diff --git a/docs/customize.rst b/docs/customize.rst index 3b0e345e..ca3ec975 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -575,8 +575,9 @@ listed below. - Reads an initial-shaped suffix word after a comma as a suffix: ``"John Smith, V"`` is John Smith the fifth when ``True`` (default); ``False`` reads ``V`` as a given-name initial - instead. Multi-letter suffixes (``III``, ``MD``) are - unaffected. + instead, and ``"john smith, v"`` the same way, since a name + written all in one case reads as its capitals. Multi-letter + suffixes (``III``, ``MD``) are unaffected. * - ``unlisted_dotted_suffixes`` - ``bool`` - Reads an unlisted dotted acronym such as ``X.Y.Z.`` as a diff --git a/docs/design/decisions/T4-bare-letter-as-an-initial.md b/docs/design/decisions/T4-bare-letter-as-an-initial.md index 33c3e555..f6c9f111 100644 --- a/docs/design/decisions/T4-bare-letter-as-an-initial.md +++ b/docs/design/decisions/T4-bare-letter-as-an-initial.md @@ -3,7 +3,9 @@ - 2026-10-18, #723 — A ONE-CASE BARE LETTER IS THE INITIAL ITS CAPITAL IS. #721 applied S10's "a name written wholly in one case says nothing about any letter in it" to the one-letter roman numerals only (decisions/S10, the Open line of its /simplify entry), so a bare lowercase `j` was a name word where `J` was an initial, and every rule that reads an initial differently read the two spellings of one name differently: M4 (`j nee smith` family 'j', `J NEE SMITH` given 'J'), O5 (a lone `j` reported `given-or-family`, `J` did not), and P3's count, which passes over an initial (`jose j y lopez` family 'j y lopez', `JOSE J Y LOPEZ` middle 'J Y', family 'LOPEZ', which 1.4.0 read too). The rule is now stated once, for every letter, and the numeral case is an instance of it: in a name written in one case, a bare letter is tagged an initial wherever its capital would be. Its capital is asked of the bare-capital shape (one ASCII capital), so `é` and `ж`, whose capitals are no initial, read as before in both spellings. `ı` and `ſ`, whose capitals are `I` and `S`, are tagged initials too; `ı`'s twin is not a true one, since `I` is also the marked Catalan connective and `ı` is in no vocabulary (`JOSE I LOPEZ` reports `conjunction-or-initial`, `jose ı lopez` nothing). Measured on the first commit's tree against 16ee2851, a dated snapshot from a scratch grid not committed: letters a–z, `é` and `ж`; heads (a, b, c) = (john, smith, jones), (jose, lopez, garcia), (mary, ma, lee); all three orders; the 35 shapes `{n}`, `{n} nee {b}`, `{a} {n}`, `{n} {a}`, `{a} {n} {b}`, `{a} {n} y {b}`, `{a} {n} e {b}`, `{b}, {n}`, `{b}, {n} y {c}`, `{b}, {a} {n}`, `, {n}`, `{n}, phd`, `{a} {n}, phd`, `{a} {n}, jr.`, `{a} nee {b} {n}`, `{a} nee {n}`, `{a} {b} nee {n} {c}`, `dr {n}`, `dr {n} {b}`, `sir {n} {b}`, `{a} van {n}`, `{n} van {b}`, `{a} van {n} {b}`, `abdul {n}`, `{n} {a} {b}`, `{a} {n} {n} {b}`, `{a} {n}. {b}`, `({c}) {n} {b}`, `{n} y {b}`, `{a} {n} phd`, `{a} {b} {n} prof.`, `{n} {b} {n}`, `{n} {n}`, `{a} {n} de la {b}`, `{b} {n} jr`; each lowercase text against its `.upper()`, compared on case-folded fields, sorted report kinds and `initials()`, keyed by order and text, so the 8,820 products collapse to 7,794 distinct pairs where a shape has no head. 16ee2851 disagreed on 504 of them, this tree on none, and no capital spelling moved. The committed invariant is `test_a_one_case_name_reads_as_its_capitals` (tests/v2/test_properties.py), whose grid gains the issue's shapes and the letters `j`, `y` and `e`; its stated controls are the recompute. WHAT MOVES: lower-case letters in a name whose own words are written in one case, each onto its capital twin's reading — wholly lower-case names, and (since the case question is asked of the own words, P3) a one-case name beside a clause of another case, so `j nee Smith` reads given 'j' as `J nee Smith` does. In three classes — a lone letter loses O5's report, a letter before a maiden marker keeps the given name M4 would have taken, and a letter beside a single-letter link is passed over by P3's count, so the link in what is then a three-word name stays a name word. No corpus name moved (the snapshot and the CLDR, GND and native-script agreement records are unchanged). And P3's Accepted paragraph on a declined marker loses its "one shape" in which the name's own words read differently in the two spellings: `dr. née juan y lopez` read family 'juan y lopez' where `DR. NÉE JUAN Y LOPEZ` reads middle 'JUAN Y', because the `y` past the declined marker is outside the own words P3's fork reads; it is this rule's letter now, and reads as its capital does. And one move the snapshot cannot see (R4): case repair of a connective letter in a one-case maiden clause, now tagged an initial as its capital is, so `jose lopez nee y garcia` repairs to `Jose Lopez (Y Garcia)` where it gave `(y Garcia)` — the same tag in the two spellings that P3's rewritten paragraph intends. No corpus name's `capitalized()` moved (the design-docs review compared every one across the two trees). - A GAP THE RULE DOES NOT CLOSE, found by the design-docs review and present on 16ee2851 too: under `Policy(lenient_comma_suffixes=False)` the strict comma-suffix test asks a one-letter numeral's shape as written, not its tag, so `john smith, v` reads suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reports nothing where `SMITH, JOHN, I` reports `comma-structure`. The committed grid runs the default policy only. Left for its own change. - Open: #727 — whether the strict comma test should read the tag. + A GAP THE RULE DOES NOT CLOSE, found by the design-docs review and present on 16ee2851 too: under `Policy(lenient_comma_suffixes=False)` the strict comma-suffix test asks a one-letter numeral's shape as written, not its tag, so `john smith, v` reads suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reports nothing where `SMITH, JOHN, I` reports `comma-structure`. The committed grid runs the default policy only. Left for its own change, and taken by the #727 entry below (was Open: #727). The connective letters are left where P3 put them. Among the name's own words, P3's fork answers by vocabulary before this rule is asked — the marked `e` and `i` read as initials and report, the unmarked `y` as a connective — and does so in both spellings, so the invariant already held for them; a connective letter outside the own words was always read by the bare-capital shape in its capital spelling, and its lower-case spelling now follows. - 2026-10-18, #723's code review — A LETTER OUTSIDE THE OWN WORDS ASKS THE WHOLE NAME'S CASE. The first version gated the new tag on the one-case fact classify records, which is asked of the name's own words only (P3), so a lower-case letter in a clause of a name whose own words are capitals passed it: `MARIA GARCIA NEE LOPEZ y FERNANDEZ` and `MARIA GARCIA (LOPEZ y FERNANDEZ)` tagged the `y` an initial (case repair wrote `Lopez Y Fernandez`), and `DR. NÉE JUAN y LOPEZ` moved from family 'JUAN y LOPEZ' to middle 'JUAN y' — mixed-case writing, which this rule reads as no initial, and moves the twin-only grids could not see, since every pair in them is wholly one case. The fix asks a word outside the own words about the whole name instead; an own word keeps the own-words fact, so a clause beside the name still changes nothing about the name's own letters (P3). The defect predated #723 for the numeral letters, which #721 gated the same way; they are read by this rule now and move with it, and the move is not only a tag (found by the design-docs review). After a marker among capitals, a lower-case numeral no longer holds the clause up as an initial would: `MARY SMITH NEE v JONES` reads middle 'SMITH NEE v', family 'JONES' (on 16ee2851, maiden 'v JONES'), `MARY SMITH NEE i JONES` family 'NEE i JONES', and `JANE DOE NEE SMITH v` drops `suffix-or-name`. Each is the reading of its mixed-case spelling (`Mary Smith nee v Jones`) and of 2.4.2 (and 1.4.0 for the first), so only the unreleased #721 had moved them and no release bullet is owed; the capital twin `MARY SMITH NEE V JONES` keeps maiden 'V JONES'. Pinned by the row `a_lowercase_numeral_after_a_marker_among_capitals_is_no_initial`. Pinned by `test_a_lowercase_clause_letter_among_capitals_is_no_initial` and the case row `a_lowercase_letter_past_a_declined_marker_among_capitals_joins`. The first entry's figures were measured on the first version, and the twin grids are unchanged by the fix (both spellings of every pair in them are wholly one case). +- 2026-10-18, #727 — THE STRICT SUFFIX TEST VETOES A ONE-CASE LETTER AS IT VETOES ITS CAPITAL. The gap the first entry recorded: `is_suffix_strict` vetoes an initial by the word's text (`is_initial`), not by classify's tag, so a one-case lowercase `v` or `i` — an initial under this rule — passed where its capital was vetoed. Two callers reached it. Under `Policy(lenient_comma_suffixes=False)`, C1's comma tests (`is_wholly_suffix` and the name-word count): `john smith, v` read suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reported nothing where its capitals report `comma-structure`. And under the default policy, W2's scan-back (`_script_segment._is_post_nominal`): `田中さん v` peeled さん and read suffix 'さん v' where `田中さん V` reads given '田中さん', family 'V' — a CJK name with a Latin letter beside it is one case under the own-words test, the letter being the only cased word in it, so nothing in the writing contrasts it. The strict test now takes the one-case fact as an argument and vetoes a bare lowercase letter whose capital is A to Z where it is true; segment, `_comma` and script_segment pass the fact they already hold, and segment asks for it only under the strict knob. A separate keyword from `is_wholly_suffix`'s `one_case`, which is the credential LEAN and is None by design at callers that want no lean; they still want the letters read by this rule. The lenient default takes a listed suffix word whatever its shape and never reads the argument. + Measured: the committed invariant's grid gains the three comma shapes and `田中さん {n}`, and runs every comma-holding shape again under the strict knob (the knob reaches nothing else); 2444f847 disagreed on 72 of its 4,554 pairs (54 strict comma pairs, 18 honorific), this tree on none, and the third recorded control is that 72. `田中さん v` now reads as 1.4.0 read it; 2.1 through 2.4.2 peeled it (suffix 'さん, v' in 2.1 and 2.2, 'さん v' from 2.3). No corpus name moved. + Open: none from this entry; #727 closes the gap the first entry left open. diff --git a/docs/design/rules/C1-suffix-comma-decision.md b/docs/design/rules/C1-suffix-comma-decision.md index 355f7978..62aef943 100644 --- a/docs/design/rules/C1-suffix-comma-decision.md +++ b/docs/design/rules/C1-suffix-comma-decision.md @@ -138,7 +138,8 @@ C1. Rationale: a credential run after the comma means the name is in parse could not recognize, not a fork it called. By default a recognized suffix word counts even written like an initial ("V."), while strict mode vetoes - initial-shaped words. In the listing form the part after the + a word read as an initial — in a name written in one case a + bare lower-case letter as well, as its capital is (T4). In the listing form the part after the comma is still read for what it is: a part that is nothing but suffix words is the credential run and reads as suffixes, whole — the slot after a family comma is postnominal position, so the @@ -195,6 +196,7 @@ C1. Rationale: a credential run after the comma means the name is in "John Smith, PhD" → suffix="PhD" "John Smith, V." → suffix="V." "John Smith, V." strict-comma-suffixes → family="John Smith" + "john smith, v" strict-comma-suffixes → family="john smith" "Smith, PhD" → family="Smith" · boundary "Smith, PhD" → suffix="PhD" "Smith, Jr." → suffix="Jr." @@ -348,5 +350,5 @@ C1. Rationale: a credential run after the comma means the name is in V` reads the suffix and `Smith, John PhD I.` continues the run, while adding a suffix comma after either turns that same letter into the middle initial. - history: decisions/C1 · interacts: H1, H2, H6, P1, P2, P3, P5, P6, W3, S3, M2, S5, S6, S7, S8 · implemented: nameparser/_pipeline/_comma.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py + history: decisions/C1 · interacts: H1, H2, H6, P1, P2, P3, P5, P6, W3, S3, M2, S5, S6, S7, S8, T4 · implemented: nameparser/_pipeline/_comma.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py diff --git a/docs/design/rules/T4-bare-letter-as-an-initial.md b/docs/design/rules/T4-bare-letter-as-an-initial.md index 4ff914ae..fa367d8a 100644 --- a/docs/design/rules/T4-bare-letter-as-an-initial.md +++ b/docs/design/rules/T4-bare-letter-as-an-initial.md @@ -14,8 +14,10 @@ T4. Rationale: one letter standing for a name word is how an an initial differently from a name word reads the lower-case spelling as it reads the capitals: M4 leaves an initial its given-name reading, O5 reports no convention for one, P3's - count passes over one, and S10 reads a numeral written as one - as the capital numeral it is. A letter whose capital is not one + count passes over one, S10 reads a numeral written as one + as the capital numeral it is, and a suffix test that vetoes an + initial vetoes it — C1's strict mode, and the scan-back of W2's + peel. A letter whose capital is not one of those — é, ж — is no initial in either spelling, and in a name of mixed case a bare lower-case letter is no initial. As P3 asks it, the case question is asked of the name's own words, @@ -42,4 +44,4 @@ T4. Rationale: one letter standing for a name word is how an "DR. NÉE JUAN y LOPEZ" → family="JUAN y LOPEZ" · boundary "jose e lopez" → ambiguities=("conjunction-or-initial",) "jose y lopez" → middle="y" - history: decisions/T4 · interacts: M4, O5, P2, P3, P7, S10, R3, R4 · implemented: nameparser/_pipeline/_classify.py + history: decisions/T4 · interacts: M4, O5, P2, P3, P7, S10, R3, R4, C1, W2 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_vocab.py diff --git a/docs/design/rules/W2-glued-honorific-peel.md b/docs/design/rules/W2-glued-honorific-peel.md index 5440d5f9..a2d303b1 100644 --- a/docs/design/rules/W2-glued-honorific-peel.md +++ b/docs/design/rules/W2-glued-honorific-peel.md @@ -41,5 +41,5 @@ W2. Rationale: some East Asian honorifics glue directly onto the end more right (#707). "田中 副社長" → given="副" "岸田文雄衆議院議員" → family="岸田文雄衆議院" - history: decisions/W2 · interacts: W3 · implemented: nameparser/_pipeline/_script_segment.py + history: decisions/W2 · interacts: W3, T4 · implemented: nameparser/_pipeline/_script_segment.py diff --git a/docs/release_log.rst b/docs/release_log.rst index 2b1e3bf1..e7d73746 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -15,7 +15,7 @@ Release Log - **Fix case repair capitalizing the Dutch particle 't.** ``jacobus henricus van 't hoff`` repairs to ``Jacobus Henricus van 't Hoff``, where 1.4.0 through 2.4.2 gave ``van 'T Hoff``. :issue:`700` - **Add Gov as a title.** ``Gov Smith`` gives title ``Gov``, last ``Smith``, where 1.4.0 through 2.4.2 gave first ``Gov``, and ``Lt. Gov. Smith, John`` gives title ``Lt. Gov.`` with the change above. :issue:`701` - **Fix a capital V, I or X after a lone first name being read as a generational suffix, leaving no last name.** ``John V`` gives first ``John``, last ``V``, and ``Malcolm X`` last ``X``, where 2.0.0 through 2.4.2 gave suffix ``V`` and ``X`` with no last name, restoring 1.4.0's reading. A title is no word to spare, so ``Dr. Smith V`` gives first ``Smith``, last ``V`` (1.4.0's reading too) where 2.0.0 through 2.4.2 gave last ``Smith``, suffix ``V``; a name with a word to spare keeps the suffix, and ``John Smith V`` still gives suffix ``V``. A name written all in lower case reads the same way (``john v`` gives last ``v``). :issue:`718`, :issue:`721` - - **Fix a single letter in a name written all in lower case reading differently from the same name in capitals.** ``jose j y lopez`` gives first ``jose``, middle ``j y``, last ``lopez``, as ``JOSE J Y LOPEZ`` does, where 1.4.0 through 2.4.2 gave last ``j y lopez``. A lone letter before a maiden marker stays the first name (``j nee smith`` as ``J NEE SMITH``). :issue:`723` + - **Fix a single letter in a name written all in lower case reading differently from the same name in capitals.** ``jose j y lopez`` gives first ``jose``, middle ``j y``, last ``lopez``, as ``JOSE J Y LOPEZ`` does, where 1.4.0 through 2.4.2 gave last ``j y lopez``. A lone letter before a maiden marker stays the first name (``j nee smith`` as ``J NEE SMITH``), ``田中さん v`` reads as ``田中さん V`` does, and ``Policy(lenient_comma_suffixes=False)`` reads ``john smith, v`` as it reads ``JOHN SMITH, V``. :issue:`723`, :issue:`727` * 2.4.2 - October 13, 2026 diff --git a/nameparser/_pipeline/_classify.py b/nameparser/_pipeline/_classify.py index 639b65fe..7e7106e3 100644 --- a/nameparser/_pipeline/_classify.py +++ b/nameparser/_pipeline/_classify.py @@ -171,12 +171,13 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, # nothing about any letter in it, a bare letter is an initial # wherever its capital would be" -- so every reader of the tag # after classify reads 'j' as it reads 'J' (#723; the roman - # numerals first, #721) -- not the honorific peel upstream, - # which asks the shape itself ('田中さん v' still reads - # otherwise than '田中さん V'). `one_case` is the fact about - # the name's own words, applied to a clause's words too, as - # #516's caps branch applies it, since a capital 'J' in a - # clause is an initial as well. The capital is asked of the + # numerals first, #721). Readers that run before the tag + # exists, or ask the word's text rather than its tag -- the + # strict suffix test, which the honorific peel and + # `lenient_comma_suffixes=False` ask -- take the same fact as + # an argument (#727). `letter_one_case` is the own-words fact + # for an own word and the whole name's for a clause's + # (docstring). The capital is asked of the # bare pattern at C level, no frame per one-letter word, and # needs no script test: no capital `_INITIAL` matches is in a # script without initials. 'é' and 'ж' stay as they were, diff --git a/nameparser/_pipeline/_comma.py b/nameparser/_pipeline/_comma.py index cd72e904..a6bfa658 100644 --- a/nameparser/_pipeline/_comma.py +++ b/nameparser/_pipeline/_comma.py @@ -244,7 +244,8 @@ def decide(state: ParseState) -> ParseState: licensed: set[int] = set() run = False if maybe: - names = name_word_count([tokens[i].text for i in seg0], lex, pol) + names = name_word_count([tokens[i].text for i in seg0], lex, pol, + letters_one_case=state.one_case) if names >= 2: # rules/C1: "An unlisted all-caps word joins the class in such # a part only where the name carries the contrast" @@ -283,7 +284,8 @@ def decide(state: ParseState) -> ParseState: for i in p: others.append(tokens[i].text) run = (bool(licensed) - and (not others or is_wholly_suffix(others, lex, pol)) + and (not others or is_wholly_suffix( + others, lex, pol, letters_one_case=state.one_case)) and not any(len(p) == 1 and is_single_letter_numeral(tokens[p[0]].text) for p in p1)) @@ -336,7 +338,8 @@ def decide(state: ParseState) -> ParseState: # rules/C1: "By default a recognized suffix word counts even # written like an initial" -- behind a whole name ('John Smith, V.') if reading is None and whole and is_wholly_suffix( - [tokens[i].text for i in part], lex, pol, one_case=None): + [tokens[i].text for i in part], lex, pol, one_case=None, + letters_one_case=state.one_case): reading = (True,) * len(p1) if reading is None: return state diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index e3f14252..c659f057 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -800,10 +800,10 @@ def segment_suffix_reading(pieces: Sequence[Sequence[int]], title to read by). `lenient` is Policy.lenient_comma_suffixes, and only the numeral - continuation consults it. C1: "by default a recognized suffix word - counts even written like an initial, while strict mode vetoes - initial-shaped words" -- so under strict the veto stands and the - run ends where it always did. Reading no policy here silently + continuation consults it. C1: "while strict mode vetoes a word read + as an initial" -- so under strict the veto stands and the run ends + where it always did; the walk reads the token's `initial` tag, so a + one-case lowercase letter is vetoed as its capital is (rules/T4). Reading no policy here silently overrode the one knob a caller sets to prevent exactly this. The FAMILY_COMMA rule that segment 0 is the family name rests diff --git a/nameparser/_pipeline/_script_segment.py b/nameparser/_pipeline/_script_segment.py index e5f756d1..859a140c 100644 --- a/nameparser/_pipeline/_script_segment.py +++ b/nameparser/_pipeline/_script_segment.py @@ -242,7 +242,13 @@ def _is_post_nominal(state: ParseState, i: int) -> bool: not: a title never trails, so widening the peel's scan-back would move it off the real last name token. Split the predicate then, not before.""" - return is_suffix_strict(state.tokens[i].text, state.lexicon) + # rules/T4, #727: a bare lowercase letter in a name written in one + # case is vetoed as its capital is; the case fact is asked only of + # such a letter, so a CJK honorific (no case) never asks it + text = state.tokens[i].text + return is_suffix_strict(text, state.lexicon, + len(text) == 1 and text.islower() + and _one_case(state)) # rules/W2: "A part that is not name text — a post-nominal word @@ -553,10 +559,13 @@ def _peel_honorific_tail(state: ParseState, # the site test first: it is the cheaper half, and false # for every name whose part before the comma holds no glued # honorific, which spares those the case fact + # one fact, asked for the lean and for the strict test's + # initial veto alike (rules/T4, #727) if not (site is not None and is_wholly_suffix(second, state.lexicon, state.policy, - one_case=_one_case(state))): + one_case=(case := _one_case(state)), + letters_one_case=case)): site = both if site is None: return state diff --git a/nameparser/_pipeline/_segment.py b/nameparser/_pipeline/_segment.py index 77e7393e..4e17581d 100644 --- a/nameparser/_pipeline/_segment.py +++ b/nameparser/_pipeline/_segment.py @@ -99,9 +99,15 @@ def case_class() -> bool: # walk below passes `case_class()` for the leaning one. One # closure with a default rather than two whose bodies differ only # in that argument. + # The strict test also reads the case fact, for its initial veto + # (rules/T4, #727): asked only under that opt-in policy, so the + # default pays nothing for it. def suffixy(seg: tuple[int, ...], case: bool | None = None) -> bool: - return is_wholly_suffix([state.tokens[i].text for i in seg], - state.lexicon, state.policy, one_case=case) + return is_wholly_suffix( + [state.tokens[i].text for i in seg], state.lexicon, + state.policy, one_case=case, + letters_one_case=(None if state.policy.lenient_comma_suffixes + else case_class())) def class_run(seg: tuple[int, ...]) -> bool: # Every token of the run joins the ambiguous credential class diff --git a/nameparser/_pipeline/_vocab.py b/nameparser/_pipeline/_vocab.py index c5fc5c82..abd31427 100644 --- a/nameparser/_pipeline/_vocab.py +++ b/nameparser/_pipeline/_vocab.py @@ -497,18 +497,33 @@ def suffix_as_written(n: str, text: str, lexicon: Lexicon) -> bool: or n in lexicon.suffix_words -def _is_suffix_strict_n(n: str, text: str, lexicon: Lexicon) -> bool: - if is_initial(text): +def _is_suffix_strict_n(n: str, text: str, lexicon: Lexicon, + one_case: bool | None = None) -> bool: + # rules/T4: "in a name written wholly in one case, which says + # nothing about any letter in it, a bare letter is an initial + # wherever its capital would be" -- so the veto takes 'v' where it + # takes 'V' (#727). Lower case first, at C level: a capital is + # is_initial's already, and only a caller with the fact passes it + if is_initial(text) or ( + one_case and len(text) == 1 and text.islower() + and _INITIAL.fullmatch(text.upper()) is not None): # period-written ambiguous acronyms are exempt from the veto return _dotted(text) and \ n.replace(".", "") in lexicon.suffix_acronyms_ambiguous return suffix_as_written(n, text, lexicon) -def is_suffix_strict(text: str, lexicon: Lexicon) -> bool: +def is_suffix_strict(text: str, lexicon: Lexicon, + one_case: bool | None = None) -> bool: """v1's is_suffix: suffix_as_written with the initial veto ('V.' in - 'John V. Smith' is a middle initial, not roman five).""" - return _is_suffix_strict_n(_normalize(text), text, lexicon) + 'John V. Smith' is a middle initial, not roman five). + + `one_case` is whether the name's own words are written in one + case (ParseState.one_case): where they are, a bare lowercase + letter is the initial its capital is (rules/T4) and is vetoed as + its capital is. None or False is the veto by the writing alone, + which is v1's.""" + return _is_suffix_strict_n(_normalize(text), text, lexicon, one_case) def is_suffix_lenient(text: str, lexicon: Lexicon) -> bool: @@ -1028,7 +1043,8 @@ def surname_unit_tags(text: str, lexicon: Lexicon, def name_word_count(texts: Sequence[str], lexicon: Lexicon, - policy: Policy) -> int: + policy: Policy, letters_one_case: bool | None = None, + ) -> int: """How many of these texts are NAME words -- not suffix vocabulary, not title vocabulary. @@ -1050,9 +1066,10 @@ def name_word_count(texts: Sequence[str], lexicon: Lexicon, family name, so 'De La Cruz' is one name word and 'De La Cruz, Ed' reads as 'Royce, Ed' does. Why the other two joins are not read here: `SURNAME_UNIT_TAGS`. + + `letters_one_case` is `is_wholly_suffix`'s, for the same reason. """ - predicate = (is_suffix_lenient if policy.lenient_comma_suffixes - else is_suffix_strict) + lenient = policy.lenient_comma_suffixes # One pass: each token's name-word verdict, and whether any token # is a particle. With none, every token is a unit of its own and @@ -1064,7 +1081,9 @@ def name_word_count(texts: Sequence[str], lexicon: Lexicon, n = _normalize(text) if n in lexicon.particles: has_particle = True - names.append(not (predicate(text, lexicon) or n in lexicon.titles + suffix = (is_suffix_lenient(text, lexicon) if lenient + else is_suffix_strict(text, lexicon, letters_one_case)) + names.append(not (suffix or n in lexicon.titles or is_title_shaped(text))) if not has_particle: return sum(names) @@ -1080,7 +1099,8 @@ def name_word_count(texts: Sequence[str], lexicon: Lexicon, def is_wholly_suffix(texts: Sequence[str], lexicon: Lexicon, - policy: Policy, one_case: bool | None = None) -> bool: + policy: Policy, one_case: bool | None = None, + letters_one_case: bool | None = None) -> bool: """Every token in a RUN counts as a suffix -- C1's suffix test of a run, shared by `_comma.decide`, segment's C2 flag on a tail part and the peel (#319). @@ -1111,6 +1131,15 @@ def is_wholly_suffix(texts: Sequence[str], lexicon: Lexicon, as no lean and is this predicate's behavior in every release before 2.4. + `letters_one_case` is the same fact asked a different question, + and the two are kept apart because a caller that wants no lean + still wants the letters read by T4: under + `lenient_comma_suffixes=False` the strict test vetoes an + initial, and in a name written in one case a bare lowercase + letter is one ('john smith, v' reads as 'JOHN SMITH, V', #727). + The lenient test takes a suffix word whatever its shape, so the + default never reads it. + Neither Policy.unlisted_dotted_suffixes NOR Policy.unlisted_caps_suffixes reaches this predicate: admitting a by-shape member here unconditionally (an earlier version of this @@ -1130,8 +1159,7 @@ def is_wholly_suffix(texts: Sequence[str], lexicon: Lexicon, """ if not texts: return False - predicate = (is_suffix_lenient if policy.lenient_comma_suffixes - else is_suffix_strict) + lenient = policy.lenient_comma_suffixes # v1 expand_suffix_delimiter parity (#206): a configured delimiter # is TRANSPARENT in the all-suffix tests -- v1 split the part string # on the delimiter before checking, so the delimiter never counted @@ -1144,7 +1172,8 @@ def counts_as_suffix(text: str) -> bool: and ambiguous_class_member(text, lexicon) and ambiguous_lean(text, one_case) == "credential"): return True - return (predicate(text, lexicon) + return ((is_suffix_lenient(text, lexicon) if lenient + else is_suffix_strict(text, lexicon, letters_one_case)) or period_joined_vocab(text, lexicon) == "suffix" or (bool(cores) and splits_into_suffixes(text, cores, lexicon))) diff --git a/nameparser/_policy.py b/nameparser/_policy.py index cc3d2868..8241d4db 100644 --- a/nameparser/_policy.py +++ b/nameparser/_policy.py @@ -700,8 +700,10 @@ class Policy: #: of the default vocabulary that means the roman numerals "I" and #: "V"): "John Smith, V" reads as John Smith the fifth when True #: (the default, v1 behavior); False reads "V" as a given-name - #: initial instead (family "John Smith", given "V"). Multi-letter - #: suffixes ("III", "MD") parse the same either way. + #: initial instead (family "John Smith", given "V"), and "john + #: smith, v" the same way, a name written in one case reading as + #: its capitals (rules/T4). Multi-letter suffixes ("III", "MD") + #: parse the same either way. lenient_comma_suffixes: bool = True #: Excludes emoji from tokenization: they appear in no token, #: field, or rendered view. The original string keeps them (input diff --git a/tests/v2/cases.py b/tests/v2/cases.py index d7774342..09f2e401 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -3978,6 +3978,24 @@ def _check_cjk_shape_purity(self) -> None: "roman numeral), so 'V' is the family -- 1.4.0's " "reading field for field, which #718 restored -- and " "both forks, 'V' and 'Ma', report"), + Case("strict_comma_reads_a_one_case_numeral_as_its_capital", + "john smith, v", + {"given": "v", "family": "john smith"}, + policy=Policy(lenient_comma_suffixes=False), + notes="the strict test vetoes an initial, and in a name " + "written in one case a bare 'v' is one (T4), so this " + "reads as 'JOHN SMITH, V' does under the same knob: " + "the knob's 'V' as a given-name initial (#727; before, " + "suffix 'v')"), + Case("strict_comma_reports_a_one_case_numeral_in_a_third_part", + "smith, john, i", + {"given": "john", "family": "smith", "suffix": "i"}, + policy=Policy(lenient_comma_suffixes=False), + ambiguities=("comma-structure",), + notes="the third part holds no word the strict test calls a " + "suffix, so C2 reads it as suffix material and reports " + "the structure, as for 'SMITH, JOHN, I' (#727; before, " + "no report)"), Case("caps_run_declines_a_bound_given_head", "John Smith, ABDUL AP", {"given": "ABDUL AP", "family": "John Smith"}, policy=Policy(unlisted_caps_suffixes=CapsSuffixes.EVERYWHERE), @@ -11465,6 +11483,16 @@ def _check_cjk_shape_purity(self) -> None: "the same token downstream -- 'V.' is a middle " "initial, not a post-nominal", tolerated=True), + Case("ja_honorific_glued_before_a_one_case_numeral", "田中さん v", + {"given": "田中さん", "family": "v"}, + ambiguities=("suffix-or-name",), + notes="nothing in this name is written in a case, so the bare " + "'v' is the initial its capital is (T4): the scan-back " + "stops at it as at 'V', and this reads as '田中さん V' " + "does -- 1.4.0's reading too (#727; 2.1 through 2.4.2 " + "peeled さん, reading suffix 'さん, v' in 2.1 and 2.2 " + "and 'さん v' from 2.3)", + tolerated=True), Case("ja_honorific_with_a_period_no_comma", "田中さん 様.", {"family": "田中", "suffix": "さん 様."}, classification="fix(#320)", diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 8e37449b..972b003d 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -191,9 +191,10 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # was read into, on those rows under each order; 1191 -> 1210 the # same day: the PR review's rows (the chain-taken, family-first, # lowercase and maiden-clause numerals); 1210 -> 1243 the same day: - # #721's rows (#722). - assert claims == 1243, ( - f"the field sweep checks {claims} claims, recorded as 1243 on " + # #721's rows (#722); 1243 -> 1246 the same day: #727's + # '田中さん v' row (one suffix-or-name claim under each order). + assert claims == 1246, ( + f"the field sweep checks {claims} claims, recorded as 1246 on " f"2026-10-18") diff --git a/tests/v2/test_facade_cases.py b/tests/v2/test_facade_cases.py index 6573cd29..bc1637b1 100644 --- a/tests/v2/test_facade_cases.py +++ b/tests/v2/test_facade_cases.py @@ -175,6 +175,8 @@ "caps_switch_leaves_a_capitalized_title_a_title", "strict_comma_reads_the_dotted_numeral_as_a_name_word", "strict_comma_reads_the_bare_numeral_into_the_run", + "strict_comma_reads_a_one_case_numeral_as_its_capital", + "strict_comma_reports_a_one_case_numeral_in_a_third_part", "the_comma_flip_is_read_under_the_declared_order", "two_caps_credentials_peel_as_a_run", "the_caps_shape_is_script_agnostic_cyrillic", diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index d8d9b470..07a77f53 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -3343,8 +3343,11 @@ def test_every_one_letter_particle_veto_is_load_bearing( # bound given name, an initial, a nickname and a credential before or # after it -- and the shapes any bare letter meets (#723), with the # letters below, three heads (one Catalan, one holding an ambiguous -# acronym) and all three orders: -# 3,168 pairs, about 6,350 parses, 0.48s on py3.11 (2026-10-18). +# acronym) and all three orders -- and every shape holding a comma +# again under Policy(lenient_comma_suffixes=False), the one switch +# whose suffix test vetoes an initial by the word's own shape (#727); +# trimmed by shape, since the switch reaches only a comma's parts: +# 4,554 pairs, about 9,100 parses, 0.73s on py3.11 (2026-10-18). _ONE_CASE_SHAPES = ( "{a} {n}", "{a} {b} {n}", "{a} {n}, phd", "{a} {b} {n}, phd", "{a} {n}, jr.", "{b}, {a} {n}", ", {a} {n}", "{a} nee {b} {n}", @@ -3360,7 +3363,11 @@ def test_every_one_letter_particle_veto_is_load_bearing( # #723: the letter alone, before a maiden marker, before or after a # connective, opening the name and after an empty head "{n}", "{n} nee {b}", "{a} {n} y {b}", "{b}, {n} y {c}", "{n} {a}", - ", {n}") + ", {n}", + # #727: the letter alone after a comma, behind a whole name or in a + # third part, and opening a credential run; and after a glued CJK + # honorific, whose peel asks the suffix test by text + "{a} {b}, {n}", "{b}, {a}, {n}", "{a} {b}, {n} phd", "田中さん {n}") #: the one-letter numerals and a listed and a shape-only spelling of #: each longer length, then one letter from each other fork a bare #: letter meets (#723): no vocabulary, a connective, a marked one @@ -3369,12 +3376,14 @@ def test_every_one_letter_particle_veto_is_load_bearing( _ONE_CASE_HEADS = (("john", "smith", "jones"), ("carod", "puig", "rovira"), ("mary", "ma", "lee")) #: the negative controls, measured 2026-10-18 on this grid, which -#: 16ee2851 (before #723, its numerals already guarded) disagreed on -#: 30 of: with classify's one-case letter initial taken out, and with -#: the numeral fork's one-case letter in front taken out -- each guard -#: seen on its own -_ONE_CASE_UNGUARDED = 306 +#: 2444f847 (before #727) disagreed on 72 of: with classify's one-case +#: letter initial taken out, with the numeral fork's one-case letter +#: in front taken out, and with the strict suffix test's one-case veto +#: taken out (#727: the 54 strict comma pairs and the 18 honorific +#: ones, which are 2444f847's 72) -- each guard seen on its own +_ONE_CASE_UNGUARDED = 387 _ONE_CASE_LETTER_IN_FRONT_UNGUARDED = 117 +_ONE_CASE_STRICT_VETO_UNGUARDED = 72 def _one_case_disagreements() -> list[str]: @@ -3382,13 +3391,17 @@ def sig(name: ParsedName) -> tuple[object, ...]: return ({k: v.lower() for k, v in name.as_dict().items() if v}, tuple(sorted(a.kind.value for a in name.ambiguities))) out = [] + commas = tuple(s for s in _ONE_CASE_SHAPES if "," in s) for order in (GIVEN_FIRST, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST): - parser = Parser(policy=Policy(name_order=order)) - for shape, letter, (a, b, c) in itertools.product( - _ONE_CASE_SHAPES, _ONE_CASE_LETTERS, _ONE_CASE_HEADS): - lower = shape.format(a=a, b=b, c=c, n=letter) - if sig(parser.parse(lower)) != sig(parser.parse(lower.upper())): - out.append(f"[{order}] {lower!r}") + for strict, shapes in ((False, _ONE_CASE_SHAPES), (True, commas)): + parser = Parser(policy=Policy( + name_order=order, lenient_comma_suffixes=not strict)) + for shape, letter, (a, b, c) in itertools.product( + shapes, _ONE_CASE_LETTERS, _ONE_CASE_HEADS): + lower = shape.format(a=a, b=b, c=c, n=letter) + if (sig(parser.parse(lower)) + != sig(parser.parse(lower.upper()))): + out.append(f"[{order}{' strict' * strict}] {lower!r}") return out @@ -3405,6 +3418,19 @@ def test_the_one_case_grid_sees_the_letter_initial( assert len(_one_case_disagreements()) == _ONE_CASE_UNGUARDED +def test_the_one_case_grid_sees_the_strict_veto( + monkeypatch: pytest.MonkeyPatch) -> None: + # the strict suffix test without its one-case input (#727): the + # veto by the word's own writing alone, as before + from nameparser._pipeline import _vocab + real = _vocab._is_suffix_strict_n + monkeypatch.setattr( + _vocab, "_is_suffix_strict_n", + lambda n, text, lexicon, one_case=None: real(n, text, lexicon)) + assert (len(_one_case_disagreements()) + == _ONE_CASE_STRICT_VETO_UNGUARDED) + + def test_the_one_case_grid_sees_the_letter_in_front( monkeypatch: pytest.MonkeyPatch) -> None: from nameparser._pipeline import _vocab diff --git a/tools/differential/corpus_cjk.jsonl b/tools/differential/corpus_cjk.jsonl index d18a8191..de45f006 100644 --- a/tools/differential/corpus_cjk.jsonl +++ b/tools/differential/corpus_cjk.jsonl @@ -85,6 +85,7 @@ "田中さん" "田中さん II" "田中さん V." +"田中さん v" "田中さん 様." "田中さん, Dr." "田中さん, Ph. D." diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index a6d645e1..3486a1d6 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -5627,6 +5627,7 @@ {"name": "田中さん", "family": "田中", "suffix": "さん", "initials": "田."} {"name": "田中さん II", "family": "田中", "suffix": "さん II", "initials": "田."} {"name": "田中さん V.", "given": "田中さん", "family": "V.", "initials": "田. V."} +{"name": "田中さん v", "given": "田中さん", "family": "v", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "田. v."} {"name": "田中さん 様.", "family": "田中", "suffix": "さん 様.", "initials": "田."} {"name": "田中さん, Dr.", "title": "Dr.", "family": "田中さん", "initials": "田."} {"name": "田中さん, Ph. D.", "family": "田中", "suffix": "さん, Ph. D.", "initials": "田."}