From 1bea4aba32e52d169012fea23e219d79864349db Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 18 Oct 2026 19:35:37 -0700 Subject: [PATCH 1/2] Veto a one-case letter in the strict suffix test as its capital (#727) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit T4 tags a bare lowercase letter in a one-case name an initial, but the strict suffix test vetoes an initial by the word's text, so a lowercase 'v' or 'i' passed where its capital was vetoed. Under lenient_comma_suffixes=False 'john smith, v' read suffix 'v' where 'JOHN SMITH, V' reads given 'V', and 'smith, john, i' reported nothing where its capitals report comma-structure. Under the default policy the honorific peel's scan-back asks the same test, so '田中さん v' peeled さん where '田中さん V' does not. is_suffix_strict takes the one-case fact and vetoes such a letter; segment, _comma and script_segment pass the fact they hold (segment asks for it only under the strict knob). It is a keyword apart from is_wholly_suffix's one_case, which is the credential lean and None by design at callers that want no lean. The lenient default never reads it. The one-case invariant's grid gains three comma shapes and the honorific shape, and runs its comma shapes again under the strict knob: 2444f847 disagreed on 72 of 4,554 pairs, this tree on none; the new control records the 72. '田中さん v' reads as 1.4.0 did. No corpus name moved. Docs: T4 and C1 state it, decisions/T4 records it, the knob's docstring and customize.rst, and the release bullet. Closes #727 Co-Authored-By: Claude Opus 5.5 --- docs/customize.rst | 5 +- .../decisions/T4-bare-letter-as-an-initial.md | 3 + docs/design/rules/C1-suffix-comma-decision.md | 6 +- .../rules/T4-bare-letter-as-an-initial.md | 8 ++- docs/design/rules/W2-glued-honorific-peel.md | 2 +- docs/release_log.rst | 2 +- nameparser/_pipeline/_classify.py | 13 +++-- nameparser/_pipeline/_comma.py | 9 ++- nameparser/_pipeline/_script_segment.py | 13 ++++- nameparser/_pipeline/_segment.py | 10 +++- nameparser/_pipeline/_vocab.py | 55 ++++++++++++++----- nameparser/_policy.py | 6 +- tests/v2/cases.py | 27 +++++++++ tests/v2/test_cases.py | 7 ++- tests/v2/test_facade_cases.py | 2 + tests/v2/test_properties.py | 54 +++++++++++++----- tools/differential/corpus_cjk.jsonl | 1 + tools/differential/snapshot.jsonl | 1 + 18 files changed, 170 insertions(+), 54 deletions(-) diff --git a/docs/customize.rst b/docs/customize.rst index 3b0e345e..ca3ec975 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -575,8 +575,9 @@ listed below. - Reads an initial-shaped suffix word after a comma as a suffix: ``"John Smith, V"`` is John Smith the fifth when ``True`` (default); ``False`` reads ``V`` as a given-name initial - instead. Multi-letter suffixes (``III``, ``MD``) are - unaffected. + instead, and ``"john smith, v"`` the same way, since a name + written all in one case reads as its capitals. Multi-letter + suffixes (``III``, ``MD``) are unaffected. * - ``unlisted_dotted_suffixes`` - ``bool`` - Reads an unlisted dotted acronym such as ``X.Y.Z.`` as a diff --git a/docs/design/decisions/T4-bare-letter-as-an-initial.md b/docs/design/decisions/T4-bare-letter-as-an-initial.md index 33c3e555..e65aaeb1 100644 --- a/docs/design/decisions/T4-bare-letter-as-an-initial.md +++ b/docs/design/decisions/T4-bare-letter-as-an-initial.md @@ -7,3 +7,6 @@ Open: #727 — whether the strict comma test should read the tag. The connective letters are left where P3 put them. Among the name's own words, P3's fork answers by vocabulary before this rule is asked — the marked `e` and `i` read as initials and report, the unmarked `y` as a connective — and does so in both spellings, so the invariant already held for them; a connective letter outside the own words was always read by the bare-capital shape in its capital spelling, and its lower-case spelling now follows. - 2026-10-18, #723's code review — A LETTER OUTSIDE THE OWN WORDS ASKS THE WHOLE NAME'S CASE. The first version gated the new tag on the one-case fact classify records, which is asked of the name's own words only (P3), so a lower-case letter in a clause of a name whose own words are capitals passed it: `MARIA GARCIA NEE LOPEZ y FERNANDEZ` and `MARIA GARCIA (LOPEZ y FERNANDEZ)` tagged the `y` an initial (case repair wrote `Lopez Y Fernandez`), and `DR. NÉE JUAN y LOPEZ` moved from family 'JUAN y LOPEZ' to middle 'JUAN y' — mixed-case writing, which this rule reads as no initial, and moves the twin-only grids could not see, since every pair in them is wholly one case. The fix asks a word outside the own words about the whole name instead; an own word keeps the own-words fact, so a clause beside the name still changes nothing about the name's own letters (P3). The defect predated #723 for the numeral letters, which #721 gated the same way; they are read by this rule now and move with it, and the move is not only a tag (found by the design-docs review). After a marker among capitals, a lower-case numeral no longer holds the clause up as an initial would: `MARY SMITH NEE v JONES` reads middle 'SMITH NEE v', family 'JONES' (on 16ee2851, maiden 'v JONES'), `MARY SMITH NEE i JONES` family 'NEE i JONES', and `JANE DOE NEE SMITH v` drops `suffix-or-name`. Each is the reading of its mixed-case spelling (`Mary Smith nee v Jones`) and of 2.4.2 (and 1.4.0 for the first), so only the unreleased #721 had moved them and no release bullet is owed; the capital twin `MARY SMITH NEE V JONES` keeps maiden 'V JONES'. Pinned by the row `a_lowercase_numeral_after_a_marker_among_capitals_is_no_initial`. Pinned by `test_a_lowercase_clause_letter_among_capitals_is_no_initial` and the case row `a_lowercase_letter_past_a_declined_marker_among_capitals_joins`. The first entry's figures were measured on the first version, and the twin grids are unchanged by the fix (both spellings of every pair in them are wholly one case). +- 2026-10-18, #727 — THE STRICT SUFFIX TEST VETOES A ONE-CASE LETTER AS IT VETOES ITS CAPITAL. The gap the first entry recorded: `is_suffix_strict` vetoes an initial by the word's text (`is_initial`), not by classify's tag, so a one-case lowercase `v` or `i` — an initial under this rule — passed where its capital was vetoed. Two callers reached it. Under `Policy(lenient_comma_suffixes=False)`, C1's comma tests (`is_wholly_suffix` and the name-word count): `john smith, v` read suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reported nothing where its capitals report `comma-structure`. And under the default policy, W2's scan-back (`_script_segment._is_post_nominal`): `田中さん v` peeled さん and read suffix 'さん v' where `田中さん V` reads given '田中さん', family 'V' — a CJK name with a Latin letter beside it is one case under the own-words test, since neither script contrasts the letter. The strict test now takes the one-case fact as an argument and vetoes a bare lowercase letter whose capital is A to Z where it is true; segment, `_comma` and script_segment pass the fact they already hold, and segment asks for it only under the strict knob. A separate keyword from `is_wholly_suffix`'s `one_case`, which is the credential LEAN and is None by design at callers that want no lean; they still want the letters read by this rule. The lenient default takes a listed suffix word whatever its shape and never reads the argument. + Measured: the committed invariant's grid gains the three comma shapes and `田中さん {n}`, and runs every comma-holding shape again under the strict knob (the knob reaches nothing else); 2444f847 disagreed on 72 of its 4,554 pairs (54 strict comma pairs, 18 honorific), this tree on none, and the third recorded control is that 72. `田中さん v` now reads as 1.4.0 read it; 2.1 through 2.4.2 peeled it. No corpus name moved. + Open: none from this entry; #727 closes the gap the first entry left open. diff --git a/docs/design/rules/C1-suffix-comma-decision.md b/docs/design/rules/C1-suffix-comma-decision.md index 355f7978..62aef943 100644 --- a/docs/design/rules/C1-suffix-comma-decision.md +++ b/docs/design/rules/C1-suffix-comma-decision.md @@ -138,7 +138,8 @@ C1. Rationale: a credential run after the comma means the name is in parse could not recognize, not a fork it called. By default a recognized suffix word counts even written like an initial ("V."), while strict mode vetoes - initial-shaped words. In the listing form the part after the + a word read as an initial — in a name written in one case a + bare lower-case letter as well, as its capital is (T4). In the listing form the part after the comma is still read for what it is: a part that is nothing but suffix words is the credential run and reads as suffixes, whole — the slot after a family comma is postnominal position, so the @@ -195,6 +196,7 @@ C1. Rationale: a credential run after the comma means the name is in "John Smith, PhD" → suffix="PhD" "John Smith, V." → suffix="V." "John Smith, V." strict-comma-suffixes → family="John Smith" + "john smith, v" strict-comma-suffixes → family="john smith" "Smith, PhD" → family="Smith" · boundary "Smith, PhD" → suffix="PhD" "Smith, Jr." → suffix="Jr." @@ -348,5 +350,5 @@ C1. Rationale: a credential run after the comma means the name is in V` reads the suffix and `Smith, John PhD I.` continues the run, while adding a suffix comma after either turns that same letter into the middle initial. - history: decisions/C1 · interacts: H1, H2, H6, P1, P2, P3, P5, P6, W3, S3, M2, S5, S6, S7, S8 · implemented: nameparser/_pipeline/_comma.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py + history: decisions/C1 · interacts: H1, H2, H6, P1, P2, P3, P5, P6, W3, S3, M2, S5, S6, S7, S8, T4 · implemented: nameparser/_pipeline/_comma.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py diff --git a/docs/design/rules/T4-bare-letter-as-an-initial.md b/docs/design/rules/T4-bare-letter-as-an-initial.md index 4ff914ae..fa367d8a 100644 --- a/docs/design/rules/T4-bare-letter-as-an-initial.md +++ b/docs/design/rules/T4-bare-letter-as-an-initial.md @@ -14,8 +14,10 @@ T4. Rationale: one letter standing for a name word is how an an initial differently from a name word reads the lower-case spelling as it reads the capitals: M4 leaves an initial its given-name reading, O5 reports no convention for one, P3's - count passes over one, and S10 reads a numeral written as one - as the capital numeral it is. A letter whose capital is not one + count passes over one, S10 reads a numeral written as one + as the capital numeral it is, and a suffix test that vetoes an + initial vetoes it — C1's strict mode, and the scan-back of W2's + peel. A letter whose capital is not one of those — é, ж — is no initial in either spelling, and in a name of mixed case a bare lower-case letter is no initial. As P3 asks it, the case question is asked of the name's own words, @@ -42,4 +44,4 @@ T4. Rationale: one letter standing for a name word is how an "DR. NÉE JUAN y LOPEZ" → family="JUAN y LOPEZ" · boundary "jose e lopez" → ambiguities=("conjunction-or-initial",) "jose y lopez" → middle="y" - history: decisions/T4 · interacts: M4, O5, P2, P3, P7, S10, R3, R4 · implemented: nameparser/_pipeline/_classify.py + history: decisions/T4 · interacts: M4, O5, P2, P3, P7, S10, R3, R4, C1, W2 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_vocab.py diff --git a/docs/design/rules/W2-glued-honorific-peel.md b/docs/design/rules/W2-glued-honorific-peel.md index 5440d5f9..a2d303b1 100644 --- a/docs/design/rules/W2-glued-honorific-peel.md +++ b/docs/design/rules/W2-glued-honorific-peel.md @@ -41,5 +41,5 @@ W2. Rationale: some East Asian honorifics glue directly onto the end more right (#707). "田中 副社長" → given="副" "岸田文雄衆議院議員" → family="岸田文雄衆議院" - history: decisions/W2 · interacts: W3 · implemented: nameparser/_pipeline/_script_segment.py + history: decisions/W2 · interacts: W3, T4 · implemented: nameparser/_pipeline/_script_segment.py diff --git a/docs/release_log.rst b/docs/release_log.rst index 2b1e3bf1..e7d73746 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -15,7 +15,7 @@ Release Log - **Fix case repair capitalizing the Dutch particle 't.** ``jacobus henricus van 't hoff`` repairs to ``Jacobus Henricus van 't Hoff``, where 1.4.0 through 2.4.2 gave ``van 'T Hoff``. :issue:`700` - **Add Gov as a title.** ``Gov Smith`` gives title ``Gov``, last ``Smith``, where 1.4.0 through 2.4.2 gave first ``Gov``, and ``Lt. Gov. Smith, John`` gives title ``Lt. Gov.`` with the change above. :issue:`701` - **Fix a capital V, I or X after a lone first name being read as a generational suffix, leaving no last name.** ``John V`` gives first ``John``, last ``V``, and ``Malcolm X`` last ``X``, where 2.0.0 through 2.4.2 gave suffix ``V`` and ``X`` with no last name, restoring 1.4.0's reading. A title is no word to spare, so ``Dr. Smith V`` gives first ``Smith``, last ``V`` (1.4.0's reading too) where 2.0.0 through 2.4.2 gave last ``Smith``, suffix ``V``; a name with a word to spare keeps the suffix, and ``John Smith V`` still gives suffix ``V``. A name written all in lower case reads the same way (``john v`` gives last ``v``). :issue:`718`, :issue:`721` - - **Fix a single letter in a name written all in lower case reading differently from the same name in capitals.** ``jose j y lopez`` gives first ``jose``, middle ``j y``, last ``lopez``, as ``JOSE J Y LOPEZ`` does, where 1.4.0 through 2.4.2 gave last ``j y lopez``. A lone letter before a maiden marker stays the first name (``j nee smith`` as ``J NEE SMITH``). :issue:`723` + - **Fix a single letter in a name written all in lower case reading differently from the same name in capitals.** ``jose j y lopez`` gives first ``jose``, middle ``j y``, last ``lopez``, as ``JOSE J Y LOPEZ`` does, where 1.4.0 through 2.4.2 gave last ``j y lopez``. A lone letter before a maiden marker stays the first name (``j nee smith`` as ``J NEE SMITH``), ``田中さん v`` reads as ``田中さん V`` does, and ``Policy(lenient_comma_suffixes=False)`` reads ``john smith, v`` as it reads ``JOHN SMITH, V``. :issue:`723`, :issue:`727` * 2.4.2 - October 13, 2026 diff --git a/nameparser/_pipeline/_classify.py b/nameparser/_pipeline/_classify.py index 639b65fe..7e7106e3 100644 --- a/nameparser/_pipeline/_classify.py +++ b/nameparser/_pipeline/_classify.py @@ -171,12 +171,13 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, # nothing about any letter in it, a bare letter is an initial # wherever its capital would be" -- so every reader of the tag # after classify reads 'j' as it reads 'J' (#723; the roman - # numerals first, #721) -- not the honorific peel upstream, - # which asks the shape itself ('田中さん v' still reads - # otherwise than '田中さん V'). `one_case` is the fact about - # the name's own words, applied to a clause's words too, as - # #516's caps branch applies it, since a capital 'J' in a - # clause is an initial as well. The capital is asked of the + # numerals first, #721). Readers that run before the tag + # exists, or ask the word's text rather than its tag -- the + # strict suffix test, which the honorific peel and + # `lenient_comma_suffixes=False` ask -- take the same fact as + # an argument (#727). `letter_one_case` is the own-words fact + # for an own word and the whole name's for a clause's + # (docstring). The capital is asked of the # bare pattern at C level, no frame per one-letter word, and # needs no script test: no capital `_INITIAL` matches is in a # script without initials. 'é' and 'ж' stay as they were, diff --git a/nameparser/_pipeline/_comma.py b/nameparser/_pipeline/_comma.py index cd72e904..a6bfa658 100644 --- a/nameparser/_pipeline/_comma.py +++ b/nameparser/_pipeline/_comma.py @@ -244,7 +244,8 @@ def decide(state: ParseState) -> ParseState: licensed: set[int] = set() run = False if maybe: - names = name_word_count([tokens[i].text for i in seg0], lex, pol) + names = name_word_count([tokens[i].text for i in seg0], lex, pol, + letters_one_case=state.one_case) if names >= 2: # rules/C1: "An unlisted all-caps word joins the class in such # a part only where the name carries the contrast" @@ -283,7 +284,8 @@ def decide(state: ParseState) -> ParseState: for i in p: others.append(tokens[i].text) run = (bool(licensed) - and (not others or is_wholly_suffix(others, lex, pol)) + and (not others or is_wholly_suffix( + others, lex, pol, letters_one_case=state.one_case)) and not any(len(p) == 1 and is_single_letter_numeral(tokens[p[0]].text) for p in p1)) @@ -336,7 +338,8 @@ def decide(state: ParseState) -> ParseState: # rules/C1: "By default a recognized suffix word counts even # written like an initial" -- behind a whole name ('John Smith, V.') if reading is None and whole and is_wholly_suffix( - [tokens[i].text for i in part], lex, pol, one_case=None): + [tokens[i].text for i in part], lex, pol, one_case=None, + letters_one_case=state.one_case): reading = (True,) * len(p1) if reading is None: return state diff --git a/nameparser/_pipeline/_script_segment.py b/nameparser/_pipeline/_script_segment.py index e5f756d1..859a140c 100644 --- a/nameparser/_pipeline/_script_segment.py +++ b/nameparser/_pipeline/_script_segment.py @@ -242,7 +242,13 @@ def _is_post_nominal(state: ParseState, i: int) -> bool: not: a title never trails, so widening the peel's scan-back would move it off the real last name token. Split the predicate then, not before.""" - return is_suffix_strict(state.tokens[i].text, state.lexicon) + # rules/T4, #727: a bare lowercase letter in a name written in one + # case is vetoed as its capital is; the case fact is asked only of + # such a letter, so a CJK honorific (no case) never asks it + text = state.tokens[i].text + return is_suffix_strict(text, state.lexicon, + len(text) == 1 and text.islower() + and _one_case(state)) # rules/W2: "A part that is not name text — a post-nominal word @@ -553,10 +559,13 @@ def _peel_honorific_tail(state: ParseState, # the site test first: it is the cheaper half, and false # for every name whose part before the comma holds no glued # honorific, which spares those the case fact + # one fact, asked for the lean and for the strict test's + # initial veto alike (rules/T4, #727) if not (site is not None and is_wholly_suffix(second, state.lexicon, state.policy, - one_case=_one_case(state))): + one_case=(case := _one_case(state)), + letters_one_case=case)): site = both if site is None: return state diff --git a/nameparser/_pipeline/_segment.py b/nameparser/_pipeline/_segment.py index 77e7393e..4e17581d 100644 --- a/nameparser/_pipeline/_segment.py +++ b/nameparser/_pipeline/_segment.py @@ -99,9 +99,15 @@ def case_class() -> bool: # walk below passes `case_class()` for the leaning one. One # closure with a default rather than two whose bodies differ only # in that argument. + # The strict test also reads the case fact, for its initial veto + # (rules/T4, #727): asked only under that opt-in policy, so the + # default pays nothing for it. def suffixy(seg: tuple[int, ...], case: bool | None = None) -> bool: - return is_wholly_suffix([state.tokens[i].text for i in seg], - state.lexicon, state.policy, one_case=case) + return is_wholly_suffix( + [state.tokens[i].text for i in seg], state.lexicon, + state.policy, one_case=case, + letters_one_case=(None if state.policy.lenient_comma_suffixes + else case_class())) def class_run(seg: tuple[int, ...]) -> bool: # Every token of the run joins the ambiguous credential class diff --git a/nameparser/_pipeline/_vocab.py b/nameparser/_pipeline/_vocab.py index c5fc5c82..abd31427 100644 --- a/nameparser/_pipeline/_vocab.py +++ b/nameparser/_pipeline/_vocab.py @@ -497,18 +497,33 @@ def suffix_as_written(n: str, text: str, lexicon: Lexicon) -> bool: or n in lexicon.suffix_words -def _is_suffix_strict_n(n: str, text: str, lexicon: Lexicon) -> bool: - if is_initial(text): +def _is_suffix_strict_n(n: str, text: str, lexicon: Lexicon, + one_case: bool | None = None) -> bool: + # rules/T4: "in a name written wholly in one case, which says + # nothing about any letter in it, a bare letter is an initial + # wherever its capital would be" -- so the veto takes 'v' where it + # takes 'V' (#727). Lower case first, at C level: a capital is + # is_initial's already, and only a caller with the fact passes it + if is_initial(text) or ( + one_case and len(text) == 1 and text.islower() + and _INITIAL.fullmatch(text.upper()) is not None): # period-written ambiguous acronyms are exempt from the veto return _dotted(text) and \ n.replace(".", "") in lexicon.suffix_acronyms_ambiguous return suffix_as_written(n, text, lexicon) -def is_suffix_strict(text: str, lexicon: Lexicon) -> bool: +def is_suffix_strict(text: str, lexicon: Lexicon, + one_case: bool | None = None) -> bool: """v1's is_suffix: suffix_as_written with the initial veto ('V.' in - 'John V. Smith' is a middle initial, not roman five).""" - return _is_suffix_strict_n(_normalize(text), text, lexicon) + 'John V. Smith' is a middle initial, not roman five). + + `one_case` is whether the name's own words are written in one + case (ParseState.one_case): where they are, a bare lowercase + letter is the initial its capital is (rules/T4) and is vetoed as + its capital is. None or False is the veto by the writing alone, + which is v1's.""" + return _is_suffix_strict_n(_normalize(text), text, lexicon, one_case) def is_suffix_lenient(text: str, lexicon: Lexicon) -> bool: @@ -1028,7 +1043,8 @@ def surname_unit_tags(text: str, lexicon: Lexicon, def name_word_count(texts: Sequence[str], lexicon: Lexicon, - policy: Policy) -> int: + policy: Policy, letters_one_case: bool | None = None, + ) -> int: """How many of these texts are NAME words -- not suffix vocabulary, not title vocabulary. @@ -1050,9 +1066,10 @@ def name_word_count(texts: Sequence[str], lexicon: Lexicon, family name, so 'De La Cruz' is one name word and 'De La Cruz, Ed' reads as 'Royce, Ed' does. Why the other two joins are not read here: `SURNAME_UNIT_TAGS`. + + `letters_one_case` is `is_wholly_suffix`'s, for the same reason. """ - predicate = (is_suffix_lenient if policy.lenient_comma_suffixes - else is_suffix_strict) + lenient = policy.lenient_comma_suffixes # One pass: each token's name-word verdict, and whether any token # is a particle. With none, every token is a unit of its own and @@ -1064,7 +1081,9 @@ def name_word_count(texts: Sequence[str], lexicon: Lexicon, n = _normalize(text) if n in lexicon.particles: has_particle = True - names.append(not (predicate(text, lexicon) or n in lexicon.titles + suffix = (is_suffix_lenient(text, lexicon) if lenient + else is_suffix_strict(text, lexicon, letters_one_case)) + names.append(not (suffix or n in lexicon.titles or is_title_shaped(text))) if not has_particle: return sum(names) @@ -1080,7 +1099,8 @@ def name_word_count(texts: Sequence[str], lexicon: Lexicon, def is_wholly_suffix(texts: Sequence[str], lexicon: Lexicon, - policy: Policy, one_case: bool | None = None) -> bool: + policy: Policy, one_case: bool | None = None, + letters_one_case: bool | None = None) -> bool: """Every token in a RUN counts as a suffix -- C1's suffix test of a run, shared by `_comma.decide`, segment's C2 flag on a tail part and the peel (#319). @@ -1111,6 +1131,15 @@ def is_wholly_suffix(texts: Sequence[str], lexicon: Lexicon, as no lean and is this predicate's behavior in every release before 2.4. + `letters_one_case` is the same fact asked a different question, + and the two are kept apart because a caller that wants no lean + still wants the letters read by T4: under + `lenient_comma_suffixes=False` the strict test vetoes an + initial, and in a name written in one case a bare lowercase + letter is one ('john smith, v' reads as 'JOHN SMITH, V', #727). + The lenient test takes a suffix word whatever its shape, so the + default never reads it. + Neither Policy.unlisted_dotted_suffixes NOR Policy.unlisted_caps_suffixes reaches this predicate: admitting a by-shape member here unconditionally (an earlier version of this @@ -1130,8 +1159,7 @@ def is_wholly_suffix(texts: Sequence[str], lexicon: Lexicon, """ if not texts: return False - predicate = (is_suffix_lenient if policy.lenient_comma_suffixes - else is_suffix_strict) + lenient = policy.lenient_comma_suffixes # v1 expand_suffix_delimiter parity (#206): a configured delimiter # is TRANSPARENT in the all-suffix tests -- v1 split the part string # on the delimiter before checking, so the delimiter never counted @@ -1144,7 +1172,8 @@ def counts_as_suffix(text: str) -> bool: and ambiguous_class_member(text, lexicon) and ambiguous_lean(text, one_case) == "credential"): return True - return (predicate(text, lexicon) + return ((is_suffix_lenient(text, lexicon) if lenient + else is_suffix_strict(text, lexicon, letters_one_case)) or period_joined_vocab(text, lexicon) == "suffix" or (bool(cores) and splits_into_suffixes(text, cores, lexicon))) diff --git a/nameparser/_policy.py b/nameparser/_policy.py index cc3d2868..8241d4db 100644 --- a/nameparser/_policy.py +++ b/nameparser/_policy.py @@ -700,8 +700,10 @@ class Policy: #: of the default vocabulary that means the roman numerals "I" and #: "V"): "John Smith, V" reads as John Smith the fifth when True #: (the default, v1 behavior); False reads "V" as a given-name - #: initial instead (family "John Smith", given "V"). Multi-letter - #: suffixes ("III", "MD") parse the same either way. + #: initial instead (family "John Smith", given "V"), and "john + #: smith, v" the same way, a name written in one case reading as + #: its capitals (rules/T4). Multi-letter suffixes ("III", "MD") + #: parse the same either way. lenient_comma_suffixes: bool = True #: Excludes emoji from tokenization: they appear in no token, #: field, or rendered view. The original string keeps them (input diff --git a/tests/v2/cases.py b/tests/v2/cases.py index d7774342..45116506 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -3978,6 +3978,24 @@ def _check_cjk_shape_purity(self) -> None: "roman numeral), so 'V' is the family -- 1.4.0's " "reading field for field, which #718 restored -- and " "both forks, 'V' and 'Ma', report"), + Case("strict_comma_reads_a_one_case_numeral_as_its_capital", + "john smith, v", + {"given": "v", "family": "john smith"}, + policy=Policy(lenient_comma_suffixes=False), + notes="the strict test vetoes an initial, and in a name " + "written in one case a bare 'v' is one (T4), so this " + "reads as 'JOHN SMITH, V' does under the same knob: " + "the knob's 'V' as a given-name initial (#727; before, " + "suffix 'v')"), + Case("strict_comma_reports_a_one_case_numeral_in_a_third_part", + "smith, john, i", + {"given": "john", "family": "smith", "suffix": "i"}, + policy=Policy(lenient_comma_suffixes=False), + ambiguities=("comma-structure",), + notes="the third part holds no word the strict test calls a " + "suffix, so C2 reads it as suffix material and reports " + "the structure, as for 'SMITH, JOHN, I' (#727; before, " + "no report)"), Case("caps_run_declines_a_bound_given_head", "John Smith, ABDUL AP", {"given": "ABDUL AP", "family": "John Smith"}, policy=Policy(unlisted_caps_suffixes=CapsSuffixes.EVERYWHERE), @@ -11465,6 +11483,15 @@ def _check_cjk_shape_purity(self) -> None: "the same token downstream -- 'V.' is a middle " "initial, not a post-nominal", tolerated=True), + Case("ja_honorific_glued_before_a_one_case_numeral", "田中さん v", + {"given": "田中さん", "family": "v"}, + ambiguities=("suffix-or-name",), + notes="nothing in this name is written in a case, so the bare " + "'v' is the initial its capital is (T4): the scan-back " + "stops at it as at 'V', and this reads as '田中さん V' " + "does -- 1.4.0's reading too (#727; 2.1 through 2.4 " + "peeled さん and read suffix 'さん v')", + tolerated=True), Case("ja_honorific_with_a_period_no_comma", "田中さん 様.", {"family": "田中", "suffix": "さん 様."}, classification="fix(#320)", diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 8e37449b..972b003d 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -191,9 +191,10 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # was read into, on those rows under each order; 1191 -> 1210 the # same day: the PR review's rows (the chain-taken, family-first, # lowercase and maiden-clause numerals); 1210 -> 1243 the same day: - # #721's rows (#722). - assert claims == 1243, ( - f"the field sweep checks {claims} claims, recorded as 1243 on " + # #721's rows (#722); 1243 -> 1246 the same day: #727's + # '田中さん v' row (one suffix-or-name claim under each order). + assert claims == 1246, ( + f"the field sweep checks {claims} claims, recorded as 1246 on " f"2026-10-18") diff --git a/tests/v2/test_facade_cases.py b/tests/v2/test_facade_cases.py index 6573cd29..bc1637b1 100644 --- a/tests/v2/test_facade_cases.py +++ b/tests/v2/test_facade_cases.py @@ -175,6 +175,8 @@ "caps_switch_leaves_a_capitalized_title_a_title", "strict_comma_reads_the_dotted_numeral_as_a_name_word", "strict_comma_reads_the_bare_numeral_into_the_run", + "strict_comma_reads_a_one_case_numeral_as_its_capital", + "strict_comma_reports_a_one_case_numeral_in_a_third_part", "the_comma_flip_is_read_under_the_declared_order", "two_caps_credentials_peel_as_a_run", "the_caps_shape_is_script_agnostic_cyrillic", diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index d8d9b470..07a77f53 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -3343,8 +3343,11 @@ def test_every_one_letter_particle_veto_is_load_bearing( # bound given name, an initial, a nickname and a credential before or # after it -- and the shapes any bare letter meets (#723), with the # letters below, three heads (one Catalan, one holding an ambiguous -# acronym) and all three orders: -# 3,168 pairs, about 6,350 parses, 0.48s on py3.11 (2026-10-18). +# acronym) and all three orders -- and every shape holding a comma +# again under Policy(lenient_comma_suffixes=False), the one switch +# whose suffix test vetoes an initial by the word's own shape (#727); +# trimmed by shape, since the switch reaches only a comma's parts: +# 4,554 pairs, about 9,100 parses, 0.73s on py3.11 (2026-10-18). _ONE_CASE_SHAPES = ( "{a} {n}", "{a} {b} {n}", "{a} {n}, phd", "{a} {b} {n}, phd", "{a} {n}, jr.", "{b}, {a} {n}", ", {a} {n}", "{a} nee {b} {n}", @@ -3360,7 +3363,11 @@ def test_every_one_letter_particle_veto_is_load_bearing( # #723: the letter alone, before a maiden marker, before or after a # connective, opening the name and after an empty head "{n}", "{n} nee {b}", "{a} {n} y {b}", "{b}, {n} y {c}", "{n} {a}", - ", {n}") + ", {n}", + # #727: the letter alone after a comma, behind a whole name or in a + # third part, and opening a credential run; and after a glued CJK + # honorific, whose peel asks the suffix test by text + "{a} {b}, {n}", "{b}, {a}, {n}", "{a} {b}, {n} phd", "田中さん {n}") #: the one-letter numerals and a listed and a shape-only spelling of #: each longer length, then one letter from each other fork a bare #: letter meets (#723): no vocabulary, a connective, a marked one @@ -3369,12 +3376,14 @@ def test_every_one_letter_particle_veto_is_load_bearing( _ONE_CASE_HEADS = (("john", "smith", "jones"), ("carod", "puig", "rovira"), ("mary", "ma", "lee")) #: the negative controls, measured 2026-10-18 on this grid, which -#: 16ee2851 (before #723, its numerals already guarded) disagreed on -#: 30 of: with classify's one-case letter initial taken out, and with -#: the numeral fork's one-case letter in front taken out -- each guard -#: seen on its own -_ONE_CASE_UNGUARDED = 306 +#: 2444f847 (before #727) disagreed on 72 of: with classify's one-case +#: letter initial taken out, with the numeral fork's one-case letter +#: in front taken out, and with the strict suffix test's one-case veto +#: taken out (#727: the 54 strict comma pairs and the 18 honorific +#: ones, which are 2444f847's 72) -- each guard seen on its own +_ONE_CASE_UNGUARDED = 387 _ONE_CASE_LETTER_IN_FRONT_UNGUARDED = 117 +_ONE_CASE_STRICT_VETO_UNGUARDED = 72 def _one_case_disagreements() -> list[str]: @@ -3382,13 +3391,17 @@ def sig(name: ParsedName) -> tuple[object, ...]: return ({k: v.lower() for k, v in name.as_dict().items() if v}, tuple(sorted(a.kind.value for a in name.ambiguities))) out = [] + commas = tuple(s for s in _ONE_CASE_SHAPES if "," in s) for order in (GIVEN_FIRST, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST): - parser = Parser(policy=Policy(name_order=order)) - for shape, letter, (a, b, c) in itertools.product( - _ONE_CASE_SHAPES, _ONE_CASE_LETTERS, _ONE_CASE_HEADS): - lower = shape.format(a=a, b=b, c=c, n=letter) - if sig(parser.parse(lower)) != sig(parser.parse(lower.upper())): - out.append(f"[{order}] {lower!r}") + for strict, shapes in ((False, _ONE_CASE_SHAPES), (True, commas)): + parser = Parser(policy=Policy( + name_order=order, lenient_comma_suffixes=not strict)) + for shape, letter, (a, b, c) in itertools.product( + shapes, _ONE_CASE_LETTERS, _ONE_CASE_HEADS): + lower = shape.format(a=a, b=b, c=c, n=letter) + if (sig(parser.parse(lower)) + != sig(parser.parse(lower.upper()))): + out.append(f"[{order}{' strict' * strict}] {lower!r}") return out @@ -3405,6 +3418,19 @@ def test_the_one_case_grid_sees_the_letter_initial( assert len(_one_case_disagreements()) == _ONE_CASE_UNGUARDED +def test_the_one_case_grid_sees_the_strict_veto( + monkeypatch: pytest.MonkeyPatch) -> None: + # the strict suffix test without its one-case input (#727): the + # veto by the word's own writing alone, as before + from nameparser._pipeline import _vocab + real = _vocab._is_suffix_strict_n + monkeypatch.setattr( + _vocab, "_is_suffix_strict_n", + lambda n, text, lexicon, one_case=None: real(n, text, lexicon)) + assert (len(_one_case_disagreements()) + == _ONE_CASE_STRICT_VETO_UNGUARDED) + + def test_the_one_case_grid_sees_the_letter_in_front( monkeypatch: pytest.MonkeyPatch) -> None: from nameparser._pipeline import _vocab diff --git a/tools/differential/corpus_cjk.jsonl b/tools/differential/corpus_cjk.jsonl index d18a8191..de45f006 100644 --- a/tools/differential/corpus_cjk.jsonl +++ b/tools/differential/corpus_cjk.jsonl @@ -85,6 +85,7 @@ "田中さん" "田中さん II" "田中さん V." +"田中さん v" "田中さん 様." "田中さん, Dr." "田中さん, Ph. D." diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index a6d645e1..3486a1d6 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -5627,6 +5627,7 @@ {"name": "田中さん", "family": "田中", "suffix": "さん", "initials": "田."} {"name": "田中さん II", "family": "田中", "suffix": "さん II", "initials": "田."} {"name": "田中さん V.", "given": "田中さん", "family": "V.", "initials": "田. V."} +{"name": "田中さん v", "given": "田中さん", "family": "v", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "田. v."} {"name": "田中さん 様.", "family": "田中", "suffix": "さん 様.", "initials": "田."} {"name": "田中さん, Dr.", "title": "Dr.", "family": "田中さん", "initials": "田."} {"name": "田中さん, Ph. D.", "family": "田中", "suffix": "さん, Ph. D.", "initials": "田."} From 5d552c1538ebe28ed0d2127bae4be368990f1aee Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 18 Oct 2026 19:44:04 -0700 Subject: [PATCH 2/2] Close #727's Open line and correct the strict-veto quote (#727 review) Co-Authored-By: Claude Opus 5.5 --- docs/design/decisions/T4-bare-letter-as-an-initial.md | 7 +++---- nameparser/_pipeline/_pieces.py | 8 ++++---- tests/v2/cases.py | 5 +++-- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/design/decisions/T4-bare-letter-as-an-initial.md b/docs/design/decisions/T4-bare-letter-as-an-initial.md index e65aaeb1..f6c9f111 100644 --- a/docs/design/decisions/T4-bare-letter-as-an-initial.md +++ b/docs/design/decisions/T4-bare-letter-as-an-initial.md @@ -3,10 +3,9 @@ - 2026-10-18, #723 — A ONE-CASE BARE LETTER IS THE INITIAL ITS CAPITAL IS. #721 applied S10's "a name written wholly in one case says nothing about any letter in it" to the one-letter roman numerals only (decisions/S10, the Open line of its /simplify entry), so a bare lowercase `j` was a name word where `J` was an initial, and every rule that reads an initial differently read the two spellings of one name differently: M4 (`j nee smith` family 'j', `J NEE SMITH` given 'J'), O5 (a lone `j` reported `given-or-family`, `J` did not), and P3's count, which passes over an initial (`jose j y lopez` family 'j y lopez', `JOSE J Y LOPEZ` middle 'J Y', family 'LOPEZ', which 1.4.0 read too). The rule is now stated once, for every letter, and the numeral case is an instance of it: in a name written in one case, a bare letter is tagged an initial wherever its capital would be. Its capital is asked of the bare-capital shape (one ASCII capital), so `é` and `ж`, whose capitals are no initial, read as before in both spellings. `ı` and `ſ`, whose capitals are `I` and `S`, are tagged initials too; `ı`'s twin is not a true one, since `I` is also the marked Catalan connective and `ı` is in no vocabulary (`JOSE I LOPEZ` reports `conjunction-or-initial`, `jose ı lopez` nothing). Measured on the first commit's tree against 16ee2851, a dated snapshot from a scratch grid not committed: letters a–z, `é` and `ж`; heads (a, b, c) = (john, smith, jones), (jose, lopez, garcia), (mary, ma, lee); all three orders; the 35 shapes `{n}`, `{n} nee {b}`, `{a} {n}`, `{n} {a}`, `{a} {n} {b}`, `{a} {n} y {b}`, `{a} {n} e {b}`, `{b}, {n}`, `{b}, {n} y {c}`, `{b}, {a} {n}`, `, {n}`, `{n}, phd`, `{a} {n}, phd`, `{a} {n}, jr.`, `{a} nee {b} {n}`, `{a} nee {n}`, `{a} {b} nee {n} {c}`, `dr {n}`, `dr {n} {b}`, `sir {n} {b}`, `{a} van {n}`, `{n} van {b}`, `{a} van {n} {b}`, `abdul {n}`, `{n} {a} {b}`, `{a} {n} {n} {b}`, `{a} {n}. {b}`, `({c}) {n} {b}`, `{n} y {b}`, `{a} {n} phd`, `{a} {b} {n} prof.`, `{n} {b} {n}`, `{n} {n}`, `{a} {n} de la {b}`, `{b} {n} jr`; each lowercase text against its `.upper()`, compared on case-folded fields, sorted report kinds and `initials()`, keyed by order and text, so the 8,820 products collapse to 7,794 distinct pairs where a shape has no head. 16ee2851 disagreed on 504 of them, this tree on none, and no capital spelling moved. The committed invariant is `test_a_one_case_name_reads_as_its_capitals` (tests/v2/test_properties.py), whose grid gains the issue's shapes and the letters `j`, `y` and `e`; its stated controls are the recompute. WHAT MOVES: lower-case letters in a name whose own words are written in one case, each onto its capital twin's reading — wholly lower-case names, and (since the case question is asked of the own words, P3) a one-case name beside a clause of another case, so `j nee Smith` reads given 'j' as `J nee Smith` does. In three classes — a lone letter loses O5's report, a letter before a maiden marker keeps the given name M4 would have taken, and a letter beside a single-letter link is passed over by P3's count, so the link in what is then a three-word name stays a name word. No corpus name moved (the snapshot and the CLDR, GND and native-script agreement records are unchanged). And P3's Accepted paragraph on a declined marker loses its "one shape" in which the name's own words read differently in the two spellings: `dr. née juan y lopez` read family 'juan y lopez' where `DR. NÉE JUAN Y LOPEZ` reads middle 'JUAN Y', because the `y` past the declined marker is outside the own words P3's fork reads; it is this rule's letter now, and reads as its capital does. And one move the snapshot cannot see (R4): case repair of a connective letter in a one-case maiden clause, now tagged an initial as its capital is, so `jose lopez nee y garcia` repairs to `Jose Lopez (Y Garcia)` where it gave `(y Garcia)` — the same tag in the two spellings that P3's rewritten paragraph intends. No corpus name's `capitalized()` moved (the design-docs review compared every one across the two trees). - A GAP THE RULE DOES NOT CLOSE, found by the design-docs review and present on 16ee2851 too: under `Policy(lenient_comma_suffixes=False)` the strict comma-suffix test asks a one-letter numeral's shape as written, not its tag, so `john smith, v` reads suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reports nothing where `SMITH, JOHN, I` reports `comma-structure`. The committed grid runs the default policy only. Left for its own change. - Open: #727 — whether the strict comma test should read the tag. + A GAP THE RULE DOES NOT CLOSE, found by the design-docs review and present on 16ee2851 too: under `Policy(lenient_comma_suffixes=False)` the strict comma-suffix test asks a one-letter numeral's shape as written, not its tag, so `john smith, v` reads suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reports nothing where `SMITH, JOHN, I` reports `comma-structure`. The committed grid runs the default policy only. Left for its own change, and taken by the #727 entry below (was Open: #727). The connective letters are left where P3 put them. Among the name's own words, P3's fork answers by vocabulary before this rule is asked — the marked `e` and `i` read as initials and report, the unmarked `y` as a connective — and does so in both spellings, so the invariant already held for them; a connective letter outside the own words was always read by the bare-capital shape in its capital spelling, and its lower-case spelling now follows. - 2026-10-18, #723's code review — A LETTER OUTSIDE THE OWN WORDS ASKS THE WHOLE NAME'S CASE. The first version gated the new tag on the one-case fact classify records, which is asked of the name's own words only (P3), so a lower-case letter in a clause of a name whose own words are capitals passed it: `MARIA GARCIA NEE LOPEZ y FERNANDEZ` and `MARIA GARCIA (LOPEZ y FERNANDEZ)` tagged the `y` an initial (case repair wrote `Lopez Y Fernandez`), and `DR. NÉE JUAN y LOPEZ` moved from family 'JUAN y LOPEZ' to middle 'JUAN y' — mixed-case writing, which this rule reads as no initial, and moves the twin-only grids could not see, since every pair in them is wholly one case. The fix asks a word outside the own words about the whole name instead; an own word keeps the own-words fact, so a clause beside the name still changes nothing about the name's own letters (P3). The defect predated #723 for the numeral letters, which #721 gated the same way; they are read by this rule now and move with it, and the move is not only a tag (found by the design-docs review). After a marker among capitals, a lower-case numeral no longer holds the clause up as an initial would: `MARY SMITH NEE v JONES` reads middle 'SMITH NEE v', family 'JONES' (on 16ee2851, maiden 'v JONES'), `MARY SMITH NEE i JONES` family 'NEE i JONES', and `JANE DOE NEE SMITH v` drops `suffix-or-name`. Each is the reading of its mixed-case spelling (`Mary Smith nee v Jones`) and of 2.4.2 (and 1.4.0 for the first), so only the unreleased #721 had moved them and no release bullet is owed; the capital twin `MARY SMITH NEE V JONES` keeps maiden 'V JONES'. Pinned by the row `a_lowercase_numeral_after_a_marker_among_capitals_is_no_initial`. Pinned by `test_a_lowercase_clause_letter_among_capitals_is_no_initial` and the case row `a_lowercase_letter_past_a_declined_marker_among_capitals_joins`. The first entry's figures were measured on the first version, and the twin grids are unchanged by the fix (both spellings of every pair in them are wholly one case). -- 2026-10-18, #727 — THE STRICT SUFFIX TEST VETOES A ONE-CASE LETTER AS IT VETOES ITS CAPITAL. The gap the first entry recorded: `is_suffix_strict` vetoes an initial by the word's text (`is_initial`), not by classify's tag, so a one-case lowercase `v` or `i` — an initial under this rule — passed where its capital was vetoed. Two callers reached it. Under `Policy(lenient_comma_suffixes=False)`, C1's comma tests (`is_wholly_suffix` and the name-word count): `john smith, v` read suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reported nothing where its capitals report `comma-structure`. And under the default policy, W2's scan-back (`_script_segment._is_post_nominal`): `田中さん v` peeled さん and read suffix 'さん v' where `田中さん V` reads given '田中さん', family 'V' — a CJK name with a Latin letter beside it is one case under the own-words test, since neither script contrasts the letter. The strict test now takes the one-case fact as an argument and vetoes a bare lowercase letter whose capital is A to Z where it is true; segment, `_comma` and script_segment pass the fact they already hold, and segment asks for it only under the strict knob. A separate keyword from `is_wholly_suffix`'s `one_case`, which is the credential LEAN and is None by design at callers that want no lean; they still want the letters read by this rule. The lenient default takes a listed suffix word whatever its shape and never reads the argument. - Measured: the committed invariant's grid gains the three comma shapes and `田中さん {n}`, and runs every comma-holding shape again under the strict knob (the knob reaches nothing else); 2444f847 disagreed on 72 of its 4,554 pairs (54 strict comma pairs, 18 honorific), this tree on none, and the third recorded control is that 72. `田中さん v` now reads as 1.4.0 read it; 2.1 through 2.4.2 peeled it. No corpus name moved. +- 2026-10-18, #727 — THE STRICT SUFFIX TEST VETOES A ONE-CASE LETTER AS IT VETOES ITS CAPITAL. The gap the first entry recorded: `is_suffix_strict` vetoes an initial by the word's text (`is_initial`), not by classify's tag, so a one-case lowercase `v` or `i` — an initial under this rule — passed where its capital was vetoed. Two callers reached it. Under `Policy(lenient_comma_suffixes=False)`, C1's comma tests (`is_wholly_suffix` and the name-word count): `john smith, v` read suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reported nothing where its capitals report `comma-structure`. And under the default policy, W2's scan-back (`_script_segment._is_post_nominal`): `田中さん v` peeled さん and read suffix 'さん v' where `田中さん V` reads given '田中さん', family 'V' — a CJK name with a Latin letter beside it is one case under the own-words test, the letter being the only cased word in it, so nothing in the writing contrasts it. The strict test now takes the one-case fact as an argument and vetoes a bare lowercase letter whose capital is A to Z where it is true; segment, `_comma` and script_segment pass the fact they already hold, and segment asks for it only under the strict knob. A separate keyword from `is_wholly_suffix`'s `one_case`, which is the credential LEAN and is None by design at callers that want no lean; they still want the letters read by this rule. The lenient default takes a listed suffix word whatever its shape and never reads the argument. + Measured: the committed invariant's grid gains the three comma shapes and `田中さん {n}`, and runs every comma-holding shape again under the strict knob (the knob reaches nothing else); 2444f847 disagreed on 72 of its 4,554 pairs (54 strict comma pairs, 18 honorific), this tree on none, and the third recorded control is that 72. `田中さん v` now reads as 1.4.0 read it; 2.1 through 2.4.2 peeled it (suffix 'さん, v' in 2.1 and 2.2, 'さん v' from 2.3). No corpus name moved. Open: none from this entry; #727 closes the gap the first entry left open. diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index e3f14252..c659f057 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -800,10 +800,10 @@ def segment_suffix_reading(pieces: Sequence[Sequence[int]], title to read by). `lenient` is Policy.lenient_comma_suffixes, and only the numeral - continuation consults it. C1: "by default a recognized suffix word - counts even written like an initial, while strict mode vetoes - initial-shaped words" -- so under strict the veto stands and the - run ends where it always did. Reading no policy here silently + continuation consults it. C1: "while strict mode vetoes a word read + as an initial" -- so under strict the veto stands and the run ends + where it always did; the walk reads the token's `initial` tag, so a + one-case lowercase letter is vetoed as its capital is (rules/T4). Reading no policy here silently overrode the one knob a caller sets to prevent exactly this. The FAMILY_COMMA rule that segment 0 is the family name rests diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 45116506..09f2e401 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -11489,8 +11489,9 @@ def _check_cjk_shape_purity(self) -> None: notes="nothing in this name is written in a case, so the bare " "'v' is the initial its capital is (T4): the scan-back " "stops at it as at 'V', and this reads as '田中さん V' " - "does -- 1.4.0's reading too (#727; 2.1 through 2.4 " - "peeled さん and read suffix 'さん v')", + "does -- 1.4.0's reading too (#727; 2.1 through 2.4.2 " + "peeled さん, reading suffix 'さん, v' in 2.1 and 2.2 " + "and 'さん v' from 2.3)", tolerated=True), Case("ja_honorific_with_a_period_no_comma", "田中さん 様.", {"family": "田中", "suffix": "さん 様."},