From 7cca615a7125966a72978836502e221ed802cbf4 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 18 Oct 2026 20:01:29 -0700 Subject: [PATCH 1/3] Ask the maiden take's numeral candidates the one-case fact (#731) M2's take builds a clause-free view from the clause's trailing candidates, and its candidate test asked the numeral fork's shape test with no one-case fact, as a superset. But a candidate is also cut out of the view, which changes the word in front of the numeral: in 'john smith nee x x' the lowercase first 'x' is not initial-shaped, so the second became a candidate and the view read it behind 'smith' as the generation (suffix 'x', maiden 'x'), where 'JOHN SMITH NEE X X' keeps maiden 'X X'. trailing_candidates now takes the one-case fact; 'john smith nee j x' moves the same way. The superset check still passes. The one-case grid gains the two clause shapes: ad486f66 disagreed on 36 of 4,752 pairs, this tree on none; a fourth recorded control holds the 36. 2.3.0 through 2.4.2 gave the old lowercase reading, so #723's release bullet links this too. No corpus name moved. T4 says it, with an example; decisions/T4 records it. Closes #731 Co-Authored-By: Claude Opus 5.5 --- .../decisions/T4-bare-letter-as-an-initial.md | 2 + docs/design/rules/M2-maiden-marker-rule.md | 2 +- .../rules/T4-bare-letter-as-an-initial.md | 6 ++- docs/release_log.rst | 2 +- nameparser/_pipeline/_group.py | 2 +- nameparser/_pipeline/_pieces.py | 14 +++++-- nameparser/_pipeline/_vocab.py | 8 ++-- tests/v2/cases.py | 15 ++++++++ tests/v2/pipeline/test_pieces.py | 4 +- tests/v2/test_properties.py | 37 +++++++++++++++---- tools/differential/corpus_rules.jsonl | 1 + tools/differential/snapshot.jsonl | 1 + 12 files changed, 73 insertions(+), 21 deletions(-) diff --git a/docs/design/decisions/T4-bare-letter-as-an-initial.md b/docs/design/decisions/T4-bare-letter-as-an-initial.md index f6c9f111..6b3c939f 100644 --- a/docs/design/decisions/T4-bare-letter-as-an-initial.md +++ b/docs/design/decisions/T4-bare-letter-as-an-initial.md @@ -9,3 +9,5 @@ - 2026-10-18, #727 — THE STRICT SUFFIX TEST VETOES A ONE-CASE LETTER AS IT VETOES ITS CAPITAL. The gap the first entry recorded: `is_suffix_strict` vetoes an initial by the word's text (`is_initial`), not by classify's tag, so a one-case lowercase `v` or `i` — an initial under this rule — passed where its capital was vetoed. Two callers reached it. Under `Policy(lenient_comma_suffixes=False)`, C1's comma tests (`is_wholly_suffix` and the name-word count): `john smith, v` read suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reported nothing where its capitals report `comma-structure`. And under the default policy, W2's scan-back (`_script_segment._is_post_nominal`): `田中さん v` peeled さん and read suffix 'さん v' where `田中さん V` reads given '田中さん', family 'V' — a CJK name with a Latin letter beside it is one case under the own-words test, the letter being the only cased word in it, so nothing in the writing contrasts it. The strict test now takes the one-case fact as an argument and vetoes a bare lowercase letter whose capital is A to Z where it is true; segment, `_comma` and script_segment pass the fact they already hold, and segment asks for it only under the strict knob. A separate keyword from `is_wholly_suffix`'s `one_case`, which is the credential LEAN and is None by design at callers that want no lean; they still want the letters read by this rule. The lenient default takes a listed suffix word whatever its shape and never reads the argument. Measured: the committed invariant's grid gains the three comma shapes and `田中さん {n}`, and runs every comma-holding shape again under the strict knob (the knob reaches nothing else); 2444f847 disagreed on 72 of its 4,554 pairs (54 strict comma pairs, 18 honorific), this tree on none, and the third recorded control is that 72. `田中さん v` now reads as 1.4.0 read it; 2.1 through 2.4.2 peeled it (suffix 'さん, v' in 2.1 and 2.2, 'さん v' from 2.3). No corpus name moved. Open: none from this entry; #727 closes the gap the first entry left open. +- 2026-10-18, #731 — THE MAIDEN TAKE'S CANDIDATE TEST ASKS THE ONE-CASE FACT. M2's take (#601, #610) builds a clause-free view of the name from the clause's trailing candidates and lets `tail_reading` decide what the run takes. Its candidate test (`_pieces.trailing_candidates`) asked the numeral fork's shape test with no one-case fact, as a superset of what the fork admits. But a candidate is also cut out of the view, which changes the word in front of the numeral there: in `john smith nee x x` the lowercase first `x` is not initial-shaped, so the second became a candidate, and the view read it behind `smith` with words to spare, as the generation (suffix 'x', maiden 'x', reported). `JOHN SMITH NEE X X` keeps maiden 'X X', the capital first `X` refusing the candidate. Now the test asks with the name's one-case fact, as the fork does, so the lowercase letter in front refuses it too; `john smith nee j x` moves the same way. The superset check (`test_the_trailing_candidates_cover_what_the_tail_reading_takes`) still passes: its oracle reads the view with every clause word after the first, where the numeral keeps its real neighbour. + Measured: the one-case grid gains `{a} {b} nee {n} {n}` and `{a} {b} nee j {n}`. ad486f66 disagreed on 36 of its 4,752 pairs, all in those shapes; this tree on none, and the fourth recorded control is that 36. 2.3.0 through 2.4.2 gave the lowercase reading above, so the release bullet for #723 carries this too. No corpus name moved. diff --git a/docs/design/rules/M2-maiden-marker-rule.md b/docs/design/rules/M2-maiden-marker-rule.md index 6293b4aa..32e1d6ac 100644 --- a/docs/design/rules/M2-maiden-marker-rule.md +++ b/docs/design/rules/M2-maiden-marker-rule.md @@ -143,5 +143,5 @@ M2. Rationale: a maiden marker announces that what follows it is the Accepted: a marker behind a credential is an ordinary word, and the credential's run (S8) takes it in with everything after it. "Jane Doe Jr. nee Smith" → suffix="Jr. nee Smith" · boundary - history: decisions/M2 · interacts: P2, P3, P5, P6, R1, R2, M1, S1, S2, H1, H5, C1, S4, S8 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py + history: decisions/M2 · interacts: P2, P3, P5, P6, R1, R2, M1, S1, S2, H1, H5, C1, S4, S8, T4 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py diff --git a/docs/design/rules/T4-bare-letter-as-an-initial.md b/docs/design/rules/T4-bare-letter-as-an-initial.md index fa367d8a..e86e7681 100644 --- a/docs/design/rules/T4-bare-letter-as-an-initial.md +++ b/docs/design/rules/T4-bare-letter-as-an-initial.md @@ -17,7 +17,8 @@ T4. Rationale: one letter standing for a name word is how an count passes over one, S10 reads a numeral written as one as the capital numeral it is, and a suffix test that vetoes an initial vetoes it — C1's strict mode, and the scan-back of W2's - peel. A letter whose capital is not one + peel — and M2's take keeps a numeral its clause ends with where + the letter in front is one. A letter whose capital is not one of those — é, ж — is no initial in either spelling, and in a name of mixed case a bare lower-case letter is no initial. As P3 asks it, the case question is asked of the name's own words, @@ -38,10 +39,11 @@ T4. Rationale: one letter standing for a name word is how an "smith, j y lopez" → given="j" "dr. née juan y lopez" → middle="juan y" "john j v" → family="v" + "john smith nee x x" → maiden="x x" "Jose j y Lopez" → family="j y Lopez" · boundary "é nee smith" → family="é" · boundary "j nee Smith" → given="j" "DR. NÉE JUAN y LOPEZ" → family="JUAN y LOPEZ" · boundary "jose e lopez" → ambiguities=("conjunction-or-initial",) "jose y lopez" → middle="y" - history: decisions/T4 · interacts: M4, O5, P2, P3, P7, S10, R3, R4, C1, W2 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_vocab.py + history: decisions/T4 · interacts: M2, M4, O5, P2, P3, P7, S10, R3, R4, C1, W2 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_vocab.py diff --git a/docs/release_log.rst b/docs/release_log.rst index e7d73746..a592a68a 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -15,7 +15,7 @@ Release Log - **Fix case repair capitalizing the Dutch particle 't.** ``jacobus henricus van 't hoff`` repairs to ``Jacobus Henricus van 't Hoff``, where 1.4.0 through 2.4.2 gave ``van 'T Hoff``. :issue:`700` - **Add Gov as a title.** ``Gov Smith`` gives title ``Gov``, last ``Smith``, where 1.4.0 through 2.4.2 gave first ``Gov``, and ``Lt. Gov. Smith, John`` gives title ``Lt. Gov.`` with the change above. :issue:`701` - **Fix a capital V, I or X after a lone first name being read as a generational suffix, leaving no last name.** ``John V`` gives first ``John``, last ``V``, and ``Malcolm X`` last ``X``, where 2.0.0 through 2.4.2 gave suffix ``V`` and ``X`` with no last name, restoring 1.4.0's reading. A title is no word to spare, so ``Dr. Smith V`` gives first ``Smith``, last ``V`` (1.4.0's reading too) where 2.0.0 through 2.4.2 gave last ``Smith``, suffix ``V``; a name with a word to spare keeps the suffix, and ``John Smith V`` still gives suffix ``V``. A name written all in lower case reads the same way (``john v`` gives last ``v``). :issue:`718`, :issue:`721` - - **Fix a single letter in a name written all in lower case reading differently from the same name in capitals.** ``jose j y lopez`` gives first ``jose``, middle ``j y``, last ``lopez``, as ``JOSE J Y LOPEZ`` does, where 1.4.0 through 2.4.2 gave last ``j y lopez``. A lone letter before a maiden marker stays the first name (``j nee smith`` as ``J NEE SMITH``), ``田中さん v`` reads as ``田中さん V`` does, and ``Policy(lenient_comma_suffixes=False)`` reads ``john smith, v`` as it reads ``JOHN SMITH, V``. :issue:`723`, :issue:`727` + - **Fix a single letter in a name written all in lower case reading differently from the same name in capitals.** ``jose j y lopez`` gives first ``jose``, middle ``j y``, last ``lopez``, as ``JOSE J Y LOPEZ`` does, where 1.4.0 through 2.4.2 gave last ``j y lopez``. A lone letter before a maiden marker stays the first name (``j nee smith`` as ``J NEE SMITH``), ``田中さん v`` reads as ``田中さん V`` does, and ``Policy(lenient_comma_suffixes=False)`` reads ``john smith, v`` as it reads ``JOHN SMITH, V``. :issue:`723`, :issue:`727`, :issue:`731` * 2.4.2 - October 13, 2026 diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 3383bf12..667713e1 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -303,7 +303,7 @@ def _maiden_take(pieces: Sequence[Sequence[int]], for j in range(lo, end)] if site is not ClauseSite.TRAILING: assert_never(site) - c = trailing_candidates(lo, pieces, ptags, tokens) + c = trailing_candidates(lo, pieces, ptags, tokens, one_case) end = len(pieces) released: list[tuple[int, Role]] = [] # the view's picks, for the kept member below (#718) diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index c659f057..4c061706 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -1506,7 +1506,8 @@ def is_trailing_title_word(piece: Sequence[int], ptags: Set[str], # beside the peel and the chain whose admissions it has to cover (#610) def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], - tokens: Sequence[WorkToken]) -> int: + tokens: Sequence[WorkToken], + one_case: bool | None) -> int: """The first of the pieces after `lo` that `tail_reading` might read into the trailing run: `pieces[c:]`, a SUPERSET of what it takes, for the maiden take to build its clause-free view over and let @@ -1520,7 +1521,14 @@ def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], but such title words and group-flagged suffix pieces: the chain takes the titles first and the walk never holds the flagged pieces ('Ph. D.'), so either way the numeral is the walk's last piece and - the fork reads it. Then back to the first + the fork reads it. The shape test is asked with the name's + one-case fact, as the fork asks it: a candidate is not only + admitted but cut out of the view, so the view puts a different + word in front of it, and a numeral the fork would refuse for the + letter in front is a numeral the view reads. A bare lowercase + letter in front of one in a one-case name is the initial its + capital is (rules/T4), so 'john smith nee x x' keeps its clause as + 'JOHN SMITH NEE X X' does (#731). Then back to the first credential that starts #602's run, whose words the run takes whatever they are. A BARE title word is no candidate: H5's chain does not take one, and in the view it would only inflate the count @@ -1541,7 +1549,7 @@ def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], not tokens[piece[0]].tags.isdisjoint(_TRAILING_WORD_TAGS) or (titles_behind and is_trailing_numeral_suffix( tokens[piece[0]].text, - tokens[pieces[k - 1][0]].text, None))))): + tokens[pieces[k - 1][0]].text, one_case))))): break # a group-flagged suffix piece is outside the peel's walk # altogether (`peel_walk` drops it), so it is behind the diff --git a/nameparser/_pipeline/_vocab.py b/nameparser/_pipeline/_vocab.py index abd31427..f0c202d8 100644 --- a/nameparser/_pipeline/_vocab.py +++ b/nameparser/_pipeline/_vocab.py @@ -277,9 +277,11 @@ def is_trailing_numeral_suffix(text: str, preceding: str, the word in front is read as its capital would be, so a bare 'j' is initial-shaped as 'J' is (rules/S10, #721's review: 'john j v' reads as 'JOHN J V'), while 'é', 'ж' or an uncased word, whose - capitals are not, stay as they were. None (the maiden take's - superset view passes it) gets the shape test alone, which admits - more and so stays a superset.""" + capitals are not, stay as they were. None is the shape test + alone. The maiden take's candidate view once passed it as a safe + superset, but admitting a candidate also cuts it out of the view, + changing the word in front of the numeral, so it asks with the + fact as the fork does (#731).""" return (_ROMAN.match(text) is not None and not is_initial_shaped(preceding) and not (one_case and is_initial_shaped(preceding.upper()))) diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 09f2e401..c1faa9cc 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -7686,6 +7686,21 @@ def _check_cjk_shape_purity(self) -> None: "capitals, but the name is not written in one case, so " "the lower-case 'y' past the marker is no initial (T4) " "and joins (#723's review)"), + Case("a_one_case_doubled_numeral_ending_a_clause_stays_in_it", + "john smith nee x x", + {"given": "john", "family": "smith", "maiden": "x x"}, + classification="fix(#274)", + notes="read as 'JOHN SMITH NEE X X' is (T4): the first 'x' is " + "the initial its capital is, so the take's view does " + "not cut the second out of the clause and read it as " + "the generation; 2.3 through 2.4.2 read suffix 'x', " + "maiden 'x' (#731). 1.4.0 had no maiden markers"), + Case("a_one_case_numeral_behind_a_letter_ending_a_clause", + "john smith nee j x", + {"given": "john", "family": "smith", "maiden": "j x"}, + classification="fix(#274)", + notes="the row above with a plain letter in front, as 'JOHN " + "SMITH NEE J X' reads (#731)"), Case("a_lowercase_numeral_after_a_marker_among_capitals_is_no_initial", "MARY SMITH NEE v JONES", {"given": "MARY", "middle": "SMITH NEE v", "family": "JONES"}, diff --git a/tests/v2/pipeline/test_pieces.py b/tests/v2/pipeline/test_pieces.py index 79bb3198..1165d980 100644 --- a/tests/v2/pipeline/test_pieces.py +++ b/tests/v2/pipeline/test_pieces.py @@ -921,8 +921,8 @@ def _candidate_misses(monkeypatch: pytest.MonkeyPatch, def cut(lo: int, pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], - tokens: Sequence[WorkToken]) -> int: - c = candidates(lo, pieces, ptags, tokens) + tokens: Sequence[WorkToken], one_case: bool | None) -> int: + c = candidates(lo, pieces, ptags, tokens, one_case) seen.append((lo, c)) return c diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index 07a77f53..cec802a0 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -3347,7 +3347,7 @@ def test_every_one_letter_particle_veto_is_load_bearing( # again under Policy(lenient_comma_suffixes=False), the one switch # whose suffix test vetoes an initial by the word's own shape (#727); # trimmed by shape, since the switch reaches only a comma's parts: -# 4,554 pairs, about 9,100 parses, 0.73s on py3.11 (2026-10-18). +# 4,752 pairs, about 9,500 parses, 0.75s on py3.11 (2026-10-18). _ONE_CASE_SHAPES = ( "{a} {n}", "{a} {b} {n}", "{a} {n}, phd", "{a} {b} {n}, phd", "{a} {n}, jr.", "{b}, {a} {n}", ", {a} {n}", "{a} nee {b} {n}", @@ -3367,7 +3367,11 @@ def test_every_one_letter_particle_veto_is_load_bearing( # #727: the letter alone after a comma, behind a whole name or in a # third part, and opening a credential run; and after a glued CJK # honorific, whose peel asks the suffix test by text - "{a} {b}, {n}", "{b}, {a}, {n}", "{a} {b}, {n} phd", "田中さん {n}") + "{a} {b}, {n}", "{b}, {a}, {n}", "{a} {b}, {n} phd", "田中さん {n}", + # #731: the letter doubled, and behind a bare letter, ending a + # maiden clause, where the take reads the numeral fork's shape test + # to choose the words its clause-free view holds + "{a} {b} nee {n} {n}", "{a} {b} nee j {n}") #: the one-letter numerals and a listed and a shape-only spelling of #: each longer length, then one letter from each other fork a bare #: letter meets (#723): no vocabulary, a connective, a marked one @@ -3376,14 +3380,18 @@ def test_every_one_letter_particle_veto_is_load_bearing( _ONE_CASE_HEADS = (("john", "smith", "jones"), ("carod", "puig", "rovira"), ("mary", "ma", "lee")) #: the negative controls, measured 2026-10-18 on this grid, which -#: 2444f847 (before #727) disagreed on 72 of: with classify's one-case +#: ad486f66 (before #731) disagreed on 36 of: with classify's one-case #: letter initial taken out, with the numeral fork's one-case letter -#: in front taken out, and with the strict suffix test's one-case veto -#: taken out (#727: the 54 strict comma pairs and the 18 honorific -#: ones, which are 2444f847's 72) -- each guard seen on its own -_ONE_CASE_UNGUARDED = 387 -_ONE_CASE_LETTER_IN_FRONT_UNGUARDED = 117 +#: in front taken out (the fork and the maiden take's candidate test +#: alike, the two sharing the name in `_pieces`), with the strict +#: suffix test's one-case veto taken out (#727: 54 strict comma pairs +#: and 18 honorific ones), and with the maiden take's candidate test +#: alone given no one-case fact (#731: ad486f66's 36) -- each guard +#: seen on its own +_ONE_CASE_UNGUARDED = 423 +_ONE_CASE_LETTER_IN_FRONT_UNGUARDED = 153 _ONE_CASE_STRICT_VETO_UNGUARDED = 72 +_ONE_CASE_MAIDEN_CANDIDATES_UNGUARDED = 36 def _one_case_disagreements() -> list[str]: @@ -3431,6 +3439,19 @@ def test_the_one_case_grid_sees_the_strict_veto( == _ONE_CASE_STRICT_VETO_UNGUARDED) +def test_the_one_case_grid_sees_the_maiden_candidates( + monkeypatch: pytest.MonkeyPatch) -> None: + # the maiden take's candidate test without the one-case fact + # (#731): the shape test alone, as before + from nameparser._pipeline import _group, _pieces + monkeypatch.setattr( + _group, "trailing_candidates", + lambda lo, pieces, ptags, tokens, one_case: + _pieces.trailing_candidates(lo, pieces, ptags, tokens, None)) + assert (len(_one_case_disagreements()) + == _ONE_CASE_MAIDEN_CANDIDATES_UNGUARDED) + + def test_the_one_case_grid_sees_the_letter_in_front( monkeypatch: pytest.MonkeyPatch) -> None: from nameparser._pipeline import _vocab diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 9ce2290d..1d7627ab 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -541,6 +541,7 @@ "john smith mcse" "john smith md" "john smith meng" +"john smith nee x x" "john smith p.h.d." "john smith ph. d." "john smith ph.d." diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 3486a1d6..2ea9be13 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -3412,6 +3412,7 @@ {"name": "john smith mcse", "given": "john", "family": "smith", "suffix": "mcse", "initials": "j. s."} {"name": "john smith md", "given": "john", "family": "smith", "suffix": "md", "initials": "j. s."} {"name": "john smith meng", "given": "john", "family": "smith", "suffix": "meng", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. s."} +{"name": "john smith nee x x", "given": "john", "family": "smith", "maiden": "x x", "initials": "j. s."} {"name": "john smith p.h.d.", "given": "john", "family": "smith", "suffix": "p.h.d.", "initials": "j. s."} {"name": "john smith ph. d.", "given": "john", "family": "smith", "suffix": "ph. d.", "initials": "j. s."} {"name": "john smith ph.d.", "given": "john", "family": "smith", "suffix": "ph.d.", "initials": "j. s."} From ee0962e49eaa59b0dc94de8470416b1d175824c5 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 18 Oct 2026 20:11:44 -0700 Subject: [PATCH 2/3] Ask a clause numeral's candidate test the whole name's case (#731 review) The first version passed the own-words fact, but the word in front of a candidate numeral is always a clause word, which T4 reads by the whole name's case: 'MARY SMITH NEE j x' (mixed case) moved to maiden 'j x' where every release and 'Mary Smith nee j x' read suffix 'x', maiden 'j'. The take now passes the whole name's fact. A new invariant pairs own words in capitals with Title case, clause letters lowercase in both, over fields: 0 now, 72 with the first version's fact (its control); reports still differ on 108 pairs, on master too, since S10's case-settles rule asks the own words there -- recorded, not decided. Docs: trailing_candidates states its one exception to the superset; M2 names S10 among the trailing rules; decisions/M2 and decisions/T4 record it; version claims corrected (2.2.0 through 2.4.2 read the lowercase spelling as the generation, 2.0 and 2.1 as now). Co-Authored-By: Claude Opus 5.5 --- .../design/decisions/M2-maiden-marker-rule.md | 2 +- .../decisions/T4-bare-letter-as-an-initial.md | 6 +- docs/design/rules/M2-maiden-marker-rule.md | 4 +- .../rules/T4-bare-letter-as-an-initial.md | 2 +- nameparser/_pipeline/_group.py | 13 ++++- nameparser/_pipeline/_pieces.py | 9 ++- tests/v2/cases.py | 5 +- tests/v2/test_properties.py | 55 +++++++++++++++++++ 8 files changed, 84 insertions(+), 12 deletions(-) diff --git a/docs/design/decisions/M2-maiden-marker-rule.md b/docs/design/decisions/M2-maiden-marker-rule.md index 42cc12df..717c6657 100644 --- a/docs/design/decisions/M2-maiden-marker-rule.md +++ b/docs/design/decisions/M2-maiden-marker-rule.md @@ -75,4 +75,4 @@ the fullwidth-colon marker (旧姓：佐藤 arrives as one word; the head-peel q - 2026-10-04 (Derek), #601 — A MARKER COUNTS ONLY BEHIND A SURNAME, AND IT TAKES THE WORDS AFTER IT UP TO THE TRAILING RUN THE NAME WOULD END WITH IF THE CLAUSE WERE NOT WRITTEN. This SUPERSEDES the release machinery of the #533 and #535 entries above (the release question, the per-stop checks, the anchored-head boundary of the #544 entry) and the given-part and suffix-comma reach the 2026-07-03 rule had; the #399, #420, #434 and delimiter entries stand. The trigger was #548, `Dr. nee Smith PhD Prof.`, where 2.3.0 read family 'PhD', maiden 'Smith': a clause whose head was nothing but a title, and a release check asking whether each word it gave up could still read as a post-nominal. #548 was closed into this issue and #602 (S2's run) because both halves of its answer were rules rather than repairs. THE HEAD RULE (Derek): a marker is one only in the name before any comma or the surname part before a family comma, behind at least one name word past the leading title run, and not straight behind an unambiguous suffix word or a connective. Anywhere else it is an ordinary word — in the given part after a family comma (`Doe, Jane nee Smith` reads middle 'nee Smith', 1.4.0's reading) and in a part after a suffix comma (`Smith, John, PhD née Jones` reads suffix 'PhD née Jones', also 1.4.0's). Derek's framing was that a maiden marker always follows a family name, and that a marker following something the parser has decided is a title, suffix, particle or connective is not a marker. THE PARTICLE HALF WAS DROPPED BY MEASUREMENT: prototype variant 1 refused a marker behind particle vocabulary and so declined `Anh Do née Tran` and `Mai Le née Nguyen`, Do, Le and Van being particles that are also the commonest Vietnamese surnames; a particle counts as the surname (`Jane van nee Smith` reads family 'van', maiden 'Smith'). THE BOUND (Derek): the clause takes its words up to the run of post-nominals and titles that the trailing rules read at the end of the name with the clause removed, and the words it gives up take the roles that reading gives them — the TRAILING rule's reading of the clause-free name, not the full parse of it, so a head the full parse would read differently (H1's lone title, a P5 join) cannot change where the clause ends. The first word after the marker is always taken, the marker having announced a name — a lone roman numeral included, so `Jane Smith née V` reads maiden 'V' where the clause-free `Jane Smith V` reads suffix 'V', accepted rather than given a numeral exception (rules/M2's Accepted block) — unless it is an unambiguous suffix word, where the marker declines (`Jane Smith nee PhD` reads family 'nee', suffix 'PhD', as before). Before a family comma the clause keeps the 2026-07-03 stop at the first suffix word, since no trailing rule reads that part's end. VARIANTS, prototyped 2026-10-03 behind an environment switch on one tree (the numbers are in #601's comments): v2/v3 grew the clause until the existing release check accepted it and reached 0 invariant violations but inherited the check's caution about particles (`Anh Do geb. de la Vega PhD Prof.` kept 'PhD Prof.' in the maiden name); v4 read the run once over the name as written without binding it, and is the recorded negative control in tests/v2/test_properties.py (350 of the clause grid's 4,482 parses fail under it, measured 2026-10-04); v5/v6 bound that reading as roles but still counted the clause's words among the words to spare, so `John nee Smith ba` read 'ba' as a credential where `John ba` reads a name; v7, the clause-free reading bound as roles, was adopted. Against #535's grids v7 left 0 violations of the M2 invariant where the tree then had 15,586 (grid A, 327,936 texts) and 8,713 (grid B, 173,057), measured 2026-10-03 with an invariant check written to #535's recipe. WHAT THE PROTOTYPE MISSED, found implementing it: the numeral fork. A trailing roman numeral is a suffix only by its shape against the word before it, and v7 read the clause's last word without that test, so `John née Jones Smith VI` kept 'VI' in the maiden name; the take now asks `is_trailing_numeral_suffix` of the last piece, and cases.py's `a_numeral_the_clause_free_name_reads_ends_the_clause` pins it. And the head's title run is measured over the words before the marker, a lone title excepted: measured over the whole segment, `Lord Chancellor née Jones` counted 'Chancellor' as a title and refused the marker. TRIAGE, 2026-10-04: the prototype's 68 failing case rows (40 facade failures among them) were re-pinned as `fix(#601)` with their group named in the note, bar the head-rule bug above; tests/v2/test_parser.py's corpus-append invariant became two-sided (appending ` née X` moves no other field, OR the take records why it declined), with a named set for the suffixes that themselves hold a marker. DEGENERATE INPUT, NO READING DESIGNED FOR (Derek, 2026-10-04: garbage in, garbage out). Two names that read worse by eye moved, and they are recorded here so the move is not mistaken for a regression, not because either reading is wanted: `Berg, Jane van der nee Smith DO` reads middle 'van der nee Smith', family 'Berg' where 2.3.0 read family 'van der Berg', maiden 'Smith DO' — the marker in a given part being a word, the particles no longer trail the given name (P6); and `Jane Doe, PhD née Smith` reads given 'PhD', middle 'née Smith', family 'Jane Doe' where 2.3.0 read given 'Jane', family 'Doe', suffix 'PhD', maiden 'Smith', which 2.3.0 reached only through the clause taking the marker (the comma part opening with a credential is #603's shape). Measured 2026-10-04 against 2.3.0. COST: the take is one function and no release helper survives; the #533 review's comma-head grids could no longer reach a take at all and were replaced by counting heads (tests/v2/test_properties.py's `_maiden_clause_grid`, 4,482 parses). - 2026-10-04 #610 — THE MAIDEN TAKE'S CANDIDATE WORDS ARE ONE FUNCTION BESIDE THE PEEL, AND SHARING IT FIXED A GAP THE HAND COPY HAD. `_maiden_take` chose the clause words to read its clause-free view over with a filter of its own (`_clause_tail_word` plus a numeral clause), a hand copy of the peel's and the chain's admissions whose contract — a superset of what `tail_reading` takes — nothing stated or tested. It is now `_pieces.trailing_candidates`, written beside `peel_trailing`, and tests/v2/pipeline/test_pieces.py pins the superset at the take itself, on the unjoined pieces the take is handed, over a generated grid (37,044 clauses through `group`). The grid found the copy wrong twice. The numeral fork reads a shape-only roman numeral (`VI`, in no wordlist) as the WALK's last piece, and two things leave it last without standing at the end of the name: a period-marked title behind it, which the chain takes first, and the merged `Ph. D.`, which group flags and the walk never holds. The copy admitted the numeral only as the clause's last word, so `Jane Doe nee Smith VI Prof.` kept maiden 'Smith VI' where `John Smith VI Prof.` reads suffix 'VI', and `Jane Doe nee Smith VI Ph. D.` the same way. Both now end the clause at the numeral, as rules/M2's statement already said they should. Recorded negative control: the loop as #601 shipped it misses 432 of that grid's clauses, every one this shape. The test's first version read plain names through all of `group`, on pieces the joins had already merged, which review caught: there the old loop missed 288 names, and a first draft of the fix, covering the title half only, still missed 111 — the `Ph. D.` half, which that grid is what found. Against master 8a8459a8 over the 95,813-text equivalence grid the change moved 10 readings, all of this class. The same issue lifted P6's particle-tail walk into `_pieces.particle_tail`, read off roles, so post_rules' attachment and assign's given-part credential run (#602) ask one walk; that half moved none of the 95,813 readings. - +- 2026-10-18, #731 — THE CANDIDATE TEST IS A SUPERSET WITH ONE STATED EXCEPTION. The #610 entry's contract — `trailing_candidates` a superset of what `tail_reading` takes — now has one exception: a numeral whose word in front reads as an initial is refused, since the clause keeps that word and the view would otherwise read the numeral behind a different one. And the statement names S10 among the trailing rules the take asks, the fork that reads a shape-only numeral being S10's and not S2's. History: decisions/T4's #731 entry. diff --git a/docs/design/decisions/T4-bare-letter-as-an-initial.md b/docs/design/decisions/T4-bare-letter-as-an-initial.md index 6b3c939f..dac404a2 100644 --- a/docs/design/decisions/T4-bare-letter-as-an-initial.md +++ b/docs/design/decisions/T4-bare-letter-as-an-initial.md @@ -9,5 +9,7 @@ - 2026-10-18, #727 — THE STRICT SUFFIX TEST VETOES A ONE-CASE LETTER AS IT VETOES ITS CAPITAL. The gap the first entry recorded: `is_suffix_strict` vetoes an initial by the word's text (`is_initial`), not by classify's tag, so a one-case lowercase `v` or `i` — an initial under this rule — passed where its capital was vetoed. Two callers reached it. Under `Policy(lenient_comma_suffixes=False)`, C1's comma tests (`is_wholly_suffix` and the name-word count): `john smith, v` read suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reported nothing where its capitals report `comma-structure`. And under the default policy, W2's scan-back (`_script_segment._is_post_nominal`): `田中さん v` peeled さん and read suffix 'さん v' where `田中さん V` reads given '田中さん', family 'V' — a CJK name with a Latin letter beside it is one case under the own-words test, the letter being the only cased word in it, so nothing in the writing contrasts it. The strict test now takes the one-case fact as an argument and vetoes a bare lowercase letter whose capital is A to Z where it is true; segment, `_comma` and script_segment pass the fact they already hold, and segment asks for it only under the strict knob. A separate keyword from `is_wholly_suffix`'s `one_case`, which is the credential LEAN and is None by design at callers that want no lean; they still want the letters read by this rule. The lenient default takes a listed suffix word whatever its shape and never reads the argument. Measured: the committed invariant's grid gains the three comma shapes and `田中さん {n}`, and runs every comma-holding shape again under the strict knob (the knob reaches nothing else); 2444f847 disagreed on 72 of its 4,554 pairs (54 strict comma pairs, 18 honorific), this tree on none, and the third recorded control is that 72. `田中さん v` now reads as 1.4.0 read it; 2.1 through 2.4.2 peeled it (suffix 'さん, v' in 2.1 and 2.2, 'さん v' from 2.3). No corpus name moved. Open: none from this entry; #727 closes the gap the first entry left open. -- 2026-10-18, #731 — THE MAIDEN TAKE'S CANDIDATE TEST ASKS THE ONE-CASE FACT. M2's take (#601, #610) builds a clause-free view of the name from the clause's trailing candidates and lets `tail_reading` decide what the run takes. Its candidate test (`_pieces.trailing_candidates`) asked the numeral fork's shape test with no one-case fact, as a superset of what the fork admits. But a candidate is also cut out of the view, which changes the word in front of the numeral there: in `john smith nee x x` the lowercase first `x` is not initial-shaped, so the second became a candidate, and the view read it behind `smith` with words to spare, as the generation (suffix 'x', maiden 'x', reported). `JOHN SMITH NEE X X` keeps maiden 'X X', the capital first `X` refusing the candidate. Now the test asks with the name's one-case fact, as the fork does, so the lowercase letter in front refuses it too; `john smith nee j x` moves the same way. The superset check (`test_the_trailing_candidates_cover_what_the_tail_reading_takes`) still passes: its oracle reads the view with every clause word after the first, where the numeral keeps its real neighbour. - Measured: the one-case grid gains `{a} {b} nee {n} {n}` and `{a} {b} nee j {n}`. ad486f66 disagreed on 36 of its 4,752 pairs, all in those shapes; this tree on none, and the fourth recorded control is that 36. 2.3.0 through 2.4.2 gave the lowercase reading above, so the release bullet for #723 carries this too. No corpus name moved. +- 2026-10-18, #731 — THE MAIDEN TAKE'S CANDIDATE TEST ASKS THE ONE-CASE FACT. M2's take (#601, #610) builds a clause-free view of the name from the clause's trailing candidates and lets `tail_reading` decide what the run takes. Its candidate test (`_pieces.trailing_candidates`) asked the numeral fork's shape test with no one-case fact, as a superset of what the fork admits. But a candidate is also cut out of the view, which changes the word in front of the numeral there: in `john smith nee x x` the lowercase first `x` is not initial-shaped, so the second became a candidate, and the view read it behind `smith` with words to spare, as the generation (suffix 'x', maiden 'x', reported). `JOHN SMITH NEE X X` keeps maiden 'X X', the capital first `X` refusing the candidate. Now the test asks with a one-case fact, as the fork does, so the lowercase letter in front refuses it too; `john smith nee j x` moves the same way. The fact is the WHOLE name's, since the word in front of a candidate is always a clause word, which this rule reads by the whole name's case: the first version passed the own-words fact, and the design-docs review found `MARY SMITH NEE j x` (mixed-case writing) moving to maiden 'j x' where every release and `Mary Smith nee j x` read suffix 'x', maiden 'j' — 72 field moves over a grid pairing own words in capitals with Title case, the clause letters lowercase in both, now `test_a_clause_letter_beside_capitals_reads_as_beside_title_case` with that 72 as its control. + `trailing_candidates` is no longer a strict superset of what `tail_reading` takes from a view missing the clause's first word, and says so: a numeral whose word in front reads as an initial is refused even where a view without that word would read it, which is the point. The superset check (`test_the_trailing_candidates_cover_what_the_tail_reading_takes`) still passes only because its clauses all open with `Smith`; a clause opening with the letter shows the refusal as a "miss" (the review measured 8 of 48 such texts, the capital ones on ad486f66 too). + A GAP LEFT OPEN: on the same capitals-against-Title-case grid, 108 pairs still differ in their REPORT, on ad486f66 as on this tree — S10's "a numeral written in lower case in a name of mixed case is the generation and reports nothing" is asked of the own words' case, so `JOHN SMITH nee x x` reports `suffix-or-name` where `John Smith nee x x` does not. The view the take reads also asks S6's credential lean, which P3 says is asked of the own words, so moving the fact there is a design question rather than this fix's. + Measured: the one-case grid gains `{a} {b} nee {n} {n}` and `{a} {b} nee j {n}`. ad486f66 disagreed on 36 of its 4,752 pairs, all in those shapes; this tree on none, and the fourth recorded control is that 36. 2.2.0 through 2.4.2 gave the lowercase reading above, and 2.0.0 and 2.1.0 the reading this restores (maiden 'x x'), so the release bullet for #723 carries this too. No corpus name moved. diff --git a/docs/design/rules/M2-maiden-marker-rule.md b/docs/design/rules/M2-maiden-marker-rule.md index 32e1d6ac..d62bdb4d 100644 --- a/docs/design/rules/M2-maiden-marker-rule.md +++ b/docs/design/rules/M2-maiden-marker-rule.md @@ -18,7 +18,7 @@ M2. Rationale: a maiden marker announces that what follows it is the A marker that counts takes the words after it as the maiden name, and is itself dropped. It takes them up to the trailing run of post-nominals and titles that the end of the name reads as if the - clause were not written — the trailing rules (S2, H5) asked over + clause were not written — the trailing rules (S2, S10, H5) asked over the words before the marker and the post-nominal or title words that end it, before any join — and always takes the first word after it, the marker having announced a name. A suffix word @@ -143,5 +143,5 @@ M2. Rationale: a maiden marker announces that what follows it is the Accepted: a marker behind a credential is an ordinary word, and the credential's run (S8) takes it in with everything after it. "Jane Doe Jr. nee Smith" → suffix="Jr. nee Smith" · boundary - history: decisions/M2 · interacts: P2, P3, P5, P6, R1, R2, M1, S1, S2, H1, H5, C1, S4, S8, T4 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py + history: decisions/M2 · interacts: P2, P3, P5, P6, R1, R2, M1, S1, S2, S10, H1, H5, C1, S4, S8, T4 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py diff --git a/docs/design/rules/T4-bare-letter-as-an-initial.md b/docs/design/rules/T4-bare-letter-as-an-initial.md index e86e7681..babe41c0 100644 --- a/docs/design/rules/T4-bare-letter-as-an-initial.md +++ b/docs/design/rules/T4-bare-letter-as-an-initial.md @@ -46,4 +46,4 @@ T4. Rationale: one letter standing for a name word is how an "DR. NÉE JUAN y LOPEZ" → family="JUAN y LOPEZ" · boundary "jose e lopez" → ambiguities=("conjunction-or-initial",) "jose y lopez" → middle="y" - history: decisions/T4 · interacts: M2, M4, O5, P2, P3, P7, S10, R3, R4, C1, W2 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_vocab.py + history: decisions/T4 · interacts: M2, M4, O5, P2, P3, P7, S10, R3, R4, C1, W2 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_vocab.py diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index 667713e1..28301f7a 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -81,7 +81,8 @@ import nameparser._pipeline._comma as _comma from nameparser._pipeline._vocab import D, PH from nameparser._pipeline._vocab import ( - comma_follows_marker, delimiter_cores, reads_as_a_reported_numeral, + comma_follows_marker, delimiter_cores, is_one_case, + reads_as_a_reported_numeral, ) from nameparser._types import AmbiguityKind, Role @@ -303,7 +304,15 @@ def _maiden_take(pieces: Sequence[Sequence[int]], for j in range(lo, end)] if site is not ClauseSite.TRAILING: assert_never(site) - c = trailing_candidates(lo, pieces, ptags, tokens, one_case) + # rules/T4: a letter outside the name's own words "asks the case + # of the whole name, the clause included" -- and the word in front + # of a candidate numeral is always a clause word, so the candidate + # test takes the whole name's fact, not the own words' (#731's + # review: 'MARY SMITH NEE j x' is mixed case, and its 'j' no + # initial). Asked only where the own words are one case. + c = trailing_candidates( + lo, pieces, ptags, tokens, + one_case and is_one_case([t.text for t in tokens])) end = len(pieces) released: list[tuple[int, Role]] = [] # the view's picks, for the kept member below (#718) diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index 4c061706..41d158c5 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -1512,6 +1512,9 @@ def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], into the trailing run: `pieces[c:]`, a SUPERSET of what it takes, for the maiden take to build its clause-free view over and let `tail_reading` decide. `pieces[lo]` itself is never a candidate. + One exception, on purpose: a numeral whose word in front reads as + an initial is refused even where a view without that word would + read it, since the clause keeps the word (#731, below). Back from the end over every piece the peel or the chain admits on its own terms -- a suffix piece, a member of the ambiguous class, @@ -1521,8 +1524,10 @@ def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], but such title words and group-flagged suffix pieces: the chain takes the titles first and the walk never holds the flagged pieces ('Ph. D.'), so either way the numeral is the walk's last piece and - the fork reads it. The shape test is asked with the name's - one-case fact, as the fork asks it: a candidate is not only + the fork reads it. The shape test is asked with a one-case fact, + as the fork asks it -- the WHOLE name's, which the caller passes, + since the word in front is a clause word (rules/T4): a candidate + is not only admitted but cut out of the view, so the view puts a different word in front of it, and a numeral the fork would refuse for the letter in front is a numeral the view reads. A bare lowercase diff --git a/tests/v2/cases.py b/tests/v2/cases.py index c1faa9cc..6f477a01 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -7693,8 +7693,9 @@ def _check_cjk_shape_purity(self) -> None: notes="read as 'JOHN SMITH NEE X X' is (T4): the first 'x' is " "the initial its capital is, so the take's view does " "not cut the second out of the clause and read it as " - "the generation; 2.3 through 2.4.2 read suffix 'x', " - "maiden 'x' (#731). 1.4.0 had no maiden markers"), + "the generation; 2.2 through 2.4.2 read suffix 'x', " + "maiden 'x', and 2.0 and 2.1 read as here (#731). 1.4.0 " + "had no maiden markers"), Case("a_one_case_numeral_behind_a_letter_ending_a_clause", "john smith nee j x", {"given": "john", "family": "smith", "maiden": "j x"}, diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index cec802a0..9a884444 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -3439,6 +3439,61 @@ def test_the_one_case_grid_sees_the_strict_veto( == _ONE_CASE_STRICT_VETO_UNGUARDED) +# --- #731's review: a clause's letter asks the whole name's case -- +# T4 reads a letter outside the name's own words by the case of the +# WHOLE name, so a lower-case letter in a maiden clause behind own +# words in capitals is mixed-case writing and reads as it does behind +# the same words in Title case. The one-case grid above cannot see +# this (both spellings of each of its pairs are wholly one case); this +# pairs capitals against Title case, the clause letters lower-case in +# both: 396 pairs, about 800 parses, 0.07s on py3.11 (2026-10-18). +# FIELDS only: the report on a numeral the take reads as a suffix +# still differs on 108 of these pairs, on ad486f66 as here -- S10's +# "a numeral written in lower case in a name of mixed case is the +# generation and reports nothing" is asked of the own words' case +# there, so behind capitals the lower-case numeral reports. Recorded +# in decisions/T4's #731 entry, not decided here. +_CLAUSE_CASE_SHAPES = ("{a} {b} nee {n} {n}", "{a} {b} nee j {n}", + "{a} {b} nee {c} j {n}", "{a} {b} nee {c} {n} {n}") +#: the negative control, measured 2026-10-18: the maiden take's +#: candidate test given the own-words fact instead (#731's first +#: version), which disagreed on that many +_CLAUSE_CASE_OWN_WORDS = 72 + + +def _clause_case_disagreements() -> list[str]: + def sig(name: ParsedName) -> dict[str, str]: + return {k: v.lower() for k, v in name.as_dict().items() if v} + out = [] + for order in (GIVEN_FIRST, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST): + parser = Parser(policy=Policy(name_order=order)) + for shape, letter, (a, b, c) in itertools.product( + _CLAUSE_CASE_SHAPES, _ONE_CASE_LETTERS, _ONE_CASE_HEADS): + caps = shape.format(a=a.upper(), b=b.upper(), c=c.upper(), + n=letter) + title = shape.format(a=a.title(), b=b.title(), c=c.title(), + n=letter) + if sig(parser.parse(caps)) != sig(parser.parse(title)): + out.append(f"[{order}] {caps!r}") + return out + + +def test_a_clause_letter_beside_capitals_reads_as_beside_title_case( + ) -> None: + failures = _clause_case_disagreements() + assert not failures, "\n".join(failures[:10]) + + +def test_the_clause_case_grid_sees_the_whole_name_fact( + monkeypatch: pytest.MonkeyPatch) -> None: + # the take asks `one_case and is_one_case()`; with the + # second half always true it is the own-words fact alone, which is + # what #731's first version passed + monkeypatch.setattr("nameparser._pipeline._group.is_one_case", + lambda texts: True) + assert len(_clause_case_disagreements()) == _CLAUSE_CASE_OWN_WORDS + + def test_the_one_case_grid_sees_the_maiden_candidates( monkeypatch: pytest.MonkeyPatch) -> None: # the maiden take's candidate test without the one-case fact From 5f989e224edd5a6b68460cd1594971982b75ba4a Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 18 Oct 2026 20:26:37 -0700 Subject: [PATCH 3/3] State the clause-case gap's reach and why the view keeps its fact (#731 review) The code review found the gap recorded as report-only also moves fields with one own name word in front ('SMITH NEE j x' against 'Smith Nee j x'), on master too. Passing the whole name's fact to the take's view closes it but moves S6's credential lean, which P3 asks of the own words: test_a_maiden_clause_does_not_change_how_a_trailing_word_reads fails on 270 of 20,412 pairs. Declined with that evidence; the test comment and decisions/T4 now state the grid's two-own-word reach and the open gap, and two docstrings no longer say the candidate test asks "as the fork does". Co-Authored-By: Claude Opus 5.5 --- .../decisions/T4-bare-letter-as-an-initial.md | 3 ++- nameparser/_pipeline/_pieces.py | 8 ++++---- nameparser/_pipeline/_vocab.py | 7 ++++--- tests/v2/test_properties.py | 14 ++++++++------ 4 files changed, 18 insertions(+), 14 deletions(-) diff --git a/docs/design/decisions/T4-bare-letter-as-an-initial.md b/docs/design/decisions/T4-bare-letter-as-an-initial.md index dac404a2..f7432bb8 100644 --- a/docs/design/decisions/T4-bare-letter-as-an-initial.md +++ b/docs/design/decisions/T4-bare-letter-as-an-initial.md @@ -11,5 +11,6 @@ Open: none from this entry; #727 closes the gap the first entry left open. - 2026-10-18, #731 — THE MAIDEN TAKE'S CANDIDATE TEST ASKS THE ONE-CASE FACT. M2's take (#601, #610) builds a clause-free view of the name from the clause's trailing candidates and lets `tail_reading` decide what the run takes. Its candidate test (`_pieces.trailing_candidates`) asked the numeral fork's shape test with no one-case fact, as a superset of what the fork admits. But a candidate is also cut out of the view, which changes the word in front of the numeral there: in `john smith nee x x` the lowercase first `x` is not initial-shaped, so the second became a candidate, and the view read it behind `smith` with words to spare, as the generation (suffix 'x', maiden 'x', reported). `JOHN SMITH NEE X X` keeps maiden 'X X', the capital first `X` refusing the candidate. Now the test asks with a one-case fact, as the fork does, so the lowercase letter in front refuses it too; `john smith nee j x` moves the same way. The fact is the WHOLE name's, since the word in front of a candidate is always a clause word, which this rule reads by the whole name's case: the first version passed the own-words fact, and the design-docs review found `MARY SMITH NEE j x` (mixed-case writing) moving to maiden 'j x' where every release and `Mary Smith nee j x` read suffix 'x', maiden 'j' — 72 field moves over a grid pairing own words in capitals with Title case, the clause letters lowercase in both, now `test_a_clause_letter_beside_capitals_reads_as_beside_title_case` with that 72 as its control. `trailing_candidates` is no longer a strict superset of what `tail_reading` takes from a view missing the clause's first word, and says so: a numeral whose word in front reads as an initial is refused even where a view without that word would read it, which is the point. The superset check (`test_the_trailing_candidates_cover_what_the_tail_reading_takes`) still passes only because its clauses all open with `Smith`; a clause opening with the letter shows the refusal as a "miss" (the review measured 8 of 48 such texts, the capital ones on ad486f66 too). - A GAP LEFT OPEN: on the same capitals-against-Title-case grid, 108 pairs still differ in their REPORT, on ad486f66 as on this tree — S10's "a numeral written in lower case in a name of mixed case is the generation and reports nothing" is asked of the own words' case, so `JOHN SMITH nee x x` reports `suffix-or-name` where `John Smith nee x x` does not. The view the take reads also asks S6's credential lean, which P3 says is asked of the own words, so moving the fact there is a design question rather than this fix's. + A GAP LEFT OPEN, on ad486f66 as on this tree: the view the take reads asks the own words' case, so a lower-case clause numeral behind own words in capitals is counted as a capital would be, where behind Title case it is S10's "lower case in a name of mixed case", the generation and unreported. With two own name words the fields agree and only the report differs (108 pairs of the committed grid: `JOHN SMITH nee x x` reports `suffix-or-name`, `John Smith nee x x` does not); with ONE own name word there is no word to spare and the fields differ too (`SMITH NEE j x` maiden 'j x', `Smith Nee j x` suffix 'x', maiden 'j'; the code review's finding). The committed grid therefore holds two own words in front only, and says so. + Declined, measured: passing the whole name's fact to the view as well. It closes both (0 field and 0 report differences on a grid with one-word, two-word and titled heads) but the view also asks S6's credential lean, which P3 says is asked of the own words, and moving it there fails `test_a_maiden_clause_does_not_change_how_a_trailing_word_reads` (tests/v2/test_properties.py) on 270 of its 20,412 pairs (with it, `J. nee Smith BA` would read the credential where `J. BA` reads a name). Closing the gap needs the peel to take the numeral's case fact apart from the lean's — a design change beyond this fix. Measured: the one-case grid gains `{a} {b} nee {n} {n}` and `{a} {b} nee j {n}`. ad486f66 disagreed on 36 of its 4,752 pairs, all in those shapes; this tree on none, and the fourth recorded control is that 36. 2.2.0 through 2.4.2 gave the lowercase reading above, and 2.0.0 and 2.1.0 the reading this restores (maiden 'x x'), so the release bullet for #723 carries this too. No corpus name moved. diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index 41d158c5..2e71cccc 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -1524,10 +1524,10 @@ def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], but such title words and group-flagged suffix pieces: the chain takes the titles first and the walk never holds the flagged pieces ('Ph. D.'), so either way the numeral is the walk's last piece and - the fork reads it. The shape test is asked with a one-case fact, - as the fork asks it -- the WHOLE name's, which the caller passes, - since the word in front is a clause word (rules/T4): a candidate - is not only + the fork reads it. The shape test is asked with a one-case fact + -- the WHOLE name's, which the caller passes, since the word in + front is a clause word (rules/T4), where the view's reading keeps + the own words' fact: a candidate is not only admitted but cut out of the view, so the view puts a different word in front of it, and a numeral the fork would refuse for the letter in front is a numeral the view reads. A bare lowercase diff --git a/nameparser/_pipeline/_vocab.py b/nameparser/_pipeline/_vocab.py index f0c202d8..39e80ecc 100644 --- a/nameparser/_pipeline/_vocab.py +++ b/nameparser/_pipeline/_vocab.py @@ -278,10 +278,11 @@ def is_trailing_numeral_suffix(text: str, preceding: str, is initial-shaped as 'J' is (rules/S10, #721's review: 'john j v' reads as 'JOHN J V'), while 'é', 'ж' or an uncased word, whose capitals are not, stay as they were. None is the shape test - alone. The maiden take's candidate view once passed it as a safe + alone. The maiden take's candidate test once passed it as a safe superset, but admitting a candidate also cuts it out of the view, - changing the word in front of the numeral, so it asks with the - fact as the fork does (#731).""" + changing the word in front of the numeral, so it passes a fact + now: the whole name's, the word in front being a clause word + (#731), where the view's own reading keeps the own words'.""" return (_ROMAN.match(text) is not None and not is_initial_shaped(preceding) and not (one_case and is_initial_shaped(preceding.upper()))) diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index 9a884444..4de6a6c0 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -3447,12 +3447,14 @@ def test_the_one_case_grid_sees_the_strict_veto( # this (both spellings of each of its pairs are wholly one case); this # pairs capitals against Title case, the clause letters lower-case in # both: 396 pairs, about 800 parses, 0.07s on py3.11 (2026-10-18). -# FIELDS only: the report on a numeral the take reads as a suffix -# still differs on 108 of these pairs, on ad486f66 as here -- S10's -# "a numeral written in lower case in a name of mixed case is the -# generation and reports nothing" is asked of the own words' case -# there, so behind capitals the lower-case numeral reports. Recorded -# in decisions/T4's #731 entry, not decided here. +# FIELDS only, and TWO own name words in front only, which is this +# grid's whole claim: the take's view reads the numeral by the own +# words' case, so behind capitals a lower-case clause numeral is +# counted like a capital -- the report differs on 108 of these pairs +# (on ad486f66 as here), and with ONE own name word, no word to +# spare, the fields differ too ('SMITH NEE j x' maiden 'j x', 'Smith +# Nee j x' suffix 'x'). Recorded in decisions/T4's #731 entry with +# why the view keeps the own words' fact. _CLAUSE_CASE_SHAPES = ("{a} {b} nee {n} {n}", "{a} {b} nee j {n}", "{a} {b} nee {c} j {n}", "{a} {b} nee {c} {n} {n}") #: the negative control, measured 2026-10-18: the maiden take's