diff --git a/docs/design/decisions/H1-title-and-one-name-word.md b/docs/design/decisions/H1-title-and-one-name-word.md index e32a5457..2815c1df 100644 --- a/docs/design/decisions/H1-title-and-one-name-word.md +++ b/docs/design/decisions/H1-title-and-one-name-word.md @@ -1,6 +1,6 @@ # H1 — a title and one name word -- 2026-08-25 (#410) — H1's "and nothing else" counted a suffix, a nickname and a maiden name as further name words, so a title-plus-surname name reported no family the moment any of them stood beside it: `Dr. Smith` reads family 'Smith' and `Dr. Smith née Jones` read given 'Smith' with the family empty. #410 reports the maiden flavor; measured, all three roles suppress the rule identically and the fix is one term in one guard, so the term went rather than the maiden role being special-cased. What decided the width is H1's own rationale — a title addresses by surname — which says nothing about what stands BESIDE the name. The rationale is stated for H1 rather than as a doc-wide principle about what a suffix is, deliberately: N3 counts a suffix the other way (`'Smitty' Jones Jr.` reads given 'Jones', family ''), and which of the two readings is right for a nickname-led name is not decided here. #399 is what made this urgent rather than latent: stopping the particle chain at the marker routed the canonical title-and-particle shape (`Freiherr von Richthofen geb. Albrecht`) into H1 for the first time. N3 moves with it, without N3 changing: `'Smitty' Dr. Jones` declines N3's one-piece count as it always did, and H1 now names the family behind it, so it reads family 'Jones' where it read given 'Jones' through 2.1. Five corpus names change reading at every baseline, of which four arrive as new diffs and are classified with the fix. The fifth, `Freiherr von Richthofen V`, was already classified under the `fix(#424)` rule: #410 narrows its diff from {given, family, suffix} to {family, suffix}, and `classify()` takes the first rule whose declared `fields` are a SUPERSET of the observed diff, so a shrinking diff kept matching and no run ever named it. A real movement behind a green gate, and the second of that kind here: the fields-only `fix(suffix-routing)` catch-all was absorbing three names at 1.4.0 the same way, its heading growing with nothing to announce it. Both are one shape — a rule broader than the diff it explains — and that is the lesson worth keeping rather than either fix. It is also, being a title, one name word and a suffix, the suffix flavor of this very shape, which two ledger comments had claimed no corpus carries. At 1.4.0 three of the four were being absorbed by the fields-only `fix(suffix-routing)` rule (its heading went 14 -> 17) — invisible to `_CORPUS_CLAIMS`, which records the whole corpus for a rule with no `name_regex`, so the guard could not see them arrive. They have their own rule now, as #372's names got one off the same catch-all. The v1 suite shipped the nickname reading as a strict xfail (`tests/test_nicknames.py::test_nickname_and_last_name_with_title`), which passes now. Known gap between the statement and the guard: H1 says "exactly one name word" and "that word", but the guard never counts units. It tests which ROLES are unoccupied — no middle, no family — and then retags every given token, so a name whose given run is two name WORDS (units, not tokens) reaches it and moves both: `Freiherr de V Jr` reads family 'de V'. The distinction matters for reading the claim that follows: a multi-token given run is ordinary — the rules.md example `Freiherr von Richthofen V` has one, its two tokens being a single particle unit — and it is a run of two UNITS that no real input produces. A 58,338-name sweep found no corpus name and no rules.md example with one — but it is recorded because decisions.md elsewhere leans on H1's "exactly one name word" as a settled scope when narrowing P5's licence, and that reading of H1 is stronger than the code behind it. Not fixed here: the widening this entry records is about which roles disqualify the rule, not about how it counts. +- 2026-08-25 (#410) — H1's "and nothing else" counted a suffix, a nickname and a maiden name as further name words, so a title-plus-surname name reported no family the moment any of them stood beside it: `Dr. Smith` reads family 'Smith' and `Dr. Smith née Jones` read given 'Smith' with the family empty. #410 reports the maiden flavor; measured, all three roles suppress the rule identically and the fix is one term in one guard, so the term went rather than the maiden role being special-cased. What decided the width is H1's own rationale — a title addresses by surname — which says nothing about what stands BESIDE the name. The rationale is stated for H1 rather than as a doc-wide principle about what a suffix is, deliberately: N3 counts a suffix the other way (`'Smitty' Jones Jr.` reads given 'Jones', family ''), and which of the two readings is right for a nickname-led name is not decided here. #399 is what made this urgent rather than latent: stopping the particle chain at the marker routed the canonical title-and-particle shape (`Freiherr von Richthofen geb. Albrecht`) into H1 for the first time. N3 moves with it, without N3 changing: `'Smitty' Dr. Jones` declines N3's one-piece count as it always did, and H1 now names the family behind it, so it reads family 'Jones' where it read given 'Jones' through 2.1. Five corpus names change reading at every baseline (PR #444); one of them, `Freiherr von Richthofen V`, only narrowed a diff the `fix(#424)` ledger rule already classified, so no run named it. The v1 suite shipped the nickname reading as a strict xfail (`tests/test_nicknames.py::test_nickname_and_last_name_with_title`), which passes now. Known gap between the statement and the guard: H1 says "exactly one name word" and "that word", but the guard never counts units. It tests which ROLES are unoccupied — no middle, no family — and then retags every given token, so a name whose given run is two name WORDS (units, not tokens) reaches it and moves both: `Freiherr de V Jr` reads family 'de V'. The distinction matters for reading the claim that follows: a multi-token given run is ordinary — the rules.md example `Freiherr von Richthofen V` has one, its two tokens being a single particle unit — and it is a run of two UNITS that no real input produces. A 58,338-name sweep found no corpus name and no rules.md example with one — but it is recorded because decisions.md elsewhere leans on H1's "exactly one name word" as a settled scope when narrowing P5's licence, and that reading of H1 is stronger than the code behind it. Not fixed here: the widening this entry records is about which roles disqualify the rule, not about how it counts. - 2026-09-08 [#489](https://github.com/derek73/python-nameparser/issues/489) (the #316/#489 bundle) — a title RUN addresses as its LAST title does. `Her Majesty Queen Elizabeth` read family `Elizabeth` because the `given_name_titles` lookup was keyed on the whole run, and `her majesty queen` is no shipped entry. A run is written as several titles and is addressed the way its final one is, so the run's folded key is now matched whole OR by its last word. **This AMENDS the 2026-08-22 #369 entry under #P5**, which keyed both sites on the whole run so they could never disagree; the amendment keys both on the LAST word and the invariant is untouched — H1's retag and P5's licence still cannot read one run two ways, through one predicate beside `_title_key`. What the amendment reopens is that entry's EXAMPLE and not its reasoning: `'mr sir'` was #369's own instance of a run neither site treats as a given-name title, and it now IS one to both, `sir` being the last word. The whole-run arm is kept for a caller's multi-word phrase entry — `lt col` is stored and matched as one key — and its order carries nothing: the `or` short-circuits but decides nothing, since for a ONE-WORD run the key IS its last word. Every shipped given-name title is a single word, asserted in tests/v2/test_lexicon.py rather than counted here, so under the default vocabulary that arm can only ever match a one-word run, which the last-word arm reads the same way. Recompute the single-word property with `L = Parser().lexicon; sorted(t for t in L.given_name_titles if " " in t)`, which gives `[]` on 2026-09-08; the set holds 47 entries that day, and the argument survives the digit moving. @@ -12,7 +12,7 @@ - 2026-09-09 (the #316/#489 bundle, review round 2) — WHICH run addresses, where a name carries a title run at BOTH ends. The TRAILING REACH paragraph above is right about the trailing slot and was incomplete about the front one: H1 collected every `Role.TITLE` token into ONE run, and with #316(a) shipped that run can span the whole name. `Sir John Prof.` keyed `sir prof`, whose last word `prof` is no given-name title, so it read title `Sir Prof.`, family `John` where `Sir John` reads given `John` — the title written behind the name changing what the title in front of it addressed by. `Queen Elizabeth Prof.` lost its given name the same way, and `Dr. Smith Sir.` moved in the other direction, the trailing `sir` making the leading `dr` address by given name and turning family `Smith` into given `Smith`. THE RULE: the run standing BEFORE the one name word is the run that addresses; a run standing behind it decides only when none stands before. The split is at the NAME WORD, and the first fix split at the first token of ANOTHER ROLE instead, which a nickname written in front of the titles satisfies: `'Smitty' Sir Jones Prof.` then found no leading run and read family `Jones` where `'Smitty' Sir Jones` reads given, and `Dr. 'Smitty' Sir John` lost the run it had — the same defect one nickname away, caught by the design review's two-input invariant. H1's own rationale is what decides it, saying in as many words that what stands beside the name word does not decide this reading; so a run written around a nickname is ONE run and `Dr. 'Smitty' Sir John` reads given `John` as `Dr. Sir John` does. That is #H5's transparency principle applied to the one-word name — `X Prof.` is `X` plus a title, so adding the title cannot change how the words in front of it are read — and it leaves the trailing slot's own reading untouched, `Smith Sir.` still reading given `Smith` because no run stands in front of it. rules/H1's statement carries the ordering and gains two examples, `"Sir John Prof." → given="John"` and `"Dr. Smith Sir." → family="Smith"`; #H5's one-name-word clause is conditioned the same way. FOUND BY THE PR REVIEW'S P5 FINDING, and that is the part worth keeping. The reserve fix under #P5 was correct on its own terms and left `Sir abdul rahman Prof.` reading family `abdul rahman`: the join fired for the right reason and the field was still wrong. What the review then asked was which run each site reads, and the answer was that they read DIFFERENT runs — group's licence has always read only the pieces ahead of the bound word (`range(fk)`), while H1 read every title token — so the 2026-08-22 #369 invariant under #P5, that H1 and P5 cannot read one run two ways, had been broken by the composite keying without any site's own tests noticing. Both now read the leading run, through the same predicate, and `nameparser/_pipeline/_group.py` records that beside the licence. - MEASURED 2026-09-09, over every distinct name in `tools/differential/corpus*.jsonl`, parsed on the fix's parent (`e489dc1`) and on the fix and diffed across the seven fields and the ambiguity kinds: over the corpus AS THE PARENT HAD IT, not one name moves; over the corpus as this commit leaves it, exactly TWO do and they are the two rules.md examples this entry adds (`Sir John Prof.`, `Dr. Smith Sir.`). The composite shape needs a title run at BOTH ends and no corpus name had one before those two, which is why a defect this plain sat behind four green gates. `Dr. King Sir.` is untouched and stays #H4's: its leading run is `dr king`, whose last word `king` IS a given-name title, so the word the peel left standing reads given `Sir.` with `title-or-name` reported. The end-to-end invariant is `tests/v2/test_parser.py::test_the_p5_licence_and_h1_read_a_title_run_the_same_way`, whose trailing-title half now requires `f"{title} abdul rahman Prof."` and `f"{title} John Prof."` to give the same given and family as the spellings without the trailing title; restoring the whole-`titles` key fails six of its eleven rows, which is the mutation that proves it discriminates. It carries the nickname spellings as well, and those are the only rows that separate the two splits above. Frame delta zero on both entry points: the run is built inside H1's guard, after the role counts have short-circuited, and the reference name `Dr. Juan0000 de la Vega III` has a family and never enters it. + MEASURED 2026-09-09 against the fix's parent `e489dc1` over every corpus name (PR #520): only the two rules.md examples this entry adds move (`Sir John Prof.`, `Dr. Smith Sir.`), no corpus name having had a title run at both ends. `Dr. King Sir.` is untouched and stays #H4's: its leading run is `dr king`, whose last word `king` IS a given-name title, so the word the peel left standing reads given `Sir.` with `title-or-name` reported. The end-to-end invariant is `tests/v2/test_parser.py::test_the_p5_licence_and_h1_read_a_title_run_the_same_way`, whose nickname rows are the only ones that separate the two splits above; frame delta zero on both entry points. - 2026-09-10 [#519](https://github.com/derek73/python-nameparser/issues/519) — `prince` and `princess` join the given-name titles; `lord` and `lady` do not. This RESOLVES the VOCABULARY SCOPE paragraph of the 2026-09-08 #489 bullet above, which is left as written: it named the four as not given-name titles and filed the question, and two of the four have now moved. The criterion is rules.md H Background's — membership follows how the title addresses — and `prince` and `princess` address by the given name (`Prince Harry`, `Princess Anne`), with no surname reading of the word behind it. The clause is for the two words and not for a royal class: TITLES holds `emperor`, `tsar`, `sultan`, `pharaoh`, `archduke`, `maharani` and their pairs, none of them weighed here, and `Emperor Akihito` still reads family `Akihito`. `lord` and `lady` split the same way as each other, by the bearer's rank, which the text does not carry: the given-name form is a courtesy style for children of the senior ranks alone — younger sons of dukes and marquesses (`Lord Peter`), daughters of dukes, marquesses and earls (`Lady Diana`) — and every peer and every wife takes the title or surname (`Lord Byron`, `Lady Thatcher`). [Debrett's "The trouble with titles"](https://debretts.com/the-trouble-with-titles/) and [Bratman's peerage primer](https://www.dbratman.net/nobility.html), both read 2026-09-10, agree on that boundary, Debrett's naming `Lady Jane Debrett` for a baronet's wife as the typical error. The issue gave `lord` the surname reading outright, hedged with a "mostly"; the hedge is this class. The set has no way to say "sometimes" — the `venerable` reasoning under #indic-honorifics — so Derek's call was to leave both out, decided rather than deferred, and the issue's proposed frequency test on `lady` was not run. Both are pinned as negative controls (`Lord Byron`, `Lady Gaga`) beside `His Excellency Lord Duncan`. Both read sites move, through the one predicate: H1's fold reads `Prince Harry` given `Harry` and `Her Royal Highness Princess Anne` given `Anne` (the #489 bundle's negative control, flipped), and P5's licence reads `Prince abdul Rahman` given `abdul Rahman` with an empty family, as `Sir abdul Rahman` does. What membership does NOT decide is whether the word is a title at all: `Prince Fielder` reads given `Fielder` now and read family `Fielder` before, wrong both ways, and that is [#348](https://github.com/derek73/python-nameparser/issues/348)'s collision. @@ -21,5 +21,5 @@ - 2026-10-18 [#720](https://github.com/derek73/python-nameparser/issues/720) — H1 decides the field of the one name word before a comma with no given name after it, and of the one name word of a name read family-first. Option A of the issue (H1 decides), Derek's call over option B (the comma decides, 1.4.0's reading of `Elizabeth II, Queen`, `John III, Sir` and `Smith, Sir`; 1.4.0 read `Anne, Princess` as first `Anne`, which this restores). `Elizabeth II, Queen` read family `Elizabeth` because assign writes the word before a family comma to the family and H1's site only ever moved given → family; the shape-only `John VI, Sir` read given `John` by accident, its numeral counting as a second word and making the comma a suffix comma, so the reading turned on how the numeral was spelled. The site now has the other direction: a family with no given and no middle, whose addressing run is a given-name title, moves to the given name. It is not gated on the structure, and that is what closes the issue's third open question too: under either family-first order assign places the one word of `Sir John` in the family, and H1 had never moved it back, as the 2026-08-22 #P5 entry recorded for `Sheik abdul salam` ("H1's exemption never runs under a family-first order"); both now read given under every order, P5's Accepted clause amended to match. A particle in the word keeps the family where something settled it: after a comma, C3's evidence that the part is a surname (`van der Berg, Sir`, `Van, Sir`); without one, a never-given particle, which P1's fold takes (`Sir de Mesnil`). An AMBIGUOUS particle with no comma is a word the declared order placed, so it reads given: the first draft skipped every particle and left `Aunt Della`, `Sir Van` and the corpus name `Sheik Abu Bakar` reading family under FAMILY_FIRST, against this clause and P5's amended one; the design-docs review found it by comparing `T X` under all three orders against `X, T` over every given-name title. Accepted, from the issue: a listing record whose given field holds only a given-name title (`Smith, Sir`) reads given `Smith` with the family empty, as `Sir Smith` does. - Measured 2026-10-18 over every distinct name in `tools/differential/corpus*.jsonl`, on this change against its parent (a detached worktree, `__file__` checked on both sides), across the seven fields and ambiguity kinds under all three orders: under the default order no name the corpus already held moves, and the four new H1 examples with a given-name title read differently than the parent would have read them (`Anne, Princess`, `Elizabeth II, Queen`, `John III, Sir`, `Smith, Sir`); under each family-first order 30 names move, every one a given-name title and one name word, comma-less or not (`Pope Francis`, `Swami Vivekananda`, `Sheik abdul salam`, `Sheik Abu Bakar`, `Sir Ph. D. Van Johnson`), each from family to given; the last two arrived with the review's particle fix, the first draft's count having been 28. + Measured 2026-10-18 over every corpus name against the parent under all three orders (PR #735): under the default order only the four new H1 examples with a given-name title move (`Anne, Princess`, `Elizabeth II, Queen`, `John III, Sir`, `Smith, Sir`); under each family-first order 30 names move from family to given, every one a given-name title and one name word (`Pope Francis`, `Sheik Abu Bakar`). THE FOLD'S OWN PREDICATE (eb0081d7, the PR's /simplify). The never-given hold had been a hand copy of P1's test asked of the first family TOKEN; the cleanup made it ask P1's `is_lone_never_given_particle` of the leading name PIECE, as the fold does. The two disagree where a title-particle (`St`, `Freiherr`) opens the title run ahead of a given-name title: group then chains the never-given particle forward, the fold does not fire, and the default order already read given `de Mesnil` in `St Sir de Mesnil`; the copy held the family under the family-first orders, so the order decided what the default did not. Now every order reads given, as this entry's clause says. Found by the refactor's own review, which measured 868 texts where the two predicates disagreed, every one with `St` or `Freiherr`, and moved readings only under a family-first order; no corpus name has the shape, so the snapshot did not move. The commit's "No reading moves" is wrong for this shape; it is pinned by the H1 example `St Sir de Mesnil` family-first. The snapshot reads the default order only, so the family-first moves are this entry's to record. Recompute by dumping the corpus glob under the three orders on both trees and diffing. diff --git a/docs/design/decisions/H5-trailing-title-run.md b/docs/design/decisions/H5-trailing-title-run.md index fbb55915..47db134c 100644 --- a/docs/design/decisions/H5-trailing-title-run.md +++ b/docs/design/decisions/H5-trailing-title-run.md @@ -9,22 +9,22 @@ - **A1, the peel order, decided by Derek: the first peel is PROVISIONAL and a trailing title is TRANSPARENT to the suffix reading.** `X Prof. Y` reads exactly as `X Y` reads, plus the title. SCOPED 2026-09-09, in review of the docs commit: that is a claim about inputs where a name word still stands on both sides of the chain — two or more name words. Where the chain leaves ONE name word there is no second reading for it to be transparent to, and #H1 decides the field instead: `Smith Prof.` reads family `Smith` where `Smith` alone reads given `Smith` and reports `given-or-family`, and `Smith Sir.` reads given `Smith` with an empty family, `sir` being a given-name title. All three measured on this tree; rules/H5's statement carries the scope and `_assign.py`'s excerpt of it was updated in the same edit. The problem is order: with ONE peel, `Prof.` standing behind `Jr.` stopped the suffix peel before `Jr.`, and the title walk then removed the very word that had been blocking it — `John Smith Jr. Prof.` read family `Jr.`, a generational suffix promoted to the family name, which is worse output than the reading it replaced. So the pieces the walk takes are spliced out and ONE peel runs over what stands, in original order, and that second answer alone places a piece or reports a fork. Measured: `John Smith Jr. Prof.` reads suffix `Jr.`; `John Prof. MA` reads the family `MA` that `John MA` reads, S2's reserve keeping a bare ambiguous acronym the family of a two-word name where a second peel laid over a first read family `John`, suffix `MA`; and `John Smith V Prof. VI` reads what `John Smith V VI` reads — middle `Smith V`, family `VI`, nothing reported — where two peels each reporting their own last piece reported twice. The choice is CORPUS-NEUTRAL: both variants move the same eight names, no more and no fewer, so it is a question about output quality alone. - **A2, the family-comma segment-1 path, decided: it gets the walk.** The spec's condition was "if the comma segment gate already routes those, say so and leave it". Measured, it does not: `Smith, John Prof.` read middle `Prof.` at 1.4.0, 2.0.0, 2.1.0, 2.2.0 and at this branch's parent — all five measured — while `Smith, John Prof. Dr.` read middle `Prof.`, suffix `Dr.` through 2.1.0 and middle `Prof. Dr.` from 2.2.0, `dr` having left the suffix vocabulary in #296. The eleven comma rows the spec calls "already routed" are the `Smith, Prof.` shape, where segment 1 holds NO name word and `segment_suffix_reading` reads it piece by piece — a different gate and a different mechanism. So the segment-1 walk was genuinely missing; with it, `Smith, John Prof.` reads title `Prof.`, given `John`, family `Smith`. It moved NO corpus name at the fix; `Smith, John Prof.` is a rules/H5 example, so it ENTERS the corpus in the docs commit that follows and is one of the seventeen in the population bullet below. Case rows pin it either way. The segment's walk follows the same transparency principle with its own lenient loop, which is why the candidates are the pieces that walk would not read as a suffix rather than the strict suffix test alone (`Smith, John Prof. Jr.` must reach past the post-nominal, `Smith, John Prof. V` past the numeral the lenient tail test claims, #144). - **What A2 does NOT reach, and why that is right.** `Smith, John Prof. MA` still reads middle `Prof. MA`: after a family comma a bare ambiguous acronym is a MIDDLE name and has been since 2.0 (the `Smith, Ed` cost S2 already accepted), so `MA` stands at the end of the segment, is not a period-marked title word, and stops the walk before it starts. The walk reads from the end; it does not hunt. -- **Measured population: FIVE corpus names at the fix, SEVENTEEN after the docs commit that follows it and the two review rounds on it.** The five are the four planted no-comma rows in corpus_issues.jsonl (`John Smith Dr.`, `John Smith Mr.`, `John Smith Prof.`, `John Smith Rev.`) plus `Andrew Perkins (Mgr.)`, which the drafting did not expect. Eleven of the other twelve are rules.md examples entering corpus_rules.jsonl, eight of them this rule's own — `Smith Prof.`, `Dr. John Smith Prof.`, `John Smith Prof. Dr.`, `John Smith Prof. Jr.`, `John Smith Jr. Prof.`, `John Prof. MA`, `Smith, John Prof.` and `Mary Jane King.`, the last of those added by the first 2026-09-09 review of that commit — plus three of #H1's, which reach this walk because the run each puts behind the name word is what the walk takes: `Smith Sir.`, and `Sir John Prof.` and `Dr. Smith Sir.` from the second review round the same day. The twelfth is `毛 泽东 Dr.`, which entered corpus_cjk_tolerated.jsonl in the FIRST review round and is the one member of the population in a native script: the walk runs before the positional read, so the pieces the script test sees are all Han and the family-first order stands. It is a corpus name the walk moves like any other, and the reason it took its own ledger rule rather than a seventeenth alternative is a guard about alternations, not a difference in the argument (the ledger comment beside it says which). So the growth is documentation rather than reach, and both numbers come off the same recipe — parse every corpus name twice, once on the tree and once with the walk disabled, and diff — run on the two trees. **The disabling stub is NOT the same on the two trees, corrected 2026-09-09 in review of the /simplify round below.** At the fix (`5568fdc`) `trailing_titles` returned the count the chain TAKES and both call sites were in assign, so a stub returning 0 took nothing and disabled the walk. Since the /simplify round it returns the count it LEAVES STANDING, so 0 now takes EVERYTHING and `John Smith` raises IndexError before any measurement happens; the disabling stub on this tree is `lambda rest, *a: len(rest)`, and it has to be bound over both `_pieces.trailing_titles`, which is what `tail_reading` calls, and the name `_assign` imported for the segment-1 walk, patching either alone leaving the other walk live. Re-run that way the two figures hold: FIVE corpus names at `5568fdc`, SEVENTEEN on this branch (2026-09-09). That fifth is a genuine hit rather than a misfire: rules/S1 drops the brackets and reads the content exactly as if written bare, so `(Mgr.)` is a trailing period-marked title word and reads as one; its own test stays green. Recompute by parsing every name in `tools/differential/corpus*.jsonl` on the tree and on the branch's third commit's parent and diffing the seven name fields plus `ambiguities`. Baselines differ per name — `John Smith Dr.` read suffix `Dr.` at 1.4.0, 2.0.0 and 2.1.0 (`dr` left the suffix vocabulary in 2.2, #296) and family `Dr.` at 2.2.0 — so the ledger entries are per-baseline. +- **Measured population: FIVE corpus names at the fix, SEVENTEEN after the docs commit that follows it and the two review rounds on it.** Re-measured 2026-09-09 at `5568fdc` and on the branch of PR #520 by parsing every `tools/differential/corpus*.jsonl` name with the walk on and disabled (recipe, and the disabling stub, which changed with the /simplify round below, in PR #520): the five are the four planted `John Smith Dr.`/`Mr.`/`Prof.`/`Rev.` rows and `Andrew Perkins (Mgr.)`, a genuine hit since rules/S1 reads bracketed content as if written bare, eleven of the twelve added are rules examples entering corpus_rules.jsonl, and the twelfth, `毛 泽东 Dr.`, entered the CJK corpus in the first review round, the one member in a native script; so the growth is documentation rather than reach. - **The inline frame-free gates at the two walk sites were REMOVED in review**, and this is the ONE-PREDICATE-PER-QUESTION half of the entry. Each site had a cheap inline test written to match the walk's own first condition; that is a second implementation of the question, and the measurement that justified it did not survive re-running. The band test runs early in a session where the facade sits at 453, so the "one frame of headroom" claim did not reproduce. With the gates gone the walk costs +1 frame on each entry point, inside the plan's target of two and inside `test_facade_cost_stays_within_its_band`. The walk's own cheapness is where the saving lives instead: the abbreviation-shape test is a compiled regex (a C call, no Python frame) and runs BEFORE the vocabulary call, and almost no name ends in a period-marked word, so the ordinary parse pays one match and stops. - **A pre-existing detail mismatch, recorded and NOT fixed.** #H4's join shape reports `title-or-name` with a `detail` that says the unit was "read as a given name by convention", while under the default order H1 retags the unit to the family — so `Dr. John of Prince` reports that text with the unit in `family`. This rule adds a second input with the same mismatch, `John of Prince Prof.`, and fixes neither: the wording predates this bundle, the fork the kind reports is title-versus-name which no field answers either way, and #H4 already records why the detail names no field for the peel shape. - 2026-09-09 (review round 2) — **A1's transparency holds for the ONE-name-word case too, and the scope written a day earlier was the defect rather than the boundary.** A1 was scoped that morning to inputs where a name word still stands on both sides of the chain, handing the one-word case to #H1 unconditionally: "the title behind that word decides its field as a title in front of it would". #H1 was then reading BOTH ends of the name as one title run, so `Sir John Prof.` read family `John` where `Sir John` reads given `John` — the trailing title changing what the leading one addressed by, which is exactly the non-transparency A1 denies. The clause is now conditioned on there being no run in FRONT of the word: where one stands it addresses and the chained trailing title only joins the title field, so `X Prof.` reads as `X` plus the title for one name word as for two. Where no run stands in front the trailing one still decides, unchanged — `Smith Prof.` reads family `Smith`, `Smith Sir.` given `Smith`. Nothing in the walk moved; the fix is entirely in which run H1 keys, and #H1's 2026-09-09 entry carries it, the measurement and the mutation. The #P5 clause below reads the same way afterwards and gains its second half: a bound given-name word behind a trailing title joins as it joins with the title absent AND lands in the same field. Said of the COMMA-LESS writing, and the review that found the H1 defect measured why the qualifier is needed: after a family comma the reserve reads no peel at all, so the join fires over the trailing title word and takes it into the given name — `Berg, abdul Prof.` reads given `abdul Prof.` where `Berg, John Prof.` reads title `Prof.`. That is PARITY (1.4.0 reads it the same, measured) and a gap in the segment path rather than a boundary of the rule, so it is recorded rather than fixed here: the clause carries the scope, and the case row `title_word_trailing_behind_a_bound_pair_after_a_comma` pins it without making it normative. Tracked as part of #316. (FIXED by #637, decisions/P5, 2026-10-11: the join declines the trailing title word, and the row pins the title.) - 2026-09-09 (the /simplify round on the bundle's PR) — **the trailing peel and the title chain are ONE reading run to a FIXED POINT, and A1's "ONE peel runs over what stands" was a truncation of it.** Two contrived readings contradicted clauses this section states, both found by an altitude review reading the code against the rule rather than by a test. (1) THE SUBTRACTION MODEL. #P5's reserve modelled assign's reading as the peel's count minus the chain's take, on each of the two views the join compares. That arithmetic equals assign's answer only while the chain's take does not change what the peel would do, and at rules/S4's bare-ambiguous reserve it does: `abdul rahman MA` reads given `abdul`, family `rahman`, suffix `MA` — the join declining because peeling the acronym unjoined and not joined is a suffix reading the join would change — while `abdul rahman MA Prof.` read given `abdul rahman`, family `MA`, the subtraction having counted `MA` a name word to spare. The Accepted clause below, "a bound given-name word behind one joins exactly as it joins with the title absent", was false as stated. (2) THE TRUNCATED FIXED POINT. The re-peel ran ONCE, so a second title was read half way: `John Prof. MA` reads title `Prof.`, given `John`, family `MA`, and `John Prof. MA Prof.` read title `Prof.`, given `John`, family `Prof.`, suffix `MA` — the re-peel un-peeled the acronym and re-exposed the FIRST title, which nothing then took. Both measured on the branch tip, 2026-09-09. - **The fix is one function, `_pieces.tail_reading`:** peel, chain, splice the chained pieces out, peel again over what is left, until the chain takes nothing. Assign calls it in place of its inline sequence and all its reports still come from the final peel; the reserve calls it on both views and compares the suffix lists it returns, and both subtractions are deleted. `abdul rahman MA Prof.` now reads given `abdul`, family `rahman`, suffix `MA` under title `Prof.`, and `John Prof. MA Prof.` reads title `Prof. Prof.`, given `John`, family `MA`. One further clause had to be said in the code rather than left to the arithmetic: the join must not take a piece the chain takes (#P5 — "a trailing title word the run takes is no word to spare"), which the subtraction had enforced by accident, by leaving that piece in the tail it compared. Without it `Sir abdul Prof.` joins the title into the given name; with it the reading is title `Sir Prof.`, given `abdul`, as before. `trailing_titles` returns the count it LEAVES STANDING in the same edit, the way `peel_trailing` counts, so the two compose at the call site without arithmetic at all. - **Cost, and what paid it.** The shared helper is one frame on the common path, where the inline sequence was none. It is paid by `_effective_order` taking the segment's pieces and the name-piece indices — the shape every piece-layer predicate takes — instead of a list built for it, which on 3.11 is a comprehension frame on every parse; the reference name stays at 416 parse / 453 facade. The helper returns a bare 3-tuple rather than a NamedTuple for the same budget: a NamedTuple's `__new__` is itself a frame, measured at +1 here. -- **How big each defect's class is, measured as the rule states it.** The two readings named above are the shapes that made the defects visible; the class both sit in is the TRANSPARENCY class A1 is stated over, and the two defects reach different parts of it — an earlier wording of this bullet flattened them into one width and named the wrong shape for it. Recipe: take the alphabet `John Smith abdul rahman Prof. Dr. Sir MA Jr. V Xyz. King de née`, form every 3- and 4-word arrangement ending in `Prof.` or `Dr.` whose spelling WITHOUT that last word still fills two of given/middle/family (3,210 of them — the two-or-more-name-words scope the A1 clause is stated for), and compare the seven fields other than `title` against that shorter spelling. rules/H5 says they must be equal. On the branch tip 786 differ; on this commit 698 do, none of them new, and all 698 are the P2/M2 residue the rule already accepts as a boundary — a particle chain or a maiden marker took the trailing word before the H5 chain could see it (`John de Prof.`, `Mary née Prof.`). The 88 that close PARTITION by defect, and that partition is what makes them two blast radii rather than one. 56 of them carry a SINGLE period-marked title word, and every one of those carries a bound given-name word as well — the subtraction model's reach, `abdul rahman MA Prof.` itself and `abdul John MA Prof.` beside it, where the arithmetic counted the acronym a name word to spare. The other 32 carry TWO, which is the truncated fixed point's reach: `John Prof. MA Prof.` and `John Dr. MA Dr.`, where the single re-peel re-exposed a title nothing then took. The two-title members of the 3,210 number 784, which is the population that could have witnessed the fixed point at all: 164 of them differed on the branch tip and 132 still do, every one of those 132 the same P2/M2 residue the 698 are, so the defect accounted for exactly the 32 that closed and no more. Over the same sweep's 82,712 inputs (both comma-less and family-comma spellings, 2 to 4 words) exactly 156 parses move, splitting the same way — 96 carry two period-marked title words, 60 carry one, and all 60 of those are bound-given — and not one mover is a comma spelling, the family-comma segment path reaching neither defect. Every figure in this bullet re-measured 2026-09-09 on this tree against `7ac2218`, the round's parent. +- **How big each defect's class is, measured as the rule states it.** Measured 2026-09-09 against `7ac2218` (the round's parent; PR #520), over the 3,210 three- and four-word arrangements in A1's two-name-word scope ending in `Prof.` or `Dr.` (recipe in PR #520): 786 broke transparency on the branch tip and 698 still do, all the P2/M2 residue the rule accepts as a boundary (`John de Prof.`, `Mary née Prof.`); of the 88 that close, 56 carry one period-marked title word and a bound given-name word (the subtraction model's reach) and 32 carry two (the truncated fixed point's), and over the sweep's 82,712 comma-less and family-comma inputs 156 parses move, none a comma spelling. -- **Nothing else moved, and it was measured rather than argued.** Snapshot all seven fields, the ambiguity kinds and the recorded `order` for every name in `tools/differential/corpus*.jsonl` plus every string literal in `tests/v2/cases.py` (`ast.walk` over that file, which sweeps up the notes and the case ids too — harmless, they parse like anything else), on the branch tip and on the fix, and diff: 2682 strings before and 2681 after, 2674 of them on both trees, and NONE of those 2674 reads differently. The seven that differ are the case rows this round added and dropped, not parses that moved. The two readings above are inputs no corpus holds — as is every other member of the class the bullet below measures — and each takes a case row (`title_word_trailing_run_is_read_to_a_fixed_point`, `title_word_trailing_behind_a_bound_pair_at_the_peel_reserve`). The same round dropped the `Sir Jr` example from rules/S2 and its case row: it reads by the same branches as `Dr Jr`, the same roles and the same reported kind, the run being empty by then and no branch reading `vocab:given-title` — so the pair pinned one reading twice. Its leaving corpus_rules.jsonl moves two ledger rosters: the jr suffix-routing rule 7 → 6 corpus names in the 1.4.0 ledger, and the #489 peel-floor rule 4 → 3 in all four. +- **Nothing else moved, and it was measured rather than argued.** Over every corpus name and every string literal in `tests/v2/cases.py` on the branch tip and on the fix (PR #520), none of the 2,674 strings on both trees reads differently; the two readings above are in no corpus and each takes a case row (`title_word_trailing_run_is_read_to_a_fixed_point`, `title_word_trailing_behind_a_bound_pair_at_the_peel_reserve`). The same round dropped the `Sir Jr` example from rules/S2 and its case row: it reads by the same branches as `Dr Jr`, the same roles and the same reported kind, the run being empty by then and no branch reading `vocab:given-title` — so the pair pinned one reading twice. - 2026-09-26 #535 — CROSS-REFERENCE: the maiden half of rules/H5's "the chain reads PIECES" Accepted row now reads the title — a maiden clause is no longer a join that puts the trailing word out of the chain's reach, so `Mary Smith née Jones Prof.` reads title 'Prof.' — decided in the 2026-09-26 #535 entry under `### M2`. -- 2026-10-07 #614 — the accepted cost 'the chain reads PIECES, so a join that ran earlier puts the word out of reach' is retired for the particle chain on a main segment: the chain is read with S2's run, once, after P3's connective joins and before the particle chain is made, so a particle chain stops in front of a trailing title -- `John van der Berg Prof.` reads title 'Prof.', family 'van der Berg' (every release from 1.4.0: family 'van der Berg Prof.'), the example rules/H5 carried for the cost now pinning the reading. It stands for a connective's join, made before the read, so a title it joined into a longer unit is no single word (`John Smith Prof. and Dr.` keeps family 'Prof. and Dr.', the case row that pins the one-word gate), and after a family comma, whose given part is read once all its joins are made (`Smith, John van der Berg Prof.` keeps middle 'van der Berg Prof.'; SUPERSEDED IN PART by #623, below: there the chain now stops in front of the title). decisions/S2, this date. +- 2026-10-07 #614 — the accepted cost 'the chain reads PIECES, so a join that ran earlier puts the word out of reach' is retired for the particle chain on a main segment: the chain is read with S2's run, once, after P3's connective joins and before the particle chain is made, so a particle chain stops in front of a trailing title -- `John van der Berg Prof.` reads title 'Prof.', family 'van der Berg' (every release from 1.4.0: family 'van der Berg Prof.'), the example rules/H5 carried for the cost now pinning the reading. It stands for a connective's join, made before the read, so a title it joined into a longer unit is no single word (`John Smith Prof. and Dr.` keeps family 'Prof. and Dr.', the case row that pins the one-word gate), and it stood after a family comma (SUPERSEDED IN PART by #623, below: there the chain now stops in front of the title). decisions/S2, this date. - 2026-10-10 (Derek), #623 — AFTER A FAMILY COMMA THE PARTICLE CHAIN STOPS IN FRONT OF A TRAILING TITLE TOO. Assign's walk over the given part already reads the trailing title run, with its own suffix reading (#144's lenient tail with two parts, #602's run, P6's attachment), so `Smith, John Prof.` reads title 'Prof.'. What it lacked was a chain that left the title standing: group bounded the chain there by `trailing_start`, the suffix peel alone, so the chain took the title into the particle's name before the walk met it, and `Smith, John van der Berg Prof.` read middle 'van der Berg Prof.' where `John van der Berg Prof.` reads title 'Prof.' since #614 (1.4.0 and 2.3.0 read middle 'van der Berg Prof.' for the comma spelling). At that site the bound is now `_pieces.trailing_run_start`, where the run `tail_reading` takes begins, its titles included, so the chain stops in front of the title and the walk reads it: title 'Prof.', middle 'van der Berg'. Nothing new is counted -- only the chain's stop moves -- so the issue's question of which words-to-spare floor the given part uses does not arise, and the comma's own readings of a numeral or a bare acronym there (`Smith, John van der Berg VI` middle 'van der Berg VI') do not move. P6 (the issue's open question) reads after the title run, as though the title were absent: `Smith, Jan van Prof.` → title 'Prof.', family 'van Smith', as `Smith, Jan van` reads. The bound is asked only where a particle stands past the chain's lead, by an inline tag scan, since the chain otherwise merges nothing -- which also drops the `trailing_start` call master made for every family-comma part. DECIDED (Derek): the narrower bound. Declined: the issue's proposal, reading the given part's run once ahead of the chain and splitting it off as the main segment's is (#614). It would have assign place the given part off group's read, a second reader beside the walk's own predicates, for a class the bound alone closes: the transparency measurement below finds no text the narrower fix leaves broken that the read would mend. - MEASURED 2026-10-10 against 4d5631c2 (#634's merge), py3.11, `nameparser.__file__` asserted on each side. A given-part grid -- `{family}, {head} {middle} {tail}`, and the same with `, Jr.` appended, for family in `Smith`, `van der Berg`, `Ortega y Gasset`, (empty), `(Bob)`; head in `John`, `Dr. John`, `Freiherr`, `abdul`, `Freiherr St`, `J.`; middle in (none), `van`, `van der`, `de la`, `van Berg`, `van der Berg`, `Mc Mc`, `von und zu`, `Berg`, `Berg van`, `do`, `St`, `y Lopez`; tail in (none), `Prof.`, `Dr.`, `Prof. Dr.`, `Prof`, `Sir`, `Prof. PhD`, `PhD Prof.`, `Jr. Prof.`, `Prof. Jr.`, `MA`, `Ma`, `VI`, `V`, `X.Y.Z.`, `Ph. D.`, `Esq.`, `née Jones`, `Prof. Jones`, `Xyz.`, `King.`, `Prof. V`, `MA Prof.`, `Lt.Gov.` -- 18,480 texts under the three orders and `lenient_comma_suffixes=False`, comparing the seven fields and every report: 17,340 of the 73,920 parses move a role (4,335 texts under each configuration), and no parse moves a report alone. All but 500 parses (125 texts) put a trailing period-marked title into the title field. H5's transparency over the grid's default-order texts whose given part ends in `Prof.`, `Dr.`, `King.` or `Prof. Dr.` (5,390; the parse with the title against the parse without it, plus the title): broken on 392, against 3,048 on master, and on no text master kept. By cause, 352 of the 392 are H1's move of the name behind a title into an empty family (an empty or nickname-only head), and 40 are P5's lenient post-comma join taking the title into a bound given name -- 24 after a named family (`Smith, abdul Dr.` reads given 'abdul Dr.') and 16 behind an empty head (`, abdul Prof.`), the gap rules/H5's Accepted P5 paragraph records (fixed by #637, decisions/P5, 2026-10-11). The 500 put a title in front of a numeral with a third comma part (`(Bob), Freiherr Mc Mc Prof. V, Jr.`): the bound reads 'V' as a numeral and stops before 'Prof.', where the walk keeps 'V' a name word with a third part (#144), so 'Prof.' lands in the middle 'Prof. V' rather than the given 'Mc Mc Prof.' -- a name word either way, garbage. The S2 stress grid's comma half (#614's recipe): 56 moves, all a trailing title behind a particle after a family comma (`Smith, Jan Freiherr von Berg Prof.`); #624's title-particle grid: 24, the same class (`Smith, John Freiherr Dr.` → title 'Dr.', family 'Freiherr Smith'). #625's particle grid, every corpus name and case text, and #624's comma-tail grid: 0. The five gates exit 0 with the new `fix(#623)` rule in every ledger. Frames (3.11, warm): the reference rows do not move; a family-comma name with a particle pays 2 with no trailing run (`Smith, John van der Berg` 282 → 284), 4 with a suffix (`Smith, John van der Berg Jr.`), and the moved shapes 38 to 73 in a sample (`Smith, John van der Berg Prof. PhD` +38, `Smith, John van der Berg Prof.` 310 → 360, `Smith, Jan van Prof.` +73), with an unmoved particle name ending in a suffix and a title 20 to 22 (`Smith, John van der Berg PhD Prof.` 382 → 402, `Smith, John van der Berg Jr. Prof.` 382 → 404), rare shapes no guard sees; one without a particle pays less (`Smith, John Quincy` 221 → 213, `Doe, John MA` 254 → 227), which `test_a_given_part_with_no_particle_skips_the_chain_bound` holds (the bound skipped, 'Smith, John Prof.' 250 frames with it asked against 207). Recorded negative control: with the old bound, 8 tests fail -- the two `fix(#623)` rows on both surfaces, the three new rules/H5 examples and the field-sweep count. + MEASURED 2026-10-10 against 4d5631c2 (#634's merge), py3.11, `nameparser.__file__` asserted on each side. A given-part grid -- `{family}, {head} {middle} {tail}`, and the same with `, Jr.` appended, for family in `Smith`, `van der Berg`, `Ortega y Gasset`, (empty), `(Bob)`; head in `John`, `Dr. John`, `Freiherr`, `abdul`, `Freiherr St`, `J.`; middle in (none), `van`, `van der`, `de la`, `van Berg`, `van der Berg`, `Mc Mc`, `von und zu`, `Berg`, `Berg van`, `do`, `St`, `y Lopez`; tail in (none), `Prof.`, `Dr.`, `Prof. Dr.`, `Prof`, `Sir`, `Prof. PhD`, `PhD Prof.`, `Jr. Prof.`, `Prof. Jr.`, `MA`, `Ma`, `VI`, `V`, `X.Y.Z.`, `Ph. D.`, `Esq.`, `née Jones`, `Prof. Jones`, `Xyz.`, `King.`, `Prof. V`, `MA Prof.`, `Lt.Gov.` -- 18,480 texts under the three orders and `lenient_comma_suffixes=False`, comparing the seven fields and every report: 17,340 of the 73,920 parses move a role and none a report alone, all but 500 parses (125 texts) putting a trailing period-marked title into the title field (PR #635); H5's transparency over the grid's 5,390 default-order texts ending in a trailing title breaks on 392 against master's 3,048, and on no text master kept (352 H1's move behind an empty head, 40 P5's post-comma join taking the title, fixed by #637); 500 parses (125 texts) put a title in front of a numeral with a third comma part into the middle, garbage; the S2 stress grid's comma half (#614's recipe) and #624's title-particle grid move 56 and 24 of the same class, #625's particle grid, every corpus name and case text, and #624's comma-tail grid none. Frames: a family-comma name with a particle pays 2 with no trailing run, 4 with a suffix, and the moved shapes 38 to 73 in a sample, rare shapes no guard sees, and one without a particle pays less (`Smith, John Quincy` 221 → 213, `Doe, John MA` 254 → 227), which `test_a_given_part_with_no_particle_skips_the_chain_bound` holds; recorded negative control: with the old bound, 8 tests fail. THE REVIEW (2026-10-10). The bound is `tail_reading`'s run, read as the comma-less name reads it, and the walk reads the given part by its own predicates, so where the two disagree on a word behind a title the chain stops for a title the walk does not take, or cannot cross one to reach a particle P6 takes: a title in front of a shape-only numeral (`Smith, John Mc Prof. VI` → suffix 'Mc', middle 'Prof. VI'; master middle 'Mc Prof. VI'; the walk never reads 'VI' as a suffix after a comma), in front of a numeral with a third comma part (the 500 above), in the middle of the given part in front of a particle (`Smith, John van Prof. Mc` → title 'Prof.', middle 'van', family 'Mc Smith'), and a period-marked title-particle ending it, which P6 attaches (`Smith, John van MA St.` → suffix 'MA', family 'St. Smith'). Every such shape is garbage, most read as master already reads the text without the title (`Smith, John Mc VI` suffix 'Mc'), and none is a reading a well-formed name loses: over about 1.6 million generated rows under every order, the opt-in policies, one-case writing and caller lexicons, and over a 134,720-pair transparency grid (`Smith, ` and `, ` heads; one to three words from `John`, `van`, `der`, `von und zu`, `Berg`, `Ma`, `MA`, `V`, `VI`, `Jr.`, `PhD`, `Ph. D.`, `Mc`, `St.`, `St`, `Freiherr`, `X.Y.Z.`, `Esq.`, `abdul`, `Prof.`; a trailing ` Prof.`, with and without `, Jr.` after it; the three orders and `lenient_comma_suffixes=False`; transparent where given, middle, family, nickname and maiden equal the title-less text's and the title gains exactly ` Prof.`; master broken on 53,384 pairs, the tree on 9,734), no text transparent to its title on master is broken now. (The first review named `Smith, John Ph. D. St. Prof.` as one, its oracle reading the direction backwards: master gives suffix 'Ph. D. St. Prof.' against the title-less 'family St. Smith', and the tree is transparent.) Recorded, not fixed (Derek): garbage, and narrowing the bound would add a condition for it. The review also found the bound's particle gate held by no test, two of its narrowings undoing readings master gives -- a scan from the name's start rather than the chain's lead (`Smith, Freiherr von Berg Prof.`) and a test of single-token particles alone (`Smith, John von und zu Berg Prof.`) -- now rows of their own, with an empty-head row (`, Jan van Prof.`, AGENTS.md's comma sweep), and the frame saving now held as above. Each of the three mutations fails a test. The re-ask after the chain (`left < tail`) still decides readings, though its documented case (`Freiherr von Berg Ma`) takes the read-once path since #614: removed, it moves 400 parses of the given-part grid on master and here alike, a title-particle head before a particle run and a dotted acronym (`Smith, Freiherr van Berg X.Y.Z.` → given 'van Berg X.Y.Z.', as 1.4.0 reads it; without it suffix 'X.Y.Z.'). It decides a main segment's run with a name piece inside it too (`Freiherr van Berg Dr. X.Y.Z.` → family 'van Berg Dr. X.Y.Z.', as master and 2.3.0 read it; without it given 'van Berg Dr.', family 'X.Y.Z.', 1.4.0's reading), a garbage shape no row holds. The first review reported it changing nothing over a grid with no such shape, and #638 was filed on that; the fix review found it live and held by no test, and the row `the_chain_runs_again_where_its_merges_free_the_acronym` (parity) holds it now. Left for follow-ups, two gaps outside this change: a part past the second comma reads a trailing title behind a particle into the suffix (`John Smith, Jr., van der Berg Prof.`, where `John Smith, Jr., Prof.` reads title 'Prof.'; #636), and P5's post-comma join taking a title (the 40 above; #637). diff --git a/docs/design/decisions/M2-maiden-marker-rule.md b/docs/design/decisions/M2-maiden-marker-rule.md index 8eef1817..2d973b04 100644 --- a/docs/design/decisions/M2-maiden-marker-rule.md +++ b/docs/design/decisions/M2-maiden-marker-rule.md @@ -14,7 +14,7 @@ - 2026-08-21 #412 / #417 / #418 (PR #420) — the marker pass runs before every join in the grouping stage. The three issues were one shape: a join rule computed something — P3's word count (#418), the chain's stop condition (#417), the connective join's range (#412) — against a piece list the marker pass would rewrite a few lines later. #411 had fixed the same shape in P5's reserve by hand, excluding the departing words and declining a join onto the marker, and #418 argued the general fix was a guard at the merge primitive: no join may span a marker piece. Stated unconditionally that guard is the ungated stop the 2026-08-19 entry above measured and rejected (it strands `Jane van der Berg née`), so it would have to be gated on "a marker the consumer will take", evaluated live at every merge — #417's shared predicate, plus a per-merge scan the `"and " * n` benchmark shape would have to tolerate, plus the count exclusions kept in step. The reorder set aside above gets the property by construction: the consumer runs first, what it takes is gone before any count or join looks, and the stop, the exclusion and the shared-predicate question all disappear, a net deletion. P5's decline-onto-a-marker test stays, because it is about what a name word is, not about counting — measured: removing it re-read `Berg, abdul née PhD` as given 'abdul née'. It tests the LONE marker piece, which is the only marker the consumer can have declined on its own; a declined marker the particle chain has already taken travels with the chain, and P5 joins the chain — `Abd van der Berg née Jr Jones` reads given 'Abd van der Berg née' where the old stop had kept the marker out of it. Review found this; rules/P5 now says "standing as a word of its own" for that reason. - The recorded ground for setting the reorder aside was that nothing separated it from the stop. That stopped being true when #413's harvest landed, two hours after #418 was filed: eleven corpus names flip P3's carve-out under an appended maiden clause (eight losing the family — #418 counted seven and six, before the harvest), and seven show the connective join swallowing the marker — `johnny y`, `Juan de y`, `e and e` with the marker behind the connective, and `Claus SchroederVP of`, `Jon Dough and`, `Jon Dough and of`, `jones, john e` with the marker ahead of it — where #412 recorded no corpus name behind it at all (review counted the seven; the first draft of this entry had three). Measured over every corpus name, every corpus name with ` nee jones` / ` née Jones` / ` née Jr Jones` appended (comma names included) and every case row, 4,180 inputs: 55 differ, three of them inputs this change itself adds (the P3 example, its appended forms, the new case row), and each of the rest either a shape of the three issues or the clause ceasing to change how the rest of the name reads — `The Hon nee jones` reads given 'The', family 'Hon' because `The Hon` does; `der, y van nee jones` reads family 'van der' because `der, y van` does. No ambiguity report moves. The suffix-comma tails, the ordering cost the #411 entry left unmeasured, were byte-identical in every shape tried — under the default policy, which review showed was the wrong population. With `extra_suffix_delimiters` configured, a delimiter core after a marker in a tail became a word the pass could take (`Smith, John, PhD née - Jones` read maiden '- Jones'), and a core the pass left standing alone stopped being dropped (`Smith, John, - née Jones` read suffix '-', maiden 'Jones'): the tail's core drop used to run before the pass and now ran after it. Closed before merge by having the pass read a tail segment with its cores screened out and leave them for the drop — the old order restored for cores — with both shapes pinned at the piece level. Two corpus names move at the 2.0.0 and 2.1.0 baselines, the #412 pair; at 1.4.0 fix(#274) already classifies them. A third, `Juan y Garcia née Jones`, is the example this change adds to P3 and so to the rules corpus, and moves at all three — the one #418 shape the gate can see, the other eight showing it only under an appended clause. + The recorded ground for setting the reorder aside was that nothing separated it from the stop. That stopped being true when #413's harvest landed, two hours after #418 was filed: eleven corpus names flip P3's carve-out under an appended maiden clause (eight losing the family — #418 counted seven and six, before the harvest), and seven show the connective join swallowing the marker — `johnny y`, `Juan de y`, `e and e` with the marker behind the connective, and `Claus SchroederVP of`, `Jon Dough and`, `Jon Dough and of`, `jones, john e` with the marker ahead of it — where #412 recorded no corpus name behind it at all (review counted the seven; the first draft of this entry had three). WHAT MOVES, measured over every corpus name, every corpus name with a maiden clause appended (comma names included) and every case row, 4,180 inputs, against the pre-change tree (PR #420): 55 differ, each an input this change adds, a shape of the three issues, or the clause ceasing to change how the rest of the name reads, and no ambiguity report moves. The suffix-comma tails, the ordering cost the #411 entry left unmeasured, were byte-identical under the default policy, which review showed was the wrong population: with `extra_suffix_delimiters` configured a delimiter core after a marker in a tail became a word the pass could take, and the repair made before merge, screening a tail's cores out of the pass, was reversed by #549 (the 2026-10-03 entry below). At the gate the #412 pair moved at 2.0.0 and 2.1.0 (fix(#274) at 1.4.0), and P3's new example `Juan y Garcia née Jones` at all three, the one #418 shape the gate can see. The one reading the order costs: the consumer sees unjoined pieces, so "up to any trailing suffix" now means the first suffix WORD after the marker, and a connective beside it cannot un-suffix it first. `Jane née Jr y Jones` read maiden 'Jr y Jones' and now declines (family 'Jr y Jones'). The shape is a suffix homograph ANYWHERE in the maiden name with a connective beside it, not only in first position — review measured `Jane Smith née Jones Jr y Smith` going from maiden 'Jones Jr y Smith' to maiden 'Jones', family 'Jr y Smith', middle 'Smith': the maiden walk stops at the homograph, and the connective then builds a family name out of what is left, demoting the real one. The first draft of this entry priced it as the first-word shape, from a sweep that appended only ` née Jr Jones`. Accepted rather than built around, and the one place a merge-time guard would have read differently. @@ -48,15 +48,15 @@ the fullwidth-colon marker (旧姓:佐藤 arrives as one word; the head-peel q The contract test should have caught this and did not, and the hole is worth naming because it is a property of how the contract was FORMULATED. It varied the marker's SPELLING across nine forms and held its PLACEMENT fixed, so every run it built was structurally contiguous by construction and no probe ever asked what the sites do when a run straddles a boundary — which is precisely where two sites walking different token populations can disagree. The placement axis is now its own test over eight placements, asserting two things no spelling can reach: that every tagged run has one role and one comma bucket, and that group drops a run whole or declines it. Mutation-checked in three pieces, which is what shows the two halves of the rule are both load-bearing: removing the contiguity bound outright fails both new case rows and all three straddling placements; removing only the ROLE half fails the two clause placements and both rows; removing only the COMMA half fails exactly one placement and NO case row, because a comma-split run happens to be declined downstream for an unrelated reason today. That last one is the argument for a tag-level contract test in its own right — the only witness to half the rule is an assertion about tags, not about fields. - Perf, and how it was measured, because the first attempt at it was wrong in a way worth recording. Timing four source trees in separate processes and comparing them across sessions gave figures whose spread was the size of the effect: merged master alone measured 0.579s, 0.593s and 0.607s on one machine inside an hour. Every number below is a median of five INTERLEAVED rounds — every variant timed once per round, in order, so drift lands on all of them alike — over 8000 parses of a four-name mix (plain, titled-with-initials, phrase-marked, comma-and-particle). Against master: the run pass alone +6.8%, a head-word fast path inside the predicate +1.4%, the contiguity rule on top of that +1.6%, and after the simplify round -0.3% and -1.3% on two separate five-round runs. So the shipped mechanism has no cost this benchmark can resolve. That is not a free lunch and should not be read as one: classify no longer does the per-token `n in lexicon.maiden_markers` membership test master did — the marker decision moved into one pass that answers for every token at once — so what the marker pass adds and what it removed are within noise of each other. The reviewer reproduced the +7.2% and +1.6% stages independently on other hardware, which is the corroboration a percentage of this size needs. + Perf, measured as medians of five INTERLEAVED rounds (every variant timed once per round, since timing trees in separate processes across sessions gave a spread the size of the effect) over 8000 parses of a four-name mix, against master before PR #448: the shipped mechanism costs nothing this benchmark resolves (-0.3% and -1.3% on two runs, the run pass alone having been +6.8%), the marker pass having replaced classify's per-token membership test, and the reviewer reproduced the stages on other hardware. Four structural choices got it there, and each is a smaller version of the same move: ask the question once and pass the answer along. The fast path lives INSIDE the predicate rather than at its callers, so it stays an implementation detail and not a second answer. maiden_marker_head takes the FOLD rather than the text (suffix_as_written's shape, "n is _normalize(text), passed in so callers normalize once"), and classify folds once per token for the marker pass and the vocabulary tags together, which took the common token from two folds to one — fewer than before this work. maiden_marker_run folds each word once and keys from a prefix, where _title_key(words[:n]) per candidate length re-folds the whole prefix and made a ONE-word hit cost more than a two-word one; sixteen of the seventeen shipped entries are one word. And the marker pass returns its decision as index -> tag for classify's single token-building loop to write, rather than rebuilding every marker token a second time with dataclasses.replace, which is the most expensive call in a parse. The comma-bucket sweep the contiguity rule needs is deferred until a marker head is actually found, so a single-word vocabulary and a phrase vocabulary over a name holding no marker both skip it entirely. One limit is recorded rather than closed. The contiguity test is role plus comma bucket, and role is one-directional: a role change is always a clause edge, but two ADJACENT clauses of the same role are indistinguishable, so 'Jane (z) (domu) Jones' does tag a run across two separate nickname clauses. Nothing reads it — the piece walk never sees role-bearing tokens and the clause drop is scoped to one span — and the parse is identical with and without the phrase mechanism. The clean fix is a per-token structural UNIT id derived from state.segments and state.extracted, which would replace role-plus-bucket with one comparison and close the adjacent-clause case as a side effect. Deliberately not taken here: it changes a mechanism four correctness reviewers had just cleared, inside a pass whose stated business was not to change behavior. It is the shape to reach for if this test ever needs to grow a third term. -- 2026-09-19 (Derek), #533 — THE CLAUSE ENDS AT A TRAILING CREDENTIAL, AND THE DOUBLE CHECK THE 2026-08-22 BULLET DEFERRED NOW EXISTS FOR THE ACRONYM TOO. That bullet stopped the walk before the trailing numeral and left the bare-acronym fork alone, with its reason stated: the acronym fork COUNTS pieces and the walk removes the very pieces it counted, so a peel read over the pieces as they stand said "credential with words to spare" about `John née Jones Smith Ma` and would have left maiden 'Jones Smith', family 'Ma'. The reason was right and is unchanged; what it argued for was a deferral, and the answer is the one the numeral got — ask the peel TWICE, once over the pieces as written and once over the name the take would leave, and stop only where both read the word as the credential. "Left to assign" meant "read in silence", assign never getting the question at all, the clause's tokens carrying the maiden role before pieces exist for it: the end of a maiden clause was the last trailing position in the library where a member of the ambiguous credential class was read without a report. WHICH RULE reads the name left standing is the part the numeral's version did not have to answer, because a numeral reads the same way everywhere and a credential does not. With no comma it is the trailing peel; after a family comma it is #531's slot, where the comma has already settled the count and the writing decides alone; before that comma the words are the family the comma named and a stop would hand one of them to `family` rather than to `suffix`; past a second comma the segment is read as credentials whole and no trailing rule is consulted. The last two READ AS NONE, which is a statement and not a default — the first prototype reported on `Smith, John, Jr nee Jones MA`, where the walk had asked nothing. A FIFTH CASE, found by the design-docs review of this commit and not by the plan, and THE PR REVIEW THEN SETTLED IT THE OTHER WAY — see the review bullet below, which is where the reasoning now lives. The shape: after a family comma the reader may take the word and nothing may then report it. `Jane Doe, Jr nee Smith MA` is a family comma whose segment 1 is `Jr nee Smith MA`, so the given-slot reader is asked and would take the member, but the take leaves that segment as `Jr MA`, post-nominals only, which the credential-run gate reads whole without ever reaching the emitter — and the released word therefore lands in `given`, not in `suffix`. This commit read it as maiden 'Smith', suffix 'Jr MA', silently; the review round declines the stop instead, so the clause keeps 'Smith MA' and reports, which is 2f57ff21's reading. The KEPT direction always reported here, the clause's own emitter being what raises it: `Jane Doe, Jr nee Smith Ma` and `Jane Doe, Jr nee MA` both say so, and now so does `Jane Doe, Jr nee Smith MA`. Pinned by a unit test rather than by a rules.md example line: the name differs from the tree at all five baselines and most of the diff is not this change's (1.4.0 reads title 'Jr', given 'nee', family 'Jane Doe'; 2.0.0 and 2.1.0 title 'Jr', family 'Jane Doe'; 2.2.0 and 2.3.0 already read given 'Jane', family 'Doe', suffix 'Jr'), so an example line would have put five ledger rules into this docs commit to attribute four other changes' readings. THE SECOND CHECK ASKS WHETHER THE READER TAKES THAT WORD, not whether it takes something, and the distinction is not academic: the numeral's existing re-ask tests whether the view's trailing run starts at the view's end, exactly right for a fork that reads one piece, and under it `JOHN NEE JONES SMITH MA PHD` leaves `JOHN MA PHD`, whose peel takes 'PHD' and then DECLINES 'MA' for want of words to spare — so the weaker question answers yes while the member becomes the FAMILY name, which is the 2026-08-22 disaster one word further on. Measured both ways on the prototype; the shipped check compares the view's run start against the member's own index in that view, and `JOHN NEE JONES SMITH MA PHD` keeps maiden 'JONES SMITH MA' with suffix 'PHD' and reports. AFTER A FAMILY COMMA THE CHECK NEEDS #531'S FLOOR, NOT ONLY ITS LEAN: `Doe, Jane nee Smith MA do` leaves `Jane MA do`, where 'MA' leans credential but the 'do' behind it does not read as a suffix (P6 keeps it, and `Doe, Jane MA do` reads middle 'MA', family 'do Doe'), so #531's slot would read 'MA' as a MIDDLE name — and a check that asked only about 'MA' released it from the clause into the current name's middle, a word crossing from one person's name to another's. With the floor the clause keeps 'MA do' whole and reports the 'do'. THE FLOOR ON THE FIRST WORD IS A CLAMP, NOT A VETO, and the first draft got two names wrong by making it one: stated as "a member that is the ONLY word after the marker stays the maiden name" and implemented as "cancel the stop when nothing would be left", it cancels too much wherever the peel consumed that first word AND words behind it — `Doe, J. nee MA ba` peels 'ba' and then 'MA', so the first piece the peel took IS the only maiden word, and the veto handed 'ba' back to the clause, giving maiden 'MA ba' where `Doe, J. ba` reads suffix 'ba'. Clamped to the piece after the marker instead, that name keeps maiden 'MA' and reads suffix 'ba', reporting both. The floor itself is deliberate and diverges from the `née V` / `née PhD` precedent, where the marker declines and stays an ordinary word: certain suffix vocabulary declines the marker, while an ambiguous word is kept by the clause it ends, because that class is the one carrying no evidence of which it is — its members are borne surnames as well as credentials, and nobody writes a credential straight after the marker. 1.4.0 read `Jane Doe nee MA` as last 'nee', suffix 'MA' (measured on the wheel), so this half restores nothing and does not try to. THE `do` PAIR is #531's reading, unchanged and now SHARED as a predicate rather than spelled twice (`credential_at_the_given_slot` in `_pipeline/_pieces.py`; the drift would have been silent, each site's own tests going on passing): `Doe, Jane nee Smith do` keeps maiden 'Smith do', `Doe, Jane nee Smith Do` keeps 'Smith Do', `Doe, Jane nee Smith DO` gives maiden 'Smith' with suffix 'DO', and the comma-less `Jane Doe nee Smith do` reads suffix 'do' — right for the reason the whole change is, since `John Doe do` reads suffix 'do' today, P6 does not run without a comma, and a clause must not change how a word outside it reads. ACCEPTED COSTS, each measured on this tree: `JANE DOE NEE YO-YO MA` reads maiden 'YO-YO', suffix 'MA', a two-word birth surname losing its last word in a one-case record — and `JANE YO-YO MA` reads suffix 'MA' too, so the cost is the one-case reading's rather than this rule's, and in mixed case the writing saves it (`Jane Doe nee Yo-Yo Ma` keeps maiden 'Yo-Yo Ma' and reports); `Jane Doe nee Smith X.Y.Z.` reads suffix where 1.4.0 read last 'X.Y.Z.', the same trade #531 took at its own slot and for agreement with `John Doe X.Y.Z.` rather than for parity; (two costs listed here were WITHDRAWN by the PR review and are no longer costs at all — `Doe, Dr. nee Smith MA` and `Berg, abdul nee Jones MA` both keep their maiden words and report; the bullet below carries the reason); `Jane Smith nee Jones and MA` reads maiden 'Jones and' with suffix 'MA', the take running before the joins (#420) so the connective is a piece of its own when the walk stops; and a report on a declined word adds noise without moving a field, which is #530's stated rule and the trade #531 took at its slot. NOT TRANSPARENT HERE: a trailing title. `Jane Doe nee Smith MA Prof.` is unchanged and silent while `Jane Doe nee Smith Prof. MA` gives maiden 'Smith Prof.' with suffix 'MA' and reports, because the walk reads the trailing credential run and the H5 title chain is read by the rule the walk does not run. Accepted for #533, pinned as a pair of boundary example lines under M2; moving the walk onto the title-aware reading would change the numeral half too, which the 2026-08-22 bullet settled with its own measurements, and the pair belongs to one decision about what "trailing" means inside a clause — a follow-up is drafted for Derek rather than filed here. THE `one_case` PLUMBING AT THESE SITES IS NOW LIVE, and decisions/S6's 2026-09-18 bullet headed THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT stands as what was true then: `numeral_only` answered off the peeled numeral and the numeral fork is decided before the peel reads a lean, so the fact reached only the bare-acronym fork, which that reading discarded. The acronym fork is asked now. Re-measured 2026-09-19 with the same wrapper shape over the same population that entry's recipe names — AND THE PAIR IS THE FINDING: over the corpus as it stood the day before this change, dropping the argument inside the walk moved 0 parses; over the corpus WITH this change's own rows it moves 36 of 10,752 (1,792 names), on six distinct names (`Doe, Jane nee Smith DO`, `Doe, Jane nee Smith Ma`, `Jane Doe nee Smith Ma`, `Jane Doe nee Smith Ma JD`, `Jane Doe nee Yo-Yo Ma`, `John née Jones Smith MA`). The population, the six policies and the denominator are recorded at the call site in `_pipeline/_group.py`, as the 2026-09-18 entry records its own; both were re-run on 2026-09-19 after the review found the figures first written there (1,790 names, 10,740 and 21,480 parses) did not reproduce from the recipe as stated — they had been taken two corpus names earlier in the same day's work. The FINDING was unharmed, 36 parses on the same six names. The plumbing was live either way and the corpus could not show it, which is the blindness mechanisms.md's corpus field note asks to be measured before any "N names move" is written down. +- 2026-09-19 (Derek), #533 — THE CLAUSE ENDS AT A TRAILING CREDENTIAL, AND THE DOUBLE CHECK THE 2026-08-22 BULLET DEFERRED NOW EXISTS FOR THE ACRONYM TOO. That bullet stopped the walk before the trailing numeral and left the bare-acronym fork alone, with its reason stated: the acronym fork COUNTS pieces and the walk removes the very pieces it counted, so a peel read over the pieces as they stand said "credential with words to spare" about `John née Jones Smith Ma` and would have left maiden 'Jones Smith', family 'Ma'. The reason was right and is unchanged; what it argued for was a deferral, and the answer is the one the numeral got — ask the peel TWICE, once over the pieces as written and once over the name the take would leave, and stop only where both read the word as the credential. "Left to assign" meant "read in silence", assign never getting the question at all, the clause's tokens carrying the maiden role before pieces exist for it: the end of a maiden clause was the last trailing position in the library where a member of the ambiguous credential class was read without a report. WHICH RULE reads the name left standing is the part the numeral's version did not have to answer, because a numeral reads the same way everywhere and a credential does not. With no comma it is the trailing peel; after a family comma it is #531's slot, where the comma has already settled the count and the writing decides alone; before that comma the words are the family the comma named and a stop would hand one of them to `family` rather than to `suffix`; past a second comma the segment is read as credentials whole and no trailing rule is consulted. The last two READ AS NONE, which is a statement and not a default — the first prototype reported on `Smith, John, Jr nee Jones MA`, where the walk had asked nothing. A FIFTH CASE, after a family comma (`Jane Doe, Jr nee Smith MA`), was read here as maiden 'Smith', suffix 'Jr MA', silently; the review bullet below settled it the other way, the clause keeping 'Smith MA' and reporting, and #601 then made a marker in that given part an ordinary word. THE SECOND CHECK ASKS WHETHER THE READER TAKES THAT WORD, not whether it takes something, and the distinction is not academic: the numeral's existing re-ask tests whether the view's trailing run starts at the view's end, exactly right for a fork that reads one piece, and under it `JOHN NEE JONES SMITH MA PHD` leaves `JOHN MA PHD`, whose peel takes 'PHD' and then DECLINES 'MA' for want of words to spare — so the weaker question answers yes while the member becomes the FAMILY name, which is the 2026-08-22 disaster one word further on. Measured both ways on the prototype; the shipped check compares the view's run start against the member's own index in that view, and `JOHN NEE JONES SMITH MA PHD` keeps maiden 'JONES SMITH MA' with suffix 'PHD' and reports. AFTER A FAMILY COMMA THE CHECK NEEDED #531'S FLOOR as well as its lean (`Doe, Jane nee Smith MA do` kept 'MA do' whole), a given-part reach #601 removed. THE FLOOR ON THE FIRST WORD IS A CLAMP, NOT A VETO, and the first draft got two names wrong by making it one: stated as "a member that is the ONLY word after the marker stays the maiden name" and implemented as "cancel the stop when nothing would be left", it cancels too much wherever the peel consumed that first word AND words behind it — `Doe, J. nee MA ba` peels 'ba' and then 'MA', so the first piece the peel took IS the only maiden word, and the veto handed 'ba' back to the clause, giving maiden 'MA ba' where `Doe, J. ba` reads suffix 'ba'. Clamped to the piece after the marker instead, that name keeps maiden 'MA' and reads suffix 'ba', reporting both. The floor itself is deliberate and diverges from the `née V` / `née PhD` precedent, where the marker declines and stays an ordinary word: certain suffix vocabulary declines the marker, while an ambiguous word is kept by the clause it ends, because that class is the one carrying no evidence of which it is — its members are borne surnames as well as credentials, and nobody writes a credential straight after the marker. 1.4.0 read `Jane Doe nee MA` as last 'nee', suffix 'MA' (measured on the wheel), so this half restores nothing and does not try to. THE `do` PAIR is #531's reading, unchanged and now SHARED as a predicate rather than spelled twice (`credential_at_the_given_slot` in `_pipeline/_pieces.py`; the drift would have been silent, each site's own tests going on passing): `Doe, Jane nee Smith do` keeps maiden 'Smith do', `Doe, Jane nee Smith Do` keeps 'Smith Do', `Doe, Jane nee Smith DO` gives maiden 'Smith' with suffix 'DO', and the comma-less `Jane Doe nee Smith do` reads suffix 'do' — right for the reason the whole change is, since `John Doe do` reads suffix 'do' today, P6 does not run without a comma, and a clause must not change how a word outside it reads. ACCEPTED COSTS, each measured on this tree: `JANE DOE NEE YO-YO MA` reads maiden 'YO-YO', suffix 'MA', a two-word birth surname losing its last word in a one-case record — and `JANE YO-YO MA` reads suffix 'MA' too, so the cost is the one-case reading's rather than this rule's, and in mixed case the writing saves it (`Jane Doe nee Yo-Yo Ma` keeps maiden 'Yo-Yo Ma' and reports); `Jane Doe nee Smith X.Y.Z.` reads suffix where 1.4.0 read last 'X.Y.Z.', the same trade #531 took at its own slot and for agreement with `John Doe X.Y.Z.` rather than for parity; (two costs listed here were WITHDRAWN by the PR review and are no longer costs at all — `Doe, Dr. nee Smith MA` and `Berg, abdul nee Jones MA` both keep their maiden words and report; the bullet below carries the reason); `Jane Smith nee Jones and MA` reads maiden 'Jones and' with suffix 'MA', the take running before the joins (#420) so the connective is a piece of its own when the walk stops; and a report on a declined word adds noise without moving a field, which is #530's stated rule and the trade #531 took at its slot. NOT TRANSPARENT HERE: a trailing title. `Jane Doe nee Smith MA Prof.` is unchanged and silent while `Jane Doe nee Smith Prof. MA` gives maiden 'Smith Prof.' with suffix 'MA' and reports, because the walk reads the trailing credential run and the H5 title chain is read by the rule the walk does not run. Accepted for #533, pinned as a pair of boundary example lines under M2; moving the walk onto the title-aware reading would change the numeral half too, which the 2026-08-22 bullet settled with its own measurements, and the pair belongs to one decision about what "trailing" means inside a clause — a follow-up is drafted for Derek rather than filed here. THE `one_case` PLUMBING AT THESE SITES IS NOW LIVE, and decisions/S6's 2026-09-18 bullet headed THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT stands as what was true then: `numeral_only` answered off the peeled numeral and the numeral fork is decided before the peel reads a lean, so the fact reached only the bare-acronym fork, which that reading discarded. The acronym fork is asked now. Re-measured 2026-09-19 over the population that entry's recipe names (PR #534): dropping the argument inside the walk moved 0 parses over the corpus of the day before and 36 of 10,752 (1,792 names, six distinct names) with this change's own rows, so the plumbing was live and the corpus could not show it, the blindness mechanisms.md's corpus field note names. -- 2026-09-19 (Derek), #533 review — A WORD THE CLAUSE GIVES UP READS AS A POST-NOMINAL OR THE CLAUSE KEEPS IT, AND THAT INVARIANT REPLACES THE ACCEPTED SILENT MOVERS ABOVE. The PR review measured what the released words actually do, and three of them do not land in `suffix`: `Doe, Prof. nee Smith A.B.` read given 'A.B.' (66 distinct names in the review's own sweep, the same shape reaching `ba`, `X.Y.Z.`, caps-on `XYZ` and the lower-case spellings); `Berg, Jane van der nee Smith DO` read family 'van der DO Berg' and `Jane Doe nee Smith DO DO` read family 'DO DO', a word crossing from the BIRTH name into the current one, which is #424's failure from the other side; and `Berg, abdul nee Jones MA` read given 'abdul MA'. Two of those had been written up as ACCEPTED COSTS in the bullet above, on the argument that the clause form AGREES with the clause-less one. The agreement is real and it is not the question: the clause-less `Berg, abdul MA` puts the word in the given name because nobody promised otherwise, while a CLAUSE promised that the word was either the birth name or a credential, and a third answer is the promise broken in silence. So the direction is the parent's, conservative: THE STOP IS RIGHT ONLY WHERE THE RELEASED WORD ENDS THE PARSE SUFFIX-ROLED, and where the take cannot promise that, the clause keeps the word (and reports it, the emitter asking whether a trailing rule reads these words at all rather than whether the view check then passed). Two view checks carry it, each modelling a LATER decision rather than predicting one: the given-slot reader requires the view to still hold a name word AHEAD of the member, since a trailing slot needs a part to be the end of; and a release is withdrawn where a join reached below the marker pass would absorb the word — P2's chain, for a particle-tagged member behind a particle piece or beside another released one, and P5's bound-given join, for the word after the bound one. Both over-decline rather than predict, which is the direction the invariant asks for. MEASURED, and the pair of numbers is the argument: over the review's 142,518-name corpus under six policies, 855,108 parses, the invariant holds 0 violations at 2f57ff21 and 0 here, against 1,446 in 40 shapes at the commit this round started from; and a token-level diff against 2f57ff21 finds 128,424 moved parses on 24,958 names with ZERO of the moved tokens unnamed by a `suffix-or-name` report, against 4,100 silent moved tokens in 59 shapes before. `tests/v2/test_properties.py` carries the invariant as a property over a generated grid rich enough to hold every shape above; the grid fails on 790 of its 8,466 parses at d97d3eb7 and passes at 2f57ff21, which is what makes it a test rather than a restatement. TWO ACCEPTED SILENCES REMAIN AND ARE NOT SILENCES OF THIS KIND: a member with a name word BEHIND it was never asked about, and a delimited clause is not a fork at all (below). ONE ROUGH EDGE IS RECORDED RATHER THAN FIXED, and it PREDATES this change: with `extra_suffix_delimiters=('-',)`, `Jane Doe nee Smith - MA` gives maiden 'Smith -' — a clause ending on a bare delimiter token. 2f57ff21 does the same thing one word later (`Jane Doe nee Smith - PhD` gives maiden 'Smith -' there too), so what #533 changes is only which words reach the edge. Walking the stop back over it is not the two-line change it looks like: the delimiter-core set the walk already steps over is EMPTY off a tail segment, which is exactly where this name's `-` sits, so the fix would have to widen what counts as a core on every path and would move the parent's reading as well. Left for its own issue. +- 2026-09-19 (Derek), #533 review — A WORD THE CLAUSE GIVES UP READS AS A POST-NOMINAL OR THE CLAUSE KEEPS IT, AND THAT INVARIANT REPLACES THE ACCEPTED SILENT MOVERS ABOVE. The PR review measured what the released words actually do, and three of them do not land in `suffix`: `Doe, Prof. nee Smith A.B.` read given 'A.B.' (66 distinct names in the review's own sweep, the same shape reaching `ba`, `X.Y.Z.`, caps-on `XYZ` and the lower-case spellings); `Berg, Jane van der nee Smith DO` read family 'van der DO Berg' and `Jane Doe nee Smith DO DO` read family 'DO DO', a word crossing from the BIRTH name into the current one, which is #424's failure from the other side; and `Berg, abdul nee Jones MA` read given 'abdul MA'. Two of those had been written up as ACCEPTED COSTS in the bullet above, on the argument that the clause form AGREES with the clause-less one. The agreement is real and it is not the question: the clause-less `Berg, abdul MA` puts the word in the given name because nobody promised otherwise, while a CLAUSE promised that the word was either the birth name or a credential, and a third answer is the promise broken in silence. So the direction is the parent's, conservative: THE STOP IS RIGHT ONLY WHERE THE RELEASED WORD ENDS THE PARSE SUFFIX-ROLED, and where the take cannot promise that, the clause keeps the word (and reports it, the emitter asking whether a trailing rule reads these words at all rather than whether the view check then passed). Two view checks carried it, a name word ahead of the member at the given slot and the withdrawal of a release a P2 chain or P5 join would absorb; #601 replaced the release checks with the clause-free reading bound as roles. MEASURED over the review's 142,518-name corpus under six policies (855,108 parses): 0 invariant violations at 2f57ff21 and on this tree against 1,446 in 40 shapes at the round's starting commit, and no moved token left unreported among 128,424 moved parses against 2f57ff21, against 4,100 silent moved tokens in 59 shapes before; the property grid in `tests/v2/test_properties.py` fails 790 of its 8,466 parses at d97d3eb7. TWO ACCEPTED SILENCES REMAIN AND ARE NOT SILENCES OF THIS KIND: a member with a name word BEHIND it was never asked about, and a delimited clause is not a fork at all (below). ONE ROUGH EDGE IS RECORDED RATHER THAN FIXED, and it PREDATES this change: with `extra_suffix_delimiters=('-',)`, `Jane Doe nee Smith - MA` gives maiden 'Smith -' — a clause ending on a bare delimiter token. 2f57ff21 does the same thing one word later (`Jane Doe nee Smith - PhD` gives maiden 'Smith -' there too), so what #533 changes is only which words reach the edge. Walking the stop back over it is not the two-line change it looks like: the delimiter-core set the walk already steps over is EMPTY off a tail segment, which is exactly where this name's `-` sits, so the fix would have to widen what counts as a core on every path and would move the parent's reading as well. Left for its own issue. - 2026-09-19 (Derek), #533 review — THE DELIMITERS TRUMP ANY OTHER READING, and that is a settled position rather than an un-asked fork. `Jane Doe (nee Smith Ma)` and `Jane Doe (nee Smith MA)` both keep the WHOLE span as the maiden name and report nothing, in every release including this one; the marker INSIDE the delimiters is what tells the parser the span is a maiden clause, even where the pair is not configured in `maiden_delimiters` (M3). The writer drew the boundary, so no fork is called and there is nothing to report — which is a different kind of quiet from the silences M2 and `AmbiguityKind.SUFFIX_OR_NAME` document, where a fork WAS available and the parser declined to ask it. The boundary cuts both ways: `Jane Doe (nee Smith) MA` gives suffix 'MA' and reports, the member standing outside the span being an ordinary trailing credential. Behavior is unchanged from 2f57ff21 and was not touched; what the review added is the three rules.md example lines, the case rows, and the statement in M2 and in the `SUFFIX_OR_NAME` docstring, because an undocumented settled position reads as an oversight to the next person who measures it. @@ -67,13 +67,13 @@ the fullwidth-colon marker (旧姓:佐藤 arrives as one word; the head-peel q INV6's off-switch exemption is NOT widened, and the reason is the one AGENTS.md gives for not widening an exemption a grid cannot reach: measured 2026-09-20, 1,680 rows of that grid carry both a class letter and a marker and 0 of them put the letter after the marker, its generator placing the connective among the name's own words. A clause link has its own grid instead. TWO SECOND-ORDER MOVEMENTS, both recorded rather than repaired. The two one-case spellings part company, which is P8's own Accepted clause (P3's until the 2026-10-19 split) reaching a taken marker rather than a declined one: classify leaves a letter after a marker to the mixed-case rule, so `JANE DOE NEE PUIG I SOLER` never moved (the capital is an initial, and the initial veto kept the walk going all along) while `jane doe nee puig i soler` goes from family 'doe i soler' to maiden 'puig i soler'. And a report is GAINED where a longer clause ends on an ambiguous credential: `Jane Doe nee Puig i Ma` keeps maiden 'Puig i Ma' and says `suffix-or-name` where 0fbcaa0b read family 'Doe i Ma' in silence — the clause's own emitter, reaching a word the truncation had put out of its reach. A THIRD, found by the second review and the mirror of that one: a report is LOST where the clause takes a class member it used to end at. `Jane Doe nee MA i Soler` reads maiden 'MA i Soler' in silence where the parent 46651750 read maiden 'MA' with a `suffix-or-name` on it. That FOLLOWS from the link no longer ending the clause and is not a second decision: the report is the walk's own, raised on the LAST word the clause kept where a trailing rule was asked about it, and with the link joining, 'MA' has a name word behind it and is no longer that word — which is exactly what `Jane Doe nee MA Smith` has always done with the same acronym. The `y` twin is the oracle and it AGREES, in fields and in reports: `Jane Doe nee MA y Soler` reads maiden 'MA y Soler' in silence at the parent and here. Measured 2026-09-20 over the review grid, 24 names lose the report this way and all 24 agree with their `y` twin on every field and every report. -- 2026-09-22 #397 follow-up — A DELIMITER CORE PAST THE CLAUSE'S FIRST WORD PASSES FOR THE NAME WORD BESIDE A LINK, RECORDED AS A DEVIATION RATHER THAN REPAIRED. The link exception above wants "a connective standing between two name words of the clause", and a separator the caller declared through `Policy.extra_suffix_delimiters` is structure, not a name word — so a link with one beside it joins nothing and should end the clause like any other suffix word. Between the marker and the clause's first word that already holds, the bound refusing the core before either piece test is asked. PAST that first word it does not: the core is an ordinary index to the run walk, which steps over connectives and nothing else, so it stands in for the name word on the link's left and the clause runs on past a title it would otherwise stop at. MEASURED 2026-09-22 under `extra_suffix_delimiters=(" - ",)`: `Smith, John, PhD née Puig Mr. - i Soler` reads maiden 'Puig Mr. i Soler' where its separator-less twin `Smith, John, PhD née Puig Mr. i Soler` stops at 'Puig Mr.'. THE POPULATION is the branch's own sweep, recorded with the code it describes (2026-09-21, corpus ∪ cases.py ∪ the property grids ∪ a 50,925-name generated set with cores, under thirteen core-bearing policies): the predicate is asked about a core in 51,072 of 900,023 calls, the answer differs from a core-skipping reading in 8,094 parses over 1,278 texts, and 1,824 of those move `maiden` on 288 texts — none of the 288 reachable at the default policy, `extra_suffix_delimiters` being empty there. NOT REPAIRED HERE: the fix threads the core set through three call sites into the run walk and moves the parent's reading as well, which makes it its own change rather than a rider on a review round. Open: #538. PINNED TWICE MEANWHILE. rules/M2 carries the shape as a `deviates: #538` example under an `extra_suffix_delimiters-dash` annotation — the first entry `tests/v2/rules_doc.py`'s registry has had for that field, named after the Policy field and carrying the delimiter in the suffix because the field's value is a set rather than a flag. And `tests/v2/pipeline/test_group.py::test_a_core_beside_a_link_wrongly_passes_for_a_word_until_538` holds the pair at the piece level, named so nobody reads it as the contract. ONE COST OF THE DOC EXAMPLE, worth knowing before the repair lands: its string enters `corpus_rules.jsonl`, where the differential gate parses it with the DEFAULT facade — no delimiter declared, so the dash is an ordinary name word and #538's reading is off the path entirely. It moves there for the 2026-09-20 link fix instead, and is classified as that at all five baselines: added to the `fix(#397) a link inside a maiden clause stays in the birth name` alternation at the four 2.x ledgers (suffix 'PhD i Soler' → 'PhD', maiden 'Puig Mr. -' → 'Puig Mr. - i Soler', identical at each), and given its own rule at 1.4.0, where v1 had no maiden markers and read the whole suffix-comma tail as one suffix. -- 2026-09-26 #538 — A DELIMITER CORE IS STEPPED OVER BY THE LINK'S NEIGHBOUR SEARCH, AS A CONNECTIVE IS (Derek, 2026-09-26). The 2026-09-22 follow-up above recorded the deviation; this resolves it. `_run_neighbours` treats a lone core like a connective, so the word on a link's side is the one past the core and a maiden clause reads exactly as the same text written without the core: `Smith, John, PhD née Puig Mr. - i Soler` under `extra_suffix_delimiters=(" - ",)` now stops at maiden 'Puig Mr.' as its separator-less twin does. INVARIANT, pinned by `tests/v2/test_properties.py::test_a_delimiter_core_reads_as_if_it_were_not_written` over 81 generated texts, maiden field: 6 disagreed at e0f1a2fa, 0 after. Declined: THE BOUNDARY READING, which the 2026-09-22 entry's wording implied ("a link with one beside it joins nothing") — a core ending the neighbour search on its side. It fixes the reported shape equally, and it moves `Smith, John, PhD née Puig - i Soler` and `… Puig i - Soler` from maiden 'Puig i Soler' to maiden 'Puig', suffix 'PhD i Soler' — a birth-name link pushed into the credentials, where the skip reading leaves both as they read. Default-unreachable either way. THE SAME STEPPING APPLIES OUTSIDE A CLAUSE: the `frozen` loop's own `_run_neighbours` call also takes `cores`, so a connective beside a core looks past it; where a credential or nothing stands beyond, it stays a lone suffix word rather than joining, and the core it stood beside -- now a lone piece with nothing joined to it -- is dropped as #206 drops any lone core, the same way it already drops one between two ordinary post-nominals: `Smith, John, PhD - i Soler` 'PhD - i Soler' -> 'PhD, i Soler', `Smith, John, PhD - i - MD` 'PhD - i - MD' -> 'PhD, i, MD', `Smith, John, - i Puig` '- i Puig' -> 'i Puig', `Smith, John, Puig i -` 'Puig i -' -> 'Puig i'. MEASURED 2026-09-26: 191 of 8,097 generated core-bearing tail texts (the three heads `Smith, John,`, `Smith, John, Jr.,` and `John Smith,`, each followed by every 2-4-word product of {PhD, MD, Puig, i, y, -, Jr., Soler, Mr.} containing a core) move `suffix` and no other field, none reachable at the default policy; the comparison is each text parsed under `extra_suffix_delimiters=(" - ",)` with `_run_neighbours` handed its cores and with it handed none. Not repaired here: a core the join MERGES into a joined piece -- interior (`… Puig Dr. i - y Soler`, still 'PhD i - y Soler') or at the edge a link joins across, a name word standing beyond it rather than a credential or nothing (`Smith, John, Puig - i Soler` 'Puig - i Soler', its separator-less twin 'Puig i Soler'; `Smith, John, Puig i - Soler` 'Puig i - Soler', measured 2026-09-26) -- is not dropped from the suffix text; the join is right, the surviving separator is the gap. Also in that family: an ORDINARY connective's join still takes a core as its neighbour, the stepping above being for generational vocabulary alone (rules/P3) -- `Smith, John, PhD - and MD` keeps suffix 'PhD - and MD' (measured 2026-09-26), pre-existing and unchanged here. Open: #549, which carries both the surviving core and this one. +- 2026-09-22 #397 follow-up — A DELIMITER CORE PAST THE CLAUSE'S FIRST WORD PASSES FOR THE NAME WORD BESIDE A LINK, RECORDED AS A DEVIATION RATHER THAN REPAIRED. The link exception above wants "a connective standing between two name words of the clause", and a separator the caller declared through `Policy.extra_suffix_delimiters` is structure, not a name word — so a link with one beside it joins nothing and should end the clause like any other suffix word. Between the marker and the clause's first word that already holds, the bound refusing the core before either piece test is asked. PAST that first word it does not: the core is an ordinary index to the run walk, which steps over connectives and nothing else, so it stands in for the name word on the link's left and the clause runs on past a title it would otherwise stop at. MEASURED 2026-09-22 under `extra_suffix_delimiters=(" - ",)`: `Smith, John, PhD née Puig Mr. - i Soler` reads maiden 'Puig Mr. i Soler' where its separator-less twin `Smith, John, PhD née Puig Mr. i Soler` stops at 'Puig Mr.'. THE POPULATION, measured 2026-09-21 over the branch's own sweep (corpus, cases.py, the property grids and a 50,925-name generated set with cores, under thirteen core-bearing policies): the answer differs from a core-skipping reading in 8,094 parses over 1,278 texts, 288 of them moving `maiden`, none reachable at the default policy. NOT REPAIRED HERE: the fix threads the core set through three call sites into the run walk and moves the parent's reading as well, which makes it its own change rather than a rider on a review round. Open: #538. PINNED TWICE MEANWHILE. rules/M2 carries the shape as a `deviates: #538` example under an `extra_suffix_delimiters-dash` annotation — the first entry `tests/v2/rules_doc.py`'s registry has had for that field, named after the Policy field and carrying the delimiter in the suffix because the field's value is a set rather than a flag. And `tests/v2/pipeline/test_group.py::test_a_core_beside_a_link_wrongly_passes_for_a_word_until_538` holds the pair at the piece level, named so nobody reads it as the contract. ONE COST OF THE DOC EXAMPLE: the differential gate parsed it with the default facade, where the dash is a name word, so it moved there for the 2026-09-20 link fix and was classified as that at all five baselines. +- 2026-09-26 #538 — A DELIMITER CORE IS STEPPED OVER BY THE LINK'S NEIGHBOUR SEARCH, AS A CONNECTIVE IS (Derek, 2026-09-26). The 2026-09-22 follow-up above recorded the deviation; this resolves it. `_run_neighbours` treats a lone core like a connective, so the word on a link's side is the one past the core and a maiden clause reads exactly as the same text written without the core: `Smith, John, PhD née Puig Mr. - i Soler` under `extra_suffix_delimiters=(" - ",)` now stops at maiden 'Puig Mr.' as its separator-less twin does. Its invariant (6 of 81 generated texts disagreed at e0f1a2fa, 0 after) was replaced with the reading by #549's, under decisions/C10. Declined: THE BOUNDARY READING, which the 2026-09-22 entry's wording implied ("a link with one beside it joins nothing") — a core ending the neighbour search on its side. It fixes the reported shape equally, and it moves `Smith, John, PhD née Puig - i Soler` and `… Puig i - Soler` from maiden 'Puig i Soler' to maiden 'Puig', suffix 'PhD i Soler' — a birth-name link pushed into the credentials, where the skip reading leaves both as they read. Default-unreachable either way. THE SAME STEPPING APPLIED OUTSIDE A CLAUSE, a lone core beside a connective being dropped as #206 drops any (191 of 8,097 generated core-bearing tail texts moved `suffix` and no other field, measured 2026-09-26 against the same tree with no cores handed in), while a core a join merged survived in the suffix text; #549 (the next entry) reversed the skip reading and settled both. - 2026-10-03 #549 — SUPERSEDED: the skip reading the #538 entry above settles is reversed, and a declared delimiter in a trailing part now parts it as a comma would, so `Smith, John, PhD née Puig - i Soler` reads maiden 'Puig' with suffix 'PhD, i Soler', the reading #538 declined and the one 2.3.0 gave, and a marker with a core straight before or after it takes no clause (`Smith, John, PhD née - Jones` reads suffix 'PhD née, Jones', reversing the #418 entry's repair). The decision, its measurement and the invariant that replaces #538's are under decisions/C10's 2026-10-03 entry. -- 2026-09-26 #535 — THE MAIDEN WALK READS THE TRAILING TITLE CHAIN, AND EVERY STOP ASKS ONE RELEASE QUESTION OF THE SPAN IT GIVES UP (Derek, 2026-09-26). This resolves the NOT TRANSPARENT HERE clause of the 2026-09-19 #533 entry above, which recorded `Jane Doe nee Smith MA Prof.` and `Jane Doe nee Smith Prof. MA` disagreeing and deferred the pair as one decision about what "trailing" means inside a clause. BOTH HALVES WERE TAKEN TOGETHER: a trailing title ends the clause (`Jane Doe nee Smith Prof.` reads title 'Prof.', maiden 'Smith', where 2.3.0 read maiden 'Smith Prof.'), and a credential or numeral standing in front of that title gets the stop it gets with the title absent (`… Smith MA Prof.` gives suffix 'MA', `… Smith V Prof.` suffix 'V'). Only the first half would have left the pair disagreeing in the other direction; H5's transparency is a statement about the peel and the chain read together to their fixed point, so the walk now reads the end of the clause the way assign reads the end of the name — `tail_reading`, the shared predicate mechanisms.md#ONE-PREDICATE-PER-QUESTION names — and over the forms the transparency property test below exercises, the two spellings give one answer. They split where something in or ahead of the clause stands to take the title — for example a particle chain (ahead of the clause or inside it), a bound-given join, a name left with no name word, or, after a family comma, a title word in front of a class member with a title behind it, which the given part's chain stops short of exactly as it does with no marker (`Doe, Jane nee Smith Rev. MA Prof.` reads maiden 'Smith Rev.', title 'Prof.', suffix 'MA', where `… Rev. Prof. MA` reads title 'Rev. Prof.', maiden 'Smith', suffix 'MA'; bare `Doe, Jane Rev. MA Prof.` reads middle 'Rev.') — and the Accepted pairs recorded here and in rules/M2 are examples of that, not a complete list. `tests/v2/test_properties.py::test_a_trailing_title_is_transparent_to_the_maiden_clause` holds that as an invariant over the two inputs, over heads and runs where the credential is given up and nothing ahead would take the title, with its negative control at e0f1a2fa in its docstring. ACCEPTED, THE TRANSPARENCY BOUNDARY: where the clause KEEPS the credential the spellings differ, because a clause is one contiguous run and a title inside the kept text cannot leave without the words behind it — `Doe nee Smith ba Prof.` reads title 'Prof.', maiden 'Smith ba', and `Doe nee Smith Prof. ba` maiden 'Smith Prof. ba'; `abdul nee Smith MA Prof.` reads title 'Prof.', family 'abdul', maiden 'Smith MA', and `abdul nee Smith Prof. MA` given 'abdul', maiden 'Smith Prof. MA' (2.3.0 kept every word in all four). The report differs with them: `Doe, Dr. nee Smith MA` and `… Prof. MA` report `suffix-or-name` on the kept MA, `… MA Prof.` reports nothing, the kept member no longer ENDING the clause, which is what M2's report is asked of. rules/M2 carries the first pair as an Accepted example. ACCEPTED, FURTHER SPLITS, measured 2026-09-26. Where something ahead of the clause would take the title, the spellings differ even with the credential given up in both, because the first-suffix-word stop ends the clause before the title check runs and the title check alone declines: `Jane van der Berg nee Smith PhD Prof.` reads title 'Prof.', maiden 'Smith', suffix 'PhD', and `… Prof. PhD` maiden 'Smith Prof.', suffix 'PhD' — H5's accepted particle-chain boundary inherited, the bare `Jane van der Berg Smith PhD Prof.` / `… Prof. PhD` splitting the same way — and `Berg, abdul nee Smith PhD Prof.` (bound-given join) and `Doe, Dr. nee Smith PhD Prof.` (no name word left) split likewise; the parent d9d80492 and 2.3.0 read all of these as the tree does, so only the claim is new. And a released particle with a title behind it is withdrawn: `Jane Doe nee Smith DO Prof.` reads title 'Prof.', maiden 'Smith DO', where `… Prof. DO` reads title 'Prof.', maiden 'Smith', suffix 'DO' (2.3.0 maiden 'Smith DO Prof.' and 'Smith Prof. DO'); `do` likewise. A particle INSIDE the clause does it from the other side, its chain able to take the credential behind it: `Jane Doe nee Smith do MA Prof.` reads title 'Prof.', maiden 'Smith do MA', and `… do Prof. MA` title 'Prof.', maiden 'Smith do', suffix 'MA' (2.3.0 kept every word in both). rules/M2 carries a pair of each shape as Accepted examples. ACCEPTED, H5'S REACH: `Jane Doe nee Smith King.` reads title 'King.', maiden 'Smith' (2.3.0 maiden 'Smith King.'), H5's accepted `Mary Jane King.` cost now reaching the last word of a birth name as it reaches the last word of a current one; rules/M2 carries it as an Accepted example. READER SCOPE IS UNCHANGED: the chain is consulted only where a trailing rule reads the clause's words (no comma, the part before a suffix comma, the given part after a family comma); before a family comma and in a tail segment the walk reads the peel alone and the clause keeps the title (`Doe nee Smith Prof., Jane` keeps maiden 'Smith Prof.'). THE FIRST-WORD FLOOR MOVED INTO THE CHAIN. `trailing_titles` and `tail_reading` take a `floor` (1 everywhere else), and the walk passes the position just past the marker's first word, so the chain never takes that word: `Jane Doe nee King.` keeps its maiden name (TITLES holds borne surnames and the marker announced one), and `Jane Doe nee Prof. Dr.` keeps 'Prof.' and gives up 'Dr.'. The first draft applied the floor as a clamp on the title stop after the chain had run, and that is measurably wrong rather than merely inelegant: a chain allowed to take the first word has already spliced it out of the count the re-peel reads, so `Jane Doe nee King. ba` kept 'ba' in the clause where `Jane Doe nee Smith ba` gives it up. `test_a_title_first_word_counts_as_a_word` is the invariant (a title-vocabulary first word against an ordinary one, the credential behind it read alike) and its docstring records the clamp's failure count. ONE RELEASE CHECK FOR THREE STOPS. The numeral, the credential and the title stop each ask `_release_reads_off` whether the name the take would leave reads what they give up as titles or suffixes with no join below the take absorbing it — rules/M2's invariant, "a word the clause gives up reads as a post-nominal or the clause keeps it", which the title stop now answers to as well. It is asked of the whole SPAN a stop gives up, not of the stop's own word: a first draft that checked the word alone broke the invariant on 513 of a 5,198-parse sweep (measured 2026-09-26 on that draft, which is not in the tree to recompute), because a stop gives up every word behind it — `DOE NEE SMITH PROF. MA` stopped at the title and handed the MA to the family, where the name left standing reads MA as a name; it keeps maiden 'SMITH PROF. MA' now. The question is asked per reader, the way that reader reads: the TRAILING reader is assign's own `tail_reading` over the view; the GIVEN_SLOT reader has its count settled by the comma, so it is the writing alone with a name word ahead. The name-word-ahead half is what keeps `Dr. nee Jones Smith Prof.` whole — the take would leave `Dr. Prof.`, whose Prof. would be the family name. THE JOINS. A released span holding a title with a particle ahead of it declines, because P2's chain runs on over a trailing title (rules/H5's Accepted `John van der Berg Prof.`): `Jane van der Berg nee Smith Prof.` keeps maiden 'Smith Prof.', and a released particle with a title behind it is withdrawn for the same reason (`Jane Doe nee Smith MA do Prof.` keeps maiden 'Smith MA do'). The bound-given (P5) half of `_join_takes_the_member` is asked for the GIVEN_SLOT reader alone: P5 is `BoundJoin.LENIENT` only after a family comma, and before one its STRICT reserve already refuses a join that would change a suffix reading, so asking there over-declined — `abdul nee Smith V` kept 'V' in the clause, where the scoped check lets it go to suffix as 2.3.0 did. The consequence is recorded by a case row rather than prevented: `abdul nee Smith Dr.` reads title 'Dr.', family 'abdul', maiden 'Smith', which is how `abdul Dr.` reads bare (2.3.0 read given 'abdul', maiden 'Smith Dr.'). THREE NUMERAL-STOP READINGS MOVE WITH IT, decided in rather than deferred, since each is the same invariant broken at the stop this change was already rewriting. The numeral stop never asked the join question: `Berg, abdul nee Smith V` read given 'abdul V' at 2.2.0 and 2.3.0 (given 'abdul nee', suffix 'V' at 2.0.0 and 2.1.0, #411's reserve differing between those pairs before the walk runs) and reads maiden 'Smith V' now. After a family comma the given slot reads a lone numeral as a suffix only where the given part is the LAST comma part — assign's own two-segment condition from #144 — and the walk now asks it too (`tail_follows`): `Doe, Jane nee Smith V, PhD` read middle 'V' at 2.2.0, 2.3.0 and e0f1a2fa, an M2 violation predating this change that its first draft had extended to `… V Prof., PhD`, and it reads maiden 'Smith V' now, as 2.0.0 read it. The withdrawal reaches through a title behind the numeral, so `Doe, Jane nee Smith Prof. V, PhD` keeps maiden 'Smith Prof. V' where `Doe, Jane nee Smith V Prof., PhD` gives up the title — the same asymmetry the bare given slot has with no marker in it (`Doe, Jane Prof. V, PhD` against `Doe, Jane V Prof., PhD`). And the numeral stop reads FROM the marker, unlike the other two, so a numeral straight after the marker is not held to the first-word floor, and it may decline the clause; examples, not a rule over every shape: `Dr. nee V` and `Doe, J. nee V` keep maiden 'V' as 2.3.0 did, though `Doe, J. V` reads suffix 'V', and `Doe nee V, Jane` and `Smith, John, PhD nee V` read no clause at all (family 'Doe nee V', suffix 'PhD nee V', as at 2.3.0) — `Jane Doe nee V Prof.` read maiden 'V Prof.' at 2.3.0 and reads family 'nee', suffix 'V', title 'Prof.' now, as `Jane Doe nee V` reads plus the title. After a family comma with another comma part behind the given one, the same #144 condition that keeps `Doe, Jane nee Smith V, PhD` whole now keeps a lone numeral in the clause as well: `Doe, Jane nee V, PhD` read given 'Jane', middle 'nee V', suffix 'PhD' at 2.3.0 and at the parent d9d80492 and reads maiden 'V', suffix 'PhD' now, as 2.0.0 and 2.1.0 read it, `Doe, Jane nee V, Jr.` likewise — the marker had been read as a name word there at 2.2.0 and 2.3.0, and a case row records it. A LINK THE WALK STOPS AT ASKS THE RELEASE QUESTION TOO. Reading the end of the clause through the title chain makes the link exception refuse a link it used to join — in `Jane Doe nee Smith i DO Prof.` the DO is the peel's once the title is chained — and a stop at a link gives up the words behind it. Unchecked, that put 'Doe i' in the middle name and 'DO Prof.' in the family (e0f1a2fa read maiden 'Smith i DO Prof.'), and `Berg, abdul nee Smith i V Prof., MD` read middle 'V'. So a link the exception refuses asks `_release_reads_off` of the run it would give up — only where the exception, bounded by the peel over the words as WRITTEN, would have joined it (the refusal is the title chain's), where a trailing rule reads the clause, and where the link is not the first word after the marker (a first-word stop declines the clause and gives nothing up); where that fails the clause keeps the link and walks on. Scoped that narrowly because a first version asked it of every refused link and kept links the name left standing reads off: `Doe, Jane nee Smith i V` kept maiden 'Smith i' where every release gives up suffix 'i V' (23,472 of a 411,936-parse link grid moved clean readings, measured 2026-09-26). With it, the given-part model reads the lenient numeral in both of assign's passes — the literal last piece first, then the last one standing once the chain has taken the titles behind it — and only where no comma part follows (`Doe, Jane nee Smith i V Prof.` gives up 'i V' and the title, as bare `Doe, Jane i V Prof.` reads them; `… i V Prof., PhD` keeps maiden 'Smith i V'). The same model moves the credential stop where a lenient numeral follows the credential: `Doe, Jane nee Smith MA V` now gives up 'MA V' (suffix 'MA V', maiden 'Smith') as bare `Doe, Jane MA V` reads it, where d9d80492 and 2.3.0 read maiden 'Smith MA', suffix 'V'; `… MA V, PhD` keeps maiden 'Smith MA V'. Over that link grid (links i, y, e, and, i y, y i, and i × 8 heads × bodies × `, MD` tail × 3 cases × 3 name orders), against the tree before the link check: 0 new violations, 6,195 fixed, and the clean readings that move now match the bare given part's (`DOE, JANE NEE SMITH MA I` gives up 'MA I', as `DOE, JANE MA I` reads suffix 'MA I'). `Jane Doe nee Smith i DO Prof.` now reads title 'Prof.', maiden 'Smith i DO' and reports the kept DO, and `Berg, abdul nee Smith i V Prof., MD` maiden 'Smith i V Prof.', suffix 'MD'. A RELEASED TITLE THAT IS ALSO A PARTICLE IS KEPT AFTER A FAMILY COMMA, because P6 attaches a particle trailing the given part to the family: released by the title stop, 'St.' in `Doe, Jane nee Smith St.` reads family 'St. Doe', so `_join_takes_the_member` declines it and the name reads maiden 'Smith St.', as e0f1a2fa and 2.3.0 read it, and so does `Doe, Jane nee Smith MA St.`. Titles only — a credential that is also a particle ('DO') is the given slot's own lean, which the credential stop has already asked (#533). A FOURTH MEASUREMENT, against e0f1a2fa rather than d9d80492: the M2 invariant over a fuzz of twelve heads ({`Jane Doe`, `Doe, Jane`, `Doe, Prof.`, `Jane Doe, PhD`, `Doe, J.`, `Berg, Jane van der`, `Jane van der Berg`, `Berg, abdul`, `abdul Berg`, `J. Doe`, `Prof. Jane Doe`, `Dr.`}), ` nee Smith ` and every one-to-three-word sequence over {MA, Ma, V, Prof., St., King., PhD, ba, do, DO, Jr., i, van, y} holding at least one of Prof., St., King., then nothing or `, MD`, as written, upper- and lower-cased (107,352 parses, measured 2026-09-26 on the narrowed tree; the intermediate tree above read 2,228): 588 texts that violated it at e0f1a2fa no longer do, and 12 newly do, all `Dr. nee Smith i MA|V` followed by `Prof.`, `King.` or `St.`, with or without `, MD`. Each is the title-carrying twin of `Dr. nee Smith i MA` / `Dr. nee Smith i V`, which read family 'i' at e0f1a2fa already: the bare-title head leaves no name word, the unguarded stop's class (#548), which the title now reaches because it leaves the clause. ACCEPTED: `Jane Doe (nee Smith Prof.)` reads title 'Prof.', maiden 'Smith' — bracket content ending in a period is suffix-shaped (S1), so the brackets are dropped and the clause is read bare, outside the reach of the delimiter precedence; rules/M2 carries it as an Accepted example. MEASURED 2026-09-26, the tree against its parent d9d80492, over three generated grids, and these are dated snapshots. Recipe: grid A is each head in {`Doe, Jane`, `Doe, J.`, `Doe, Prof.`, `Berg, abdul`, `Doe, Jane van`, `Jane Doe`, `J. Doe`, `Dr.`, `abdul`, `Jane van der Berg`, `John`}, then ` nee `, then every sequence of one to three words (repetition allowed) from {Smith, Jones, Prof., Dr., Sir, MA, DO, PhD, V, III, i, do, van, Ma, M.A., King., Rev., ba}, then either nothing or `, PhD`, each text as written, upper-cased and lower-cased, duplicates removed — 327,936 texts; grid B, aimed at the bound-given join, is the same construction over heads {`Dr. abdul`, `abdul rahman`, `Dr. abdul rahman`, `Berg, Dr. abdul`, `Berg, abdul rahman`, `abu`, `Berg, abu`, `Mr. abu`, `Dr.`, `Berg, Dr.`, `abdul`, `Berg, abdul`}, words {Smith, Prof., MA, V, do, van, PhD, Ma, ba, III, Jr., M.A.} and tails {nothing, `, PhD`, `, Jr., MD`}, plus these fourteen texts, as written only: `Jane Doe nee Ph. D. Prof.`, `Jane Doe nee Ph. D. Smith Prof.`, `Jane Doe z domu King. ba`, `Jane Doe z domu Smith Prof.`, `Jane Doe nee King. Prof. ba`, `Jane Doe nee King. Prof.`, `Doe, Jane nee Smith V,`, `Doe, Jane nee Smith V, ` (with the trailing space), `Doe, Jane, nee Smith V`, `Doe, Jane nee Smith V Prof., PhD`, `Doe, Jane nee Smith Prof. V, PhD`, `Doe, Jane nee Smith MA, PhD`, `Doe, Jane nee Smith V (Jr.)`, `Doe, Jane nee Smith V "Bo"` — 173,057 texts. The invariant tested is M2's: in a parse with a non-empty maiden field, every token written after the ` nee ` marker is roled maiden, title or suffix (so the two `z domu` texts are parsed and not checked). Grid C, a four-word probe of the title chain after a family comma, is each head in {`Doe, Jane`, `Jane Doe`, `Doe, J.`, `John Smith`, `Doe, Jane Mary`}, then ` nee `, then every sequence of four words (repetition allowed) from {Smith, Jones, Prof., Dr., King., MA, PhD, V, Jr., ba, do}, then either nothing or `, PhD`, as written only — 146,410 texts. Per grid, the counts are: texts that newly violate the invariant at the tree, texts that violated it at the parent and no longer do, texts that violate it at both. Grid A: 0, 2,012 and 17,854; grid C: 0, 1,368 and 15,534; grid B: 0, 1,222 and 11,149 (all re-measured 2026-09-26 on the tree with the link and particle-title checks below, as narrowed; an intermediate tree whose link check fired on every refused link read grid A 0, 3,772 and 16,094, its extra 1,760 fixes being links the as-written reading refuses too, which are #548's class), one of those last being the check itself counting the quoted nickname in `Doe, Jane nee Smith V "Bo"`. In grids A and B, every other remaining violation has the clause ending immediately before a word of the unambiguous suffix vocabulary written the way the plain suffix-word stop takes it (PhD, III, M.A., Jr., and a lower-case i or v; never a bare capital, which reads as an initial), which is where that stop ends a clause without asking a release question. DEFERRED: the walk's plain suffix-word stop — the one ending the clause at the first suffix word, as distinct from the trailing numeral, credential and title stops — asks no release question at all, and where the words behind it cannot read as post-nominals they land in a name part: on degenerate heads the stop word itself does (`Dr. nee Smith PhD Prof.` reads family 'PhD', `Doe nee Smith Jr. Prof., Jane` family 'Doe Prof.'), and with an ordinary head the words behind it do (`Doe, Jane nee Smith PhD Smith` reads middle 'Smith'). Unchanged here, and what "the clause keeps it" should mean where the kept words would follow a credential the clause itself ended at is its own question; rules/M2 carries `Doe nee Smith Jr. Prof., Jane` as an Accepted example meanwhile. The trailing numeral's stop has the same gap before a family comma, where it is made over the peel alone with no release question: `Doe nee Smith V, Jane` reads family 'Doe V', maiden 'Smith' (so did 2.2.0, 2.3.0 and the parent d9d80492; 2.0.0 and 2.1.0 read maiden 'Smith V'), and rules/M2 carries it beside the other. Open: #548. COST, measured 2026-09-26 on CPython 3.11.16 as profiler call events in one `Parser.parse` after a warm-up parse, parent → tree: `John Smith` 171 unchanged, as are `Jane Doe Prof.` and `John Smith MA`; `Jane Doe nee Smith` 244 → 246; `Jane Doe nee Smith MA` 377 → 388; `Jane Doe nee Smith Prof.` 276 → 372, the one shape that now runs the chain and a release check it never ran. Recompute: count `sys.setprofile` call events around the second of two `Parser().parse` calls, with the tree and d9d80492 each first on `sys.path`. -- 2026-09-27 (Derek), #544 — S2'S COMPANY CLAUSE DOES NOT REACH ACROSS A CLAUSE, recorded as an Accepted boundary rather than repaired. `Jane Doe Jr. nee Smith Ma` keeps maiden 'Smith Ma' and reports it, the clause's own words standing between 'Jr.' and 'Ma', where the clause-less `Jane Doe Jr. Ma` reads suffix 'Jr. Ma'. tests/v2/test_properties.py's clause-agreement walk pins the pairs that differ for exactly this reason as its `anchored_head` class — 810 of its 20,412 pairs, recorded 2026-09-27, 0 with the anchor off. Inside the clause the company reads as it does anywhere: `Jane Doe nee Smith PhD MEng` ends the clause at 'PhD' and reads suffix 'PhD MEng', maiden 'Smith'. +- 2026-09-26 #535 — THE MAIDEN WALK READS THE TRAILING TITLE CHAIN, AND EVERY STOP ASKS ONE RELEASE QUESTION OF THE SPAN IT GIVES UP (Derek, 2026-09-26). This resolves the NOT TRANSPARENT HERE clause of the 2026-09-19 #533 entry above, which recorded `Jane Doe nee Smith MA Prof.` and `Jane Doe nee Smith Prof. MA` disagreeing and deferred the pair as one decision about what "trailing" means inside a clause. BOTH HALVES WERE TAKEN TOGETHER: a trailing title ends the clause (`Jane Doe nee Smith Prof.` reads title 'Prof.', maiden 'Smith', where 2.3.0 read maiden 'Smith Prof.'), and a credential or numeral standing in front of that title gets the stop it gets with the title absent (`… Smith MA Prof.` gives suffix 'MA', `… Smith V Prof.` suffix 'V'). Only the first half would have left the pair disagreeing in the other direction; H5's transparency is a statement about the peel and the chain read together to their fixed point, so the walk now reads the end of the clause the way assign reads the end of the name — `tail_reading`, the shared predicate mechanisms.md#ONE-PREDICATE-PER-QUESTION names — and over the forms the transparency property test below exercises, the two spellings give one answer. They split where something in or ahead of the clause stands to take the title — for example a particle chain (ahead of the clause or inside it), a bound-given join, a name left with no name word, or, after a family comma, a title word in front of a class member with a title behind it, which the given part's chain stops short of exactly as it does with no marker (`Doe, Jane nee Smith Rev. MA Prof.` reads maiden 'Smith Rev.', title 'Prof.', suffix 'MA', where `… Rev. Prof. MA` reads title 'Rev. Prof.', maiden 'Smith', suffix 'MA'; bare `Doe, Jane Rev. MA Prof.` reads middle 'Rev.') — and the Accepted pairs recorded here and in rules/M2 are examples of that, not a complete list. `tests/v2/test_properties.py::test_a_trailing_title_is_transparent_to_the_maiden_clause` holds that as an invariant over the two inputs, over heads and runs where the credential is given up and nothing ahead would take the title, with its negative control at e0f1a2fa in its docstring. ACCEPTED, THE TRANSPARENCY BOUNDARY: where the clause KEEPS the credential the spellings differ, because a clause is one contiguous run and a title inside the kept text cannot leave without the words behind it — `Doe nee Smith ba Prof.` reads title 'Prof.', maiden 'Smith ba', and `Doe nee Smith Prof. ba` maiden 'Smith Prof. ba'; `abdul nee Smith MA Prof.` reads title 'Prof.', family 'abdul', maiden 'Smith MA', and `abdul nee Smith Prof. MA` given 'abdul', maiden 'Smith Prof. MA' (2.3.0 kept every word in all four). The report differs with them: `Doe, Dr. nee Smith MA` and `… Prof. MA` report `suffix-or-name` on the kept MA, `… MA Prof.` reports nothing, the kept member no longer ENDING the clause, which is what M2's report is asked of. rules/M2 carries the first pair as an Accepted example. ACCEPTED, FURTHER SPLITS, measured 2026-09-26. Where something ahead of the clause would take the title, the spellings differ even with the credential given up in both, because the first-suffix-word stop ends the clause before the title check runs and the title check alone declines: `Jane van der Berg nee Smith PhD Prof.` reads title 'Prof.', maiden 'Smith', suffix 'PhD', and `… Prof. PhD` maiden 'Smith Prof.', suffix 'PhD' — H5's accepted particle-chain boundary inherited, the bare `Jane van der Berg Smith PhD Prof.` / `… Prof. PhD` splitting the same way — and `Berg, abdul nee Smith PhD Prof.` (bound-given join) and `Doe, Dr. nee Smith PhD Prof.` (no name word left) split likewise; the parent d9d80492 and 2.3.0 read all of these as the tree does, so only the claim is new. And a released particle with a title behind it is withdrawn: `Jane Doe nee Smith DO Prof.` reads title 'Prof.', maiden 'Smith DO', where `… Prof. DO` reads title 'Prof.', maiden 'Smith', suffix 'DO' (2.3.0 maiden 'Smith DO Prof.' and 'Smith Prof. DO'); `do` likewise. A particle INSIDE the clause does it from the other side, its chain able to take the credential behind it: `Jane Doe nee Smith do MA Prof.` reads title 'Prof.', maiden 'Smith do MA', and `… do Prof. MA` title 'Prof.', maiden 'Smith do', suffix 'MA' (2.3.0 kept every word in both). rules/M2 carries a pair of each shape as Accepted examples. ACCEPTED, H5'S REACH: `Jane Doe nee Smith King.` reads title 'King.', maiden 'Smith' (2.3.0 maiden 'Smith King.'), H5's accepted `Mary Jane King.` cost now reaching the last word of a birth name as it reaches the last word of a current one; rules/M2 carries it as an Accepted example. READER SCOPE IS UNCHANGED: the chain is consulted only where a trailing rule reads the clause's words (no comma, the part before a suffix comma, the given part after a family comma); before a family comma and in a tail segment the walk reads the peel alone and the clause keeps the title (`Doe nee Smith Prof., Jane` keeps maiden 'Smith Prof.'). THE FIRST-WORD FLOOR MOVED INTO THE CHAIN. `trailing_titles` and `tail_reading` take a `floor` (1 everywhere else), and the walk passes the position just past the marker's first word, so the chain never takes that word: `Jane Doe nee King.` keeps its maiden name (TITLES holds borne surnames and the marker announced one), and `Jane Doe nee Prof. Dr.` keeps 'Prof.' and gives up 'Dr.'. The first draft applied the floor as a clamp on the title stop after the chain had run, and that is measurably wrong rather than merely inelegant: a chain allowed to take the first word has already spliced it out of the count the re-peel reads, so `Jane Doe nee King. ba` kept 'ba' in the clause where `Jane Doe nee Smith ba` gives it up. `test_a_title_first_word_counts_as_a_word` is the invariant (a title-vocabulary first word against an ordinary one, the credential behind it read alike) and its docstring records the clamp's failure count. ONE RELEASE CHECK FOR THREE STOPS: the numeral, credential and title stops each asked `_release_reads_off`, per reader, whether the name the take would leave reads the whole span a stop gives up as titles or suffixes; #601 replaced the release question with the clause-free reading bound as roles. THE JOINS: a released span a P2 chain or the P5 join would absorb was withdrawn, release machinery #601 removed. THREE NUMERAL-STOP READINGS moved with it, each the release invariant at the numeral stop and mostly after a family comma (`Berg, abdul nee Smith V`, `Doe, Jane nee Smith V, PhD`, `Doe, Jane nee V, PhD`), where #601 made the marker an ordinary word. A LINK THE WALK STOPS AT asked the release question too, where the title chain refused a link the exception would have joined (`Jane Doe nee Smith i DO Prof.`), scoped by a 411,936-parse link grid (6,195 fixed, 0 new violations, measured 2026-09-26); release machinery #601 removed. A RELEASED TITLE THAT IS ALSO A PARTICLE was kept after a family comma (`Doe, Jane nee Smith St.`), P6 attaching it to the family; a given-part reach #601 removed. A FOURTH MEASUREMENT, of the M2 invariant over a twelve-head fuzz (107,352 parses) against e0f1a2fa: 588 texts fixed and 12 newly violating, all `Dr. nee Smith i MA|V` behind a title, #548's class. ACCEPTED: `Jane Doe (nee Smith Prof.)` reads title 'Prof.', maiden 'Smith' — bracket content ending in a period is suffix-shaped (S1), so the brackets are dropped and the clause is read bare, outside the reach of the delimiter precedence; rules/M2 carries it as an Accepted example. MEASURED 2026-09-26, the tree against its parent d9d80492, over three generated grids, and these are dated snapshots. Recipe: grid A is each head in {`Doe, Jane`, `Doe, J.`, `Doe, Prof.`, `Berg, abdul`, `Doe, Jane van`, `Jane Doe`, `J. Doe`, `Dr.`, `abdul`, `Jane van der Berg`, `John`}, then ` nee `, then every sequence of one to three words (repetition allowed) from {Smith, Jones, Prof., Dr., Sir, MA, DO, PhD, V, III, i, do, van, Ma, M.A., King., Rev., ba}, then either nothing or `, PhD`, each text as written, upper-cased and lower-cased, duplicates removed — 327,936 texts; grid B, aimed at the bound-given join, is the same construction over heads {`Dr. abdul`, `abdul rahman`, `Dr. abdul rahman`, `Berg, Dr. abdul`, `Berg, abdul rahman`, `abu`, `Berg, abu`, `Mr. abu`, `Dr.`, `Berg, Dr.`, `abdul`, `Berg, abdul`}, words {Smith, Prof., MA, V, do, van, PhD, Ma, ba, III, Jr., M.A.} and tails {nothing, `, PhD`, `, Jr., MD`}, plus these fourteen texts, as written only: `Jane Doe nee Ph. D. Prof.`, `Jane Doe nee Ph. D. Smith Prof.`, `Jane Doe z domu King. ba`, `Jane Doe z domu Smith Prof.`, `Jane Doe nee King. Prof. ba`, `Jane Doe nee King. Prof.`, `Doe, Jane nee Smith V,`, `Doe, Jane nee Smith V, ` (with the trailing space), `Doe, Jane, nee Smith V`, `Doe, Jane nee Smith V Prof., PhD`, `Doe, Jane nee Smith Prof. V, PhD`, `Doe, Jane nee Smith MA, PhD`, `Doe, Jane nee Smith V (Jr.)`, `Doe, Jane nee Smith V "Bo"` — 173,057 texts. The invariant tested is M2's: in a parse with a non-empty maiden field, every token written after the ` nee ` marker is roled maiden, title or suffix (so the two `z domu` texts are parsed and not checked). Grid C, a four-word probe of the title chain after a family comma, is each head in {`Doe, Jane`, `Jane Doe`, `Doe, J.`, `John Smith`, `Doe, Jane Mary`}, then ` nee `, then every sequence of four words (repetition allowed) from {Smith, Jones, Prof., Dr., King., MA, PhD, V, Jr., ba, do}, then either nothing or `, PhD`, as written only — 146,410 texts. Per grid (newly violating at the tree, fixed, violating at both), measured on the tree as narrowed: A 0, 2,012 and 17,854; B 0, 1,222 and 11,149, one of those last the check itself counting the quoted nickname in `Doe, Jane nee Smith V "Bo"`; C 0, 1,368 and 15,534. In grids A and B, every other remaining violation has the clause ending immediately before a word of the unambiguous suffix vocabulary written the way the plain suffix-word stop takes it (PhD, III, M.A., Jr., and a lower-case i or v; never a bare capital, which reads as an initial), which is where that stop ends a clause without asking a release question. DEFERRED: the walk's plain suffix-word stop — the one ending the clause at the first suffix word, as distinct from the trailing numeral, credential and title stops — asks no release question at all, and where the words behind it cannot read as post-nominals they land in a name part: on degenerate heads the stop word itself does (`Dr. nee Smith PhD Prof.` reads family 'PhD', `Doe nee Smith Jr. Prof., Jane` family 'Doe Prof.'), and with an ordinary head the words behind it do (`Doe, Jane nee Smith PhD Smith` reads middle 'Smith'). Unchanged here, and what "the clause keeps it" should mean where the kept words would follow a credential the clause itself ended at is its own question; rules/M2 carries `Doe nee Smith Jr. Prof., Jane` as an Accepted example meanwhile. The trailing numeral's stop has the same gap before a family comma, where it is made over the peel alone with no release question: `Doe nee Smith V, Jane` reads family 'Doe V', maiden 'Smith' (so did 2.2.0, 2.3.0 and the parent d9d80492; 2.0.0 and 2.1.0 read maiden 'Smith V'), and rules/M2 carries it beside the other. Open: #548. COST, `sys.setprofile` call events in one parse after a warm-up on CPython 3.11.16, d9d80492 → tree (PR #550): `John Smith`, `Jane Doe Prof.` and `John Smith MA` unchanged, `Jane Doe nee Smith` 244 → 246, `… MA` 377 → 388, `… Prof.` 276 → 372, the one shape that now runs the chain and a release check. +- 2026-09-27 (Derek), #544 — S2'S COMPANY CLAUSE DOES NOT REACH ACROSS A CLAUSE, recorded as an Accepted boundary rather than repaired. #601 superseded it: a marker behind a credential is an ordinary word now, and the credential's run takes the clause in (rules/M2's Accepted `Jane Doe Jr. nee Smith`). Inside the clause the company reads as it does anywhere: `Jane Doe nee Smith PhD MEng` ends the clause at 'PhD' and reads suffix 'PhD MEng', maiden 'Smith'. - 2026-10-04 (Derek), #601 — A MARKER COUNTS ONLY BEHIND A SURNAME, AND IT TAKES THE WORDS AFTER IT UP TO THE TRAILING RUN THE NAME WOULD END WITH IF THE CLAUSE WERE NOT WRITTEN. This SUPERSEDES the release machinery of the #533 and #535 entries above (the release question, the per-stop checks, the anchored-head boundary of the #544 entry) and the given-part and suffix-comma reach the 2026-07-03 rule had; the #399, #420, #434 and delimiter entries stand. The trigger was #548, `Dr. nee Smith PhD Prof.`, where 2.3.0 read family 'PhD', maiden 'Smith': a clause whose head was nothing but a title, and a release check asking whether each word it gave up could still read as a post-nominal. #548 was closed into this issue and #602 (S2's run) because both halves of its answer were rules rather than repairs. THE HEAD RULE (Derek): a marker is one only in the name before any comma or the surname part before a family comma, behind at least one name word past the leading title run, and not straight behind an unambiguous suffix word or a connective. Anywhere else it is an ordinary word — in the given part after a family comma (`Doe, Jane nee Smith` reads middle 'nee Smith', 1.4.0's reading) and in a part after a suffix comma (`Smith, John, PhD née Jones` reads suffix 'PhD née Jones', also 1.4.0's). Derek's framing was that a maiden marker always follows a family name, and that a marker following something the parser has decided is a title, suffix, particle or connective is not a marker. THE PARTICLE HALF WAS DROPPED BY MEASUREMENT: prototype variant 1 refused a marker behind particle vocabulary and so declined `Anh Do née Tran` and `Mai Le née Nguyen`, Do, Le and Van being particles that are also the commonest Vietnamese surnames; a particle counts as the surname (`Jane van nee Smith` reads family 'van', maiden 'Smith'). THE BOUND (Derek): the clause takes its words up to the run of post-nominals and titles that the trailing rules read at the end of the name with the clause removed, and the words it gives up take the roles that reading gives them — the TRAILING rule's reading of the clause-free name, not the full parse of it, so a head the full parse would read differently (H1's lone title, a P5 join) cannot change where the clause ends. The first word after the marker is always taken, the marker having announced a name — a lone roman numeral included, so `Jane Smith née V` reads maiden 'V' where the clause-free `Jane Smith V` reads suffix 'V', accepted rather than given a numeral exception (rules/M2's Accepted block) — unless it is an unambiguous suffix word, where the marker declines (`Jane Smith nee PhD` reads family 'nee', suffix 'PhD', as before). Before a family comma the clause keeps the 2026-07-03 stop at the first suffix word, since no trailing rule reads that part's end. VARIANTS, prototyped 2026-10-03 behind an environment switch on one tree (the numbers are in #601's comments): v2/v3 grew the clause until the existing release check accepted it and reached 0 invariant violations but inherited the check's caution about particles (`Anh Do geb. de la Vega PhD Prof.` kept 'PhD Prof.' in the maiden name); v4 read the run once over the name as written without binding it, and is the recorded negative control in tests/v2/test_properties.py (350 of the clause grid's 4,482 parses fail under it, measured 2026-10-04); v5/v6 bound that reading as roles but still counted the clause's words among the words to spare, so `John nee Smith ba` read 'ba' as a credential where `John ba` reads a name; v7, the clause-free reading bound as roles, was adopted. Against #535's grids v7 left 0 violations of the M2 invariant where the tree then had 15,586 (grid A, 327,936 texts) and 8,713 (grid B, 173,057), measured 2026-10-03 with an invariant check written to #535's recipe. WHAT THE PROTOTYPE MISSED, found implementing it: the numeral fork. A trailing roman numeral is a suffix only by its shape against the word before it, and v7 read the clause's last word without that test, so `John née Jones Smith VI` kept 'VI' in the maiden name; the take now asks `is_trailing_numeral_suffix` of the last piece, and cases.py's `a_numeral_the_clause_free_name_reads_ends_the_clause` pins it. And the head's title run is measured over the words before the marker, a lone title excepted: measured over the whole segment, `Lord Chancellor née Jones` counted 'Chancellor' as a title and refused the marker. TRIAGE, 2026-10-04: the prototype's 68 failing case rows (40 facade failures among them) were re-pinned as `fix(#601)` with their group named in the note, bar the head-rule bug above; tests/v2/test_parser.py's corpus-append invariant became two-sided (appending ` née X` moves no other field, OR the take records why it declined), with a named set for the suffixes that themselves hold a marker. DEGENERATE INPUT, NO READING DESIGNED FOR (Derek, 2026-10-04: garbage in, garbage out). Two names that read worse by eye moved, and they are recorded here so the move is not mistaken for a regression, not because either reading is wanted: `Berg, Jane van der nee Smith DO` reads middle 'van der nee Smith', family 'Berg' where 2.3.0 read family 'van der Berg', maiden 'Smith DO' — the marker in a given part being a word, the particles no longer trail the given name (P6); and `Jane Doe, PhD née Smith` reads given 'PhD', middle 'née Smith', family 'Jane Doe' where 2.3.0 read given 'Jane', family 'Doe', suffix 'PhD', maiden 'Smith', which 2.3.0 reached only through the clause taking the marker (the comma part opening with a credential is #603's shape). Measured 2026-10-04 against 2.3.0. COST: the take is one function and no release helper survives; the #533 review's comma-head grids could no longer reach a take at all and were replaced by counting heads (tests/v2/test_properties.py's `_maiden_clause_grid`, 4,482 parses). - 2026-10-04 #610 — THE MAIDEN TAKE'S CANDIDATE WORDS ARE ONE FUNCTION BESIDE THE PEEL, AND SHARING IT FIXED A GAP THE HAND COPY HAD. `_maiden_take` chose the clause words to read its clause-free view over with a filter of its own (`_clause_tail_word` plus a numeral clause), a hand copy of the peel's and the chain's admissions whose contract — a superset of what `tail_reading` takes — nothing stated or tested. It is now `_pieces.trailing_candidates`, written beside `peel_trailing`, and tests/v2/pipeline/test_pieces.py pins the superset at the take itself, on the unjoined pieces the take is handed, over a generated grid (37,044 clauses through `group`). The grid found the copy wrong twice. The numeral fork reads a shape-only roman numeral (`VI`, in no wordlist) as the WALK's last piece, and two things leave it last without standing at the end of the name: a period-marked title behind it, which the chain takes first, and the merged `Ph. D.`, which group flags and the walk never holds. The copy admitted the numeral only as the clause's last word, so `Jane Doe nee Smith VI Prof.` kept maiden 'Smith VI' where `John Smith VI Prof.` reads suffix 'VI', and `Jane Doe nee Smith VI Ph. D.` the same way. Both now end the clause at the numeral, as rules/M2's statement already said they should. Recorded negative control: the loop as #601 shipped it misses 432 of that grid's clauses, every one this shape. The test's first version read plain names through all of `group`, on pieces the joins had already merged, which review caught: there the old loop missed 288 names, and a first draft of the fix, covering the title half only, still missed 111 — the `Ph. D.` half, which that grid is what found. Against master 8a8459a8 over the 95,813-text equivalence grid the change moved 10 readings, all of this class. The same issue lifted P6's particle-tail walk into `_pieces.particle_tail`, read off roles, so post_rules' attachment and assign's given-part credential run (#602) ask one walk; that half moved none of the 95,813 readings. -- 2026-10-18, #731 — THE CANDIDATE TEST IS A SUPERSET WITH ONE STATED EXCEPTION. The #610 entry's contract — `trailing_candidates` a superset of what `tail_reading` takes — now has one exception: a numeral whose word in front reads as an initial is refused, since the clause keeps that word and the view would otherwise read the numeral behind a different one. And the statement names S10 among the trailing rules the take asks, the fork that reads a shape-only numeral being S10's and not S2's. History: decisions/T4's #731 entry. +- 2026-10-18, #731 — THE CANDIDATE TEST IS A SUPERSET WITH ONE STATED EXCEPTION. SUPERSEDED by 2026-10-18 #731 reverted (PR #733) — the entry below; the full text is in git and in PR #732. - 2026-10-18 (Derek), #731 reverted — the #731 entry above is superseded: `trailing_candidates` takes no case fact again and is a superset of what `tail_reading` takes, without the exception. Its statement keeps naming S10. History: decisions/T4's revert entry. diff --git a/docs/design/decisions/M3-marker-led-clause.md b/docs/design/decisions/M3-marker-led-clause.md index 65478fe9..59b952ed 100644 --- a/docs/design/decisions/M3-marker-led-clause.md +++ b/docs/design/decisions/M3-marker-led-clause.md @@ -5,7 +5,7 @@ - 2026-08-26 #335 — the condition is a word AFTER the marker, not the marker alone. The reason is M1's: a one-word clause keeps its word because Nee is an attested surname (Irish Ní/Nee, and a Chinese romanization), and a marker with nothing after it marks nothing. Be precise about what pins it, because the obvious answer is wrong: M1's own boundary line carries the maiden-parens annotation, so it asserts the CONFIGURED reading and passes either way — built and measured 2026-08-26, a len(words) > 0 predicate still gives it maiden "Nee". What the loosened predicate actually breaks, measured by mutation 2026-08-26, is M3's own boundary example ("Jane Smith (née)" → nickname "née") and tests/v2/cases.py::maiden_marked_clause_one_word_stays_a_nickname in both runners — three pytest failures — AND the 2.1.0 differential gate, which goes red with three unexplained names: "Jane Smith (Nee)", "Jane Smith (Nee) (Jones)" and "Jane Smith (née)". The gate is the stronger signal and the one to look at first. The middle name is why: loosened, it reads maiden "Nee" and nickname "Jones", splitting one bracket pair into the maiden field and the next into the nickname field, which is the direct contradiction of M1's "two enclosures read as one maiden name". Anyone re-litigating the second-word test should check those, not M1's line. So the real division of labour is NOT markerless versus marker-led, which is how it was first written into rules.md and had to be corrected at review: it is one word versus a marker plus a word. "Jane Smith (Nee)" is marker-led and still reads nickname "Nee" by default, maiden "Nee" only with the pair configured (both measured 2026-08-26). - 2026-08-26 #335 — accepted consequence: the word taken after the marker is not tested for being a name word, so M3 does not stop at a suffix word where M2's bare take does. "Jane Smith (née V)" reads maiden "V" while the bare "Jane Smith née V" reads suffix "V" and family "née" (both measured). The reason is structural rather than an oversight: M3 hands the WHOLE clause to Role.MAIDEN and the bracket is its right boundary, so the trailing-suffix rule never sees the word as trailing the name at all. Pinned as M3's Accepted line in rules.md, which puts "Jane Smith (née V)" in the differential corpus and classified in all three ledgers. - 2026-08-26 #335 — rules/M3 lists implemented: nameparser/_pipeline/_extract.py alone, and that is deliberate rather than an omission. M3 decides a REGION'S ROLE; the marker drop it describes ("the marker itself dropped, as M1 drops it") is M1's mechanism in _group.py, which cites M1 and is governed by it. Adding _group.py to M3's list would mean inventing an M3 citation there for behavior M1 already owns — and test_doc_citations requires implemented: to equal the set of modules that cite the rule, so the two would have to move together. Do not re-open this. -- 2026-08-26 #335 — the durable finding, and the one worth reading before touching either side: _maiden_marked asks a WHITESPACE-SPLIT, _normalize'd question of the clause's first word, while tokenize and classify ask a token-level one. _normalize strips a trailing period but not a comma, so 'née,' is not a marker to M3 while the tokenizer splits the comma off and still tags the token vocab:maiden-marker. That divergence is the only thing keeping the "role is not Role.MAIDEN" branch of _group.group's clause-scoped drop pass reachable at all: without it, a marker-led NICKNAME clause could no longer exist, since M3 would have converted every one to MAIDEN before group ran. Anyone "unifying" the two tests would orphan that filter silently. The pin is tests/v2/cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname, and it needs BOTH of its clauses — the drop pass is gated on the name holding a maiden region at all, so a marker-glued nickname clause standing alone leaves the branch unexercised (measured by mutation, 2026-08-26). The row that used to pin the filter, maiden_marker_delimited_beside_a_nickname_clause, lost the job to M3: both its clauses are marker-led, so both are maiden now and there is no contrast left in it. SUPERSEDED IN PART 2026-10-12 by #653: a clause's comma now stays on its word, so 'née,' is no marker to the tokenizer either and the comma row no longer reaches the branch; the divergence lives on through rules/T2's nakaguro, and the pin is cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname (see #N1's #653 bullet). +- 2026-08-26 #335 — the durable finding, and the one worth reading before touching either side: _maiden_marked asks a WHITESPACE-SPLIT, _normalize'd question of the clause's first word, while tokenize and classify ask a token-level one. _normalize strips a trailing period but not a comma, so 'née,' is not a marker to M3 while the tokenizer splits the comma off and still tags the token vocab:maiden-marker. That divergence is the only thing keeping the "role is not Role.MAIDEN" branch of _group.group's clause-scoped drop pass reachable at all: without it, a marker-led NICKNAME clause could no longer exist, since M3 would have converted every one to MAIDEN before group ran. Anyone "unifying" the two tests would orphan that filter silently. Its pin was tests/v2/cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname, with a maiden clause beside the nickname one because the drop pass is gated on the name holding a maiden region at all. The row that used to pin the filter, maiden_marker_delimited_beside_a_nickname_clause, lost the job to M3: both its clauses are marker-led, so both are maiden now and there is no contrast left in it. SUPERSEDED IN PART 2026-10-12 by #653: a clause's comma now stays on its word, so 'née,' is no marker to the tokenizer either and the comma row no longer reaches the branch; the divergence lives on through rules/T2's nakaguro, and the pin is cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname (see #N1's #653 bullet). - 2026-08-26 #335 — M3 makes the marker vocabulary act in a THIRD position. M2's claim acts on the word after a name word; M1's on the content of a configured pair; M3's now on the first word of any bracketed clause, under the default policy. #vocabulary-collisions C-i is keyed on the position a claim acts on, so extending where a rule acts re-asks C-i of the whole set it reads, and nothing in the process prompts that — the rule was designed, reviewed and approved without the set being re-examined. Review afterwards found one entry that fails C-i, the Czech abbreviation roz, and its disposition is the Excluded (MAIDEN_MARKERS) block in decisions/vocabulary-exclusions, which is keyed to the vocabulary set rather than to any rule: it was removed. Two things worth separating, since the finding surfaced during this work and is easy to file under it — the defect was M2's and predates #335 (bare "Rosalind Roz Smith" lost its family name on master), and M3's own boundary already excluded the common one-word "(Roz)" spelling. What belongs to M3 is only the widening, and the lesson is the general one: re-ask C-i of a vocabulary set whenever a rule extends where its claim acts. - 2026-08-26 #335 — what configuring maiden_delimiters still buys, since M3 removes the commonest reason to reach for it: markerless clauses, and one-word clauses. "Cherice J. (Johnson) Williams" is a real US convention and a corpus name, and nothing in that clause says maiden, so it stays a nickname by default — only a caller who knows their data can say otherwise. The knob is a declaration about the DATA; M3 is a reading of what a clause declares about itself. diff --git a/docs/design/decisions/P3-connectives.md b/docs/design/decisions/P3-connectives.md index 57277802..0ab23bb2 100644 --- a/docs/design/decisions/P3-connectives.md +++ b/docs/design/decisions/P3-connectives.md @@ -5,7 +5,7 @@ - 2026-08-21 #418 (PR #420) — the three-word carve-out counts the name that remains once the maiden name leaves. The count was taken while the marker and the maiden name were still pieces, so `juan y garcia nee jones` counted five, 'y' joined, and the family was empty once the clause left — given 'juan y garcia', family ''. Nine corpus names show it under an appended clause. Closed by order, not by an exclusion: the marker pass now runs before the count (decisions/M2, 2026-08-21), and the three-piece gate ahead of the count, which had the same exposure, is closed the same way. rules/P3 now says the count is of the name's own words. The question #397 and #411 left open — whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another — is untouched: this was never a membership question, the words are simply not in the name. - 2026-09-20 #397 — DECIDED AND SHIPPED: `i` joins `conjunctions` and `conjunctions_ambiguous` together, which is what decisions/P8's 2026-09-13 #383 + #479 entry said would happen and which half of it the conjunctions Open block in decisions/vocabulary-exclusions called contested. `parse("Josep Carod i Rovira")` gives family "Carod i Rovira" where every release from 1.4.0 through 2.3.0 gave middle "Carod i" with family "Rovira"; `parse("Josep Lluis Carod i Rovira")` gives middle "Lluis" with that same family; and `parse("Carod i Rovira, Josep")` gives family "Carod i Rovira" where they read family "Carod Rovira" and took the link into `suffix` as a generation marker (all three measured 2026-09-20 on the released 1.4.0, 2.0.0, 2.1.0, 2.2.0 and 2.3.0 wheels from a throwaway environment, this tree shadowing a wheel otherwise). - THE OFF SWITCH is `Lexicon.default().remove(conjunctions={"i"})`, and on the v1 surface deleting `i` from `C.conjunctions`; both restore all three readings above byte for byte, measured 2026-09-20. That is a PROPERTY rather than an anecdote, and it is this change's durable statement, and it takes THREE exemptions rather than two once case repair is included — the earlier wording of this sentence named two and the release note named none. A name's FIELDS equal its reading with the letter out of the vocabulary unless (1) a link joined between two name words or (2) the two readings disagree about the letter being an initial; its REPAIR, plain and forced, equals the off switch's unless one of those two holds or (3) the parse placed the letter among the NAME words as a connective, where R4 keeps it lower-case and the off switch — for which it is no connective at all — title-cases it: `parse("Dr. John i Smith").capitalized(force=True)` keeps 'i' against the off switch's 'Dr. John I Smith', and `Carod i Rovira` and `Josep i Rovira` are the same shape. Measured 2026-09-20 over the shipped grid's 28,448 class-letter rows: 1,462 take the join exemption, 9,296 the initial-reading one, and 2,228 more reach the third, which is INV7's alone. `tests/v2/test_properties.py::test_a_link_that_joins_nothing_changes_no_field` runs it over a 16,576-text grid paired with the configurations that can read each text differently — 66,752 rows, of which 28,448 carry a class letter and are parsed twice — and carries the recorded control: 1,480 failing parses at `c8550b64`, the commit the first review was written against, 154 at `dc3bdf9c`, and 0 here. (Quoted over the full cross product of texts × lexicons × policies the earlier numbers were 2,360 and a 12,880-name grid times eight parsers; the shipped grid is trimmed by SHAPE and grew again with the second review, so the test's own docstring is the live figure and this is a dated snapshot.) + THE OFF SWITCH is `Lexicon.default().remove(conjunctions={"i"})`, and on the v1 surface deleting `i` from `C.conjunctions`; both restore all three readings above byte for byte, measured 2026-09-20. That is a PROPERTY rather than an anecdote, and it is this change's durable statement, and it takes THREE exemptions rather than two once case repair is included — the earlier wording of this sentence named two and the release note named none. A name's FIELDS equal its reading with the letter out of the vocabulary unless (1) a link joined between two name words or (2) the two readings disagree about the letter being an initial; its REPAIR, plain and forced, equals the off switch's unless one of those two holds or (3) the parse placed the letter among the NAME words as a connective, where R4 keeps it lower-case and the off switch — for which it is no connective at all — title-cases it: `parse("Dr. John i Smith").capitalized(force=True)` keeps 'i' against the off switch's 'Dr. John I Smith', and `Carod i Rovira` and `Josep i Rovira` are the same shape. Measured 2026-09-20 (PR #536) over the shipped grid's 28,448 class-letter rows: 1,462 take the join exemption, 9,296 the initial-reading one and 2,228 more the third, which is INV7's alone; `tests/v2/test_properties.py::test_a_link_that_joins_nothing_changes_no_field` holds it, its recorded control 1,480 failing parses at `c8550b64`, 154 at `dc3bdf9c` and 0 here, and its docstring is the live figure. THE COUNT REACHES THE CLASS (Derek, 2026-09-20), which ANSWERS the C-ii question the 2026-08-21 #418 bullet above left open — "whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another". It does not: a single-letter connective counts as a name word toward P3's three-word carve-out whatever else the vocabulary says it is, and `i` is also the roman numeral. The alternative was rejected on its own arithmetic rather than on taste — counting the link as the generation it also spells lets a connective raise the bar for ITS OWN join, so "Josep Carod i Rovira", the shortest linked name and the commonest one, would be the only shape that failed to link while the four-word spellings joined. The arithmetic runs the other way too, which is why the count is scoped to where the word is PLACED TO JOIN: a connective joining nothing would otherwise LOWER the bar for an unrelated one, and "Carod y Rovira i" is that shape — it counted the trailing generation and let the "y" join. It reads given "Carod", middle "y", family "Rovira", suffix "i" here, the same FIELDS the parent gave — its initials moved, but under R3's rule rather than this one. ACCEPTED CONSEQUENCE OF REACHING THE CLASS, and the one row the design's reading tables show it touching: under `Lexicon.default().add(particles={"y"})`, `Juan Velasquez y Garcia` goes from given "Juan" / middle "Velasquez" / family "y Garcia" to given "Juan" / family "Velasquez y Garcia" — the custom lexicon now AGREES with the default one, which is the right direction — and its `family_base` becomes "Velasquez Garcia", a pre-existing R2 limit (`family_base` drops a particle wherever it stands, not only leading) surfaced rather than created here. Measured 2026-09-20 on this branch. WIDER THAN ONE ROW, and the second review measured how much: the design's reading tables show one name because they hold one of this shape, not because one moves. Over the review's 43,925-name generated grid, `add(particles={"y"})` moves a word the parent roled SUFFIX into a name part on 130 names BEYOND what the default lexicon already moves there — "ABDUL Y ROVIRA MA" goes from given "ABDUL" / family "Y ROVIRA" / suffix "MA" to given "ABDUL Y ROVIRA" / family "MA" — and 71 of those 130 still report `suffix-or-name` on the moved word, so the reading is marked rather than silent. Four more lose a title, all four the same name in its four spellings ("Jr. y Soler Puig MA" and its casings). The direction is the one the row above describes: the custom lexicon converges on the default one, the particle joining what the connective already joined. Measured 2026-09-20. diff --git a/docs/design/decisions/P5-bound-given-names.md b/docs/design/decisions/P5-bound-given-names.md index 2c6496e0..fef86ed0 100644 --- a/docs/design/decisions/P5-bound-given-names.md +++ b/docs/design/decisions/P5-bound-given-names.md @@ -25,7 +25,7 @@ AMENDED 2026-09-08 (#489, the #316/#489 bundle): the key is no longer the whole run. Both sites now match the run's folded key WHOLE or by its LAST WORD, and the invariant this entry rests on — that H1 and P5 cannot read one run two ways — is what the amendment preserves rather than what it costs, both sites still reading through one predicate beside `_title_key`. What moves is this entry's own example: `'mr sir'` IS a given-name-title run to both sites now, `sir` being its last word, so `mr sir abdul rahman` joins where the sentence above says it does not. The leaf test that pinned the old reading, test_group's `test_the_title_run_is_one_key_as_h1_reads_it`, is renamed `test_the_title_run_is_read_as_h1_reads_it`; it runs the GROUP stage alone, so what it pins is P5's side — that `mr sir abdul rahman` joins — and not the agreement between the two sites. The agreement rests on the shared predicate, `_run_addresses_by_given` beside `_title_key`, which is the one place either site asks the question, and it is ASSERTED end to end by `tests/v2/test_parser.py::test_the_p5_licence_and_h1_read_a_title_run_the_same_way`: eleven title runs, each parsed in both spellings, requiring `parse(f'{title} John').family == ''` and `parse(f'{title} abdul rahman').family == ''` to agree run by run. Nothing else in this entry changes: `Dr. abdul salam` still keeps given `abdul`, family `salam`, `dr` addressing by family. See #H1 for the measurement and the vocabulary scope. - Measured at all three baselines: `Sheik abdul salam` moves (given, family), and with it the `Sheik abdul salam Jr` rules example — the same move with the suffix standing — and no other corpus name's fields do; `Sheik Abu Bakar` returns to the fields it had at 1.4.0, 2.0.0 and 2.1.0 — a restoration, which the ledgers record as a non-event — with one v2-surface change: the PARTICLE_OR_GIVEN report it carried through 2.1 is gone, because the join that takes 'Bakar' is P5's rather than the chain's, and a bound word read as the bound word is not a fork. The Arabic-script spelling comes along: `الشيخ عبد الله` read given 'عبد', family 'الله' and now reads given 'عبد الله', which is the name. The previous #369 ledger rules at all three baselines classified the regression "because the cause is known and recorded, NOT because the reading is wanted"; they are rewritten for the fix, and the 1.4.0 one is what keeps `Sheik abdul salam` explained at all: it landed on the fields-only suffix-routing catch-all before #451 deleted that rule, and with no fields-only rule left in any ledger the name now arrives UNEXPLAINED without it — measured by driving `classify()` over the 1.4.0 ledger minus this rule, which returns None for the name and for the `Sheik abdul salam Jr` example. A stronger reason to keep the rule than the one this entry first gave: the alternative is a red gate, not a mislabel. + Measured at all three baselines (PR #422): `Sheik abdul salam` and the `Sheik abdul salam Jr` rules example move (given, family) and no other corpus name's fields do; `Sheik Abu Bakar` returns to its 1.4.0, 2.0.0 and 2.1.0 fields and loses the PARTICLE_OR_GIVEN report it carried through 2.1, a bound word read as the bound word not being a fork; `الشيخ عبد الله` reads given 'عبد الله' where it read given 'عبد', family 'الله'. Two things review added before merge. The licence lifts the reserve for two WORDS, so the piece it lets the join take must be a single word: a particle chain is one piece, but it is the family name P2 built, and without that clause `Sir abdul van der Berg` read given 'abdul van der Berg', family '' — where the untitled name keeps family 'van der Berg', and so does `Sir John van der Berg`. H1's own scope is "exactly one name word", and the licence now matches it. `Sheik abdul al Rahman` therefore stays given 'abdul', family 'al Rahman', the same limit the `abd` entry above records for the three-token spelling. `Sir abdul van der Berg` is a rules.md example and so sits in the rules corpus, where it reads byte-identical to every baseline — the gate witnesses the clause; `Sheik abdul al Rahman` is in no corpus, and the group test is its only witness. And P5 now states the precedence `Sheik Abu Bakar` only exercised: a bound word that is also a particle is read as the bound word, the join outranking P4's leading-position reading, with no fork reported. That is the reading `Abu Bakar Salim` has always had — P5's join and P4's "the words stay separate" were never reconciled in writing — and the two rules now point at each other. @@ -38,10 +38,10 @@ - 2026-08-22 #425 — the reserve runs assign's peel over the view the join would leave. #423 had the reserve re-derive assign's trailing peel by hand — the flagged-credential drop, the walk's start, the numeral fork's conditions, the preceding piece remapped to "as the join would leave it" — with each seam held by a pin, and the bare-ambiguous-acronym fork not mirrored at all: `abdul Smith Jr Ma` read given 'abdul Smith', family '', suffix 'Jr, Ma' where `John Smith Jr Ma` keeps family 'Smith', the whole residue of #401's shape on the main walk (every family-losing bound-word shape in the sweep below contains one). The altitude review's framing is taken: the peel is one function, `_peel_trailing` over the list `_peel_walk` builds (the walk's input defined once too, after the /simplify pass found it spelled at three sites), housed in group beside the piece predicates because assign imports those and group cannot import assign; assign calls it and reports what it returns (the numeral fork's piece, the acronym picks, in the order it always reported them — byte-identical over the suite and the three gates); and the reserve builds the VIEW merge() would leave — the pair one piece, its tags the union — and calls the same function. `BoundJoin`'s "the value IS the threshold" trick survives restated as post-join name pieces: STRICT 2, LENIENT 1. The remap and the `k == first_name_k` carve-out are gone, the joined piece being two tokens and never a suffix piece. - 2026-10-07 #614 — THE RESERVE READS NO VIEW: S2's trailing run is read once, after P3's joins and before the bound-given join is tried, and split off, so the words to spare are the name pieces it left, and the join can move no suffix reading because nothing in group reads the run again (rules/P5 amended; decisions/S2, this date). SUPERSEDES the two-view comparison, which survives only where the run is not read there (a run with a name piece inside it), and with it two pinned readings: `abdul J. V` and `Sir abdul J. V` now join, given 'abdul J.' with family 'V' -- unjoined the initial in front suppresses the numeral fork, and that is the one reading taken now, where the joined view had read the V as a suffix and declined; 1.4.0 read the same, 2.0 and 2.1 suffix 'V', 2.2 and 2.3 middle 'J.' -- and `Sheik abdul Ma` reports the pick the read weighed (`suffix-or-name`), superseding the #425 bullet's application of the #369 precedent to the pair's suffix fork: the read decides 'Ma' before the join, so the fork is a fork. #369's own precedent, a bound word that is also a particle read as the bound word with no PARTICLE_OR_GIVEN report, is untouched (`Sheik Abu Bakar` reports nothing). Accepted by Derek as a class with the rest of #614's moves. - One principle came out of writing the ledger that the view alone did not state. `abdul Smith Ma` had read given 'abdul Smith', family 'Ma' since 2.0, and the #423 entry called that the lossless shape; the 1.4.0 gate's catch-all count said otherwise — 1.4.0 reads first 'abdul', last 'Smith', suffix 'Ma', as `John Smith Ma` reads, and the fields-only catch-all had absorbed the diff since the example went in (the count is the tell, per the #401 entry). Unjoined, the acronym has words to spare and peels as a credential; joined, the view is two pieces and the fork keeps it as the family — the join was CHANGING what assign reads. So the rule is stated as P5 now has it: the join joins two name words into one and changes nothing else, which the code asks directly: the pieces the peel takes are the same either way (a count proxy, "the view leaves exactly one name word fewer", was the first spelling; the altitude review pointed out it held only because the peel is a tail walk, and the comparison states the rule). That comparison subsumes the unjoined-piece check #423 added for `abdul V` (unjoined the peel takes the V, joined nothing) and restores `abdul Smith Ma` to 1.4.0's reading; `abdul Smith Berg Ma` keeps its join, the acronym peeling either way. Every #423 pin holds. Measured: the 1.4.0 gate is byte-identical at the code change, before the examples went in — the three `Ma`-final examples are v1 parity, the catch-all back at 15 — and `abdul Smith Jr Ma` and `abdul Smith Ma` move against 2.0.0 and 2.1.0, classified there; no hand corpus has the shapes. The population the corpus cannot see, from the design-docs review's sweep of 67,860 constructed main-walk shapes (three bound words — abdul, abd, abu — × twelve words — Smith, Berg, Ed, J., Jr, M.A., Ma, PhD, V, X, 'Ph. D.', 'van der', none a title word — × tails of one to four; no comma shapes; a different twelve-word pool gives different counts, so the pool is the claim): before this change 2,160 lost the family and every one contained a bare ambiguous acronym (864 with further suffixes behind it, so "ending in" was an artefact of #423's shorter sweep); after it none does, and all 67,860 match the family and suffix their ordinary-given twin reads. 5,166 move against master in all — 3,006 where the acronym was the family and is now the suffix (`Sir abdul Smith Ma`), 2,160 where the family is restored. The title-word class below was outside that population and moves after a family comma too (`Berg, abdul Sir`); so was the licence class at the end of this entry. + One principle came out of writing the ledger that the view alone did not state. `abdul Smith Ma` had read given 'abdul Smith', family 'Ma' since 2.0, and the #423 entry called that the lossless shape; the 1.4.0 gate's catch-all count said otherwise — 1.4.0 reads first 'abdul', last 'Smith', suffix 'Ma', as `John Smith Ma` reads, and the fields-only catch-all had absorbed the diff since the example went in (the count is the tell, per the #401 entry). Unjoined, the acronym has words to spare and peels as a credential; joined, the view is two pieces and the fork keeps it as the family — the join was CHANGING what assign reads. So the rule is stated as P5 now has it: the join joins two name words into one and changes nothing else, which the code asks directly: the pieces the peel takes are the same either way (a count proxy, "the view leaves exactly one name word fewer", was the first spelling; the altitude review pointed out it held only because the peel is a tail walk, and the comparison states the rule). That comparison subsumes the unjoined-piece check #423 added for `abdul V` (unjoined the peel takes the V, joined nothing) and restores `abdul Smith Ma` to 1.4.0's reading; `abdul Smith Berg Ma` keeps its join, the acronym peeling either way. Every #423 pin holds. Measured (PR #426): the 1.4.0 gate byte-identical at the code change, the catch-all back at 15, and `abdul Smith Jr Ma` and `abdul Smith Ma` moving against 2.0.0 and 2.1.0; over the design-docs review's 67,860 constructed main-walk shapes (abdul, abd, abu × a twelve-word pool × tails of one to four, no comma shapes; the pool is the claim), 2,160 had lost the family, every one holding a bare ambiguous acronym, none does after, and 5,166 move against master in all. The title-word class below was outside that population and moves after a family comma too (`Berg, abdul Sir`); so was the licence class at the end of this entry. Three things the principle does not reach, recorded so nobody applies it there. The post-comma LENIENT path is untouched (AMENDED by #637, below: its decline now reads H5's trailing run where a title word follows the bound word), and NOT for the reason first written here ("its walk has no peel" — it reads a trailing numeral by the lenient last-of-two rule, as the #423 entry says): the family is fixed there and the pair is the given whatever follows, so the principle is not applied, and `Berg, abdul V` keeps given 'abdul V' though the V would be the suffix unjoined — 1.4.0 parity, and exactly the regression #423 measured when a mirror was applied there; P5 now carries it as Accepted. The vocabulary decline is the UNAMBIGUOUS vocabulary: a bare ambiguous acronym the peel does not take is a name word, so `abdul Ma Smith` reads given 'abdul Ma' (1.4.0 read middle 'Ma' — a 2.x difference no corpus witnesses, now an Accepted example). And the principle is about the suffix reading, not the report: `Sheik abdul Ma` reports no SUFFIX_OR_NAME where `Sheik John Ma` does, the #369 precedent — a bound word read as the bound word is not a fork. One more class moves, found by the code review and then measured against 1.4.0: the 2.x reserve had excluded TITLE pieces from its count — an unrecorded deviation from v1, whose join counted suffixes out and nothing else — and the shared peel, following assign, reads a mid-name title word as the name word it is. `Berg, abdul Sir` (given 'abdul', middle 'Sir' since 2.0) and `abdul Smith Sir` (declined since 2.0) read given 'abdul Sir' and given 'abdul Smith', family 'Sir' — 1.4.0's readings, both; `abdul Sir Smith Berg` never stopped reading given 'abdul Sir'. No corpus name has the shapes that move, and no rules example carries one — `abdul Sir Smith Berg`, P5's example of the reading, is byte-identical at every baseline, so the class is pinned in tests alone (the release log says so; the late review of this entry found the sentence here implying otherwise). The same review found the clause had taken `do`, `freiherr` and `st` from P6 — recorded under P6. What the review actually caught was the tag artefact that exclusion had been hiding: the conjunction merge derives a `title` piece tag for a chain such as "Sheikh and Ahmad", merge()'s union handed it to the pair, and assign peeled the pair as a leading title — `abdul Sheikh and Ahmad Bakar Smith` read title 'abdul Sheikh and Ahmad' on 2.0 and 2.1 (v1 read it as the given), and `abdul Sheikh and Ahmad Bakar`, which the exclusion had kept unjoined, would have too once it went. The bound join drops the tag, and the view drops it the same way — the pair is a given name whatever tag the word carried. One more class the comparison moves, found by the code review's classification of every residual master-to-branch diff (12 of 17,640 constructed names — a fourteen-word pool, bare and behind `Sir`, tails of two and three, 6 × (14² + 14³); the review did not record the pool — one shape; 360 of 360 over {Sir, Sheikh, Dame, Sheik, King, Father} × {abdul, abd, abu, umm} × {J., V, X, A., I} × {V, X, I}): behind a given-name title, a numeral after an initial-shaped piece. `Sir abdul J. V` read given 'abdul J.', suffix 'V' since #369 — the fork suppressed unjoined, fired joined, and the licence's threshold of one let the join through — and the comparison declines it: given 'abdul', middle 'J.', family 'V', exactly as `Sir John J. V`. Pinned at both levels; it was the only shape a looser comparison let through under mutation at the test review. #424 — P2's chain and M2's walk taking a trailing numeral — is the remaining asker of the same question and now has `_peel_trailing` to ask it of. - 2026-10-11 (Derek), #637 and #640 — THE BOUND JOIN DECLINES A TITLE WORD A TRAILING RULE READS AS A TITLE. P5 joins a bound given-name word to the word after it, and declined a maiden marker and a suffix piece; a title word it took, so where a trailing rule would have read that word as a title the joined piece was no title word to read. After a family comma the join reads no peel (the LENIENT reserve, #423), so `Berg, abdul Prof.` read given 'abdul Prof.' where `Berg, John Prof.` reads title 'Prof.' (#637, the gap the 2026-09-09 A1 entry recorded); in a part past the second comma after a suffix comma, where the join runs STRICT, `John Smith, Jr., abdul Dr. Smith` read suffix 'Jr., abdul Dr. Smith' where `John Smith, Jr., Berg Dr. Smith` reads title 'Dr.' (#640). Now the join declines a title word a trailing rule reads as a title: after a family comma, one H5's trailing run takes from the given part (`tail_reading`, asked only when a title word follows the bound word), and in a part past the second comma any title word, which C2 reads as a title. `Berg, abdul Prof.` → title 'Prof.', given 'abdul'; `John Smith, Jr., abdul Dr. Smith` → title 'Dr.'. DECIDED (Derek): this narrower clause (a′) rather than declining every title word (a). P5 states that a title word standing in the name is a name word and joins like one (H3), and (a) undid it: it failed six tests, a P5 rules example and the two tests named for that clause among them, and over the bound-given grid below moved 4,044 parses, 3,000 of them a join split with no title read (`(Bob), Abu Bishop` → given 'Abu', middle 'Bishop') beside the 1,044 that (a′) moves (six tests measured against 4455962f's suite). (a′) leaves every join on a title word inside the name standing -- `abdul Sir Smith Berg`, `Berg, abdul Prof. Ali`, `Smith, abdul Bishop`. - MEASURED 2026-10-11 against 4455962f (#639's merge), py3.11, `nameparser.__file__` asserted on each side. A bound-given grid -- `{head} {bound} {next} {rest}` for head in (none), `Dr.`, `Sir`, `Smith,`, `van der Berg,`, `,`, `(Bob),`, `Smith, John,`, `John Smith, Jr.,`, `John Smith, PhD,`, `John`, `Mr. and Mrs.`; bound in `abdul`, `Abu`, `abd al`; next in `Dr.`, `Prof.`, `Prof`, `Bishop`, `King`, `King.`, `Sir`, `St`, `Freiherr`, `Salam`, `Rahman`, `Lt.Gov.`, `Mr.`, `van`, `MA`, `J.`; rest in (none), `Smith`, `Ali Smith`, `Dr.`, `Smith Dr.`, `Jr.`, `MA`, `Smith, Jr.`, `Prof. Smith` -- 5,184 texts under the three orders, comparing the seven fields and every report: 1,044 parses move, every one a title word into the title field and no report moved alone -- 660 in a part past the second comma (`John Smith, Jr., Abu Bishop Ali Smith` → title 'Bishop', as C2 reads `John Smith, Jr., Bishop`), the rest after a family comma or an empty head (`(Bob), Abu King.` → title 'King.', given 'Abu'). #623's given-part grid (decisions/H5): 280 -- 260 the same class (`(Bob), abdul Dr.` → title 'Dr.', family 'abdul', H1's move behind a title), and 20 that split the join with no title read (`Smith, abdul Prof. V, Jr.` → given 'abdul', middle 'Prof. V'), the class below. Every corpus name and case text at 4455962f: only `Berg, abdul Prof.`, the row that had pinned the gap. The S2 stress grid, #624's title-particle grid and #625's particle grid: 0. The five gates exit 0, with `fix(#637)` and `fix(#640)` rules in each 2.x ledger; at 1.4.0 the broad `fix(comma-family) lone post-comma piece routes to suffix/title` rule explains both new examples. Frames: the reference rows do not move; a comma-less name with a bound given word pays nothing, the site asked before the title test (`abdul salam ahmed salem` 241), a family-comma one with no title behind it one for the title test (`Berg, abdul salam` 208 → 209), and one with a title word behind it after a family comma about ten to twenty for the H5 read (`Berg, abdul Bishop` 208 → 218, `Berg, abdul Prof.` 207 → 228), rare shapes no guard sees. Recorded negative controls (with the review's rows): with no title declined, 11 tests fail (the four #637/#640 rows on both surfaces, H5's example and two P5 examples); with the past-the-second-comma half alone removed, 5 (its two rows on both surfaces and one P5 example). + MEASURED 2026-10-11 against 4455962f (#639's merge; this change is PR #643), over a 5,184-text bound-given grid (twelve heads × `abdul`, `Abu`, `abd al` × sixteen next words × nine tails) under the three orders: 1,044 parses move, every one a title word into the title field, 660 of them in a part past the second comma; #623's given-part grid moves 280, 260 of the same class and 20 the declined join below; no corpus name or case text but `Berg, abdul Prof.`, and nothing on the S2, #624 or #625 grids; the five gates exit 0; frames unmoved on the reference rows, up to `Berg, abdul Prof.` 207 → 228 on rare shapes no guard sees; recorded negative controls, 11 failing tests with no title declined and 5 with the past-the-second-comma half removed. THE REVIEW (2026-10-11). The decline after a family comma asks H5's run as the comma-less name reads it (`tail_reading`), and the given part's own walk reads some words behind a title otherwise, the disagreement #623's entry (decisions/H5) records for the chain's bound: there the join is declined and no title is read. A shape-only numeral VI to X, or V before a third comma part, which the walk keeps a name word (`Smith, abdul Prof. VI` → given 'abdul', middle 'Prof. VI', where master and 1.4.0 read given 'abdul Prof.', middle 'VI'); a period-marked title that is also a particle, which P6 then attaches with its report (`Smith, abdul St.` → family 'St. Smith', particle-or-given); and a bound word that is also a particle before one (`Smith, Abu St.` → title 'St.', where `Smith, Zed St.` reads family 'St. Smith'). Every such shape is garbage; the first two read as the same name with an ordinary given word does. Recorded, not narrowed, as #623's disagreement was (Derek, #623), and rules/P5 states it as an Accepted clause; the review's sweep found nothing reading worse than master on a real name and no comma-less name moving. It also found two mechanisms no test held, now rows: the bare-title half past the second comma (`John Smith, Jr., abdul Sir Smith`) and the whole trailing run after a family comma rather than the last word (`Berg, abdul Prof. Dr.`). diff --git a/docs/design/decisions/P6-trailing-orphan-particle.md b/docs/design/decisions/P6-trailing-orphan-particle.md index 205a1d66..3c8a3e2f 100644 --- a/docs/design/decisions/P6-trailing-orphan-particle.md +++ b/docs/design/decisions/P6-trailing-orphan-particle.md @@ -2,7 +2,7 @@ - 2026-08-23 — `Jong, vd` is C1's now, not P6's: C1's listing-form clause (#296's bundle, rules/C8 since #728) reads a post-comma part that is nothing but suffix words as the credential run, and `vd` is suffix vocabulary, so the row reads family 'Jong', suffix 'vd' by statement (it read so before, by the lone-piece deviation). The #380 scope note below calling it deliberately unresolved is superseded for that one row; the P6 question — a tussenvoegsel with no given word left — is unchanged. - 2026-08-18 #379/#380 IMPLEMENTED — the rule shipped as stated, both `deviates:` markers out. Keyed on the token's VOCABULARY rather than its assigned role, which is what gives the S2 precedence its effect: assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma. The words-to-spare guard is a PIECE test, not a count — every trailing piece that is wholly particles attaches, and the run must leave a GIVEN word ahead of it, so "Nguyen, Van" keeps its only given word. Not "a name word": see the defect below, where that wider reading passed on family text P1 had just produced. MIDDLE was in the test briefly and came out — 0 hits over 740,552 instrumented guard sites, and structurally unreachable, since the only rule that leaves a MIDDLE with no GIVEN ahead of it is gated on state.order, which assign never records on the family-comma path P6 runs on. - Measured: ONE of 751 corpus names moves ("Vega, Juan de la"), at all three baselines, and it needed a new ledger rule in each. The LEDGER rule's regex — not the parse rule, which has no regex and fires on the whole particle vocabulary through the tag — is a PARTIAL copy of PARTICLES, deliberately: the words that actually end a Dutch, Iberian or German listing, not all 70. The ledger guards forced that narrowing: the first draft used the neighbours' bare comma regex, which claimed 215 names and reached two protected exclusions. A member `des` had to come out as well, and for a reason worth keeping straight: it is not in PARTICLES at all, so it could copy nothing, and the text it reached was the opening of "Designated Judge David A. Ezra" rather than a person's name. `_reaches_non_vocabulary("des", PARTICLES)` prints the FRAGMENT it matched, `['Des']`, not the name containing it — which is how the retracted wording arose in the first place. + Measured (PR #403): ONE of 751 corpus names moves ("Vega, Juan de la"), at all three baselines, and it needed a new ledger rule in each. Two defects found in review, both from the rule's neighbours rather than from its own logic. P1's fold runs EARLIER in the same function and retags all of segment 1 to FAMILY, so a words-to-spare test written as "some name role remains" passed on family text P1 had just produced — and the attachment then hoisted the particle in front of a base it never preceded ("Smith, de Mesnil van" → 'van Smith de Mesnil'). 1.4 gives 'de Mesnil van Smith' for that input and 2.1 gives 'Smith de Mesnil van', so neither the defect nor the fix restores 1.4 — what the fix restores is 2.1, the pre-comma-family reading having moved back in 2.0. The test has to name GIVEN, which is what the rule says. And a post-nominal sits BEHIND the tussenvoegsel in this listing, so the backward walk stopped on it and the attachment never fired: "Berg, Jan van Jr." and "Berg, Jan van, Jr." parsed two ways on nothing but a comma. The walk now steps over a trailing piece holding no name, unless it is itself particle vocabulary — `vd` arrives suffix-roled and IS the run. Fallout worth its own record: "Vega, Juan de la" was the input tests/test_initials.py used to build an ALL-PARTICLE middle name, and this rule takes that shape away from THAT input. It does not remove the shape — "Vega, Santa de la", "Berg, Van der" and "Nguyen, Van Le" all still produce one in the default order, because the trailing run swallows the given word too and the attachment then declines. The test is repointed at the last of those rather than deleted. Whether such a middle should exist at all is #402 — for a never-given particle it is #365's defect, and for an ambiguous one the parser reports it as a middle name and then declines to initial it, which is a miscount either way. @@ -33,7 +33,7 @@ Decided 2026-08-19 (was Open: [#380](https://github.com/derek73/python-nameparse The key is the READING and not the vocabulary, and the docs review caught a first draft that said otherwise ("a word that is also suffix vocabulary declines a post-nominal"). That wording is wrong for both of the words it most obviously describes. `do` is the third member of the particle ∩ suffix-acronym overlap and reports `particle-or-given`: it sits in the AMBIGUOUS acronym half, which had already left it a name word, so no credential reading was ever taken for the attachment to override. And a run of several words is read whole — `Berg, Jan de vd` reports NOTHING though `vd` is suffix vocabulary, because behind `de` the run was read as name words, where the lone `Berg, Jan vd` reports `suffix-or-name`. Both are pinned in tests/v2/cases.py; neither is in any corpus, so nothing else would have caught the wrong statement. The two arms are ORDERED rather than asserted disjoint, and that is a correction to the plan, which recorded them as disjoint by construction ("a suffix-vocabulary particle arrives suffix-roled and IS the whole run"). That premise is a property of the SHIPPED vocabulary, not of the rule: `vd` and `mc` are the only words in both the particle and the unambiguous suffix sets and neither is ambiguous particle vocabulary, so no shipped input reaches both arms — measured, 0 both-arm runs over 29,400 generated names (every ordered pair of the 70-word particle vocabulary in six family-comma shapes: bare, behind a given word, behind a title, before `Jr.`, before `MD`, and with the pair split across the comma). But a caller's `Lexicon` may put one word in both, and the `assert` the plan asked for DID fire on `Berg, Jan zz` under a Lexicon adding `zz` to `particles_ambiguous` and `suffix_acronyms`. `rules/A1` says parsing never fails on any input and names exactly one exception, a caller's segmenter; an `AssertionError` out of a stage would be a second. So the code decides instead: the suffix arm wins, because assign had read the word as a post-nominal and the name-word reading was never on the table for the attachment to decline. Pinned in tests/v2/pipeline/test_post_rules.py. The never-given half stays SILENT, and that is the substantive half of the decision. `Jong, Piet de` has no live alternative: `de` is not a name in any reading and is not suffix vocabulary either, so the attachment chose between one reading and nothing, which is not a fork. Reporting it would be "the code got here" keying, the failure this project already has on record (a fork reported for every ambiguous particle at once on `Dr. Van Jr.` — the count is left out on purpose, since decisions/vocabulary-collisions says to recompute before quoting it). - Measured over the 1094-name differential corpus: P6's attachment fires on 9 names, of which 5 take the ambiguous arm (`Beethoven, Ludwig van`, `Berg, Jan van der`, `Nguyen, Thi Van`, `Vega, Juan de la`, `der, y van`), 2 the suffix arm (`Berg, Jan vd`, `Berg, abdul vd`) and 2 stay silent (`Jong, Anke de`, `Jong, Piet de`). Seven names gain an ambiguity kind and no role field moves that was not already moving. Read that seven as evidence about the corpus rather than about the blast radius, the caution the 2026-08-16 entry above already gives for this rule: 304 of the 1094 names carry a comma and only 13 end in a particle, 11 of those being rows the #379/#380 arc added itself (this change added four corpus names and not one of them carries a comma, so the 11 is unchanged by it). The reach is a SHAPE — every family-comma listing ending in an ambiguous particle, which is to say ordinary Dutch, Flemish and unaccented Vietnamese listings — and a caller holding such records sees a report on all of them, not on seven. The LEDGER shape that follows from that is not the one the plan predicted, and the difference is worth recording because it will recur for any report added to a rule whose fields already move. A new `fields = ["_ambiguities"]` rule explains none of the seven: `classify` matches on a rule's `fields` being a SUPERSET of the whole diff, and against 2.0.0 and 2.1.0 these names diff in `middle`/`family` (or `family`/`suffix`) from #379 and #380 as well. So the repair is to WIDEN those two existing rules to `{middle, family, _ambiguities}` and `{family, suffix, _ambiguities}` in both ledgers, not to add a rule. Each widened rule still passes the #452 equality check, since the union over the names it explains is exactly what it declares — the two silent `Jong` names contribute `{middle, family}` and the five ambiguous ones contribute `_ambiguities` on top. 1.4.0 needs nothing at all, having no ambiguity surface to diff. Gate before and after: byte-identical output at all three baselines apart from the provenance header — 1094 names, 229/194/102 intentional, 0 unexplained, and no per-heading count moved. The seven names were already in the diff; only their field sets grew. + Measured over the 1094-name differential corpus (PR #460): P6's attachment fires on 9 names, 5 on the ambiguous arm, 2 on the suffix arm (`Berg, Jan vd`, `Berg, abdul vd`) and 2 silent (`Jong, Anke de`, `Jong, Piet de`), and seven gain an ambiguity kind with no role field moving that was not already moving; 304 of the 1094 carry a comma and only 13 end in a particle, 11 of them rows the #379/#380 arc added, so read the seven as evidence about the corpus rather than the blast radius — the reach is a SHAPE, every family-comma listing ending in an ambiguous particle, and a caller holding such records sees a report on all of them; the gate was byte-identical at all three baselines once the two #379/#380 ledger rules were widened to carry `_ambiguities`. The most visible consequence, and it is not the Vietnamese one: `Beethoven, Ludwig van` — the textbook-correct Dutch listing, read exactly right — now carries a report. That is honest rather than regrettable. The parser cannot separate it from `Nguyen, Thi Van`; the two are the same string shape over the same vocabulary, and a report on one is a report on the other. A caller who wants only the doubtful cases has nothing here to filter on, because there is nothing to filter on in the input either. - 2026-08-30 #467 (the no-comma site, and the given position given back) — a declared family-first order names the family as a comma does, so P6's attachment fires without a comma too. It is keyed on TWO things, and review showed the second is not implied by the first: the run must stand in a MIDDLE — the one position that means nothing for a particle, middles being further given names — and it must END the name. Together they name the order without asking it: the default order ends with the FAMILY and FAMILY_FIRST_GIVEN_LAST with the GIVEN name, so only FAMILY_FIRST can put a name's last word in a middle. This DECIDES the parenthesis the 2026-08-16 Scope bullet left open ("arguably a missing comma under a declared family-first order") and narrows that bullet to the default order. @@ -55,7 +55,7 @@ Decided 2026-08-19 (was Open: [#380](https://github.com/derek73/python-nameparse - 2026-09-18 #531 (cross-amendment) — THE PRECEDENCE OVER S2 GAINS ONE EXCEPTION, AND THE RUN WALK GAINS ONE STAND-DOWN CONDITION. An all-caps member of the AMBIGUOUS credential class, in a name written in more than one case, reads as the credential and this attachment stands down: `Doe, John DO` is suffix `DO` where `Doe, John Do` and `SMITH, JOHN DO` still attach. The argument, the accepted cost (`SMITH, JOHN DO` paired with `NASCIMENTO, EDSON ARANTES DO`) and the two carve-outs are in decisions/S5's 2026-09-18 `do` bullet; what belongs here is the CONDITION, which is narrowed to the ambiguous tag rather than to the suffix role: a suffix-roled single-token piece carrying that tag is not part of the run, while `vd` and `mc` — unambiguous suffix vocabulary, also particles, also suffix-roled — stay inside it and are unaffected. Keying on the role instead would have stood the attachment down for them too, and keying on nothing would have let it silently override the slot's reading (`Doe, John DO` read family `DO Doe` with the condition absent, verified). The 2026-08-18 entry's "assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma" is unchanged for `vd` and `mc` and now has this one exception for `do`. - 2026-10-02 (Derek), #573 — A word of both the particle and the unambiguous suffix vocabulary reads alike in every case at three sites, P6's third exception among them. The record moved to decisions/P9 with that exception (2026-10-19). -- 2026-10-06 #613 — THE COMMA SITE IS DECIDED IN THE PASS THAT READS THE GIVEN PART. #611 found P6 showing the decisive sign smaller: assign held back from #602's run exactly the particle tail post_rules would attach a stage later, by asking `particle_tail` over roles that meant "not yet read" there and "suffix" in post_rules. The attachment and its third exception (`_inside_a_credential_run`) moved into assign, right after the given-part walk, and `particle_tail` is asked ONCE, at the moment the walk reaches #602's run: the pieces in front of the run hold the walk's roles and the run's words none, the state in which its answer equals the one over the finished roles (argued piece by piece in its docstring). post_rules ran P1's leading fold before P6, so assign asks P1's condition itself (`is_lone_never_given_particle` on the given piece), which is what keeps `Smith, de Mesnil van` the fold's. H1 and M4, the other rules ahead of the old site, are gated on there being no family, which a family comma always has. The no-comma site (#467) stays in post_rules, since it runs over the order P1's family-first redistribution leaves. NOTHING MOVES, measured 2026-10-06 against the commit before it: the P6 grid (103,786 distinct texts: each of the heads `Berg, Jan`, `Berg, Jan Piet`, `BERG, JAN`, `Berg, Dr. Jan`, `Berg, de`, `Berg, van`, `Berg, abdul`, `Smith, de Mesnil`, `de la, Jan` and `Nguyen, Thi`, followed by every sequence of zero to three of `van de der vd VD mc do DO Do PhD Jr. MA Ma Prof. V la Piet`, with and without a trailing `, Jr.` -- under the default policy, `lenient_comma_suffixes=False`, a family-first order and `middle_as_family`; comparator every token's role and folded or unjoined tag, every ambiguity's kind, detail and token IN ORDER, and `initials()`) and the #613 harness's 80,423 texts are identical, the gate's output byte-identical at all five baselines but for the baseline path, and the suite unchanged. ONE SHAPE DID MOVE, and the PR review found it on a grid with an EMPTY part before the comma, a head no measurement above held: there no family is named, H1 and M4 are not gated off, and they had made the given word the family before the old site attached -- `, Mr. Jones vd` read title 'Mr.', family 'Jones', suffix 'vd', and assign attaching first read given 'Jones', family 'vd' (177 texts on that grid). Reading P6 strictly instead (no family named, no attachment) moved 1,302 texts against master, `, Jan van` among them, so the attachment behind an empty family part stays in post_rules, after H1 and M4, through the same helper; that grid now differs from b547c247 only in C1's decided classes. Frames drop on every family-comma name, post_rules' block having paid a second `particle_tail` and a generator-driven search for the given word on every one, tail or not, where assign now attaches only behind a non-empty tail: `Smith, John` 169 → 160, `Smith, John, Jr.` 213 → 204, `Doe, Jane nee Smith` 302 → 293, a no-comma name unchanged. +- 2026-10-06 #613 — THE COMMA SITE IS DECIDED IN THE PASS THAT READS THE GIVEN PART. #611 found P6 showing the decisive sign smaller: assign held back from #602's run exactly the particle tail post_rules would attach a stage later, by asking `particle_tail` over roles that meant "not yet read" there and "suffix" in post_rules. The attachment and its third exception (`_inside_a_credential_run`) moved into assign, right after the given-part walk, and `particle_tail` is asked ONCE, at the moment the walk reaches #602's run: the pieces in front of the run hold the walk's roles and the run's words none, the state in which its answer equals the one over the finished roles (argued piece by piece in its docstring). post_rules ran P1's leading fold before P6, so assign asks P1's condition itself (`is_lone_never_given_particle` on the given piece), which is what keeps `Smith, de Mesnil van` the fold's. H1 and M4, the other rules ahead of the old site, are gated on there being no family, which a family comma always has. The no-comma site (#467) stays in post_rules, since it runs over the order P1's family-first redistribution leaves. NOTHING MOVES, measured 2026-10-06 against the commit before it (PR #616): a 103,786-text P6 grid (ten family-comma heads behind which zero to three of seventeen particle, credential and name words, with and without a trailing `, Jr.`, under four policies, comparing every token's role and tag, every report in order and `initials()`) and the #613 harness's 80,423 texts are identical, the gate byte-identical at all five baselines and the suite unchanged. ONE SHAPE DID MOVE, and the PR review found it on a grid with an EMPTY part before the comma, a head no measurement above held: there no family is named, H1 and M4 are not gated off, and they had made the given word the family before the old site attached -- `, Mr. Jones vd` read title 'Mr.', family 'Jones', suffix 'vd', and assign attaching first read given 'Jones', family 'vd' (177 texts on that grid). Reading P6 strictly instead (no family named, no attachment) moved 1,302 texts against master, `, Jan van` among them, so the attachment behind an empty family part stays in post_rules, after H1 and M4, through the same helper; that grid now differs from b547c247 only in C1's decided classes. Frames drop on every family-comma name, post_rules' block having paid a second `particle_tail` and a generator-driven search for the given word on every one, tail or not, where assign now attaches only behind a non-empty tail: `Smith, John` 169 → 160, `Smith, John, Jr.` 213 → 204, `Doe, Jane nee Smith` 302 → 293, a no-comma name unchanged. - 2026-10-17 #700 — THE ELIDED PARTICLE AND THE HYPHEN-ATTACHED ARABIC ARTICLE ARE PARTICLE VOCABULARY, NEVER-GIVEN, AND THE RULE IS UNCHANGED. Library catalogs file `Paul d'Ivoi` as `Ivoi, Paul d'` and `Ḥusnī az-Zaʿīm` as `Zaʿīm, Ḥusnī az-`, the particle split off with its mark still on it; both read as a middle name (1.4.0 and 2.4 alike) because neither written form was in `PARTICLES`. P6 asks only the particle tag, so the fix is the vocabulary alone: no code moved. Three decisions, each taken by Derek on the #700 session's options. ELIDED SET: `d'`, `l'`, `dell'`, each also in its U+2019 spelling, and `de’` and `’t` beside the existing `de'` and `'t` (`’t` added in the PR review, a gap the test review found), since nothing folds the typographic apostrophe to the ASCII one; particles.py asserts at import that every elided entry has both spellings (a vocabulary build check since 2026-10-18, decisions/vocabulary-model). ARTICLES: a CLOSED list — `al-`, `ul-`, the article assimilated to each of the fourteen sun letters in ASCII and DMG spellings (`ar-`, `az-`, `aš-`, `ash-`, `aṭ-`, …; lām's being `al-` itself), and the Egyptian `el-` with its assimilations in ASCII spellings only (`es-`, `esh-`, `ed-`, …; no `eš-` or `eṣ-`) — all never-given, where the bare `al` stays ambiguous (Al Gore): written apart with its hyphen, the word is the article and nothing else. RENDERING: the attached family joins with a space like every particle, `d' Ivoi` and `al- Rashid`. Declined: a shape rule (a vowel, consonants, a hyphen) in place of the list. It would need a new predicate and the bare-word collision check the issue named, for a population the list already covers; the corpora attest seven article spellings (`al- az- aš- ar- es- as- aṭ-`), all in it — recompute: every whitespace- or comma-bounded token of `tools/differential/corpus*.jsonl` matching `\w{1,4}-` at a name's end. Not decided here: the GLUED forms (`Paul d'Ivoi`, `Harun al-Rashid`) stay one word with no particle split, out of #700's scope by its own statement. Hyphen-minus only: U+2010 and the non-breaking hyphen are not listed. The Scottish Gaelic `a’` (`Mac a’ Ghobhainn`, in corpus_cldr.jsonl) is mid-name and not an index-form particle, and was not added. Nor are the elisions no corpus holds (`dall'`, `nell'`, `degl'`): the list is closed, so such a spelling reads as a middle name until it is listed. diff --git a/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md b/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md index fa76b59b..0c7c7c03 100644 --- a/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md +++ b/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md @@ -6,7 +6,7 @@ WHERE THE WRITING SAYS NOTHING. A name written wholly in ONE case — all upper and all lower alike — carries no case evidence about any letter in it. The reading there comes from a new marker subset `Lexicon.conjunctions_ambiguous` ⊂ `conjunctions`, default `{e}`: a member reads as an INITIAL and reports the new `AmbiguityKind.CONJUNCTION_OR_INITIAL`, a non-member JOINS even written as a bare capital and reports nothing. That is the third instance of the ambiguous-subset pattern, after `particles_ambiguous` and `suffix_acronyms_ambiguous`, so no mechanisms.md entry is owed. ROW 1 OF #479 IS THE DEFECT, NOT ROW 4 (Derek, 2026-09-12). The one-case lowercase reading is the one that was wrong, not the one-case uppercase reading that looked wrong beside it. WHY `e` AND NOT `y`. A bare E initial is common (Edward, Elizabeth) and an "e" between two surnames is rare outside couple listings; a bare Y initial is rare and "y" between two surnames is the commonest Hispanic compound and this library's oldest fixture ("Velasquez y Garcia"). The 2026-08-18 LOCALE CONTROL bullet under #3-0-reevaluations already said that which letters a tradition wants joined is per-word POLICY rather than membership; this subset is that policy made configurable, which is the smaller half of what that bullet asked for and ships without locale packs. `i` is not here because it is not conjunction vocabulary at all yet; it ships in the subset when #397 adds it, a bare I initial being as common as a bare E. - MEASURED (2026-09-13, this branch, over the deduped `tools/differential/corpus*.jsonl` glob — 1174 distinct names). SEVENTEEN names are written in one case and carry a cased single-letter connective: that is the WHOLE population this fork can reach, and ten of them move something against 2.3.0 (a role, `initials()`, or the report). Recompute the population by keeping every corpus name whose space-joined text equals its own `upper()` or `lower()` and which holds a one-character token that is in `conjunctions` after `_lexicon._normalize` and whose `upper()` and `lower()` differ; recompute the movers by parsing each of them here and under a released wheel and diffing the seven role fields, `initials()` and the ambiguity kinds. Of the EIGHT such names the corpora already held before this PR added its own examples, not one moves a role — the three-word carve-out or an existing initial reading covers every one of them. The two role moves are both new example names, and they break 1.4.0 parity in OPPOSITE directions: `jose e maria santos` read given "jose e maria" at every release from 1.4.0 through 2.3.0 and now reads given "jose", middle "e maria"; `JUAN GARCIA Y LOPEZ` read middle "GARCIA Y" and now reads family "GARCIA Y LOPEZ" (measured on the released 1.4.0 and 2.3.0 wheels from a throwaway environment, 2026-09-13 — this tree shadows the wheel otherwise). The ledgers at all five baselines classify them. + MEASURED 2026-09-13 over the 1174 distinct names of the deduped `tools/differential/corpus*.jsonl` glob (PR #527, whose text of this entry carries the recompute recipe): SEVENTEEN are written in one case and carry a cased single-letter connective, the WHOLE population this fork can reach, and ten move something against 2.3.0; none of the eight held before the PR moves a role, and the two role moves are new example names breaking 1.4.0 parity in OPPOSITE directions — `jose e maria santos` now given "jose", middle "e maria", and `JUAN GARCIA Y LOPEZ` now family "GARCIA Y LOPEZ" (on the 1.4.0 and 2.3.0 wheels); the ledgers at all five baselines classify them. THE RULE IS PER TOKEN AT CLASSIFY TIME AND REACHES EVERY NAME LENGTH (Derek, 2026-09-13), the three-word shape included: `john e smith` reads middle "e" tagged as an initial, so `parse(...).initials()` gives "j. e. s." where 2.3.0 gave "j. s." and `capitalized()` gives "John E Smith" where 2.3.0 gave "John e Smith". P3's three-word carve-out is untouched — it already says a single letter in a short name is more likely an initial, and this is the same judgement extended to the letter's case — so no ROLE moves at three words and the visible change is in the two derived views. BOTH QUESTIONS ARE ASKED OF THE NAME'S OWN WORDS — the case class AND the fork, not just the three-word count. Tokens standing before the first maiden-marker run head and not already carrying a role from a delimited clause are the name; the rest are not. Found by measurement rather than by design: appending " née Jones" made an all-caps name mixed-case and flipped the fork, failing the repo's invariant that a maiden clause changes nothing else, and a delimited `"Y"` was being conjunction-tagged (so R4 lowercased it) while a delimited `"E"` reported. ACCEPTED edge, and it is WIDER than an early draft of this entry said (measured 2026-09-13, in review): a maiden marker's run and every word AFTER it leave the own-words span the moment classify tags the marker, and they stay outside whether or not the marker pass later DECLINES the marker and leaves it a word. So a declined marker does two things, not one. It still does not count toward the case class, which is visible only when the declined marker word is the only differently-cased token (`JUAN Y GARCIA née` joins where `JUAN Y GARCIA née Jones` keeps "Y" a name word) — that much the early draft had. But it also leaves any connective standing after it to the MIXED-CASE rule rather than to the one-case fork, so a bare Latin capital there is an initial and its lowercase spelling joins. That is the one shape in which a name's OWN words read differently in its two one-case spellings: `JUAN NÉE JR Y LOPEZ` reads given "JUAN", middle "NÉE JR Y" with its "Y" tagged `initial`, while `juan née jr y lopez` reads given "juan", middle "née", family "jr y lopez" with its "y" tagged `conjunction`. SCOPE THAT PHRASE TO THE OWN WORDS, because a clause's words are not covered by it and a re-review caught the over-reach: a letter inside a delimited or maiden clause is read as it always was, which still means a different TAG in the two spellings, and case repair shows it — measured 2026-09-13, `JANE VAN DER BERG NÉE Y JONES` capitalizes to `Jane van der Berg (Y Jones)` where `jane van der berg née y jones` gives `(y Jones)`, and `JUAN "Y" GARCIA LOPEZ` keeps nickname `"Y"` where the lowercase spelling keeps `"y"`. The FIELDS agree in both pairs; what differs is the repaired case of a clause's word, which is that clause's rules doing their job rather than this one's. ACCEPTED rather than repaired, on two grounds: classify cannot know what group will decline, and a clause's words are read by the clause's own rules — which is the same doctrine the own-words span states, applied to the one shape where it bites. rules/P8's Accepted clause (P3's until the 2026-10-19 split) carries the same wording and the same example pair. (2026-10-11, release sweep: #602's credential run now reads `JR Y LOPEZ` as suffixes in the capitals spelling, so the clause carries the pair `DR. NÉE JUAN Y LOPEZ` / `dr. née juan y lopez` instead, the marker declined behind a title the same way: middle "JUAN Y" with "Y" an initial, against family "juan y lopez".) REPORT SCOPE IS SUBSET MEMBERS ONLY. Both readings are live only there; "y" in a one-case name is not in doubt, and reporting it would be noise on every lowercase Hispanic record. Matches how `particles_ambiguous` and `suffix_acronyms_ambiguous` report, and matches AGENTS.md's "a kind is worth adding only if a reader would hesitate too". diff --git a/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md b/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md index 1624c25b..ce370a44 100644 --- a/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md +++ b/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md @@ -18,6 +18,6 @@ ROUND-TRIPPING is stable on the shapes #429 fixed, which is what that entry recorded as the one load-bearing consequence of leaving this open: `str()` of a fixed parse is a no-comma string, and the no-comma path put the comma back. `test_a_suffix_run_survives_a_render_and_a_reparse` in tests/v2/test_render.py asserts `parse(str(parse(s))).suffix == parse(s).suffix` over the three writings. NOT stable in general, and the general claim is the one to resist — measured 2026-09-06 by evaluating that same equality over every corpus name, at the parent tree and here: 35 of 1116 names failed at the parent and 23 of 1117 fail here, thirteen fixed and ONE newly failing. The one is `Smith, John PhD I.`, and it is this same artifact seen from the other side: it round-tripped only because the render emitted 'John Smith PhD, I.' with a comma the writer never typed, which the no-comma path read back as a run. The render writes 'John Smith PhD I.' now, and that string reads middle 'Smith PhD', family 'I.' at the parent and here alike — a bare trailing initial behind a name is name material, #430's own boundary — so the two spellings genuinely disagree and the comma had been hiding it. The 23 that remain are the standing class: `str()` is a rendering, not a canonical form, and no rule promises it re-parses. NOT FIXED, and named because this change makes it reachable on more inputs: `revise(n, suffix=n.suffix)` is not the identity on a space-joined run. `Parser.revise` classifies each value by a full sub-parse of that value alone — its own docstring records the limit, the value being "classified ON ITS OWN" — and a bare 'MD PhD' carries no comma for the entry rule to route by, so the two words come back as two entries and the field re-renders 'MD, PhD'. Pre-existing, and the same shape as the `str()` limit above rather than a new one. Measured 2026-09-06 over every corpus name, comparing `revise(parse(n), suffix=parse(n).suffix).suffix` against `parse(n).suffix` at the parent tree and here: 24 of the parent's 1116 names fail and 38 of the 1117 here do, none of the parent's 24 recovering, and the 14 added are exactly the 14 movers — the 13 pre-existing names plus the R1 example. Not this change's to fix: the entry rule reads the commas in a whole name, and what `revise` hands its sub-parse is a field, so closing it means deciding what a field value's separators mean, which is a `revise` question. Answered, and FIXED, by #511 — the next bullet. DECLINED, all four with the evidence: marking the boundary at the core-drop site (#437's MARK-DONT-STRIP shape) — `dropped` already holds the fact with its span, and a second recording of it is the duplication that mechanism exists to prevent, one level up; making the `"joined"` tag role-aware — within a piece it is role-blind and correct for every role, `Smith, Ph. D. Smith` giving `first_list == ['Ph. D.']`, and only the between-piece half was ever a suffix concept; a render-time span scan instead of the recorded tag — `_facade.__setstate__` and `ParsedName.replace()` synthesise span-less tokens, so an unpickled name has nothing to scan and the tag IS the entry structure a pickle carries; and fixing the no-comma path inside group's block as a third branch beside `tail` and `reading`, which is the shape #429 took. -- 2026-09-06 #511 — a suffix value handed to `revise()` derives its entries from its own commas, by the rule a whole name uses. `Parser.revise` sub-parsed each value and forced every harvested token to the named role AFTER the sub-parse had run R1's entry pass, which keys on Role.SUFFIX; a bare 'MD PhD' reads there as a title and a family name, so the pass joined nothing and the field rendered 'MD, PhD'. The fix runs the same pass again: `revise` now sub-parses to a ParseState, forces the role on every non-dropped token, and calls `suffix_entries` — the pass, lifted unedited out of post_rules' tail into a function the two callers share (mechanisms.md#ONE-PREDICATE-PER-QUESTION) — over the forced state, then assembles and harvests as before. So a comma in the value parts two credentials and a space joins them. TWO SPELLINGS of the lifted pass, and the reason is the call budget: a first draft had post_rules call the state-in/state-out wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06, tools/perf/call_count.py: 450 calls/name before the move, 451 with the in-place worker `_mark_suffix_entries` that post_rules now calls, 454 with the draft; the facade band tops at 455.9), so the worker writes in place as every other post rule does and the wrapper exists for the one caller with no token list of its own. NO `_run` HELPER for the same reason: a draft routed `parse()` through a private state builder shared with `revise`, and that cost one frame per parse on the hot path (py3.11, 2026-09-06: 415 calls/name against 414 without it, in a 402-418 band), so the four-line construction is spelled twice and `parse()` is untouched. MEASURED 2026-09-06 over the 1117 distinct names in `tools/differential/corpus*.jsonl`, comparing `revise(p, suffix=p.suffix).suffix` against `p.suffix` under the default parser for every name with a non-empty suffix (368 of the 1117): 38 differed before and 1 after; the differential gate is byte-identical at all four baselines, `revise` not being on the compare path. RECOMPUTE: parse each corpus name, skip an empty suffix, revise the parse with its own suffix, count the names whose suffix moved (the script is in the #511 issue body). THE ONE LEFT is '김민준씨, J.씨', and it is not entry structure: the whole-name parse keeps 'J.씨' one glued suffix token, suffix '씨, J.씨', while the sub-parse of the bare value '씨, J.씨' peels the honorific off the initial, so the revised field renders '씨, J. 씨' — right entries, the spurious comma of before ('씨, J., 씨') gone, one word read differently by the value's own parse than by the whole name's. That is the "classified ON ITS OWN" limit `revise`'s docstring has always recorded, and CJK honorific peeling is a W-rule question this change does not move; pinned in `test_revise_reads_a_glued_honorific_on_its_own`. SUPERSEDES the phd-merge acceptance of 'Ph., D.' on this path (its bullet says how). STALE TAGS, tried and backed out: a draft cleared the sub-parse's own "joined" with the forcing and re-derived it, because the pass only ADDS the tag and `revise(n, family="Jones MD PhD")` carries the sub-parse's between-piece suffix mark on 'PhD' onto a FAMILY token. Measured 2026-09-06 against `330ee55`, the clear also destroyed every WITHIN-piece mark on a non-suffix value — 'D.' of `revise(n, family="John Ph. D. Smith")` lost the merge mark the #436 bullet's DECLINED list calls role-blind and correct for every role — while on a ParsedName only the suffix string view reads "joined" (`_text_for`'s suffix_join gate) — the facade's `_list_for` heals it for every role, but no path puts a revised name into a HumanName, the v1 setters going through `replace()`, so wiring those setters onto `revise` is the change that would show the stale mark, as `last_list == ['Jones', 'MD PhD']` — `initials()` and every field string being identical at both trees for every shape measured. So the tags are kept minus FOLDED_TAG as before; the between-piece mark on a forced non-suffix role is a tag-only oddity that predates this change and stays, and for a suffix value nothing depends on the sub-parse's marks, every pair it joined sharing a bucket with no parting token and the pass setting it again; pinned in `test_revise_keeps_the_sub_parses_within_piece_mark`. Dropped tokens keep their role and tags through the forcing, because the pass filters them by index and assemble omits them. ONE MORE LIMIT, pinned in `test_revise_leaves_a_policy_delimiter_unparted_without_a_tail_segment`, and it is the "classified ON ITS OWN" limit again rather than a rule of revise's: a delimiter the policy names through `extra_suffix_delimiters` is dropped, and so parts entries, only on a segment after a comma that the reading of the words makes a tail, and a value with no comma of its own has none — under `Policy(extra_suffix_delimiters=frozenset({" - "}))` the whole name 'Doe, John, MD PhD - FACS' renders 'MD PhD, FACS' while `revise(n, suffix="MD PhD - FACS")` renders 'MD PhD - FACS', the dash surviving as a token and the forced role making it a suffix word; measured 2026-09-06, a value whose own words read with a tail segment does part ('John Doe, MD - FACS' revises to 'John Doe, MD, FACS') and one after a suffix comma does not ('MD, PhD - FACS' stays), which is how the value's words read as a name deciding it. The round-trip is unaffected, the whole-name view having already rendered that boundary as a comma. DECLINED: a list-valued `revise(p, suffix=["MD PhD", "FACS"])`, which widens the API and leaves the string path where it was; documenting the limit with pins alone, the docstring already recording it and #511's measurement showing it reachable on every space-joined run #436 produced; and a text-only read of the delimiter cores inside `revise`, a second rule for a case no round-trip reaches. Out of scope and left as is: `ParsedName.replace(suffix="MD, PhD")` renders 'MD,, PhD', `replace` whitespace-splitting by contract and the facade setters riding on it for v1 parity. +- 2026-09-06 #511 — a suffix value handed to `revise()` derives its entries from its own commas, by the rule a whole name uses. `Parser.revise` sub-parsed each value and forced every harvested token to the named role AFTER the sub-parse had run R1's entry pass, which keys on Role.SUFFIX; a bare 'MD PhD' reads there as a title and a family name, so the pass joined nothing and the field rendered 'MD, PhD'. The fix runs the same pass again: `revise` now sub-parses to a ParseState, forces the role on every non-dropped token, and calls `suffix_entries` — the pass, lifted unedited out of post_rules' tail into a function the two callers share (mechanisms.md#ONE-PREDICATE-PER-QUESTION) — over the forced state, then assembles and harvests as before. So a comma in the value parts two credentials and a space joins them. TWO SPELLINGS of the lifted pass, and the reason is the call budget: a first draft had post_rules call the state-in/state-out wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06, tools/perf/call_count.py: 450 calls/name before the move, 451 with the in-place worker `_mark_suffix_entries` that post_rules now calls, 454 with the draft; the facade band tops at 455.9), so the worker writes in place as every other post rule does and the wrapper exists for the one caller with no token list of its own. NO `_run` HELPER for the same reason: a draft routed `parse()` through a private state builder shared with `revise`, and that cost one frame per parse on the hot path (py3.11, 2026-09-06: 415 calls/name against 414 without it, in a 402-418 band), so the four-line construction is spelled twice and `parse()` is untouched. MEASURED 2026-09-06 over the 1117 distinct corpus names, revising each of the 368 with a non-empty suffix by its own suffix: 38 suffixes moved before and 1 after, and the differential gate was byte-identical at all four baselines (recipe in the #511 issue body). THE ONE LEFT is '김민준씨, J.씨', and it is not entry structure: the whole-name parse keeps 'J.씨' one glued suffix token, suffix '씨, J.씨', while the sub-parse of the bare value '씨, J.씨' peels the honorific off the initial, so the revised field renders '씨, J. 씨' — right entries, the spurious comma of before ('씨, J., 씨') gone, one word read differently by the value's own parse than by the whole name's. That is the "classified ON ITS OWN" limit `revise`'s docstring has always recorded, and CJK honorific peeling is a W-rule question this change does not move; pinned in `test_revise_reads_a_glued_honorific_on_its_own`. SUPERSEDES the phd-merge acceptance of 'Ph., D.' on this path (its bullet says how). STALE TAGS, tried and backed out: a draft cleared the sub-parse's own "joined" with the forcing and re-derived it, because the pass only ADDS the tag and `revise(n, family="Jones MD PhD")` carries the sub-parse's between-piece suffix mark on 'PhD' onto a FAMILY token. Measured 2026-09-06 against `330ee55`, the clear also destroyed every WITHIN-piece mark on a non-suffix value — 'D.' of `revise(n, family="John Ph. D. Smith")` lost the merge mark the #436 bullet's DECLINED list calls role-blind and correct for every role — while on a ParsedName only the suffix string view reads "joined" (`_text_for`'s suffix_join gate) — the facade's `_list_for` heals it for every role, but no path puts a revised name into a HumanName, the v1 setters going through `replace()`, so wiring those setters onto `revise` is the change that would show the stale mark, as `last_list == ['Jones', 'MD PhD']` — `initials()` and every field string being identical at both trees for every shape measured. So the tags are kept minus FOLDED_TAG as before; the between-piece mark on a forced non-suffix role is a tag-only oddity that predates this change and stays, and for a suffix value nothing depends on the sub-parse's marks, every pair it joined sharing a bucket with no parting token and the pass setting it again; pinned in `test_revise_keeps_the_sub_parses_within_piece_mark`. Dropped tokens keep their role and tags through the forcing, because the pass filters them by index and assemble omits them. ONE MORE LIMIT, pinned in `test_revise_leaves_a_policy_delimiter_unparted_without_a_tail_segment`, and it is the "classified ON ITS OWN" limit again rather than a rule of revise's: a delimiter the policy names through `extra_suffix_delimiters` is dropped, and so parts entries, only on a segment after a comma that the reading of the words makes a tail, and a value with no comma of its own has none — under `Policy(extra_suffix_delimiters=frozenset({" - "}))` the whole name 'Doe, John, MD PhD - FACS' renders 'MD PhD, FACS' while `revise(n, suffix="MD PhD - FACS")` renders 'MD PhD - FACS', the dash surviving as a token and the forced role making it a suffix word; measured 2026-09-06, a value whose own words read with a tail segment does part ('John Doe, MD - FACS' revises to 'John Doe, MD, FACS') and one after a suffix comma does not ('MD, PhD - FACS' stays), which is how the value's words read as a name deciding it. The round-trip is unaffected, the whole-name view having already rendered that boundary as a comma. DECLINED: a list-valued `revise(p, suffix=["MD PhD", "FACS"])`, which widens the API and leaves the string path where it was; documenting the limit with pins alone, the docstring already recording it and #511's measurement showing it reachable on every space-joined run #436 produced; and a text-only read of the delimiter cores inside `revise`, a second rule for a case no round-trip reaches. Out of scope and left as is: `ParsedName.replace(suffix="MD, PhD")` renders 'MD,, PhD', `replace` whitespace-splitting by contract and the facade setters riding on it for v1 parity. - 2026-10-18 #728 — MOVED FROM decisions/C1. These entries decide that one comma segment is one suffix entry, which is R1's entry pass rather than the comma decision, and were filed under decisions/C1 until #728; they moved unchanged but for the pointers the move broke, the measurement paragraphs, each cut to one line naming its population, counts, baseline and PR, and the superseded passages, each cut to its decision and what replaced it. - 2026-10-19 #760 — MOVED FROM the entry pass's code comment (`_post_rules._mark_suffix_entries`), the facts the entries above did not hold. ORDER: the pass runs after assign because it keys on Role.SUFFIX; the same span rule run role-blind would join the A and B of `John A B Smith` into one middle_list element, pinned by `test_the_pass_runs_after_roles_are_settled` in tests/v2/pipeline/test_post_rules.py. Within post_rules it runs last by convention, not by necessity: every retag there targets a name role (`_retag` is called with FAMILY, GIVEN, MIDDLE and `_name_positions`' return, which is those three, and the `role=Role.FAMILY` replaces of P6's no-comma attachment and O3's fold are FAMILY too), so no rule moves a token into or out of SUFFIX or {TITLE, NICKNAME, MAIDEN} and the predicate reads the same answer wherever in the stage it stands (measured 2026-09-06 by reading the stage's retag targets). THE UNGATED CORE READ: group drops a core only on a `tail` segment, while the pass reads `delimiter_cores` whole and asks by TEXT alone, so a dropped token whose text the policy names parts the run whatever dropped it. The gate is not needed: a core group dropped was on a tail segment by construction, and the one case the ungated read adds is a maiden marker the policy also lists — under `Policy(extra_suffix_delimiters=frozenset({" nee "}))`, `John Doe, MD nee Jones PhD` renders 'MD, PhD' where the default policy renders 'MD PhD' (measured 2026-09-06), the writer's configuration having named that text a separator. That example no longer reaches the case: re-measured 2026-10-19 while moving it here, `John Doe, MD nee Jones PhD` reads title 'MD', given 'nee', middle 'Jones', family 'John Doe' and suffix 'PhD' under both policies, the comma now read as a family comma, so the marker is never dropped and the pass sees one post-nominal; `Smith, MD nee Jones PhD`, the bare-marker example of the #436/#437 entry above, reads the same way. The witness today is the bracketed clause: `Smith, MD (nee Jones) PhD` renders 'MD PhD' under the default policy and 'MD, PhD' under that one, extract having dropped the marker as Role.MAIDEN and group never dropping it, so the ungated text read is what parts the run (re-measured 2026-10-19 in #760's review). The same input is the filter's example below. THE DROPPED-INDEX FILTER is belt-and-braces: cores leave `pieces` before assign and get no role, and the one dropped token that arrives roled is the MAIDEN one tokenize roles from an extracted clause (`Smith, MD (nee Jones) PhD` drops index 2, the marker, as Role.MAIDEN); the filter keeps the suffix list and the parting scan agreeing about what a dropped index is. diff --git a/docs/design/decisions/R3-initials-honor-the-fold.md b/docs/design/decisions/R3-initials-honor-the-fold.md index 530ea696..cbd0e071 100644 --- a/docs/design/decisions/R3-initials-honor-the-fold.md +++ b/docs/design/decisions/R3-initials-honor-the-fold.md @@ -6,7 +6,7 @@ R3's earlier history is under `decisions/R2`, which this entry does not repeat: WHAT THE MEASUREMENT ADDED, and it reframes the issue from a consistency fix to a parity restoration. v1 had this feature as `CONSTANTS.middle_name_as_last` (`_config_shim` maps it to `Policy.middle_as_family`) and had NO general `name_order`, so only the two default-order policies have a v1 reference at all. Measured on the released 1.4.0 wheel over the 1094-name deduped corpus, from a script run outside the worktree that asserts `nameparser.__version__` first: at the default policy 1 name moves and agrees with 1.4.0 neither before nor after; under `middle_as_family` 71 move, of which **54 come into exact 1.4.0 agreement and NOT ONE moves away from it**. Worked pair: `Doe, Dr. John A.` gives `J. A. D.` at 1.4.0, gave `J. D. A.` here, and gives `J. A. D.` again; `Brundridge, Contessa A` likewise `C. A. B.` / `C. B. A.` / `C. A. B.`. The facade had been giving the folded answer all along — `HumanName.initials()` reads `first_list`/`middle_list`/`last_list`, and `_facade._list_for` prepends the carriers exactly as `_text_for` does — so the core view was simultaneously out of step with the field beside it, with the facade above it, and with 1.4.0 behind it. A view that stopped honoring a record had been reproducing a v1 bug v1 did not have. THE 17 THAT AGREE WITH 1.4.0 NEITHER WAY, recorded because the obvious explanation is wrong — and the FIRST REPLACEMENT for it was wrong the same way, corrected by this branch's docs review before merge. It is not that their parse already differs: only 2 of the 17 parse differently from 1.4.0 under this policy, `Smith, John V.` (#432's dotted numeral) and `남궁민수 지훈` (the unspaced-hangul surname split and the script order, rules/W1). The other 15 have SEVEN BYTE-IDENTICAL FIELDS at 1.4.0 and here — `der, y van` INCLUDED, which an earlier draft of this bullet filed with the two above by carrying over a measurement taken at the OTHER policy, where its parse does differ. Of those 15, 14 differ from 1.4.0 only in v1's initials GRANULARITY — v1 initials one group per `*_list` element, so a run its parse joined into one element (`Velasquez y Garcia`, `abdul salam`) yields `V G.` where 2.x, one token per word, yields `V. G.` — and strip the periods and spaces and all 14 come into 1.4.0's letter ORDER after this change, none having been in it before (`JVGQ` → `JQVG`). The 15th is `der, y van`, and it is worth naming rather than lumping: its fields match 1.4.0 exactly and its initials do not, because 1.4.0 contributes NOTHING for a family that is all particles (`van der` → `y.`) where 2.x contributes its words. That is R3's own adjudicated clause and decisions/R2's territory — R3 carries `"Juan van der" → initials="J. v. d."` against 1.4.0's `J.` — so this one name is a divergence this document has already decided in favor of, not an unexplained residue. So the parity claim is stronger than "54 of 71 and the rest unexplained": 68 of the 71 come into 1.4.0's ordering, 54 of them exactly, one more is R2's decided divergence, and 2 have a parse difference that predates this change. BLAST RADIUS by policy, over the same corpus × three `name_order` values × `middle_as_family` off/on (6564 parses, 660 move). Default order: 1 / 71. FAMILY_FIRST: 1 / 294. FAMILY_FIRST_GIVEN_LAST: 1 / 292. STATE THE POPULATION BESIDE THE COUNT, per this file's own rule: the shape an order change can be observed in is a contributing folded token standing AFTER a contributing unfolded one of the same role, and at the default order the corpus holds exactly 1 such name without `middle_as_family` and 72 with it. So the movers are 1 of 1 and 71 of 72 — a near-total hit rate over a corpus that carries one instance of P6's half, not a blast radius near zero. (The one non-mover is `Chang, Andy C I`, where the folded `C` and the family `Chang` share a letter and the reorder is invisible in initials.) Nine corpus names carry the tag at all at the default order; the other eight have a single contributing family token, which makes the partition the identity. The `middle_as_family` half is where the reach is, which follows from the producers: O3 folds every middle word, while P6's attachment moves initials only where the attached run is READMITTED by the unjoined mark — a family that is all particles — so it reaches one corpus name in each order and the same one. RECOMPUTE, and the COMPARATOR is the half that has to be stated, because the obvious phrasing of it measures nothing: parse the `tools/differential/corpus*.jsonl` files deduped under each of the six policies and compare `initials()` against THE PRE-CHANGE RENDERING — the same contributing tokens left in WRITTEN order. What this sentence said until 2026-08-30 was "against a folded-first partition of `tokens_for(role)`", which is what `initials()` now IS: that recipe compares the shipped implementation with itself and yields 0 differing parses, the inert-measurement shape (mechanisms.md's Verification shapes section) written into the very sentence meant to prevent it, and the only recompute path the repository offered for docs/release_log.rst's counts as well. CONTRIBUTING is the other half of the recipe and it comes FIRST: apply this view's own skip filter — outside the given group a token tagged `particle` or `conjunction` contributes nothing unless it also carries UNJOINED_TAG — and compare the two orders only over what survives it. Leave the filter out and every skipped word reads as a difference instead: 1546 rather than 660 against written order, and 1018 rather than 0 against the folded partition. Re-measured this way on 2026-08-30 and reproducing the counts above: 660 of 6564 parses, 1 / 71 at the default order, 1 / 294 and 1 / 292 under the two family-first orders. - THE VERIFICATION LIMIT, and it is decisions/R4's, restated because a plausible reading of "one name moves at the default policy" is that the ledger owes it a rule. It does not, and could not: `tools/differential/compare.py` compares the seven role fields plus `_ambiguities`, and `initials()` is not among them — the same blindness recorded for `capitalized()` under R4. Measured rather than reasoned: the gate was run at all three baselines on the pre-change tree (exported with `git archive HEAD` into a scratch directory, so nothing in the working tree moved) and on the changed one, and the two outputs are byte-identical apart from the tree path and the baseline-worker temp path — 1094 corpus names, 229 / 194 / 102 intentional diffs, `unexplained: 0`, and 67 / 58 / 51 headings with identical per-heading counts, at 1.4.0 / 2.0.0 / 2.1.0. NO LEDGER RULE WAS ADDED, and none was needed. What stands in its place is `rules/R3`'s example line and `tests/v2/test_render.py`, both mutation-checked. Amended 2026-09-02 (#484): the blindness this paragraph rests on is retired — the gate compares `initials()` for roles-identical names now, so an initials-only change DOES owe a ledger rule. #408 still owes none, and for a different reason than the one recorded here: its movement is 1.4.0 parity, and the one corpus name it moves at the default order (`der, y van`) diffs on ROLES at all three baselines, under `fix(#379)`, so the pseudo-field never enters. The semantics are in `decisions/differential-ledger, the initials view`. + THE VERIFICATION LIMIT, and it is decisions/R4's, restated because a plausible reading of "one name moves at the default policy" is that the ledger owes it a rule. It does not, and could not: `tools/differential/compare.py` compares the seven role fields plus `_ambiguities`, and `initials()` is not among them — the same blindness recorded for `capitalized()` under R4. Measured rather than reasoned: the gate at 1.4.0 / 2.0.0 / 2.1.0 on the pre-change tree and on the changed one gave byte-identical outputs over 1094 corpus names (229 / 194 / 102 intentional diffs, `unexplained: 0`). NO LEDGER RULE WAS ADDED, and none was needed. What stands in its place is `rules/R3`'s example line and `tests/v2/test_render.py`, both mutation-checked. Amended 2026-09-02 (#484): the blindness this paragraph rests on is retired — the gate compares `initials()` for roles-identical names now, so an initials-only change DOES owe a ledger rule. #408 still owes none, and for a different reason than the one recorded here: its movement is 1.4.0 parity, and the one corpus name it moves at the default order (`der, y van`) diffs on ROLES at all three baselines, under `fix(#379)`, so the pseudo-field never enters. The semantics are in `decisions/differential-ledger, the initials view`. WHY R3 GAINED A SENTENCE rather than leaning on R1. R1 already says folded family words "render before the rest of the family wherever they stood in the string" — but its subject is "Every FIELD", and `initials()` is not a field: R3 calls it "this view", and it takes a format spec and two separators rather than being read as an attribute. So R1 does not reach it by its own words, and #408 is precisely what leaving that to inference costs. R3's new clause states the order in R3's own voice, carries the O3 example line (`"Hassan, Mohamad Ahmad Ali"` under `middle_as_family` → `initials="M. A. A. H."`, reusing R1's own input so `corpus_rules.jsonl` is unchanged at 241 names), and R1 and R3 now name each other in `interacts:` — the field says which order, the view says it follows the field. P6's half gets no example line, for the same reason R1 carries none: the only input that witnesses it is `der, y van`, whose PARSE is itself contested, so pinning its initials normatively would pin a value another open question can move. It is pinned in the unit test instead, which is where today's-behavior pins belong. THE PARTITION IS PER ROLE, not scoped to FAMILY, and this is a deliberate choice rather than a copied line. Both producers re-role to FAMILY, so FAMILY is the only role a parse can put the tag on today and the GIVEN and MIDDLE arms are unreachable. They are there because `_text_for` partitions for every role it renders and the two views must not diverge again — the same uniformity `_post_rules`' own UNJOINED_TAG loop takes for its three roles, "for uniformity with the rule, not because all three are observable". A producer that ever folded into another part would otherwise reopen #408 there with nothing to fail. `test_initials_folds_in_every_role_it_renders` pins it from a hand-built name, since no input string can. Mutation-checked four ways — scoping the partition to FAMILY, and skipping it for GIVEN, for MIDDLE, for FAMILY — and each fails that test. It took THREE drafts of the fixture to make that true, and the two misses are the same miss one role apart. The first carried two GIVEN tokens and ONE MIDDLE: a one-element partition is the identity, so the MIDDLE arm was asserted and unpinned, and skipping the partition for MIDDLE passed the entire suite. The second fixed MIDDLE and carried NO FAMILY token at all, in a test named for every role it renders — so skipping the partition for FAMILY passed this test, and was caught only by its siblings and by R3's example line, neither of which is about per-role application. A zero-element group is the identity too, and reads even less like a gap than a one-element one, which is why the second miss survived a review that had just named the first. Two tokens per role is what closes it, and the general form is carried as mechanisms.md#TWO-ELEMENT-GROUPS rather than left here: a test written to pin a partition, a sort or a dedup needs at least two elements in every group it claims to cover, or the claim rides on a no-op. - 2026-09-01 #462 — DONE, the facade's twin: `HumanName.initials()` dropped a dotted or bare-capital `E`/`Y` from the middle and family groups because `_process_initial` tested conjunction membership alone, where 1.4.0's `is_conjunction` was "in the set AND NOT `is_an_initial`". `Scott E. Werner` gave `S. W.` from 2.0.0 through 2.2.0 and gives `S. E. W.` again; `John E Smith` (bare capital) and `Juan Y. Garcia` likewise; `parse().initials()` never had the bug, since `_classify` tags `E.` an initial and the render reads the tag (mechanisms.md#RENDER-HONORS-THE-PARSE). The issue said the bare form was "still correctly dropped", which is true of lowercase `e` and false of bare capital `E` — 7 of the 14 corpus names that move are bare capitals. Restored with `_render._INITIAL`, v1's own `initial` shape, because the facade may import `_render` and not `_pipeline`; scoped to `_process_initial`, its only caller, so a future reader of `_is_conjunction` does not inherit a decision made for initials. Found by #484's pseudo-field at the 1.4.0 baseline, where it had sat as 14 unreported diffs; the gate could not see it before because the seven fields never moved. @@ -16,17 +16,18 @@ R3's earlier history is under `decisions/R2`, which this entry does not repeat: A SECOND STATED BREAK, FOUND AND FIXED IN REVIEW (2026-09-14): before #528, `_initials_lists` read `self.first_list`/`self.middle_list`/`self.last_list` — public properties a v1 subclass may override — and the move to `_list_tokens_for(member)` above dropped that: an overridden property was silently ignored by `initials()` alone, while `last_base` (via `_split_last`, which reads `self.last_list`), `surnames` (`self.middle_list + self.last_list`) and `given_names` (`self.first_list + self.middle_list`, which never reaches `last_list` at all) kept honoring it. Fixed by checking, per member, whether the class overrides that property (`getattr(type(self), f"{member}_list") is not getattr(HumanName, f"{member}_list")` — three class-attribute identity checks, nothing on the parse path) and, where it is overridden, taking the pre-#528 STRING path over the override's own strings instead of the token walk. Measured 2026-09-14 against `git show 338daf7:nameparser/_facade.py` (before #528): a three-property `Sub` (`first_list` uppercasing every element of `super().first_list`; `middle_list` dropping every element of `super().middle_list` that starts with `"X"`; `last_list` always returning `["Zorro"]`) on `"john Xavier smith"` gives "J. Z." both there and here — reading only the `last_list` override into the recipe predicts "j. X. Z." instead (measured 2026-09-14 with `last_list` alone overridden on this tree), since the upper-cased first and the filtered-out `Xavier` both feed the result too — where the unfixed tree gave "j. X. s."; a one-property `Sub` (only `last_list` overridden, same `["Zorro"]`) on `"john e smith"` gives "j. e. Z." — the un-overridden first/middle keep the (post-#528) token path's own answer, and only the overridden last takes the pre-#528 fallback. Honoring the override, rather than leaving it silently ignored, is why "This is the only break in #528" above still holds: a second silent break on a PUBLIC surface would have gone unnoticed by every existing test, unlike the private-method break above, which is loud by design. Pinned at `tests/v2/test_facade.py::test_initials_honor_an_overridden_list_property`. ACCEPTED COST, MEASURED (2026-09-13, this branch): for a word backed by a token the connective answer is fixed at parse time, as `capitalize()`'s already was. `C = Constants(); h = HumanName("juan y garcia", constants=C)` gives "j. g."; `C.conjunctions.remove("y")` leaves it "j. g." with no re-parse, where before #528 it gave "j. y. g." immediately; `h.full_name = "juan y garcia"` re-parses and gives "j. y. g.". Unpinned before this change and pinned by it, at `tests/v2/test_facade.py::test_initials_freeze_the_connective_answer_at_parse_time`. - ACCEPTED COST, THE SECOND ONE, and it is the no-parse paths rather than the configuration, THREE of them, not one: state restoration (pickle, `copy.copy`, `copy.deepcopy`) and the keyword constructor all skip the tokenizer, so every word they carry takes the vocabulary fallback instead of a parse's tag. `__setstate__` stamps every restored token `UNCLASSIFIED_TAG`, because a v1 pickle carries the `*_list` STRINGS and no tags, so a restored name is spliced text throughout; `copy.copy` and `copy.deepcopy` use the same `__getstate__`/`__setstate__` state hooks, so a copy takes the identical fallback (measured 2026-09-14). The THIRD producer, unrecorded until this review round: `HumanName(first=..., middle=..., last=...)` routes straight through the field-setter properties in `__init__` and never calls `_apply_full_name`, so its tokens are `UNCLASSIFIED_TAG` for the identical reason a restored pickle's are — not a new bug, a third route to a cost already accepted. `HumanName(first="john", middle="e", last="smith").initials()` gives "j. s." where `HumanName("john e smith").initials()` gives "j. e. s."; `HumanName(first="JUAN", middle="Y", last="GARCIA").initials()` gives "J. Y. G." where `HumanName("JUAN Y GARCIA").initials()` gives "J. G." (measured 2026-09-14). A round-tripped, copied or deep-copied `JUAN Y GARCIA` therefore gives "J. Y. G." where the live parse gives "J. G.", and a round-tripped, copied or deep-copied `john e smith` gives "j. s." where the live parse gives "j. e. s." (the pickle round trip measured 2026-09-13, copy and deepcopy measured 2026-09-14, all three giving the identical two values). `capitalize()` has disagreed with the live parse on exactly those two names for exactly that reason since the tag was introduced — "Juan Y Garcia" against "Juan y Garcia", "John e Smith" against "John E Smith", all four measured the same day — and the keyword constructor gives the identical disagreement, `HumanName(first="john", middle="e", last="smith").capitalize()` leaving "John e Smith" against the live parse's "John E Smith" on both this tree and 338daf7 (measured 2026-09-14) — so this is the existing shape reaching one more view rather than a new one, and the remedy for the state-restoration pair is the one `__setstate__`'s own comment gives: the restored state is a v1 blob, not a parse; the keyword constructor has no state to restore an argument from — the caller supplied fields, not a string, so there was never a tokenizer run to have tagged them. RECOMPUTE ACROSS THE CORPUS, ALL THREE PATHS TOGETHER: rebuild every one of the deduped `tools/differential/corpus*.jsonl` glob's 1174 distinct names from its own parse's seven keyword-constructible fields (`first`, `middle`, `last`, `title`, `suffix`, `nickname`, `maiden`) and diff `initials()` against the live parse — 1 of 1174 differs before #528 (`Ph. D., John`, the pre-existing merge the ONE CORPUS NAME paragraph below names, unrelated to this mechanism) and 7 now, the added six being the same six `initials()` movers named throughout this entry. Pinned at `tests/v2/test_facade.py::test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too`. - THE POPULATION, MEASURED 2026-09-14: the deduped `tools/differential/corpus*.jsonl` glob holds two line shapes — `{"name": ...}` objects in `corpus.jsonl` and `corpus_shapes.jsonl`, bare JSON strings in the other four — so reading both and deduping is what gives its 1174 distinct names; a dict-only read stops at 528. Over that glob, exactly 6 give a different `initials()` after a pickle round trip than the live parse, up from 0 before #528 — the same six the STATED BREAK paragraph above names: `john e smith`, `john e jones`, `jones, john e`, `jose e maria santos`, `JUAN GARCIA Y LOPEZ` and `JUAN Y GARCIA`. `copy.copy` and `copy.deepcopy` move the identical six over the same glob, confirming the state hooks are one mechanism rather than a pickle-specific accident. + ACCEPTED COST, THE SECOND ONE, and it is the no-parse paths rather than the configuration, THREE of them, not one: state restoration (pickle, `copy.copy`, `copy.deepcopy`) and the keyword constructor all skip the tokenizer, so every word they carry takes the vocabulary fallback instead of a parse's tag. `__setstate__` stamps every restored token `UNCLASSIFIED_TAG`, because a v1 pickle carries the `*_list` STRINGS and no tags, so a restored name is spliced text throughout; `copy.copy` and `copy.deepcopy` use the same `__getstate__`/`__setstate__` state hooks, so a copy takes the identical fallback (measured 2026-09-14). The THIRD producer, unrecorded until this review round: `HumanName(first=..., middle=..., last=...)` routes straight through the field-setter properties in `__init__` and never calls `_apply_full_name`, so its tokens are `UNCLASSIFIED_TAG` for the identical reason a restored pickle's are — not a new bug, a third route to a cost already accepted. `HumanName(first="john", middle="e", last="smith").initials()` gives "j. s." where `HumanName("john e smith").initials()` gives "j. e. s."; `HumanName(first="JUAN", middle="Y", last="GARCIA").initials()` gives "J. Y. G." where `HumanName("JUAN Y GARCIA").initials()` gives "J. G." (measured 2026-09-14). A round-tripped, copied or deep-copied `JUAN Y GARCIA` therefore gives "J. Y. G." where the live parse gives "J. G.", and a round-tripped, copied or deep-copied `john e smith` gives "j. s." where the live parse gives "j. e. s." (the pickle round trip measured 2026-09-13, copy and deepcopy measured 2026-09-14, all three giving the identical two values). `capitalize()` has disagreed with the live parse on exactly those two names for exactly that reason since the tag was introduced — "Juan Y Garcia" against "Juan y Garcia", "John e Smith" against "John E Smith", all four measured the same day — and the keyword constructor gives the identical disagreement, `HumanName(first="john", middle="e", last="smith").capitalize()` leaving "John e Smith" against the live parse's "John E Smith" on both this tree and 338daf7 (measured 2026-09-14) — so this is the existing shape reaching one more view rather than a new one, and the remedy for the state-restoration pair is the one `__setstate__`'s own comment gives: the restored state is a v1 blob, not a parse; the keyword constructor has no state to restore an argument from — the caller supplied fields, not a string, so there was never a tokenizer run to have tagged them. Over the deduped corpus glob's 1174 names rebuilt from their seven keyword-constructible fields, 1 differs from the live parse before #528 (`Ph. D., John`, below) and 7 after, the added six being this entry's six movers. Pinned at `tests/v2/test_facade.py::test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too`. + THE POPULATION, MEASURED 2026-09-14 over the deduped `tools/differential/corpus*.jsonl` glob read in both line shapes (1174 names; a dict-only read stops at 528): exactly the six movers above give a different `initials()` after a pickle round trip, `copy.copy` or `copy.deepcopy` than the live parse, up from 0 before #528. ONE CORPUS NAME STILL DIVERGES and it is not this rule's. Measured over the deduped `tools/differential/corpus*.jsonl` glob (1174 distinct names) before and after: seven names had the two views disagreeing, six moved here, and `Ph. D., John` remains — the facade merges "Ph." and "D." into ONE list element (v1's `fix_phd`) and renders "P D" with the separator and no inner delimiter, giving "J. P D." against the core's "J. P. D.". That is `fix(initials-per-word) the Ph. D. merge`, a 2.0.0 rendering change ledgered at 1.4.0 since #484, and #528 preserves the element boundaries exactly so it neither moves nor is absorbed. RECOMPUTE by parsing every name of the glob on both surfaces and diffing `initials()`; the before half needs the pre-#528 `_facade.py` and `_render.py` on the path, which `git show` writes into a scratch copy of the package — never a checkout in a shared worktree. - DIFFERENTIAL. The facade's `_initials` is compared at every baseline, 1.4.0 included, and the core's from 2.0.0; the pseudo-field enters a name's diff only where every role and every ambiguity kind agrees. So at 1.4.0 FOUR of the six take a new rule (`fix(#528) the facade's initials follow the parse's connective tags`) and two do not — `jose e maria santos` and `JUAN GARCIA Y LOPEZ` move roles against that baseline, and #383/#479's role rule explains them. At 2.0.0 through 2.2.0 nothing new: the e-names' `_ambiguities` diff keeps `_initials` out of their diff, and `JUAN Y GARCIA`'s single `_initials` row survives with the facade half of its two causes closed, which `fix(#462)`'s dated paragraph in `expected_since_2.0.0.toml` now says. At 2.3.0 the existing rule stands and both surfaces move together. All five gates re-run at 0 unexplained on 2026-09-13. + DIFFERENTIAL, re-run 2026-09-13 at 0 unexplained at all five baselines: at 1.4.0 four of the six took a new rule (`fix(#528) the facade's initials follow the parse's connective tags`) and two moved roles under #383/#479's rule; nothing new at 2.0.0 through 2.2.0; at 2.3.0 the existing rule stands. + THE FACADE'S FALLBACK CAN SHOW WHAT THE CORE'S CANNOT, accepted rather than repaired. Recorded in mechanisms.md#RENDER-HONORS-THE-PARSE until #759 moved it here (2026-10-19), being a decision rather than a mechanism. The shape is a caller-added conjunction written initial-shaped in a script with no initials (`太.`, `씨.`): classify reads it through `is_initial()`, whose #320 repertoire veto the fallback's copy of `_INITIAL` does not carry, layering forbidding the import (test_regex_sync pins the two copies to each other and to config). Before #528 case repair was the fallback's only reader and could show nothing, `lower()` and `capitalize()` giving the same string over a caseless script; the facade's `initials()` is a second reader and can, since a word the fallback calls a connective contributes no initial while one it does not contributes a letter. Measured 2026-09-13 under `Constants()` with `conjunctions.add("太")`: `HumanName("Wang Chen 太. Li").initials()` is "W. C. L.", classify tagging `太.` a conjunction in the family, while splicing the identical family text in, `h.last = "Chen 太. Li"`, gives "W. C. 太. L."; `capitalized(force=True)` shows nothing on either. Accepted because the fallback is reached only for text no parse read, a spliced field being the caller's own text, and the remedy is the documented crossing, which differs by view: `Parser.revise()` for the parsed name (rules/R3's Accepted clause), and for the facade, which has no `revise`, assigning `full_name` again, which restores "W. C. L." (measured the same day). The core `ParsedName.initials()` cannot show it, being handed no vocabulary. - 2026-09-20 #461 — DONE, and the clause is settled for every group at once. THE RULE AS TAKEN: a connective contributes nothing where it is JOINING — a part holding another name word for it to join, the part's working particles set aside — and where its part holds nothing else it is joining nothing, so it initials like any other name word and agrees with the base. That picks ONE of the four candidate answers the 2026-08-29 bullet under `decisions/R2` listed for "John and Jane Smith" and makes the other three unreachable: `parse("John and Jane Smith").initials()` is "J. J. S.", not the "J. a. J. S." of 2.0 through 2.3, not P3's one-name-word "J. S.", and not 1.4.0's run-together "J a J. S.". THE CRITERION IS NEVER A WORD COUNT, which is what keeps `parse("Jon Dough and")` at "J. D." over base "Dough and" and `parse("Juan Velasquez y Garcia")` at "J. V. G." — a part of two words where one is the connective is still a part where it joins, and a part of three is no different. BOTH VIEWS, ONE MARK. The fact is about a PART, so it is decided once where the parts are settled and recorded on the token (`UNJOINED_CONJUNCTION_TAG`, mechanisms.md#MARK-DONT-STRIP), and the core view and the v1 facade both read it. That is the direct answer to what the first #461 attempt found: a mark honored for some of a part's words and not for one of them is honoring nothing. A SEPARATE marker rather than a widening of `UNJOINED_TAG`, because that one is also read by `family_particles`/`family_base` and by `_cap_word`, and none of those three may move. `Parser.revise` recomputes both marks, the same reason `_remarked` recomputes the first. - BLAST RADIUS, MEASURED 2026-09-20 against the parent commit `46651750` over the deduped corpus-union-cases set under eight configurations, both processes asserting their own `nameparser.__file__`. RE-MEASURED 2026-09-20 at the end of the second review round, the population having grown again with its own rows. Over the population AS IT STANDS — 1568 non-empty names — the default order moves 12 names' roles, 3 more names' reports with the roles fixed, and 56 names' `initials()`, 22 gaining a letter and 34 losing one; `HumanName.initials()` moves on THE SAME 56 names with THE SAME values. Family-first moves 11 / 3 / 34 and `FAMILY_FIRST_GIVEN_LAST` 12 / 3 / 64. The twelve role movers at the default order are "Carod i Rovira, Josep", "Doe, Jane nee Puig i Soler", "Henry i of England", "Jane Doe nee Puig i Ma", "Jane Doe nee Puig i Soler", "Josep Carod i Rovira", "Josep Carod i Rovira III", "Josep Carod i de Rovira", "Josep Lluis Carod i Rovira", "john smith i jr", "maier, amy i, jr." and "rovira, i" — seven of the twelve are rows this bundle added, which is why the restricted figure below is the comparable one. Restricted to the 1543 names that stood in the corpus at the parent, the same sweep reads 2 / 3 / 46 at the default order, 18 gaining and 28 losing, with 1 / 3 / 24 family-first and 2 / 3 / 52 given-last — unmoved by the second review, whose report change touches only "JOHN QUINCY SMITH I", a name the round itself added. (The 1558-name sweep that stood here read 6 / 3 / 50 with 19 gaining and 31 losing, and 5 / 3 / 28 and 6 / 3 / 58 for the two family-first orders; it is a dated snapshot over a smaller population.) RECOMPUTE by parsing every name of the deduped `tools/differential/corpus*.jsonl` glob plus `tests/v2/cases.py`'s texts under those eight configurations in two processes, one on this tree and one on the parent's, and diffing the seven role fields, the ambiguity kinds and `initials()` on both surfaces. The gate cannot answer this on its own: its `_initials` pseudo-field sees only names whose roles stayed put, so a count taken from the classified summary is a FLOOR. + BLAST RADIUS, MEASURED 2026-09-20 (re-measured at the end of the second review round) against the parent `46651750` (PR #536) over the deduped corpus glob plus `tests/v2/cases.py`'s texts, 1568 non-empty names, under eight configurations: at the default order 12 names move roles, 3 more reports, and 56 `initials()` (22 gaining a letter, 34 losing one; `HumanName.initials()` the same 56), family-first 11 / 3 / 34 and `FAMILY_FIRST_GIVEN_LAST` 12 / 3 / 64; restricted to the 1543 names at the parent, 2 / 3 / 46, 1 / 3 / 24 and 2 / 3 / 52. The gate's `_initials` sees only names whose roles stayed put, so a count from its classified summary is a FLOOR. TWO OF THE GAINS ARE 1.4.0 PARITY RESTORED, and that is what retires two ledger rules: `JUAN Y GARCIA` gives "J. Y. G." and `محمد و علي` gives "م. و. ع.", which is what 1.4.0 gave for both. The two ranges differ and an earlier draft of this bullet ran them together, so state them apart (measured on all five wheels 2026-09-20): `محمد و علي` gave "م. ع." at 2.0.0, 2.1.0, 2.2.0 AND 2.3.0, while `JUAN Y GARCIA` gave "J. G." at 2.0.0, 2.1.0 and 2.2.0 and went back to 1.4.0's "J. Y. G." at 2.3.0 — the reason being the one the 2026-09-13 #383/#479 entry under P8 records, and the 2.4.0 release note says the same. The readmission is a parity RESTORATION on those two names rather than a deviation, and the given-group half in the other direction is the deviation — see rules/R3's Accepted clause, which states both. - THE FRAME BUDGET WENT DOWN: `tools/perf/call_count.py` reads `py3.11 parse=406.00 facade=443.00` where the parent read `412.00 / 449.00`. The two marks now come off ONE walk over the roles instead of two generator expressions, and on 3.11 a genexp costs a frame for its creation and one per resume, so the reference name's two non-empty parts pay exactly six. `_CALL_BASELINE[(3,11)]` is `{"parse": 410, "facade": 447}` with `_BAND = 0.02`, so the bands are [401.8, 418.2] and [438.06, 455.94] and both figures land inside: NO baseline edit and no `decisions/parse-cost` decision is owed. PEP 709 inlined comprehensions and not generator expressions, so 3.12 through 3.15 should drop by the same six. + THE FRAME BUDGET WENT DOWN: `tools/perf/call_count.py` reads `py3.11 parse=406.00 facade=443.00` against the parent's `412.00 / 449.00`, the two marks coming off ONE walk over the roles instead of two generator expressions; both land inside the `_CALL_BASELINE` bands, so no baseline edit and no `decisions/parse-cost` decision is owed. THE NO-PARSE PATHS KEEP THE OLDER READING, the same degradation the 2026-09-13 #528 bullet above accepted and for the same reason: a token the parse never saw carries no mark, so the view falls back to the vocabulary. Measured 2026-09-20, `HumanName("JUAN GARCIA Y LOPEZ").initials()` is "J. G. L." live and "J. G. Y. L." after `pickle`, `copy.copy` or `copy.deepcopy`; a name built from keyword fields carries no tags either, and so does one whose `*_list` property a subclass overrides. The remedy is the v1 one — assign `full_name` again — since the facade has no `revise`. - HOW FAR IT REACHES, over a STATED population rather than an example, because this bundle MULTIPLIES it and the earlier wording did not say by how much: over the deduped `tools/differential/corpus*.jsonl` glob plus `tests/v2/cases.py`'s texts — 1,568 non-empty names — the v1 `initials()` of a pickled, copied or deep-copied name differs from the live parse's on 34 names here against 6 at the parent 46651750, measured 2026-09-20 in two processes each asserting its own `nameparser.__file__`. #461's lone-connective half is what multiplies it: every part whose only word is the connective initials live and drops after the round trip, so `HumanName("Juan de y").initials()` is "J. y." live and "J." after `copy.deepcopy`. RECOMPUTE by building that population and comparing `initials()` before and after each of the three state paths. A larger figure measured over a generated grid says nothing this does not — the population is what the number is about. -- 2026-10-02 #585 — DONE: an initial is the shortest start of its word that composes to the composed word's first character, written in the form the writer used. `initials()` on both surfaces had taken a token's first CODE POINT (`t.text[0]` in `_render.initials`, `word[0]` and `w[0]` in `_facade`), which for a name typed decomposed (NFD) is a base letter without the mark that completes it: `émile zola` initialled `e. z.` against `é. z.`, and — the reach the issue did not name, found by the corpus walk's control — decomposed katakana lost its dakuten, `マイケル ジャクソン` initialling `マ. シ.` against `マ. ジ.`. `_render._first_letter` is the one helper both surfaces call. One shape has no such start: a mark composing with nothing that canonical order puts between the letter and a mark that does (`o` + U+0331 + U+0301, whose composed spelling is `ó` + U+0331), where the initial is the composed character decomposed, `o` + U+0301 — found by the docs review, 2026-10-02. That fallback is for a start LONGER than one code point: a first code point that is a whole character Unicode excludes from composition (`क़`, U+0958, which NFC writes `क` + a nukta) is kept as written, its nukta with it, as before this change; the second review found the first draft of the fallback stripping it (`ज़ाकिर` initialling `ज.`). And the fallback writes the character wholly decomposed even where the writer composed part of it (`ẹ` + U+0331 + U+0302 gives `e` + U+0323 + U+0302) — the twin holds, the written form does not, accepted as contrived. WHY NOT "THE FIRST LETTER AND THE COMBINING MARKS AFTER IT", the shape #542 used for case repair and the issue's own option 1, measured before writing it: decomposed hangul jamo are LETTERS (category Lo), not marks, so it would initial a decomposed `김` as `ᄀ` where the composed name gives `김`; and a Bengali vowel sign IS a mark (Mc), so it would move a COMPOSED `কিরণ` from `ক.` to `কি.`, which is a question about Indic syllables nobody has decided, and a change the gate would see on every composed Indic name. Composing the prefix answers both: a composed word gives `text[0]` exactly as before, so no composed name's initials move. DECLINED: composing the initial to NFC (the issue's option 2, and what outside PR #586 did, on the core surface only). It fixes the dropped mark but hands back a composed initial for a decomposed name, where #542 settled that the render views keep the form they were given (decisions/R4, 2026-10-02). VERIFICATION, which the gate DOES see, unlike #542's: `_initials` is a compared pseudo-field (#484), and the two NFD example lines this change adds to rules/R3 enter the rules corpus and diff at every baseline, classified by a `fix(#585)` `_initials` rule in each of the five ledgers; the CJK rule's corpus claim grows 138 → 139 at 1.4.0 and 2.0.0 with the katakana row, which lies in its script span. `tests/v2/test_properties.py::test_a_decomposed_name_renders_as_its_composed_twin` joins #542's walk rather than opening its own and records 44 disagreements — 22 texts, each on both surfaces — over the 139 texts compared with `text[0]` restored (this change's corpus, its two R3 rows included; measured 2026-10-02), with a live control beside it. + HOW FAR IT REACHES, measured 2026-09-20 over the same 1,568 names against 46651750: the v1 `initials()` of a pickled, copied or deep-copied name differs from the live parse's on 34 names against 6 at the parent. #461's lone-connective half is what multiplies it: every part whose only word is the connective initials live and drops after the round trip, so `HumanName("Juan de y").initials()` is "J. y." live and "J." after `copy.deepcopy`. +- 2026-10-02 #585 — DONE: an initial is the shortest start of its word that composes to the composed word's first character, written in the form the writer used. `initials()` on both surfaces had taken a token's first CODE POINT (`t.text[0]` in `_render.initials`, `word[0]` and `w[0]` in `_facade`), which for a name typed decomposed (NFD) is a base letter without the mark that completes it: `émile zola` initialled `e. z.` against `é. z.`, and — the reach the issue did not name, found by the corpus walk's control — decomposed katakana lost its dakuten, `マイケル ジャクソン` initialling `マ. シ.` against `マ. ジ.`. `_render._first_letter` is the one helper both surfaces call. One shape has no such start: a mark composing with nothing that canonical order puts between the letter and a mark that does (`o` + U+0331 + U+0301, whose composed spelling is `ó` + U+0331), where the initial is the composed character decomposed, `o` + U+0301 — found by the docs review, 2026-10-02. That fallback is for a start LONGER than one code point: a first code point that is a whole character Unicode excludes from composition (`क़`, U+0958, which NFC writes `क` + a nukta) is kept as written, its nukta with it, as before this change; the second review found the first draft of the fallback stripping it (`ज़ाकिर` initialling `ज.`). And the fallback writes the character wholly decomposed even where the writer composed part of it (`ẹ` + U+0331 + U+0302 gives `e` + U+0323 + U+0302) — the twin holds, the written form does not, accepted as contrived. WHY NOT "THE FIRST LETTER AND THE COMBINING MARKS AFTER IT", the shape #542 used for case repair and the issue's own option 1, measured before writing it: decomposed hangul jamo are LETTERS (category Lo), not marks, so it would initial a decomposed `김` as `ᄀ` where the composed name gives `김`; and a Bengali vowel sign IS a mark (Mc), so it would move a COMPOSED `কিরণ` from `ক.` to `কি.`, which is a question about Indic syllables nobody has decided, and a change the gate would see on every composed Indic name. Composing the prefix answers both: a composed word gives `text[0]` exactly as before, so no composed name's initials move. DECLINED: composing the initial to NFC (the issue's option 2, and what outside PR #586 did, on the core surface only). It fixes the dropped mark but hands back a composed initial for a decomposed name, where #542 settled that the render views keep the form they were given (decisions/R4, 2026-10-02). VERIFICATION, which the gate DOES see, unlike #542's: `_initials` is a compared pseudo-field (#484), and the two NFD example lines this change adds to rules/R3 diff at every baseline, classified by a `fix(#585)` `_initials` rule in each of the five ledgers. `tests/v2/test_properties.py::test_a_decomposed_name_renders_as_its_composed_twin` joins #542's walk rather than opening its own and records 44 disagreements (22 texts, each on both surfaces) over 139 texts with `text[0]` restored (measured 2026-10-02), with a live control beside it. diff --git a/docs/design/decisions/R4-case-repair-reads-the-unjoined-mark.md b/docs/design/decisions/R4-case-repair-reads-the-unjoined-mark.md index c2460056..231f11b9 100644 --- a/docs/design/decisions/R4-case-repair-reads-the-unjoined-mark.md +++ b/docs/design/decisions/R4-case-repair-reads-the-unjoined-mark.md @@ -2,8 +2,8 @@ - 2026-08-29 #407 — DECIDED: case repair consults the mark #404 introduced, and consults it for the WHOLE PART rather than for a particle standing alone. Until now three views read it and the fourth did not, which is how `parse("ANH DO")` came to report family_base 'DO' and initials 'A. D.' while `capitalized()` returned 'Anh do' — the same word treated as an ordinary name word by three readers and as a particle by the fourth. The scope question is the only real choice here, and standing-alone loses on the first Vietnamese name that tests it: `anh van do` has family `van do`, two particle-vocabulary words and neither of them alone, so a rule keyed on aloneness would capitalize `Anh Do` and leave `Anh van do`, making the same surname read differently for having a given-name word in front of it. rules/R2 states the test as "Every word", not "standing alone", and this is the case that shows why it is stated that way. One predicate, four views. - 2026-08-29 — this is a DELIBERATE DEVIATION from 1.4.0, not a parity restoration, said plainly because the surrounding branch work is mostly restorations and a reader will otherwise assume this is one. Measured on the released 1.4.0 wheel: `HumanName('ANH DO').capitalize()` leaves `'Anh do'` and `HumanName('anh van do').capitalize()` leaves `'Anh van do'`; both are `'Anh Do'` and `'Anh Van Do'` on this branch. There was nothing to restore — v1 lowercased on vocabulary membership alone and had no notion of a particle with nothing to join, the mark being a 2.x invention. -- 2026-08-29 — the ACCEPTED COST, agreed before the change rather than discovered after it: a degenerate family that is nothing but particles capitalizes too, so `juan van der` renders `Juan Van Der` where 1.4.0 gave `Juan van der`. This is the same trade rules/R2 already took for `family_base` — those words ARE the base, so they are name words, and a parser that called them name words for sorting and particles for display would be the incoherent option. Blast radius, measured by running both predicates in one process, and stated over BOTH populations because the neighbouring bullets in this entry measure over 1094 and a bare "4 and 23" would read as measured there too. Over the 1090 names the corpora carried before this change: 4 names move under a plain `capitalized()` — `ANH DO`, `vai la`, `van ma van`, and `der, y van`, which becomes `y Van Der` and keeps its lowercase conjunction — and 23 under `force=True`, the extra 19 being mixed-case inputs the gate holds back (rules/R5). Over today's 1094 it is 5 and 24: this change's own R4 example rows added `anh van do`, and that name is itself a mover. -- 2026-08-29 — THE VERIFICATION LIMIT, which matters more than the change. `tools/differential/compare.py` compares the seven role fields plus `_ambiguities` and nothing else, so `capitalized()` is not a compared field and NO ledger run at any baseline can see this deviation: the corpus counts and the three intentional-diff totals hold across it precisely because the harness is blind to it, and a green run is evidence about roles, not about case. mechanisms.md#FACADE-CONTRACT promises that v1-visible changes are "release-log-classified fixes, each verified in the differential ledger". The first half is kept — docs/release_log.rst carries the 2.2.0 bullet — and the SECOND HALF CANNOT BE, for this change or for any other that moves only case. Standing in its place: rules/R4's own example rows (`ANH DO`, `anh van do`, and the `juan de la vega` boundary), rules/R5's, and the v1-facade tests in tests/test_capitalization.py. The ones that pin the fix are mutation-checked against deletion of the tag consult; the `juan de la vega` control passes that mutation BY DESIGN, which is what makes it a control rather than a witness. Read the silence as a SCOPE and not as a gap: the harness compares parses, case repair is a render view, and nothing is owed on the tracker for it — the next bullet is why that is a decision rather than a deferral. Amended 2026-09-02 (#484): the premise moved and the conclusion did not. `compare.py` compares the seven role fields plus `_ambiguities` AND `_initials`, so "nothing else" is now one pseudo-field short and a render view is no longer invisible by construction — but `initials()` is the only one added, deliberately (`decisions/differential-ledger, the initials view`), so `capitalized()` is still not a compared field, and the next bullet's "the compared surfaces stay the seven roles plus `_ambiguities`" should be read as naming the surfaces case repair is absent from. Do NOT read #484 as the precedent that answers the next bullet: it MEETS that bullet's two objections rather than waiving them. `_initials` is a diff-of-diffs gated on the fields agreeing — precisely the instrument the next bullet declines for case — and the column did arrive pre-failed, by 140 entries at 1.4.0 as measured on this branch's base — 126 of them classified by six new 1.4.0 rules on the shipped tree, and the other 14 by `fix(#462)` at the three 2.x baselines once the facade fix moved them there. What earned it its place is a demonstrated ESCAPE: #408 moved initials for a whole minor with the gate green, and the 98 per-word names had been moving since 2.0.0, unseen and unclassified. So the next bullet's case for no `_capitalized` column stands until a case change escapes the same way, and that is the test a future proposal has to meet. +- 2026-08-29 — the ACCEPTED COST, agreed before the change rather than discovered after it: a degenerate family that is nothing but particles capitalizes too, so `juan van der` renders `Juan Van Der` where 1.4.0 gave `Juan van der`. This is the same trade rules/R2 already took for `family_base` — those words ARE the base, so they are name words, and a parser that called them name words for sorting and particles for display would be the incoherent option. Blast radius, measured 2026-08-29 by running both predicates in one process (PR #460): 4 of the 1090 pre-change corpus names move under plain `capitalized()` and 23 under `force=True`, the extra 19 being mixed-case inputs the gate holds back (rules/R5); 5 and 24 over 1094 with this change's own mover `anh van do`. +- 2026-08-29 — THE VERIFICATION LIMIT, which matters more than the change. `tools/differential/compare.py` compares the seven role fields plus `_ambiguities` and nothing else, so `capitalized()` is not a compared field and NO ledger run at any baseline can see this deviation: the corpus counts and the three intentional-diff totals hold across it precisely because the harness is blind to it, and a green run is evidence about roles, not about case. mechanisms.md#FACADE-CONTRACT promises that v1-visible changes are "release-log-classified fixes, each verified in the differential ledger". The first half is kept — docs/release_log.rst carries the 2.2.0 bullet — and the SECOND HALF CANNOT BE, for this change or for any other that moves only case. Standing in its place: rules/R4's own example rows (`ANH DO`, `anh van do`, and the `juan de la vega` boundary), rules/R5's, and the v1-facade tests in tests/test_capitalization.py. The ones that pin the fix are mutation-checked against deletion of the tag consult; the `juan de la vega` control passes that mutation BY DESIGN, which is what makes it a control rather than a witness. Read the silence as a SCOPE and not as a gap: the harness compares parses, case repair is a render view, and nothing is owed on the tracker for it — the next bullet is why that is a decision rather than a deferral. Amended 2026-09-02 (#484): the premise moved and the conclusion did not. `compare.py` compares the seven role fields plus `_ambiguities` AND `_initials`, so "nothing else" is now one pseudo-field short and a render view is no longer invisible by construction — but `initials()` is the only one added, deliberately (`decisions/differential-ledger, the initials view`), so `capitalized()` is still not a compared field, and the next bullet's "the compared surfaces stay the seven roles plus `_ambiguities`" should be read as naming the surfaces case repair is absent from. Do NOT read #484 as the precedent that answers the next bullet: it MEETS that bullet's two objections rather than waiving them. `_initials` is a diff-of-diffs gated on the fields agreeing — precisely the instrument the next bullet declines for case — and the column did arrive pre-failed, by 140 entries at 1.4.0 as measured on this branch's base, all of them since classified. What earned it its place is a demonstrated ESCAPE: #408 moved initials for a whole minor with the gate green, and the 98 per-word names had been moving since 2.0.0, unseen and unclassified. So the next bullet's case for no `_capitalized` column stands until a case change escapes the same way, and that is the test a future proposal has to meet. - 2026-08-29 — WHY CASE IS NOT A COMPARED SURFACE, settled here rather than left open, because "add a column" is the obvious answer to the bullet above and it is the wrong instrument. What the harness is FOR is parsing behavior — tools/differential/README.md says to run it "when touching parsing behavior, and before cutting a release" — and what earns it its two-process cost is that a role is EMERGENT: the eight stages fold over one name and each can move what the next sees, so the reach of a vocabulary or rule edit is not derivable by reading the diff and has to be measured over a corpus instead. Case repair is the opposite kind of thing. `_render._cap_word` is a pure function of `(word, role, tags, lexicon)` with four branches — the particle/conjunction lowercasing, the exceptions map, the Mac/Mc convention, and plain `capitalize` — sitting behind R5's single gate, so an edit to it is legible: you can see what you touched. The measurement agrees with the reasoning. Over the 1094 corpus names the 1.4.0 and 2.1.0 wheels ALREADY disagree on `capitalize(force=True)` for 139 of them (recompute by running `HumanName(n).capitalize(force=True)` under each wheel over the corpus files deduped, and diffing the two maps; the count was the same over the 1090 names the corpora carried before this change). Nearly every one is an echo of a parse diff the field comparison already classifies — a token that changed role capitalizes differently — so a column would re-report the ledger's own explained diffs in a vocabulary its rules cannot match, and would arrive pre-failed by 139 entries. Isolating the residue means comparing case only where the fields agree, a diff-of-diffs, to reach a class that a unit test reaches directly. So the compared surfaces stay the seven roles plus `_ambiguities`, by design and not by omission, and case is pinned where case belongs: the `rules/R4` and `#R5` example rows, and tests/test_capitalization.py. - 2026-08-29 — DEBT DISCHARGED, the obligation the last bullet of decisions/R5 left this commit; recorded here so the pointer resolves rather than dangling. The FALSEHOOD: R4 promised repair "vocabulary exceptions (McDonald) included" flatly, which `str(parse('Juan Mcdonald').capitalized())` — `'Juan Mcdonald'` — falsifies, the gate refusing before any vocabulary is consulted. The statement now scopes that promise to where repair acts at all and hands the where to R5. A SECOND error in the same eight words turned up while rewriting them, and is worth recording because it had survived unread for as long as the first: McDonald is not a vocabulary exception. The shipped exceptions are five entries — II, III, IV, M.D., Ph.D. (recompute with `Lexicon.default().capitalization_exceptions_map`) — and `Mc`/`Mac` is a separate convention applied to any name shaped like one, so the old parenthetical illustrated the promise with the one mechanism it was not about. The statement now names an exception the vocabulary actually records (Ph.D.) and the Mac/Mc convention separately. The AMBIGUITY: "an already-correct name comes back unchanged" is true read as idempotence and reads as false under bearer-correctness, and the repair was to say WHICH is meant — a name already written the way repair would write it comes back unchanged — plus one clause refusing the inference a reader might otherwise draw, that a spelling its bearer chose is not spared for having been chosen. That clause is deliberately a REFUSAL and not a carve-out: sparing deliberately single-cased names is behavior the parser does not have and that R5's rationale declines on the record, so `str(parse('bell hooks').capitalized())` is still `'Bell Hooks'`. R4's boundary moved too: the old `"Juan McDonald" → capitalized="Juan McDonald"` row was inert — it passes with R5's gate deleted — so it became a `capitalized_forced=` row, which asserts what repair APPLIES rather than what the gate keeps and fails when the Mac/Mc convention is removed (measured by neutering it in one process: `'Juan Mcdonald'`). The boundary is now `juan de la vega`, a working particle run R4's new clause deliberately does not reach. @@ -11,7 +11,7 @@ - 2026-08-29 — the REPLACE/REVISE BOUNDARY, raised by the review of this branch as a gap needing a systematic fix and corrected by Derek to what it actually is: the documented boundary, with a supported path across it. `_cap_word` keys the particle TEST on lexicon membership but the REPAIR on the unjoined mark, and the mark is a parse product — so `parse('de la').capitalized(force=True)` is `'De La'` while `base.replace(family='de la').capitalized(force=True)` is `'de la'`, `replace()` splicing raw text into a field without classifying it. `Parser.revise()` is the method that classifies it, and its docstring already promised exactly this: each value gets a full sub-parse "so the stable tags survive and the tag-driven views ... behave as if the text had been parsed". Measured: `Parser().revise(base, family='de la').capitalized(force=True)` is `'De La'`. So there is nothing to fix — rules/R4 carries it as an Accepted boundary naming `revise()`, not as a deviation, and `revise()`'s enumeration of the tag-driven views gains `capitalized()`, which #407 made the fourth. (#458 would additionally make `replace()` agree, by keying the test on the `particle` tag rather than re-deriving membership from the word — the same shape it already names for `conjunction`/`initial`. That is a convenience, not the resolution.) It is a limit and not a regression either way: 2.1.0 gives `'de la'` on the spliced path too, so only the parsed path moved. -- 2026-08-29 #458 — DONE: case repair's conjunction test reads the `conjunction` TAG instead of re-deriving the word's class at render time, so a view honors the decision the parse recorded (mechanisms.md#VOCAB-TAGS) rather than taking it again. The two copies of the question were not the same question, which is the reason this was worth doing over a corpus that cannot see it: `_classify.py` asks `is_initial(token.text)` — the shape test ANDed with a repertoire test since #320 — while `_render.py` asked a bare pattern, shape only, over each WORD of a token's text rather than the token. It does NOT retire the hand-sync obligation the module carried in a comment ("keep in sync with `nameparser/_pipeline/_vocab.py` by hand"), and an earlier draft of this bullet said it did: the tag read has nothing to keep in sync, but `_render._INITIAL` was deleted and then restored in the same PR, the fallback in the bullet below giving it a reader again. The obligation is narrower and sharper than before — the two copies no longer answer the same question about the same token, so what has to hold is that the fallback answers as `_vocab` would — and test_regex_sync is what holds it. MEASURED, re-derived on this branch rather than taken from the issue: 0 of the 1094 corpus names move under `capitalized()` or `capitalized(force=True)`, and 0 move with each name also re-run uppercased and lowercased (6564 name/variant/lexicon rows = 1094 x 3 spellings x 2 lexicons, the second lexicon being `Lexicon.default().add(particles={'y'})`, and 13128 `capitalized` calls over them); the R5 force-versus-case counts recorded above hold — 62 upper / 16 lower through the facade, 63 upper / 18 lower through the core. Movement EXISTS outside that population, and the issue's "no output moves" is a claim about the corpus only: the old predicate re-decided per word, so `juan e-f smith` capitalized to `Juan e-F Smith` — the Italian conjunction inside a hyphenated middle name — and now gives `Juan E-F Smith`, agreeing with the uppercase spelling it always gave. An earlier draft of this change moved untagged tokens the same way — a family `replace()` splices in as `de y` repairing to `de Y` — and that was a REGRESSION, caught in review before it shipped and fixed in the bullet below; read this bullet with that one. NOT DONE, and the parenthetical in the boundary bullet above should be read as still open: the PARTICLE conjunct still keys on lexicon membership, so `replace()` does not agree there. Making it read the `particle` tag is a second change with its own blast radius — it moves a boundary rules/R4 states in prose — and #458 as filed does not ask for it. +- 2026-08-29 #458 — DONE: case repair's conjunction test reads the `conjunction` TAG instead of re-deriving the word's class at render time, so a view honors the decision the parse recorded (mechanisms.md#VOCAB-TAGS) rather than taking it again. The two copies of the question were not the same question, which is the reason this was worth doing over a corpus that cannot see it: `_classify.py` asks `is_initial(token.text)` — the shape test ANDed with a repertoire test since #320 — while `_render.py` asked a bare pattern, shape only, over each WORD of a token's text rather than the token. It does NOT retire the hand-sync obligation the module carried in a comment ("keep in sync with `nameparser/_pipeline/_vocab.py` by hand"), and an earlier draft of this bullet said it did: the tag read has nothing to keep in sync, but `_render._INITIAL` was deleted and then restored in the same PR, the fallback in the bullet below giving it a reader again. The obligation is narrower and sharper than before — the two copies no longer answer the same question about the same token, so what has to hold is that the fallback answers as `_vocab` would — and test_regex_sync is what holds it. MEASURED on the #458 branch (PR #463): 0 of the 1094 corpus names move under `capitalized()` or `capitalized(force=True)`, over 6564 rows (three spellings × the default lexicon and one adding particle `y`), and decisions/R5's force-versus-case counts hold. Movement EXISTS outside that population, and the issue's "no output moves" is a claim about the corpus only: the old predicate re-decided per word, so `juan e-f smith` capitalized to `Juan e-F Smith` — the Italian conjunction inside a hyphenated middle name — and now gives `Juan E-F Smith`, agreeing with the uppercase spelling it always gave. An earlier draft of this change moved untagged tokens the same way — a family `replace()` splices in as `de y` repairing to `de Y` — and that was a REGRESSION, caught in review before it shipped and fixed in the bullet below; read this bullet with that one. NOT DONE, and the parenthetical in the boundary bullet above should be read as still open: the PARTICLE conjunct still keys on lexicon membership, so `replace()` does not agree there. Making it read the `particle` tag is a second change with its own blast radius — it moves a boundary rules/R4 states in prose — and #458 as filed does not ask for it. - 2026-08-29 (#458 review) — THE FALLBACK FOR TEXT THE PARSE NEVER SAW, and why it is scoped to one of the two questions `_cap_word` asks. Reading the tag and nothing else regressed every ASSIGNED field holding a conjunction, because a value `replace()` splices in is never classified and so carries no tag to read. Measured on the released 1.4.0 and 2.1.0 wheels, all agreeing and all broken by the tag-only reading: `h.last = 'velasquez y garcia'` then `capitalize(force=True)` gives `John Velasquez y Garcia`, `h.middle = 'e.'` gives `John E. Smith`, `h.last = 'smith-y'` gives `John Smith-y`, `h.first = 'y'` gives `y Smith`. That is the canonical Spanish surname and the shape `y` is in the vocabulary FOR, so an Accepted clause was not available: it had to be fixed. RULE: a token carrying `UNCLASSIFIED_TAG` was never read, so there is no decision to honor and the view falls back to the vocabulary. Note what the discriminator is NOT, in two directions, the second of which took a second review round to find. It is not UNTAGGEDNESS: an ordinary parsed name word carries no tags either (`velasquez` parses to `tags=[]`), so that would have swept in the whole parsed corpus. And it is not the absent SPAN, which is what the first fix used. Span-less means SYNTHETIC, a wider set than unclassified: `Parser.revise()` builds span-less tokens too, from a full sub-parse whose tags it keeps deliberately, and its docstring promises the tag-driven views "behave as if the text had been parsed" — so keying the fallback on the span overrode exactly the tags `revise()` exists to preserve. Measured on the branch that did it: `revise(name, middle='e-f').capitalized(force=True)` gave `e-F` where `parse('john e-f smith')` gave `E-F`, and under `Parser(lexicon=Lexicon())` a revised family `de la vega` initialed `j. v.` where the parse initialed `j. d. l. v.`. So the mark is stamped by the two producers of genuinely unclassified text — `ParsedName.replace()`, and the facade's v1 pickle load, which rebuilds a name from `*_list` strings and no tags — and a HAND-BUILT span-less token is not marked, so it takes the tag path like everything else in the library. That is the tag-driven semantics every other view already has, and it is a change from the span reading in exactly one place: a hand-built span-less token whose text is CONJUNCTION vocabulary now capitalizes rather than lowercasing (measured: a hand-built family `velasquez y garcia` repairs to `Velasquez Y Garcia` untagged, `Velasquez y Garcia` with either the `conjunction` tag or the mark). The PARTICLE conjunct is unaffected either way — it keys on lexicon membership rather than on the mark, which is the open item this bullet's neighbours record — so a hand-built `de la Vega` is unchanged. SCOPED TO THE PER-WORD QUESTION, and this is the durable half. *Is this word a conjunction rather than an initial* is a property of the word and its spelling: an unread token has no ambiguity the parse would have resolved differently, so the vocabulary gives the answer the parser would have given, v1's initial carve-out included — which is why `_render._INITIAL` came back and why test_regex_sync's two-copy assertion matters MORE now, the fallback being right only while it answers as `_vocab` would. *Is this particle acting as a particle here* is a property of the whole PART, and answering it needs a reading on every word of the part, which is exactly what a spliced field lacks. `_cap_word` is handed the whole token's tags — it is not blind to the part because it walks one word, and an earlier draft of this bullet said it was — but with no tag on any word of the part there is nothing to re-derive the part answer FROM, so the fallback here is the word question alone and the particle half falls through to plain particle treatment. Measured cost of the wide reading IN CASE REPAIR, so nobody re-derives it: `replace(family='de la')` gives `de la` today and a parse gives `De La`; widening case repair's fallback would make them agree and would thereby REVERSE the replace/revise boundary recorded two bullets above, with `revise()` as its documented crossing. Do not widen it without reopening the decision. Verified after the fix: `replace(family='velasquez y garcia').capitalized(force=True)` is `Velasquez y Garcia`, `replace(family='de la').capitalized(force=True)` is `de la`, and the parsed and `revise()` paths are unchanged. @@ -20,16 +20,16 @@ - 2026-08-29 — WHY THE BOUNDARY WENT UNNOTICED UNTIL #407, which is where a future reader should look for it. For an ALL-PARTICLE part the other three tag-driven views give the same answer through `replace()` and `revise()` alike: measured over `de la`, `van der`, `do`, `de` and `van de la`, all five agree on `family_particles=''`, on a `family_base` holding the whole part, and on initials from every word. They converge because an UNTAGGED part and a MARKED all-particle part reach the same place by different routes — untagged, no word is recognized as a particle; marked, none is ACTING as one — and all three views only ask which words are particles. Case repair is the one view that asks a second question, since it must also decide whether to lowercase, so it is where the two routes first come apart. The mirror case confirms the reading: on a MIXED part the convergence is the other way round — `de la vega` and `van der berg` diverge in all three views between `replace()` and `revise()` (`replace()` reports particles `''` and base `'de la vega'` where `revise()` reports `'de la'` and `'vega'`) and AGREE on case repair, R4's all-particle clause not reaching them. So before #407 the distinction was invisible on exactly the shape the clause is about, and visible only on shapes the clause does not govern. - 2026-09-13 #459 (landed 2026-09-22, PR #521) — DECIDED: a credential acronym the exceptions map does not carry is an initialism, so a single-case word the parse put in the SUFFIX role from `suffix_acronyms` repairs to its all-caps spelling rather than a title-cased one (the clause in `_render._cap_word`, after the exceptions-map lookup and before the Mac/Mc rule — the order is load-bearing, since `mcse` is in the acronym set and matches the Mac/Mc shape, and reads `MCSE` only because the acronym clause is asked first; rules/R4 states the precedence and pins it with a row). The scope is narrow on purpose. The exceptions map is consulted first, so its five entries — the spellings `str.capitalize()` gets wrong, `md` → M.D. and `phd` → Ph.D. because they are punctuated, and `ii`/`iii`/`iv` → II/III/IV because a numeral is written all-caps and they are `suffix_words` the acronym clause would never see — keep theirs and the clause never touches them. The repair is gated on the SUFFIX role, so a word that is in the acronym vocabulary but parsed as an ordinary name word (`anh van do` → `Anh Van Do`) still repairs as that name word -- the gate is the whole reason the fix is safe on surnames that share a spelling with a credential. Because it is the ROLE and not a parse tag, the gate also reaches a suffix spliced in as raw text: `parse("john smith").replace(suffix="mba").capitalized()` gives `MBA` like the parsed name does, which is the contrast R4's Accepted paragraph draws against the all-particle clause. The same PR removed `ph` from `suffix_acronyms`; see `Excluded (SUFFIX_ACRONYMS — ph ...)`, beside the esq block, for why. The wider design the issue proposed (letter masks, `md` leaving the map, the given-role half of `QC MP`) stays on the rescoped #459; this clause is the narrow part #459 already accepts, not a re-litigation of those. -Reach, population first, measured 2026-09-22 on the PR branch merged with master at 23e52dc6, over the deduped corpus glob (1338 distinct names): 178 names carry acronym vocabulary in a suffix token and 30 of those are written in a single case, and 22 move on the default `capitalized()` path, 119 under `force=True`. Recompute by exec'ing the pre-change `_cap_word` -- `git show 23e52dc6:nameparser/_render.py`, the master this branch merged, which carries #397/#461's generation gate but not this clause -- into `nameparser._render`'s namespace and swapping it in for the changed one in a single process, then diffing `str(capitalized(parse(n), None, force=...))` over the glob. The digits have moved twice and neither move is the clause's: the PR's first head (2026-09-11, `ph` still in the set, 1143 names) measured 5 default and 71 forced; the commit that dropped `ph` took the 13 `PH. D.` forced movers back to `Ph. D.` and left 111 in the population, 5 default and 58 forced; and the merge brought in the corpus rows of PR #530 (#289/#516), PR #532 (#531) and PR #534 (#533), whose single-case trailing `MA` credentials (`DOE, JOHN MA`, `jane doe nee smith ma`, `JOHN SMITH, MA`, the CJK `田中 太郎, MA` family; their mixed-case twins hold still under R5) account for 15 of the 17 further default movers; the other two are `STEVEN HARDMAN, MD, DO, DDS` and `john smith mcse`, the row this PR's own rules.md amendment adds (`McSe` under the pre-change function, which is the precedence claim above measured). The ratio is the stable claim: every single-case name with an acronym suffix the map does not carry moves, and nothing else does. +Reach, measured 2026-09-22 on the PR #521 branch merged with master at 23e52dc6, against the pre-change `_cap_word` swapped in one process, over the deduped corpus glob (1338 names): of 178 names with acronym vocabulary in a suffix token, 30 written in a single case, 22 move on the default path and 119 under `force=True`. The ratio is the stable claim: every single-case name with an acronym suffix the map does not carry moves, and nothing else does. Accepted costs, deferred to the rescoped #459 rather than relitigated here: the all-caps default reaches words conventionally written mixed-case -- `bsc`/`msc` read `BSC`/`MSC` under `force=True`, and `Dr. med. univ. Margit Popp, MSc` is a corpus name that reads `MSC` under `force=True` only, its mixed case holding it back on the default path under R5 -- which the letter-mask design #459 defers is meant to recover; `ii`/`iii`/`iv` are `suffix_words` rather than acronyms and need the exceptions map precisely because the clause would not see them there; and because the ambiguous five (`ba`, `do`, `ed`, `jd`, `ma`) are in `suffix_acronyms`, the clause moves WHICH parse triggers the repair rather than preventing it -- `john smith ed` → `John Smith ED`, `john smith ba` → `BA`, and `smith, ms.` → `MS.` on the default path. That is the same #342/#454-class cost #459 already accepts, and the alternative (reading classify's `vocab:suffix` tag) was measured and costs `jd` → `Jd`, because `jd` carries `vocab:suffix-ambiguous` and not `vocab:suffix`, so the role gate is the right instrument. -- 2026-09-20 #461/#397 — THE GROUNDING MOVED, AND THEN A DEFECT THE GROUNDING EXPOSED. R4's sentence rested on R3 by name ("the carve-out R3 states for initials"), and R3's carve-out is conditional now while R4's is not, so the cross-reference is CUT and replaced by this rule's own reason: a connective that initials because it joins nothing is still not written the way a name is written. Measured 2026-09-20, plain and forced, over `juan y`, `john and jane smith`, `duke of edinburgh`, `juan de y`, `juan y garcia`, `JUAN Y GARCIA`, `josep carod i rovira`, `JOSEP CAROD I ROVIRA` and — under `add(particles={"y"})` — `Anh y Van`: all nine are byte-identical to the same call at the parent, re-measured after the repair below and still byte-identical, and `tests/test_capitalization.py` is green. +- 2026-09-20 #461/#397 — THE GROUNDING MOVED, AND THEN A DEFECT THE GROUNDING EXPOSED. R4's sentence rested on R3 by name ("the carve-out R3 states for initials"), and R3's carve-out is conditional now while R4's is not, so the cross-reference is CUT and replaced by this rule's own reason: a connective that initials because it joins nothing is still not written the way a name is written. Measured 2026-09-20 (PR #536), plain and forced, over nine connective names: byte-identical to the parent, before and after the repair below. WHAT THE PLAIN CALL DOES, stated over a population rather than over nine names, because "capitalized() does not move" was drafted for this bullet and is false as written. Swept 2026-09-20 over the 1558 non-empty corpus-union-cases names under eight configurations against the parent commit, `capitalized()` moves on SIX distinct names and every one of them is a name whose ROLES moved under the join — it renders different fields rather than treating a word differently. The correct statement is therefore: where only `initials()` moves, `capitalized()` is byte-identical; where the roles move, it follows them. - WHAT THE FORCED CALL DID, AND THE DEFECT THAT WAS FOUND WRITING THIS RECORD. The sweep above was run for the PLAIN claim and turned up a second answer: `capitalized(force=True)` moved on ten further names whose roles, initials and plain repair all stayed put, every one a lower-case `i` — `parse("Carod i")` forced gave "Carod i" where 1.4.0 and 2.3.0 gave "Carod I", and so did "John Quincy Smith i", "Josep Carod i", "Lluis Carod i", "Josep Lluis Carod i III", "Josep Lluis Carod i V", "Carod y de Rovira i", "Rovira, Josep Carod i Jr.", "Carod i Rovira" and "Josep i Rovira". Nine of those were a DEFECT and are REPAIRED here; the remaining two are the rule. + WHAT THE FORCED CALL DID, AND THE DEFECT THAT WAS FOUND WRITING THIS RECORD. The sweep above was run for the PLAIN claim and turned up a second answer: `capitalized(force=True)` moved on ten further names whose roles, initials and plain repair all stayed put, every one a lower-case `i` — `parse("Carod i")` forced gave "Carod i" where 1.4.0 and 2.3.0 gave "Carod I", and nine more names likewise. Nine of those were a DEFECT and are REPAIRED here; the remaining two are the rule. WHY IT WAS A DEFECT, and the line R4 now draws. In those names the parse read the letter as the GENERATION it also spells and roled the token SUFFIX — but classify had tagged it `conjunction`, and repair's conjunction conjunct read the tag without asking the role, so a generation was lower-cased as a connective. Derek's principle decides it: a connective with nothing to join is not acting as a connective. Repair now asks the ROLE the parse decided rather than the tag alone, and a suffix-roled token is repaired as the suffix it was read as. `parse("John Quincy Smith i")` forced gives "John Quincy Smith I" again on BOTH surfaces, the facade repairing through the same helper. WHAT REMAINS AND IS NOT A DEFECT: `Carod i Rovira` and `Josep i Rovira`, where the letter is a lone MIDDLE word the parse tagged `conjunction` — three-word names, so P3's carve-out leaves the letter a name word and nothing joined. Forced repair keeps it lower-case, which is R4's own rule and exactly the answer the same shape has always had for `y`: `Carod y Rovira` and `Josep y Rovira` give "Carod y Rovira" and "Josep y Rovira" too. Those two names are the whole of what #397 moves on this view. AND THEN THE REPAIR ITSELF WAS TOO WIDE, found by the second review round (2026-09-20). The guard shipped as the suffix ROLE alone, and the role says only where a word LANDED. A plain connective can land in the suffix field without being generational vocabulary at all — the third part of a comma form is read as the suffix run whatever its words are — so `parse("Smith, John, and").capitalized(force=True)` gave 'John Smith And' where 1.4.0, 2.0.0 through 2.3.0 and the parent 46651750 all give 'John Smith and', and the same for `, y`, `, e`, `, und`, `, of` and `Doe, Jane, and Jr.`. A field SPLICED in went the same way and against R4's own Accepted paragraph, which says a field nobody read asks the vocabulary: `HumanName(first="John", last="Smith", suffix="de y")` repaired forced to 'John Smith De, Y' against 'John Smith De, y' on both wheels. THE GUARD IS NOW THE PAIR — the suffix role AND the `vocab:suffix` tag — because the clause turns on the GENERATION and the role alone cannot see it; classify's tag is the vocabulary half of the decision and reading both re-derives neither. Invisible to everything already watching: no name of the off-switch grid carries the shape (its oracle is a letter of the class, and none of these words is one) and no corpus name carries it either, which is why INV7b was written over a grid of its own. ACCEPTED, and the one reading this moves rather than restores: a field spliced to `suffix="i"` now repairs to 'John Smith i' where the parent gave 'John Smith I'. `i` is connective vocabulary on this branch and was not at the parent, so the spliced field follows `y` now — which is R4's Accepted paragraph working as written, the PARSED name being unaffected because it has a reading and the reading is the generation. - MEASURED, and the numbers are the fix's own control. Over the same eight configurations the forced-only set is ten names before the fix and two after; the invariant `tests/v2/test_properties.py::test_a_letter_that_did_not_join_repairs_as_the_off_switch_does` — a letter that became no connective of this name repairs as the off-switch parse repairs it, plain and forced, on both surfaces — fails on 7,860 repairs at `e540d4c5` (6,843 core, 1,017 v1) and on 0 here, and removing the generation conjunct outright fails it on 8,054. (11,341 / 10,424 / 917 was the same control over the full cross product before the grid was trimmed by shape and then grown by the second review; it is a dated snapshot and the test's docstring is the live figure.) The differential still cannot see any of it, case being no compared surface (the 2026-08-29 bullets above settle why), which is why this needed an invariant rather than a ledger rule. + MEASURED over the same eight configurations (PR #536): the forced-only set is ten names before the fix and two after; the invariant `tests/v2/test_properties.py::test_a_letter_that_did_not_join_repairs_as_the_off_switch_does` fails on 7,860 repairs at `e540d4c5` (6,843 core, 1,017 v1) and on 0 after, and on 8,054 with the generation conjunct removed (its docstring holds the live figure). The differential cannot see case (the 2026-08-29 bullets above), which is why this needed an invariant rather than a ledger rule. - 2026-09-23 #459 — DECIDED (Derek, 2026-09-23): the exceptions map holds case MASKS, and nothing else. A value is its key's own letters and digits, each in the case it takes, and case repair lays it over the word as the writer punctuated it, keeping every other character where the writer put it — so one `phd` entry repairs `phd` to `PhD`, `ph.d.` to `Ph.D.` and `PH.D.` to `Ph.D.`, and the map can hold a conventionally mixed-case acronym (`BSc`, `MSc`) without a dotted spelling per source. A replacement string is substitution, not capitalization: it is how `md` became `M.D.`, how `iii.` lost its period, and why the map never grew past the five entries it was imported with in 2011. The shipped map is `phd` → `PhD`, `bsc` → `BSc`, `msc` → `MSc`; `md`, `ii`, `iii` and `iv` left it, a suffix `md` now repairing by the acronym clause and a suffix numeral by the numeral clause (both below). Lookup is unchanged — the edge-stripped key, then the period-free one — and the map stays ROLE-FREE: an entry is the caller saying how a word is written wherever it stands (`phd smith` gives `PhD Smith`). rules/R4 now states the invariant every clause serves — repair changes case and nothing else — and `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` holds it over every corpus name and case-table text, plain and forced, on both surfaces, under the shipped lexicon and one with the map emptied, comparing under `str.casefold()` (the Unicode-boundary bullet below says why that comparator). `md` → `MD` and `phd` → `PhD` are DELIBERATE DEVIATIONS from every release, 1.4.0 through 2.3.0, all of which gave the dotted forms (measured 2026-09-23 on all five wheels). There is no contributor intent behind the period insertion to preserve: the map, its `md` entry and `capitalize()` itself arrived together in the 2011 svn import (45a15394), authored by Derek. @@ -49,22 +49,22 @@ Accepted costs, deferred to the rescoped #459 rather than relitigated here: the - 2026-09-23 #459 — NUMERALS: a SUFFIX-roled word whose text, edge periods stripped, fullmatches the roman shape repairs to capitals. It reaches `i` through `x`, which is why `ii`/`iii`/`iv` left the map: `vi` through `ix` read as suffix by that shape, carry no vocabulary tag, and title-cased to `Vi`, `Vii`, `Viii`, `Ix` (measured on 4d0680e6, and `Vi`, `Vii` on every release wheel 1.4.0 through 2.3.0); and `john smith iii.` now keeps its period, `III.`, where every release gave `III`. The shape is `_vocab._ROMAN`, v1's `roman_numeral`, and `_render` holds a HAND COPY pinned against config by `tests/v2/test_regex_sync.py`: the approved spec said the clause would import the predicate, but tests/v2/test_layering.py's ALLOWED table gives `_render.py` `_types` and `_lexicon` and nothing else, and `_render._INITIAL` is the precedent for exactly this. The copy re-derives no parse decision — the role is the parse's, read as recorded; the pattern answers only how a numeral is written. THE PARSE LIMIT beside it, recorded and not fixed here: the roman shape stops at `x`, so `john smith xi` reads family `xi` and repairs to `John Smith Xi`, and after a family comma `john smith, vi` reads given `vi` and repairs to `Vi John Smith` — both rules/R4 boundary lines, and both for #459's close-out. The #397 generation guard is unchanged and still decides first: `Carod i` forced gives `Carod I`, the numeral clause now being what writes the capital. -- 2026-09-23 #459 — A SUFFIX READ BY ITS DOTTED SHAPE is an acronym to repair: the acronym clause fires on a SUFFIX-roled token whose period-free text is listed OR which carries `SHAPE_ACRONYM_TAG` (#516's by-shape reading, rules/S3), so `john smith x.y.z.` repairs to `John Smith X.Y.Z.` where the parent gave `X.y.z.`. The constant moved from `_pipeline/_state.py` to `_types.py`, which `_render` may import, and `_state` imports it back — still one spelling, which is what the constant exists for. The role gate holds: `Jack X.Y.Z.` keeps its surname and repairs as a name word, `Jack X.y.z.` under force. ACCEPTED: an unlisted dotted credential written in mixed case reads all-caps wherever repair acts, since a by-shape member has no listed casing to honor — `John Smith B.Tech.` gives `John Smith B.TECH.` under force, where every release gave `B.tech.`, no better. And since the suffix-free gate (decisions/R5, 2026-09-23) this reaches the DEFAULT path too wherever the name words are one-case: `john smith B.Tech.` gives `John Smith B.TECH.` where every release returned it untouched, while the corpus spelling `John Smith B.Tech.` stays untouched there, its name words being mixed. A caller who writes such a credential adds a mask: `('btech', 'BTech')` gives `B.Tech.` on either spelling (measured 2026-09-23). +- 2026-09-23 #459 — A SUFFIX READ BY ITS DOTTED SHAPE is an acronym to repair: the acronym clause fires on a SUFFIX-roled token whose period-free text is listed OR which carries `SHAPE_ACRONYM_TAG` (#516's by-shape reading, rules/S3), so `john smith x.y.z.` repairs to `John Smith X.Y.Z.` where the parent gave `X.y.z.`. The constant moved from `_pipeline/_state.py` to `_types.py`, which `_render` may import, and `_state` imports it back — still one spelling, which is what the constant exists for. The role gate holds: `Jack X.Y.Z.` keeps its surname and repairs as a name word, `Jack X.y.z.` under force. ACCEPTED: an unlisted dotted credential written in mixed case reads all-caps wherever repair acts, since a by-shape member has no listed casing to honor — `John Smith B.Tech.` gives `John Smith B.TECH.` under force, where every release gave `B.tech.`, no better. Its reach onto the DEFAULT path through the suffix-free gate (decisions/R5, 2026-09-23), `john smith B.Tech.` giving `John Smith B.TECH.`, is superseded by the 2026-09-24 SUPERSEDED IN PART bullet below. A caller who writes such a credential adds a mask: `('btech', 'BTech')` gives `B.Tech.` on either spelling (measured 2026-09-23). - 2026-09-23 #478 — DECIDED: inside `_cap_text`, a part of a HYPHENATED token that is connective vocabulary keeps its lowercase when a WORDED part stands on each side of it within the token (split on `-`; a part counts only if it holds a word, and an empty part does not count and is skipped, so a trailing hyphen supplies no neighbour while a doubled hyphen changes nothing: `juan garcia--y-lopez` gives `Juan Garcia--y-Lopez`). An edge part repairs as ordinary name text. `jose ortega-y-gasset` and `JOSE ORTEGA-Y-GASSET` both give `Jose Ortega-y-Gasset`, and `maria silva-e-sousa` gives `Maria Silva-e-Sousa`. It reads POSITION and VOCABULARY, as the particle arm already does per sub-word (`maria da-silva` → `Maria da-Silva`, unchanged) — a question about a sub-word the parse never asked, the parse having read the hyphenated word as one untagged token, which is the same standing mechanisms.md#RENDER-HONORS-THE-PARSE records for the particle arm — and it does not re-derive the initial-versus-conjunction class from a word's case, which is what #458 removed. The edges are pinned on both sides: `juan e-f smith` keeps `Juan E-F Smith` and `juan y-garcia` gives `Juan Y-Garcia`, and in a three-part word `juan y-garcia-lopez` gives `Juan Y-Garcia-Lopez` and `juan garcia-lopez-y` gives `Juan Garcia-Lopez-Y`. A SINGLE LETTER MARKED WITH A PERIOD is an initial there, as the parse reads it in spaced text, never the connective (the guard is `not re.fullmatch(r"\w\.", part)`): the first draft lowered the `e.` in `j.-e.-p. dupont`, giving `J.-e.-P. Dupont` where every release from 1.4.0 through 2.3.0 gave `J.-E.-P. Dupont`, a regression found in review and pinned since as a rules/R4 boundary line. A MULTI-letter word marked with a period is not an initial and stays reachable, as its spaced reading is (`hans smith und. jones` tags `und.` a conjunction), so `hans smith-und.-jones` gives `Hans Smith-und.-Jones`. This reverses, for the interior position only, what #458's tag read did to the shape: 2.2.0 and 2.3.0 gave `Ortega-Y-Gasset` for both spellings — the side effect #478 reported, which the 2026-08-29 #458 bullet above does not record — and #458's edge answer stands. Parity, measured on the released wheels: 1.4.0, 2.0.0 and 2.1.0 re-decided per word and gave `Jose Ortega-y-Gasset` lowered but `Jose Ortega-Y-Gasset` upper, so the all-caps half is a parity break with them on #479's one-case precedent; they also gave `Juan y-Garcia` at the edge, which neither 2.2.0, 2.3.0 nor this tree does. KNOWN NON-REACH, with its witness: the Māori corpus name `Te Awanui-a-Rangi Black` still repairs to `Te Awanui-A-Rangi Black` under force, because `a` is not connective vocabulary and must not become one (a spaced `a` is an initial or a name elsewhere; the connectives' Excluded block in decisions/vocabulary-exclusions says so). ACCEPTED: `mary-e-smith` reads `Mary-e-Smith` — no evidence the shape exists, and the hyphen is the writer joining the parts. Every role, since a hyphenated given name is the same text shape; a middle-role row is pinned beside the family ones (`smith, jose ortega-y-gasset` forced gives `Jose Ortega-y-Gasset Smith`). -- 2026-09-23 #459/#478 — MEASURED, with the recipe; none of these numbers is a live count. Population: the deduped `tools/differential/corpus*.jsonl` glob, both line shapes (an object with `name`, a bare string), as this change leaves it — 1375 distinct names, the new rules.md example lines among them (1371 before the two 2026-09-24 decisions below added four; re-measured 2026-09-24). Comparator: the parent 4d0680e6 — `git archive 4d0680e6` into a scratch directory, both trees imported BY PATH in separate processes over the SAME corpus directory, each asserting `nameparser.__file__` — comparing every token's `capitalized()` and `capitalized(force=True)` text on the core, and the rendered string on the facade, and attributing each moved token PER SUB-WORD: the token is split into its `_WORD` sub-words (`(\w|\.)+`), and the FIRST sub-word whose repair changed decides the bucket, tried in this order — its period-free key is a shipped mask key (`phd`, `bsc`, `msc`, and since 2026-09-24 the fifteen below) or one that left the map (`md`, `ii`, `iii`, `iv`); else the token is a suffix carrying `SHAPE_ACRONYM_TAG` or the sub-word fits the roman shape; else the sub-word is an interior connective part of a hyphenated token; else, on the default path only, the token's repair used to equal its input, which is R5's admission — with a bucket for anything else, asserted empty. The per-sub-word split is load-bearing: keyed on the token as written, the corpus name `John Doe, MD-PhD-` (key `md-phd`) falls to the hyphen bucket instead, giving 15 and 82, 3 and 5 over that day's 1371 names and 11 and 77, 0 and 1 over the parent's corpus (measured in review on 2026-09-23). Mask, or a word that left the map: 16 names on the default path and 84 under force over the 1375 (15 and 83 over the 1371 names before the 2026-09-24 decisions). Dotted shape and numerals: 3 and 16 over both. Hyphenated link: 4 and 5 over the 1375 (3 and 4 over the 1371), every one of them a rules/R4 example line — no corpus name written before this change carries a connective part in a hyphenated word, at the interior or an edge (0 of 1340 at 4d0680e6). R5's gate is R5's bullet. The same comparison over the parent's own corpus directory, 1340 names and none of this change's example lines, gives 11 and 78, 1 and 14, and 0 and 0 — the same before and after the two 2026-09-24 decisions, which move no name written before them: the difference is this change's own rules.md lines, and those are the counts to read as reach into names written for other reasons. The core and the facade move on the same number of names in both views. No role field moves on any name, which is also all the differential gate can say: its compared surface is the seven role fields, `_ambiguities` and `_initials`, never case (the 2026-08-29 bullet above). +- 2026-09-23 #459/#478 — MEASURED against the parent 4d0680e6 (PR #539), both trees imported by path over the same deduped corpus glob, every token's plain and forced repair on core and facade, each moved token attributed by its first changed sub-word: over 1375 names (re-measured 2026-09-24), a mask or a word that left the map 16 names default / 84 forced, the dotted shape and numerals 3 / 16, a hyphenated link 4 / 5 (every one a rules/R4 example line); over the parent's own 1340 names 11 / 78, 1 / 14 and 0 / 0. Core and facade move on the same number of names, and no role field moves. - 2026-09-23 — mechanisms.md#RENDER-HONORS-THE-PARSE CORRECTED, found in this change's re-read of that entry: it still said case repair carries the connective carve-out "ON R3's authority", quoting R4's pre-#461 text ("the carve-out R3 states for initials"), and that `.initials()` gives `j. a. j. s.` for `john and jane smith`. Both had been stale since #461 (2026-09-20), which cut R4's cross-reference to R3 in favor of R4's own reason and stopped a connective with a name word beside it from initialing. The entry now states R4's own reason, dates the cut, and names where the two views part today (`Juan de y`: initials `J. y.`, forced repair `Juan de y`), re-measured 2026-09-23. -- 2026-09-24 #459/#492/#478 (branch review) — FOUR FINDINGS FROM THE REVIEW ROUND, recorded together because each is a small correction to a bullet already above rather than a new decision. (a) THE UNICODE BOUNDARY IS NOT SCOPED TO THE TWO SHAPE CLAUSES, AND NOT TO ONE WAY A MASK LETTER COMES OUT UPPER EITHER. The 2026-09-23 bullet above states the `ß` and dotless-`ı` boundary against `_cap_word`'s acronym/numeral clauses, but the MASK path can lengthen a word through `ß` exactly the same way. `_apply_mask` decides per letter whether it is written upper -- by the mask's own case, or by the split-off-initial force -- and then takes one casing step: the letter at that position in the whole-word `word.upper()`/`word.lower()` when both keep the word's length, otherwise a per-character `c.upper()`/`c.lower()`. `'ß'.upper()` is `'SS'`, so wherever the word holds a `ß` the whole-word form changes the length and every letter goes through `c.upper()`, which lengthens a `ß` written upper to `SS` whichever of the two reasons made it upper. Through the force: an identity mask validates fine (`('a.ß', 'aß')` spells the key's own letters unchanged), so `john a.ß smith` forced gives middle `A.SS` -- casefold-equal to its input and one character longer, the same trade the shape clauses make. Through the mask's own case, no full stop or split letter involved: `('straße', 'STRAẞE')`, the value spelling `straße`'s own letters unchanged with the actual German capital `ẞ` (not the digraph `'STRASSE'`, which the validator refuses as a VALUE for failing to spell `ß` at all), validates and `john straße` forced gives family `STRASSE`. The validator's refusal of `('straße', 'STRASSE')` is a different question -- whether the VALUE spells the KEY -- and says nothing about what the applier can produce from a mask that does validate. This same boundary is where `capitalized()`'s own "Idempotent" docstring claim narrows: forcing `A.SS` (that first pass's own output) through the SAME lexicon a second time gives `A.ss`, not `A.SS` back, because the second pass folds the word to `a.ss` and the exceptions map's key is still `a.ß` -- a different letter sequence, not merely a different case of one -- so the lookup that found the entry on the first pass misses on the second and the word falls through to plain title-casing. Pinned by `tests/v2/test_render.py::test_a_masks_upper_fallback_can_lengthen_a_word_through_ss` and `tests/v2/test_render.py::test_a_lengthening_mask_is_not_a_fixpoint_under_a_second_forced_pass`. The same boundary reaches plain title-casing with no mask involved, by two distinct mechanisms: LENGTHENING, where a letter's own upper() is a different letter SEQUENCE that stays \w -- 'ʼn' (one letter) upper-cases to the two cased characters 'ʼN', so `parse("ʼna smith").capitalized(force=True)` gives given `ʼNa`, and forcing THAT output a second time gives `ʼna` ('N' is no longer the word's first character, so `str.capitalize()` lower-cases it on the second pass) -- and SPLITTING, where a letter's own upper() is a base letter plus a COMBINING MARK that `_WORD` does not match at all -- 'ǰ' upper-cases to 'J' + a combining caron, so `parse("ǰo smith").capitalized(force=True)` gives given `J̌o`, and forcing THAT output a second time gives `J̌O` (`_cap_text` reads the output as two separate words, 'J' and 'o', the combining mark matching neither, and 'o' capitalized alone is 'O'). (b) MASK VALUES ARE STORED NFC-COMPOSED. `_normpairs` compared a value against its key NFC-composed but stored the value AS WRITTEN, so a value written in decomposed form spelled one composed letter as a base letter plus a non-alpha combining mark -- changing which letters `_letter_run_ge2` reads as split off beside a full stop from what the composed spelling of the identical value would read. `john smith pé.x.` forced gave `Pé.X.` under a composed mask and `Pé.x.` under the decomposed spelling of the SAME value. Fixed by NFC-composing the value at the same point the key is composed; pinned by `tests/v2/test_lexicon.py::test_a_decomposed_value_is_compared_and_stored_nfc_composed` and `tests/v2/test_render.py::test_a_decomposed_mask_value_reads_the_same_split_as_composed`. (c) THE HYPHEN CLAUSE'S INITIAL TEST IS NOW A PINNED MODULE CONSTANT. `_cap_text`'s inline `re.fullmatch(r"\w\.", part)` was a third hand copy of the initial shape's period half, invisible to `tests/v2/test_regex_sync.py`'s hand-copy sweep because it was never hoisted to a module-level pattern. Hoisted to `_render._DOTTED_INITIAL` and registered as `_INITIAL`'s period alternative; pinned by `tests/v2/test_regex_sync.py::test_dotted_initial_is_the_period_alternative_of_initial`. (d) THE V1 RAISE NOW CARRIES A V1-SPELLED HINT, VIA A TYPED EXCEPTION. The 2026-09-23 bullet above records that a mismatched-mask value raises `ValueError` at a v1 `Constants`' first parse; the message it raised was `Lexicon`'s own, offering a `Lexicon(capitalization_exceptions=...)` constructor call to a caller looking at a `TupleManager` assignment through `Constants`. `_lexicon._normpairs` now raises a dedicated `_MaskValueError(ValueError)` for exactly that one shape, built in the `UnicodeDecodeError` idiom: `__init__(self, key, normalized_key, value)` stores the three offending inputs and computes `self.offered` (from `_offered_mask`) and the message from them, rather than taking a pre-built message. `__reduce__` returns `(type(self), (self.key, self.normalized_key, self.value), self.__dict__)` -- reconstructing from the three fields and then restoring `self.__dict__` (so an attribute added after construction, e.g. `add_note()`, survives too) -- since the inherited default (`type(self)(*self.args)`, `self.args` holding only the message) cannot reconstruct an instance that needs three fields, not one, and raises `TypeError` trying; a worker in a `ProcessPoolExecutor` pickles an exception it raises to deliver it to the caller, so an unpicklable one there surfaces as `BrokenProcessPool` instead. `_config_shim._build_snapshot` wraps ONLY the `Lexicon(...)` construction in `try`/`except _MaskValueError` and re-raises a message built fresh from `e.key`/`e.value`/`e.offered` -- not from `str(e)`, which is the 2.0 exception's own message offering the 2.0 `capitalization_exceptions=((...),)` constructor spelling no v1 caller can paste -- with `constants.capitalization_exceptions[e.key] = e.offered`. Every OTHER `ValueError` `Lexicon(...)` can raise passes through unchanged, by TYPE rather than by a message-substring test. Of those, the three SUBSET checks (`particles_ambiguous` inside `particles`, `suffix_acronyms_ambiguous` inside `suffix_acronyms`, `honorific_tails` inside `suffix_words`) cannot fire from here: each is built as an intersection or union with its OWN base, so the normalized image of that relation is still a subset whatever the raw spelling. THE CONTRADICTION CHECK AND THE GATE-BYPASS CHECK ARE A DIFFERENT SHAPE, AND DO REACH `Lexicon(...)` -- this bullet claimed otherwise ("cannot arise ... observing no raise") until the confirmation round measured the opposite. Both compare two sets built by SEPARATE set arithmetic in `_build_snapshot`, on the RAW SetManager entries, before Lexicon's own per-field normalization ever runs: `SetManager` strips edge periods but not edge whitespace, while `Lexicon`'s normalization strips both, so an entry carrying edge whitespace can slip past the shim's own subtraction or union and still collide once Lexicon normalizes it independently. Measured: `c.suffix_not_acronyms.add('ba ')` then a parse raises Lexicon's own gate-bypass check (`an ambiguous suffix acronym must not also be a suffix word; in both: ba`), and `c.bound_first_names.add("'t ")` then a parse raises the contradiction check -- both pass through the `except _MaskValueError` clause above unchanged, being a different exception type. Pre-existing at HEAD and at master alike, not introduced by this session and not fixed here -- left for the orchestrator to file as its own issue. What DOES reach `Lexicon(...)` here unchanged, always, is an entry normalizing to empty, in `capitalization_exceptions` or in ANY set field -- measured: `c.titles.add("...")` then a parse raises Lexicon's own `titles entry '' normalizes to empty`, unaltered -- since nothing above translates that away, and a non-str value's `TypeError`, a raise v1 also had, later, at `capitalize()`. Declined: a first draft pre-validated `capitalization_exceptions` in isolation, a second call to `_normpairs` ahead of `Lexicon(...)` gated on a message-substring test; measured to make a multi-word key's dead-entry warning fire twice on the shim path, and to rest on a filter that was untested (deleting it passed the whole suite) and wrong (an unrelated ValueError then got the v1 hint appended regardless). Pinned by `tests/test_capitalization.py::test_a_mismatched_exception_value_raises_at_the_first_parse` (the hint, and that it carries no 2.0 constructor spelling), `tests/v2/test_config_shim.py::test_an_unrelated_capitalization_exceptions_valueerror_has_no_v1_hint` (the pass-through), `tests/v2/test_config_shim.py::test_a_multiword_capitalization_exceptions_key_warns_exactly_once` (the single warning), and `tests/v2/test_lexicon.py::test_mask_value_error_survives_pickle_and_copy` (the pickle/copy round-trip). +- 2026-09-24 #459/#492/#478 (branch review) — FOUR FINDINGS FROM THE REVIEW ROUND, recorded together because each is a small correction to a bullet already above rather than a new decision. (a) THE UNICODE BOUNDARY IS NOT SCOPED TO THE TWO SHAPE CLAUSES, AND NOT TO ONE WAY A MASK LETTER COMES OUT UPPER EITHER. The 2026-09-23 bullet above states the `ß` and dotless-`ı` boundary against `_cap_word`'s acronym/numeral clauses, but the MASK path can lengthen a word through `ß` exactly the same way. `_apply_mask` decides per letter whether it is written upper -- by the mask's own case, or by the split-off-initial force -- and then takes one casing step: the letter at that position in the whole-word `word.upper()`/`word.lower()` when both keep the word's length, otherwise a per-character `c.upper()`/`c.lower()`. `'ß'.upper()` is `'SS'`, so wherever the word holds a `ß` the whole-word form changes the length and every letter goes through `c.upper()`, which lengthens a `ß` written upper to `SS` whichever of the two reasons made it upper. Through the force: an identity mask validates fine (`('a.ß', 'aß')` spells the key's own letters unchanged), so `john a.ß smith` forced gives middle `A.SS` -- casefold-equal to its input and one character longer, the same trade the shape clauses make. Through the mask's own case, no full stop or split letter involved: `('straße', 'STRAẞE')`, the value spelling `straße`'s own letters unchanged with the actual German capital `ẞ` (not the digraph `'STRASSE'`, which the validator refuses as a VALUE for failing to spell `ß` at all), validates and `john straße` forced gives family `STRASSE`. The validator's refusal of `('straße', 'STRASSE')` is a different question -- whether the VALUE spells the KEY -- and says nothing about what the applier can produce from a mask that does validate. This same boundary is where `capitalized()`'s own "Idempotent" docstring claim narrows: forcing `A.SS` (that first pass's own output) through the SAME lexicon a second time gives `A.ss`, not `A.SS` back, because the second pass folds the word to `a.ss` and the exceptions map's key is still `a.ß` -- a different letter sequence, not merely a different case of one -- so the lookup that found the entry on the first pass misses on the second and the word falls through to plain title-casing. Pinned by `tests/v2/test_render.py::test_a_masks_upper_fallback_can_lengthen_a_word_through_ss` and `tests/v2/test_render.py::test_a_lengthening_mask_is_not_a_fixpoint_under_a_second_forced_pass`. The same boundary reaches plain title-casing with no mask involved, by two distinct mechanisms: LENGTHENING, where a letter's own upper() is a different letter SEQUENCE that stays \w -- 'ʼn' (one letter) upper-cases to the two cased characters 'ʼN', so `parse("ʼna smith").capitalized(force=True)` gives given `ʼNa`, and forcing THAT output a second time gives `ʼna` ('N' is no longer the word's first character, so `str.capitalize()` lower-cases it on the second pass) -- and SPLITTING, through a letter whose upper() is a base letter plus a combining mark ('ǰ'), retired by #542 (the 2026-10-02 bullet below). (b) MASK VALUES ARE STORED NFC-COMPOSED. `_normpairs` compared a value against its key NFC-composed but stored the value AS WRITTEN, so a value written in decomposed form spelled one composed letter as a base letter plus a non-alpha combining mark -- changing which letters `_letter_run_ge2` reads as split off beside a full stop from what the composed spelling of the identical value would read. `john smith pé.x.` forced gave `Pé.X.` under a composed mask and `Pé.x.` under the decomposed spelling of the SAME value. Fixed by NFC-composing the value at the same point the key is composed; pinned by `tests/v2/test_lexicon.py::test_a_decomposed_value_is_compared_and_stored_nfc_composed` and `tests/v2/test_render.py::test_a_decomposed_mask_value_reads_the_same_split_as_composed`. (c) THE HYPHEN CLAUSE'S INITIAL TEST IS NOW A PINNED MODULE CONSTANT. `_cap_text`'s inline `re.fullmatch(r"\w\.", part)` was a third hand copy of the initial shape's period half, invisible to `tests/v2/test_regex_sync.py`'s hand-copy sweep because it was never hoisted to a module-level pattern. Hoisted to `_render._DOTTED_INITIAL` and registered as `_INITIAL`'s period alternative; pinned by `tests/v2/test_regex_sync.py::test_dotted_initial_is_the_period_alternative_of_initial`. (d) THE V1 RAISE NOW CARRIES A V1-SPELLED HINT, VIA A TYPED EXCEPTION. The 2026-09-23 bullet above records that a mismatched-mask value raises `ValueError` at a v1 `Constants`' first parse; the message it raised was `Lexicon`'s own, offering a `Lexicon(capitalization_exceptions=...)` constructor call to a caller looking at a `TupleManager` assignment through `Constants`. `_lexicon._normpairs` now raises a dedicated `_MaskValueError(ValueError)` for exactly that one shape, built in the `UnicodeDecodeError` idiom: `__init__(self, key, normalized_key, value)` stores the three offending inputs and computes `self.offered` (from `_offered_mask`) and the message from them, rather than taking a pre-built message. `__reduce__` returns `(type(self), (self.key, self.normalized_key, self.value), self.__dict__)` -- reconstructing from the three fields and then restoring `self.__dict__` (so an attribute added after construction, e.g. `add_note()`, survives too) -- since the inherited default (`type(self)(*self.args)`, `self.args` holding only the message) cannot reconstruct an instance that needs three fields, not one, and raises `TypeError` trying; a worker in a `ProcessPoolExecutor` pickles an exception it raises to deliver it to the caller, so an unpicklable one there surfaces as `BrokenProcessPool` instead. `_config_shim._build_snapshot` wraps ONLY the `Lexicon(...)` construction in `try`/`except _MaskValueError` and re-raises a message built fresh from `e.key`/`e.value`/`e.offered` -- not from `str(e)`, which is the 2.0 exception's own message offering the 2.0 `capitalization_exceptions=((...),)` constructor spelling no v1 caller can paste -- with `constants.capitalization_exceptions[e.key] = e.offered`. Every OTHER `ValueError` `Lexicon(...)` can raise passes through unchanged, by TYPE rather than by a message-substring test. Of those, the three SUBSET checks (`particles_ambiguous` inside `particles`, `suffix_acronyms_ambiguous` inside `suffix_acronyms`, `honorific_tails` inside `suffix_words`) cannot fire from here: each is built as an intersection or union with its OWN base, so the normalized image of that relation is still a subset whatever the raw spelling. THE CONTRADICTION CHECK AND THE GATE-BYPASS CHECK ARE A DIFFERENT SHAPE, AND DO REACH `Lexicon(...)` -- this bullet claimed otherwise ("cannot arise ... observing no raise") until the confirmation round measured the opposite. Both compare two sets built by SEPARATE set arithmetic in `_build_snapshot`, on the RAW SetManager entries, before Lexicon's own per-field normalization ever runs: `SetManager` strips edge periods but not edge whitespace, while `Lexicon`'s normalization strips both, so an entry carrying edge whitespace can slip past the shim's own subtraction or union and still collide once Lexicon normalizes it independently. Measured: `c.suffix_not_acronyms.add('ba ')` then a parse raises Lexicon's own gate-bypass check (`an ambiguous suffix acronym must not also be a suffix word; in both: ba`), and `c.bound_first_names.add("'t ")` then a parse raises the contradiction check -- both pass through the `except _MaskValueError` clause above unchanged, being a different exception type. Pre-existing at HEAD and at master alike, not introduced by this session and not fixed here -- left for the orchestrator to file as its own issue. What DOES reach `Lexicon(...)` here unchanged, always, is an entry normalizing to empty, in `capitalization_exceptions` or in ANY set field -- measured: `c.titles.add("...")` then a parse raises Lexicon's own `titles entry '' normalizes to empty`, unaltered -- since nothing above translates that away, and a non-str value's `TypeError`, a raise v1 also had, later, at `capitalize()`. Declined: a first draft pre-validated `capitalization_exceptions` in isolation, a second call to `_normpairs` ahead of `Lexicon(...)` gated on a message-substring test; measured to make a multi-word key's dead-entry warning fire twice on the shim path, and to rest on a filter that was untested (deleting it passed the whole suite) and wrong (an unrelated ValueError then got the v1 hint appended regardless). Pinned by `tests/test_capitalization.py::test_a_mismatched_exception_value_raises_at_the_first_parse` (the hint, and that it carries no 2.0 constructor spelling), `tests/v2/test_config_shim.py::test_an_unrelated_capitalization_exceptions_valueerror_has_no_v1_hint` (the pass-through), `tests/v2/test_config_shim.py::test_a_multiword_capitalization_exceptions_key_warns_exactly_once` (the single warning), and `tests/v2/test_lexicon.py::test_mask_value_error_survives_pickle_and_copy` (the pickle/copy round-trip). - 2026-09-24 #459 — DECIDED (Derek, 2026-09-24): the shipped map gains a mask for each listed acronym whose conventional spelling is mixed case, taken together with decisions/R5's keep-as-written bullet of the same day, which is the half that surfaced the need. The reviewer who measured the R5 regression named nine keys — `edd`, `dsc`, `psyd`, `meng`, `dmin`, `thd`, `drph`, `pharmd`, `mdiv` — and a sweep of `SUFFIX_ACRONYMS` for further entries whose usual spelling is unambiguously mixed found ten more: `bt`, `chfc`, `cpht`, `diplac`, `kt`, `phc`, `rph` and `thm`, and `lac` and `ded`. Of those nineteen, four are left out (below), so SHIPPED is fifteen — seven of the reviewer's nine and eight of the sweep's ten — every key a `suffix_acronyms` member: `bt` → `Bt` (Baronet), `chfc` → `ChFC` (Chartered Financial Consultant), `cpht` → `CPhT` (Certified Pharmacy Technician), `diplac` → `DiplAc` (Diplomate in Acupuncture, written `Dipl. Ac.`), `dmin` → `DMin` (Doctor of Ministry), `drph` → `DrPH` (Doctor of Public Health), `dsc` → `DSc` (Doctor of Science), `kt` → `Kt` (Knight Bachelor), `mdiv` → `MDiv` (Master of Divinity), `pharmd` → `PharmD` (Doctor of Pharmacy), `phc` → `PhC` (Pharmaceutical Chemist), `psyd` → `PsyD` (Doctor of Psychology), `rph` → `RPh` (Registered Pharmacist), `thd` → `ThD` (Doctor of Theology), `thm` → `ThM` (Master of Theology). The four LEFT OUT are two of the reviewer's nine, `meng` and `edd`, and two of the sweep's ten, `lac` and `ded` (the CAPITALIZATION_EXCEPTIONS block for meng, edd, lac and ded in decisions/vocabulary-exclusions): a mask applies wherever its word stands (the 2026-09-23 masks bullet above), so a key borne as a name re-spells a person — the #vocabulary-collisions C-i test read with the mask's reach, which is every position. Candidates considered and not taken for want of one settled spelling: `djur` (`D.Jur.` beside the German `Dr. jur.`), and `litk`, `litl`, `litp`, `mscmsm`, whose credentials could not be identified. WHY BOTH HALVES: each serves a population the other cannot. Keeping a suffix as written serves the writer who spelled it (`john smith EdD`), but a name written wholly in one case carries no spelling to keep, its suffix being one-case too, so only a mask gives `john smith psyd` → `John Smith PsyD` (`PSYD` by the acronym clause without it; `Psyd` on every release 1.4.0 through 2.3.0, measured 2026-09-24); and a mask cannot reach an unlisted credential or a caller's own spelling, which the keep rule does. No corpus name written before this change carries any of the fifteen keys (measured 2026-09-24 over the parent's 1340 names and this tree's 1375), so the MEASURED bullet above moves only by the rules.md line `john smith psyd`. Recompute the list: `sorted(Lexicon.default().capitalization_exceptions)`. Pinned on both surfaces by `tests/v2/test_render.py::test_a_shipped_mask_spells_its_suffix_in_either_single_case`, which derives its rows from the shipped pairs and so pins the property rather than the list, and the exclusion's fork by `test_a_listed_acronym_that_is_a_name_word_gets_no_mask`, whose recorded negative control is the `meng`/`edd` masks added back. - 2026-09-24 — SUPERSEDED IN PART: the 2026-09-23 bullet above headed "A SUFFIX READ BY ITS DOTTED SHAPE" accepted that `john smith B.Tech.` gives `John Smith B.TECH.` on the DEFAULT path once the suffix-free gate admitted the name. That half no longer holds: decisions/R5's keep-as-written bullet (2026-09-24) keeps a suffix written in more than one case, so the default path gives `John Smith B.Tech.`, the credential as written — every release 1.4.0 through 2.3.0 returned the whole name untouched, `john smith B.Tech.` (measured 2026-09-24). The acceptance stands under force only — `John Smith B.TECH.` with `force=True`, where every release gave `B.tech.` — and the mask remedy it names is unchanged. The rules/R5 example line proposed for this spelling (`juan garcia B.Tech.`) was not added: the differential gate reported it UNEXPLAINED at every baseline, the by-shape suffix reading being a role change no ledger rule claims for that name, so it is pinned as a unit test instead (`tests/v2/test_render.py::test_a_suffix_written_in_more_than_one_case_is_kept_as_written`). - 2026-09-24 #478 — DECIDED (Derek, 2026-09-24): the hyphen clause keeps its reading in a name written wholly in one case, and the case where that disagrees with rules/P3 is a recorded boundary, not a change. The two collide on a marked letter. Spaced, a letter the vocabulary marks as reading both ways reads as an initial in a one-case name (P8), so `maria silva e sousa` repairs to `Maria Silva E Sousa`; hyphenated, the interior part reads as the connective whatever the name's case, so `maria silva-e-sousa` repairs to `Maria Silva-e-Sousa`. Derek's reasoning: hyphenating `Silva-e-Sousa` is the writer joining the surname on purpose, so the interior word is a connective by that act, whatever the one-case rule would say of the spaced form. THE COST, measured 2026-09-24: a one-case name whose hyphenated bare initials happen to spell a connective — `J-E-P DUPONT` gives `J-e-P Dupont`, where every release 1.4.0 through 2.3.0 and the parent 4d0680e6 gave `J-E-P Dupont`, and `JOHN A-Y-B SMITH` gives `John A-y-B Smith` (every release: `John A-Y-B Smith`). The period-marked spelling is unaffected (`j.-e.-p. dupont` keeps `J.-E.-P. Dupont`, the 2026-09-23 #478 bullet). `conjunctions_ambiguous`, P8's knob, does NOT reach a hyphenated word: under `Lexicon.default().add(conjunctions_ambiguous={"y"})` the spaced `JOSE ORTEGA Y GASSET` repairs to `Jose Ortega Y Gasset` (default lexicon: `Jose Ortega y Gasset`) while `JOSE ORTEGA-Y-GASSET` stays `Jose Ortega-y-Gasset` under both. rules/R4's hyphen sentence said the hyphens join the name "as the spaced connective would", which claimed an equality the one-case case breaks; it now says the hyphens are the writer's join and states the split, with an Accepted paragraph and the `J-E-P DUPONT` boundary line. Pinned by `tests/v2/test_render.py::test_the_hyphen_is_the_writers_join_even_in_a_one_case_name`. -- 2026-10-02 #541 — THE "LEFT FOR THE ORCHESTRATOR TO FILE" FINDING of the 2026-09-24 branch-review bullet (d) above is #541, and it was wider than that bullet saw: besides the two raises it names, an entry carrying whitespace v1 never matched SILENTLY ACTIVATED in every set field the shim copies through and in `capitalization_exceptions` keys — measured 2026-10-02 against 1.4.0 (run outside the worktree, `PYTHONSAFEPATH=1`): `titles ' dean '` gave title `dean` on `dean john smith` where 1.4.0 gave first `dean`, and likewise `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a `' zzc '` key; the raise (`suffix_not_acronyms 'ma '`) and the activation (`titles ' dean '`) both reproduce on the 2.0.0, 2.2.0 and 2.3.0 wheels. DECIDED (Derek): `Constants._snapshot()` drops every entry `_config_shim._v1_matchable` rejects — empty after `lc()`, or not equal to its own single-spaced re-join — from all nine set fields and the keys, BEFORE its set algebra, and names them in one `UserWarning` whose remedy runs (`tests/v2/test_config_shim.py::test_the_offered_remedy_runs_and_silences_the_warning`). The test is exact because a v1 piece comes from a whitespace split re-joined only with single spaces; it generalizes the filter `given_name_titles` already had, and `tests/v2/test_config_shim.py::_UNFILTERED_OUTCOME` records what each swept row does with it off. One exception, decided when the 1.4 pickle test caught it: every release from at least 0.5.8 through 1.4.0 SHIPPED two such entries in `TITLES` (`'actor '`, `'television '`), so a restored 1.4 pickle carries them and its user never wrote them; they are dropped WITHOUT the warning (`_V14_SHIPPED_UNMATCHABLE`, held to the pickle's own unmatchable set by `test_the_1_4_shipped_unmatchable_roster_is_exactly_the_pickles`), keeping that pickle warning-free. The exemption is keyed by entry, not by provenance, so a user who writes `'actor '` or `'television '` themselves is not warned either — accepted, since the two strings are 1.x's own typos. The drop is a reading change for that pickle: `Actor John Smith` gave title `Actor` on 2.0.0 through 2.3.0 and gives first `Actor` now, as 1.4.0 did. DECLINED: stripping whitespace in `SetManager` (activates every v1-inert entry above), and a silent drop for user entries (Derek: the entry is certainly a typo and certainly dead, so say so). ACCEPTED WIDENING, deliberately left: a key written with capitals or edge periods (`'McDonald'`, `'phd.'`) never matched on 1.4.0, whose lookup is `lc(word)` against keys stored as written, but has matched since 2.0 through `Lexicon`'s fold; dropping it would break a working 2.x config to restore an inertness nobody wanted. Also left: a set entry `Lexicon` folds to empty that `lc()` does not, a lone non-ASCII full stop (`'。'`), still raises at the first parse in every field `Lexicon` receives directly — `titles`, `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a `capitalization_exceptions` key; `first_name_titles` drops it quietly (`if t`), and `non_first_name_prefixes` and `suffix_acronyms_ambiguous` never pass it to `Lexicon` at all — v1-matchable, so outside this rule. +- 2026-10-02 #541 — THE "LEFT FOR THE ORCHESTRATOR TO FILE" FINDING of the 2026-09-24 branch-review bullet (d) above is #541, and it was wider than that bullet saw: besides the two raises it names, an entry carrying whitespace v1 never matched SILENTLY ACTIVATED in every set field the shim copies through and in `capitalization_exceptions` keys — measured 2026-10-02 against 1.4.0 (run outside the worktree, `PYTHONSAFEPATH=1`): `titles ' dean '` gave title `dean` on `dean john smith` where 1.4.0 gave first `dean`, and likewise `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a `' zzc '` key; the raise (`suffix_not_acronyms 'ma '`) and the activation (`titles ' dean '`) both reproduce on the 2.0.0, 2.2.0 and 2.3.0 wheels. DECIDED (Derek): `Constants._snapshot()` drops every entry `_config_shim._v1_matchable` rejects — empty after `lc()`, or not equal to its own single-spaced re-join — from all nine set fields and the keys, BEFORE its set algebra, and names them in one `UserWarning` whose remedy runs (`tests/v2/test_config_shim.py::test_the_offered_remedy_runs_and_silences_the_warning`). The test is exact because a v1 piece comes from a whitespace split re-joined only with single spaces; it generalizes the filter `given_name_titles` already had, and `tests/v2/test_config_shim.py::_UNFILTERED_OUTCOME` records what each swept row does with it off. One exception, decided when the 1.4 pickle test caught it: every release from at least 0.5.8 through 1.4.0 SHIPPED two such entries in `TITLES` (`'actor '`, `'television '`), so a restored 1.4 pickle carries them and its user never wrote them; they are dropped WITHOUT the warning (`_V14_SHIPPED_UNMATCHABLE`, held to the pickle's own unmatchable set by `test_the_1_4_shipped_unmatchable_roster_is_exactly_the_pickles`), keeping that pickle warning-free. The exemption is keyed by entry, not by provenance, so a user who writes `'actor '` or `'television '` themselves is not warned either — accepted, since the two strings are 1.x's own typos. The drop is a reading change for that pickle: `Actor John Smith` gave title `Actor` on 2.0.0 through 2.3.0 and gives first `Actor` now, as 1.4.0 did. DECLINED: stripping whitespace in `SetManager` (activates every v1-inert entry above), and a silent drop for user entries (Derek: the entry is certainly a typo and certainly dead, so say so). ACCEPTED WIDENING, deliberately left: a key written with capitals or edge periods (`'McDonald'`, `'phd.'`) never matched on 1.4.0, whose lookup is `lc(word)` against keys stored as written, but has matched since 2.0 through `Lexicon`'s fold; dropping it would break a working 2.x config to restore an inertness nobody wanted. Also left: a set entry `Lexicon` folds to empty that `lc()` does not, a lone non-ASCII full stop (`'。'`), still raised at the first parse; superseded by #582 below, which drops it with the warning. - 2026-10-02 #582 — SUPERSEDES the #541 bullet's "Also left" sentence above. DECIDED (Derek): an entry `Lexicon._normalize` folds to empty is dropped by `_config_shim._v1_matchable` with the #541 warning, in every set field and key. The case is a lone CJK full stop (`'。'`, `'.'`, `'。'`, or a run of them), which 2.3.0's `FULL_STOPS` fold (#322/#323) empties while v1's `lc()` strips only `.`. Measured 2026-10-02 on released wheels run outside the checkout with `PYTHONSAFEPATH=1`: 1.4.0 through 2.2.0 accept `c.titles.add('。')` and read `。 john smith` with title `。`, while 2.3.0 raises `ValueError` at the first parse, as did this tree before the change, in `titles`, `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a key. ACCEPTED DEPARTURE from 1.4.0, the only one this filter makes: such an entry used to act on a token that is nothing but that full stop. Since 2.3.0 `Lexicon` can neither hold the entry nor match the token, its lookup fold emptying both, so reproducing v1 is not on offer and the choice was drop or raise. The warning's wording moved with it, from "nameparser 1.x never matched" to "match no name word". `given_name_titles`' `if t` filter was deleted as unreachable: `_title_key` is empty only when every word folds away, and then the whole entry folds to empty and was already dropped. With no shim-reachable plain `ValueError` left from `_normpairs`, `test_an_unrelated_capitalization_exceptions_valueerror_has_no_v1_hint` turns the filter off to keep the except-by-type clause pinned. THE SAME FOLD, ONE CHARACTER IN: an entry with a CJK full stop at its EDGE (not only full stops) passed the filter, and the set algebra compared it raw while `Lexicon` folded it, so `c.suffix_not_acronyms.add('ma。')` (beside the ambiguous `ma`) raised the gate-bypass check and a bound `'zed。'` beside a never-given particle `zed` raised the contradiction check, both from 2.3.0 on; measured accepted on 1.4.0 and 2.2.0 by the review of this change. `_build_snapshot` now compares `_normalize`d spellings in exactly the two computations `Lexicon` re-checks after its own fold — dropping a `suffix_not_acronyms` entry whose fold is an ambiguous acronym's, and taking into `particles_ambiguous` every particle whose fold is a bound given name's — so such an entry reads exactly as its folded spelling (`test_an_edge_full_stop_collision_reads_as_its_folded_spelling`; the fold-off control is `_UNFOLDED_OUTCOME`). That reading is the `dean。` widening below, not v1's: 1.4.0 matched `ma。` only on a `ma。` token. DECLINED, after being written and reviewed: folding EVERY kept entry before the set algebra. `non_first_name_prefixes` and `suffix_acronyms_ambiguous` reach `Lexicon` only through raw set algebra, so folding them is a NEW match, not the 2.3.0 one: `prefixes zz` with `non_first_name_prefixes 'zz。'` moved `zz smith` from first `zz` to last `zz smith`, an NFD/NFC pair across those fields moved the same way, and `suffix_acronyms_ambiguous 'zq。'` beside `suffix_acronyms zq` moved `john zq` from suffix to last — the particle readings the same on 1.4.0, 2.2.0, 2.3.0 and the pre-fold tree, the `zq` one on 2.2.0, 2.3.0 and the pre-fold tree (1.4.0 gives last `zq` there by its own two-word reading, not by matching the entry). They are pinned at those readings by `test_an_edge_full_stop_entry_outside_the_two_checks_reads_as_before`. ACCEPTED WIDENING, left as it is (Derek): since 2.3.0 a `titles` entry `'dean。'` also matches a bare `dean`, where 1.4.0 matched only `dean。`. The same fold makes it, and dropping the entry would lose the `dean。` match too, so it sits beside the capitalized-key widening above. - 2026-10-02 #542 — A DECOMPOSED WORD IS REPAIRED AS ITS COMPOSED TWIN, and the repaired text keeps the form it was typed in: no clause composes its OUTPUT. Two mechanisms do it. The word splitter and the mask's neighbour tests read a combining mark (Unicode category M) as part of the letter before it; the three clauses that ask a regex about a whole word's SHAPE — Mac/Mc and the two initial tests — ask it of the word's composed spelling. They differ only for a mark with no precomposed form, where composing leaves the word as written and so agrees with the parse: under a caller-added conjunction `q́`, `ivan q́. petrov` parses `q́.` as the conjunction, and the hyphen clause keeps `Petrov-q́.-Sidorov` lowercase as it does (measured by the docs review, 2026-10-02). A name typed NFD (macOS file names, some databases) had been split at each mark by `_WORD`'s `(\w|\.)+`, `\w` matching no mark, so `josé garcía` repaired to `José GarcíA` on every release from 1.4.0. Python's `re` has no `\p{M}`, and a hand-written class of the combining blocks would be a copy of Unicode data that drifts (it would already miss Adlam, a cased script whose marks sit outside them), so the test is `unicodedata.category(ch)[0] == "M"`, asked by `_render._sub_words` after each `_WORD` match and by `_render._beside` for a letter's neighbours. DECLINED: NFC-composing before repair, the issue's option 2. It fixes the split but hands back text in a form the writer did not use, which is a change beyond case under rules/R4 — `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` compares with `casefold()`, which does not normalize, and would report it. THE ISSUE'S CLAIM THAT THE MASK NEEDED NO CHANGE ONCE THE WORD REACHED IT WHOLE WAS MEASURED FALSE, and the Mac/Mc clause needed one too; both were found by fixing the splitter alone and comparing again. `_apply_mask`'s split-off-initial override and `_letter_run_ge2` read a letter's neighbour by raw index, which in NFD is the mark rather than the full stop or the letter past it: with the splitter alone, under a caller's `('éx', 'éx')` mask `john smith é.x.` forced gave `É.X.` composed and `é.X.` decomposed — a shape that AGREED before the fix (`É.X.` both ways) only because the old splitter cut the word at its mark before the mask was looked up — and under `('éex', 'éeX')` `john smith ée.x` gave `ée.X` against `éE.X` (`ÉE.X` before the fix). Both read past marks now. The Mac/Mc clause's `\w{2,}` fails at the mark, so with the splitter alone `macée` gave `Macée` decomposed against `MacÉe` composed (`MacéE` before the fix, the split having made the last `e` a word of its own); the clause is now DECIDED on the composed spelling and applied to the word as written. The two initial-shape tests, `_DOTTED_INITIAL` in the hyphen clause and `_INITIAL` in the unclassified-text fallback, matched the raw word and now match its composed spelling, so a decomposed `й.` — `й` is the one default conjunction that decomposes — is the initial its composed twin is: `ivan petrov-й.-sidorov` repaired to `Ivan Petrov-й.-Sidorov` decomposed against `Ivan Petrov-Й.-Sidorov` composed, and a middle spliced in as decomposed `й.` kept `й.` against `Й.`, at 97af1f02 and at this change's first commit alike (found by the docs review, 2026-10-02). AMENDS the 2026-09-24 branch-review bullet (a): its SPLITTING half is retired. `parse("ǰo smith").capitalized(force=True)` gives `J̌o`, and a second forced pass now gives `J̌o` back rather than `J̌O`, the mark staying with its letter (measured 2026-10-02); the LENGTHENING half (`ß`, `ʼn`) stands. A mark with no letter before it heads no word: `str.capitalize()` on a word beginning with one would upper-case the mark and leave the letter after it lowercase. Out of reach and left there, two of them: `initials()`, a rules/R3 view rather than case repair, takes a token's first character, which for a decomposed initial letter is the base letter without its accent (`parse("émile zola")` initials `é. z.` composed and `e. z.` decomposed, measured 2026-10-02; fixed the same day under #585, decisions/R3); and unspaced hangul, whose NFD parse differs from its NFC parse because segmentation matches the census list as written (docs/usage.rst, "Decomposed text"), so repair follows a different parse; hangul jamo are letters, not marks, and nothing here touches them. VERIFICATION: the differential gate cannot see this — `capitalized()` is not a compared surface (this entry, 2026-08-29); the two NFD rows this change adds to the rules corpus reach the gate through the role fields, `_ambiguities` and `_initials` only. Standing in its place: rules/R4's two NFD example lines, `tests/v2/test_properties.py::test_a_decomposed_name_repairs_as_its_composed_twin` (renamed `test_a_decomposed_name_renders_as_its_composed_twin` by #585, which joined initials to the walk), which decomposes every corpus and case-table text that decomposes, compares its repair with the composed text's on both surfaces plain and forced, and records as its control 144 disagreements over 137 texts with 97af1f02's `_render.py` over this change's corpus (measured 2026-10-02), and `tests/v2/test_render.py::test_a_decomposed_word_repairs_as_its_composed_twin` and `test_a_spliced_decomposed_initial_reads_as_its_composed_twin` for the shapes no corpus name reaches, whose two mask rows were mutation-checked by restoring raw-index neighbours. - 2026-10-04 #604 — DECIDED (Derek, 2026-10-04): a mask outranks the particle lowercase. Adding the Irish particles moved case repair the wrong way: `SEÁN Ó MURCHÚ` repaired to `Seán ó Murchú`, where 2.3.0 gave `Seán Ó Murchú` (Ó having been a middle initial there), and `ní` and `ua` the same, because repair writes every particle in lowercase while Irish writes these three capitalized. The map is already "how a word is written wherever it stands" (the role-free note in `_cap_word`), so the particle arm now steps aside for a word the map carries, and the map gains `ó`→`Ó`, `ní`→`Ní`, `ua`→`Ua`. Each mask is its word's own title case, so a bare word in any other role repairs as it would without one. The one reach past particles is the dotted spelling of the two-letter masks: each writes its word as one run, so R4's split-off-letter clause capitalizes both letters, and `u.a. smith` repairs to `U.A. Smith` where it gave `U.a. Smith`, `n.í. smith` to `N.Í. Smith` where it gave `N.í. Smith` (measured 2026-10-04, with and without the three masks; `ó` has no dotted twin), which is right for two initials. Measured before deciding, no shipped exceptions key was a particle (`set(Lexicon.default().capitalization_exceptions_map) & Lexicon.default().particles` was empty), so in the SHIPPED vocabulary the precedence moved nothing but the three new entries. A CALLER's map is another matter, and this is a deliberate change there too (Derek, 2026-10-04, after the design and code reviews both found it): `c.capitalization_exceptions['van'] = 'Van'` makes `ludwig van beethoven` repair to `Ludwig Van Beethoven`, where 1.4.0 through 2.3.0 gave `Ludwig van Beethoven`, the particle having won in v1's `cap_word` and every release since. Kept because an entry is how the caller says a word is written wherever it stands, and a caller who maps `van` to `Van` plausibly means it; the release bullet says so, since the differential gate cannot see `capitalized()` (#R3, above). Pinned by test_a_caller_particle_mask_outranks_the_particle_lowercase. Not extended to `mac`, which repairs to `mac` as before (`Liam mac Murchú`): `mac` is an ambiguous particle shared with Scottish and other spellings, and nothing in #604 weighed it. The Malay `binti`/`binte` carry no mask and keep their lowercase (`Ina binti Navalamar`, where 2.3.0 gave `Ina Binti Navalamar`). diff --git a/docs/design/decisions/R5-case-repair-gate.md b/docs/design/decisions/R5-case-repair-gate.md index 5db5cc51..af0be3de 100644 --- a/docs/design/decisions/R5-case-repair-gate.md +++ b/docs/design/decisions/R5-case-repair-gate.md @@ -4,7 +4,7 @@ - 2026-08-29 — why the gap survived, and what the new boundary buys. R4 promised that "an already-correct name comes back unchanged", which is idempotence: true of everything the gate returns, and silent about the gate. The two readings were indistinguishable through R4's boundary, because `"Juan McDonald"` comes back unchanged under `force=True` as well — an already-correct input cannot witness a rule whose subject is what gets kept REGARDLESS of correctness. `Shirley Maclaine` separates them, so R5's block pins it twice: kept as written on a plain `capitalized=` line, and repaired on the forced line. Under idempotence the plain line would have to fail. The BOUNDARY sits on the forced line, which took a correction to get right — the first draft marked the refusal, and the refusal is R5's effect, not its absence. R5's own stated exception is repair-regardless, so that is the non-firing case (rules.md's preamble: the boundary is "usually the rule's OWN stated exception"). - 2026-08-29 — WHY R5'S STATEMENT WAS REWORDED RATHER THAN GIVEN A THIRD EXAMPLE, recorded at length because the wrong turn was taken twice and the reasoning is reusable. R5 first said a mixed-case name is returned untouched "whether or not its casing is correct". That phrasing reads as a disjunction with two branches, and invites the question of which example witnesses the already-correct one. Nothing can: for a mixed-case name that repair would not alter, kept and repaired are the SAME STRING by construction, so no example line distinguishes a parser with the gate from one without it. The unwitnessable branch is a property of the claim, not a gap in our choice of names — and the bullet above had already said so ("an already-correct input cannot witness a rule whose subject is what gets kept REGARDLESS of correctness") before two review rounds pushed past it. Two rows were tried and both withdrawn. `"Juan McDonald" → capitalized="Juan McDonald"` was INERT (mechanisms.md's inert-measurement class): it passes with R5's gate deleted, which is the whole failure shape that class names. `"Vincent Van Gogh" → capitalized="Vincent Van Gogh"` did discriminate — measured by comparing the shipped call against gate-off behavior, which the forced call reproduces exactly, since deleting the gate is what `force` already does: shipped `'Vincent Van Gogh'`, gate-off `'Vincent van Gogh'`, the particle rule lowercasing `Van` the moment repair runs — but it was REDUNDANT, not complementary. It asserts the same proposition as the `Shirley Maclaine` row above it: repair would change this name, and the gate keeps it anyway. The two differ only in which repair rule would have fired, and which rule repair applies is R4's subject, not R5's. And the frame it was chosen under was itself wrong, which is the more useful half of the lesson (reframed 2026-08-29 on Derek's correction, while R4 was being reworked). The row was picked to be bearer-correct — a name whose casing the bearer would endorse — and that frame drags the document into per-name arguments about whose spelling is right: this one needed `Vincent van Gogh` to be the correct spelling, which P6's own examples contradict, and the tussenvoegsel convention behind them is amended later on this branch anyway. CORRECTNESS DOES NOT ENTER INTO IT. Mixed case is the writer making an explicit choice, and repair defers to that choice rather than judging it — a name kept is not a name endorsed. Read that way the withdrawn row needed no claim about Dutch orthography at all, and the reason it needed one is exactly that the frame was wrong. The fix was to stop claiming the unwitnessable branch: the statement now says a mixed-case name is kept and that whether its casing is right does not enter into the decision, which `Shirley Maclaine` — casing wrong, kept — witnesses whole. GENERAL LESSON, and the reason this is long: when an example cannot be found for half a rule, suspect the STATEMENT before suspecting the example set. A phrasing that promises more branches than the behavior has will absorb inert examples indefinitely, each one looking like progress. Caution for the commit that reworks R4 and will choose its own mixed-case rows: avoid a name whose family base is wholly particle vocabulary (`Anh Van Do`, base `Van Do`), because the #407 work changes how those capitalize and would neuter such a row silently; `gogh` and `vega` are in neither `particles` nor `particles_ambiguous`, `van` and `do` are in both. - 2026-08-29 — the override has TWO routes, and the rule states neither, by design. Per-call is the obvious one. The second is a v1 Constants attribute the facade still honors, `force_mixed_case_capitalization` (nameparser/_facade.py resolves it when no per-call value is given; docs/release_log.rst records it as "still honored through the facade"). Measured today: with that attribute set True, `HumanName('Shirley Maclaine')` then a bare `.capitalize()` — no argument at all — LEAVES `'Shirley MacLaine'` (the v1 call mutates in place and returns None, so the name is read back with `str()`). An earlier draft of R5 said "only an explicit request to repair regardless overrides that", which that measurement falsifies; the statement now says repair was asked for anyway, without saying by what route (it read `repair regardless of how the name is cased was asked for` until 2026-08-29, when that phrasing turned out to have a second reading -- see the override bullet below), because rules.md is implementation-free by its own preamble. -- 2026-08-29 — `capitalized_forced` is a test-side pseudo-field, not a parser field. The route the EXAMPLES use is the per-call argument to `capitalized()`, and an argument is not a policy, locale or extras gate — the only three things rules.md's grammar admits in an example's annotation slot — so it cannot ride that slot, and the doc runner grew a resolver branch instead. (The facade attribute above is a second route to the same behavior, not a second thing to assert; the core takes the argument only.) The R5 block's example lines are asserted by the suite like any others. They do enlarge one file: `corpus_rules.jsonl` is generated from this document's examples, and `Shirley Maclaine` had not been among them, so regenerating adds it (235 lines, from 234). Whether that moves the DIFFERENTIAL population is a separate question with a per-name answer, and the general answer is that a rules.md example CAN move it: the harness dedupes across corpus files, so a new example name costs a population slot exactly when no other corpus already carries that same string. `Shirley Maclaine` was already in `corpus.jsonl`, so it costs nothing and the population holds at 1090 — measured, not assumed. The row withdrawn above is the counter-case, and worth keeping for it: `Vincent Van Gogh` appears in no other corpus (`corpus.jsonl` and `corpus_issues.jsonl` carry `Vincent van Gogh`, a DIFFERENT string that does not dedupe against it), and while it was in the block the population read 1091. Neither name needed a ledger rule, which is a measurement rather than a consequence of the above: the gate exits 0 with 0 unexplained at all three baselines either way. +- 2026-08-29 — `capitalized_forced` is a test-side pseudo-field, not a parser field. The route the EXAMPLES use is the per-call argument to `capitalized()`, and an argument is not a policy, locale or extras gate — the only three things rules.md's grammar admits in an example's annotation slot — so it cannot ride that slot, and the doc runner grew a resolver branch instead. (The facade attribute above is a second route to the same behavior, not a second thing to assert; the core takes the argument only.) The R5 block's example lines are asserted by the suite like any others. Regenerating `corpus_rules.jsonl` from them adds `Shirley Maclaine` (235 lines, from 234). The harness dedupes across corpus files, so a new example name costs a population slot exactly when no other corpus already carries that same string; `corpus.jsonl` already carried this one, so the population held at 1090 (the withdrawn `Vincent Van Gogh`, in no other corpus, had made it 1091), and the gate exited 0 with 0 unexplained at all three baselines either way. - 2026-08-29 — WHAT THE OVERRIDE DOES AND DOES NOT PROMISE, from Derek's framing of R5 and then measured, because the framing implies a property that is ALMOST true and the gap is the useful part. The framing first, and it supersedes the correctness talk elsewhere in this entry: mixed case is the writer making an explicit choice, and repair defers to that choice instead of judging it. Nothing is being called correct or incorrect — a name kept is a name whose writer said something about it, and a name repaired is one whose writer did not. The property that seems to follow is that asking for repair REGARDLESS should ignore the given casing entirely, so one name repairs to one string however it was written. MEASURED over the 1094 corpus names and it does NOT hold. On the v2 core — `parse(n).capitalized(force=True)` against `parse(n.upper()).capitalized()`, rendering all seven roles — the two differ for 63 names, and the lowercase direction for 18. Naming the surface matters here, and BOTH counts move with the surface, which an earlier draft of this bullet got half right: through `HumanName` and `str()`, which is what tests/test_capitalization.py uses, the counts are 62 and 16 — and the two directions are dropped for DIFFERENT reasons, which an earlier draft of this bullet ran together under the first. UPPER: the facade's default render spec omits the maiden name, so `str()` cannot see `Jane van der Berg née y Jones` (the whole of the upper difference, and it is a conjunction inside the maiden name — `maiden` is `y Jones` forced against `Y Jones` uppercased and every other role is byte-identical). LOWER: the two the facade drops are `John van der J. V` and `abdul V Smith`, and the maiden field is EMPTY in both directions for both, so the omitted role explains neither. What hides them is that `str()` CONCATENATES adjacent roles, so a token that crosses a role BOUNDARY and moves nothing else is invisible in the joined string: `John van der J. V` is family `van der J. V` forced against family `van der J.` plus suffix `V` lowercased, and `abdul V Smith` is given `Abdul V` forced against given `Abdul` plus middle `V` lowercased — same seven roles rendered, same string joined. Re-derived 2026-08-29 on this branch, and named per name so the next reader can check the arithmetic: core 63/18, facade 62/16. Two things about those numbers are the opposite of what one would guess. UPPERCASE IS THE WORSE DIRECTION, not the clean one. And the misses are not merely the pipeline's case-sensitivity leaking in: of the 63, only 25 move a role at all, and the other 38 parse byte-identically and diverge inside the repair (25 and 37 through the facade, the missing name being one of the byte-identical ones). The mechanism is v1's initial carve-out — a conjunction is not lowercased where it is written initial-shaped, and initial-shaped means one CAPITAL letter — so uppercasing turns every one-letter conjunction into an initial (`Velasquez y Garcia, Dr. Juan Q.` forced keeps `y`; the same name uppercased then repaired gives `Y`, with a byte-identical partition either side -- the comma form is the one to cite here, the space-written `Dr. Juan Q. Velasquez y Garcia` being a member of the 25 whose roles DO move), and lowercasing turns a middle initial `E` into the Italian conjunction. 1.4.0 does the same (`JUAN Y GARCIA` capitalizes to `Juan Y Garcia`), so this is inherited, and it is recorded here rather than fixed here. Recompute both directions by running the two forms over the corpus files deduped and diffing. - 2026-08-29 — and therefore NOT stated in rules.md, which is a deliberate choice rather than an oversight. The document's examples are keyed on input STRINGS, so any statement of the property invites exactly the test that falsifies it — re-case the input, expect the same output — and the counterexamples are already in the corpora. The property is true of the repair given a parse, and rules.md speaks input-to-output; a rule stating it would be over-broad in the one direction a reader would check. What R5's statement says is enough for the promise that IS kept: a mixed-case name is kept unless repair was asked for anyway. That clause was REWORDED for this, and the reword is the whole point rather than a tidy-up. It read `unless repair regardless of how the name is cased was asked for`, which carries two readings -- the intended one, that the request overrides the keeping, and a second one, that the repair disregards the input's casing, which is this property in nearly this bullet's own words. A reader taking the second reading would run the re-casing test predicted above, land on `Velasquez y Garcia, Dr. Juan Q.` (in the corpus today), and conclude the RULE is wrong when only the phrasing was. Nine words, and they asserted the thing the paragraph exists to deny. The property is pinned in tests/test_capitalization.py instead, over names carrying no single-letter word whose class case decides, with `juan y garcia` beside it as the recorded exception. R5's example block gains `"SHIRLEY MACLAINE" → capitalized="Shirley MacLaine"` from this work, and it earns its place on its own ground rather than as half of a convergence pair: it is the only row in the block that fails when the gate is narrowed to lowercase-only, every other row passing that mutation. Measured three ways — gate deleted (passes, so it does not witness the gate's existence), gate narrowed to accept only all-lowercase (FAILS, and alone in the block), Mac/Mc convention deleted (fails, with the other two rows). Until it was added, R5 stated that repair acts on a name written entirely in one case and witnessed only the lowercase half of it. That lowercase half is still `"juan mcdonald"`, which is byte for byte an R4 row as well, and the duplication is deliberate rather than an editing slip: the two rules make different claims about the same line — R4 that the repair honors the Mac/Mc convention, R5 that an all-lowercase name is acted on at all — and dropping it from R5 would leave the gate's lowercase half unwitnessed inside the rule that states the gate. Five other rows already sit under two rules apiece for the same reason (P5/P6 twice, P5/O5, N3/M4, W1/W3). - 2026-08-29 — DEBT this extraction leaves, named so the next commit inherits an obligation rather than a rediscovery. Pulling the gate out into R5 leaves R4 carrying ONE falsehood and ONE ambiguity — different defects wanting different repairs, and `interacts: R5` carries neither, the field being advisory. FALSE: R4 promises repair "vocabulary exceptions (McDonald) included", but `str(parse('Juan Mcdonald').capitalized())` is `'Juan Mcdonald'` — the gate refuses before any vocabulary is consulted, and only `str(parse('Juan Mcdonald').capitalized(force=True))`, `'Juan McDonald'`, reaches the exception. R4 needs its promise scoped to names the gate admits. AMBIGUOUS, not false: R4's "an already-correct name comes back unchanged" means correct by the repair's own conventions, i.e. idempotence, and under that meaning it is true; a reader hears correct as the bearer writes it, and under THAT meaning `str(parse('bell hooks').capitalized())` — `'Bell Hooks'` — looks like a counterexample. It is not one, because `bell hooks` is not already-correct in R4's sense. What R4 owes is a disambiguation of "correct", NOT a narrowing to spare deliberately single-cased names: that would be new behavior, and R5's own rationale declines it on the ground that single case leaves the repair nothing to read. Also for that commit, and inert as things stand: R4's boundary row `"Juan McDonald" → capitalized="Juan McDonald"` passes with R5's gate deleted, exactly like the R5 row that was withdrawn above; rewriting it to `capitalized_forced=` makes it discriminate for R4's own subject but still witnesses nothing about the already-correct question. This commit adds R5 and touches R4 only on its pointer line, leaving both defects as found rather than half-fixed by a commit whose subject is something else. @@ -18,7 +18,7 @@ - 2026-09-23 #492 — TITLES STAY EVIDENCE, and the reason is a limit of title REPAIR rather than a claim about titles. Title repair is still plain `str.capitalize()`, and widening the gate to titles would surface its defects on the default path. Measured under `force=True`, where they already show (re-measured 2026-09-23): `Lt.Gov. juan e garcia` → `Lt.gov. Juan e Garcia`, `The Lord of the Universe` → `the Lord of the Universe`, `Dr. med. univ. Margit Popp` → `Dr. Med. Univ. Margit Popp`. So `Dr. juan garcia` is left alone (rules/R5's boundary line). This may move when title repair is trusted. -- 2026-09-23 #492 — MEASURED, population first. Recipe: for each name in the deduped corpus glob, the gate as shipped before #492 (the joined text of EVERY token) declines it AND the joined text of its non-suffix tokens is one-case; those are exactly the names the new gate admits, and the default path returns their forced repair (since 2026-09-24, less any suffix written in more than one case, which it keeps as written — the bullet below). The movers are the admitted names whose default-path string changes, counted by the R4 MEASURED bullet's comparator. At 4d0680e6, over its 1340 names, the population was 17 and 12 of them changed their default-path string — `juan garcia III`, `dr. ben alex johnson III`, `john e jones, III`, `john smith MA` and eight `pennie von bergen wessels` spellings; the other five (`威廉·莎士比亚, PhD`, `田中さん, PhD`, `田中さん, Ph. D.`, `김민준씨 Jr.`, `1 & 2, 3 4 5, Mr.`) are admitted and repair to themselves. The approved spec said "17 move": 17 is the population, 12 the movers. Over the corpus this change leaves (1375 names, R5's new example lines among them) the population is 22 and the movers 17, the five new ones being those lines (20 and 15 over 1371 names before the 2026-09-24 decision below added `john smith EdD` and `juan garcia Iii`; re-measured 2026-09-24, the parent-corpus 17 and 12 unchanged). No forced repair moves by this rule, as none can: the gate is never consulted under force. +- 2026-09-23 #492 — MEASURED against the parent 4d0680e6 (PR #539) over the deduped corpus glob, the population being the names the old gate declines and the new one admits, the movers those whose default-path string changes (by decisions/R4's MEASURED comparator): 17 and 12 over the parent's 1340 names (the approved spec's "17 move" was the population), 22 and 17 over this change's 1375, the five new movers being R5's own example lines (re-measured 2026-09-24). No forced repair moves, as none can: the gate is never consulted under force. - 2026-09-24 #492 — DECIDED (Derek, 2026-09-24): where repair was not forced, a SUFFIX token written in more than one case is kept as written; one written in a single case is repaired like any other token, and force repairs both, unchanged. It is the gate's own one-case test asked of each suffix on its own (`_render._in_one_case`, one predicate for both), and it is the 2026-09-23 #492 reasoning read to its end: a suffix is left out of the gate because a cased one says nothing about how the writer cased the NAME, and for the same reason a mixed-case one is the writer's spelling of the credential, which repair has no evidence to overrule. THE REGRESSION THAT SURFACED IT, measured by the branch's code reviewer and re-measured 2026-09-24: once the gate stopped counting the suffix, a one-case name with a conventionally mixed-case listed acronym was admitted and the acronym clause wrote the acronym in capitals — `john smith EdD` → `John Smith EDD`, `JANE DOE, DSc` → `Jane Doe DSC`, `juan garcia PsyD` → `Juan Garcia PSYD` — where the parent 4d0680e6 and every release 1.4.0 through 2.3.0 returned those names untouched. They now give `John Smith EdD`, `Jane Doe DSc` and `Juan Garcia PsyD`. The accepted `B.Tech.` → `B.TECH.` on the default path (decisions/R4, 2026-09-23) was the same shape, and its default-path half is superseded (decisions/R4, 2026-09-24). THE COST, accepted: "as written" keeps a garbled spelling with the deliberate one, so `juan garcia Iii` gives `Juan Garcia Iii` on the default path (`Juan Garcia III` forced) and `john smith Mba` gives `John Smith Mba`, where the branch before this decision gave `III` and `MBA`; every release returned both names untouched. A rules/R5 boundary line pins `Iii`. Why not keep only a spelling the vocabulary does not know how to repair: that would re-derive, per word, a judgment about which mixed spellings are deliberate, which is the evidence R5 says repair cannot read. The masks added the same day (decisions/R4) are the other half: they serve the one-case names this rule cannot reach (`john smith psyd` → `PsyD`). No corpus name written before this decision moves by it (measured 2026-09-24 against the branch before it, over this tree's 1375 names: the only movers are the new rules.md lines). Pinned by `tests/v2/test_render.py::test_a_suffix_written_in_more_than_one_case_is_kept_as_written`, core and facade; a mutant dropping the per-token test fails it, `test_the_gate_leaves_the_suffixes_out` and two rules/R5 lines, and one dropping its `force` guard fails it and the forced `EdD` line. diff --git a/docs/design/decisions/README.md b/docs/design/decisions/README.md index a1cb5d58..526cd244 100644 --- a/docs/design/decisions/README.md +++ b/docs/design/decisions/README.md @@ -9,7 +9,7 @@ Entry conventions: - **Dated decision entries**, append-only for DECISIONS, not for text. Each entry cites its issue or PR. A new entry opens with its verdict — a capitalized headline sentence (`C1 IS DECIDED ONCE, AFTER CLASSIFY, …`) ahead of the evidence, so a reader skimming the file reads the verdicts; older entries that open otherwise are left as written, rewording them being none of the edits below. The date is the DECISION's, never the release's (add "shipped in X" alongside if useful), and git author dates outrank remembered ones — two review rounds corrected exactly these two errors. Four kinds of edit to an existing entry, and no others: 1. **A changed or reversed decision is a new entry.** The old entry's verdict and reasons are never edited; the new one names it on a `SUPERSEDES:` line, and the old one gains, right after its head, `SUPERSEDED by #N —`. Reversed IN PART, the old entry's head names the part instead, as decisions/C3's #575 head does (`(where the count runs superseded by #613, decisions/C1)`). Older entries that announce what they replace by opening `SUPERSEDED:` or `SUPERSEDED IN PART:` (decisions/M2's #549, decisions/R4's 2026-09-24) are live, the marker of a superseded entry being `SUPERSEDED by`. This is the part that keeps a reason from being rewritten to fit a later outcome. 2. **A factual error is corrected in place**, with a note saying what was wrong — a date, an attribution, a measurement or a pointer. decisions/C1's 2026-07-12/13 entry, whose parenthetical moved a misdated credit, is the shape. - 3. **Compaction, on a stated trigger.** A superseded entry is cut to a tombstone: its date, issue, verdict line and `SUPERSEDED by`; a superseded passage inside a live entry is cut to its decision and what replaced it. A measurement paragraph (what moves, cost, blast radius) is cut to one line naming its population, counts, baseline and PR once its change has shipped in a release. The full text stays in git and in the PR the entry cites. #728 is the worked compaction. A harvested entry (below) keeps its framing line through compaction, a tombstone included, and its measured-vs-remembered markings wherever its measurements survive. + 3. **Compaction, on a stated trigger.** A superseded entry is cut to a tombstone: its date, issue, verdict line and `SUPERSEDED by`, plus whatever it recorded that the superseding entry did not replace — a gap it found standing before its change, or an option it declined that the reversal does not answer, stays beneath the tombstone (decisions/T4's #731); a superseded passage inside a live entry is cut to its decision and what replaced it. A measurement paragraph (what moves, cost, blast radius) is cut to one line naming its population, counts, baseline and PR once its change has shipped in a release. The full text stays in git and in the PR the entry cites. #728 is the worked compaction. A harvested entry (below) keeps its framing line through compaction, a tombstone included, and its measured-vs-remembered markings wherever its measurements survive. A trap #759's review found: a one-line replacement carries each count with the classification the old text gave that count and no other (decisions/S2's #614 line had given a whole grid the classification measured on one slice of it). 4. **A per-word fact that moves into the vocabulary's TOML is REPLACED by a query that finds it** (`uv run python tools/vocab/query.py --issue 661`, `--excluded generation`), not left beside a dated "moved" note, which would keep two copies to drift. Only the per-word sentences go; the decision, its criterion and its measurement stay as written. Sanctioned since 2026-10-18 (decisions/vocabulary-model). - **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none and no other entry shares the date, plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. Being tried on decisions/C1 since 2026-10-19; the entries follow under `## Log`. - **Harvested entries** (content landed from a session's report) keep their provenance: a "measurements are that session's, spot-checked at landing" framing line, the contributor's measured-vs-remembered markings where they matter, and — the hard-won one — a flagged uncertainty is resolved by reading the source artifact, never by inference from the neighboring argument. diff --git a/docs/design/decisions/S10-trailing-roman-numeral.md b/docs/design/decisions/S10-trailing-roman-numeral.md index 66a07049..f8acec05 100644 --- a/docs/design/decisions/S10-trailing-roman-numeral.md +++ b/docs/design/decisions/S10-trailing-roman-numeral.md @@ -1,18 +1,18 @@ # S10 — the trailing roman numeral - 2026-10-17 (Derek), #718 — A ONE-LETTER NUMERAL NEEDS WORDS TO SPARE, AND THE RULE IS WRITTEN DOWN. Until this entry no rule stated the comma-less numeral at all: the behavior was pinned in tests/v2/cases.py and credited in code comments to rules/S2, whose statement covers the suffix vocabulary and never the shape fork that reads `VI` to `X`, nor the initial veto that sends `V`, `I` and `X` there. The extraction found one reading worth changing. `John V` and `Malcolm X` read suffix with an EMPTY family in every 2.x release (2.0.0 through 2.4.2 measured), alongside 2.0.0's `Johnson PhD` fix; 1.4.0 read family `V` and `X`. Derek: a generational V is mostly archaic, X rarer still and I hardly ever written without a comma (Sr. and Senior are what people write), so a capital letter standing alone behind the given name is far likelier an initial or a one-letter surname. The numeral was never what S2's Accepted clause (an unambiguous suffix is consumed even when that leaves no family) covers, since the fork has always reported itself ambiguous; S4's count is the analogue, and the numeral now asks it. - WHAT MOVES. The count is S4's own, in name UNITS, so a particle chain is one word (P2) and a title none. Measured on the shipped change against origin/master at 958d252e, the snapshot moves four corpus names: `John V` and `Mohamad X` (family V and X, as 1.4.0 read them), `John née Jones Smith V` (the clause-free name is `John V`, so the V is no suffix and the clause, which M2's statement takes up to the trailing run the clause-free name reads, keeps it: maiden 'Jones Smith V'), and `Freiherr von Richthofen V`, family 'von Richthofen V': behind a title the chain is the one name word, so the V has no word to spare and the chain takes it as P2 takes any name word, as it already took `Freiherr von Richthofen Ma`. That last is the reading a scratch prototype counting raw pieces did not show, and the one most worth a second look. Four case rows moved with it (`strict_comma_reads_the_bare_numeral_into_the_run`: `Smith V, Ma` under the strict knob reads family V, suffix Ma, as 1.4.0 did; and the rows whose purpose needed a suffix kept their ids on a spelling that still has one — `abdul Doe nee Smith V`, `John III, Dr.`, `John III, Sir` — with the old inputs re-pinned as new rows: `abdul nee Smith V` maiden 'Smith V', `John V, Dr.` and `John V, Sir` given John, family V). `Sir abdul V` reads given 'abdul V': the V is a name word, and P5's licence behind a given-name title joins it. Nothing with a word to spare moves: `John W. Ingram V`, `King John Alexander V`, `Larry James Johnson I` (a v1 test) keep the suffix and the report. A title is no word to spare, so `Dr Smith V` reads given Smith, family V, which is 1.4.0's reading too, where `Dr. von Richthofen V` keeps suffix V, the particle and the surname being two words there. + WHAT MOVES. The count is S4's own, in name UNITS, so a particle chain is one word (P2) and a title none. Measured on the shipped change against origin/master at 958d252e, the snapshot moves four corpus names: `John V` and `Mohamad X` (family V and X, as 1.4.0 read them), `John née Jones Smith V` (the clause-free name is `John V`, so the V is no suffix and the clause, which M2's statement takes up to the trailing run the clause-free name reads, keeps it: maiden 'Jones Smith V'), and `Freiherr von Richthofen V`, family 'von Richthofen V': behind a title the chain is the one name word, so the V has no word to spare and the chain takes it as P2 takes any name word, as it already took `Freiherr von Richthofen Ma`. That last is the reading a scratch prototype counting raw pieces did not show, and the one most worth a second look. Four case rows moved with it, their purposes re-pinned on new rows (PR #719). `Sir abdul V` reads given 'abdul V': the V is a name word, and P5's licence behind a given-name title joins it. Nothing with a word to spare moves: `John W. Ingram V`, `King John Alexander V`, `Larry James Johnson I` (a v1 test) keep the suffix and the report. A title is no word to spare, so `Dr Smith V` reads given Smith, family V, which is 1.4.0's reading too, where `Dr. von Richthofen V` keeps suffix V, the particle and the surname being two words there. THE LOWERCASE HALF. A numeral in lower case in a mixed-case name is not how anyone writes an initial, so it is the generation and reports nothing (Derek). Measured before the change, the reports had no rationale: `John Smith v` and `John Smith i` were silent because the listed letters reach the vocabulary path, `John Smith vi` and `John Smith x` reported because the shape fork reports everything it takes, and the one-case `john smith i` reported because the connective tag (P8, #479) makes a one-case `i` an initial. Now the mixed-case lower case is uniformly silent and a one-case name is counted like capitals, `v` excepted (the ACCEPTED paragraph below). `john smith i` keeps its report, by Derek's call. Declined: dropping the shape fork for single capital letters outright. Prototyped in a scratch edit, it moved 26 readings and cost the real generational names the count keeps: `John W. Ingram V` read middle 'W. Ingram', family 'V', and `John van der Berg V` folded the numeral into the particle chain. A variant that dropped the fork for I and X only, on top of the count, moved four more, among them the v1 test `Larry James Johnson I`. ACCEPTED, and recorded in rules/S10: a one-case lowercase `v` is read as the vocabulary reads it, so `john v` still loses its family and `john smith v` reports nothing, where `JOHN V` and `john x` keep their family and `JOHN SMITH V` reports — the listed lowercase letter is taken as a suffix word before the numeral fork is reached, and closing that means touching the suffix-vocabulary path, which is not this change. Not `i`, though it is listed too: a one-case `i` is P8's connective, which classify reads as an initial, so it reaches the fork and is counted (`john i` → family 'i'; `john smith i` reports). And a one-letter numeral with a credential behind it is not the last word and is read as a name word (`John Smith V PhD` → family V, 1.4.0 parity), where `John Smith III PhD` reads suffix 'III PhD'. Recompute: the moved names are this commit's diff of tools/differential/snapshot.jsonl (or `uv run python tools/differential/snapshot.py` on the parent tree with this commit's code). The Declined paragraph's figures are a prototype's, over every cases.py row and snapshot name parsed before and after the patch in one process, on master at 958d252e: its 26 replace `_pieces.is_trailing_numeral_suffix` with a wrapper returning False for a text that is one capital `I`, `V` or `X` (periods stripped) and the original's answer otherwise; its "four more" apply that wrapper to `I` and `X` alone on top of the shipped count. Both are dated snapshots by Counting claims; nothing re-runs them. - 2026-10-18 (Derek), #718 — THE DECLINED NUMERAL REPORTS TOO, BOTH DIRECTIONS AS S4'S FORK DOES. The count's first form reported only the generation, so `John V` read family V in silence while `Jack Ma` reports keeping its family; Derek: report both, as `Jack Ma` does. A declined numeral is a pick of the walk, and its branches are taken in three places, so three sites report it (mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE): assign where it stays a lone name word (`John V` — "read as a family name rather than a generational suffix"), the particle chain where it joins the run (`Freiherr von Richthofen V`), and a maiden clause that keeps it as its last word (`John née Jones Smith V`, `abdul nee Smith V`). Each wording is its own, since the acronym pick's ("both a post-nominal and an ordinary name") misdescribes a numeral. - ONE STAND-DOWN, FOUND BY INV10 rather than designed. In a one-case name the connective `i` reaches the fork as a letter classify has already reported as `conjunction-or-initial`, read as an initial, and the new report put a second report on the same token: 138 rows of the connective grid (measured on this branch with the `reported_as_initial` check out of assign's numeral report, a dated snapshot) in tests/v2/test_properties.py (`AND ROVIRA I`, `HENRY I`) tripped INV10's kind pair. The two reports did not contradict each other — both read the letter as a name word — but they named one reading twice, so the numeral's report stands down where the initial reading is already reported, the shape of rules/S7's "a member its own capitals already made the credential reports nothing new". The guard was left untouched: its kind pair still means "two readings" everywhere else, and narrowing it to suit this change would have retaken its recorded control. `HENRY I` therefore reports `conjunction-or-initial` alone, and moved in test_assemble's contrast list from the withdrawn side to the side where the fork resolved to a name word. + ONE STAND-DOWN, FOUND BY INV10 rather than designed: where classify had already reported a one-case connective `i` as an initial, the numeral's report stood down rather than put a second report on the token, INV10's kind pair being left to mean "two readings" everywhere else; #722's last review (below) replaced the stand-down with one withdrawal. - 2026-10-18, #718 review — THE COUNT IS NOT ASKED IN THE GIVEN PART AFTER A FAMILY COMMA. The code review's grid (family-comma and empty-head shapes behind a title-particle, about 558,000 parses) found 2,880 parses where the first form moved a numeral from the suffix into the given or family name with no report: `Smith, Freiherr von Berg V` read given 'von Berg V', `, Freiherr von V` family 'von V', `Smith, St van V` given 'van V', where `Smith, von V` and `Smith, Freiherr John V` kept suffix V. The cause was not a choice: assign's walk over that part never counts the numeral (rules/S4: the given part's trailing slot "is read without it (S5)"), but the particle chain's bound there asks `tail_reading`, which the count had reached, so the chain took a word assign would have read as the suffix. The bound now reads with the count off (`trailing_run_start`, `count_numeral=False`), and all four read as they did on master (2.2.0 onward, since #424; 1.4.0 read the numeral into the name). The class is the one the AGENTS.md gotcha on group-side predictors of the given part names, met from the other side: the predictor did not change, the peel it asks did. The S4 analogue, `Smith, Freiherr von Berg Ma` → given 'von Berg Ma', is master's reading and is left alone here. - 2026-10-18, #718's PR review — TWO OF THE THREE REPORT SITES ARE ONE, AND THE CLAUSE ASKS THE CASE. The test-coverage review mutated each new emitter out in a copy of the tree. Removing the particle chain's numeral report moved no field and no kind — assign's pick loop reported the same `suffix-or-name`, worded from the final role ('read as a family name') — so the chain's branch was only choosing a sentence, and it is gone: a declined numeral stays one of the read's picks, and assign alone reports it, the decision having been the read's and not the chain's. The maiden clause's stand-down on a letter already reported as an initial could not fire at all (classify reads no connective inside a clause), so that call is gone too. And the clause's report on a numeral its take gave up did not ask the case: `Jane Doe nee Smith vi` and `x` reported where every other path had gone silent, the drift mechanisms.md#ONE-PREDICATE-PER-QUESTION names; the take now asks `numeral_settled_by_case` as assign does, `VI` still reporting beside them. Five rows the first draft labelled `fix(#718)` read field for field as 1.4.0 does and are `parity`, as are the two Freiherr rows added here; a restored 1.4.0 reading is parity by the table's own definition (rules/S4's `bare_surname_is_not_a_credential` is the precedent). - 2026-10-18 (Derek), #721 — A ONE-CASE ONE-LETTER NUMERAL IS THE INITIAL ITS CAPITAL IS, AND A LETTER ENDING ITS PART IS NO CONNECTIVE. Two Accepted gaps of the entries above close. First, `john v` read suffix 'v' with no family while `JOHN V` and `john x` were counted: `V` carries the `initial` tag (a bare capital) and every reader of the trailing run honors it — the suffix veto, the particle chain, the credential run — while a lowercase `v` was a plain listed suffix word. A first prototype (not shipped) routed a one-case `v` to the numeral fork at the walk's last position only, and a lower/upper invariant — every one-case name parses as its capitals, fields and report kinds — showed it too shallow (`john v phd`, `john v, phd` and `freiherr von smith v` still disagreed). The fix that holds tags the letter where its capital is tagged: in a name written in one case, a one-letter roman numeral is an `initial` (classify, reading the NAME-level one-case fact so a maiden clause's `v` is one too, as #516's caps branch reads it). And #721's reviews found the numeral fork's own test of the word in front case-dependent: a bare lowercase letter is never initial-shaped, so `john j v` read suffix 'v' where `JOHN J V` reads family 'V' (on master too). In a one-case name a bare cased letter in front now counts as written as an initial. The invariant is `test_a_one_case_name_reads_as_its_capitals` in tests/v2/test_properties.py, its grid and its two recorded controls stated there: master disagreed on 288 of its 1,575 pairs, the branch on none. Second, Derek on the issue: `john i` reporting `conjunction-or-initial` is wrong "because there's nothing for a conjunction to join to there". P3 already said a connective with nothing to its right connects nothing; classify's report now skips a letter that is the last own word of its part — before a comma, a marker's clause or the end. Such a letter is read as an initial and not reported, and where it is a roman numeral the numeral fork reports `suffix-or-name` instead (`john i`, `HENRY I`, `carod i`); a marked letter that is no numeral is simply silent (`john e`, `jones, john e`). Not all such letters are the walk's last word: a trailing title (rules/H5) is an own word to classify that the walk peels first, so in `john i dr.` the letter keeps P8's report and is still the numeral fork's declined pick. The stand-down the entry above added (a letter already reported as an initial reports nothing new) is what keeps that to one report, and it stays: a first draft of this change deleted it as dead on a mutation run over an intermediate tree, and both reviews then found `john i dr.`, `JOHN I DR.` and `henry i prof.` carrying both reports on one token, where master carries `conjunction-or-initial` alone. Two comma rows go silent rather than move — `rovira, i` and `maier, amy i, jr.` — since the comma path has never reported the numeral (decisions/C9, "Not done"). - WHAT MOVES, the snapshot against master at 2bfa9533: six corpus names, `john v` (family 'v'), `john smith v` and `larry james edward johnson v` (gain the report), `doe, jane v phd do` (middle 'v', suffix 'phd do', as `DOE, JANE V PHD DO` and `Doe, Jane V PhD Do` read; rules/S7's example keeps its input, whose point — `do` anchored by the degree in front — is unchanged), `john i` (report kind), `jones, john e` (report dropped). And a seventh reading the snapshot shows as a removal rather than a move, since the rules example that pinned it was deleted with S10's Accepted clause: `john nee smith i` read suffix 'i' and now reads maiden 'smith i', reported — the clause-free name is `john i`, whose letter has no word to spare, so M2's clause keeps it, as it keeps `john nee smith v`. And decisions/P3's off switch no longer restores 2.3.0's three one-case readings (that file's #721 entry). + WHAT MOVES, the snapshot against master at 2bfa9533: six corpus names (`john v` family 'v', `doe, jane v phd do` middle 'v', four in their reports alone) and one rules example deleted with S10's Accepted clause (`john nee smith i`, now maiden 'smith i'); decisions/P3's off switch no longer restores 2.3.0's three one-case readings (that file's #721 entry; PR #722). - 2026-10-18 (Derek), #721's PR review — A NUMERAL BEFORE A CREDENTIAL IS REPORTED, AND THE TWO-WORD COUNT STANDS. Derek: `john i phd` should report `suffix-or-name` as `john i, phd` does, and `John Smith V PhD` keeps family 'V' with the report — so the Accepted clause's reading stands and only the report is added; the fields do not move (1.4.0 read the same). The part's last name word with nothing but the trailing run behind it is the site: assign, once the trailing read has said what that run is, withdraws classify's connective report on a marked letter there (rules/P8 now says "unless no name word of its part stands to its right") and reports a roman numeral there, worded for what follows it. So `john i dr.` reports the numeral once, `john e phd` and `jose e jr` report nothing, and `john smith i jr` reports `suffix-or-name` where it reported `conjunction-or-initial`. #718's stand-down is kept as a defensive check, and says so: measured on this commit's tree, the full suite and a 1,680-parse trailing-title sweep fail and pass alike without it. Declined: two stricter readings Derek weighed on the PR, both measured over the snapshot corpus. (A) a one-letter numeral is the generation only when given, middle and family are all filled — 11 readings move, among them `Hardie Leannon V`; (B) a comma-less one-letter numeral is never the generation — 18 move, among them `John W. Ingram V`. Both are real people who write the numeral without a comma: #36 (2015) reported `Hardie Leannon V` reading last 'V' as a bug, #136 (2022) came from John W. Ingram V writing his name with and without the comma, and Derek found the same sheriff written `John W Ingram V` and, once, `John Ingram V` (Brunswick Beacon, his candidate profile). The two-word count reads all four as the generation; the rows `a_real_generation_with_a_middle_initial` and `a_real_generation_with_no_middle_name` pin the last two. And the review's other defect: the one-case test of the word in front accepted any one-character cased letter, so `IVAN Ж V` and `JOSÉ É V` moved off their mixed-case reading and a decomposed `é` read otherwise than a composed one. It asks now whether the letter's capital is initial-shaped, which is the rule the docstring states (`john j v` as `JOHN J V`; `é`, `ж` and an uncased word as before). The grid gains a non-ASCII letter in front and a shape-only three-letter numeral; master disagrees on 315 of its 1,872 pairs, the branch on none. diff --git a/docs/design/decisions/S2-suffix-word-ends-a-name-and-never-opens-the-string.md b/docs/design/decisions/S2-suffix-word-ends-a-name-and-never-opens-the-string.md index 0afa0e74..cc608ebe 100644 --- a/docs/design/decisions/S2-suffix-word-ends-a-name-and-never-opens-the-string.md +++ b/docs/design/decisions/S2-suffix-word-ends-a-name-and-never-opens-the-string.md @@ -8,19 +8,19 @@ THE SECOND REVIEW (bfa798fb, 2026-10-07) found four defects in the counting view and the report hand-off, every one on a shape the corpus, the gates and the first stress grid held no instance of, and each now has a case row that dies without its fix. (1) The read started at the end of the leading titles counted in PIECES, applied to the view: a title that is also a particle opens a unit inside the titles, so the index landed past the name and `Freiherr St van Berg MA` read family 'St van Berg MA' (0c54662b suffix 'MA'); the view now returns where the name starts in its own indices -- the first unit opened at or before the titles' end, since a unit opened inside them makes a name of them (SUPERSEDED IN PART by #624, decisions/P2, 2026-10-10: the chain now opens no unit inside the titles) even where it ends short (`Freiherr Freiherr Prof do` lost its report on 'do' until that second half). (2) Assign reported the chain's picks, in the wrong field's words, above. (3) The view made units INSIDE #602's credential run, which hid a title word from the run's title test: `John Smith MD van Secretary Jones` lost title 'Secretary' to the suffix. Past a run starter a unit is now the bare particle run, as the chain's own merge leaves it. (4) A `title` piece-tag exclusion in `_counted_as_one` made garbage differ from 0c54662b (`Freiherr von Mr. and Mrs. X.Y.Z.`) and guarded nothing (3) did not; removed rather than pinned. NOT THE SAME READ AS M2'S: the maiden take reads its own clause-free view of the words before any join, word by word, as rules/M2 states, and its Accepted clause already covers what that does where a particle chain would take the run; S2's read counts the chain's units after P3's joins. Two reads of one name answer different questions -- where the clause ends, where the name's run starts -- and are kept apart rather than made one here. RESIDUALS, the paths that read as before #614: a segment of fewer than three pieces (no join reaches it); a run with a name piece inside it (an H5 title in front of a name word the peel declined, `John Dr. G.J.`, `John van V Dr. V`), which is not split -- group re-asks and assign reads for itself; and a name group's joins leave without a name word in front of the run, which assign reads again so it keeps one -- the same procedure as before #614, not always the same answer, the Freiherr garbage above being the measured case. - MEASURED 2026-10-07 against 0c54662b (#618's merge), py3.11, `nameparser.__file__` asserted on each side; #619, merged between, moved no reading. The two #617 harnesses as decisions/P3's 2026-10-06 entry records them (the fingerprint, 78,324 parses; the connective grid, 485,300), and an S2 stress grid built for this entry: the 22 heads `John`, `John Smith`, `John Q. Smith`, `Mary Ann Smith`, `John van Smith`, `Juan de la Vega`, `John van`, `anh van`, `Jan Freiherr von Berg`, `Freiherr von Berg`, `abdul salam`, `abdul rahman al-said`, `Juan y Garcia`, `Josep Carod i Rovira`, `Jane Doe nee van der Berg`, `Dr. John Smith`, `JOHN SMITH`, `john smith`, `Smith`, `de Mesnil`, `Kim Min Jun`, `Ortega y Gasset`, each followed by every one- and two-word tail over the 35 words `PhD MA Ma DO Do do vd VD mc Mc MD Jr Jr. Sr III V V. VI I X.Y.Z. XYZ Ph. D. Esq. Prof. Dr. Sir MBA RN G.J. Ed van and Jones Ed. ba` (`Ph. D.` one word of the list); then 25,000 draws seeded `random.seed(614)`, each `choice(heads)`, `randint(3, 5)` and that many `choice(words)`, joined by spaces; then every head before a comma with every two-word pair behind it, and `Smith, ` and ` , John` for every word; deduplicated, under four policies (default, both family-first orders, strict comma suffixes) -- 81,138 names, 324,552 parses, comparing the seven roles, every ambiguity's kind and detail, `initials()` and `capitalized()`. Fingerprint: 6 parses move, `John van der Berg Prof.` under six policies, and nothing else. Connective grid: 1,248 move, no corpus or case name among them: 1,212 the title class, and 36 garbage shapes holding a title or a credential run behind a leading particle (`de der Dr. MA PhD Vega y Lopez` reads suffix 'PhD Vega y Lopez', #602's run, where 0c54662b kept every word the family), 8 of them moving a report's grouping alone (below). Stress grid: 8,472, no corpus or case name; 48 move reports alone, 20 the suffix wording above and 28 the grouping of an absorbed word's report where the run the view stops unit-making at is not the one the peel reads (`anh van Do Sr DO Jones MBA` reports 'DO' and 'Jones' where 0c54662b reported 'DO Jones'); restricted to a head and one tail word under the default policy, 12 parses, the six particle-chain heads with `Dr.` and `Prof.`, all the title class. (At bfa798fb, before the second review, these were 30, 2,936 and 20,532, the slice 33.) Whole tail readings per corpus name (`tail_reading` calls, 1,536 names): 1,125 read once, 56 twice, none three times, against 1,095, 55 and 30 on 0c54662b; peel passes, which count H5's fixed point's own re-peels, 1,215 names at one against 583. Five differential gates exit 0 with the one move classified `fix(#614)` at every baseline. - COST, `tools/perf/call_count.py` on each interpreter against adf6da88 (#619's merge): `parse` 317 -> 308 on 3.11 and 296 -> 290 on 3.12-3.15, `facade` 354 -> 345 and 333 -> 327 (decisions/parse-cost, this date). Per name on 3.11, warm: the Vega name 376 -> 368, and every segment of fewer than three pieces unchanged, never reaching the read (`John Smith` 152, `Smith, John` 161, `John Smith, PhD` 206). + MEASURED 2026-10-07 against 0c54662b (#618's merge), py3.11, `nameparser.__file__` asserted on each side; #619, merged between, moved no reading. The two #617 harnesses as decisions/P3's 2026-10-06 entry records them (the fingerprint, 78,324 parses; the connective grid, 485,300), and an S2 stress grid built for this entry: the 22 heads `John`, `John Smith`, `John Q. Smith`, `Mary Ann Smith`, `John van Smith`, `Juan de la Vega`, `John van`, `anh van`, `Jan Freiherr von Berg`, `Freiherr von Berg`, `abdul salam`, `abdul rahman al-said`, `Juan y Garcia`, `Josep Carod i Rovira`, `Jane Doe nee van der Berg`, `Dr. John Smith`, `JOHN SMITH`, `john smith`, `Smith`, `de Mesnil`, `Kim Min Jun`, `Ortega y Gasset`, each followed by every one- and two-word tail over the 35 words `PhD MA Ma DO Do do vd VD mc Mc MD Jr Jr. Sr III V V. VI I X.Y.Z. XYZ Ph. D. Esq. Prof. Dr. Sir MBA RN G.J. Ed van and Jones Ed. ba` (`Ph. D.` one word of the list); then 25,000 draws seeded `random.seed(614)`, each `choice(heads)`, `randint(3, 5)` and that many `choice(words)`, joined by spaces; then every head before a comma with every two-word pair behind it, and `Smith, ` and ` , John` for every word; deduplicated, under four policies (default, both family-first orders, strict comma suffixes) -- 81,138 names, 324,552 parses, comparing the seven roles, every ambiguity's kind and detail, `initials()` and `capitalized()`. Results: fingerprint 6 parses move (`John van der Berg Prof.` under six policies), connective grid 1,248 (1,212 the title class, 36 garbage) and stress grid 8,472 (its head-plus-one-word slice under the default policy, 12 parses, all the title class), no corpus or case name among them; five differential gates exit 0 with the one move classified `fix(#614)` (PR #621). + COST, `tools/perf/call_count.py` against adf6da88 (#619's merge): `parse` 317 -> 308 frames on 3.11, segments of fewer than three pieces unchanged (decisions/parse-cost, this date; PR #621). - 2026-10-08 (Derek), #620 — S2'S READ COUNTS THE CHAIN'S UNITS OVER THE PIECES AS WRITTEN, RATHER THAN MERGING A COPY OF THEM. The #614 read above counted through a merged copy of the pieces, a second statement of P2's chain with its own stop list, and its reviews found three defects that were that copy's: an index applied across the two layouts (`Freiherr St van Berg MA`), a unit formed inside #602's run hiding a title (`John Smith MD van Secretary Jones`), and a shape-only numeral folded into a unit (`Juan de la Vega VI`), each patched in the copy. Now `_pieces._chain_units` marks each piece by the unit the chain will put it in -- OPENS, JOINED (a name word the chain joins to the run in front of it) or BOUND (a particle inside the run, a word the run took; retired by #625, this section, 2026-10-10, group merging the run first) -- asking `chain_run_end`, the chain's run end, which `_group`'s chain now calls too. The peel's words to spare (`second_unit`) and #602's run start count in those marks and the peel stops at any piece the chain joins into a run, which is a name word whichever mark it carries; everything else reads the pieces as written, so nothing is hidden, and the copy, its view index and its credential-run gate are gone. WHAT THE COUNT KEEPS. A word the reading weighs -- a suffix word of either kind, the ambiguous class, a word in it by shape, an initial, a roman numeral by shape, a period-marked title word (`_weighed`) -- keeps a unit of its own though the chain may join it, as a position always counted it; only the plain words behind a particle run are JOINED, and a unit with nothing past its opener moves no name start into the titles (SUPERSEDED IN PART by #624, decisions/P2, 2026-10-10: no unit opens inside the titles, so the gate is gone). The first prototype marked every word the chain reaches JOINED, by the chain's own rule alone, and the review found it wrong in two shapes the earlier count read right: `Freiherr von Berg MA X.Y.Z.` kept 'MA X.Y.Z.' in the family (a member the chain would join counted as no word in front of the next), and `St St VI` and `Freiherr St MA` read title 'St St', 'Freiherr St' with the numeral or acronym the whole family (a unit of words the read then takes, which the chain never builds). Both masters read all three as they read now. (#624 since reads `St St VI` and `Freiherr St MA` as title 'St St' / 'Freiherr St' again, by a different route: the chain and the count now share one leading position, the last title-particle, so neither builds the unit -- decisions/P2, 2026-10-10.) The copy's exclusion list had been carrying the count positions always kept -- a member in front counts -- and the marks keep it without the copy. The numeral fork still reads the piece in front of the numeral as written -- reading the unit's first word broke rules/P2's `John van der J. V` -- so `John van B and Smith X` reads family 'van B and Smith X' again, the reading #614's copy had moved. ALSO COUNTED IN UNITS: #602's run start, a particle opening a chain run counting once and its bound particle not at all (`John der la Jr. Prof. Smith` keeps its run, `van la Smith Secretary Jr. Dr. Smith` starts none, as on master). And `credential_run`, on the read's path, gathers adjacent particles into the one absorbed piece the chain makes of them (`Dr. John Smith Esq. RN Mc Mc` reports 'Mc Mc'; SUPERSEDED by #625, this section, 2026-10-10: group merges the run before the read, which absorbs it as one piece); M2's clause-free view reports word by word as it always has. DECLINED, measured: a units test in `tail_reading`'s resume condition. A splice lowers a count only by taking out titles that open units of their own, and with fewer than two units in front of it every piece there is in the first piece's chain run, which the titles behind it join; fuzzed over 372,330 names, the test changed nothing. CLASSIFICATION, found writing this entry's rows: eight case rows #614 added as `parity` did not match 1.4.0 -- they had been compared against master -- and are relabelled with the change each reading came from, bisected (#424, #289, #296, #516, #602 twice; R2 and P2 for two that read so since the 2.0 pipeline). - MEASURED 2026-10-08 against 4278693d (#621's merge), py3.11, `nameparser.__file__` asserted, with the #614 harnesses and recipe above: fingerprint 0 moves; connective grid 0; S2 stress grid 12, three garbage names under four policies, none in the head-plus-one-word slice -- `Freiherr von Berg Dr. X.Y.Z. VD` reads title 'Freiherr Dr.', suffix 'X.Y.Z. VD' (H5's trailing title, the class #614 moved; master family 'von Berg Dr. X.Y.Z.'), `Jan Freiherr von Berg VD V and ba I` reads family 'VD V and ba I' as adf6da88 did (master suffix 'I'), and `de Mesnil Ma do mc and Ph. D.` reports the 'do' the peel took beside adf6da88's 'Ma'. Five gates exit 0. Frames unchanged, 308 on 3.11 and 289 on 3.12-3.15 (decisions/parse-cost). + MEASURED 2026-10-08 against 4278693d (#621's merge), with the #614 harnesses and recipe above: fingerprint and connective grid 0 moves, S2 stress grid 12 (three garbage names under four policies, none in the head-plus-one-word slice), five gates exit 0, frames unchanged (decisions/parse-cost; PR #622). /SIMPLIFY (2026-10-08), measured. Each pass of `tail_reading` had rescanned for the second unit from the front, a C-level quadratic no frame guard saw: `Freiherr von` + `Berg `*n + `MA Dr. `*n read 5.77x for 4x the input against master's 4.08x; the lookup is now made once per read and handed to every pass (4.03x), and `test_the_second_unit_is_found_once_per_read` counts it. `_weighed` is pinned to the reading by `test_a_word_the_read_takes_is_never_one_the_count_joined` (no piece marked JOINED is ever in the run the read takes; 320 grid names fail with the test answering False). The gathering of an absorbed particle run reads the BOUND marks, and #602's run start one condition. Grids byte-identical, frames unchanged. Left for a follow-up, each moving readings: merging the tail's particle runs in group before the read, which would retire the BOUND stop and the gathering (80 grid names move, `Dr. mc mc`, and the chain's particle report needs re-keying; done by #625, this section, 2026-10-10, where #624 had already moved those 80); and the name start decided once, the item below. LEFT OPEN, by Derek's scoping: reading a family comma's given part the same way, and deciding the name's start once for a title-particle head (#620's related paths, which change readings). The second was settled by #624 (decisions/P2, 2026-10-10), the first by #623 with a narrower bound rather than a second read (decisions/H5, 2026-10-10). - 2026-10-10 (Derek), #625 — THE TAIL'S PARTICLE RUNS ARE MERGED IN GROUP BEFORE S2'S READ, SO THE READ NEEDS NO BOUND MARK AND NO GATHERING. Since #614 split the trailing run off ahead of the chain, the read sees particles the chain has not merged yet, and #620 re-created the chain's particle-to-particle merge in the reader: `_chain_units` marked a particle standing straight behind another BOUND, the peel stopped at it, and `credential_run` gathered an absorbed run of them into one report. Now group merges each run of two or more adjacent particles past the chain's leading position (`_pieces.merge_particle_runs`, at the trailing site, after P3's joins and before the read) into the piece the chain would open its run with, so the read counts, stops at and absorbs the run as one piece, and the BOUND mark, the peel's stop for it and the gathering are gone. The merged piece keeps `prefix` and drops `title`: a P3 join carries the title tag in from a word in both vocabularies (`St und Berg`), and on the merged run it read the run as a leading title. The title run and the lead are read once, before the merge, and handed to the read, which had asked both again. The chain's PARTICLE_OR_GIVEN report keyed on its run claiming something past the particle (`j > k + 1`), which a merged run has done already though the chain then claims nothing more (`Freiherr von der`); it now keys on that or on the run being one the merge made (`premerged`) -- not on the piece holding several tokens, which a P3-joined phrase does too, and which moved 11,358 reports on the grid below (`Dr. St Do und`). The rule text is unchanged: rules/S2 already counts a particle run, the duals straight behind a particle included, as the one name word the chain makes. What grows is code, not concepts: the read loses a mark and two branches and re-creates nothing of the chain, group gains the merge (about 45 more lines than it removes across `_pieces.py` and `_group.py`, docstrings included). DECIDED (Derek): retiring the two reader-side mechanisms is worth the moves -- every one is garbage input, now read consistently, and none is a released reading restored. - MEASURED 2026-10-10 against 505d6dec (#632's merge), py3.11, `nameparser.__file__` asserted on each side. A particle-run grid -- `{head} {run} {tail}` for every head in (none), `Dr.`, `Freiherr`, `St`, `John`, `Dr. John`, `Freiherr von`, `John Smith`, `Sir`, `Freiherr St`; every run of one to three words over `van der de la von und mc Mc vd do Do St du`; every tail in (none), `Berg`, `MA`, `PhD`, `Ma`, `Jr.`, `Berg MA`, `Prof.`, `Berg Ph. D.`, `V`, `VI`, `X.Y.Z.`, `née Smith`, `Berg Jr. PhD`, `Berg van`, `, Jr.`, `, John` (a comma tail written against the run) -- 395,148 texts, 1,185,444 parses under the three orders, comparing the seven fields and every report (kind, detail, tokens): 774 role moves (258 texts under each order), no report-only move. Every move holds `St und`, the joined piece carrying the title tag, which master merged into the particle run in front with the tag kept and read as a title (`Freiherr Do St und Berg MA` → title 'Freiherr Do St und Berg', family 'MA'); now title 'Freiherr', family 'Do St und Berg', suffix 'MA', the name part read alike with and without the trailing 'MA'. Not a released reading: 1.4.0 reads first 'Do St und Berg', last 'MA', and 2.3.0 family 'Do St und Berg MA', and over the 258 moved texts under the default order master's seven fields equal 2.3.0's on 152, the tree's on none. The S2 stress grid (#614's recipe, above): no role move, and one text's report under its four policies -- `de Mesnil Ma do mc and Ph. D.` no longer reports 'do': P3 joins `mc and Ph. D.` into one suffix-tagged piece, the merge folds 'do' into it, and the peel takes the piece whole rather than picking 'do', the report adf6da88 did not make either and #620 added (its entry above). The #624 title-particle head grid (decisions/P2, 2026-10-10): 0. Every corpus name and case text, and #624's comma-tail grid, under the three orders: 0. The five differential gates exit 0, their reports identical line for line to the tree before. The 2026-10-08 prototype's 80 moves (`Dr. mc mc`) are not in the population: #624 already reads `Dr. Mc Mc` as H3 states. Recorded negative controls (2026-10-10): with the merge off, 30 tests in tests/v2 fail (32 over the whole suite, measured at the review's fix), `test_an_absorbed_particle_run_reports_as_the_one_word_the_chain_makes` among them; with the report keyed on `j > k + 1` alone, the case rows `a_particle_takes_the_dual_behind_it_at_the_head` and `a_particle_run_before_the_run_stays_one_name_word`; with the merge starting at the segment's second piece rather than past the lead, 13; with `title` kept, only `test_a_merged_particle_run_is_a_particle_and_no_title`, the unit test added for it. Cost: +3 frames on the reference row, which holds `de la` (decisions/parse-cost, this date). - THE REVIEW (2026-10-10) found the change reaching further, every shape garbage, none given a row (Derek): no move goes the other way, and nothing moved over comma shapes (an empty or nickname-only head, a family comma's given part), connective-separated runs, runs beside a maiden marker or holding a nickname, bound given names, one-case names, or caller lexicons adding title or suffix words as particles. (1) The `St und` class is any P3 join whose left word is in both the title and particle vocabularies, `und`, `and` or `of` (`y` moves nothing: master already reads `Freiherr Do St y Berg MA` as now), a caller's title-particle included (`Freiherr Do Freiherr and Berg MA`: master title 'Freiherr Do Freiherr and Berg', family 'MA'; now title 'Freiherr', family 'Do Freiherr and Berg', suffix 'MA'). (2) A title-particle head, a particle run, then a title and a dotted word (`Freiherr van der Dr. G.J. MA`, 88 texts on its grid under each order): the run now counts as one name word, so 'G.J.' has none to spare, the read takes nothing and group's own chain takes 'Dr. G.J.' -- family 'van der Dr. G.J.', suffix 'MA', as the one-particle `Freiherr van Dr. G.J. MA` reads (master title 'Freiherr Dr.', family 'van der', suffix 'G.J. MA'). (3) H3's give-back behind a leading title where every word is suffix vocabulary, the last two a run of duals (`Dr. vd Jr. mc mc`, about 1,800 parses over two grids): the merged run is a particle piece and no suffix word, so the title stays -- title 'Dr.', family 'vd', suffix 'Jr. mc mc', where master gave 'Dr.' back as the given name with 'vd' still standing; `Dr. vd Jr. mc` still gives it back. The review also found two runs in one name unreached by any test (an off-by-one in the merge's bookkeeping passed the suite), now the rows `two_particle_runs_merge_each` and `a_second_run_of_duals_is_the_family` (parity) and a half of the merge's unit test, which also holds that a lone particle is no merged run; and the read's title run taken before the merge held only by the frame bands, now `test_the_read_takes_the_title_run_as_written_before_the_merge` (`Dr. mc Jr mc mc`, which with the run asked again after the merge, by the read or by group, reads family 'mc Jr mc mc'; the fix's own review found the group-side half unheld by a read-level test, so it pins the parse too). + MEASURED 2026-10-10 against 505d6dec (#632's merge) over a 395,148-text particle-run grid under the three orders: 774 role moves (258 texts under each order), every one holding the title-tagged `St und` join, none a released reading (over the 258 default-order texts master equals 2.3.0 on 152, the tree on none); the #614 stress grid moves one text's report (`de Mesnil Ma do mc and Ph. D.`), and the #624 grids, every corpus name and case text and the five gates move nothing; with the merge off 30 tests in tests/v2 fail; +3 frames on the reference row (decisions/parse-cost, this date; PR #634). + THE REVIEW (2026-10-10) found the change reaching further, every shape garbage, none given a row (Derek), no move going the other way: (1) any P3 join whose left word is in both the title and particle vocabularies, (2) a title-particle head, a particle run, then a title and a dotted word (`Freiherr van der Dr. G.J. MA`), (3) H3's give-back behind a leading title over a run of duals (`Dr. vd Jr. mc mc`); two runs in one name and the read's title run taken before the merge, unheld by any test, gained the rows `two_particle_runs_merge_each` and `a_second_run_of_duals_is_the_family` and `test_the_read_takes_the_title_run_as_written_before_the_merge` (PR #634). - 2026-10-17 #682 — THE RULE WAS SPLIT, AND SO WAS THIS LOG. rules/S2 had grown to seven rules, and each now has its own ID and its own decisions entry, which took the entries written under this key about it: decisions/S3, which shared this log (the roman-chunk retirement and the digit-dotted tokens), decisions/S4 (the count and the slots that report), decisions/S5 (the given part's trailing slot after a family comma, #531, #573, #578), decisions/S6 (written case, #289, with the review and measurement rounds of the #289 + #516 arc), decisions/S7 (company, #544), decisions/S8 (the credential's run, #602, #627, #679) and decisions/S9 (the by-shape class, #516's two switches, #563, #564). What stays here is the trailing read itself (#614, #620, #625), which rules/S2 kept. The entries keep their text as written: one dated before this entry that says rules/S2 means the rule as it then stood. - 2026-10-19 — CLAUSE MOVED OUT: the ambiguous acronym written with its periods left rules/S2 for rules/S3, the rule whose signal that shape is, and S2's rationale lost the sentence that argued for it. S2's own signal is now the one position it states, a suffix word ending the name and never opening the string. Record: decisions/S3's entry of this date. diff --git a/docs/design/decisions/S6-case-signal-at-the-suffix-slot.md b/docs/design/decisions/S6-case-signal-at-the-suffix-slot.md index 77abd9ec..c59d7e96 100644 --- a/docs/design/decisions/S6-case-signal-at-the-suffix-slot.md +++ b/docs/design/decisions/S6-case-signal-at-the-suffix-slot.md @@ -5,25 +5,13 @@ - 2026-09-14 (Derek) — THE LEAN REACHES THE POST-COMMA GIVEN SLOT: `Smith, MA` → family 'Smith', suffix 'MA', while `Smith, Ma` and `Smith, ma` stay given. Positive evidence outranks position there, so the credential reading fires with ONE word before the comma where the positional rule would not. AND THE COUNT AT A COMMA IS OF NAME WORDS. For this class, "words to spare" in `X, Y` form means the part before the comma holds two or more NAME words: `Smith Jr., MA` has two tokens and one name word, and a token count hands its family to `given` (measured on the prototype). rules/C1 (the second clause rules/C8's since #728) already said both things — "more than one word precedes the comma" in its opening clause and "a part before the comma with more than one name word" later — so the doc edit is a reconciliation inside C1 rather than a new claim. - 2026-09-17, corrected — A CASELESS SCRIPT IS INERT ONLY AS A LEAN, NOT AS THE NAME-WORD COUNT. `is_one_case` answers True trivially for text with no case to write a contrast in, so `ambiguous_lean` returns `None` for `毛泽东, MA`, `마틴 킹, MA` and `田中 太郎, MA` alike — the LEAN really is inert by construction, as first written. But the comma count above is of NAME words "whatever case the name is written in", which is orthogonal to case and reaches a caseless script the same as any other: `마틴 킹, MA` and `田中 太郎, MA` each hold TWO pre-comma name words, so the structure flips and both read suffix 'MA' — a PARITY RESTORATION, not a fresh deviation (1.4.0 read both as suffix on the wheel), with the Hangul/Han surname split then running positionally over what the comma leaves (`마틴 킹` → given '틴', middle '킹', family '마'; `田中 太郎` → given '太郎', family '田中'), unrelated to this design and #271/#272's pre-existing behavior. `毛泽东, MA` does not move — one pre-comma token, the count never reaches two. Pinned `a_caseless_script_wrote_no_contrast_hangul` and `a_caseless_script_wrote_no_contrast_japanese` (tests/v2/cases.py), both `tolerated=True`. -- 2026-09-14, corrected 2026-09-17 and 2026-09-18 — WHERE THE FACT LIVES, AND WHY THERE. The comma forms need the class decision at SEGMENT time, two stages before the own-words span is computable from marker TAGS. Four options were measured on a runtime probe (baseline 410.00 frames): segment computing it with a direct marker-head walk, +15; the same re-running the whole marker pass, +16; the fact staying in classify while classify RE-DECIDES `structure`, +0 but refused ON ORDERING — `structure` is read between segment and classify by `_script_segment.py`, which returns early on FAMILY_COMMA, so a flip decided in classify would leave the surname-split gate un-run on a name that turns out to have a suffix comma (measured: `Smith 김민준씨, MA` reads given 'Smith', middle '김', family '민준', suffix '씨, MA' on this tree — the figure first written here, given 'MA' with family 'Smith 김민준씨', was the reading before the flip moved to segment, re-measured 2026-09-18 — and a DIFFERENT wrong split again with the flip decided before script_segment — two stages disagreeing about one parse is what mechanisms.md#PIPELINE-STAGE-CONTRACTS exists to forbid); the marker pass and the fact MOVING to segment, +8, refused because killing the duplicated fold means carrying it on the state through CJK splits, a new bug surface in the one stage that changes token COUNT. SHIPPED: one shared helper, `_pieces.own_words(tokens, comma_offsets, markers, marker_tags=None)`, called by classify always (passing its own tag map) and by segment only when a comma form could turn on the fact. Every figure below is a TOTAL against `1f78befd` (the merge base this whole arc branched from), re-measured 2026-09-18 rather than accumulated per-commit: a `sys.setprofile` harness shaped like `tools/perf/call_count.py`'s `calls_for` (one warm-up parse, mean of 50, module-level `nameparser.parse`, run as a SCRIPT FILE outside either checkout so neither run's `sys.path[0]` masks the other — a `-c` one-liner instead would let the invoking shell's cwd shadow the older snapshot and read the current tree twice), against a `git archive 1f78befd nameparser` extraction on one side and this tree's own editable install on the other, same resolved interpreter both sides. Reference name (`tools/perf/call_count.py`'s own): 411 → 412 (+1). A comma-less, non-trigger control, `John Smith`: 175 → 176 (+1) — segment's lazy `own_words` call never fires without a comma, so this is classify's own always-on share of the cost. Comma names: a non-trigger, `Smith, John`: 203 → 207 (+4); `John Smith, Jr.`: 229 → 233 (+4); `Smith, John, Jr.`: 253 → 257 (+4) — every non-trigger comma name pays the SAME +4 total, dominated by classify's shared span cut plus the cheap case-free candidacy tests the three reading sites each add, not by the lazy fact itself (which these names never force). A genuine trigger, `Smith, MA`: 203 → 225 (+22) — the lazy fact plus its three readers' own work once a real candidate is found. (An earlier draft of this paragraph reported "+1 to +2 on non-trigger comma names" and "+19 to +20" on `Smith, MA`; those were PER-COMMIT deltas from the fix-round commentary above, accumulated inexactly rather than measured as one total, and are superseded by the totals here.) `own_words` shares its exact span-cutting walk with classify's own cut through a second shared function, `_vocab.tag_marker_runs` (moved out of `_classify.py` so `_pieces.py`'s own layering rule — it may import `_state` and `_vocab` only — is not broken): both call sites run the identical walk over the identical inputs, including `comma_offsets`, so a marker phrase like `z domu` is bucketed the same way at a comma either side. A corpus probe over the 1174 corpus entries confirms 0 cut differences and 0 verdict differences between the two call shapes (an earlier, text-only approximation that predated sharing `tag_marker_runs` differed on the CUT for 2 names — `Anna z Nowak`, `Anna z (domu) Nowak`, where `z` opens the phrase entry `z domu` but no run completes — while agreeing on the BOOLEAN for all 1174 even then). WITH SWITCH B ON, a genuine caps candidate pays more, recorded rather than chased to zero since nothing gates it — re-measured 2026-09-18 with `Parser().parse` on both sides of the on/off comparison (the convention `tests/v2/test_policy.py`'s own test uses), against this tree's own default: `Smith, John` 206 → 213 (+7), `Smith, XYZ` 205 → 245 (+40). Recompute (`tools/perf/call_count.py`'s own `calls_for` counts only its fixed reference name, so a chosen comma name needs the same shape spelled inline): `uv run python -c "import sys; from nameparser import Parser, Policy -def calls_for(fn, name, n=50): - fn(name); calls=0 - def counter(frame,event,arg): - nonlocal calls - if event=='call': calls+=1 - sys.setprofile(counter) - try: - for _ in range(n): fn(name) - finally: sys.setprofile(None) - return calls/n -off=Parser(); on=Parser(policy=Policy(unlisted_caps_suffixes=True)) -for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_for(on.parse, n))"` (both cheaper than an unconsolidated spelling would be, after the caps predicate was consolidated into one shared function, `_vocab.caps_shape_candidate`, called from classify, `ambiguous_class_candidate` and `_segment.py`'s run test alike — the usual "a shared call costs the default a frame" objection does not apply, because every caller's own first conjunct is `policy.unlisted_caps_suffixes` itself, False by default; `tests/v2/test_policy.py`'s own dated comment carries the same two figures and the recipe for `tools/perf/call_count.py` reference readers). The band never moved at either policy — `test_benchmark.py` gates only the comma-less reference name, which no comma-path cost can reach — and `_CALL_BASELINE` is untouched throughout. +- 2026-09-14, corrected 2026-09-17 and 2026-09-18 — WHERE THE FACT LIVES, AND WHY THERE. The comma forms need the class decision at SEGMENT time, two stages before the own-words span is computable from marker TAGS. Four options were measured on a runtime probe (baseline 410.00 frames): segment computing it with a direct marker-head walk, +15; the same re-running the whole marker pass, +16; the fact staying in classify while classify RE-DECIDES `structure`, +0 but refused ON ORDERING — `structure` is read between segment and classify by `_script_segment.py`, which returns early on FAMILY_COMMA, so a flip decided in classify would leave the surname-split gate un-run on a name that turns out to have a suffix comma (measured: `Smith 김민준씨, MA` reads given 'Smith', middle '김', family '민준', suffix '씨, MA' on this tree — the figure first written here, given 'MA' with family 'Smith 김민준씨', was the reading before the flip moved to segment, re-measured 2026-09-18 — and a DIFFERENT wrong split again with the flip decided before script_segment — two stages disagreeing about one parse is what mechanisms.md#PIPELINE-STAGE-CONTRACTS exists to forbid); the marker pass and the fact MOVING to segment, +8, refused because killing the duplicated fold means carrying it on the state through CJK splits, a new bug surface in the one stage that changes token COUNT. SHIPPED: one shared helper, `_pieces.own_words(tokens, comma_offsets, markers, marker_tags=None)`, called by classify always (passing its own tag map) and by segment only when a comma form could turn on the fact. COST, as totals against `1f78befd` (the merge base this arc branched from), re-measured 2026-09-18 with a `sys.setprofile` harness shaped like `tools/perf/call_count.py`'s: the reference name 411 → 412 and `John Smith` 175 → 176, every non-trigger comma name +4 (`Smith, John` 203 → 207, `John Smith, Jr.` 229 → 233, `Smith, John, Jr.` 253 → 257), the trigger `Smith, MA` 203 → 225, and with switch B on `Smith, John` +7 and `Smith, XYZ` +40 (tests/v2/test_policy.py's dated comment carries those two and its recipe); the benchmark band and `_CALL_BASELINE` never moved (PR #530). `own_words` shares its exact span-cutting walk with classify's own cut through a second shared function, `_vocab.tag_marker_runs` (moved out of `_classify.py` so `_pieces.py`'s own layering rule — it may import `_state` and `_vocab` only — is not broken): both call sites run the identical walk over the identical inputs, including `comma_offsets`, so a marker phrase like `z domu` is bucketed the same way at a comma either side. A corpus probe over 1174 entries finds 0 cut and 0 verdict differences between the two call shapes. The caps predicate is one shared function, `_vocab.caps_shape_candidate`, called from classify, `ambiguous_class_candidate` and `_segment.py`'s run test alike — the usual "a shared call costs the default a frame" objection does not apply, because every caller's own first conjunct is `policy.unlisted_caps_suffixes` itself, False by default. NO MECHANISMS ENTRY IS OWED, and here is why: the promoted fact is STATE-OFFSET-CHANNELS' "a pre-token fact is recorded on the ParseState and consulted later" applied to a name-level boolean rather than to offsets, and `ParseState.order` is the precedent that entry and ONE-PREDICATE-PER-QUESTION both already name. That entry's own sentence — "recorded as offsets … or by presence alone where the fact is name-level" — already covers it; the one edit made there is to name `one_case` as the third channel. - 2026-09-15 (Derek) — ACCEPTED, NOT REPAIRED, TWO SHAPES. (i) `Freiherr von Berg MA` MOVES: it was listed as unreachable by the peel and as staying fix(#424)'s, and measured, the caps lean makes it reachable, because the lean ends P2's chain with no words to spare where the count would not. Today title 'Freiherr', family 'von Berg MA'; designed title 'Freiherr', family 'von Berg', suffix 'MA'; 1.4.0 first 'von Berg', last 'MA'. Three readings exist and the designed one ships; it moves a rules/P2 example line and re-scopes fix(#424)'s ledger rule, whose `fields` gains `suffix`. (ii) `abdul Smith Jr Ma` reads `Jr` as a MIDDLE NAME: the surname lean breaks the peel AT `Ma`, so the unambiguous `Jr` in front of it is never reached. Today given 'abdul', family 'Smith', suffix 'Jr Ma'; designed given 'abdul Smith', middle Jr, family 'Ma'. One case row rather than real data, but the shape is real — a name-leaning ambiguous acronym BLOCKS the peel of a genuine suffix behind it — and the walk stopping at the declined pick is the rule, not a bug the peel owes a repair. -- 2026-09-18 — THE BLAST RADIUS IS THE GATE'S OWN COUNT, NOT A PREDICTION. Measured at 2.3.0 (`uv run python tools/differential/compare.py`, the default baseline, the previous release), and RE-MEASURED after the review round later the same day and again after the verification round that followed it, which is why several sets of figures carrying one date would otherwise sit here: corpus 1234 names, intentional diffs 51, unexplained 0, radar unclassified 7 — SUFFIX_OR_NAME rows the gate reports but no rule yet claims, named so a release note is never quantified from the intentional count alone. The round added TEN corpus names, not the eight first written here (corrected in the verification round of 2026-09-18): `Doe, John X.Y.Z.`, `Dr. 김민준씨, MA`, `John Smith, MD, Ma`, `John Smith, MD, R.A.I.`, `John de Ma`, `Jo김민준씨, DO`, `Kim김민준씨, MA`, `STEVEN HARDMAN, MD, DO, DDS`, `anh van mc`, `毛泽东 MA` — case rows and rules.md example lines both. The verification round itself then added THREE more (`Anh van Do`, `Doe, John MA`, `John Smith, MD, XYZ`, all rules.md boundary examples), taking the gate's count at this baseline from 1231 to 1234. Recompute the delta rather than trusting either figure: load every `tools/differential/corpus*.jsonl` through compare.py's own `_load_entries`, dedup by name in file order, and diff that list against the same list built from `git show :tools/differential/corpus*.jsonl`. That by-name count runs four below the gate's at the 2.x baselines, the gate deduping by (name, order) and four entries carrying a declared order; the DELTAS agree, which is what a "the round added N" sentence is claiming. The round also added a FOURTH rule, `fix(#289/#516) the glued CJK honorific peel reads the case lean` (3 composed CJK names: `Dr. 김민준씨, MA`, `Jo김민준씨, DO`, `Kim김민준씨, MA` — radar-tier by demotion and classified anyway, since a radar diff this arc intended is one a release note may be written from). The four rules carry 20 + 9 + 9 + 3 = 41 of the 51, `John de Ma` having joined the first. The pre-round reading of this paragraph, kept for the shape of the argument rather than the digits: `fix(#289) a written case contrast decides a bare ambiguous acronym` (19 names, incl. `Davis Royce, Ed`, `Freiherr von Berg MA`, `Jack MA`, `Jack Wei Ma`, `John Prof. MA`, `John Smith Ma`, `John Smith, Ed`, `John Smith, MA`, `John Smith, Ma`, `John van der Berg Ma`); `fix(#516) an unlisted dotted acronym is read by position` (9: `Jack X.Y.I.`, `John Smith Q.W.E.R.T.`, `John Smith X.Y.Z.`, `John Smith, A.B.`, `john smith x.y.z.`, `John Smith B.Tech.`, `John Smith C.H.A.`, `John Smith E.S.Q.`, `Smith, E.S.Q.`); `fix(#289/#516) the ambiguous credential class reports at slots that were silent` (9: `John Smith J.u.n.i.o.r.`, `Royce, Ed`, `Smith, A.B.`, `Smith, Ma`, `J.A. K.D.`, `Jack X.Y.Z.`, `John Smith R.A.I.`, `Smith Jr., A.B.`, `Steven Hardman, MD, DO, DDS`). This is LARGER than the plan's original "11 role + 4 report-only" prediction at the default policy — entirely by corpus growth (this arc's own commits admitted the case-row texts above as contract-tier names) and by two mechanisms the runtime prototype that produced that figure predated (`ambiguous_class_candidate`'s existence and the family-comma report's shape-tag branch), never by a wider rule. Recompute with the command above and read the count the gate returns; the numbers here are a snapshot dated 2026-09-18, not a promise. +- 2026-09-18 — THE BLAST RADIUS IS THE GATE'S OWN COUNT, NOT A PREDICTION. Measured 2026-09-18 at 2.3.0 with `tools/differential/compare.py` (re-measured after the review and verification rounds): corpus 1234 names, intentional diffs 51 (41 under this arc's four fix rules), unexplained 0, radar unclassified 7; larger than the plan's "11 role + 4 report-only" prediction by corpus growth and two mechanisms the prototype predated, never by a wider rule (PR #530). - 2026-09-18, WHOLE-PR REVIEW ROUND — WHAT IT MOVED, MEASURED BEFORE AND AFTER. Ten findings; the five that changed behavior are here with the shape each was found by, since the shape is what generalizes. (1) A LOST REPORT. Where the prefix chain absorbed an acronym the S2 peel had DECLINED, nobody emitted the flag S2 promises for either reading: `assign` reports from `peel_trailing`'s picks and a pick reaches it only as a LONE piece, so the token it would have reported on no longer existed. Before: `John van der Berg Ma`, `John de Ma`, `Dr. John van Smith Ma`, `John van Smith Ma Jr.` and `John Smith Mc Ma` gave their family names with `ambiguities` empty; after, each names `suffix-or-name` once and no role moves. The emitter is at the chain's own merge and declines where the PARTICLE run took the word (`anh van do`, `anh van mc` — P6's fork, not S2's, and 1.4.0 read both silently). Recompute: parse the five names and read `ambiguities`. (2) TWO STAGES ASKING ONE QUESTION WITH DIFFERENT INPUTS. `_script_segment` asked `is_wholly_suffix` of the post-comma run WITHOUT the `one_case` fact `segment` had recorded one stage earlier, so `Kim김민준씨, MA` read family `Kim김민준씨`, suffix `MA` (씨 unpeeled) while `Kim김민준씨, PhD` read family `Kim김민준`, suffix `씨, PhD` — one name, two credential spellings, two divisions. After: both peel. `Jo김민준씨, DO` and `Dr. 김민준씨, MA` move with them; `김민준씨, MA` does NOT, being wholly one case (Hangul is caseless and `MA` contrasts with nothing), which is the doctrine holding rather than an exception. rules/W3's claim that the two stages differ "in what else they require, not in what they ask of the words" was false until this and is true now. (3) A REPORT ABOUT THE PARSER'S OWN NEW READING. rules/S3's narrow retirement moved `R.A.I.`, `X.Y.I.` and `J.u.n.i.o.r.` from the vocabulary verdict into the shape class, and `segment`'s tail-segment test reads `is_wholly_suffix`, which is blind to that class — so `John Smith, MD, R.A.I.`, `John Smith, Ph.D., R.A.I.` and `John Smith, MD, J.u.n.i.o.r.` gained a `comma-structure` flag 2.3 did not raise. After: quiet, at the default. The test is narrowed to the BY-SHAPE half deliberately — a LISTED member reaches that reading through the case lean, and `STEVEN HARDMAN, MD, DO, DDS` (one case, nothing leans) keeps its flag as the lean's own recorded negative control; `John Smith, MD, Ma` keeps it too. With `unlisted_dotted_suffixes=False` the flag stands, the token being name material there. C2 is amended to state all of this. (4) THE WORDLIST ROSTER. `_vocab.caps_shape_candidate`'s exclusion named ELEVEN of `_lexicon._VOCAB_FIELDS`' thirteen: `Lexicon.default().add(surnames={"dupont"})` with the caps switch on still read `Jean Pierre DUPONT` as given `Jean`, family `Pierre`, suffix `DUPONT` — a caller listing a word as a SURNAME and getting a credential, which is this switch's own worst failure arriving through the one wordlist that says "family name". After: family `DUPONT`, no report. The roster is derived from the field tuple, with a per-field test; `honorific_tails`, the other omission, was already excluded transitively (⊆ `suffix_words` by Lexicon invariant) and joins for completeness. (5) A LISTED MEMBER READ BY SHAPE. A caller may list a DOTTED entry in `suffix_acronyms_ambiguous` ('a.b'), and the whole token then matches that set while `suffix_as_written`'s period-free acronym lookup ('ab') misses it — so the chunk view reached classify's shape branch, `SHAPE_ACRONYM_TAG` rode along, and `_pieces.listed_lean` (which declines wherever that tag is) silenced the caller's own case lean: `Jack A.B.` read family where `Jack MA` reads suffix. After: suffix `A.B.`, the lean applying. Nothing default moves, the shipped ambiguous vocabulary carrying no periods. Also in the round and behavior-neutral: `Policy.unlisted_dotted_suffixes` and `unlisted_caps_suffixes` moved to the END of `Policy` and `PolicyPatch` — inserted mid-class, they had re-bound every positional argument from `strip_emoji` on, which is API for a dataclass that is not `kw_only`; the bool-validation roster is derived from `dataclasses.fields`, the hand-written twin in the tests having named neither new flag; `ambiguous_class_candidate` lost an optional `one_case` parameter no production caller ever passed, and with it a caps branch that answered False for every name the library has parsed; and `one_case` is REQUIRED on the four piece predicates that own it. - 2026-09-18 — THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT, kept because the failure mode is this log's recurring one. A review round called THREE `one_case` plumbing sites in `_group.py` inert and proposed removing them. Measured with a runtime wrapper over 1,642 names under six policies — the default, both family-first orders, strict commas, and each 2.4 switch flipped — 9,852 parses: the maiden walk's PAIR is inert, 0 parses move, and it is inert BY CONSTRUCTION rather than by corpus luck (`numeral_only` answers off `peeled.numeral`, and the numeral fork is decided before the peel reads a lean). The chain-tail measure is not: dropping it there moves 18, on `John van der Berg Ma`, `John de Ma` and `Freiherr von Berg MA` under every one of the six. Both figures are at the call sites. The population is the DISTINCT union, in file order, of every `tools/differential/corpus*.jsonl` entry, every `tests/v2/cases.py` text, and `tests/test_variations.TEST_NAMES` with the three comma permutations `test_variations_of_TEST_NAMES` builds (no-comma, family-comma, and suffix-comma where the name has a suffix), empty strings dropped; the wrapper patches the call sites in a copy of the module source and rebinds `_pipeline.STAGES`, which binds stage functions at import and would otherwise leave the mutation inert. This entry first carried 2,338 names and 14,028 parses, which that recipe does not reproduce by any reading of it — the distinct union is 1,642, summing the three sources' distinct counts gives 2,202, and not deduping at all gives 2,690 — so both figures were re-measured on this tree in the verification round of 2026-09-18 and restated here and at the call sites. The FINDING was unharmed: 0 and 18 reproduce exactly, on the same three names, under all six policies, and the control that drops all four sites at once moves those same 18, which is what shows the wrapper can report a difference at all. The argument was right for two sites out of three and would have taken a third with it. -- 2026-09-18, VERIFICATION ROUND — THE REPORT INVENTORY AN UPGRADER SEES, 2.3.0 → TREE, AND ITS RECIPE. docs/release_log.rst's comma-report bullet claimed TWO reports go away and named `Steven Hardman, MD, DO, DDS` and `John Smith, MD, R.A.I.`. The second is false and was never measured against the wheel: `John Smith, MD, R.A.I.` reports NOTHING at 2.3.0, having carried a `comma-structure` flag only inside this arc's own development, where rules/S3's narrow retirement raised it and the C2 amendment quieted it again. Over the whole corpus exactly ONE report is lost, and one more is SWAPPED. Measured: LOST — `comma-structure` ×1 (`Steven Hardman, MD, DO, DDS`), `given-or-family` ×2 (`Jack X.Y.I.`, and `e and e`, which is PR #527's conjunction change and not this arc's). GAINED — `suffix-or-name` ×37, `conjunction-or-initial` ×9 (#527's again), `given-or-family` ×2 (`Jack MA`, `Jack MA.`). So `Jack X.Y.I.` is the only name whose report CHANGED KIND on this arc's account, `given-or-family` → `suffix-or-name`, and the bullet now names it as the swap rather than counting it among the losses. Recompute: collect the corpus glob through compare.py's `_load_entries` deduped by name, parse each name on the tree and on the 2.3.0 wheel (a PEP 723 `uv run --no-project` script pinning `nameparser==2.3.0`, per AGENTS.md's warning that a same-tree comparison reports parity), and take the per-name multiset difference of the `ambiguities` kind values in both directions. The kind counts are multisets, not name counts — `e and e` contributes two `conjunction-or-initial` gains on its own — which is why a bullet quantified from the gate's per-rule name lists would not answer this question. +- 2026-09-18, VERIFICATION ROUND — THE REPORT INVENTORY AN UPGRADER SEES, 2.3.0 → TREE, AND ITS RECIPE. Over the corpus deduped by name, against the 2.3.0 wheel, one report is lost (`comma-structure` on `Steven Hardman, MD, DO, DDS`) and one changes kind (`Jack X.Y.I.`, `given-or-family` → `suffix-or-name`), so the release bullet's second named loss, `John Smith, MD, R.A.I.`, which reports nothing at 2.3.0, was false and was corrected; per-name multiset difference of `ambiguities` kinds in both directions, recipe in PR #530. - 2026-09-23 #492 — THE DEFERRED QUESTION the 2026-09-14 predicate paragraph and decisions/S4's 2026-09-19 #533 bullet name — whether a cased suffix token counts as case evidence — IS ANSWERED FOR RENDER ONLY. R5's gate now leaves the suffixes out (decisions/R5, 2026-09-23); the parse-time fact this section rests on keeps counting the judged token exactly as written above, so `Jack MA` still leans credential and `DOE, JANE nee Smith Ma` still reads suffix `Ma`. The split is decided and recorded under R5, with the shapes it makes visible there (`jack MA` at this slot, `john e jones III` at P8's). - 2026-10-17 #682 — SPLIT FROM decisions/S2 with the rule. The entries above were written under decisions/S2 while this rule was part of rules/S2, and moved here unchanged but for pointers the move broke; where one says rules/S2 it means the rule as it then stood, which held this one. - 2026-10-19 #760 — MOVED FROM the chain emitter's code comment in `_group.py`, the cost record the entries above did not hold. The emitter is written inline against the tags with no new walk — `pieces[j - 1]` is the last piece the merge is about to claim, one index and one frozenset test — so the ordinary chained name (`de la Vega`) pays no frame for it. Its `not is_prefix_piece(last, ...)` floor is tested LAST, behind the `isdisjoint` (a C call, no frame), because it costs a frame (two until group's predicate closures were inlined): asked ahead of the tag test, through the closure, it moved the reference name from 412 to 414 frames, every chained name paying for a question only an ambiguous acronym makes interesting (measured in the #289/#516 review round, 2026-09-18, commit 59d8f38a, interpreter unrecorded; the reference row has since moved for other reasons, decisions/parse-cost). diff --git a/docs/design/decisions/S7-company-of-an-unambiguous-credential.md b/docs/design/decisions/S7-company-of-an-unambiguous-credential.md index 5c2bc846..5d935470 100644 --- a/docs/design/decisions/S7-company-of-an-unambiguous-credential.md +++ b/docs/design/decisions/S7-company-of-an-unambiguous-credential.md @@ -6,9 +6,9 @@ ACCEPTED LIMITS, each keeping the member a name word, rules/S2's Accepted block and case rows: the merged `Ph. D.` is outside the no-comma walk (`John Smith Ph. D. MEng` family 'MEng'; the comma and given-slot spellings do anchor it, `Smith, Ph. D. MEng` and `Doe, Jane Ph. D. MEng` reading suffix 'Ph. D. MEng'), a title between the credential and the member breaks the run (`John Smith PhD Prof. Ma`), and a particle member P2 has chained before the peel is out of reach (`John Smith PhD Do Do` family 'Do Do'; `Smith, PhD Do Ma` given 'PhD', middle 'Do Ma'). THE TWO-INPUT CHECK. tests/v2/test_properties.py's M2 clause-agreement walk asks every parse it takes both directions of the company clause: a listed member behind a qualifying credential reads as one, and a member read as a credential because of the word in front has that word in the suffix too. Recorded negative controls, from that test's docstring: 48 failing parses with the anchor off, 30 with the walk's leading piece allowed to anchor, 0 on this tree; re-run 2026-09-28 after (9) and (10), unmoved. The walk holds no comma head that opens its part with a dual, so (10) is pinned by case rows and a unit test instead (`a_dual_opening_the_given_part_turns_the_anchor_off`, `a_dual_opening_the_given_part_silences_a_later_degree`, test_pieces' `test_a_dual_in_the_leading_title_run_turns_the_anchor_off`). REVERSED BY THIS ENTRY rather than edited: decisions/S6's 2026-09-15 ACCEPTED item (ii) (the bullet after this entry says so); the three "wrongly moved" reversals decisions/S9's 2026-09-14 paragraph THE RUN IS THE CAPS CLASS'S ALONE records — `John Smith, Ed Ma`, `John Smith, ma do` and `John Smith, X.Y.Z. A.B.` are credential runs now by C1's own count (rules/C4 since #728) under every policy that keeps their words in the class (with `unlisted_dotted_suffixes` off, `X.Y.Z.` and `A.B.` are no class members and the third keeps the listing form), while that paragraph's narrowing of the CAPS run test stands; and #540's two pending questions (the `suffix-acronym-collisions` bullet of this date). - MEASURED 2026-09-28, the #544 tree against its parent e10e83b4, py3.11, after (9) and (10); the 2026-09-27 figures this paragraph first carried were taken before them and are superseded. Recompute: every run of one to three words from {PhD, MD, Jr, MA, Ma, MEng, M.Eng., LAc, Ed, Do, ba, MS, V, Prof.} behind each of `Wang {}`, `John Smith {}`, `John Quincy Smith {}`, `Dr. John Smith {}`, `John Smith, {}`, `John Quincy Smith, {}`, `Smith, {}`, `Doe, Jane {}`, `Jane Doe nee Smith {}`, `Doe, Jane nee Smith {}` and `Jan van der Berg {}`, each written as given, lower-cased and upper-cased and deduped, parsed under the three name orders at otherwise default policy, the seven role fields and the ambiguity kinds compared against the same grid parsed by e10e83b4. THE DETECTOR, which both the invariants and the attribution read, looks only at the run's own words, compared case-free: {phd, md, jr, ms, m.eng.} are the unambiguous credentials, {ma, meng, lac, ed, do, ba} the members; a member is ANCHORED where an unambiguous credential stands in front of it in the run with nothing but members between, except that `ms` or `md` opening a comma part anchors nothing; and a run word is IN A NAME FIELD where it appears among the space-split words of given, middle, family or maiden more often than the run's unconstrained occurrences of it account for. The two invariants count parses with an anchored member in a name field, and with an unambiguous credential in one. THE ATTRIBUTION puts each moved parse in ONE class, the first that holds, in this order: THE ANCHOR, where the tree's two violation counts are each no higher than the parent's and one is lower; THE CHUNKED `M.Eng.`, where the run holds `M.Eng.`; A LONE `v`, where re-running the first test with `v` counted as an unambiguous credential makes it hold; THE C1 RUN (rules/C4 since #728), a comma shape with two or more name words before the comma whose run is only credentials and members, at least one a member; and anything left unattributed. 254,496 parses over 84,832 texts; 89,825 move (29,943 texts, 19,123 of them with a field move): 39,261 the anchor, 14,676 the C1 run, 35,696 the chunked `M.Eng.` (32,472 of those only losing the report it raised as a pick), 180 a lone `v` (`john smith, v phd ma`, suffix 'v phd ma'), and 12 unattributed, the `Smith, MA PhD ba` parses above. Clean-to-clean — no run word in a name field on either side — 23,454 move, every one `M.Eng.`'s dropped report, and none elsewhere. The two invariants, 0 NEW violations of either: an anchored member in a name field, 29,199 → 1,530 (the residue is a particle member P2 has chained, a dual inside a leading title run, `smith, prof. md ma`, and the member behind a credential in a part a dual opens, `Smith, MD PhD Ma`, which (10) keeps a name); an unambiguous credential in a name field, 36,675 → 4,734 (the residue is an interior single-letter `V`, a non-final `Prof.`, the `PhD` a comma part keeps as its given name when a `V` or a title follows it, and the same `PhD` behind a dual that opens the part, (10) again). Against the tree before (9) and (10), the same grid moves 5,076 parses: 4,536 (1,512 texts, every one a comma shape whose part reads wholly as credentials) gain a `suffix-or-name` report and move nothing else, and 540 (180 texts, every one `Smith, ` then `MD` or `MS` opening the run, in each case form) return to e10e83b4's reading, fields and reports alike, from a silent suffix reading of the whole part. The differential corpora at the parent (1,418 names, three orders) move four names, every one intended: `John Smith, PhD MEng`, `john smith, phd meng`, `Wang M.Eng.`, `abdul Smith Jr Ma`. Frames per parse through `Parser().parse`, py3.11: the reference band is unmoved (370/407, `uv run python tools/perf/call_count.py`), and so is every ordinary name measured (`Smith, John Quincy` 260, `Doe, John MA` 283, `Doe, Jane nee Smith PhD` 338) except a comma name with two or more words on each side of the comma, which enters the C1 run test and pays `run_word_fold`'s one call before its first name word ends it (`Doe Smith, Jane Q.` 317 → 318, `Garcia Lopez, Maria Jose` 330 → 331, measured 2026-09-28); the new questions cost where they are asked — `John Smith Ma` 244 → 250, the anchor pass that finds nothing; `John Smith, MA Jr` 355 → 382, the case fact forced to see the capitals settle the run; `John Smith PhD MEng` 286 → 326; `Smith, MD PhD Ma` 349 → 362, the given slot's pass that (10) leaves with nothing to find — and `John Smith, PhD MEng` gets cheaper, 358 → 314, as does `Smith, PhD Ma`, 286 → 271. + MEASURED 2026-09-28, the #544 tree against its parent e10e83b4, py3.11, after (9) and (10); the 2026-09-27 figures this paragraph first carried were taken before them and are superseded. Recompute: every run of one to three words from {PhD, MD, Jr, MA, Ma, MEng, M.Eng., LAc, Ed, Do, ba, MS, V, Prof.} behind each of `Wang {}`, `John Smith {}`, `John Quincy Smith {}`, `Dr. John Smith {}`, `John Smith, {}`, `John Quincy Smith, {}`, `Smith, {}`, `Doe, Jane {}`, `Jane Doe nee Smith {}`, `Doe, Jane nee Smith {}` and `Jan van der Berg {}`, each written as given, lower-cased and upper-cased and deduped, parsed under the three name orders at otherwise default policy, the seven role fields and the ambiguity kinds compared against the same grid parsed by e10e83b4. THE DETECTOR, which both the invariants and the attribution read, looks only at the run's own words, compared case-free: {phd, md, jr, ms, m.eng.} are the unambiguous credentials, {ma, meng, lac, ed, do, ba} the members; a member is ANCHORED where an unambiguous credential stands in front of it in the run with nothing but members between, except that `ms` or `md` opening a comma part anchors nothing; and a run word is IN A NAME FIELD where it appears among the space-split words of given, middle, family or maiden more often than the run's unconstrained occurrences of it account for. The two invariants count parses with an anchored member in a name field, and with an unambiguous credential in one. THE ATTRIBUTION puts each moved parse in ONE class, the first that holds, in this order: THE ANCHOR, where the tree's two violation counts are each no higher than the parent's and one is lower; THE CHUNKED `M.Eng.`, where the run holds `M.Eng.`; A LONE `v`, where re-running the first test with `v` counted as an unambiguous credential makes it hold; THE C1 RUN (rules/C4 since #728), a comma shape with two or more name words before the comma whose run is only credentials and members, at least one a member; and anything left unattributed. Results: 254,496 parses over 84,832 texts, 89,825 move (29,943 texts), every one attributed but the 12 `Smith, MA PhD ba` parses above, with 0 new violations of either invariant (an anchored member in a name field 29,199 → 1,530, an unambiguous credential in one 36,675 → 4,734); the differential corpora at the parent move four intended names; the reference frame band is unmoved, the new questions paying only where they are asked (PR #552). NO MECHANISMS ENTRY IS OWED: the anchor pass is ONE-PREDICATE-PER-QUESTION's `_pieces.credential_anchors`, which `segment_suffix_reading` reads inline for the frame budget under a keep-in-step note, and its linearity is the answer #531's trailing floor and #397's `_run_neighbours` already give — one forward pass per question, never a look-behind per member. tests/v2/test_benchmark.py's `credential_run` shape guards it: the per-member look-behind measured 15.8× for 4× the input at base 800, against 4.13–4.21× on this tree at every base (that module's own record). - ADDENDUM 2026-09-28 (Derek). (e) A PARTICLE ANCHORS NOTHING, a fifth boundary beside the four WHAT MAY ANCHOR lists: a word of both the particle and the unambiguous suffix vocabulary (`vd` and `mc` in the shipped lexicon; `do` sits in the ambiguous half, so it is a member and never an anchor) heads the family name behind it (P2 when decided, rules/P9 since 2026-10-19), and anchoring there split the family around a suffix — `Smith vd Ma, John` read family 'Smith Ma', suffix 'vd'; `Jan vd Ma` suffix 'vd Ma' and no family; `Smith Mc Ma, John` and `D. Mc Ba Ed, Smith` the same way. Each reads as e10e83b4 reads it again (family 'Smith vd Ma', 'vd Ma', 'Smith Mc Ma', 'D. Mc Ba Ed'). The exclusion is of the word in front only: a particle MEMBER behind a credential is still spoken for (`doe, jane v phd do` keeps suffix 'v phd do'), and a C1 run (rules/C4 since #728) holding the particle is C1's reading rather than the anchor's (`Jan Berg, vd Ma` reads suffix 'vd Ma', as `Jan Berg, vd` reads suffix 'vd'). Measured over the grid of MEASURED above with `vd` and `Mc` added to its words (127,578 texts, 382,734 parses), against the tree before this addendum: 9,426 parses (3,142 texts) move, every one holding `vd` or `Mc` directly in front of a member; 8,940 return to e10e83b4's reading exactly, and the other 486 all hold `M.Eng.`, whose own change is (3) — 406 of them take e10e83b4's fields and differ only in the pick `M.Eng.` no longer reports, and 80 take the fields e10e83b4 gives the same text with `PhD` in `M.Eng.`'s place (`Jane Doe nee Smith M.Eng. vd Ma` reads middle 'Doe M.Eng.', family 'vd Ma', as `Jane Doe nee Smith PhD vd Ma` reads middle 'Doe PhD'). The differential corpora move 0 of 1441 names, and the grid of MEASURED without the two words moves 0 parses, so every figure there stands. THE PASS IS ASKED ONLY WHERE A SUFFIX PIECE IS IN REACH: `_pieces.anchor_in_reach` walks back through the lone members in front of a declined member on tags alone and answers False wherever the pass must — the first piece past the members is no suffix piece, or there is none — and it is asked only before the pass exists, one lookup answering after that, so a run of members stays linear. An ordinary name ending in a declined member stops paying for the pass (frames, py3.11, before → after, e10e83b4 in brackets): `John Smith Ma` 250 → 246 (244), `Smith, John Ma` 285 → 279 (275), `Doe, Jane MA do` 374 → 369 (363), `Doe, John Q. Ma` 328 → 320 (316), `Jan vd Ma` 308 → 243 (238); a name the pass does read pays the test on top, `John Smith PhD MEng` 326 → 329 and `Doe, Jane PhD MEng` 359 → 362. On the no-comma peel the reach starts past the walk's leading piece, which never anchors — so a by-shape member, which reaches the pass only at the peel's second position (a lean of None with a word to spare is consumed first), finds nothing in reach, and the by-shape guard that stood there was deleted as unreachable. `segment_suffix_reading` calls `credential_anchors` (`first_kept=False`, a comma part's first piece being a run member like any other) in place of the inline copy NO MECHANISMS describes, and the keep-in-step note goes with it. The pass is built only where the reach test finds a suffix piece in front of a declined member, which no member opening the part has, and a part whose name word stands ahead of any member returns before one is asked: a plain comma name pays nothing (`Smith, John` 182, `Doe, John MA` 283, `Smith, J. Q.` 246, `Smith, Ed` and `Smith, Ma` 213, `Smith, Ed John` 270, all as before the call replaced the copy; built on every declined member instead, the last three paid 215, 215 and 273), a title in front pays the reach test alone (`Smith, Dr. Ma` 253 → 254), and a part the company decides pays three frames over the copy (`Smith, PhD MEng` 273 → 276, `Smith, PhD Ma` 271 → 274; `Smith, PhD Ma John` 330 → 335, its pass built for 'Ma' before 'John' ends the part). THE TWO-INPUT CHECK's second control reads 42, not 30, with no change to the property: the walk's leading piece is now held out of the pass twice on the no-comma peel, by `credential_anchors` and by the reach test, either alone suffices, and with both removed the by-shape heads the deleted guard held back (`PhD X.Y.Z.`) fail beside the listed ones; the first control is unmoved at 48. And the 2026-09-28 C2 bullet's "a TAIL segment, which assign reads wholly as suffixes" holds outside a maiden clause standing in it — `Jane Doe, PhD, Jr nee van Ma` keeps maiden 'van Ma' — which rules/C2's statement now says. The reach test's one-way exactness is held by tests/v2/pipeline/test_pieces.py's `test_anchor_in_reach_never_hides_an_anchor`, over the case table's texts and runs of one to three words behind two heads; its recorded negative control, the reach test reading a `vocab:suffix` token as no suffix piece, fails at 878 member positions. + ADDENDUM 2026-09-28 (Derek). (e) A PARTICLE ANCHORS NOTHING, a fifth boundary beside the four WHAT MAY ANCHOR lists: a word of both the particle and the unambiguous suffix vocabulary (`vd` and `mc` in the shipped lexicon; `do` sits in the ambiguous half, so it is a member and never an anchor) heads the family name behind it (P2 when decided, rules/P9 since 2026-10-19), and anchoring there split the family around a suffix — `Smith vd Ma, John` read family 'Smith Ma', suffix 'vd'; `Jan vd Ma` suffix 'vd Ma' and no family; `Smith Mc Ma, John` and `D. Mc Ba Ed, Smith` the same way. Each reads as e10e83b4 reads it again (family 'Smith vd Ma', 'vd Ma', 'Smith Mc Ma', 'D. Mc Ba Ed'). The exclusion is of the word in front only: a particle MEMBER behind a credential is still spoken for (`doe, jane v phd do` keeps suffix 'v phd do'), and a C1 run (rules/C4 since #728) holding the particle is C1's reading rather than the anchor's (`Jan Berg, vd Ma` reads suffix 'vd Ma', as `Jan Berg, vd` reads suffix 'vd'). Measured over the grid of MEASURED above with `vd` and `Mc` added (382,734 parses), against the tree before this addendum: 9,426 parses move, every one holding `vd` or `Mc` directly in front of a member, 8,940 returning to e10e83b4's reading exactly and the other 486 differing only by `M.Eng.`'s own change (3); the differential corpora and the grid without the two words move 0 (PR #552). THE PASS IS ASKED ONLY WHERE A SUFFIX PIECE IS IN REACH: `_pieces.anchor_in_reach` walks back through the lone members in front of a declined member on tags alone and answers False wherever the pass must — the first piece past the members is no suffix piece, or there is none — and it is asked only before the pass exists, one lookup answering after that, so a run of members stays linear. An ordinary name ending in a declined member stops paying for the pass (`Jan vd Ma` 308 → 243 frames, py3.11), and a name the pass does read pays the test on top (+3). On the no-comma peel the reach starts past the walk's leading piece, which never anchors — so a by-shape member, which reaches the pass only at the peel's second position (a lean of None with a word to spare is consumed first), finds nothing in reach, and the by-shape guard that stood there was deleted as unreachable. `segment_suffix_reading` calls `credential_anchors` (`first_kept=False`, a comma part's first piece being a run member like any other) in place of the inline copy NO MECHANISMS describes, and the keep-in-step note goes with it. The pass is built only where the reach test finds a suffix piece in front of a declined member, which no member opening the part has, and a part whose name word stands ahead of any member returns before one is asked: a plain comma name pays nothing (built on every declined member instead, `Smith, Ed`, `Smith, Ma` and `Smith, Ed John` paid 215, 215 and 273 against 213, 213 and 270), a title in front pays the reach test alone, and a part the company decides pays three frames over the copy. THE TWO-INPUT CHECK's second control reads 42, not 30, with no change to the property: the walk's leading piece is now held out of the pass twice on the no-comma peel, by `credential_anchors` and by the reach test, either alone suffices, and with both removed the by-shape heads the deleted guard held back (`PhD X.Y.Z.`) fail beside the listed ones; the first control is unmoved at 48. And the 2026-09-28 C2 bullet's "a TAIL segment, which assign reads wholly as suffixes" holds outside a maiden clause standing in it — `Jane Doe, PhD, Jr nee van Ma` keeps maiden 'van Ma' — which rules/C2's statement now says. The reach test's one-way exactness is held by tests/v2/pipeline/test_pieces.py's `test_anchor_in_reach_never_hides_an_anchor`, over the case table's texts and runs of one to three words behind two heads; its recorded negative control, the reach test reading a `vocab:suffix` token as no suffix piece, fails at 878 member positions. - 2026-09-27 (#544) — THE 2026-09-15 ACCEPTED ITEM (ii) IS REVERSED, and the bullet stands as it landed. `abdul Smith Jr Ma` reads given 'abdul', family 'Smith', suffix 'Jr Ma' — 2.3.0's reading — because the unambiguous 'Jr' in front of the Title-case 'Ma' anchors it (the entry above): the peel takes both, and P5's reserve, now seeing the family the join would take, declines the join. Item (i) stands. The case row is now `a_credential_in_front_anchors_a_declined_pick`, and rules/S2's Accepted block names the shapes that still keep the company out of reach. - 2026-09-27 (Derek), #544 — THE #531 PAIRING GAINS A SECOND EXCEPTION, and decisions/S5's CAPITALS DECIDE FOR `do` stands as it landed. That bullet let only a positive credential lean override P6's attachment at the given slot; an unambiguous credential IN FRONT of the member now overrides it as well, the degree being a second and stronger signal: `doe, jane v phd do` reads suffix 'v phd do' and reports `suffix-or-name` where it read family 'do doe' and reported P6's fork. The one-case record the pairing protects has nothing in front of its particle, so `NASCIMENTO, EDSON ARANTES DO` still reads family 'DO NASCIMENTO' and reports `particle-or-given`. - 2026-10-17 #682 — SPLIT FROM decisions/S2 with the rule. The entries above were written under decisions/S2 while this rule was part of rules/S2, and moved here unchanged but for pointers the move broke; where one says rules/S2 it means the rule as it then stood, which held this one. diff --git a/docs/design/decisions/S9-unlisted-words-that-join-the-ambiguous-class-by-shape.md b/docs/design/decisions/S9-unlisted-words-that-join-the-ambiguous-class-by-shape.md index 3569f6d0..f2592fca 100644 --- a/docs/design/decisions/S9-unlisted-words-that-join-the-ambiguous-class-by-shape.md +++ b/docs/design/decisions/S9-unlisted-words-that-join-the-ambiguous-class-by-shape.md @@ -12,15 +12,15 @@ - OUT OF SCOPE, each its own issue or entry: no tokenizer split (`J.R.R.`, `A.B.` stay single tokens); the trailing-period-only shape is NOT the signal (`John Smith Xyz.` → family, and AGENTS.md's trailing-abbreviation paragraph stands for bare words and sentence-final stops); no new `AmbiguityKind`; no locale packs; #492 stays deferred. NEITHER SWITCH REACHES THE v1 SHIM, and that is a consequence stated rather than repaired: `lenient_comma_suffixes` has no v1 manager either and `_build_snapshot()` simply does not translate it. Switch A is DEFAULT ON, so `HumanName` changes with it and a v1 caller cannot turn it off — the deliberate reading of the bridge, where the facade tracks the parser's defaults and 2.x behavior changes reach it (the #528 precedent). - 2026-09-18 — RULES.MD STATEMENTS NEVER NAME A POLICY FIELD, AND THIS ARC MADE THAT LOAD-BEARING. rules.md's own preamble binds statements to be implementation-free ("no stage names, no function names, no regexes"), and no existing statement names a `Policy` field either — C1 says "strict mode" and leaves `strict-comma-suffixes` to the annotation slot. S2's and S3's switch clauses follow that precedent rather than setting a new one: the prose describes `unlisted_dotted_suffixes` and `unlisted_caps_suffixes` as caller configuration and points at the example lines, and the ANNOTATIONS carry the exact field names (`unlisted_dotted_suffixes-off`, `unlisted_caps_suffixes-on`, registered in `tests/v2/rules_doc.py`'s `POLICIES` beside `strict-comma-suffixes`). Recorded here rather than as a silent edit to the preamble or to two rules' prose: if a future design wants a field named directly in a statement, that is a preamble amendment for someone to propose, not a convention this entry changes on its own authority. - 2026-09-18, corrected — `BUILD_RULES_CORPUS.PY` HARVESTING A POLICY-ANNOTATED-ONLY EXAMPLE IS LONG-STANDING, NOT NEW, AND THIS ARC ADDS TWO MORE INSTANCES OF IT. Its own docstring says "Policy annotations are deliberately ignored... a family-first-scoped example is simply one more name to diff" — an earlier draft of this bullet claimed that sentence was written when every annotated example had a default-policy sibling; measured false, at the commit that wrote it and on this tree alike. Recompute: parse every rule's examples with `tests/v2/rules_doc.py`'s own `parse_rules_doc`, group by `text` across the WHOLE document, and keep every text whose every appearance's annotation resolves (`resolve_annotation`) to kind `"policy"` rather than `"locale"`/`"gated_locale"` and never appears bare. That gives **19** pre-existing examples (`Mesnil de`, `Smith, de Mesnil Jean`, `de la Vega y Santos Juan`, `ibn Awf abdul Rahman`, `Nguyen Thi Van`, `Jane Smith (Nee)`, `Jane Smith (Nee) (Jones)`, `Garcia Juan Carlos`, `Nguyễn Thị Minh Khai`, `Del Toro`, `Anh Van Do`, `Beethoven Ludwig van`, `Berg Jan de Jr.`, `Garcia`, `Ménil Christophe de`, `Ménil de`, `Smith (Jones)`, `de Anke van y`, `van Berg Jan de`) plus this arc's own **2** (`Jean DUPONT`, `Jean Pierre DUPONT`, both `unlisted_caps_suffixes-on`, rules/S2) — **21** total on this tree. So the shape was accepted the day the docstring was written, not discovered here; what this arc adds is two more members, checked here for the reason the advice below names: `Jean Pierre DUPONT`'s DEFAULT reading (given Jean, middle Pierre, family DUPONT — measured; the annotated ON reading, given Jean, family Pierre, suffix DUPONT, is the only one rules.md states) never appeared in the doc before, and it produces no diff at any of the five baselines either way. Recorded per the same over-collection principle that keeps `(` and `.,` in the corpus (`tools/differential/README.md`'s provenance table): an annotated example's TEXT is a default-policy corpus name regardless of its annotation, full stop, and every one of the 21 — not only this PR's two — is a standing instance a reader should check for a default-policy diff on before assuming it free. -- 2026-09-18 — THE COMMA HARNESS AND THE FRAME BAND, AFTER THE ROUND. Unchanged at the default: 412/449 on the reference name (`uv run python tools/perf/call_count.py`), and the module-`parse` totals in decisions/S6's WHERE THE FACT LIVES entry hold exactly — `John Smith` 176, `Smith, John` 207, `John Smith, Jr.` 233, `Smith, John, Jr.` 257, `Smith, MA` 225. WITH SWITCH B ON the two figures recorded there each drop by one: `Smith, John` 213 → 212 and `Smith, XYZ` 245 → 244, `Parser().parse` on both sides as before. The cause is the dead caps branch leaving `ambiguous_class_candidate`: `segment`'s single-token call used to reach a `caps_shape_candidate` call that could only ever answer False, and now does not. Recompute with that entry's recipe. +- 2026-09-18 — THE COMMA HARNESS AND THE FRAME BAND, AFTER THE ROUND. Unchanged at the default (412/449 on the reference name, and the module-`parse` totals in decisions/S6's WHERE THE FACT LIVES entry hold exactly); with switch B on, `Smith, John` and `Smith, XYZ` each drop one frame, the dead caps branch having left `ambiguous_class_candidate` (PR #530). - 2026-09-30 (Derek), #563 — PAIRED INITIALS ARE THE EXCEPTION TO C1'S NAME-WORD COUNT, AND A FLIP ON THE DOTTED SHAPE ALONE IS SILENT. The 2026-09-14 (Derek) entry above, #516's DOTTED HALF IS A SWITCH, read `John Smith, A.B.` as suffix 'A.B.' on the count of two name words before the comma. The count cannot tell a given name and a family name from ONE surname written in two words, and the dotted shape's own most common member is a person's initials, so the unreleased tree read `García Márquez, G.J.` as given 'García', family 'Márquez', suffix 'G.J.', `De La Cruz, M.J.` with no given name at all, and `van der Berg, A.J.` as given 'van' — where 1.4.0 through 2.3.0 read each as initials. A particle check would have caught only the last two; `García Márquez` and `Lloyd Webber` look exactly like `John Smith`. What the parser CAN see is the token, so the line is drawn there: two single letters joined by a period, with or without one after the second (`_vocab.is_paired_initials`; `García Márquez, G.J` reads as `G.J.` does), read as the given name after the comma and report the fork, and only an UNAMBIGUOUS suffix word IN FRONT of them that is not also title vocabulary, or a second word the class admits by dotted shape in the same part, makes them the credential run (`John Smith, PhD X.Y.`, `John Smith, X.Y. P.Q.`). Three letters or more (`X.Y.Z.`) and any longer chunk (`B.Tech.`) read by the count as before. Three choices made with it (Derek, 2026-09-30): (i) FRONT ONLY — a word behind the pair does not speak, so `De La Cruz, M.J. PhD` keeps given 'M.J.' and `John Smith, A.B. PhD` reads given 'A.B.', the direction rules/S2's company clause already takes; a LISTED dotted word behind does not speak either (`John Smith, A.B. Ph.D.` → given 'A.B.'), the second-word exception being for words the class admits by shape; (ii) SILENT FLIP — a flip resting only on words the class admits by dotted shape reports nothing, `B.Tech.` included, because the only such word a reader takes for a name is a pair of initials, and the comma flips a pair only where a word beside it has already said otherwise; a LISTED member in the same part still reports the flip (`John Smith, PhD MEng`, `John Smith, X.Y.Z. MA`); (iii) THE COMMA SLOT ONLY — the trailing slot (`John Smith R.T.`), the given part's trailing slot after a family comma (`Doe, John R.T.`) and the maiden clause keep #516's reading and its report; #563 leaves the no-comma question open. A generational word in front speaks like any unambiguous suffix word (`García Márquez, Jr. G.J.` → suffix 'Jr. G.J.'), as S2's company clause has it — unless it is also a title, so `García Márquez, Sr G.J.` reads title 'Sr', given 'G.J.'. ACCEPTED: `García Márquez, G.J.R.` reads suffix 'G.J.R.' — initials are conventionally written apart, as separate words this rule never reads (`García Márquez, G. J. R.` gives given 'G.', middle 'J. R.'), and three letters run together are far likelier a credential. REVERSED BY THIS ENTRY rather than edited, for the dotted half at the first slot after a comma only: the #516 SWITCH entry's `John Smith, A.B.` → suffix and its "reports either way"; the 2026-09-14 NO NEW AmbiguityKind entry's "Every decision at these slots emits" SUFFIX_OR_NAME; and rules/C1's "A decision either way at this comma is reported" (rules/C11's since #728). ALSO FIXED: the trailing slot's report described a by-shape pick in the listed member's words ("written without periods is both a post-nominal"), false of `X.Y.Z.` on both counts; it now says the word is shaped like a post-nominal but listed in no vocabulary. - REVIEW ROUND, same day. The docs review found the first version let a word of both the title and the suffix vocabulary speak for the pair, since C1 (rules/C4 since #728) counts such a word opening the part as a suffix word: `García Márquez, Ms G.J.` read given 'García', family 'Márquez', suffix 'Ms G.J.' with NO report — #563's own defect, made silent — where 2.3.0 read title 'Ms', given 'G.J.'. The twelve title/suffix duals (`ms`, `md`, `sr`, `lt`, `sa`, `ra`, `vc`, `cpl`, `cpo`, `cpt`, `csm`, `sgm`) all reached it. Fixed by letting only a suffix word that is not title vocabulary speak: in front of a pair, a dual is the title of the given part the pair opens, as S2 already reads a dual in that part's title run, and `Ms G.J.`, `MD G.J.`, `Lt G.J.` now read title plus given as every release did. The title lookup runs only once a pair is met. The code review's frame count found the single-token path asking `ambiguous_class_member` a second time, +2 frames on every reporting comma name (`John Smith, MA`, 252 → 254, profiler call events); for a word past the candidate test LISTED is exactly "no period" (`ambiguous_class_member` declines any period), which is now asked inline. Measured after both fixes against master, same counter: `Smith, John` 183 and `John Smith, MA` 252 on both trees, `John Smith, X.Y.Z.` 295 → 286 (the report it no longer builds), and `García Márquez, G.J.` 338 → 350, the family-comma reading and its report costing more than the flip did. + REVIEW ROUND, same day. The docs review found the first version let a word of both the title and the suffix vocabulary speak for the pair, since C1 (rules/C4 since #728) counts such a word opening the part as a suffix word: `García Márquez, Ms G.J.` read given 'García', family 'Márquez', suffix 'Ms G.J.' with NO report — #563's own defect, made silent — where 2.3.0 read title 'Ms', given 'G.J.'. The twelve title/suffix duals (`ms`, `md`, `sr`, `lt`, `sa`, `ra`, `vc`, `cpl`, `cpo`, `cpt`, `csm`, `sgm`) all reached it. Fixed by letting only a suffix word that is not title vocabulary speak: in front of a pair, a dual is the title of the given part the pair opens, as S2 already reads a dual in that part's title run, and `Ms G.J.`, `MD G.J.`, `Lt G.J.` now read title plus given as every release did. The title lookup runs only once a pair is met. The code review's frame count found the single-token path asking `ambiguous_class_member` a second time, +2 frames on every reporting comma name (`John Smith, MA`, 252 → 254, profiler call events); for a word past the candidate test LISTED is exactly "no period" (`ambiguous_class_member` declines any period), which is now asked inline. Measured after both fixes against master: `John Smith, X.Y.Z.` 295 → 286 and `García Márquez, G.J.` 338 → 350 frames, plain comma names unchanged (PR #567). SECOND REVIEW ROUND, same day, on the first round's fix commit. (a) TWO PAIRS (Derek): `De La Cruz, M.J. K.L.` read family 'De La Cruz', suffix 'M.J. K.L.' — no given name — in silence, the silent-flip rationale ("only paired initials are taken for a name") failing on exactly the case where each word speaking for a pair is itself a pair. The reading stands, two dotted groups not being how anyone writes a person's initials, but that flip now REPORTS: when the only words speaking for a pair are other pairs, every shape word in the part being one. `John Smith, X.Y. P.Q.` gains the report with it; `John Smith, X.Y.Z. G.J.` and `John Smith, PhD G.J. K.L.`, where something else speaks, stay silent. The same rule reports `García Márquez, Ms G.J. K.L.`, whose honorific the run takes, where the first round's fix took it in silence. (b) A CLASS MEMBER IN FRONT SPEAKS FOR NOTHING (Derek): `García Márquez, Ed G.J.`, `Ma G.J.` and `MA G.J.` flipped on the member in front, where S2's company lets only an unambiguous credential speak for a member. Now the comma keeps the family, given 'Ed' — and 'G.J.' then ends the given part, the slot (iii) left alone, so it reads suffix 'G.J.' as `Doe, John R.T.` does, both forks reported; 1.4.0 and 2.3.0 gave middle 'G.J.'. Resolving that slot is the no-comma question #563 leaves open. (c) WORDING, no behavior: C1's (now rules/C4's) sentence counting a dual opening a part as a suffix word now defers to the pair's titles; the pair's report reaches it only where it opens the part, `Smith, Ms G.J.` having always been silent (the comma report's reach is the first post-comma piece, 2026-09-18); and the silence is stated as "no listed word takes part", which covers `García Márquez, PhD G.J.`, where "dotted-shape words alone" did not. SIMPLIFY ROUND, same day, behavior-identical (0 diffs over 21,604 parses: every corpus name, every quoted string in tests/v2/cases.py and 115 composed `pre, post` probes, under four policies, comparing fields and ambiguity details against the pre-round commit c125f69b; the same harness finds 252 diffs against master). One finding was a cost, not a style point: the speaker test scanned every word in front of EACH pair for a non-title, so `John Smith, MD MD ... G.J. G.J. ...` cost duals × pairs `_normalize` calls (163 at 8 of each, 1,387 at 32, py3.11). Only the first pair's scan can change the answer, since every later pair has the same words in front and more, so it is asked once: 107 and 395. `tests/v2/test_benchmark.py::test_the_paired_initials_title_scan_does_not_cost_quadratically` guards the ratio and fails at c125f69b. The run loop also asks LISTED as "no period", as the single-token test does, and `flip_reports` is set once after the run decision rather than piecemeal. - MEASURED 2026-09-30 against master b98b26e3, every name in this branch's `tools/differential/corpus*.jsonl` parsed on both trees with `nameparser.__file__` asserted on each side: 11 of 1453 distinct names differ, every one of them a name this change's rules.md examples and case rows put in the corpus (the two-pair names `De La Cruz, M.J. K.L.` and `John Smith, X.Y. P.Q.` are not among them: after the second round they read and report exactly as master does). THE POPULATION THAT COULD MOVE is the shape's, and the corpus barely holds it: over master's 1441 distinct names, 13 have a pair opening the part after the first comma, and `John Smith, A.B.` is the only one behind two or more NAME words with an unlisted, non-CJK pair (`Smith Jr., A.B.` has one name word, `Kenneth Clarke Q.C., M.P.` and `Virginia G. Essandoh, J.D.` hold listed acronyms, the rest one word) — so it is the only mover over that corpus, and the count is evidence about the corpus rather than about the rule's reach. Recompute: check out the parent into a separate worktree, parse every corpus name in each tree under `PYTHONSAFEPATH=1` with the tree's root first on `sys.path`, and diff `as_dict()` plus the sorted ambiguity kinds; for the population, take each name whose text after its first comma opens with a token matching `[^\W\d_]\.[^\W\d_]\.?` whole. + MEASURED 2026-09-30 against master b98b26e3 over every corpus name: 11 of 1453 distinct names differ, every one a name this change's rules examples and case rows added; over master's 1441 names only `John Smith, A.B.` holds the shape that could move, so the count is evidence about the corpus rather than about the rule's reach (recipe in PR #567). - 2026-10-01 (Derek), #564 — THE ALL-CAPS HALF READS THE COMMA POSITION BY DEFAULT, AND THE SWITCH HAS THREE SETTINGS. Supersedes the default of the 2026-09-14 entry above (its reasoning stands for the positions it was argued over). That entry turned the whole caps half off because French and Korean records write the SURNAME in capitals; but the convention writes them at the end of a name (`Jean DUPONT`) or before a comma (`DUPONT, Jean`), never after a comma behind a full name, so the reason for the off default never reached the comma position and that position was switched off with it. The corpus held three names of exactly that shape — `Ahmad Jayadi, CHA`, `John Smith, RAI`, `The Rt Hon Kenneth Clarke QC MP, HMG` — all credentials, all read as the given name at 2.3.0. - DECIDED, with Derek's calls marked. (1) `Policy.unlisted_caps_suffixes` becomes a `CapsSuffixes` StrEnum, OFF / AFTER_COMMA / EVERYWHERE, default AFTER_COMMA (Derek: an enum over two booleans or a mixed `bool | "comma"`, so every state is named and no meaningless combination exists). The field was new in 2.4 and unreleased, so no deprecation is owed; a bool raises a TypeError naming both replacements. The third value exists because redefining "off" would have left no setting that keeps a given name written in capitals (Derek's question, `García Márquez, GABRIEL`). (2) AFTER_COMMA reads only the part right after the first comma, behind two or more name words — the comma's own structure decision, which segment makes from the text before classify runs. The given part's last word after a family comma (`Doe, John XYZ`) and a maiden clause's last word are trailing slots and stay behind EVERYWHERE, as the comma-less end of a name does. (3) A lone two-letter word is never admitted at the comma, in any setting (Derek): it is the undotted form of #563's paired initials, and two words before the comma may be one surname, so `García Márquez, MJ` keeps given 'MJ'. Inside a longer part a two-letter word is read exactly as #563 reads the dotted pair — measured 2026-10-01 on the shipped tree by swapping `MJ` ↔ `M.J.` in runs of one to three words from {MJ, PhD, XYZ, CPA, Jr., MA, LEED, III} after seven name prefixes whose own words carry the contrast (`García Márquez`, `Smith Jones`, `Mr García Márquez`, `de García Márquez`, `García Márquez née Smith`, `John st Smith`, `García y López`): the comma's structure decision differed on 0 of 1295. After five all-caps prefixes it differed on 325 of 925, by design, `MJ` being no class member without the contrast while the dotted shape is case-free (`GARCÍA MÁRQUEZ, CPA MJ` keeps given 'CPA' where `CPA M.J.` makes the run). Recompute: run segment and read `ParseState.structure` for each pair. The equivalence is the comma's decision only: at the given part's last word after a family comma a dotted `M.J.` is a credential by shape (S3) where `MJ` is one only under EVERYWHERE, so `García Márquez, MA M.J.` reads suffix 'M.J.' and `MA MJ` middle 'MJ' — so `García Márquez, MJ PhD` keeps given 'MJ' as `De La Cruz, M.J. PhD` does, and C1 (rules/C5 since #728) says so by pointing at #563's own sentences rather than restating them. Two restatements were wrong: the first draft's "three letters or more, or a run holding such a word" (both reviews), and its replacement, "with only a credential behind it ... while a credential in front makes the run", which #563's vocabulary rules contradict (`John Smith, MA MJ` reads given 'MA'; `García Márquez, MJ XYZ` makes the run). (3b) An unlisted all-caps word is a by-shape member of C1's run (rules/C4's and C6's since #728) (#544), beside listed credentials, not only in a run of all-caps words (Derek, after review): before, `John Smith, XYZ` read suffix while `John Smith, PhD XYZ` read given 'PhD' and `John Smith, XYZ Jr.` given 'XYZ' — more evidence for a credential producing a name reading, already true under the old `True` and made a default by this change. The contrast has to be the NAME's, and it is carried only by one of the name's own words before the comma (`own_words`, so no maiden clause or delimited content) holding a capital and whose last LETTER is lowercase (`_vocab.written_as_a_name`), with no period and not claimed by a wordlist as a title, particle, connective, credential, generation, maiden marker or honorific (`_vocab.claimed_as_non_name`) (Derek). Every earlier draft asked for "a lowercase letter somewhere, less some exclusions", and each leaked a class the next review found: a mixed-case credential (`LLOYD WEBBER, ANDREW PhD`), a maiden clause or title (`LLOYD WEBBER née Smith, ANDREW PhD`, `Mr LLOYD WEBBER, ANDREW`), a generation, connective or shape title (`LLOYD WEBBER Jr., ANDREW`, `GARCÍA y LÓPEZ, ANDREW`, `Insp. LLOYD WEBBER, ANDREW`), and glued particles and unlisted lowercase words (`GISCARD d'ESTAING, VALÉRY`, `HAFEZ al-ASSAD, BASHAR`, `LLOYD ap RHYS, DAFYDD`) — no closed list separates an all-caps record from a mixed-case name. A Title-case draft (`str.istitle()`) closed those but read too narrowly, missing interior capitals and elisions (`DiCaprio`, `IJzerman`, `al-Rashid`, `d'Estaing`, `McDonald` — every one a mixed-case name that kept a given 'XYZ') and taking a lone capital as Title case (`de GAULLE C, CHARLES` lost its given name). Derek weighed "contains a capital" (an all-caps record with any lowercase word then always carries the contrast) and "not all capitals but contains a capital" (which takes `d'ESTAING` and `al-ASSAD`, the lowercase standing before the capitals) and first chose a capital directly followed by a lowercase letter, with a leading `Mc`/`Mac` skipped. Its review found the skip was one member of a class: any capitalized prefix glued to a surname written in capitals supplies such a pair (`LLOYD FitzGERALD, RONALD`, `PAOLO DeVITO, MARCO`, `JEAN LaFLEUR, PIERRE`, `DICK VanDYKE, JOHN`, `PAUL DuBOIS, JEAN`, `JEAN St-PIERRE, MARC`, and `LLOYD SMITH-McDONALD, RONALD`, the skip applying only at a word's start), as does `ß`, which has no capital (`MÜLLER WEIß, HANS`) — each lost its given name — while titlecase digraphs failed it (`Džokić Ljubić, XYZ`) and it cost a frame per letter. The shipped test, Derek's: a capital in the word and its last letter lowercase. A surname written in capitals ends in a capital whatever is glued in front of it, so every one of those records keeps its given name with no prefix list, and `McDonald` and `Džokić` pass. "Last letter" is the last letter that carries CASE, not the last character: the word is composed (NFC) first; every non-letter and every caseless letter is passed over (`Jones'`, `Smith2`, `Smith)`, `Jonesʼ` with U+02BC, the transliteration letters in `Wafāʾ`, a trailing `李`); and so is a lowercase letter whose capital is not one character (`ß` → `SS`, `ĸ` with none, the `fi` ligature), which is no evidence of case — `WEIß` is written in capitals, `Weiß` is not. Two cuts got this wrong: the first tested the last character, and its review found every name typed with decomposed accents and ending in an accented letter losing the contrast (`José André, XYZ`, `Lê Thị Hà, XYZ`, `René Noé, XYZ`, `Chloé Zoé, MBA XYZ` read given where their composed spellings read suffix), with `Jones'`, `Smith2` and `Smith)`; the second skipped non-letters but stopped at a caseless letter, so `Asmāʾ Wafāʾ, XYZ` and `JOHN Jonesʼ, XYZ` kept given 'XYZ' (its review). MEASURED 2026-10-02 against `a16927c6`, the capital-then-lowercase commit: the seven capitalized-surname prefixes (`FitzGERALD`, `DeVITO`, `LaFLEUR`, `VanDYKE`, `DuBOIS`, `St-PIERRE`, `SMITH-McDONALD`), `MÜLLER WEIß` and `KUJAĸ de PETERSEN` keep their given names, `Džokić Ljubić` gains the contrast, `JOHN O'NEILL's` and `JEAN-pierre DUPONT` gain one (accepted below), and nothing else moves; the decomposed-accent names, `Asmāʾ Wafāʾ` and `JOHN Jonesʼ` read as `a16927c6` reads them. OFF on this tree equals `unlisted_caps_suffixes=False` on `7394078a` (origin/master at the time) on all 4286 inputs. RECIPE, the comparator being those two commits (`git archive` each): the inputs are every name in `tools/differential/corpus*.jsonl`, every match of the regex `"([^"\n]{2,60})"` in `tests/v2/cases.py` holding a comma (a text match, not a literal parser, so fragments of case notes are included, harmlessly; extracting the literals exactly with `tokenize` gave 4273 inputs and the same conclusions), and each of these prefixes joined by ", " to each of the parts `XYZ`, `ANDREW`, `ANDREW PhD`, `PhD XYZ`, `MJ`, `XYZ Jr.` — `John Smith`, `DiCaprio LaBeouf`, `IJzerman IJsselmeer`, `al-Rashid al-Hassan`, `d'Estaing d'Orléans`, `McDonald MacLeod`, `Džokić Ljubić`, `Asmāʾ Wafāʾ`, `John Smith-Jones`, `john smith`, `García y López`, `LLOYD WEBBER`, `GISCARD d'ESTAING`, `HAFEZ al-ASSAD`, `LLOYD McDONALD`, `LLOYD FitzGERALD`, `PAOLO DeVITO`, `JEAN LaFLEUR`, `DICK VanDYKE`, `PAUL DuBOIS`, `JEAN St-PIERRE`, `LLOYD SMITH-McDONALD`, `MÜLLER WEIß`, `KUJAĸ de PETERSEN`, `LLOYD ap RHYS`, `de GAULLE C`, `Mr LLOYD WEBBER`, `LLOYD WEBBER née Smith`, `JOHN O'NEILL's`, `JOHN SMITHs`, `JEAN-pierre DUPONT`, `JOHN Jones'`, `JOHN Jonesʼ`, `JOHN Smith)`, `JOHN Smith2`, and the NFD forms of `José André`, `Lê Thị Hà`, `René Noé`, `Chloé Zoé`. Parse each under OFF, AFTER_COMMA and EVERYWHERE and compare `as_dict()`, the ambiguity kinds and `capitalized(force=True)`; the population is a fact of the tree it runs on (4286 here), since the corpora and the case table grow. ACCEPTED with it (Derek): a name written wholly in lowercase reads the listing form, `john smith, XYZ` keeping given 'XYZ', an unlisted title written in mixed case (`Doña GARCÍA LÓPEZ, MARÍA`) still carries the contrast, and so do two writings in an otherwise all-caps record that end in a lowercase letter: a glued possessive or plural (`JOHN O'NEILL's, XYZ`, `JOHN SMITHs, XYZ` read suffix `XYZ`) and a hyphenated part written in mixed case or lowercase (`JOHN SMITH-Jones, XYZ`, `JEAN-pierre DUPONT, XYZ`). ACCEPTED with (4): in a mixed-case name a capitalized given name followed by a generation goes the same way as one alone, `García Márquez, JUAN Jr.` reading suffix 'JUAN Jr.', reported; and behind a particle-led surname of two name words the flip leaves no given name at all, `De La Cruz García, MARÍA` reading family 'De La Cruz García', suffix 'MARÍA', because P1 reads a never-given particle's part as all surname. (4) ACCEPTED (Derek): a given name written in capitals behind a two-word surname without particles reads as the credential, `García Márquez, GABRIEL` giving given 'García', family 'Márquez', suffix 'GABRIEL', reported; capitalizing the given and not the family is not a convention, and OFF is the escape hatch. #575 was the prerequisite: before it, the count read `Van Buren, MARTIN` and `De La Cruz, MARIA` as two or three name words and this default would have taken their given names too. - WHAT MOVES, measured 2026-10-01 with the gate at all five baselines. The three corpus names above, plus the C1 examples (rules/C6's since #728) `John Smith, XYZ` and `John Smith, LEED AP` — the latter closing a `deviates: #291` marker rules.md had carried under a closed issue — classified by `fix(#564)` in every ledger; the radar-unclassified count is what it was before the change at every baseline. `John Smith, RAI` and `Ahmad Jayadi, CHA` read suffix again by their capitals, as they did by vocabulary before #342 removed both words: parity at 1.4.0, only the comma's report at 2.0 through 2.2, so the #342 rule's `fields` lose `given` (the OVER-DECLARED check) and the watched shape for `John Smith, RAI` is re-recorded at those four baselines. `Smith, XYZ` keeps given 'XYZ' and, at the default, reports nothing; EVERYWHERE still reports the declined fork there, as it did. + DECIDED, with Derek's calls marked. (1) `Policy.unlisted_caps_suffixes` becomes a `CapsSuffixes` StrEnum, OFF / AFTER_COMMA / EVERYWHERE, default AFTER_COMMA (Derek: an enum over two booleans or a mixed `bool | "comma"`, so every state is named and no meaningless combination exists). The field was new in 2.4 and unreleased, so no deprecation is owed; a bool raises a TypeError naming both replacements. The third value exists because redefining "off" would have left no setting that keeps a given name written in capitals (Derek's question, `García Márquez, GABRIEL`). (2) AFTER_COMMA reads only the part right after the first comma, behind two or more name words — the comma's own structure decision, which segment makes from the text before classify runs. The given part's last word after a family comma (`Doe, John XYZ`) and a maiden clause's last word are trailing slots and stay behind EVERYWHERE, as the comma-less end of a name does. (3) A lone two-letter word is never admitted at the comma, in any setting (Derek): it is the undotted form of #563's paired initials, and two words before the comma may be one surname, so `García Márquez, MJ` keeps given 'MJ'. Inside a longer part a two-letter word is read exactly as #563 reads the dotted pair — measured 2026-10-01 on the shipped tree by swapping `MJ` ↔ `M.J.` in runs of one to three words from {MJ, PhD, XYZ, CPA, Jr., MA, LEED, III} after seven name prefixes whose own words carry the contrast (`García Márquez`, `Smith Jones`, `Mr García Márquez`, `de García Márquez`, `García Márquez née Smith`, `John st Smith`, `García y López`): the comma's structure decision differed on 0 of 1295. After five all-caps prefixes it differed on 325 of 925, by design, `MJ` being no class member without the contrast while the dotted shape is case-free (`GARCÍA MÁRQUEZ, CPA MJ` keeps given 'CPA' where `CPA M.J.` makes the run). Recompute: run segment and read `ParseState.structure` for each pair. The equivalence is the comma's decision only: at the given part's last word after a family comma a dotted `M.J.` is a credential by shape (S3) where `MJ` is one only under EVERYWHERE, so `García Márquez, MA M.J.` reads suffix 'M.J.' and `MA MJ` middle 'MJ' — so `García Márquez, MJ PhD` keeps given 'MJ' as `De La Cruz, M.J. PhD` does, and C1 (rules/C5 since #728) says so by pointing at #563's own sentences rather than restating them. Two restatements were wrong: the first draft's "three letters or more, or a run holding such a word" (both reviews), and its replacement, "with only a credential behind it ... while a credential in front makes the run", which #563's vocabulary rules contradict (`John Smith, MA MJ` reads given 'MA'; `García Márquez, MJ XYZ` makes the run). (3b) An unlisted all-caps word is a by-shape member of C1's run (rules/C4's and C6's since #728) (#544), beside listed credentials, not only in a run of all-caps words (Derek, after review): before, `John Smith, XYZ` read suffix while `John Smith, PhD XYZ` read given 'PhD' and `John Smith, XYZ Jr.` given 'XYZ' — more evidence for a credential producing a name reading, already true under the old `True` and made a default by this change. The contrast has to be the NAME's, and it is carried only by one of the name's own words before the comma (`own_words`, so no maiden clause or delimited content) holding a capital and whose last LETTER is lowercase (`_vocab.written_as_a_name`), with no period and not claimed by a wordlist as a title, particle, connective, credential, generation, maiden marker or honorific (`_vocab.claimed_as_non_name`) (Derek). Every earlier draft asked for "a lowercase letter somewhere, less some exclusions", and each leaked a class the next review found: a mixed-case credential (`LLOYD WEBBER, ANDREW PhD`), a maiden clause or title (`LLOYD WEBBER née Smith, ANDREW PhD`, `Mr LLOYD WEBBER, ANDREW`), a generation, connective or shape title (`LLOYD WEBBER Jr., ANDREW`, `GARCÍA y LÓPEZ, ANDREW`, `Insp. LLOYD WEBBER, ANDREW`), and glued particles and unlisted lowercase words (`GISCARD d'ESTAING, VALÉRY`, `HAFEZ al-ASSAD, BASHAR`, `LLOYD ap RHYS, DAFYDD`) — no closed list separates an all-caps record from a mixed-case name. A Title-case draft (`str.istitle()`) closed those but read too narrowly, missing interior capitals and elisions (`DiCaprio`, `IJzerman`, `al-Rashid`, `d'Estaing`, `McDonald` — every one a mixed-case name that kept a given 'XYZ') and taking a lone capital as Title case (`de GAULLE C, CHARLES` lost its given name). Derek weighed "contains a capital" (an all-caps record with any lowercase word then always carries the contrast) and "not all capitals but contains a capital" (which takes `d'ESTAING` and `al-ASSAD`, the lowercase standing before the capitals) and first chose a capital directly followed by a lowercase letter, with a leading `Mc`/`Mac` skipped. Its review found the skip was one member of a class: any capitalized prefix glued to a surname written in capitals supplies such a pair (`LLOYD FitzGERALD, RONALD`, `PAOLO DeVITO, MARCO`, `JEAN LaFLEUR, PIERRE`, `DICK VanDYKE, JOHN`, `PAUL DuBOIS, JEAN`, `JEAN St-PIERRE, MARC`, and `LLOYD SMITH-McDONALD, RONALD`, the skip applying only at a word's start), as does `ß`, which has no capital (`MÜLLER WEIß, HANS`) — each lost its given name — while titlecase digraphs failed it (`Džokić Ljubić, XYZ`) and it cost a frame per letter. The shipped test, Derek's: a capital in the word and its last letter lowercase. A surname written in capitals ends in a capital whatever is glued in front of it, so every one of those records keeps its given name with no prefix list, and `McDonald` and `Džokić` pass. "Last letter" is the last letter that carries CASE, not the last character: the word is composed (NFC) first; every non-letter and every caseless letter is passed over (`Jones'`, `Smith2`, `Smith)`, `Jonesʼ` with U+02BC, the transliteration letters in `Wafāʾ`, a trailing `李`); and so is a lowercase letter whose capital is not one character (`ß` → `SS`, `ĸ` with none, the `fi` ligature), which is no evidence of case — `WEIß` is written in capitals, `Weiß` is not. Two cuts got this wrong: the first tested the last character, and its review found every name typed with decomposed accents and ending in an accented letter losing the contrast (`José André, XYZ`, `Lê Thị Hà, XYZ`, `René Noé, XYZ`, `Chloé Zoé, MBA XYZ` read given where their composed spellings read suffix), with `Jones'`, `Smith2` and `Smith)`; the second skipped non-letters but stopped at a caseless letter, so `Asmāʾ Wafāʾ, XYZ` and `JOHN Jonesʼ, XYZ` kept given 'XYZ' (its review). MEASURED 2026-10-02 against `a16927c6`, the capital-then-lowercase commit, over 4286 inputs (every corpus name, every comma-holding quoted string in tests/v2/cases.py, and a grid of mixed-case, all-caps and decomposed-accent prefixes before six parts, under all three settings): the capitalized-surname prefixes, `MÜLLER WEIß` and `KUJAĸ de PETERSEN` keep their given names, `Džokić Ljubić` gains the contrast, `JOHN O'NEILL's` and `JEAN-pierre DUPONT` gain one (accepted below), nothing else moves, and OFF equals `unlisted_caps_suffixes=False` on `7394078a` on all 4286 (recipe in PR #577). ACCEPTED with it (Derek): a name written wholly in lowercase reads the listing form, `john smith, XYZ` keeping given 'XYZ', an unlisted title written in mixed case (`Doña GARCÍA LÓPEZ, MARÍA`) still carries the contrast, and so do two writings in an otherwise all-caps record that end in a lowercase letter: a glued possessive or plural (`JOHN O'NEILL's, XYZ`, `JOHN SMITHs, XYZ` read suffix `XYZ`) and a hyphenated part written in mixed case or lowercase (`JOHN SMITH-Jones, XYZ`, `JEAN-pierre DUPONT, XYZ`). ACCEPTED with (4): in a mixed-case name a capitalized given name followed by a generation goes the same way as one alone, `García Márquez, JUAN Jr.` reading suffix 'JUAN Jr.', reported; and behind a particle-led surname of two name words the flip leaves no given name at all, `De La Cruz García, MARÍA` reading family 'De La Cruz García', suffix 'MARÍA', because P1 reads a never-given particle's part as all surname. (4) ACCEPTED (Derek): a given name written in capitals behind a two-word surname without particles reads as the credential, `García Márquez, GABRIEL` giving given 'García', family 'Márquez', suffix 'GABRIEL', reported; capitalizing the given and not the family is not a convention, and OFF is the escape hatch. #575 was the prerequisite: before it, the count read `Van Buren, MARTIN` and `De La Cruz, MARIA` as two or three name words and this default would have taken their given names too. + WHAT MOVES, measured 2026-10-01 with the gate at all five baselines: the three corpus names above plus `John Smith, XYZ` and `John Smith, LEED AP` (closing a `deviates: #291` marker), all classified `fix(#564)`, the radar-unclassified count unchanged; `John Smith, RAI` and `Ahmad Jayadi, CHA` read suffix again by their capitals, parity at 1.4.0; `Smith, XYZ` keeps given 'XYZ', silent at the default and reported under EVERYWHERE (PR #577). CASE REPAIR FOLLOWS THE READING. The comma decision is segment's, made from the text, while case repair keeps a word in capitals only where classify wrote the shape tag (rules/R4); the first draft tagged only under EVERYWHERE, so `parse("John Smith, XYZ").capitalized(force=True)` rendered 'John Smith Xyz' at the default and 'John Smith XYZ' under EVERYWHERE (found by the docs review, axis 5). Classify now tags a caps-shaped word in the part a suffix comma opened under AFTER_COMMA too, so both settings render that part alike, and rules/R4 names the caps shape beside the dotted one; `test_render`'s forced-repair test fails with the tagging removed (an R4 example could not witness it, R5 leaving a mixed-case name unrepaired unless forced). A caps word in a third or later comma part is still tagged only under EVERYWHERE, so `John Smith, MD, XYZ` renders 'Xyz' at the default, as master does. - COST, `sys.setprofile` call counts (mean of 50, after one warm-up) against `git archive origin/master`: `Smith, John` 183 → 183, `Smith, JOHN` 183 → 183, `Smith, XYZ` 182 → 182, `John Smith, PhD` 217 → 217, `John Smith, MA` 252 → 252, `John Smith, Ph. D.` 254 → 254, `John Smith, CPA` 217 → 217, `John Smith, MD PhD` 260 → 260, `John Smith, MBA CPA` 261 → 261, `Dr. Juan de la Vega III` 366 → 366, `John Smith, XYZ` 251 → 266, and an all-caps record with a capitalized given name and a credential, `LLOYD WEBBER, ANDREW PhD`, 338 → 364, a constant +26 that does not grow with the name's length (`'LLOYD'*64 + ' WEBBER, ANDREW PhD'` 653 → 679; the second draft's per-character contrast test had made that +350). Past the all-caps check the contrast is linear in the words before the comma, about four frames a word (the generator, the case test's call, and the own-words walk's fold and marker test; `'GAULLE ' + 'ap '*k + 'GAULLE, CHARLES'` is +7, +26, +38, +54 over master at k = 0, 1, 4, 8), and constant in a word's letters (`'de ' + 'GAULLE'*k + ', CHARLES'` 253 → 275, 343 → 365 and 631 → 653 at k = 1, 16, 64), where the capital-then-lowercase draft grew a frame per letter; all paid only once a caps word is in hand. The comma test is on by default, so three C-level checks go before any call: two or more words before the comma (the first draft lacked it, and `Smith, JOHN`, a common record format, paid +25 for a flip it can never make — the code review), the first word in capitals, and the lone-two-letter length. Every caller of the caps predicate asks first, in C, what it would decline anyway — alphabetic capitals, not a listed suffix word — so a listed credential never pays for the call (the review of the first fix found `John Smith, CPA` +4 and `John Smith, Ph. D.` +5 before these). The 2026-09-14 entry's recompute recipe predates the enum: its `on` is `Policy(unlisted_caps_suffixes=CapsSuffixes.EVERYWHERE)` now, and its `off=Parser()` is `Policy(unlisted_caps_suffixes=CapsSuffixes.OFF)`, `Parser()` being AFTER_COMMA. + COST, `sys.setprofile` call counts against `git archive origin/master`: ordinary and listed-credential comma names unchanged, `John Smith, XYZ` 251 → 266, and an all-caps record with a capitalized given name and a credential (`LLOYD WEBBER, ANDREW PhD`) a constant +26 (the second draft's per-character contrast test had made `'LLOYD'*64 + ' WEBBER, ANDREW PhD'` +350), the contrast linear in the words before the comma and constant in a word's letters, all paid only once a caps word is in hand (PR #577). The comma test is on by default, so three C-level checks go before any call: two or more words before the comma (the first draft lacked it, and `Smith, JOHN`, a common record format, paid +25 for a flip it can never make — the code review), the first word in capitals, and the lone-two-letter length. Every caller of the caps predicate asks first, in C, what it would decline anyway — alphabetic capitals, not a listed suffix word — so a listed credential never pays for the call (the review of the first fix found `John Smith, CPA` +4 and `John Smith, Ph. D.` +5 before these). - 2026-10-17 #682 — SPLIT FROM decisions/S2 with the rule. The entries above were written under decisions/S2 while this rule was part of rules/S2, and moved here unchanged but for pointers the move broke; where one says rules/S2 it means the rule as it then stood, which held this one. diff --git a/docs/design/decisions/T4-bare-letter-as-an-initial.md b/docs/design/decisions/T4-bare-letter-as-an-initial.md index 85d20ee7..fc1ee6a1 100644 --- a/docs/design/decisions/T4-bare-letter-as-an-initial.md +++ b/docs/design/decisions/T4-bare-letter-as-an-initial.md @@ -1,6 +1,6 @@ # T4 — the bare letter as an initial -- 2026-10-18, #723 — A ONE-CASE BARE LETTER IS THE INITIAL ITS CAPITAL IS. #721 applied S10's "a name written wholly in one case says nothing about any letter in it" to the one-letter roman numerals only (decisions/S10, the Open line of its /simplify entry), so a bare lowercase `j` was a name word where `J` was an initial, and every rule that reads an initial differently read the two spellings of one name differently: M4 (`j nee smith` family 'j', `J NEE SMITH` given 'J'), O5 (a lone `j` reported `given-or-family`, `J` did not), and P3's count, which passes over an initial (`jose j y lopez` family 'j y lopez', `JOSE J Y LOPEZ` middle 'J Y', family 'LOPEZ', which 1.4.0 read too). The rule is now stated once, for every letter, and the numeral case is an instance of it: in a name written in one case, a bare letter is tagged an initial wherever its capital would be. Its capital is asked of the bare-capital shape (one ASCII capital), so `é` and `ж`, whose capitals are no initial, read as before in both spellings. `ı` and `ſ`, whose capitals are `I` and `S`, are tagged initials too; `ı`'s twin is not a true one, since `I` is also the marked Catalan connective and `ı` is in no vocabulary (`JOSE I LOPEZ` reports `conjunction-or-initial`, `jose ı lopez` nothing). Measured on the first commit's tree against 16ee2851, a dated snapshot from a scratch grid not committed: letters a–z, `é` and `ж`; heads (a, b, c) = (john, smith, jones), (jose, lopez, garcia), (mary, ma, lee); all three orders; the 35 shapes `{n}`, `{n} nee {b}`, `{a} {n}`, `{n} {a}`, `{a} {n} {b}`, `{a} {n} y {b}`, `{a} {n} e {b}`, `{b}, {n}`, `{b}, {n} y {c}`, `{b}, {a} {n}`, `, {n}`, `{n}, phd`, `{a} {n}, phd`, `{a} {n}, jr.`, `{a} nee {b} {n}`, `{a} nee {n}`, `{a} {b} nee {n} {c}`, `dr {n}`, `dr {n} {b}`, `sir {n} {b}`, `{a} van {n}`, `{n} van {b}`, `{a} van {n} {b}`, `abdul {n}`, `{n} {a} {b}`, `{a} {n} {n} {b}`, `{a} {n}. {b}`, `({c}) {n} {b}`, `{n} y {b}`, `{a} {n} phd`, `{a} {b} {n} prof.`, `{n} {b} {n}`, `{n} {n}`, `{a} {n} de la {b}`, `{b} {n} jr`; each lowercase text against its `.upper()`, compared on case-folded fields, sorted report kinds and `initials()`, keyed by order and text, so the 8,820 products collapse to 7,794 distinct pairs where a shape has no head. 16ee2851 disagreed on 504 of them, this tree on none, and no capital spelling moved. The committed invariant is `test_a_one_case_name_reads_as_its_capitals` (tests/v2/test_properties.py), whose grid gains the issue's shapes and the letters `j`, `y` and `e`; its stated controls are the recompute. +- 2026-10-18, #723 — A ONE-CASE BARE LETTER IS THE INITIAL ITS CAPITAL IS. #721 applied S10's "a name written wholly in one case says nothing about any letter in it" to the one-letter roman numerals only (decisions/S10, the Open line of its /simplify entry), so a bare lowercase `j` was a name word where `J` was an initial, and every rule that reads an initial differently read the two spellings of one name differently: M4 (`j nee smith` family 'j', `J NEE SMITH` given 'J'), O5 (a lone `j` reported `given-or-family`, `J` did not), and P3's count, which passes over an initial (`jose j y lopez` family 'j y lopez', `JOSE J Y LOPEZ` middle 'J Y', family 'LOPEZ', which 1.4.0 read too). The rule is now stated once, for every letter, and the numeral case is an instance of it: in a name written in one case, a bare letter is tagged an initial wherever its capital would be. Its capital is asked of the bare-capital shape (one ASCII capital), so `é` and `ж`, whose capitals are no initial, read as before in both spellings. `ı` and `ſ`, whose capitals are `I` and `S`, are tagged initials too; `ı`'s twin is not a true one, since `I` is also the marked Catalan connective and `ı` is in no vocabulary (`JOSE I LOPEZ` reports `conjunction-or-initial`, `jose ı lopez` nothing). Measured against 16ee2851 (PR #726) on a scratch grid not committed (letters a–z, `é` and `ж` in 35 shapes, three heads and all three orders, each lowercase text against its `.upper()`): 16ee2851 disagreed on 504 of 7,794 distinct pairs, this tree on none, and no capital spelling moved. The committed invariant is `test_a_one_case_name_reads_as_its_capitals` (tests/v2/test_properties.py), whose grid gains the issue's shapes and the letters `j`, `y` and `e`; its stated controls are the recompute. WHAT MOVES: lower-case letters in a name whose own words are written in one case, each onto its capital twin's reading — wholly lower-case names, and (since the case question is asked of the own words, P3) a one-case name beside a clause of another case, so `j nee Smith` reads given 'j' as `J nee Smith` does. In three classes — a lone letter loses O5's report, a letter before a maiden marker keeps the given name M4 would have taken, and a letter beside a single-letter link is passed over by P3's count, so the link in what is then a three-word name stays a name word. No corpus name moved (the snapshot and the CLDR, GND and native-script agreement records are unchanged). And P8's Accepted paragraph (P3's until the 2026-10-19 split) on a declined marker loses its "one shape" in which the name's own words read differently in the two spellings: `dr. née juan y lopez` read family 'juan y lopez' where `DR. NÉE JUAN Y LOPEZ` reads middle 'JUAN Y', because the `y` past the declined marker is outside the own words P8's fork reads; it is this rule's letter now, and reads as its capital does. And one move the snapshot cannot see (R4): case repair of a connective letter in a one-case maiden clause, now tagged an initial as its capital is, so `jose lopez nee y garcia` repairs to `Jose Lopez (Y Garcia)` where it gave `(y Garcia)` — the same tag in the two spellings that P8's rewritten paragraph intends. No corpus name's `capitalized()` moved (the design-docs review compared every one across the two trees). A GAP THE RULE DOES NOT CLOSE, found by the design-docs review and present on 16ee2851 too: under `Policy(lenient_comma_suffixes=False)` the strict comma-suffix test asks a one-letter numeral's shape as written, not its tag, so `john smith, v` reads suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reports nothing where `SMITH, JOHN, I` reports `comma-structure`. The committed grid runs the default policy only. Left for its own change, and taken by the #727 entry below (was Open: #727). @@ -9,10 +9,8 @@ - 2026-10-18, #727 — THE STRICT SUFFIX TEST VETOES A ONE-CASE LETTER AS IT VETOES ITS CAPITAL. The gap the first entry recorded: `is_suffix_strict` vetoes an initial by the word's text (`is_initial`), not by classify's tag, so a one-case lowercase `v` or `i` — an initial under this rule — passed where its capital was vetoed. Two callers reached it. Under `Policy(lenient_comma_suffixes=False)`, C1's comma tests (`is_wholly_suffix` and the name-word count): `john smith, v` read suffix 'v' where `JOHN SMITH, V` reads given 'V', and `smith, john, i` reported nothing where its capitals report `comma-structure`. And under the default policy, W2's scan-back (`_script_segment._is_post_nominal`): `田中さん v` peeled さん and read suffix 'さん v' where `田中さん V` reads given '田中さん', family 'V' — a CJK name with a Latin letter beside it is one case under the own-words test, the letter being the only cased word in it, so nothing in the writing contrasts it. The strict test now takes the one-case fact as an argument and vetoes a bare lowercase letter whose capital is A to Z where it is true; segment, `_comma` and script_segment pass the fact they already hold, and segment asks for it only under the strict knob. A separate keyword from `is_wholly_suffix`'s `one_case`, which is the credential LEAN and is None by design at callers that want no lean; they still want the letters read by this rule. The lenient default takes a listed suffix word whatever its shape and never reads the argument. Measured: the committed invariant's grid gains the three comma shapes and `田中さん {n}`, and runs every comma-holding shape again under the strict knob (the knob reaches nothing else); 2444f847 disagreed on 72 of its 4,554 pairs (54 strict comma pairs, 18 honorific), this tree on none, and the third recorded control is that 72. `田中さん v` now reads as 1.4.0 read it; 2.1 through 2.4.2 peeled it (suffix 'さん, v' in 2.1 and 2.2, 'さん v' from 2.3). No corpus name moved. Open: none from this entry; #727 closes the gap the first entry left open. -- 2026-10-18, #731 — THE MAIDEN TAKE'S CANDIDATE TEST ASKS THE ONE-CASE FACT. M2's take (#601, #610) builds a clause-free view of the name from the clause's trailing candidates and lets `tail_reading` decide what the run takes. Its candidate test (`_pieces.trailing_candidates`) asked the numeral fork's shape test with no one-case fact, as a superset of what the fork admits. But a candidate is also cut out of the view, which changes the word in front of the numeral there: in `john smith nee x x` the lowercase first `x` is not initial-shaped, so the second became a candidate, and the view read it behind `smith` with words to spare, as the generation (suffix 'x', maiden 'x', reported). `JOHN SMITH NEE X X` keeps maiden 'X X', the capital first `X` refusing the candidate. Now the test asks with a one-case fact, as the fork does, so the lowercase letter in front refuses it too; `john smith nee j x` moves the same way. The fact is the WHOLE name's, since the word in front of a candidate is always a clause word, which this rule reads by the whole name's case: the first version passed the own-words fact, and the design-docs review found `MARY SMITH NEE j x` (mixed-case writing) moving to maiden 'j x' where every release and `Mary Smith nee j x` read suffix 'x', maiden 'j' — 72 field moves over a grid pairing own words in capitals with Title case, the clause letters lowercase in both, now `test_a_clause_letter_beside_capitals_reads_as_beside_title_case` with that 72 as its control. - `trailing_candidates` is no longer a strict superset of what `tail_reading` takes from a view missing the clause's first word, and says so: a numeral whose word in front reads as an initial is refused even where a view without that word would read it, which is the point. The superset check (`test_the_trailing_candidates_cover_what_the_tail_reading_takes`) still passes only because its clauses all open with `Smith`; a clause opening with the letter shows the refusal as a "miss" (the review measured 8 of 48 such texts, the capital ones on ad486f66 too). +- 2026-10-18, #731 — THE MAIDEN TAKE'S CANDIDATE TEST ASKS THE ONE-CASE FACT. SUPERSEDED by 2026-10-18 #731 reverted (PR #733) — the entry below; the full text is in git and in PR #732. The gap and the declined option below predate #731 (ad486f66) and still stand; they were written under #731, so "this tree" and "the committed grid" there mean #731's, a grid the revert deleted (its text is in PR #732). Re-measured 2026-10-19 on a rebuild of that grid: the readings and the 108 hold. The declined option's results ("0 field and 0 report differences", and the 270) were measured under #731 and were not re-run as that option; an approximation of it re-run the same day (whole-name fact passed to the take's view) closes the fields but leaves 27 of the 396 rebuilt pairs differing in their reports, so read "closes both" as #731's measurement, unconfirmed since. A GAP LEFT OPEN, on ad486f66 as on this tree: the view the take reads asks the own words' case, so a lower-case clause numeral behind own words in capitals is counted as a capital would be, where behind Title case it is S10's "lower case in a name of mixed case", the generation and unreported. With two own name words the fields agree and only the report differs (108 pairs of the committed grid: `JOHN SMITH nee x x` reports `suffix-or-name`, `John Smith nee x x` does not); with ONE own name word there is no word to spare and the fields differ too (`SMITH NEE j x` maiden 'j x', `Smith Nee j x` suffix 'x', maiden 'j'; the code review's finding). The committed grid therefore holds two own words in front only, and says so. Declined, measured: passing the whole name's fact to the view as well. It closes both (0 field and 0 report differences on a grid with one-word, two-word and titled heads) but the view also asks S6's credential lean, which P8 says is asked of the own words P3's span holds, and moving it there fails `test_a_maiden_clause_does_not_change_how_a_trailing_word_reads` (tests/v2/test_properties.py) on 270 of its 20,412 pairs (with it, `J. nee Smith BA` would read the credential where `J. BA` reads a name). Closing the gap needs the peel to take the numeral's case fact apart from the lean's — a design change beyond this fix. - Measured: the one-case grid gains `{a} {b} nee {n} {n}` and `{a} {b} nee j {n}`. ad486f66 disagreed on 36 of its 4,752 pairs, all in those shapes; this tree on none, and the fourth recorded control is that 36. 2.2.0 through 2.4.2 gave the lowercase reading above, and 2.0.0 and 2.1.0 the reading this restores (maiden 'x x'), so the release bullet for #723 carries this too. No corpus name moved. - 2026-10-18 (Derek), #731 reverted — A MAIDEN CLAUSE'S NUMERAL IS NOT HELD TO ITS CAPITALS. The #731 entry above is superseded and its change reverted. Derek: this rule and maiden names do not really interact, because nobody writes a maiden marker followed by an initial, or two. Asked whether settling the gap that entry left open would simplify the parser, the answer was the opposite: it needs the peel to carry two case facts, one for S6's lean and one for the numeral. And #731 itself had bought `john smith nee x x` and `john smith nee j x` — both that unwritten shape — a case argument on `trailing_candidates`, a whole-name case test in the take, a stated exception to the candidates' superset contract (decisions/M2's #731 entry), two grid shapes and a second invariant with their controls, all reverted. The readings go back to 2.2.0 through 2.4.2's (`john smith nee x x` suffix 'x', maiden 'x', reported), stated as an Accepted clause of this rule with both spellings as examples. Kept from that PR: M2 naming S10 among the trailing rules its take asks, which was true before #731 and stays true. What this rule still says of a clause's letters — classify reads one by the whole name's case (the #723 code-review entry) — stands: the shape that guards is a connective inside a clause (`MARIA GARCIA NEE LOPEZ y FERNANDEZ`), which names do take. - 2026-10-19 — RULE SPLIT: the written shape of an initial (its period, or a bare unaccented Latin capital) left rules/T4 for rules/T5, and T4 keeps the case contrast every entry here decided. Record: decisions/T5's entry of this date. diff --git a/docs/design/decisions/cjk-comma-demotion.md b/docs/design/decisions/cjk-comma-demotion.md index 892ceb7c..ad53bad4 100644 --- a/docs/design/decisions/cjk-comma-demotion.md +++ b/docs/design/decisions/cjk-comma-demotion.md @@ -19,7 +19,7 @@ Closes #469, and continues the corpus-tier arc below rather than standing apart - 2026-09-01 — the SHIPPED 2.2.0 release notes are left untouched, deliberately, and this bullet is the record of that choice. docs/release_log.rst describes the comma-crossing as a fix in two 2.1.0 entries ("Fix a comma or a 间隔号 stopping the glued-honorific peel", and #319's `田中さん, V.` entry). Both were true statements about what those releases did, made while the crossing was contract and #312 had argued it. Demoting the rule does not make a released note false about its release, and editing shipped notes to agree with a later doctrine would cost the log the one property it exists for. The demotion is 2.3's news and belongs in 2.3's notes; the log is a history, not a mirror of the current contract. -- COUNTS, 2026-09-01, with the recompute beside each. 29 case rows carry `tolerated=True` over 26 distinct texts — the flag is per ROW and the corpus is per TEXT, which is why the two numbers differ (recompute: read `CASES` and count `c.tolerated`). The 26th text arrived with the W3 demotion itself: the given-side listing `지훈, 남궁민수` was a rules.md example with no case row, and would have left the harness altogether when the rules corpus stopped carrying W3, so it was given a tolerated row rather than dropped. `corpus_cjk.jsonl` went 98 → 73 and the radar-tier `corpus_cjk_tolerated.jsonl` was created at 25, then 26; the identity is the check worth keeping, since 73 + 25 = 98 exactly says the split MOVED names and dropped none, and the 99th is that one rescued row. `corpus_rules.jsonl` went 252 → 248: six comma-bearing example texts left (`田中さん, Dr.`, `田中さん, PhD`, `남궁민수, 지훈`, `지훈, 남궁민수` when the builder skipped W3 whole, plus `김, 민준씨` and `田中さん, V.` from W2's swap) and two pure ones arrived (`김민준씨`, `田中さん 様.`). `corpus_shapes.jsonl` stands at 37, with the new shapes covered 5 names (shape 6) and 2 (shape 7). And the stop condition held: intentional diffs are UNMOVED at 226/205/113/0 across 1.4.0/2.0.0/2.1.0/2.2.0, with unexplained 0, radar unclassified 0 and exit 0 at each — 1113 names compare at 1.4.0 (7 skipped, shapes 4/5) and 1120 above it. That the counts do not move is the substance and not a formality: a classified diff counts identically on either tier, so demoting a name changes which file it loads from and nothing about what the gate makes of it. +- COUNTS, 2026-09-01 (PR #488): 29 case rows carry `tolerated=True` over 26 distinct texts; `corpus_cjk.jsonl` went 98 → 73 and `corpus_cjk_tolerated.jsonl` was created at 25, then 26, moving names and dropping none; `corpus_rules.jsonl` went 252 → 248; and the gate's intentional diffs were unmoved at every baseline with unexplained 0, a classified diff counting identically on either tier. The 26th text, the given-side listing `지훈, 남궁민수`, was a rules.md example with no case row and was given a tolerated row rather than dropped. - 2026-09-05 — THE SWEEP'S CRITERION WAS NARROWER THAN THE DOCTRINE IT IMPLEMENTED, and three rows sat in the gap. The 2026-09-01 sweep asked, over the CJK-bearing case rows, for a comma or an ASCII letter. A trailing ASCII period is neither, so `田中さん 様.`, `김민준 씨.` and `김민준 양.` — the three `fix(#320)` probes — stayed contract; and `田中さん 様.` was then chosen as rules/W2's replacement example for its "ignores surrounding punctuation" clause on the strength of passing that same test, so the miss propagated into the normative document the same day. A period standing after a CJK honorific is a listing artifact no writing system produces, which is this entry's own class in its own words: the admission test judges the FORMAT, and `様.` is no more an arrangement Japanese writes than `田中さん, PhD` is. The three rows carry `tolerated=True` now, and nothing about the parse moves — the rows still pin #320's mechanism at HEAD, and the texts are still compared and classified at every baseline, from the radar file instead of the contract one. CORRECTED IN REVIEW, 2026-09-05: a first draft of this bullet called all three "the spaced twin of a comma row that WAS demoted", and only two are. `田中さん, 様.` and `김민준, 씨.` are both case rows and both in `corpus_cjk_tolerated.jsonl`; `김민준, 양.` is no row at all — enumerating the CASES texts containing `양` returns `김지양`, `양 미선`, `김민준 양`, `김민준 양.` and `양 지훈`, and nothing with a comma. `김민준 양.` is demoted on the class argument alone, which is what carries the other two as well: an existing demoted twin is corroboration, never the warrant, or the class would end wherever the case table happens to stop. (Recompute: read `CASES` and collect the distinct texts containing the character.) @@ -31,5 +31,5 @@ Closes #469, and continues the corpus-tier arc below rather than standing apart - 2026-09-05, FOUND while rewriting the clause — A PERIOD ON THE NAME'S OWN LAST WORD IS NOT IGNORED; IT IS DECISIVE, and the moved half asserted the opposite. The half said the split-off "ignores surrounding punctuation", and W3's first descriptive rewrite of it kept that shape, attributing the crossing to the punctuation. Five parses, 2026-09-05 under the default facade, say otherwise — counted by INPUT STRING, the convention the rest of this bullet uses when it says "the five inputs" and "parse the five strings". A first draft counted four by counting the SENTENCES below instead, the last of which carries two inputs (`田中さん.` and `김민준씨.`). Six, if the no-period twin `田中さん 太郎` quoted inside the third sentence as its control counts as a parse of its own rather than as part of that item. `田中さん 様` → family `田中`, suffix `さん, 様`: the crossing happens with no punctuation in the input at all, so the vocabulary licenses it and not the period. `田中さん 様.` → family `田中`, suffix `さん, 様.`: the period rides along and moves nothing, `様.` reading post-nominal-strict exactly as `様` does. `田中さん 太郎.` → family `太郎.`, given `田中さん`: a period on a NAME word, and no crossing — and not ignored either, since the same input without it is family `田中さん`, given `太郎`, the order reading flipped by the period alone. `田中さん.` → title `田中さん.`, and `김민준씨.` → title `김민준씨.`: a period glued to the honorific's own word stands between the honorific and that word's end, no listed tail matches, the peel declines and the whole text reads as a title — the parse a period changes most, not least. Of the five inputs, the ONLY one where a period changes nothing is the one where it sits on a word the split-off was going to step past regardless. So W3 now says the step past a post-nominal word is W2's and needs no punctuation, that a period on a separate post-nominal word rides into the suffix, and that a period glued to the honorific's own word decides against the split-off. AMENDED 2026-09-10 by #cjk-full-stops, and it is the summary that goes rather than the measurements: every parse above keeps its date and is what the parser did on 2026-09-05 (the two suffix values render space-joined since #436/#437, the next day, `さん 様` for `さん, 様`). Of the four period-bearing inputs, ALL FOUR now divide exactly as their no-period twins — `田中さん 太郎.` reads family `田中さん`, given `太郎.`, the classification fold making the period invisible to the order rule, and `田中さん.` and `김민준씨.` peel their honorific with the stop riding on it — so the period changes no division at all here, and the third clause of the sentence before this one is superseded: a period glued to the honorific's own word rides WITH the honorific now. What survives is the first two clauses and the finding this bullet is titled for, inverted in its consequence rather than in its content: the period was decisive because a site was reading it, and the answer was to stop three more sites reading it. RECOMPUTE: parse the five strings, each also without its period, and read `title` alongside the name fields — a rewrite that reads only `family` and `suffix` sees two empty results and can call them a null effect. -- COUNTS, 2026-09-05, with the recompute beside each. `corpus_cjk.jsonl` went 73 → 70 and `corpus_cjk_tolerated.jsonl` 26 → 29 — the same identity as the first demotion, 99 names either side, so the split MOVED three texts and dropped none. `corpus_rules.jsonl` went 248 → 247 → 249: W2 gave up the `田中さん 様.` line, then took the two the review round restored the second half with (`김민준 박사님`, `선생님`). Both arrivals are texts `corpus_cjk.jsonl` already held, so the file grew and the compared POOL did not — the same shape as the shapes-corpus growth of 2026-09-01, and the reason a witness had to be picked from names already inside. 32 case rows carry `tolerated=True` over 29 distinct texts (recompute: read `CASES` and count `c.tolerated`, per row and per text). `_CORPUS_FLOORS["corpus_cjk.jsonl"]` is lowered 70 → 67 deliberately, and the tolerated file's floor of 22 still stands under 29. The stop condition held again: intentional diffs UNMOVED at 352 / 247 / 155 / 14 across 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, unexplained 0, radar unclassified 0 and exit 0 at each. RECOMPUTE THE SWEEP, which is the part worth having rather than the digits: it is the CJK-bearing (`_has_cjk`) rows that do NOT declare `tolerated`, carrying any ASCII character other than a space, grouped by the kind of character that trips them and joined to the corpus files holding each text. Over the tree before this bundle it returns eight distinct texts — the three period rows, the parenthesized nickname, and the four mixed-script rows — and over this one, five: the period class is gone and the letter class is untouched, which is the shape of the answer to look for rather than the count. +- COUNTS, 2026-09-05 (PR #505): `corpus_cjk.jsonl` went 73 → 70 and `corpus_cjk_tolerated.jsonl` 26 → 29, moving three texts and dropping none; `corpus_rules.jsonl` went 248 → 247 → 249 without growing the compared pool; 32 case rows carry `tolerated=True` over 29 texts; the gate's intentional diffs were unmoved at every baseline with unexplained 0; and the sweep (CJK-bearing rows not declaring `tolerated` that carry an ASCII character other than a space) returns eight texts before the bundle and five after, the period class gone and the letter class untouched. diff --git a/docs/design/decisions/cjk-full-stops.md b/docs/design/decisions/cjk-full-stops.md index 5fe151d0..9b0dfa1f 100644 --- a/docs/design/decisions/cjk-full-stops.md +++ b/docs/design/decisions/cjk-full-stops.md @@ -10,10 +10,10 @@ ONE GAP SEEN FROM TWO SIDES, which is why the two issues were answered together. - **#323 turned out broader than it was filed.** The issue named the honorific peel. Making the classification fold read through an edge stop moved three readers of the `None` it used to return, and only one of them was the peel's neighbour test: the SURNAME SITE stepped past the family name onto the given name (`양. 지훈` cut 지훈 in half — given `양.`, middle `지`, family `훈` — and now reads given `지훈`, family `양.`), the ORDER RULE fell back to positional (`양 지훈.` lost family-first and now keeps it; `田中 太郎.` likewise), and the segmenter's neighbour precondition missed a writer-drawn boundary (`山田太郎 田中.` consulted a pluggable segmenter on `山田太郎` as if it stood alone, and is blocked now). The issue weighed two candidate fixes and chose neither: teaching the surname site to consult `is_suffix_strict` beside `effective_script`, or making the classification tolerant of a trailing period, which it called the broader one and expected to interact with #322. The broader one shipped, and the narrow one would have reached the filed name alone — `김.` is no suffix vocabulary, so `김. 민준` sits outside it entirely. - **The surname-site amendment was found by measurement during execution, not designed.** With the classification fold alone, `김. 민준` read given `김`, middle `.`, family `민준`: the token classified as hangul, the site matched `김` against its own head, and what was left over — the stop — BECAME the remainder. So the site matches on `text.rstrip(FULL_STOPS)` and cuts the core, a head being a prefix, which makes the offset that cuts the core cut the text; the stop rides with the remainder (`김민준.` divides as `김` + `민준.`, `김. 민준` as family `김.` plus given `민준`). `rstrip`, not `strip`: a LEADING stop would break the prefix argument. The reading is recorded in `ebf64db`'s comment at the site. - **Two readings JOINED the bundle, approved in session on 2026-09-09.** Neither was filed. FIRST, the glued-period honorific (rules/W3): a stop on the honorific's own word stood between the listed tail and the token's end, so nothing peeled and `田中さん.` and `김민준씨.` read as titles. The peel now matches the tail on the core and cuts BEFORE it, so both divide where their stop-less spellings do and the stop rides with the honorific. The move was free because W3 had recorded that reading as measured on 2026-09-05 and pinned by nothing, its own words being that neither string is a case row or a corpus line, so no row pinned those two readings; both are case rows now, tolerated. SECOND, H2's opening-abbreviation shape on a CJK word: `田中.` read title and now reads family `田中.`, `田中. 太郎` reads family `田中.`, given `太郎`. Han is what WITNESSES this one, and hangul cannot: hangul segmentation runs first, so `김민준.` never reaches the shape test and divides as `김` + `민준.` by the surname site instead. Latin and Cyrillic are untouched (`Smith. John` still reads title `Smith.`, `Проф. Иванов` title `Проф.`), and the veto is contains-any, so `Kim김. Smith` is refused a title too — deliberately: a word carrying a script with no abbreviations is not wearing an abbreviation's period. -- **Recorded limits, all five measured 2026-09-10 on this tree — and the segmenter one is a limit the review round CLOSED, kept in this bullet because this is where the claim stood. The halfwidth-kana one was CLOSED on 2026-10-03 by #594; see the CLOSED note that ends the halfwidth passage below, and decisions/W4.** HALFWIDTH KANA sits outside the script table, so it reads by the LATIN shapes at both period sites and not just at the initial one: `_vocab.is_initial("ラ.")` is `True` where `is_initial("ラ.")` is `False`, and H2's opening-abbreviation shape MISROUTES a longer halfwidth opener — `parse("タナカ. John")` gives title `タナカ.` where `parse("タナカ. John")` gives given `タナカ.`. Both because `_policy._SCRIPT_RANGES` excludes the halfwidth kana block U+FF65–FF9F on purpose — legacy bank and CSV data writes it, and the table's comment calls that a separate normalization problem, #272's separator handling touching only the halfwidth DOT U+FF65 and none of the kana behind it — and `_NO_INITIALS`, being a tuple of Script members, silently inherits the exclusion. That is #322's second mechanism, restated here because it is the one the bundle did NOT close. An earlier wording of this bullet said "nothing misroutes today, every kana honorific being multi-character", which had the direction backwards: the misroute is at the SHAPE site rather than the peel, and being multi-character is what carries a halfwidth opener INTO H2's shape rather than what keeps it out. NOTHING PINS EITHER READING — no case row, no corpus line, for the halfwidth block or for `タナカ. John` — so both can move without the suite or the differential saying so. It stays a recorded limit rather than a fix: what to do with halfwidth kana is a normalization question (widen the script table, or fold widths at classification) and not a full-stop one, and the bundle answered the full-stop one. CLOSED 2026-10-03 by #594, which widened the table (decisions/W4 carries why not the fold): `タナカ. John` reads given `タナカ.` and `is_initial("ラ.")` is `False`, and both readings are pinned now — the case rows `ja_halfwidth_katakana_opener_with_a_period_is_a_name` and `ja_halfwidth_katakana_lone_name_with_a_period_is_not_a_title`, and `test_is_initial_script_repertoire`. RECOMPUTE: parse `タナカ. John` and `タナカ. John` and read `title` beside the name fields; `is_initial` alone reports only the shorter half. A PLUGGABLE SEGMENTER receives the CORE, the token with its trailing stops removed — the same string the vocabulary match reads, the surname site handing it `core` rather than `text` (corrected 2026-09-10, in the review round after this entry was first written: the site handed over the raw token, which this bullet recorded as a limit, and the fix costs nothing because a head is a prefix — every offset reported against the core cuts `text` in the same place, so the stops ride with the last piece and no answer of the segmenter's can make a stop a piece of its own). The shipped `ja` pack's `_wholly_japanese` therefore ACCEPTS `山田太郎.` now, where the raw token failed its repertoire test: measured 2026-09-10 with `namedivider-python` installed (`uv run --extra ja`), `parser_for(locales.JA, segmenter=locales.ja_segmenter())` reads `山田太郎.` as family 山田, given `太郎.` with a SEGMENTATION report scoring 0.44, where `d37b8ec` read title `山田太郎.` and the un-corrected branch read family `山田太郎.`; the already-divided `山田 太郎.` reads family 山田, given `太郎.` on this tree and given 山, middle 田, family `太郎.` at `d37b8ec`. Pinned by `test_a_consulted_segmenter_receives_the_core_without_the_stop`. A LEADING stop keeps the token out of the SURNAME site only, because the classification fold that gates it rstrips as it does: `.김민준` classifies as no script, never becomes a surname site and stays one whole word, reading given `.김민준` exactly as `d37b8ec` reads it (corrected 2026-09-10 in the same round: the fold stripped both edges when this entry was written, which moved the ROLE to family and — with a segmenter configured — offered the token to it raw, an answer of offset 1 dividing it into the stop and the name; the fold now matches the sites, and hiding such a token is the no-split rules/W1 already accepts). It does NOT keep the token out of the HONORIFIC PEEL, which consults no script at all — the tail alone is its license (rules/W2's Background) — so a leading stop does not defeat it: `.김민준씨` peels to `.김민준` and `씨` exactly as `김민준씨` does, pinned by `test_the_peel_reads_the_trailing_stop_only`. A BRACKETED CJK CREDENTIAL degrades: rules/S1's escape in `_extract` calls a clause suffix-shaped when it ends in an ASCII period, so `(김민준.) John Smith` is unwrapped and the H2 veto then leaves `김민준.` as name text the surname site divides — given 김, middle `민준. John`, family Smith, where 2.2.0 and `d37b8ec` read title `김민준.` — pinned as the tolerated `ko_name_with_a_period_in_a_bracketed_credential`. And a caller-authored NFD surname entry is composed at ingest, so raw NFD input then finds no entry and the name goes unsplit; pinned by `test_an_nfd_authored_entry_is_stored_composed`. +- **Recorded limits, all five measured 2026-09-10 on this tree — and the segmenter one is a limit the review round CLOSED, kept in this bullet because this is where the claim stood. The halfwidth-kana one was CLOSED on 2026-10-03 by #594; see the CLOSED note that ends the halfwidth passage below, and decisions/W4.** HALFWIDTH KANA sat outside the script table, so it read by the LATIN shapes at both period sites (`is_initial("ラ.")` was `True`, and `タナカ. John` read title `タナカ.` where `タナカ. John` reads given `タナカ.`), recorded as a normalization question rather than a full-stop one; CLOSED 2026-10-03 by #594, which widened the table (decisions/W4 carries why not the fold) and pinned both readings (`ja_halfwidth_katakana_opener_with_a_period_is_a_name`, `ja_halfwidth_katakana_lone_name_with_a_period_is_not_a_title`, `test_is_initial_script_repertoire`). A PLUGGABLE SEGMENTER receives the CORE, the token with its trailing stops removed — the same string the vocabulary match reads, the surname site handing it `core` rather than `text` (corrected 2026-09-10, in the review round after this entry was first written: the site handed over the raw token, which this bullet recorded as a limit, and the fix costs nothing because a head is a prefix — every offset reported against the core cuts `text` in the same place, so the stops ride with the last piece and no answer of the segmenter's can make a stop a piece of its own). The shipped `ja` pack's `_wholly_japanese` therefore ACCEPTS `山田太郎.` now, where the raw token failed its repertoire test: measured 2026-09-10 with `namedivider-python` installed (`uv run --extra ja`), `parser_for(locales.JA, segmenter=locales.ja_segmenter())` reads `山田太郎.` as family 山田, given `太郎.` with a SEGMENTATION report scoring 0.44, where `d37b8ec` read title `山田太郎.` and the un-corrected branch read family `山田太郎.`; the already-divided `山田 太郎.` reads family 山田, given `太郎.` on this tree and given 山, middle 田, family `太郎.` at `d37b8ec`. Pinned by `test_a_consulted_segmenter_receives_the_core_without_the_stop`. A LEADING stop keeps the token out of the SURNAME site only, because the classification fold that gates it rstrips as it does: `.김민준` classifies as no script, never becomes a surname site and stays one whole word, reading given `.김민준` exactly as `d37b8ec` reads it (corrected 2026-09-10 in the same round: the fold stripped both edges when this entry was written, which moved the ROLE to family and — with a segmenter configured — offered the token to it raw, an answer of offset 1 dividing it into the stop and the name; the fold now matches the sites, and hiding such a token is the no-split rules/W1 already accepts). It does NOT keep the token out of the HONORIFIC PEEL, which consults no script at all — the tail alone is its license (rules/W2's Background) — so a leading stop does not defeat it: `.김민준씨` peels to `.김민준` and `씨` exactly as `김민준씨` does, pinned by `test_the_peel_reads_the_trailing_stop_only`. A BRACKETED CJK CREDENTIAL degrades: rules/S1's escape in `_extract` calls a clause suffix-shaped when it ends in an ASCII period, so `(김민준.) John Smith` is unwrapped and the H2 veto then leaves `김민준.` as name text the surname site divides — given 김, middle `민준. John`, family Smith, where 2.2.0 and `d37b8ec` read title `김민준.` — pinned as the tolerated `ko_name_with_a_period_in_a_bracketed_credential`. And a caller-authored NFD surname entry is composed at ingest, so raw NFD input then finds no entry and the name goes unsplit; pinned by `test_an_nfd_authored_entry_is_stored_composed`. - **Pickle compatibility, decided 2026-09-10 (Derek): a release-note line, not a forgiving load.** `Lexicon.__setstate__` re-runs `_normalize` over the loaded state and rejects any entry the fold changes, so a `Lexicon` pickled by 2.1.x or 2.2.x whose caller-added entries the widened fold now touches (a non-ASCII entry written with a wide stop such as `씨。`, or authored in NFD) raises `ValueError: incompatible Lexicon pickle: entries are not normalized (titles: 씨。); this state was not written by this version of nameparser` — a message that is wrong about provenance for exactly these pickles, which a released version did write. Measured 2026-09-10 (recompute: take `Lexicon.default().add(titles={"씨"}).__getstate__()`, add `씨。` to its `titles`, and load it into `Lexicon.__new__(Lexicon)`). The shipped vocabulary is unaffected (zero of 1735 strings change under NFC, and no shipped entry carries a wide stop). Two remedies were weighed: a release-note line with the fix a caller applies (rebuild the `Lexicon` from its source rather than unpickle it), or a load path that accepts state whose only drift is one fold pass away, re-folds it and warns naming the entries. The note was chosen: the load guard was designed to refuse rewritten data (the guarded-raise design #3-0-reevaluations records as right regardless of the in-a-minor friction) rather than become a fourth place caller data is corrected without a word, the population is caller-added CJK entries pickled across a minor, and a rebuild is the documented way to carry a `Lexicon` across versions. The 2.3.0 release note carries the line; `tests/v2/test_lexicon.py::test_unpickling_rejects_unnormalized_entries` is the pin of the refusal itself. - **CORRECTIONS this entry makes to entries above it.** #indic-honorifics' "Abbreviation marks and periods" bullet said `_normalize` "lowercases and strips edge whitespace and ASCII periods only — no NFC, no NFKC, no casefold anywhere in that module"; it now composes NFC for a non-ASCII word and strips all four stops. The VISARGA ARGUMENT still holds and holds for the same reason: ঃ U+0983 has no canonical decomposition, so NFC leaves it exactly where it was — recompute with that bullet's own one-liner, `Lexicon.default().add(titles={"মোঃ"}).titles & {"মোঃ"}`, still non-empty. #W1's 2026-07-29 §1a bullet said "segmentation MATCHING stays raw" and called the classification fold "the one deliberate exception"; matching is now folded too, so what stays raw is SEGMENTATION — the surname site's membership test and the peel's tail slice, which index the token's own text — and the folds are two, not one. #cjk-comma-demotion's 2026-09-05 five-parse bullet is amended in place: `田中さん 太郎.` no longer flips the order (the period is invisible to the classification, so it reads family `田中さん`, given `太郎.` as its no-period twin does), and `田中さん.` and `김민준씨.` no longer read as titles, so that bullet's summary — that the ONLY one of the five where a period changes nothing is the one where it sits on a word the split-off steps past — is superseded: the period changes nothing in four of the five now, and `田中さん 様.` is joined rather than alone. #initials-repertoire's principle bullet restated the veto phonologically ("morphemes or syllables"), which `_policy._NO_INITIALS`' own comment forbids in as many words — Devanagari is an abugida and Arabic an abjad, neither has letters in that sense and both abbreviate — so the bullet now states the CLDR criterion the constant actually uses. -- **What moved, measured 2026-09-10 and re-measured the same day after the review round.** SEVENTEEN names in `tools/differential/corpus*.jsonl` read differently against `d37b8ec`, and all seventeen are the bundle's own new rows, every one of them on the RADAR tier in `corpus_cjk_tolerated.jsonl`. Twelve are the CJK movers this bundle's landing commits wrote; the thirteenth, the wholly-katakana `マイケル.`, joined in the whole-branch review that produced this entry, reading given `マイケル.` where `d37b8ec` reads title `マイケル.` — the katakana arm of the same H2 veto, pinned as `ja_katakana_lone_name_with_a_period_is_not_a_title` in `tests/v2/cases.py`. The last FOUR joined in the review round after that, as pins on readings nothing held: three stop-bearing FAMILY_COMMA spellings (`김민준씨., J.씨`, `田中さん., V.`, `이, J.씨.`) and the bracketed `(김민준.) John Smith`, the recorded degradation in the limits bullet above. A leading-stop shape moves the same way but sits in no corpus at all: `.김민준씨.` — a stop no script ever writes, glued before a honorific-bearing word — read one whole given `.김민준씨.` at `d37b8ec` and now reads given `.김민준`, suffix `씨.`, pinned only by the stage tests for its two edges taken separately (`test_the_peel_reads_the_trailing_stop_only` for the leading stop, `test_peels_a_listed_tail_through_a_trailing_full_stop` for the trailing one) rather than by any corpus row or a combined case of its own. The three comma rows agree with their stop-less twins in every field once the riding stop is removed from the suffix (`씨., J.씨` vs `씨, J.씨`; `さん.` vs `さん`; `씨.` vs `씨`) — each differs from its twin in exactly that one field, by exactly the stop that rides — which is the peel's own "one name, two spellings" argument holding as far as a raw field comparison can show it; the bracketed row does not agree with anything and is not meant to. `田中. 太郎` stays there with the rest, though it is what WITNESSES rules/H2's Accepted clause added here: `tools/differential/compare.py`'s tier comment records the 2026-09-05 precedent for exactly this collision — where a demoted file holds a text a rules.md example line names, the EXAMPLE LINE moves into the tolerated rule and the row is not promoted, because marking the row alone leaves the name enforced and documented as demoted. So the clause's CJK reading is carried in W3's tolerated example block, H2's own block keeps the Latin control `Smith. John`, and all four #323 shape rows stay `tolerated`. THE POPULATION THAT COULD HAVE MOVED AND DID NOT is two different sets depending on which criterion is read, and both are worth having in front of you. Under the SHAPE this bundle is about — an edge full stop glued to a word carrying a classified character — the corpora hold 22 names: the seventeen movers and FIVE non-movers (`田中さん 様.`, `田中さん, 様.`, `김민준 씨.`, `김민준 양.`, `김민준, 씨.`), all five already in `corpus_cjk_tolerated.jsonl` before this bundle. Two of the twenty-two need the detector to be written carefully, which is why the recipe is spelled out below: the stop on `김민준씨., J.씨` and `田中さん., V.` sits before a COMMA, so a detector splitting on whitespace alone sees `김민준씨.,` and finds no edge stop (an 18 recorded earlier in the day came from such a detector, over a corpus four names smaller), and the stop on `(김민준.)` is inside a bracketed clause until rules/S1's escape unwraps it. Under the looser reading of an ASCII PERIOD ANYWHERE in a CJK-bearing name, the corpora held 20 distinct names on 21 rows before this bundle (19 rows in the tolerated file and 2 in `corpus_issues.jsonl`, `Dr 김민준씨, Jr.` being the one name on two of them) — unmoved by the katakana row, which is a name this bundle's own work wrote, not a pre-bundle count. The looser set is wider because it sweeps in periods sitting on LATIN tokens inside a CJK name — `毛 泽东 Dr.` and `田中さん, V.` — which is precisely what the veto is scoped not to touch, it reading the word and not the name. NEITHER the five nor the twenty moves — the seventeen movers are the whole of what did, and every one of them is a row this bundle wrote. An earlier wording of this bullet said "the twenty-one CJK names carrying an ASCII period": that counted rows as names and quoted the looser criterion beside the shape's argument. RECOMPUTE: parse the union of the corpus files on both trees and diff the seven fields and the ambiguity kinds; for the SHAPE population, take each name's tokens (`_pipeline._tokenize`, not a whitespace split) and ask whether any token's `strip(FULL_STOPS)` is shorter and still carries a classified character, then add the bracketed clause by hand; for the looser one, count distinct names and rows separately. The gate exits 0 at all four baselines with `radar unclassified: 0`; one ledger rule carries them, `fix(#322/#323)`, ELEVEN members at 1.4.0 and 2.0.0 and SEVENTEEN at 2.1.0 and 2.2.0 — the three hangul names the older ledgers hand to their native-script CJK rule are one part of the difference, 2.1.0 being the release that shipped that behavior, and the three FAMILY_COMMA rows are the other: at 2.0.0 none of the three diffs at all, and at 1.4.0 the two that do (`田中さん., V.`, `이, J.씨.`) are already claimed by the broad `fix(cjk-comma-compound)` and `fix(cjk-comma-honorific-peel)` rules that ledger carries, so adding them to this rule would only take a name off a rule that describes it — the same narrow-first reasoning that keeps the three hangul names out. -- **The invariant the rules imply, run over two inputs (AGENTS.md axis 11), measured 2026-09-10.** If an edge full stop is punctuation on a CJK word, then a CJK name written with one must divide exactly as the same name written without one. Written as a check over the corpora — for every corpus name carrying a classified character and an edge stop on some word, parse it, parse the spelling with every word's edge stops removed, and compare the seven fields with the stops removed from the FIRST parse's values too — 30 of the 34 such names agree (re-measured 2026-09-10 after the review round's four new rows; it read 28 of 30 before them, the katakana row マイケル. having joined the population and agreed). FOUR do not, and only three of those are about the parser. Two are the standing boundary, and neither is a CJK word: `毛 泽东 Dr.` (the period makes `Dr.` title vocabulary the trailing walk claims, where bare `Dr` is a name word) and `田中さん V.` (`V.` is a dotted numeral where bare `V` is the generational suffix) — both LATIN words wearing a Latin period inside a CJK name, which is exactly what the veto is scoped not to touch, it reading the word and not the name. The third is new and is the bracketed degradation the limits bullet records: `(김민준.) John Smith` reads given 김 / middle `민준. John` / family Smith where `(김민준) John Smith` reads nickname 김민준 — the period is what makes rules/S1 call the clause a credential rather than a nickname, so the two spellings are genuinely different inputs to `_extract` and the invariant should not hold across them. The FOURTH is the comparison's own artifact and not a finding: `이, J.씨.` and `이, J.씨` divide identically (given `J.`, family 이, suffix `씨.` and `씨`), but stripping the first parse's VALUES takes the period off the initial `J.`, where in the bare input that period is interior and survives. A normalization that runs over field values cannot tell a word-final stop in a value from a word-interior one in the name; that is worth knowing before reading the next round's count. RECOMPUTE with that description; it consults no rule statement, which is the point of the axis. -- **Parse cost, measured 2026-09-10 on py3.11** (`tools/perf/call_count.py --against d37b8ec`, and a script counting frames the way its `calls_for` does for the other names). The reference name never reaches the veto — `Dr.` is listed, so the title piece is taken before the shape test — and `parse=409.00 facade=446.00` is unchanged at both trees. On an H2-SHAPED OPENER the veto costs three to five frames: facade 304 → 308 for `Xyz. John Smith`, 385 → 390 for `Xyz. van Berg MA`, 231 → 232 for `Smith. John` (parse-only: 267 → 271, 348 → 353, 194 → 195). The number to quote is these and not "zero"; decisions/parse-cost's budget is about the reference name and this is off it. The largest movement measured is not the veto at all: `김민준씨.` goes facade 209 → 397, the peel now firing where the parse used to bail to a title reading, which is a whole segmentation path running rather than an added test. +- **What moved, measured 2026-09-10 and re-measured the same day after the review round**, against `d37b8ec` over every `tools/differential/corpus*.jsonl` name (recipe, the shape detector included, in PR #522): SEVENTEEN names move, all of them the bundle's own new rows in `corpus_cjk_tolerated.jsonl`; of the 22 corpus names of the bundle's shape (an edge full stop glued to a word carrying a classified character) the five that predate the bundle do not move, nor do the 20 pre-bundle names carrying an ASCII period anywhere in a CJK-bearing name; the leading-stop `.김민준씨.` moves too (given `.김민준`, suffix `씨.`) but sits in no corpus, pinned only by the stage tests for its two edges. `田中. 太郎` stays there with the rest, though it is what WITNESSES rules/H2's Accepted clause added here: `tools/differential/compare.py`'s tier comment records the 2026-09-05 precedent for exactly this collision — where a demoted file holds a text a rules.md example line names, the EXAMPLE LINE moves into the tolerated rule and the row is not promoted, because marking the row alone leaves the name enforced and documented as demoted. So the clause's CJK reading is carried in W3's tolerated example block, H2's own block keeps the Latin control `Smith. John`, and all four #323 shape rows stay `tolerated`. +- **The invariant the rules imply, run over two inputs (AGENTS.md axis 11), measured 2026-09-10** (PR #522): a CJK name written with an edge full stop must divide as its stop-less spelling does, and 30 of the 34 corpus names carrying a classified character and an edge stop agree; of the four that do not, `毛 泽东 Dr.` and `田中さん V.` are LATIN words wearing a Latin period, which the veto is scoped not to touch, `(김민준.) John Smith` is the bracketed degradation the limits bullet records, and `이, J.씨.` is the comparison's own artifact, a strip over field values not telling a word-final stop from the interior one of `J.`. +- **Parse cost, measured 2026-09-10 on py3.11** against `d37b8ec` (`tools/perf/call_count.py`; PR #522): the reference name is unchanged (`parse=409.00 facade=446.00`, `Dr.` being listed and taken before the shape test), an H2-shaped opener pays three to five frames for the veto (`Xyz. John Smith` facade 304 → 308), and `김민준씨.` goes facade 209 → 397 because the peel now runs a whole segmentation path where the parse used to bail to a title. diff --git a/docs/design/decisions/differential-ledger.md b/docs/design/decisions/differential-ledger.md index 5428fe6c..cf53f63d 100644 --- a/docs/design/decisions/differential-ledger.md +++ b/docs/design/decisions/differential-ledger.md @@ -1,285 +1,34 @@ # differential-ledger — the differential ledger's arcs -## differential-ledger — tooling decisions (2.1.0 release arc) +The differential ledgers were the classified gate 2.0 shipped behind: `tools/differential/compare.py` ran every corpus name through a released wheel and through the tree, and every name that read differently had to be claimed by a rule in one `tools/differential/expected_since_.toml` ledger per baseline (a `name_regex`, the `fields` it may move, an issue and a `why`), or the run failed. They proved what they were built to prove, that 2.0 changed nothing against 1.4 except the fixes the release log lists. As a standing check they made every later behavior change classify itself once per ledger, and the guards that policed the ledgers (`tests/v2/test_ledger_guards.py`, the contest pins, the recorded and watched diff rosters) outgrew them. #664 retired them for a committed snapshot on 2026-10-13. -Harvested 2026-08-15 from the release-arc session; measurements are that session's, spot-checked at landing. +Archived 2026-10-19 under rule 3 of the decisions folder's entry conventions (#759): eleven arcs of entries, about 33,000 words, were cut to the tombstones below, every one superseded by the retirement. The full text is in git — `git show fe85bb3d:docs/design/decisions/differential-ledger.md` — and in the PRs each tombstone names. The retirement entry is the one decision in force and stands in full; the passages other entries cite stand as one line each, and three Declined entries about surviving code stand as written. -- 2026-07-24 (rc1 arc, predating this section's window) — the TWO-CORPUS design: corpus_issues.jsonl (198 tracker-harvested names, 166 absent from the v1-test-bank corpus) exists because v1 test banks are structurally blind to anything 2.0 added; compare.py globs corpus*.jsonl and fails loudly on none. Its first catch was the leading-credential case ("Ph. D. John Smith" → suffix). +## differential-ledger, the arcs before retirement (2026-07-24 to 2026-09-05) -- 2026-08-05 #332 — ledger field vocabulary is Role's names, not the facade's: canonicalizing to first/last would have put an eighth place naming roles differently from Role inside the durable record, and the facade vocabulary is removed at 3.0. +- 2026-07-24 to 2026-08-07, the tooling arc (#332, #333, #334) — harvested 2026-08-15 from the release-arc session; measurements are that session's, spot-checked at landing. The two-corpus design (corpus_issues.jsonl beside the v1-test-bank corpus, since v1 test banks are blind to anything 2.0 added), ledger fields named by `Role`, and the sentinel-set over-match check (mechanisms.md#SENTINEL-SET-OVER-MATCH-CHECK). SUPERSEDED by 2026-10-13 #664 — except three of its Declined entries, below, which are about code and process that survive the ledgers. +- 2026-08, the dormancy arc (#328–#376) — measurements are that session's, the machinery verified present at landing. `[[never]]` exclusions sorted ahead of the rules, so an exclusion only ever removes a name; a rule that claims nothing is dormant and is refused. SUPERSEDED by 2026-10-13 #664 — +- 2026-08-27, the fields-only arc (#451) — `validate_rules` refuses a rule with `fields` and no `name_regex`, the one such rule having drifted from four names to seventeen with every guard green. SUPERSEDED by 2026-10-13 #664 — +- 2026-09-01, the corpus-tier arc (#468/#469) — every corpus carries a tier, and only the contract tier can fail the run. SUPERSEDED by 2026-10-13 #664 — +- 2026-09-01, the initials view (#484/#462) — `_initials` is a pseudo-field on both surfaces that enters a name's diff only when the seven roles and `_ambiguities` agree, so `initials()` is the one render view the gate compares and `capitalized()` stays invisible to it; it found #462 and the 2.0.0 per-word initials change, classified late in the 2.3.0 log. SUPERSEDED by 2026-10-13 #664 — the snapshot records `initials()` for every name. The 2.3.0 release log's per-word initials bullet points here; its evidence (the three clusters, 27 / 14 / 98 names, and the RECOMPUTE) is in the full text at fe85bb3d. +- 2026-09-02, the rule-order arc (#382) — an order-decided contest between two rules must be declared (`precedes_narrower`), narrow-first being the declaration-free default. SUPERSEDED by 2026-10-13 #664 — +- 2026-09-03, the recorded-shapes arc (#497) — the recorded diff shapes live in `compare._RECORDED_DIFFS`, where a run verifies them, and `_CROSS_RULE_WINNERS` keeps only the winner. SUPERSEDED by 2026-10-13 #664 — +- 2026-09-05, the watched-shapes arc (#501) — a shape may be recorded without a winner, in a second roster (`compare._WATCHED_DIFFS`). SUPERSEDED by 2026-10-13 #664 — +- 2026-09-05, the #501 adjudication — a contest row is owed wherever its boundary is argued, the ledger's own comments included. SUPERSEDED by 2026-10-13 #664 — +- 2026-09-05, the #498 completeness check — the declared-contest check extends from nested `fields` to intersecting and equal ones. SUPERSEDED by 2026-10-13 #664 — +- 2026-09-05, a pin names one rule (#508) — a `_CROSS_RULE_WINNERS` pin resolves to exactly one rule's issue and is compared by equality. SUPERSEDED by 2026-10-13 #664 — -Declined: +Cited elsewhere, kept as one line each: + +- The fields-only arc declined extending its rule to `'Donald mc'` (2026-08-27): `mc` is both particle and suffix vocabulary, so the comma-less name loses its family to `suffix` under rules/S2's Accepted clause, and P6's Accepted clause, which claimed the comma-less shapes keep their positional reading, was a rules defect, fixed 2026-09-07 under #454 (decisions/suffix-acronym-collisions). +- The initials view added `initials()` alone, deliberately (2026-09-01 #484); `family_base` was considered as a second derived view and not added. + +Declined (the tooling arc's, harvested as above, kept because they are not about the ledgers): - Policy annotations widened to input unions (2026-08-05 #334) — five documented spellings fail mypy and every one has a type-clean equivalent; widening would make every READER see a union, and reading is the commoner operation. A .pyi stub typing __init__ wide and attributes narrow was deferred as a 3.0 candidate (needs a parallel signature list with its own sync test). -- Closing the differential gate's default-policy ceiling (2026-08-04 #332) — a per-row policy in the corpus needs defined behavior for baselines that cannot construct newer policy fields; tests/v2/cases.py already covers opt-in paths per row, so the ceiling is documented instead. - `_check_tree` as is_relative_to(REPO_ROOT) (2026-08-05 #332) — accepts .venv/, build/ and dist/ copies inside the repo; the invariant is "is the source package", so the predicate is is_relative_to(REPO_ROOT / "nameparser"). -- Empty-string probe as the ledger over-match guard (2026-08-05 - #332) — `.`, `.+`, `\b`, `[\s\S]` all decline "" and still match - every corpus name; replaced by the sentinel set (mechanisms.md#SENTINEL-SET-OVER-MATCH-CHECK). -- "Lists every role" checked against all eight fields entries (2026-08-05 #332) — a seven-role list passed while omitting _ambiguities, which below baseline 2.0 cannot enter a diff at all; the check is against V2_FIELDS. -- A seed ledger rule with name_regex and no fields (2026-08-05 - #332) — it matched every CJK-bearing name and would have - classified all 89 diffs on the first pass, exiting 0 having distinguished nothing. -- 2026-08-07 #333 — the canonical-rule selector (keyed on literal - #271/#272 slug substrings) was deleted rather than repaired, - option A of three: split "every hand copy equals the table" (the sweep's job) from "the table did not change shape without a decision" (the span-bearing test's job), so a selector break can never take the decision gate out as collateral. This retired the slug taboo an earlier mechanisms.md entry carried. -- The B7 sanctioned-extra span (pre-#332 arc) — added, then removed as a pure loss when review measured that every B7-divided name already matches through its guaranteed classified flanks, so the span's only effect was absorbing punt-volat Latin regressions ("Gal·la Marcet" probes on both sides, recorded in PR #305's history). The cleanest "sanctioned extras must be earned" precedent the ledger has. - An issue for rotating DEFAULT_BASELINE (2026-08-07) — it recurs every release; it lives in the AGENTS.md release checklist beside the VERSION bump instead (#333 must land first). -Excluded (ledger name_regex patterns — one over-matching rule shadows the whole ledger, since name_regex rules sort first): "", -"(?:)", ".", ".+", "\b", "[\s\S]". Enforced by the sentinel-set -check. - -## differential-ledger, the dormancy arc (2026-08, #328–#376) - -The second ledger arc; measurements are that session's, the machinery verified present at landing. - -Decisions that landed: - -- 2026-08 #328 — `[[never]]` exclusions: ledger vocabulary for - "this shape must never be explained". classify() consults - exclusions before rules, which makes them MONOTONE — an entry only ever removes a name, never moves one between rules — so an exclusion's blast radius is exactly the names it captures, independent of rule order. -- 2026-08-12 #373 — `dormant = ""`: a rule explaining nothing fails the run in both directions, with three diagnoses (reverted / shadowed by X / refused by a [[never]] entry). Always on, not behind --strict: a check nobody passes a flag to is a check that doesn't exist. -- 2026-08 #373 — `--baseline 2.0.0` joined the release checklist: that ledger's rules had no dynamic coverage at all; measured clean at 90/0, so the gap was closed rather than documented. - -Declined: - -- Ambiguity reporting on multi-matching rules (#372/#373) — measured: 28% of claimed name×role pairs already have ≥2 matching rules (732 of 2619; 432 are one pair of rules alone). A report firing on 28% of what it inspects is wallpaper. The actionable slice shipped as the "shadowed by " diagnosis, which speaks only on FULL shadowing. -- A specificity floor for fields-only rules (#372/#373) — when this was proposed, exactly one fields-only rule existed across the ledgers, naming 3 of 7 roles; a six-of-seven floor matched nothing, and nothing would reveal it vacuous. Still declined: #451 banned the SHAPE instead (the fields-only arc below), which is a different proposal and leaves no ledger with a rule for a floor to measure. -- Specificity reordering of the rule sort (#328) — measured across all 751 names, on the ledger as it then stood: width-then-regex-length moved five rule populations and sent 17 names into the generic fields-only rule, draining the CJK-specific ones. No reading of the sort produces the "exactly one label changes" originally claimed; that figure was corrected on the PR. -- Unanchoring the honorific-suffix rule to reach glued forms (#376) — it would make a future suffix regression on 김지양 classify as a recognized honorific. A confidently wrong label is worse than a catch-all's honest breadth, and the gate reads the same either way. - -## differential-ledger, the fields-only arc (2026-08-27, #451) - -The third ledger arc, and the one that ends a shape rather than adding vocabulary. Measurements are this session's, at the counts each commit records. - -Decisions that landed: - -- 2026-08-27 #451 — `validate_rules` REJECTS a rule carrying `fields` and no `name_regex`. Such a rule claims every name whose diff fits those fields, and `_CORPUS_CLAIMS` records a regexless rule's reach as the WHOLE corpus — already at its maximum, so arrivals never move it and `test_every_rule_claims_the_recorded_share_of_the_corpus` passes whether it explains 4 names or 40. The one rule with the shape drifted from the four names its prose described to 25 before #372 cut it back, and then to SEVENTEEN after — decisions/H1 records that 14 -> 17 step and #410 taking three back — standing at fourteen across six unrelated behaviour families when this bundle opened, every guard green throughout, absorbing two open v2.2 bugs (#342, #397) as intentional 2.0 changes while AGENTS.md names the 1.4 summary as the release log's source for Behavior Changes. Free to state only because the fourteen were given rules first; a role-count floor was the earlier proposal and stays declined (#372/#373, vacuous). -- 2026-08-27 #451 — a rule whose declared `fields` EXCEED every diff it explains is a defect in its own right, not untidiness. Measured on the catch-all: it declared `{given, family, suffix}`, and by the time this bundle's first three commits had given its `{given, family}` names rules of their own, nothing it still explained moved `given`. Note WHEN the excess appeared, because it is the whole mechanism — the declaration was earned when the rule was written and became excess as its explained set SHRANK under it, which is exactly `fix(#424)` at decisions/H1, where #410 narrowed a diff from three roles to two and the rule kept matching by superset with no run naming it. A rule does not have to widen to become over-declared; the diffs beneath it only have to leave. The two together are why #452 was filed — nothing counts what a rule actually explains, only whether it explains anything. -- 2026-08-27 #451 — a ledger rule may classify a reading NOBODY WANTS, provided its `issue` says so. `fix(#342)` and `fix(#397)` carry NOT WANTED in the issue text, the cause-not-consent sentence decisions/P5 established for the #369 regression, and a delete-when-fixed instruction the dormancy check enforces once the fix lands. -- 2026-08-27 #451 — one vocabulary per alternation, FORCED rather than chosen. `_LatinCopy` carries one `vocabulary` and `test_latin_alternations_mean_something_the_vocabulary_ships` asserts exactly one roster key per alternation, so `jr` (SUFFIX_WORDS) and `mc`/`mp` (SUFFIX_ACRONYMS) cannot share one. That constraint forces one of the three splits, and forces it only given the roster as written: `_LatinCopy.vocabulary` is a hand-supplied frozenset, so declaring `SUFFIX_WORDS | SUFFIX_ACRONYMS` would let `(jr\.?|v|mp|mc)` pass the member checks — measured. Declining to write a union vocabulary is a judgement, that a rule's alternation should name one wordlist a reader can go and check, and it is recorded here as one rather than as a constraint. The other two are forced separately and are recorded below under the declined single-rule tail: `m\.?a\.?` cannot be an alternation member at all, and the numeral alternation copies no wordlist. Four rules, three independent reasons, none of them taste. -- 2026-08-27 #451 — a ledger rule's character class must not reach a codepoint the IMPLEMENTATION does not act on, and is pinned to that. A SUBSET, not a copy: `test_the_emoji_boundary_rule_copies_the_dividing_ranges` asserts `claimed - divides == set()` and nothing in the other direction, deliberately — `_EMOJI_RANGES` has FOUR blocks and the rule copies the two astral ones, leaving the BMP pair out because no corpus name reaches them through its token anchor. Reading it as equality would instruct the next author to pre-claim the BMP half, which is the #372 shape this same section leans on. The emoji rule's first draft spanned `U+1F300-U+1FAFF` while the two ASTRAL blocks of `_tokenize.py`'s `_EMOJI_RANGES` are narrower than that span: it claimed 1072 codepoints where the parser leaves the token whole, so its own prose ("an emoji inside a token divides it") was false for every one of them and a `{given, family}` diff with another cause would have classified as intended. `test_the_emoji_boundary_rule_copies_the_dividing_ranges` holds it, and fails on the draft it replaced. -- 2026-08-27 #451 — `_sorted_rules` is KEPT although the ban makes it the identity on every ledger that loads. Verified against all three: the ban removes the only shape that could occupy its second tier, so file order settles every tie there is and the function reorders nothing. Kept because it is what makes the ban safe to state for any reader that does not call `validate_rules` first — a future tool, a REPL, a test fixture. Deleting it would move the guarantee from the code into a convention, which is the trade #451 was filed to undo. Recorded as a decision rather than as a consequence precisely so a later dead-code sweep finds the reasoning before the `return`. -- 2026-08-27 #451 — past-tense counts stay in a ledger comment; live ones do not. "It claimed 25 until #372" is a fixed fact. "Four corpus names, and the prose fits all four" was written when the rule explained four and still read that way at fourteen, because nothing recomputes it. Where a count is owned by another comment, point at the owner rather than restating it — two copies means one gets updated and the other does not. -- 2026-08-28 #452 — a rule's declared `fields` must EQUAL the union of the diffs it explains, checked by `compare.py` at the end of every run and failing it like an unexplained diff. The statement is exact rather than heuristic, which is what makes it cheap: `classify()` already requires `declared >= union` for the rule to match the names it matches, so the only possible error is the other direction, and the union is simultaneously the check and the repair. Narrowing to it cannot orphan a name, since every name a rule explains contributed to it. Measured before landing: 3 of 67 explaining rules over-declared at 1.4.0, 5 of 58 at 2.0.0, 6 of 51 at 2.1.0 — all fourteen narrowed first, so the check was silent the day it arrived. -- 2026-08-28 #452 — over-declaration is BASELINE-RELATIVE, and each ledger is measured on its own run rather than copied. `fix(#296) a lone post-comma credential is a suffix` declared `{family, given, suffix, title}` in all three ledgers: exactly exercised at 1.4.0, where v1 reads the pre-comma word as `first` and all four roles move, and over-declared at both 2.x baselines, where the same behaviour moves only `{suffix, title}` — which is all it declares in those two files now. A reader comparing the three sees one rule with TWO different field lists and should read that as correct rather than as drift. Two and not three, measured: the 2.x pair narrowed to the same set, so the split is 1.4.0 against both 2.x ledgers, not one list per file. -- 2026-08-28 #452 — NO escape hatch, decided rather than deferred. `dormant` already covers the explains-nothing case in both directions, a rule with no `fields` has nothing to over-declare, and the ledger's own doctrine — "a rule that pre-claims shapes it has never seen is the #372 failure mode" — makes strictness the existing principle. Accepted cost, stated so it is not rediscovered as a surprise: the first rule that genuinely needs a wider declaration has to argue for a key the way `dormant` was argued for in #373, rather than reaching for one that already exists. -- 2026-08-29 #456 — a rule carrying `name_regex` and no `fields` is REJECTED, the symmetric twin of #451's ban. It narrows by name and by nothing else, so on any name its regex reaches it claims every diff shape there is — measured, 255 shapes from baseline 2.0 on and 127 below it. #452 is what made it urgent rather than merely untidy: `over_declared_rules` skips a rule with no `fields`, correctly, since one declaring no roles cannot over-declare them — so deleting the line is the cheapest way to silence an OVER-DECLARED failure AND the most permissive thing that can be done to the rule, which is the #372 failure mode reached by following a gate error message. Free to enforce on #451's own terms: measured, 0 of 179 rules across the three ledgers had the shape. After it, every rule carries both keys, and the three rejections in `validate_rules` read as one rule — narrow by name and by role, or it is not a rule. NO escape hatch for a "genuinely unbounded" rule, declined until one appears, same call and same reason as #452's. - -Found rather than decided, and worth as much: - -- **A fields-only rule silently falsifies OTHER comments.** The 2.x ledgers' idiom "such a name should arrive UNEXPLAINED and be read once" holds there because no rule above the one making the promise claims that diff shape. A fields-only rule is the extreme way to break it, not the only way: measured while fixing #452, a `{family, maiden}` diff on `fix(#367)`'s own probe goes to `fix(#274)` in the 1.4.0 ledger, which has no fields-only rule anywhere. Note the shape, not the role: `{given, family, maiden}` on the same probe is claimed by nothing, because `fix(#274)`'s `fields` stop at maiden/middle/family. That is the point — the question is only ever answerable per shape. The reusable check is "does anything above me claim this shape", which is `classify()` driven directly — NOT "is there a catch-all". In the 1.4.0 ledger it was false for anything of the catch-all's shape, and nothing said so. Measured while the rule still lived, by driving `compare.classify()` directly over the seven `_MUST_NOT_MATCH` probes of the two rules #451 ported, at diff shape `{given, family}`: ZERO arrived unexplained. Four went to the catch-all ('Mcintyre Smith Jr.', 'Los Santos', 'John Sir de Mesnil', 'Sir Smith'), two to `fix(comma-precomma-family)` ('McDonald, Ronald', 'Smith, Sir de Vaux'), and one ('Ste Marie') to the neighbouring `fix(#360) ste` rule that exists to claim it. Driven through `classify()` rather than read off a gate run, deliberately: six of the seven are in no corpus, so a run would never reach them — which is also why the promise they carried could go false unnoticed. The blast radius of the shape was never just its own heading. -- **Copy-paste is how a ledger comment goes false without being edited.** All three ledgers carried the same `fix(#379)` comment; four of its sentences were true only in the 1.4.0 file they were written for, including one citing `fix(suffix-routing)`, which exists in no 2.x ledger, and one saying a name "falls to the fields-only catch-all" twenty lines after the same file says no fields-only rule exists there. Corrected in `8237b54`. A cross-ledger claim needs re-deriving per file, not copying. - -Declined: - -- A `[[never]]` for #342 and #397 (2026-08-27) — honest, and it reddens the gate: an excluded name reports UNEXPLAINED and exits non-zero, so every release would block until both bugs are fixed. Both are on v2.2, so the window is real. The cause-not-consent rule says the same thing in the ledger and keeps the gate green. -- One replacement rule with a `(jr|v|mp|x|m\.?a\.?|mc)` tail (2026-08-27) — measured, not argued: `m\.?a\.?` matches the corpus fragment `'M.A.'`, which `_normalize` leaves as `'m.a'` and is not a `SUFFIX_ACRONYMS` entry, so the member guard rejects it — the same finding the `ambiguous-surname-acronym` rule already records. `x` is in no vocabulary set at all, reaching `suffix` through the trailing-roman-numeral fork, so it could not be a member either. -- A literal alternation of all 14 absorbed names (2026-08-27), on #413's literal-anchored precedent — kept as the fallback if a reading turned out contested, and not needed: every family had an existing rule in another ledger, a decision to cite, or an open issue. -- Extending #451 to the trailing-numeral ambiguity (2026-08-27) — `'Mohamad X'` loses its family to `_ROMAN`, and by C-i's "under uncertainty, default to AMBIGUOUS" a trailing `X` IS borne as an ordinary name in that position (Malcolm X). Numerals reach `suffix` through a pattern rather than a vocabulary set, so there is no ambiguous SUBSET for `x` to join — but the fork does not go unreported: measured, `parse('Mohamad X')` and `parse('John V')` both carry `AmbiguityKind.SUFFIX_OR_NAME`, which `_assign` raises from the fork itself because "V/X/I are ordinary middle initials, so taking it as a suffix is a call, not a fact". The contrast worth keeping is with the acronym half: `parse('Aishwarya Rai')` loses its family and reports NOTHING, because `rai` is unambiguous vocabulary. So the numeral fork already flags its own uncertainty and the vocabulary path does not — which is an argument about `SUFFIX_ACRONYMS_AMBIGUOUS`, not about `_ROMAN`, and it belongs to #342 rather than here. `'Malcolm X'` is in no corpus (`'Mohamad X'` is, and is what this bundle classified); no open issue on the numeral half. -- `explains = N` per rule (2026-08-28, #452's own first sketch) — catches a rule explaining fewer NAMES even while its fields stay exercised, which the field check does not. Declined on maintenance cost rather than merit: 179 numbers across the three ledgers, every one of them liable to move whenever a corpus does rather than only when a rule does. Measured on the precedent — #414's harvest moved 17 of the 30 `_CORPUS_CLAIMS` entries that existed then and added 14 more, parsing the roster at `7a10689^` and `7a10689`. (That roster's own comment said "Twenty of these moved" and was wrong; measured three ways — that commit, the whole of PR #415, and the backtick-harvest commit after it — the figure is 17 every time, and the comment now says so with the recipe.) The field check needs no number and cannot go stale. This is the design to revisit if a name-count shrink ever bites. -- Extending it to `'Donald mc'` (2026-08-27) — `mc` is `SUFFIX_ACRONYMS` AND `PARTICLES`, the collision C-ii decided for `vd`, and C-ii reaches this shape only through P6's attachment, which is scoped to the comma form. The authority is rules/S2's Accepted — "an unambiguous suffix is consumed even when that leaves no family name at all" — and NOT P6's scope note, which was this entry's first citation and does not support it. P6 promises the comma-less shapes "keep their positional reading", and for the words that are both particle and suffix vocabulary it does not hold: measured, `'Jong de'` and `'Smith do'` keep theirs (family 'de', family 'do') while `'Donald mc'` and `'Smith vd'` lose the family entirely to `suffix`. So `'Donald mc'` is out of #451's scope on S2's statement, and P6's own Accepted clause is falsified by two of the three words it sweeps in — its only example, `"Jong Anke de"`, is `de`-shaped, which is why no test catches it. Recorded here as a rules.md defect rather than a ledger one, and FIXED 2026-09-07 under #454: P6's Accepted clause now states the exception, naming mc and vd as the two words the positional reading does not hold for. The measurement above is unchanged and is what the repaired clause says. See #suffix-acronym-collisions. - -## differential-ledger, the corpus-tier arc (2026-09-01, #468/#469) - -The fourth ledger arc, and the first to change what the gate is FOR rather than what a rule may say. Mechanics — the tier table, the promotion recipe, the shape key, the `corpora:` line — are owned by tools/differential/README.md and not restated here; these are the decisions behind them. Counts are this session's, each recomputable by running `uv run python tools/differential/compare.py` (add `--baseline 1.4.0` for the skip figures) over the checked-in corpora and reading its `corpora:` and `corpus:` lines. - -Decisions that landed: - -- 2026-09-01 #468 — every corpus carries a TIER, and only the CONTRACT tier can fail the run. `corpus.jsonl` (scraped from v1's test banks) and `corpus_issues.jsonl` (harvested from the tracker) became RADAR: their diffs still classify against the ledger, so release notes keep their grouping, but an unmatched one prints under `UNCLASSIFIED (radar)` and the run still exits 0. `corpus_cjk.jsonl`, `corpus_rules.jsonl` and `corpus_shapes.jsonl` keep the full discipline. The reason is a VOCABULARY gap and not a volume one: the gate demanded a rule-precision explanation for names nobody chose, and the honest explanation for many of them was "this string exercises nothing" — which no ledger key can say: a rule means "this diff is intended and here is why", an exclusion means "this shape must never be explained", and neither is the sentence wanted. A rule invented to say it would be the #372 shape by construction, since the only thing it could narrow on is which file the name happens to sit in. `_CORPUS_TIERS` in compare.py is the roster and is fail-closed like `_CORPUS_FLOORS`: a corpus added without a tier is a hard error, so the split forces a decision when a corpus arrives rather than defaulting one. The tier rule is about UNMATCHED diffs, and two recorded things outrank it on a radar name — a `[[never]]` exclusion (two bullets below) and a contest row in `_RECORDED_DIFFS` (the watched-shapes arc, 2026-09-05, which states the rule and its ground); a watched row that fails does so on an entry whose tier reads contract — the default-order entry when the name has one, the first-loaded otherwise — which is this rule applied and not an exception to it. -- 2026-09-01 #468 — nothing is DELETED to quiet the gate; promotion is the answer to "then how does a radar name ever bind again". A name enters the contract by being CHOSEN — a tests/v2/cases.py row plus a shape tag, which puts it in `corpus_shapes.jsonl` at the next regeneration. Deleting the string was the alternative and is worse in both directions: it loses the radar signal, and it makes the gate's own population editable by whoever finds the gate inconvenient, which is the one edit a differential gate must not make cheap. A meaningless radar string costs one parse and one report line against a comparison of 1120 names today (1113 when this bullet was written, hours earlier the same day; #486's bullet below carries the arithmetic). -- 2026-09-01 (decided in review) — a `[[never]]` exclusion OUTRANKS the tier: a diff it refuses reports UNEXPLAINED and exits 1 on either tier. Not a carve-out but the tier's own principle applied — the split governs names nobody looked at, and an exclusion is chosen exactly as a rule is, someone having written its `why` and its `examples`. It is also what the shipped ledgers already promise, and the measurement is what settled it: the 1.4.0 ledger's trailing-`Ph. D.` exclusion, whose `why` says "a diff here is a regression", reaches exactly three corpus names (`John Smith Ph. D.`, `John Smith, Ph. D.`, `John Smith, Ph.D.`) and every one of them lives in `corpus.jsonl` or `corpus_issues.jsonl` alone. Reading the tier as outranking the exclusion would have retired that entry outright, silently, in the commit that split the corpora. Recompute by matching each `[[never]]` entry's `name_regex` over the corpus files and reading which files the matches come from. -- 2026-09-01 (decided in review) — a ledger RULE may narrow by comparison order, through a new `orders` key; it is optional, and its absence is the order-blind behavior every rule written before shape-tagged entries has. Forced by the first order-scoped rule anyone wrote: `feat(#395)`'s fold on `de la Cruz Juan Carlos` moves `{family, given, middle}` under the two family-first orders, and the most plausible regression of that very change — the fold leaking into the DEFAULT order — moves those same three roles on that same string, which `corpus_rules.jsonl` carries as a bare string and therefore compares. Order-blind, the rule describing the feature would have absorbed the regression of the feature and called it intentional: #372's failure mode reached from a new direction, and on the one name best placed to hide it. The legal set is BORROWED from shapes.py rather than hand-copied (`_legal_orders`), the same call build_cjk_corpus.py makes for the script table — an order no shape declares is an order no comparison runs under, so a rule scoped to a typo could only ever be dormant. Amended 2026-09-01 (decided in review): the legal set gains a `DEFAULT` SENTINEL, and the two legacy rules that needed it were scoped. Found when the reviewer traced the absorption running the OTHER way from the one above — the order-blind `fix(#399) a maiden marker bounds the particle chain that swallowed it` rule reaches `de la Cruz née Vega`, which `corpus_shapes.jsonl` compares under both family-first orders, and it sorts AHEAD of the two rules written for that name, so a family-first-only `{family, maiden}` regression there would have classified as intentional. The fix a rule wants is `orders`, and no rule could say "default order only": the members are constant NAMES, the default order is the absence of one, and TOML has no null inside an array. So `"DEFAULT"` names it. `fix(#399) …that swallowed it` took `["DEFAULT"]` in both 2.x ledgers and `fix(#296) … 'John Smith, Dr.' keeps its split and its title` took `["DEFAULT", "FAMILY_FIRST"]`, those being the comparisons each one's diffs actually come from. Order-blind rules stay legal — every ledger written before shape tags is full of them — so the absorption is made VISIBLE rather than illegal: a run prints an `ORDER-BLIND` block for each (issue, name, order) where a rule with no `orders` key explained an order-bearing diff, informational and outside the exit code. Recompute by running the gate at each of the four baselines and reading that block; it is empty at all four today. -- 2026-09-01 (decided in review) — EXCLUSIONS stay order-blind, and `validate_exclusions` rejects an `orders` key as unknown. The dormancy arc's monotone-refusal reasoning is the whole argument: an exclusion only ever removes a name from classification, so the widest thing an over-wide one can do is make a name report UNEXPLAINED and fail the run. There is no silent direction to protect against, and the narrowing a rule needs is the one an exclusion must not have — "never explain this, but only under FAMILY_FIRST" is a promise with a hole in it. -- 2026-09-01 #469 — "some names are only supported under one order" is answered by the SHAPE rather than by a ledger exception. shapes.py records, per input shape, the `name_order` the shape is an input FOR and the oldest baseline whose worker can honor that order; `compare.py` parses a tagged entry under that order on both sides, compares it on the v2 surface alone (the facade is the v1-compat surface, and a family-first name is not a v1 contract), and leaves out an entry whose run predates its `min_baseline` — reported as `skipped N names tagged shape(s) [...]` and counted in the `corpora:` line, so a shrunken comparison is never silent. Structural, and that is the point: nothing has to be written into any ledger to say a family-first name is not a default-order promise. Today's arithmetic, all of it recomputable from the checked-in files and one gate run: 13 shape entries, of which 4 dedupe into names already compared under the default order, for 1113 comparisons; shapes 4 and 5 declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1106 names compare there. -- 2026-09-01 #486 (later the same day, so read this bullet as superseding the arithmetic in the one above rather than the two figures disagreeing) — the shape 1-3 variation matrix filled the given-first half of the inventory, which the feature work that authored shapes 4 and 5 had left at whatever the pipeline PRs happened to tag. 14 existing case rows gained a tag and 3 rows were authored for slots no row instantiated, taking `corpus_shapes.jsonl` from 13 entries to 30 and shapes 1/2/3 from 1/4/1 names to 9/11/3. The arithmetic moves with it and the recipe is unchanged — run `uv run python tools/differential/compare.py` and read its `corpora:` and `corpus:` lines, adding `--baseline 1.4.0` for the skip figures: 30 shape entries, of which 14 dedupe into names already compared under the default order, for 1120 comparisons; shapes 4 and 5 still declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1113 names compare there. The dedupe count is the half worth reading, because it is what the tier promotion looks like from this side: 8 of the newly tagged names were in a RADAR corpus only, so the tag moved them into the contract tier without adding a comparison, and 2 more were already contract through the rules corpus. None of the 17 names #486 tagged or authored needed a ledger rule — the seven of them that were in no corpus at all diff at no baseline, and intentional stayed 226/205/113/0 across the change. Not a claim about all 30 entries: the seven shape 4/5 entries are classified by ledger rules today, and scoping exactly those rules is what the `orders` bullet three above is about. -- 2026-09-01 #469 — the CJK arrangement is deliberately OUTSIDE the shape inventory. Whether an unspaced CJK name is a third family-first shape is the open question on #469, and `corpus_cjk.jsonl` — contract-tier already, generated from the case table already — covers that ground meanwhile. Tagging it in would have to ANSWER the question first, since a shape row cannot be written without a `name_order` and a `min_baseline` for it, and the inventory is a table of arrangements rather than where that argument belongs. - -- 2026-09-01 #469 (later the same day, so read this bullet as SUPERSEDING the one above rather than the two disagreeing) — the CJK arrangement is IN the inventory after all, as shapes 6 and 7, and the bullet above is right about why it could not be until the question was answered: a shape row cannot be written without settling what it admits. What settled it was splitting the question the bullet treats as one. There is no single "unspaced CJK name" shape — there is a pure family-first arrangement (6) and a source-order transcription listing (7), and the composed forms are not arrangements at all. Both take `order=None`, which is the part that had looked impossible: the family-first reading is SCRIPT-carried rather than declared, so a pure shape 6/7 string already parses correctly under the DEFAULT policy and there is nothing for `order` to name. `min_baseline` is then documentary rather than a skip trigger, an asymmetry with shapes 4/5 that tools/differential/shapes.py's docstring states so it reads as designed. The doctrine half is its own entry (cjk-comma-demotion above); what belongs here is that `corpus_cjk.jsonl` no longer "covers that ground meanwhile" — the ground is split between it and a radar file now. -- 2026-09-01 #469 — the `tolerated` flag on a case row is the demotion's VEHICLE, and it is recorded here rather than as a mechanisms.md entry. It is an explicit declaration, reviewed one row at a time exactly as a `shape=` tag is, mutually exclusive with one, and restricted to rows whose text bears a classified codepoint; `build_cjk_corpus.py` partitions on it, and a text marked on one row and not another is a hard error, the flag being per-text in effect. Why not a mechanisms entry: that catalog is keyed by RECURRING problem shape and its contract statements are citable verbatim from code comments, and this is one arc's device with no second instance and no comment needing to cite it — the tier mechanics it rides on are already owned by tools/differential/README.md, so an entry would restate a source rather than say something the source does not (the #473 lesson under decisions/review-agent-single-source). If a second arc ever needs "record the behavior, promise nothing", that is the point to promote it and this bullet is the first instance to cite. - -## differential-ledger, the initials view (2026-09-01, #484/#462) - -The fifth ledger arc: the gate gains a column it never had. `initials()` is a derived render view, so a change in it with every role identical was invisible to the seven-field comparison by construction — the `middle_as_family_fold_order` row's `notes` in `tests/v2/cases.py` say as much, and #408 moved initials for a whole minor with the gate green. Mechanics (which rows carry `_initials`, the roles-identical guard, the one-field rule constraint) are owned by tools/differential/README.md; these are the decisions. - -- 2026-09-01 #484 — `_initials` is a pseudo-field on BOTH surfaces, the facade's at every baseline (1.4.0 has `initials()`) and the core's from 2.0 on, and it enters a name's diff ONLY when the seven roles and `_ambiguities` agree on every compared surface. The alternative — strict subset semantics, as `_ambiguities` has — was measured and rejected: names whose roles AND initials both move are claimed by 53 of the 72 explaining rules at 1.4.0, 43 of 66 at 2.0.0 and 36 of 59 at 2.1.0 (recompute below), every one of which would have had to grow `_initials`, and a rule declaring it absorbs any initials change on its names anyway, so the churn bought no discrimination. A role move drags its initials with it; the pseudo-field is a render-layer instrument, not a second copy of the role comparison. Consequences a rule author must know: `fields = ["_initials"]` is a rule of its own kind and lists nothing else — validate_rules refuses the mix as dead; and a count taken from the `_initials` summary is a FLOOR on initials movement, not the population (AGENTS.md carries the 83 / 28 / 55 measurement). `family_base` was considered as a second derived view and NOT added (YAGNI; adding one is the same two rows per surface, recorded here rather than on the issue). -- WHAT IT FOUND, measured 2026-09-01 by patching compare.py's own worker to emit `initials()` and comparing against the tree — and measured on THIS BRANCH'S BASE, before `31623d9` fixed #462, which is the tree the parenthetical describes: there the tree matched 2.2.0 exactly, 0 diffs of any kind. Three clusters of initials-only names, 140 at 1.4.0 and 28 at each of 2.0.0 and 2.1.0. On the SHIPPED tree the same recipe gives 126 at 1.4.0 and 42 / 42 / 14 at 2.0.0 / 2.1.0 / 2.2.0 (re-measured 2026-09-02; the gate itself is the recipe now, see RECOMPUTE). The two readings reconcile through cluster (ii) alone — 126 + 14 = 140, 28 + 14 = 42, 0 + 14 = 14 — the 14 #462 names having moved from the 1.4.0 side to the 2.x side when the fix landed. (i) The R2 readmission, 27 names at 1.4.0, 2.0.0 and 2.1.0 — `Anh Do` → `A. D.`, `Juan van der` → `J. v. d.` — shipped in 2.2.0 with a release note and the R2 entry above but no ledger rule, because none could see it; now `fix(#385/#402)`, a literal list of the 27 (the shape "a part of nothing but particles" is no property of the raw string), plus `de los Santos` under its own `fix(#360)` rule since ITS initials moved when `los` became a particle, not with R2. (ii) The facade dropping a middle `E`/`Y` initial, 14 names at 1.4.0 only, all but one radar tier — the population beside the count, per docs/design/AGENTS.md's population rule, is the names carrying such a letter in the MIDDLE or FAMILY group, which is the only position the bug reached, and there are 14 of them: 14 of 14 in that position moved. The `fix(#462)` rule's regex (`(?:^|[\s,])(?:[EY]|[EeYy]\.)(?=[\s,]|$)`) reaches 18 corpus names, four more than the population, and the other four (`E Anne D`, `E Jones`, `E Maria`, `Y. L.`) carry the letter in the GIVEN group, which has always initialed every word it holds, so nothing moved. The discriminator is the GROUP and not the position: `E Anne D,Leonardo` leads with the E too, but its comma re-roles the run into the family group, and it is one of the 14 that move: #462, a 1.4 parity break shipped in 2.0.0 and recorded nowhere. Fixed in the same PR (R3's bullet below); the 14 then move against the three 2.x baselines instead and carry a `fix(#462)` rule in each. (iii) Per-word grouping, 98 names at 1.4.0 only: 1.4.0 initialed one group per `*_list` element and rendered a multi-word element's letters with the separator and no delimiter — `Juan Velasquez y Garcia` → `J. V G.` at 1.4.0 and `J. V. G.` here, `Abdul Salam Hassan` → `A S. H.` and `A. S. H.` — and the 2.0 facade initials each word. Measure with the FULL name, not with the run alone: `Velasquez y Garcia` and `Abdul Salam` as whole inputs give `V. G.` and `A. S.` at 1.4.0 and here alike, the run being the family and initialed per word either way, so the bare forms this bullet illustrated with until 2026-09-02 reproduced nothing. Strip periods and spaces and all 98 agree with 1.4.0 letter for letter. Changed in 2.0.0, mentioned once in passing (the #408 bullet under R3: "v1's initials GRANULARITY"), never classified. Classified LATE, on purpose: mechanisms.md#FACADE-CONTRACT promises 1.4-warning-free code keeps working except for release-log-classified fixes, and a classification three minors overdue is still owed, so the 2.3.0 log carries a bullet that says the behavior has held since 2.0.0. Four rules rather than one, because the Latin-alternation guard keys a rule to a single copied vocabulary and the 98 copy three (CONJUNCTIONS 66, BOUND_GIVEN_NAMES 19, PARTICLES 11) plus the `Ph. D.` merge (2). The connective rule cannot claim cluster (ii), and it takes BOTH of its exclusions — the case-sensitivity and the trailing whitespace lookahead — to keep it out. Of the 14, 7 carry a bare capital `E` and 7 a dotted `E.`/`e.`: the case-sensitivity excludes the bare capitals, and the alternation's trailing whitespace lookahead excludes the dotted ones whatever their case, a period not being whitespace. Since the fix in `31623d9` those names agree with 1.4.0 and diff only against the 2.x baselines, so neither half is keeping two live clusters apart today. What they do is make any FUTURE facade initials change on these shapes surface as UNEXPLAINED at 1.4.0 rather than be absorbed here as per-word grouping. Gate totals moved from 226 / 205 / 113 / 0 intentional to 352 / 247 / 155 / 14, zero unexplained, exactly the new rules' explained counts. -- RECOMPUTE: load `tools/differential/compare.py` by path; build the entries as main() does (`_load_entries`, `_load_shapes`, `_CORPUS_TIERS`, dedup by (name, order), drop order-bearing entries below their shape's minimum); patch `_worker_source` to append `_initials` to both rows; `_run_worker`; compare against the tree's facade and core. TWO RECIPES, and only the second of them needs that patch. (a) The CLASSIFIED TOTAL per baseline is read off the gate itself, no patch and no in-memory run: `compare.py --baseline X` prints a `## issue (N)` heading per rule, and the total is the sum of the headings of the rules whose `fields = ["_initials"]` — at 1.4.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and the four `fix(initials-per-word)` rules 19 / 66 / 11 / 2, summing to 126; at 2.0.0 and at 2.1.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and `fix(#462)` 14, summing to 42; at 2.2.0 `fix(#462)` alone, 14. What the gate does NOT print is the per-name detail — a heading lists at most ten of its names and no diff fields at all, and the UNEXPLAINED and radar blocks are empty on a green tree — so it answers how many and under which rule, never which names or which surface moved. (b) The PER-NAME SPLIT, and the pre-fix 140 / 28, need the in-memory run described above: load compare.py by path, run main()'s comparison loop or hook `diffing`, and keep the names whose diff is exactly `{"_initials"}`. READ THE PATCH STEP AS HISTORY, and it belongs to (b) alone: since `a748862` the shipped worker template emits `_initials` itself, so patching `_worker_source` is what a measurement BELOW that commit needs and re-applying it on this tree changes nothing. The pre-fix 140 / 28 must be taken on the branch's base, before `31623d9`, rather than on this tree. Split "initials only" from "roles and initials"; within the former, lowercase and strip `[.\s]` to separate per-word grouping (equal after stripping) from content changes. The guard's liveness is proved by mutation, not by the green run: with `not diff and` removed from the guard, a roles-and-initials name has its role diff REPLACED by `{_initials}`, which no existing rule declares, and the 1.4.0 run reports 117 of them as no longer classified — 79 unexplained plus 38 radar, from 0 and 0. The same mutation is where the first bullet's churn figures come from: the rules that lose at least one name are the rules strict subset semantics would have had to widen, counted against the rules that explain anything at that baseline (53 of 72, 43 of 66, 36 of 59 at 1.4.0 / 2.0.0 / 2.1.0; re-derived 2026-09-02, the 1.4.0 numerator having been recorded as 43 until then). - -## differential-ledger, the rule-order arc (2026-09-02, #382) - -The sixth ledger arc, and the one that closes the question mechanisms.md#LEDGER-RULE-SEPARATION has carried open since #375's reorder mutation: file order settles every contest, and nothing said which contests existed. Mechanics — the TOML shape, the required `why`, the trap that a rule key written below the block joins the exemption — are owned by tools/differential/README.md; these are the decisions. Every figure below was measured 2026-09-02 against the real 1.4.0 wheel or recomputed from the checked-in files, and each carries its recipe. - -Decisions that landed: - -- 2026-09-02 #382 — an order-decided contest must be DECLARED, and narrow-first is the declaration-free default. A pair is a contest when the later rule's `fields` are a strict subset of the earlier one's, some corpus name's `name_regex` reaches both, and some comparison order reaches both; where the EARLIER rule is the wider one it carries a `precedes_narrower` block naming the later rule and saying why it is the classifier of record — usually what it describes that the later one does not, and in the three regex-accident pairs (`fix(#296)`/jr, and the two `fix(cjk-glued-honorific-peel)`/`fix(suffix-routing)` pairs) that the later rule reaches the name through its REGEX rather than by describing it — both `fix(suffix-routing)` patterns are anchored at both ends and open on a bare `\S+` run, which swallows the trailing comma of `Smith,` and matches hangul as readily as Latin, while the prose they carry is scoped to a comma-less two-token Latin name. Only ONE of the three is false outright — `fix(#296)`/jr, where 1.4.0 read `'Smith, Jr.'` as title `'Jr.'`, first `'Smith'`, so the family→suffix movement the jr rule describes never happens on it. The other two are PARTIAL, and symmetrically so: on `김민준씨 Jr.` and on `田中さん II` alike the trailing token really does leave `family` for `suffix` (1.4.0 first `'田中さん'`, last `'II'` → tree last `'田中'`, suffix `'さん, II'`, exactly the movement the numeral rule shows on its own `John V`), and what neither `fix(suffix-routing)` rule can describe is the peel and the segmentation that move `given`. Recomputed 2026-09-02 against the 1.4.0 wheel; the exemptions themselves claim only the scope mismatch, so do not read a falsity claim into the two partial ones. `compare.py` refuses the run, before the worker spawns, where such a pair goes undeclared. Reordering is NOT the alternative fix and the failure message says so: it moves which rule classifies a name and breaks `_CROSS_RULE_WINNERS`. What the check buys is that nothing else in the suite can see the hazard at all: `_CORPUS_CLAIMS` measures each rule alone, the gate total is per-corpus, and `_CROSS_RULE_WINNERS` pins contested outcomes only for names somebody hand-added. -- 2026-09-02 #382 — the check covers NESTED pairs, and nesting is sufficient for an order-decided contest rather than necessary. Two rules whose `fields` merely INTERSECT both admit any diff inside that intersection, so `classify()` hands such a name to whichever is written first exactly as it does for a nested pair, and `order_contests` cannot see it. That class sits outside the check by the reasoning `order_contests`' docstring already states for EQUAL `fields` — neither rule is narrower, so "narrow-first" says nothing about the pair, `precedes_narrower` has no narrower rule to name, and `_CROSS_RULE_WINNERS` stays the instrument — which covers every pair where neither `fields` set contains the other, equal `fields` being its special case. Not oversight, and not free either: measured 2026-09-02 over the four ledgers, counting pairs that share a corpus name and whose `orders` are not disjoint, 11 are strictly nested wide-first (what the check refuses), 40 nest in either direction, 11 have equal `fields`, and 111 have any non-empty intersection — so 60 overlap without nesting or equality. Widening to the general predicate would demand 111 written justifications where the real number is eleven, the same kind of answer this entry already gives below about the 657 figure: a predicate whose roster nobody can write out one justification at a time is not a usable predicate, whichever condition inflated it. The blind spot is worked and filed as [#498](https://github.com/derek73/python-nameparser/issues/498): `fix(#271/#272/#298)` (`{family, given, middle}`) and `fix(cjk-delimited-nickname)` (`{family, given, nickname}`) intersect in `{family, given}` without nesting, three contract-tier corpus names (`マイケル・ジャクソン`, `威廉・莎士比亚`, `高橋・一郎`) diff exactly those two roles against the 1.4.0 wheel, swapping the two rules reattributes all three, none is in `_CROSS_RULE_WINNERS`, and `order_contests` lists the pair in neither arrangement. RECOMPUTE the five figures with `_rule_reach` per ledger over the `corpus*.jsonl` union, classifying each pair by whether `a.fields & b.fields`, `b.fields < a.fields`, `a.fields < b.fields` or `a.fields == b.fields`; #498's body carries the loop. Closed 2026-09-05 by #498, which did not widen this predicate: the class is now covered at RUN time, where a measured diff makes the population 14 rather than the 71 pairs of the equal-or-overlap class (the 60 counted here plus the 11 with equal `fields`), and every name in it is pinned — the three named here among them, so read that "none is in `_CROSS_RULE_WINNERS`" as the state this measurement found (the #498 completeness check below). -- 2026-09-02 #382 — an ESCAPE HATCH here, where #452's and #456's were declined, and on the terms #452 set. Both of those bans were free to state: measured at the time, 0 of the 179 rules across the three ledgers then on disk had #456's shape, and #452's fourteen over-declarations (3 of 67 EXPLAINING rules at 1.4.0, 5 of 58 at 2.0.0, 6 of 51 at 2.1.0 — a different and smaller population than #456's 179, which counts every rule) were all narrowed before the check landed, so neither ban had to argue with a rule that was correct as written. #452's entry states the price of that strictness — "the first rule that genuinely needs a wider declaration has to argue for a key the way `dormant` was argued for in #373". Read that as the PROCEDURE it sets, not as a hatch this key opens: `precedes_narrower` is not a wider `fields` declaration and does nothing for an over-declared rule, which still exits the run non-zero. What carries over is the standard of proof, and here it is measured rather than asserted: eleven pairs in `expected_since_1.4.0.toml` are wide-first and every one of them is correct where it sits, so a ban would have had eleven rules to reorder or eleven prose descriptions to falsify. The hatch is narrowed the way `dormant` is: it names ONE rule (a blanket opt-out would be inherited by every narrower rule added later, which is the widening the check exists to refuse), the `why` is required, and a declaration standing over a pair that is no longer contested is refused as loudly as an undeclared contest. -- 2026-09-02 #382 (decided in review) — the vacancy half REFUSES only on a full run and prints a NOTE under `--corpus`. The two checks are not symmetric under a narrowed name set, and the asymmetry is the whole reason: narrowing removes contests, so for the undeclared check `--corpus` is only ever more lenient (it can only under-report, never false-alarm — do not call that fail-closed, which this repo uses for the rosters that REFUSE on a missing entry), while for the vacancy check it INVERTS — a live declaration whose contested names all sit outside the subset reads as vacant. Shipped as a regression and caught in review: as first written, a `--corpus` run against the 1.4 ledger exited 1 and told the contributor to delete exemptions the full gate needs — measured, each of the six corpora run ALONE reports vacancies, 11 of the 11 for `corpus.jsonl`, `corpus_cjk.jsonl` and `corpus_shapes.jsonl`, and 8 / 7 / 5 for the other three — after which deleting them would have made the full run refuse with that many undeclared contests. Read the shape and not the digits: the number varies with the subset, and only zero would have been safe. The file already treats `--corpus` differently in two places and this is the third, so the three should be read together rather than made uniform: the corpus-floor roster is SKIPPED entirely under the flag, `over_declared_rules` still FAILS the run and appends a NOTE saying the union it computed is over a subset, and the vacancy check does not fail at all. The strengths differ because the error directions do — only the vacancy check inverts under narrowing. -- 2026-09-02 #382 — TWO name populations, deliberately, rather than one shared function. `main()` must check the corpus it ACTUALLY compares, because `--corpus` narrows it; the unit guard in tests/v2/test_ledger_guards.py must check every corpus on disk, so that a rule added by a later bundle is checked at pytest speed with no baseline wheel. Forcing one function would break `--corpus`. They agree by construction instead: `_entry_name` in tests/v2/_differential_fixtures.py says in its docstring that it mirrors `compare.py`'s `_load_entries`, and both read the same `corpus*.jsonl` glob. -- 2026-09-02 #382 — the recorded negative control is deliberately BLIND to `precedes_narrower`. `_ORDER_EXEMPTION_EFFECT` records the eleven pairs and their name counts from `order_contests`, which never reads a declaration; a control that consulted the mechanism it controls for would measure nothing, and would go green the moment the predicate stopped finding anything at all. Its assertion that not every ledger's list is empty is there for exactly that failure (mechanisms.md#RECORDED-ROSTERS). - -Found rather than decided, and worth as much: - -- **#382 option 3's premise is FALSE, and the crux name is in the corpus.** The issue proposed a mechanical narrow-first check — refuse the wide-first pair, or sort by specificity. `fields`-subset is a proxy for specificity, and it is the wrong one where a wider rule describes a COMPOUND behavior its component rule does not. `马丁·路德·金씨` divides on the nakaguro AND peels its glued hangul honorific: `fix(#272/#308) nakaguro division and a glued hangul honorific in one name` describes what happens to it, `fix(cjk-glued-honorific-peel) glued honorific peels into suffix` describes half of it, and the WIDER rule wins by position, correctly. Narrow-first would have reattributed the name to the rule describing half — #372's defect reintroduced by the check meant to prevent it. The name is contract tier (`corpus_rules.jsonl`) and appears nowhere in `_CROSS_RULE_WINNERS`, so it is exactly the crux #382 was filed over and exactly the name no existing guard was watching. -- **The predicate needed a THIRD key, found in review.** The first implementation read `name_regex` and `fields`. `_entry_matches` narrows by `orders` as well: two rules declaring disjoint `orders` never see the same comparison, so file order decides nothing between them however nested their `fields` are. Omitting it made the detector read different boundaries from the predicate it models — docs/design/AGENTS.md axis 2 — and would have demanded a written justification for a hazard that cannot occur. It changes no figure today and is kept for correctness, not for its yield: measured over the four ledgers, adding the `orders` test removes 2 of 1350 nested pairs and 0 of the 657 wide-first ones. The nearest live shape is in both 2.x ledgers, where `fix(#399) a maiden marker bounds the particle chain that swallowed it` (`orders = ["DEFAULT"]`) and `fix(#399)/feat(#395) a consumed maiden marker leaves the family-first fold no given name` (`["FAMILY_FIRST", "FAMILY_FIRST_GIVEN_LAST"]`) have nested `fields` and share a corpus name, `de la Cruz née Vega` — narrow-first today, so nothing reports it either way; invert that nesting and the omission would have demanded a `why` for a comparison that never happens. -- **Nine of the eleven pairs are LATENT, not live**, which is the honest statement of what a static predicate costs and what it buys. Measured against the 1.4.0 wheel, a pair is a live order-decided contest only where some co-matched name's ACTUAL diff is a subset of the narrower rule's `fields`. In nine of eleven the real diff needs a role the narrower rule does not declare, so that rule is ineligible for those names wherever it sits. Only two pairs are live: `fix(comma-family)`/`fix(comma-precomma-family)`, where `John Smith, Mr.` is the one such name, and the compound/peel pair below, where 15 of the 17 co-matched names have such a diff and nine of those are this pair's to decide — the other six go to rules written above both, so the two counts answer different questions and neither is derivable from the other. So the predicate OVER-REPORTS relative to the measured diffs, and the price of that is eleven reasons somebody had to write. What the nine buy is the hazard that would ACTIVATE if a rule's `fields` ever widened — written down before the three later bundles add rules, which is when it is cheap. -- **A whole error class in the first draft of the exemption prose: concluding a RULE's tier from its NAMES'.** Three separate `why` texts did it. Tier is a property of the corpus file a name comes from, and a rule's reach usually spans both tiers. Measured: `fix(cjk-glued-honorific-peel)` explains 17 names, 14 of them CONTRACT tier — say the tier and not the file, even though all 14 happen to sit in `corpus_cjk.jsonl` today: the tier is what the gate acts on and `_CORPUS_TIERS` is where to read it, while a name can join or leave a file without changing either — and deleting the rule sends 12 contract-tier names to UNEXPLAINED (two of the fourteen re-home to `fix(cjk-honorific-suffix)`; the other three of the seventeen are radar) — so the gate does demand that rule, whatever the tier of the names it is contested over. Only `fix(cjk-comma-compound)` is radar-only in the strong sense: 11 explained names, all radar, and its whole 23-name regex reach radar too. Worth recording as a caution and not only as a corrected fact — the review rounds on this prose found six false claims, then one, then two more of this class. Every one was in prose nothing can machine-check, and MOST were caught by re-running the wheel rather than by reading — but not all, which is the part worth keeping: of the first round's six, two came only from reading, one citing "a few rules below" for a quote 52 rules away and one opening "the one live pair of the eleven" while a second exemption in the same file declared itself live too. Distance-in-the-file and prose contradicting prose are the classes no wheel run can reach. -- **One exemption says "least wrong reading available", and that is a finding about the ledger's vocabulary.** `fix(cjk-comma-compound)` outranks `fix(cjk-glued-honorific-peel)` on nine names — 17 reach both regexes, 15 have a diff the peel rule's `{family, given, suffix}` admits, and 6 of those 15 are taken by rules written above both — and its label is untrue of three of the nine. `王先生, V.`, `田中さん, V.` and `김민준씨, V.` show no comma routing and no order flip; their whole diff is `{family, suffix}`, the glued peel alone, against `{family, given, suffix}` on the six genuine compounds. They land on the compound rule because its criterion is "the diff includes `family`", and `family` moves only because the peel took the honorific off it. The order must still stay: the peel rule's prose disclaims comma names, so promoting it would mislabel the six. The rule that would describe them is a family-side twin of `fix(cjk-comma-honorific-peel)`, which covers this shape for a POST-comma given name; writing one was weighed as [#496](https://github.com/derek73/python-nameparser/issues/496) and DECLINED (2026-09-02), and the Declined entry below carries the evidence, the #495 tension and the reopen trigger. Declining it discards no part of this finding: the compound rule's label is still wider than those three names are, which is what this bullet and the exemption's own `why` exist to say. -- **A guard can pin the winner of a contest and still let the recorded diff shape be wrong.** `test_the_recorded_rule_still_wins_each_contested_name` feeds `classify()` the RECORDED shape and never checks that shape against a real comparison, so a wrong shape that still routes to the same rule passes forever. This branch fixed one: `田中さん II` was recorded as diffing `{given, suffix}` and measures `{family, given, suffix}` against the 1.4.0 wheel, and the claim had been copied to four sites. The winner did not move, so every argument resting on it survived — which is why nothing noticed. Filed with the general shape as [#497](https://github.com/derek73/python-nameparser/issues/497). - -The measurement, and how to redo it. Eleven wide-first contests in `expected_since_1.4.0.toml`; 0 in each of the three 2.x ledgers. Six of the eleven are contested over at least one contract-tier name, five only over radar names — the division [#495](https://github.com/derek73/python-nameparser/issues/495) argues from, and it survives a name changing tier even though the two counts do not. The load-bearing contrast is between that eleven and what `fields`-subset alone says: dropped to nesting alone, with no corpus-reach and no `orders` condition, the 1.4 ledger holds 646 nested pairs of which 367 are wide-first, against the eleven contests it actually has — and across all four ledgers, 1350 nested pairs of which 657 are wide-first. Read the ratio and not the digits: the two answers differ by more than an order of magnitude and would still differ by one after any plausible drift, which is the argument — `fields`-subset alone is not a usable predicate, and it is the corpus-reach condition that makes the check something a person can answer eleven times. RECOMPUTE: load `tools/differential/compare.py` by path and call `order_contests(rules, names)` per ledger, with `names` the union of `_load_entries` over the `corpus*.jsonl` glob — 1116 distinct names today, 326 of them reached by a corpus `_CORPUS_TIERS` marks contract. READ THAT ROSTER, do not reconstruct it from a file's name or size: `corpus.jsonl` is the LARGEST corpus at 486 distinct names and it is RADAR, having been demoted by #468 as a v1-test-bank scrape, so the contract tier is the three SMALL files (`corpus_cjk.jsonl` 73, `corpus_rules.jsonl` 248, `corpus_shapes.jsonl` 35) and the split runs 326 / 790 rather than the other way about. Reading `corpus.jsonl` as contract gives exactly 786 / 330, which is the wrong answer this figure attracts — it was measured that way twice before a third recompute caught it, and the gate's own `corpora:` line is the cheap check, since it prints the contract files first; for the contrast, re-run the same combinations keeping only the strict-`fields`-subset test. `undeclared_contests` and `vacant_exemptions` both return empty over every ledger, which is what the unit guard asserts. Note WHICH population each caller reads: `main()` checks the entries the run loaded — 1120 at every baseline, of which 1113 compare at 1.4.0 once the shape-minimum skip runs, since that skip happens after the check — while the unit guard reads every corpus on disk. The arc moved no classification: the gate reports 352 / 247 / 155 / 14 intentional diffs at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 0 unexplained and 0 radar-unclassified at all four, unchanged across the whole branch. - -Declined: - -- Reordering the eleven (2026-09-02) — the obvious way to make the check unnecessary, and wrong twice. Reordering moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; and no pair is even a reorder CANDIDATE, because in all eleven the narrower rule's own prose would be FALSE or MERELY PARTIAL of the co-matched names if it won, and either disqualifies it as classifier-of-record. Partiality is the commoner half and the arc's load-bearing distinction — compound versus component, which the `fix(#400/#274)`/`fix(#400)` exemption calls "the canonical compound-versus-component shape". Read the eleven `why` texts for it: `fix(#400)` "says nothing about a maiden marker"; a widened `fix(comma-family) lone post-comma piece routes to suffix/title` "would take the union and report only half of it"; `fix(cjk-glued-honorific-peel)` says "nothing about a division"; `feat(#273)`'s prose "has nothing to say about" the remainder rejoining as a name; and two of the three regex-accident pairs are partial rather than false, the `fix(suffix-routing)` jr and numeral rules genuinely naming the `{family, suffix}` half of `김민준씨 Jr.` and of `田中さん II` and describing neither the peel nor the segmentation that moves `given`. Falsity is the minority, and one of the eleven says so of itself: the exemption `fix(comma-family) a comma followed only by titles keeps the given/family split` carries over `fix(comma-precomma-family)` calls its own pair "the one where the narrower rule would be actively WRONG rather than merely partial", and adds that the claim "is about THIS name and is deliberately not generalised". `马丁·路德·金씨` is the clearest compound, and `王先生, V.` the reverse case, where the winner is already the least wrong reading available — and stays so, since the family-side twin that would describe it was weighed as #496 and declined (the next bullet). -- A family-side twin of `fix(cjk-comma-honorific-peel)` for a PRE-comma glued honorific (2026-09-02, [#496](https://github.com/derek73/python-nameparser/issues/496), closed as not planned) — the rule that would honestly describe `王先生, V.`, `田中さん, V.` and `김민준씨, V.`, whose diff is the glued peel alone while the label they carry says comma compound (the finding bullet above). Declined on two grounds. FIRST, no gate can demand it: all 17 names the compound/peel pair is contested across measure radar — but by two different warrants, and an audit that assumes one will find the count short. Sixteen sit in `corpus_cjk_tolerated.jsonl`, the file #488 created by demoting the composed comma/Latin-wrapper CJK forms to tolerated input, `김, 민준씨` — the name #382 was filed over — among them. The seventeenth, `Dr 田中さん, V.`, is in `corpus_issues.jsonl` alone: harvested and append-only, radar since #468, and untouched by #488. (`Dr 김민준씨, Jr.` is in both files, and reads radar from either.) Radar means an unmatched diff is reported rather than fatal for want of a RULE; it is not an unconditional never-fatal, since a `[[never]]` exclusion outranks the tier and stays fatal on a radar name — `_CORPUS_TIERS`' own note, and `main()`'s two-reasons comment, which routes an excluded name to `unexplained` rather than to `radar`. No exclusion refuses these, so nothing here can demand the twin. SECOND, [#495](https://github.com/derek73/python-nameparser/issues/495) points the other way over the same corner: it asks whether the radar-only rules already in the ledger still earn their place and records `fix(cjk-comma-compound)` as having zero contract-tier reach, so #496 proposed a SECOND rule for the very names #495 is weighing a first one away from. Fewer rules for this corner is the coherent direction, and that is what makes the decline decisive rather than merely permissive. REOPEN it if a pre-comma glued-honorific name ever reads contract — and watch BOTH routes, because the two files promote by different mechanisms. A `corpus_cjk_tolerated.jsonl` name is promoted by clearing `tolerated` on its case rows, which moves the text into `corpus_cjk.jsonl` (`_CORPUS_TIERS`, and build_cjk_corpus.py's split). `Dr 田中さん, V.` has no case row at all, so there is no flag to clear: it changes tier only by being CHOSEN — a new unmarked row, or a rules.md example — which puts the text in a contract corpus, and the (name, order) dedup loads contract files first and keeps that reading. Read the tier off `_CORPUS_TIERS` either way, never off a hand-built file→tier map (the roster caution above). -- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, was still under a second — that last one is NETWORK- and environment-dependent and the RECOMPUTE below does not reach it, covering the warm run alone. Reproducing it means pointing `UV_CACHE_DIR` at an empty directory so the wheel is fetched rather than reused, and the number will differ on a different link. Read it as "the download is not the story either", not as a figure to check against. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. This decline is unchanged and is not what #498 did: the #498 completeness check ADDS a run-time predicate over a class the static one covers not at all, rather than replacing the static one over the class it does cover — so the coverage argument above is the reason both exist (the #498 completeness check below). -- Giving the peel rule a predicate the compound rule fails (2026-09-02, #382 option 1) — narrowing `fix(cjk-comma-honorific-peel)`'s `name_regex` to the honorific-bearing shapes, whose stated effect in #382 was that the pair "becomes order-independent and the original contract holds again". Declined because that effect is UNREACHABLE, and measured rather than argued. The two rules ship a BYTE-IDENTICAL `name_regex` (verified 2026-09-02 by comparing the two strings), and `_rule_reach` computes each rule's names from its OWN pattern, so narrowing the peel rule's regex narrows the peel rule's reach and nothing else: the compound rule goes on reaching every one of the names. The general form, which no regex edit escapes — where the earlier rule's `fields` are a SUBSET of the later one's, order-independence requires the LATER rule to stop reaching the name, and an edit to the earlier rule's own regex cannot cause that. Here the peel rule's `{given, suffix}` is a strict subset of the compound rule's `{family, given, suffix, title}`, so every diff the peel rule admits the compound rule admits too. Implemented to check it: narrowing the peel regex to a `GLUED_HONORIFICS` alternation keeps all seven of the co-matched names whose 1.4.0 diff is exactly `{given, suffix}`, and swapping the two rules in the NARROWED ledger still reattributes all seven. So the pair keeps the arrangement the arc settled on, and that is the answer rather than a cost trade: it is narrow-first, the declaration-free default, so it appears in none of the eleven contests `order_contests` reports and owes no `precedes_narrower` block, and all seven names are pinned by name in `_CROSS_RULE_WINNERS` (23 corpus names reach both regexes; the seven are the ones #375's reorder mutation moves). The cost stands as a second reason and not the first: the narrowing would hand-copy more honorific vocabulary into the pattern and grow the `_HONORIFIC_SOURCES` sync-roster surface (mechanisms.md#CURATED-VOCABULARY-ALTERNATION's second half). Option 2's convention plus a check on the wide-first exceptions to it is what the arc took, and option 3 it REFUSED on a false premise (the finding above). One thing this does not settle, and the ledger comment on the peel rule overstates it: `fields` separates the two only for the UNION rows whose diff includes `family`; for those seven it is file order that decides. -- Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, MOST of the names it pins are radar-tier — so the repo would be inconsistent with itself about whether a radar contest matters. The majority is the whole of the argument and no digit belongs here: a count written at this spot was stale before the branch that wrote it merged, since the 2026-09-03 #497 entry below then deleted four of the roster's rows. RECOMPUTE: union the names in `_CROSS_RULE_WINNERS` (tests/v2/test_ledger_guards.py) and read each one's tier off `compare._CORPUS_TIERS`, via the first `corpus*.jsonl` file holding the name with contract files sorted first, as `main()` loads them. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. -- A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. -- Deleting the rules whose explained names are all radar (2026-09-02, [#495](https://github.com/derek73/python-nameparser/issues/495), closed as not planned) — filed off #488's demotion on the reasoning that a rule nothing can demand is a rule to remove, and refused on what deleting one actually does. Measured at baseline 1.4.0, 19 of the 72 explaining rules explain only radar names, between them 48 of the 352 intentional diffs — and that set is a SUPERSET of what the issue proposed deleting, not a different set. Three of the five rules #495 tabulated are in it — `fix(#296) a credential-only comma string reads a name and its postnominal`, `fix(cjk-comma-compound)`, and the roman-numeral `fix(suffix-routing)` — and they are exactly the three its option 2 named as having zero contract reach. The other two are outside it for the reason that disqualified them as candidates there too: they explain contract names, fourteen of them for `fix(cjk-glued-honorific-peel)` and `'Smith Jr.'` (corpus_rules.jsonl) for the jr `fix(suffix-routing)`. So the criterion reaches everything the issue wanted gone and more, and what refuses it is the fate of those 48 diffs. Measured by driving `classify()` over each one with its own rule removed: 25 report UNCLASSIFIED and 23 are claimed by ANOTHER rule (26 and 22 with all 19 deleted at once). For nearly half the population, deleting does not leave the diff watched-but-unexplained — it leaves it explained, by a rule whose comment is about something else. Nine of `fix(cjk-comma-compound)`'s eleven land on `fix(cjk-glued-honorific-peel)`; all seven of `fix(cjk-comma-honorific-peel)`'s land on `fix(cjk-comma-compound)`; `'Carod i'` lands on the roman-numeral `fix(suffix-routing)`, and `'Bob Jones, author'` on `fix(comma-precomma-family)`. That is the rule-broader-than-the-diff absorption this file already names twice — `fix(#424)` at decisions/H1, going on classifying a diff that shrank underneath it while the fields-only catch-all's heading stepped 14 -> 17 with nothing announcing either, and the fields-only arc's "a fields-only rule silently falsifies OTHER comments" — reached this time on purpose rather than by accident, which is not an improvement. Applied, it deletes `fix(#342)` and `fix(#397)`, the two NOT WANTED rules whose whole job is classifying a reading nobody wants so the gate stays green while the bug is open, and which carry the delete-when-fixed instruction the dormancy check enforces once the fix lands (the fields-only arc above). The cost there is NOT a red gate, and a first draft of this bullet said it was: both names sit in corpus_issues.jsonl, which is radar, and on a radar corpus an unmatched diff prints UNCLASSIFIED (radar) while the run keeps exiting 0 — UNEXPLAINED is the contract tier's word (tools/differential/README.md). Measured with both rules gone, the run reports `unexplained: 0; radar unclassified: 1` and exits 0. The one is `'Aishwarya Rai'`; `'Carod i'` is not reported at all, the numeral rule having taken it — so one of the two open bugs loses its marker not to silence but to a wrong label. Verified end to end against a ledger with `fix(#397)` removed: exit 0, 352 intentional diffs still, no UNCLASSIFIED, no EXPLAINED NOTHING, and `'Carod i'` printed under `fix(suffix-routing) a two-token name ending in a roman numeral`. It also deletes `feat(#273)`, `feat(#269) Arabic بن prefix chains onto family (non-Latin new-recognition)` and `fix(#360) ste moved into the never-given particles with mc` — records of new recognition behavior and of a vocabulary move — and the last two have to be named to the RULE rather than to the issue, as the ledger holds sibling rules under both strings. The `feat(#269)` sibling is the sharp one: `feat(#269) non-Latin titles/conjunctions recognized` does not qualify, explaining the contract-tier `'Хосе И Мария Сантос'` (corpus_rules.jsonl), so a delete by issue prefix takes both and sends that name UNEXPLAINED — measured, and a red gate. The criterion that does justify deleting a rule is that it explains NOTHING, which `dormant` already names and the dormancy check already enforces; a rule explaining eleven radar names is not dormant. What the radar tier is FOR answers the rest: decisions/cjk-comma-demotion puts it as giving the differential "somewhere to WATCH a name without promising it", and a rule explaining a radar name IS the watching. The five exemptions citing #495 are rewritten in the same diff rather than left standing: four of them put it as a LIVE question — the two under `fix(#296) a credential-only comma string`, the one on the `fix(cjk-comma-compound)` block, and the one on the `'田中さん II'` numeral contest — and the fifth, on the jr contest, called neither rule a candidate in the present tense. Each records the answer now and keeps every measurement it carried. One of those measurements was wrong and is corrected with them: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` explains eight names of which FIVE are contract, not six — the six having shipped in #499. It is not a mis-map: six is what tiering by FILE MEMBERSHIP gives from a CORRECT `_CORPUS_TIERS`, and the single name it gets wrong is `'John Smith, Dr.'`, the third RECOMPUTE trap below. #496's decline was argued inside the `fix(cjk-comma-compound)` one and stands without it: the twin rule was refused because nothing can demand it, which is untouched by what #495 answered. RECOMPUTE: do not join names to files by hand. `_CORPUS_TIERS` maps FILES, the report prints names with no file, and 24 of the 352 explained names sit in files of BOTH tiers, so a hand join has to reproduce `main()`'s contract-first dedup AND its baseline-minimum skip to get them right. Let the run do it: call `compare.main()` unchanged under two wrappers — one around `_run_worker`, whose `entries` argument is the final list with `tier` and `file` already stamped on every entry, and one around `classify()`, recording (name, order, diff, issue) for each of its 352 calls — then reclassify offline from those records against `rules` minus whichever rule is under test, having first confirmed the wrapped run reproduced the shipped one (352 intentional, 0 unexplained, 0 radar unclassified). Parse no printed repr by stripping quotes: three of the 352 names do not survive it — `"'Smitty' Dr. Jones"`, `"'Smitty' Jones Jr."`, and the spelling of محمد بن سلمان wrapped in RIGHT-TO-LEFT MARKs — and dropping them silently is what produced the wrong count of 20 on the first pass, the first of the three being `fix(#410)`'s only contract name. `ast.literal_eval` reads the printout correctly. The ten-name truncation is NOT the hazard it looks: contract files load first, so any contract name a rule explains sits in the printed prefix, and tiering from the first ten alone gives the same 19. THIRD, and the only one of the three that changes an answer on this tree: a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in. The two part where a string sits in files of both tiers AND its contract copy carries a shape tag, because the dedup key is `(name, order)` — a shape declaring an order leaves TWO entries rather than one contract winner, and the baseline-minimum skip can then drop the contract one. Measured, 2 of those 24 names carry a shape tag and exactly 1 bites at 1.4.0: `'John Smith, Dr.'` is in corpus_issues.jsonl (radar, no shape) and corpus_shapes.jsonl (contract, shape 4, FAMILY_FIRST, min_baseline 2.0.0); both entries survive the dedup, which the 2.0.0 run shows by comparing both, and at 1.4.0 the shape-4 one is among the seven skipped, leaving the radar entry as the only comparison. `'Beethoven, Ludwig van'`'s shape 2 is order-None with a 1.4.0 minimum, so the dedup collapses it and contract wins at every baseline. File-membership tiering gets exactly 1 of the 352 wrong, and that one is what `fix(comma-family)`'s six-versus-five turned on; `entry["tier"]` gets it right for free. The radar-only count of 19 is the same under either tiering. To exercise a ledger edit, point `_allowlist_for` at a scratch file rather than copying the tree — the corpus glob resolves from `HERE` and `_check_tree` refuses a nameparser outside `REPO_ROOT`, so a copy elsewhere cannot run at all. - -## differential-ledger, the recorded-shapes arc (2026-09-03, #497) - -The seventh ledger arc, and the one that makes a recorded INPUT measurable. The arc above closed on the finding that `test_the_recorded_rule_still_wins_each_contested_name` feeds `classify()` a diff shape nothing ever checks, so a shape that is wrong but still routes to the same rule passes forever; this is what was done about it, and about the second thing that search turned up — that the cost the same arc declined a check on had never been measured at all. Mechanics — where the shapes live, which name list each half of the check reads, what a `MOVED SHAPE` block may and may not claim — are owned by `tools/differential/compare.py` (`_RECORDED_DIFFS`' provenance note, `recorded_diff_mismatches`' docstring, and the two placement comments in `main()`), by `tests/v2/test_ledger_guards.py` (`_CROSS_RULE_WINNERS`' header and the note over its 2.x sections) and by tools/differential/README.md; these are the decisions. Every figure below was measured or recomputed 2026-09-03 on this branch; each one that can drift carries its recipe, and the two that cannot be re-derived from the tree at all — the sweep scan's own hit counts — say so where they appear. - -Decisions that landed: - -- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is false: measured 2026-09-03 on this branch, of 171 `def test_*` functions in that module 20 mention `_run_worker` at all and six reach the real one with `Popen` faked beneath them by `_fake_popen`. RECOMPUTE by walking the module's `def test_*` bodies for the name. That short form is what compare.py's `_RECORDED_DIFFS` provenance note SAID when this entry was written; `f70a0ad`, the very next commit, withdrew it there, and the note now rejects the paraphrase rather than carrying it — so follow that pointer to the history and not to the text. The denominator has moved once already inside this arc: `f70a0ad`'s own message says 168, which is why the argument here is "most never mention it" and the digits carry a date. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. -- 2026-09-03 #497 — ONE roster, TWO checks, TWO PLACEMENTS — and, since `9360919`, a NOTE over the gap between them, recorded at the end of this entry. (Superseded in its first word 2026-09-05: TWO rosters since the watched-shapes arc below, `_WATCHED_DIFFS` beside `_RECORDED_DIFFS`, both placements unchanged; the departed-name refusal and the `NOT CHECKED` note read the union of the two, while the shape comparison runs once per roster, because the two halves carry different severities — a contest row's mismatch feeds the exit code on either tier, a watched row's only where its default-order entry is contract tier — and one call could not tell them apart.) The asymmetry of the two checks is the decision rather than where the code happened to fit. The ABSENT-NAME half — a recorded row naming a string no corpus holds any more, which nothing else would ever report — refuses PRE-worker, beside `vacant`, because it reads the ledger and the loaded names and nothing the worker produces. It takes the PLACEMENT #382 chose for its own pre-worker refusal and NOT #382's reason for it, which was the cost claim below — `compare.py`'s comment at that placement is one of the sites the timing finding retires. The ground it stands on was rewritten on this branch: a refusal raised after the wheel is installed and the corpus compared prints below the run's own published `baseline:` header, for a comparison it will never report. Not "disowning a comparison it has just published" — measured print order is `baseline:` at the tell, then the comparison loop, then `corpus: … intentional diffs:`, so at that position the header has printed and no line of the comparison has. The ORDER is the argument, and that corrected wording is what `compare.py` now carries at all three of its placement comments. Recorded because it is the second decision the false number was under, and the two came apart differently — the declined check below kept its conclusion on a new argument, and here the ORDER was right all along while the reason for it was not. Only re-deriving a claim says which of its supports was doing the work; a placement that reads unchanged is not thereby unaffected. The MOVED-SHAPE half is the opposite case — it needs the diffs, so it stays after the comparison — and there it may not RAISE. It PRINTS and feeds the exit code, like `over_declared_rules`, its structural sibling and the other post-worker check on recorded data: a raise at that point lands MID-report and takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it, which compare.py measures on a narrowed 1.4.0 run beside the check. A stale roster row must never hide an unexplained diff, that being the gate's primary output. The two pre-worker refusals may raise because nothing has printed yet; this one cannot, and the difference is PRINT ORDER rather than severity. The two halves also read opposite name lists, which is `recorded_diff_mismatches`' docstring's to state and is not restated here. What IS decided here is the THIRD thing this roster produces, added by `9360919` once the gap between those two lists was measured and found not empty: an order-bearing entry the baseline cannot honor sits in a corpus file and outside the compared set at once, so a row on such a name is checked by neither half and reported by neither. `main()` prints a `NOT CHECKED` note over it. A NOTE and not a refusal — the docstring's argument against refusing off the post-skip list stands, and it is an argument against REFUSING and not against SAYING, which are two decisions of which only the first was ever made. Informational, outside the exit code, and gated on neither `full_corpus` nor `args.corpus`, the intersection narrowing itself. Measured 2026-09-03 at 1.4.0 — both the only baseline where the skip fires and the only ledger with rows — the window is three names, none of which carries a roster row today, so the note prints on no run yet. The measurement that motivated it put a deliberately wrong shape on `'de la Cruz née Vega'` over the full corpus at 1.4.0: before `9360919` that exited 0 in 375 stdout lines naming the name in none of them, and it now exits 0 in 378 with the `NOT CHECKED` note naming it — re-measured 2026-09-03 by corrupting the row in memory around `main()`, which leaves the worktree alone. -- 2026-09-03 #497 — a SHAPE may be pinned where a COUNT may not, and that line is the whole of what keeps this roster affordable. A diff shape is one name under two parser versions, so adding a name to a corpus cannot move it; an explained-name count moves on every corpus edit. Shapes drift only when the PARSER changes those names — so the drift and the reason to re-read the roster are the same event, which is exactly what a recorded number usually fails to be. That is why #452's churn objection to `explains = N` (the fields-only arc above, 2026-08-28) does not reach this roster, and why pinning shapes here does not weaken it: the four ledgers carry 201 rules today, one `_CORPUS_CLAIMS` entry each, every one liable to move whenever a corpus does, against 31 recorded shapes that move only with the parser (51 since #498, 45 of them at 1.4.0). Read the two CAUSES and not the two totals — the argument is that one set is coupled to corpus edits and the other is not, and it survives either number moving. RECOMPUTE: `sum(len(v) for v in _CORPUS_CLAIMS.values())` in tests/v2/test_ledger_guards.py, and `sum(len(v) for v in _RECORDED_DIFFS.values())` in compare.py. -- 2026-09-03 #497 — four of the 35 rows the roster carried were DELETED rather than corrected. Two names across the two 2.x ledgers, so two rows each, and one of the two NAMES had a correctable shape — count rows or names deliberately, since the provenance note that holds the measurements speaks of the name. Deleted because neither name pinned a CONTEST, which is what the roster is for: at the shape each really produces exactly one rule admits it, so `classify()` was never adjudicating anything. `'Nguyen, Van'` diffs at no baseline at all, so no run ever asks `classify()` about it; `'Jane née and Jones Smith'` is a malformed harvest from a radar corpus whose right parse nobody can state, so a pin on it defends no boundary anyone would argue for — which is why the correctable one went too. compare.py's provenance note holds the per-name, per-baseline measurements behind both. The deletion emptied both 2.x sections, which stay present as explicit empty mappings — the completeness guards require it, and it is what makes emptiness sayable as a POSITION: a row is owed when someone argues a boundary, not before. What the emptiness is NOT — that these ledgers hold no contested diff — is measured and stated beside those sections, and whether the position should change now that those contests are measured rather than merely unexamined is [#501](https://github.com/derek73/python-nameparser/issues/501), open. (Superseded in its emptiness 2026-09-05: the two 2.x sections are no longer empty, the #501 adjudication below having pinned FIVE rows at 2.0.0 and one at 2.1.0, so `_CROSS_RULE_WINNERS` read 31 / 5 / 1 / 0 at that point — measured 2026-09-05 — and #501 is closed. It reads 45 / 5 / 1 / 0 since #498's fourteen pins, landed the same day; only the 1.4.0 figure moved. RECOMPUTE for either: `{k: len(v) for k, v in _CROSS_RULE_WINNERS.items()}` in tests/v2/test_ledger_guards.py. The criterion is not superseded and is what filled them: a row is owed when someone argues a boundary, and three of the six were found already argued, in the 2.0.0 ledger's own comment. Read the sentences above as the position emptiness stood on for as long as it stood.) Recorded with the deletion because it is the half a later sweep will get wrong: "only one rule admits it" is NOT on its own grounds to delete a row. A substantial minority of the surviving 1.4.0 rows are in that position and stay — the count and its recompute are beside them in test_ledger_guards.py, where six of the thirteen say so in their own comments and the other seven do not, which is why the count is recomputed rather than read off the roster — because the shapes they pin are shapes runs actually make, so a widened `fields` or a moved rule hands the name over and the test says so. Counting admitters is not the test at all: 11 of those 13 route to a DIFFERENT rule under some other shape, and mechanisms.md#RECORDED-ROSTERS now carries that measurement in place of the criterion it stated first. The four deleted rows could not do that work at any edit, which is the difference. -- 2026-09-03 #497 — the prose sweep is sorted by this file's own criterion (#451, "past-tense counts stay in a ledger comment; live ones do not") into three treatments, and the treatment follows from what the count is DOING rather than from how wrong it is. A count that IS the finding and reads past tense stays. A count standing beside the enumeration it counts loses the digit — the list is self-checking and the digit is the only part that can rot. A live exhaustiveness claim with nothing recomputing it is REGROUNDED on the property it was reaching for, so that the sentence stops depending on a number — saying a widening claims all but a handful of the whole population the assertion is over says what a fraction said and cannot go stale. One judgement goes with it and is the arc's own: a figure that could NOT be reproduced was not replaced with a fresh one. `_reaches_non_vocabulary`'s docstring now carries the shape of its finding, the instruction for measuring it, and no LIVE figure for the magnitude the old figures asserted — a replacement digit nobody can re-derive is the defect rather than the repair, and this file has already been burned by that shape (the #414 roster comment under the fields-only arc's `explains = N` bullet). The past-tense digits explaining why the live ones are gone stay, which is the first treatment applied inside the third, and a later sweep should not read them as survivors of the pass. - -Found rather than decided, and worth as much: - -- **The cost that decided a decline had never been measured, and the decline survived losing it.** "The multi-minute worker pass" entered 2026-08-05 in `7767ba2`, at ONE site, with no measurement beside it. The census of where it reached, the timing figures that replace it, their recompute and the `_run_worker` timing trap all live ONCE, in the rule-order arc's `Declined:` bullet on precise per-name contest detection above — the argument they falsify — and none of them is restated here; that census is the figure in this arc most likely to move, so pointing at its owner rather than copying it is the #451 rule applied to the very claim that demonstrates why. Two notes on reading it that belong here rather than there. Not every site spells the phrase — one says the pass "costs minutes" — so count the CLAIM and not the wording (`git grep -n -i minute 514a2a1`, this branch's base). And measured, the worker pass is a fraction of a second and a whole gate run is seconds at most, where the phrase said minutes: read that class and not the digits. Three things transfer past them, and a coda about removing it. The claim was never a measurement anyone got wrong — it was a plausible sentence about a subprocess that installs a wheel, and it propagated by COPYING, the mechanism the fields-only arc recorded for ledger comments and which reaches code comments identically. It was load-bearing and its conclusion still stands, rewritten on COVERAGE: a run-time check sees one ledger, for the baseline that run was invoked with, and only when somebody invokes the tool, where a static predicate covers a rule from the moment it is written. And that is how a false number stays invisible for a month — an argument with two supports loses one and nothing downstream of it changes, so no reader ever meets a contradiction. Removing it was itself hard to do cleanly: review of the commit that removed the phrase found four instances of the very defect that commit exists to remove, fixed in `40c82a7`. -- **A row pinning a race with one runner is never exercised as a contest, so its shape only ever had to agree with itself.** That is why nothing caught the four, and it is the general shape rather than a fact about two names. The four arrived with #452's work (`ee6aaa6`, PR #455) under two claims of measurement — the roster header's "The diff shapes are measured against the 1.4.0 wheel, not guessed" and the 2.1.0 block's own "measured against this baseline's own run rather than copied from the 2.0.0 rows" — and every one of the 31 rows at 1.4.0 agreed with the run on 2026-09-03, at all four baselines; the 45 rows there since #498 all agree on 2026-09-05, the gate exiting 0 with no `MOVED SHAPE` at any baseline. Do NOT read that as the 1.4.0 half having been sound: one of those 31 was guessed too, `田中さん II`, wrong on a genuinely CONTESTED row, found by hand a day before this branch opened (`e5e4955`, the finding the arc above filed as #497) and already corrected when the 31 were re-measured. So the one-runner mechanism explains why THESE four survived a check that would have caught a contested row, not why every wrong shape survived — the contested one needed somebody to sit down with the wheel, which is the cost the run now removes. The transferable test is what to ask of a recorded INPUT: not "is it checked" but "would its CONSUMER answer differently if it were wrong". mechanisms.md#RECORDED-ROSTERS forbids re-deriving the EXPECTATION from the inputs the check reads; this roster obeyed that to the letter and still went wrong, on the other side of the assertion, which is why that entry now carries the limit. -- **The scan that scoped the sweep was wrong three ways, and it is the scan rather than the three fixed lines that the next sweep will reuse.** Two of its 19 machine-readable hits were false positives — a version number, and a hypothesis rather than a measurement — so the real population was 17. Its file list omitted `expected_since_2.2.0.toml`, which held a further copy of a claim it was already treating in two other ledgers, so the sweep would have shipped two of three siblings corrected. And its predicate matched only lines BEGINNING with `#`, which made every docstring invisible to it: the stalest figures in the whole sweep survived the first pass for that reason alone, and a docstring pass over the two guard modules then found two more. Read the sweep as BOUNDED and not exhaustive — that docstring pass covered `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`, and compare.py's own docstrings were never swept. And a fourth thing, which is the finding about the finding: the scan's PREDICATE was not preserved. Its 19 hits and their 17 survivors are recorded in `8593f3a`'s commit message and cannot be re-derived from the tree, because nothing says what it matched beyond the leading `#`. That is the reusable artifact of a sweep — not the lines it fixed — so the next one has to write its own and has no earlier hit count to check itself against. Write the predicate down where the results go. -- **What the stale claims were, because the classes recur and the digits do not.** (i) A count of a population that had grown by half underneath it: a guard comment gave a Latin-half widening as claiming "644 of the 654 unclassified corpus names", where the same expression claims 994 of 1008 today (`len(_UNCLASSIFIED_NAMES)`, intersected with `_claimed("[A-Za-z]")`, in tests/v2/test_ledger_guards.py) — and since the argument was that the widening reaches essentially the whole population the assertion is over, it is written that way now and needs no recompute. (ii) A count of FILES: "all five corpora", in three places, where six `corpus*.jsonl` sit on disk and a run globs every one of them — the corpus count was never the argument, and the gate's own `corpora:` line is the check. (iii) A live count inside the very paragraph that names stale counts as the file's standing hazard — `fix(#274)`'s ledger comment walked "all 31 the regex claims" where that regex claims 32 today (`len(_claimed(rule["name_regex"]))`). (iv) A count over a self-checking enumeration: "Six such widenings" above a list of four, where nobody can now recover which two are missing and the list was always the evidence. The fifth is the one worth the space, and the docstring pass found it: `_carries` said the `isascii()` split covers "12 of the 16 entries", where at sweep time `MAIDEN_MARKERS` shipped 17 with 12 of them non-ASCII (measured 2026-09-03; RECOMPUTE with `len(MAIDEN_MARKERS)` and `sum(not m.isascii() for m in MAIDEN_MARKERS)` from `nameparser.config.maiden_markers`) — the NUMERATOR was right and stayed right while the denominator rotted. The repair names the predicate rather than counting its output, which is why the guard's own note past-tenses both figures and points the reader at the `isascii()` split itself. A count can go stale in one of its two halves, which is the shape a re-reader is least likely to catch, and it is a reason to prefer naming the predicate over counting its output. -- **The tier split inverts the ranking that #452's still-open half rests on.** Beside the fields-equals-union check that landed, #452 carries a second proposal: a ceiling on how much of the corpus one rule may REACH, measured there as a share of the whole corpus. #468's tiering makes that the wrong denominator, because a radar diff cannot fail the run — a rule reaching many radar names is not the hazard a rule reaching many contract names is. Measured over the corpus population and tier split the rule-order arc's RECOMPUTE above already owns and this bullet does not restate, the two comma rules reach 288 names, 25.8% of all of them, but 64 contract names, 19.6% of that tier; `fix(#271/#272/#298)` reaches 108, 9.7%, and 80 contract, 24.5%. So by total reach the comma rules are the outlier and by contract reach the CJK rule is, and a ceiling on total reach would rank the wrong rule as the biggest hazard. Read the INVERSION rather than the four percentages: it is a claim about which tier the denominator counts, and it survives any drift that does not reverse the two tiers' composition. Recorded here because the proposal is open and the measurements in it predate the tiering. RECOMPUTE: tier every corpus name off `compare._CORPUS_TIERS`, contract winning where a string sits in files of both tiers, then take each rule's `name_regex` reach split by tier. - -Declined: - -- `explains = N` per rule as ledger data (2026-09-03, again) — #452's own first sketch, declined there on churn, re-examined here only because this arc pins a different recorded value and owed an account of why the two differ, and declined again on the same grounds. The third decision above is the argument in full: a shape cannot move when a corpus does and a count can. The landed fields-equals-union check already catches the drift that motivated the sketch — a rule's explained set changing SHAPE underneath it — and needs no number to do it. The question belongs to [#452](https://github.com/derek73/python-nameparser/issues/452), which is OPEN and is exactly it; nothing here forecloses it, and its remaining reach-ceiling half should be re-measured on the tier finding above. -- The differential in CI (2026-09-03) — considered and dropped rather than deferred. It is a release gate that already runs locally on demand, and no sibling check — dormancy, over-declaration, the undeclared-contest refusal, this one — needed a CI job to be worth having. Worth recording as a DECLINE and not an omission precisely because the timing finding removes the only reason nobody proposed it: a job costing seconds is cheap, and cheapness was never the criterion. -- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17 — the pair of figures that cannot be re-derived from the tree, since the scan's predicate was never preserved; they live in `8593f3a`'s commit message and the bullet above says so. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, comment lines carrying a digit run to four figures across the differential tooling, its four ledgers and the two guard modules, against 17 claim-shaped ones (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). The exact figure is deliberately not given: an earlier draft of this bullet carried one, and the NEXT commit on this branch added two comment lines and falsified it — inside the entry declining a sweep for exactly that. Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). - -## differential-ledger, the watched-shapes arc (2026-09-05, #501) - -The eighth ledger arc, and the one that lets a diff shape be recorded WITHOUT a winner. The arc above moved the recorded shapes to where a run verifies them and, in the same commit (`962ae43`), bolted them to the winners with `set(winners) == set(shapes)` — never argued as a constraint, and the reason 51 corpus names whose only watcher was a classification rule could not be watched at the shape level: recording a shape meant pinning a winner, and a winner pin asserts an argument nobody had made. Mechanics — the two dicts and their contracts, the three print blocks, which name list each check reads — are owned by `tools/differential/compare.py` (`_WATCHED_DIFFS`' header, which carries the population and its recompute) and tools/differential/README.md's `MOVED SHAPE` section; these are the decisions. Every figure below was measured 2026-09-05 on this branch from real `main()` runs with `classify()` spied, and each carries its recipe; the one figure that goes stale by design, the quiet-tree count, is dated where it appears. Refs [#501](https://github.com/derek73/python-nameparser/issues/501), which stays open. - -Decisions that landed: - -- 2026-09-05 #501 — a shape MAY be recorded without a winner, in a SECOND roster (`compare._WATCHED_DIFFS`, keyed per ledger like `_RECORDED_DIFFS`, disjoint from it per ledger, both keyed by every ledger on disk — held by the guards at pytest speed and, since the PR's review round, by `main()` pre-worker, which refuses a ledger either dict has no section for) and NOT by relaxing the equality. Two grounds, and the first is the arc above's own placement decision turned around: the run cannot see `_CROSS_RULE_WINNERS` — `compare.py` cannot import the test module, and names that roster only in comments and in the advice it prints — so if the KIND of a row is to decide anything at run time, the kind must be DATA in the tool, and one dict with a relaxed guard leaves the run unable to tell a contest row from a standalone one, which means one repair text and one severity for both. The second is that the equality was doing work the relaxation gives up: a shape whose winner was deleted and forgotten is a deleted-winner slip, and under `<=` it is indistinguishable from a deliberate standalone row. So `set(winners) == set(shapes)` stays, both directions, and its docstring now says why the reverse direction is defended — a shape with no winner is not an orphan to tolerate but a row in the wrong dict. A name in both dicts is refused twice, by the guard at pytest speed and by `main()` pre-worker beside the departed-name refusal, because the tool may not assume the suite ran; the argument is the stronger claim, so such a name belongs in `_RECORDED_DIFFS` alone. The day a winner is argued for a watched name, the row MOVES and the pin goes beside it there; it never gains a partner where it is. -- 2026-09-05 #501 — SEVERITY follows the row's kind first and the tier second: a row with an argument behind it (a pinned winner) is fatal on both tiers; a row that is only a snapshot follows its tier, fatal on a contract name and printed under a non-fatal `MOVED SHAPE (radar)` block, parallel to `UNCLASSIFIED (radar)`, on a radar one. Three grounds, from the spec's review amendment A. A per-name shape is a STRONGER promise than a contract name gets — a contract name needs a RULE, which covers a shape class and batches across names, so a feature that legitimately moves thirty of these would produce thirty findings each needing its own "say why", the per-name burden #468 removed, reintroduced one level down on the very file #468 demoted for that reason (`corpus_issues.jsonl`, 49 of the 51; 48 of the 50 since the #501 adjudication below, and the burden argument is unmoved by either count). The `[[never]]` precedent cuts the other way — exclusions are fatal on both tiers because each is a per-name deliberate choice carrying a `why`, and these rows are measured, not reasoned: a sweep, not a choice. And when a snapshot row fires the only repair is to re-snapshot — a contest row has an argument to consult; `'Ph. D., Jr.'` or `'QC MP'` has no adjudicated parse, so "correct the row" means "record whatever it does now", which is mechanisms.md#RECORDED-ROSTERS's re-derivation hazard in a gate's clothing. Two limits on what this decides, stated so nobody reads more into it. It answers the NARROW tier question only — whether a snapshot on a radar name may fail the gate, which `shape_bad` being tier-blind had answered yes by default — and leaves the gate-and-changelog hypothesis where the spec left it: the framing that the ledger is a gate AND a changelog and `_CORPUS_TIERS` splits only the gate is a hypothesis, not codified, and the radar tier's explanation-relief has read 0 at every baseline because no feature work has offered it a case, not because it was declined. And it does not reverse #468; it applies it — "shown, never blocking" reaching a per-name snapshot on a radar name exactly as it reaches an unmatched diff there. It also turns the arc above's radar overlap from unreconciled history into a stated rule: measured 2026-09-03, 21 of the 31 contest rows at 1.4.0 sat on radar-tier names, every one carrying a winner, and they stay fatal because it is the argument and not the tier a contest row defends (RECOMPUTE in tools/differential/README.md's tier paragraph). -- 2026-09-05 #501 (found in the branch's code review) — the tier a watched row's severity follows is the DEFAULT-ORDER ENTRY's, not the name's. A recorded shape is read off the default-order comparison, so that comparison's entry is the one whose tier says whether the shape was promised; the first-loaded (contract-preferring) entry decides only for a name with no default-order entry, which is the name compared under a declared order alone. This is the rule-order arc's third RECOMPUTE trap applied to severity — "a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in", under its #495 decline above, whose measurement is not restated here. The live case is that arc's own: `'John Smith, Dr.'` is contract in `corpus_shapes.jsonl` only as shape 4 (`FAMILY_FIRST`) and radar in `corpus_issues.jsonl` under the default order, so a watched row on it would print and not fail, and the family-first promise would be untouched — it carries no row today, a test naming it. The dict header first said "a name both tiers hold reads contract", which was false of the code as written (`tier_of` is built over the post-skip entries, an order-None entry winning where one exists), and was corrected before the commit was reviewed again. -- 2026-09-05 #501 (found in the design-docs review) — the roster's DEFINING property is "no winner pinned", not "nothing else watches". The four #501 rows are pinned by an exact `Case(...)` literal in `tests/v2/cases.py` and two sit in contract corpora (`'김민준 박사님'` and `'선생님'`, in `corpus_cjk.jsonl` and `corpus_rules.jsonl` alike; the two `様.` strings are in `corpus_cjk_tolerated.jsonl` only — measured 2026-09-05 on this tree and at `e004ccf`, where the four sit in exactly the same files), so "nothing else watches" is the property of the sole-watched SUBPOPULATION and false of the dict as a whole, and a run whose message told a reader of a `'선생님'` row that nothing else watched it would send that reader past the case row that already adjudicates the parse. What every row has, and what the two checks enforce (the disjointness guard, the `both` refusal), is that no winner is pinned. The dict header, the fatal block's lead, the `(radar)` block's lead and the README say "no winner is pinned"; the population comment says which rows are also sole-watched, and said until this branch that a contested row's reader consults the case row before the roster — a line the header no longer needs and no longer carries, the four rows having moved to `_RECORDED_DIFFS` with winners pinned beside them. (The four moved to `_RECORDED_DIFFS` with the #501 adjudication below, 2026-09-05, so every row in `_WATCHED_DIFFS` is sole-watched today; the decision is about the roster's DEFINING property, which is unchanged — no row there pins a winner.) -- 2026-09-05 #501 — every sole-watched diff is recorded — 109 sole-watched rows plus the four #501 contests, 113 in all — not the exposed subset. (AMENDED 2026-09-05, later the same day: the #501 adjudication below took six rows to `_RECORDED_DIFFS` — the four contests and `'MD, PHD'`'s two, which this count read as sole-watched and which were not — so the roster is 107 rows over 50 names and is now exactly the sole-watched population. The decision is about WHICH diffs get a row and survives the totals moving.) Airtightness is a property of today's rule widths and not of the name: a rule whose `fields` happens to equal a name's diff admits no alternative today, and widening that rule later exposes the name with nothing saying so. And the 35 alternatives over the sole-watched rows that route to a DIFFERENT rule (the finding below; 44 with the four contests counted) are silent whatever the winner's width is, which is a stronger reason than airtightness: "every sole-watched diffing name" survives a rule edit, "the exposed subset" is a snapshot of the widths. - -Found rather than decided, and worth as much: - -- **The spec's 385 / 71 was GROSS of #452 and measured on a population the arc had already retracted; the net figure over the roster is the one to argue from.** `over_declared_rules` requires a rule's `fields` to EQUAL the union of the diffs it explains, so on a rule explaining only one name any shrink fires OVER-DECLARED and is not silent at all. Measured 2026-09-05 over the 113 rows `_WATCHED_DIFFS` holds: 390 alternative shapes classify to the same rule and 72 diffs are exposed by that count; net of #452 — keeping only the subsets `S` for which the union of the rule's OTHER explained diffs at that baseline, joined with `S`, still equals the rule's `fields` — 337 alternatives over 64 exposed diffs, and 44 further alternatives route to a different rule. Per baseline the gross / net / other-rule split reads 105 / 96 / 23 at 1.4.0, 158 / 137 / 20 at 2.0.0, 127 / 104 / 1 at 2.1.0 and 0 / 0 / 0 at 2.2.0. Over the 109 sole-watched rows alone — the roster minus the four #501 contests, which is the population the label "sole-watched" names — 367 gross over 68 exposed diffs, 314 net over 60, 35 other-rule, the whole difference sitting at 2.0.0 where the four contested rows are. The spec's first draft measured 385 / 71 gross, 330 / 62 net and 35 other-rule, with 123 / 112 / 23 at 1.4.0, over the population the next bullet retracts: 52 names and 113 diffs, holding the four names with 1.4.0 contest rows (45 rows there, `'MD, PHD'` alone contributing 14 alternatives) and not the four #501 rows — the same total by coincidence, so those figures read as the roster's and were not. The 8 diffs that leave the exposed count under the net test — `'Esq. van Gogh'` at all three baselines where it diffs, `'Dr. Do Van Johnson, MD'` at both 2.x, `'Aishwarya Rai'`, `'Jack M.A.'`, `'Jane van der Berg 旧姓 Jones'` — are exactly the diffs that sit ALONE on their rule (the 72 → 64 drop; the spec's population had 9, `'Carod i'` the ninth, its 71 → 62), but they do NOT account for all the alternatives #452 catches: they carry 24 of the 53 caught over the roster (26 of 55 over the spec's population), and the other 29 come from diffs whose rule explains other names that do not between them cover its `fields`, so the diff stays exposed while some of its alternatives are caught. Two figures are robust across all three populations, and they are what the argument rests on: the worst case, `'Ph. D., Jr.'` at 2.0.0 and 2.1.0, 30 alternatives against a five-field rule, all 30 surviving #452; and the 35 other-rule alternatives over the sole-watched rows, equally unpinned and equally silent, a handover nobody would see — the metric UNDERCOUNTS in that direction whatever it counts in the other. Both corrections leave the conclusion where it was. AMENDED 2026-09-05, later the same day: the #501 adjudication below moved six of these rows out of `_WATCHED_DIFFS`, so the population every figure in this bullet is over went from 113 rows to 107 and the roster figures above are a snapshot of the tree this arc's own commits left. Re-run by the RECOMPUTE below over the 107: 363 gross alternatives over 66 exposed diffs, 310 net of #452 over 58, 35 other-rule, with 105 / 96 / 23 at 1.4.0, 133 / 112 / 11 at 2.0.0, 125 / 102 / 1 at 2.1.0 and 0 / 0 / 0 at 2.2.0, and 8 diffs alone on their rule. Both of the figures the argument actually rests on reproduce unchanged, which is the point of having named them: `'Ph. D., Jr.'` at 2.0.0 and at 2.1.0 is still 30 alternatives against a five-field rule with all 30 surviving #452, and the other-rule count is still 35 — the two rows that left carried no other-rule alternative between them. AMENDED again since #498, which moved five more 1.4.0 rows to `_RECORDED_DIFFS`: the roster holds 102 rows, 36 of them at 1.4.0, and re-running the RECOMPUTE below over those 102 (measured 2026-09-05) gives 337 gross alternatives over 61 exposed diffs, 284 net of #452 over 53, 35 other-rule, with 79 / 70 / 23 at 1.4.0 and 133 / 112 / 11, 125 / 102 / 1 and 0 / 0 / 0 unmoved at the three 2.x baselines, and 8 diffs alone on their rule — only the 1.4.0 line moves, since only 1.4.0 rows left. Both figures the argument rests on reproduce a third time: `'Ph. D., Jr.'` at 2.0.0 and at 2.1.0 is still 30 alternatives against a five-field rule with all 30 surviving #452, and the other-rule count is still 35. RECOMPUTE: spy on `compare.classify` through a real `main()` run at each baseline, keep the default-order calls on the names keyed in `_WATCHED_DIFFS` for that ledger (the roster's own keys, not a scan of `tests/`; drop the four #501 names for the sole-watched figures), and for each enumerate the non-empty subsets of the winning rule's `fields`, sorting each by whether `classify()` routes it to the winner, to another rule, or to nothing; the #452-aware count applies the union test above, and a diff is "alone on its rule" when no other `(name, order)` call at that baseline classified to the same rule. -- **A "where is this named" population must include the tool.** The `tests/` scan cannot see `compare.py`, so four names carrying 1.4.0 contest rows — `'Bob Jones, author'`, `'Carod i'`, `'MD, PHD'`, `'van ma van'` — scored as sole-watched there, watched by exactly the mechanism the arc extends. The definition gained its last clause (not already keyed in `_RECORDED_DIFFS` for that ledger); three of the four return at a 2.x baseline where they have no contest row, and `'Carod i'` diffs under the default order at 1.4.0 ONLY, so it leaves the population entirely: 51 names, not the 52 the `tests/`-only scan gave, and 41 rows at 1.4.0 where the spec's first table said 45. (AMENDED 2026-09-05, later the same day: `'MD, PHD'` leaves entirely too once the #501 adjudication below pins its 2.x pair, since it then carries a contest row at every baseline it diffs at — so TWO of the four return at a 2.x baseline and two leave, and the population is 50 names against the same scan's 52. Re-measured by the recipe: 41 / 31 / 30 / 5 = 107 rows over 50 names, the scan's own 45 / 32 / 31 / 5 = 113 over 52 unchanged, and the roster is exactly the 50-name set.) One more clause the recipe carried and the definition did not until the design-docs review: explained by a ledger rule at that baseline — a diff no rule explains is already printed by every run as unclassified, so a rule is the only weak watcher a row is needed for; a definition clause and not a live count, since `radar unclassified` reads 0 at every baseline today and the two sets coincide. RECOMPUTE: the population recipe is in `_WATCHED_DIFFS`' header; print the names its last clause removes per ledger, and the four appear at 1.4.0 and none elsewhere. (Stale twice over by the end of 2026-09-05, and re-measured that day by exactly that recipe: the clause removes NINE at 1.4.0 — the four plus `'Bob Jones, compositeur'`, `'MD, DO, DDS'`, `'Smith, Ph. D. Jr. MD'`, `'Smith, Ph. D. MD'` and `'Smith, Ph.D. Jr.'`, the five #498 pinned — and `'MD, PHD'` again at 2.0.0 and at 2.1.0 since #501 pinned its 2.x pair, none at 2.2.0. Rows per ledger after the clause: 36 / 31 / 30 / 5.) -- **`corpus_shapes.jsonl`'s table row counted ENTRIES where the population counts NAMES.** A `(name, order)` pair is an entry, so a name compared under three orders is three, while the population is names diffing under the default order. Measured 2026-09-05 over the four baselines: 14 distinct names diff there, 11 under the default order, and the other three — `'de Mesnil Jean, Dr.'`, `'de la Cruz Juan Carlos, Dr.'`, `'de la Cruz née Vega'` — diff only under a declared order, the same trio that is the `NOT CHECKED` window at 1.4.0. The spec's row read "16 entries"; by the recipe here — distinct `(name, order)` pairs on which `classify()` was called across the four baselines — the count is 18, and the basis for 16 was not recorded, so 16 is not carried. Names and default-order names are the figures the population argument rests on, and both reproduce. RECOMPUTE with the same spy, restricted to that file's names, counting pairs, names, and names with `order is None`. -- **The per-file split is not a partition.** 49 of the 51 sole-watched names sit in `corpus_issues.jsonl` and 3 in `corpus.jsonl`, with `'dr Vincent van Gogh dr'` in both, so the per-file counts overlap by one and 49 + 3 reads 52 for a 51-name population. (AMENDED 2026-09-05, later the same day: 48 and 3 over the 50 the #501 adjudication below leaves, the same name overlapping, so 48 + 3 reads 51 for a 50-name population — the shape of the finding, which is that the split is not a partition, is what survives the digits.) RECOMPUTE from the population script's JSON against `_load_entries` over the two radar files. -- **A missing roster section was read as an empty one (found 2026-09-05, in the PR's review round).** `main()` read both shape rosters with `.get(ledger.name, {})`, so a ledger with no section checked nothing and said nothing: with the 1.4.0 key deleted from `_WATCHED_DIFFS` in memory, a full run at that baseline checked its 41 rows against nothing, printed the same 375 lines (differing only in the worker environment's path on the `baseline:` line, which differs between any two runs) and exited 0 — the guards would have caught the deletion at pytest speed, but the run may not assume the suite ran, which is `_CORPUS_TIERS`' own ground. Now refused pre-worker, beside the `both` refusal, naming the dict and the ledger; `_CORPUS_TIERS` refuses a VALUE outside `contract`/`radar` at load on the same principle, since every downstream tier read is an `!=` against one literal and a misspelling would land on whichever side it happened to fall. RECOMPUTE on a tree before the refusal: `del compare._WATCHED_DIFFS['expected_since_1.4.0.toml']` around `main()` at `--baseline 1.4.0`, and diff stdout and the exit code against an unmodified run, with two unmodified runs diffed as the noise control. -- **The tier rule a watched row follows was stated three ways and pinned by none (found 2026-09-05, in the PR's review round).** Three mutants of `tier_of` — order-None entries only, first-loaded entry always, `== "contract"` in place of `!= "radar"` — each passed the whole suite and all four gates byte-identically, because every test drove one bare-string corpus, on which the three readings coincide. What pins the rule now is two fixtures `_run_main` cannot build: the `'John Smith, Dr.'` shape (one string, contract under shape 4 in one file and radar under the default order in another, watched row moved on the default-order comparison — prints `MOVED SHAPE (radar)`, exit 0, which refuses first-loaded-always) and a declared-order-only name (a shape-4 entry with no default-order entry, watched row reporting `measured` None — a tier map over order-None entries alone has no key for it and dies mid-report). RECOMPUTE by applying each mutant to a copy of `compare.py` outside the worktree and running `tests/v2/test_differential.py`; before the two tests, all three pass it. - -Declined: - -- Relaxing `set(winners) == set(shapes)` to `<=` (2026-09-05, the spec's first section) — the first decision above in full: the run cannot read the winner roster, so the kind of a row has to be data, and the equality catches the deleted-winner slip the relaxation would legalize. The spec's own review amendment B withdrew it. -- Fatal-on-radar for snapshot rows (2026-09-05, the spec's default by inheritance from `shape_bad`) — the second decision above; the spec's own "What this costs" section accepted 52 radar names becoming able to fail a release, and amendment A refused that cost as the wrong one for a row nobody chose. Had the 51 — as measured 2026-09-05 before the #501 adjudication below, 50 after — been made fatal after all, that would have been a reversal of #468 for those names and would sit here as one. -- Pinning winners for the six #501 contests (2026-09-05) — stays #501's. The four 2.0.0 rows here and `'MD, PHD'`'s two record SHAPES, not winners: a shape move is now a finding on them, while a handover to another rule is not — file order still decides the winner, and only a winner pin sees it. #501's own objection to pinning a boundary nobody has argued stands, and the issue is scoped down by that comment, not closed. (REVERSED the same day, 2026-09-05, by the #501 adjudication below, which argued all six and moved the rows — read this bullet as what THIS ARC declined to do in its own commits and not as a standing position. What it got right is kept there: the rows recorded shapes and not winners for as long as nobody had argued the boundaries. What it got wrong is that three of the six had already been argued, in the 2.0.0 ledger's own comment, so the objection was about where the argument was written rather than about whether one existed.) -- A checked-in generator for the rows (2026-09-05) — the rows are a snapshot, and a generator that re-derives them on demand is the re-derivation hazard with a command line (mechanisms.md#RECORDED-ROSTERS). A scratch script and a recipe in the roster comment, as `_CORPUS_FLOORS` and `_CORPUS_CLAIMS` do. - -The measurement, and how to redo it. `_WATCHED_DIFFS` holds 41 / 36 / 31 / 5 rows at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 113 in all; the 2.0.0 section is 32 sole-watched rows plus the four #501 contests (`'김민준 박사님'` and `'선생님'`, contract tier and fatal; `'田中さん 様.'` and `'田中さん, 様.'`, `corpus_cjk_tolerated.jsonl`, radar and printed — the first of that pair read contract when this was measured earlier the same day, and the period-class demotion under #cjk-comma-demotion above moved it hours later, which is the point the snapshot makes better than the prose could: a row's fatality is a property of the tier its NAME reads at, and no roster in `compare.py` records it). 51 distinct sole-watched names; every row a default-order shape, so `NOT CHECKED` names none of them. (AMENDED 2026-09-05, hours later: those four rows and `'MD, PHD'`'s two — at 2.0.0 and 2.1.0 — left for `_RECORDED_DIFFS` with the #501 adjudication below, so the roster now reads 41 / 31 / 30 / 5 = 107 over 50 names and the 2.0.0 section is 31 sole-watched rows and nothing else — since #498, 36 / 31 / 30 / 5 = 102 over the same 50, the five names it pinned at 1.4.0 having left that section alone. The figures in this paragraph were true of the tree this arc's commits left; run the RECOMPUTE at the end of it for today's.) Four mutation controls, in memory around `main()` with the worktree untouched, re-run 2026-09-05 against the amended data commit: `'QC MP'` corrupted at 1.4.0 (its only row — it does not diff at 2.0.0) prints `MOVED SHAPE (radar) expected_since_1.4.0.toml: 1 watched diff shape(s) disagree with this run` and exits 0, `'Smith Dr'` at 2.0.0 likewise; `'선생님'` corrupted at 2.0.0 prints `MOVED SHAPE expected_since_2.0.0.toml: 1 watched diff shape(s) disagree with this run` ending "This fails the run because the name is contract tier" and exits 1; `'Ph. D., Jr.'` corrupted at 2.1.0 prints the `(radar)` block and exits 0; and `'MD, PHD'` inserted into the 1.4.0 watched section beside its contest row is refused pre-worker, before any `baseline:` line, as "1 name(s) sit in both `_RECORDED_DIFFS['expected_since_1.4.0.toml']` and `_WATCHED_DIFFS['expected_since_1.4.0.toml']`", and fails `test_the_watched_roster_is_disjoint_and_names_every_ledger` naming it — the guard half wrapping `load_tool`, since that helper executes a fresh `compare` module per call and a mutation on an imported one never reaches it. A control whose corruption changed nothing would have meant an inert row; none did. The arc moved no classification: 352 / 247 / 155 / 14 intentional diffs, 0 unexplained and 0 radar-unclassified at every baseline, byte-identical summary lines before and after every commit. The quiet-tree claim that made this the moment to snapshot: measured 2026-09-05 at `4dd776a`, `git rev-list --count a965272..4dd776a -- nameparser/` is 5 commits since the tier split, moving 38 inserted and 15 deleted lines across two files, `_facade.py` (the #462 initials fix and its review rounds) and `_pipeline/_script_segment.py` (`f046d5d`, the tier-split day's own W3 demotion). PINNED TO THAT COMMIT deliberately, and NO LIVE HEAD FIGURE STANDS BESIDE IT — a first draft of this paragraph put one here as a contrast, and that number was already wrong at the commit that wrote it, which is precisely the defect the pin exists to avoid rather than an illustration of it. What can be said without a number that rots: the period-class bundle's commits after `4dd776a` touched `nameparser/_pipeline/_script_segment.py`'s comments and its W2/W3 citations — and the commit correcting this very sentence was one of them, after a draft that named two by hash had been made stale by the third — so the same range recomputed at HEAD is LARGER than the pinned one on all three counts, and it moves again with every further comment-only commit. Neither the count of such commits nor their hashes belongs here: `git log --oneline 4dd776a..HEAD -- nameparser/` lists them, and a figure that would have to be rewritten by each commit that changes it is not a figure prose can hold. So this is a POINT measurement of the tree the snapshot was taken on, and rereading it as a live one is the error to avoid — the figure goes stale by design, since the next feature is what the snapshot exists to make legible. RECOMPUTE the row counts with `{k: len(v) for k, v in compare._WATCHED_DIFFS.items()}`, the population with the script in the dict's header, the controls with `compare._WATCHED_DIFFS[ledger][name] = (...)` around `main()`, and the quiet tree with the command above plus `git diff --shortstat a965272..4dd776a -- nameparser/` — with the PINNED endpoint and not `HEAD`. A `HEAD` substitution answers a different question, and this paragraph deliberately does not state its answer. - -## differential-ledger, the #501 adjudication (2026-09-05) - -The ninth ledger arc, and the one that answers the question the arc above left open hours earlier in its own `Declined:` list — whether a contest that is MEASURED but unargued is owed a pinned winner. Six were measured across the two 2.x ledgers as this branch opened — five of the 247 diffs at 2.0.0 and one of the 155 at 2.1.0 moving a shape two or more rules admit — and all six are owed one: three of the six boundaries had ALREADY been argued, in a ledger comment, which is a place nothing in the tree can check, and one of those three arguments was false. Closing that one took a regex narrowing rather than a pin, and the narrowing then dissolved two of the six contests, which is this arc's sharpest finding about what a pin is for. Mechanics — the two rosters and their contracts, the severity rule, the population recipe, the six rows' own arguments — are owned by `tools/differential/compare.py` (`_RECORDED_DIFFS`' and `_WATCHED_DIFFS`' headers), by `tests/v2/test_ledger_guards.py` (the comment over `_CROSS_RULE_WINNERS`' two 2.x sections, and the per-row comments) and by tools/differential/README.md; these are the decisions. Every figure below was measured 2026-09-05 on this branch at `ddd8b2b`, from real `main()` runs at all four baselines with `classify()` and `_run_worker` spied, and each carries its recipe. Closes [#501](https://github.com/derek73/python-nameparser/issues/501). - -Decisions that landed: - -- 2026-09-05 #501 — a row is owed where the boundary is ARGUED ANYWHERE, not where it is argued in the guard module. The position the arc above recorded — "a row is owed when somebody ARGUES a boundary, not when one is merely measured" — is kept whole; what moves is WHERE an argument counts. Three of the six boundaries were already argued in the 2.0.0 ledger's own comment on `fix(#308/#312/#319/#320)`, which named `'田中さん 様.'`, `'田中さん, 様.'` and `'김민준 박사님'` one by one and said for each why the glued rule should win. That is a claim about which rule wins a name, and it is checked by NOTHING: `_CORPUS_CLAIMS` measures a rule's regex reach alone and cannot see a name changing hands inside it, the gate's totals are per-corpus, `order_contests` reports only wide-first nested pairs, and `classify()` is never asked the question unless a roster row asks it. `_CROSS_RULE_WINNERS` is the only thing in the tree that can check a ledger comment's adjudication, so scoping its entry criterion to arguments made in its own file left exactly the arguments it exists to check outside it. The proof is the third of the three, below. -- 2026-09-05 #501 — `'田中さん 様.'` and `'田中さん, 様.'` → `fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix`, at 2.0.0. The only one of the six that is a live two-runner contest after the narrowing: both names are admitted by the glued rule and by `fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix`, and the glued rule is written first (sorted indices 1 and 2; lines 71 and 149 of the ledger on this branch). Measured against the 2.0.0 wheel, both diff `{family, given, suffix}` — `'田中さん 様.'` reads first `'田中さん'`, last `'様.'` on the wheel and family `'田中'`, suffix `'さん, 様.'` on the tree; `'田中さん, 様.'` reads first `'様.'`, last `'田中さん'` on the wheel and the same family `'田中'`, suffix `'さん, 様.'` on the tree. Both names move through TWO mechanisms at once — the glued さん peels off 田中さん while the spaced 様 routes to `suffix` — with one rule per half, so neither rule describes the whole diff and the label is PARTIAL in #382's accepted sense. The prose test lands where the file order already does: the glued rule's own #320 clause names both strings outright — "#320 stops an ASCII period vetoing the honorific it finds there ('田中さん 様.', '田中さん, 様.')" — and no clause of the spaced rule mentions either, its examples being `'王小明 先生'`, `'김민준 씨'`, `'田中 太郎 様'`, `'田中 殿'`, `'김민준 씨.'` and `'김민준 양.'`. What the pin buys, measured: swapping the two rules in memory hands BOTH names to the spaced rule and touches nothing else — 2 of the 247 classifications move, `unexplained` stays 0 so the gate's summary line does not move, and neither rule's `_CORPUS_CLAIMS` reach can see a reorder at all. The guard fails on both names, first line "expected_since_2.0.0.toml: '田中さん 様.' diffing ['family', 'given', 'suffix'] is now explained by 'fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix', not 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix'." -- 2026-09-05 #501 — `'김민준 박사님'` → `fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix`, at 2.0.0, and the pin holds only because of the 2026-09-05 narrowing of the glued-peel `name_regex` in both ledgers (`a6a82d6`). Before the narrowing the glued rule reached the name on the 님 interior to 박사님 and, sitting first, took it. Measured against the 2.0.0 wheel the diff is `{family, given, suffix}`: the wheel reads first `'김민준'`, last `'박사님'`, and the tree reads given `'민준'`, family `'김'`, suffix `'박사님'` — the honorific WHOLE and standing as its own word, with nothing peeled off the name. The prose test is rules/W2's second sentence, "A part that is not name text — a post-nominal word standing on its own — is never the name's end: the split-off steps past it to the name word behind, and never dissects it", and that rule's witness line for this very string, `"김민준 박사님" → suffix="박사님"`. So the glued rule's title was FALSE of the name, on a contract-tier string (`corpus_cjk.jsonl`), held green by every guard in the suite from the date the comment's own parenthetical carries (2026-08-05) until this arc. What the pin buys, measured: restoring the pre-narrowing regex ALONE — one edit, no reorder, the glued rule still sitting ahead of the spaced one — hands the name straight back, 1 of the 247 classifications moving with `unexplained` still 0, and this row is the only thing that says so. The guard's first line is "expected_since_2.0.0.toml: '김민준 박사님' diffing ['family', 'given', 'suffix'] is now explained by 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix', not 'fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix'." -- 2026-09-05 #501 — `'선생님'` → `fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots`, at 2.0.0. Measured against the wheel the whole diff is `{family, given}` — the wheel reads first `'선생님'` and the tree reads family `'선생님'`, the 2.1 order flip on a hangul string and nothing else — which is exactly the shape that rule's comment claims for itself, "these are one diff shape -- pieces moving between given/middle/family on a native-script CJK name", and which rules/W2's own boundary line `"선생님" → family="선생님"` says is name text rather than a peel. The glued rule's claim on it was the same regex accident, the interior 님, and worse than a partial label: it declares `suffix` and NO `suffix` moves here at all. It never won, the order rule being written first (sorted index 0 against the glued rule's 1), and the narrowing has since taken its admission away. Recorded because the pair is the #498 class and nothing else can see it: the two rules' `fields` INTERSECT on `{family, given}` — `{family, given, suffix}` against `{_ambiguities, family, given, middle}` — with neither nested, so `undeclared_contests` looks past them in either arrangement and no `precedes_narrower` block is available to declare them. What the pin buys, measured: a reorder ALONE now moves nothing here (0 of the 247), so the row's whole work is against a RE-WIDENING — restore the wide regex and swap the two rules and 2 classifications move, `unexplained` staying 0, with the guard's first line "expected_since_2.0.0.toml: '선생님' diffing ['family', 'given'] is now explained by 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix', not 'fix(#271/#272/#298) native-script CJK: …'". -- 2026-09-05 #501 — `'MD, PHD'` → `fix(#296) a credential-only comma string reads a name and its postnominal`, at 2.0.0 AND at 2.1.0, as a row apiece rather than one shared row. Both #296 rules reach the string and declare EQUAL `fields` at each 2.x baseline, `["title", "suffix"]`, so `fields` cannot separate them at all and file order is the whole decision (sorted indices 25 and 26 at 2.0.0, 12 and 13 at 2.1.0). Measured against both wheels the diff is `{suffix, title}` — title `'PHD'` on the wheel becomes suffix `'PHD'` on the tree, family `'MD'` on both sides. The distinguishing clause is a property of the STRING and not of the rules' `fields`: the credential-only rule opens "'Jr., PhD' / 'MD, PHD': a string that is nothing BUT credentials", while the lone-post-comma rule is written for "A one-word name, a comma, one short word". `'MD, PHD'` holds no name, so the first rule's prose is true of it and the second's is not. Two rows and not one because the roster is keyed per ledger and a shape is baseline-relative — the 1.4.0 row on the same string and the same winner rests on a different measurement, `{family, given, suffix, title}` there, the comma-family move not yet having landed. What the pin buys, measured: swapping the two rules moves 1 of the 247 at 2.0.0 and 1 of the 155 at 2.1.0, `unexplained` staying 0 at both, and the guard fails at BOTH baselines, first line "expected_since_2.0.0.toml: 'MD, PHD' diffing ['suffix', 'title'] is now explained by 'fix(#296) a lone post-comma credential is a suffix', not 'fix(#296) a credential-only comma string reads a name and its postnominal'." and the same sentence under `expected_since_2.1.0.toml`. -- 2026-09-05 #501 — the glued-peel `name_regex` is NARROWED so it stops matching a listed honorific INTERIOR to a longer listed one, in BOTH ledgers that ship it (`expected_since_1.4.0.toml` and `expected_since_2.0.0.toml` carry it byte-identically). `(? fix(#271/#272/#298), the order flip with no comma routing …` — and carries `middle`, which neither name moves. The measurement is what settles the pair, not the table. What the pin buys, measured: moving the loser ahead hands both over and moves 9 of the 352 — the two, plus seven collateral names off `fix(cjk-comma-honorific-peel) glued honorific peels off a post-comma given name`; dropping `given` or `family` from the winner also hands both over — to `fix(cjk-comma-compound)`, this pair's loser — but at the price the nakaguro bullet above measures, since it is the same rule and the same two drops: 43 moved and 35 of them UNEXPLAINED. So here too the reorder is the edit whose cost stays inside this ledger's explained set. -- 2026-09-05 #498 — `'田中さん, Dr.'` → `fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots`, over `fix(cjk-glued-honorific-peel) glued honorific peels into suffix`, at 1.4.0: the SECOND unowned pair on one name, and the reason fifteen rows sit on fourteen names. `{family, given, middle}` against `{family, given, suffix}`, intersecting in `{family, given}` without nesting (sorted indices 0 and 34). Measured, `suffix` is `''` on both sides and the tree's family is the whole token `'田中さん'`: the glued さん is NOT peeled, where the peel this loser is named for would have taken family to `'田中'` and put さん into `suffix` — which is what does happen to `'田中さん, 様.'` in the next bullet, so the contrast between a mechanism that fired and one that did not sits inside this arc's own measurements. Its stated scope excludes the string anyway, the rule being written for a name with "no space before the honorific and no comma anywhere", and this name has a comma. What the pin buys, measured: moving the loser ahead hands the name over and moves 15 of the 352, much the widest of the seven pairs, the peel rule then also taking names off `fix(cjk-comma-compound)` and `fix(cjk-comma-honorific-peel)`. A `fields` narrowing does NOT buy this pair: dropping `given` or `family` from the winner sends the name to `fix(cjk-comma-compound)` and not to this loser, so a reorder is the only single edit that hands it over. -- 2026-09-05 #498 — `'田中さん, 様.'` → `fix(cjk-comma-compound) comma routing compounds with the CJK order flip`, over `fix(cjk-honorific-suffix) postnominal honorifics recognized, compounding with the CJK order flip`, at 1.4.0. `{family, given, title, suffix}` against `{family, given, middle, suffix}`, intersecting in `{family, given, suffix}` with `title` and `middle` keeping either from containing the other (sorted indices 33 and 35). Measured against the wheel: first `'様.'` → given `''`, family `'田中さん'` → `'田中'`, suffix `''` → `'さん, 様.'`, a diff of `{family, given, suffix}`. A genuine TWO-MECHANISM compound, and both halves are legible in the values — the glued さん peels off `田中さん` while the post-comma `様.` leaves `first` for `suffix`, the two arriving there together. The winner describes the comma half; the other half belongs to `fix(cjk-glued-honorific-peel)`, which is nested inside the winner and DECLARED over by a `precedes_narrower` block, so that half of this name's contest is justified where it stands and only the honorific-suffix pair is owed a row. That is the partial label #382 accepts, with every half of the compound owned between the two rules. The loser is the weaker candidate for the same half on its own terms: its prose is about a SPACED trailing honorific whose remainder takes an order flip, it disclaims the glued forms outright — "GLUED diffs (田中さん, 김민준씨, and the 김씨 that segmentation already reached) never match this rule" — and it says nothing about a comma, where the comma is the winner's subject; measured, no `middle` moves and no order flip occurs, `given` going empty rather than gaining a name. What the pin buys, measured: moving the loser ahead hands the name over and moves 2 of the 352 — this name, which comes off `fix(cjk-comma-compound)`, this bullet's winner, and `'田中さん 様.'`, which is the one that comes off the glued rule; a `fields` narrowing buys nothing here either, dropping `given`, `family` or `suffix` from the winner sending the name to the glued rule, so the reorder is again the only single edit. (The 2.0.0 section of `_CROSS_RULE_WINNERS` pins this same string to the GLUED rule. That is a different ledger with a different rule set, where the pair nests narrow-first and no declaration is owed — the #501 adjudication above — and it does not contradict this 1.4.0 verdict.) -- 2026-09-05 #498 — `'Bob Jones, compositeur'` → `fix(comma-family) a comma followed only by titles keeps the given/family split`, over `fix(comma-precomma-family) pre-comma run reads as family, not given`, at 1.4.0. EQUAL `fields`, both `{given, family}`, so file order is the whole decision (sorted indices 5 and 13). Measured against the wheel: first `'Bob Jones'` → given `'Bob'`, last `''` → family `'Jones'`, and title `'compositeur'` on BOTH sides — diff `{family, given}`. The winner's prose names the string and describes those values to the token, "the pre-comma name keeps its positional read -- first 'Bob Jones' -> given 'Bob', family 'Jones', the title staying where it was"; the loser's stated behavior, the pre-comma run reading as family, is half true at most here, `'Jones'` becoming the family and `'Bob'` not, the run not moving whole. The loser's own comment already concedes the sibling string `'Bob Jones, author'` to the winner on exactly that reasoning, and this row records the same for `compositeur`, which the winner's regex reaches by the same literal. It is also why a pin is spelled with the FULL issue string: three rules in this ledger begin `fix(comma-family)` (measured 2026-09-05 with tomllib over the ledger's `[[change]]` issues). What the pin buys, measured: moving the loser ahead moves 3 of the 352 — this name plus `'Bob Jones, author'` and `'John Smith, Mr.'`, and the three do not come off one rule: this name and `'Bob Jones, author'` come off the WINNER, and `'John Smith, Mr.'` off a third rule whose issue extends the winner's, `fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example`. The lone-post-comma rule, the third of the three rules whose issue begins `fix(comma-family)`, loses nothing here — which is the full-issue-string point made in measured form rather than by counting rules that share a prefix. Dropping `given` or `family` from the winner hands this name over on its own, moving 2 (this name and `'Bob Jones, author'`) with nothing left unexplained. -- 2026-09-05 #498 — `'MD, DO, DDS'` → `fix(#296) a dropped prenominal takes the name position it occupies`, over `fix(comma-family) lone post-comma piece routes to suffix/title, not first`, at 1.4.0. `{title, given, middle}` against `{given, title, suffix}`, intersecting in `{given, title}` — which is the measured diff — with neither containing the other (sorted indices 7 and 12). Measured against the wheel: title `'DO'` → `''` and first `''` → given `'DO'`, with family `'MD'` and suffix `'DDS'` byte-identical on both sides. The winner names the string and states its diff verbatim, "the rule's second name is 'MD, DO, DDS': title 'DO' -> given 'DO', the comma anchor's one corpus hit", which is the whole of what moved. The loser is about a one-word NAME followed by a comma and a lone credential; this string holds no name at all and has two post-comma pieces, and its direction is wrong as well — the loser routes a piece OUT of `first` where this name routes one IN. What the pin buys, measured: moving the loser ahead moves 8 of the 352 — this name, all six of the `fix(#325)` pair above, and `'Smith, Jr., PhD'`, since a move above index 7 necessarily passes `fix(#325)` at index 10 — so the reorder is the strictly larger edit here. The `fields` narrowing is smaller and still not an alternative: dropping `given` or `title` from the winner moves 2, handing `'MD, DO, DDS'` to this loser and sending `'Do Quang Minh'` — a radar-tier `corpus_issues.jsonl` name the winner is the sole explainer of — to NO rule, so radar unclassified goes from 0 to 1. Both edits cost something the pin does not; the pin is what records which rule is right without paying either. - -Found rather than decided, and worth as much: - -- **The class is measured in NAMES, not pairs, and the issue's framing had it in pairs.** 60 pairs overlap without nesting and 11 more have equal `fields`, each sharing a corpus name; measured with a diff in hand, 14 names have one that two such rules admit, over seven of those pairs. The static count over-reports for the reason #382's nine-of-eleven finding gave — most co-matched names' real diff needs a role the other rule does not declare — so the rule-order arc's objection to a widened STATIC predicate, that it would demand 111 justifications where the real number is eleven, does not reach a MEASURED one. That is the whole of why option 3 was cheaper than it read. -- **Seven of the fourteen sit in an EQUAL-`fields` pair — the clearest case of the class the docs name `_CROSS_RULE_WINNERS` the instrument for — and not one of the seven was pinned.** Two pairs, and in both `fields` separates the rules not at all and file order is the whole decision: `fix(#325) a split credential followed by another suffix after a one-word family comma reads as suffixes` ahead of `fix(comma-family) lone post-comma piece routes to suffix/title, not first` over six names, and `fix(comma-family) a comma followed only by titles keeps the given/family split` ahead of `fix(comma-precomma-family) pre-comma run reads as family, not given` over `'Bob Jones, compositeur'`. Equal `fields` is the special case the 2026-09-02 entry reasons FROM — the class it leaves to the instrument is every pair where neither set contains the other, and the other seven names sit in that wider half, over eight of the fifteen rows. The instrument was named there and checked by nothing until this arc, and its clearest cases were the unowned ones. -- **Fourteen names, SEVEN (winner, loser) pairs, fifteen (name, pair) rows — and a first draft counted five, by grouping the rows by WINNER.** `fix(#271/#272/#298)` heads three distinct pairs, against the delimited-nickname rule, the comma-compound rule and the glued-peel rule, which collapse to one under a by-winner count; and `'田中さん, Dr.'` is unowned against TWO losers, so it carries two of the fifteen rows and appears under two of the verdict bullets above. Seven pairs over fifteen rows is what the sweep reports. The fourteen is a count of NAMES, which is the unit the pins are written in — one row per name in each roster, whatever the number of losers behind it — and the three figures answer three different questions rather than disagreeing. - -Declined: - -- The wider narrow-first predicate (2026-09-05, the scope decision above) — 9 more names at 1.4.0 and 23 in all, declined as a pin the declaration-free default already justifies. -- A `precedes_narrower`-shaped per-pair declaration for non-nested pairs (2026-09-05, the issue's option 1) — the roster is per-name and already carries the argument, and a second key would let a PAIR be declared without any name's shape being measured. That is the #501 adjudication's own finding one level down: an argument nothing can check, given a key of its own. -- A static sibling of the check (2026-09-05) — there is none to write, for the reason the first decision above gives: without a diff the population is the 71-pair reach — 60 overlapping pairs and 11 with equal `fields` — and the roster is unwritable. -- Reordering any of the seven pairs (2026-09-05) — a reorder moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; every current winner is argued where it sits, and the seven verdict bullets above are that argument, name by name. A wrong winner would have been closed by a narrowing or a declared reorder and not by a pin; the adjudication found none. - -The measurement, and how to redo it. `_RECORDED_DIFFS` reads 45 / 5 / 1 / 0 = 51 rows at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0 and `_WATCHED_DIFFS` 36 / 31 / 30 / 5 = 102 over the same 50 distinct names — five names moved rosters at 1.4.0 and stayed watched at both 2.x baselines, where each has one admitter and no contest to own, so only the 1.4.0 row count fell and the population did not move. 32 of the 45 rows at 1.4.0 sit on radar-tier names and 13 have a single admitter at their recorded shape; none of the fourteen joins those 13, every one having two or more admitters at its recorded shape by the definition of the sweep that found it. THE POPULATION FIGURES, recomputed: drive `main()` at each baseline with `classify()` wrapped, keep the calls whose issue is not None, and for each list the rules satisfying `compare._entry_matches` at that call's measured shape and order — fewer than two admitters is no contest; for each admitter after the first, the winner's `fields` a strict subset of the loser's is narrow-first and justified, the loser's a strict subset of the winner's WITH the loser's issue in `compare._declared_over(winner)` is declared and justified, and anything else is owed a pin unless the name is a key of `_RECORDED_DIFFS` for that ledger. That gives 47 / 3 / 1 / 0 contested, 18 / 3 / 1 / 0 pinned and 14 / 0 / 0 / 0 owed on the tree this arc opened on, and 0 owed at every baseline since. Do NOT replay the corpus load by hand: the `(name, order)` dedup, the baseline-minimum skip and the tier stamp all happen inside `main()`, so a row's tier is read as `main()` reads it — the default-order entry's where there is one, the first-loaded entry's otherwise — by wrapping `_run_worker` for the entries it was handed. THE VERDICTS' values come off that same run — the old side from the worker's rows for the entries `main()` was handed, the new side from `HumanName` and `_tree_parse` as `main()` computes them — and only the facade is compared at a pre-2.0 baseline, `_surfaces_for` returning `{"facade"}` below 2.0.0, so every old and new value in the bullets above is `HumanName`'s. THE "WHAT THE PIN BUYS" FIGURES are measured the same way and their recipe sits beside the rows in tests/v2/test_ledger_guards.py: move the loser above the winner in a COPY of the run's own sorted rule list and re-classify the run's own 352 classified calls, counting the labels that change; the `fields` readings are that recipe with one role dropped from the winner instead of a rule moved, and nothing on disk moves for either. MUTATION CONTROL, in memory around `main()` at 1.4.0 with the worktree untouched: `'Smith, Ph. D. Jr.'` deleted from `_RECORDED_DIFFS['expected_since_1.4.0.toml']` — the contract-tier row of the six-name equal-`fields` pair — makes the run exit 1 and print `UNPINNED CONTEST expected_since_1.4.0.toml: 1 contested pair(s), one per name and losing rule, that no rule outranks by nesting or declaration and no row pins`, naming the name under its winner as `'Smith, Ph. D. Jr.' [contract] ['suffix', 'title']` with the reason "equal `fields`, file order the whole decision". Unmutated, no `UNPINNED CONTEST` block prints at any of the four baselines; a corruption that changed nothing would have meant an inert check, and this one did not. The arc moved no classification: 352 / 247 / 155 / 14 intentional diffs, 0 unexplained and 0 radar unclassified, exit 0, summary lines byte-identical at every commit of the branch — a pin moves no classification, and neither does a check that fires on nothing. RECOMPUTE the row counts with `{k: len(v) for k, v in compare._RECORDED_DIFFS.items()}` and the same over `_WATCHED_DIFFS`; the radar and one-runner splits by taking each row's tier as above and classifying it at its recorded shape against `_sorted_rules`. - -## differential-ledger, a pin names one rule (2026-09-05, #508) - -The eleventh ledger arc, and a small one: it changes how a `_CROSS_RULE_WINNERS` pin is read, not what any pin says. The winner guard compared `classify()`'s answer to the pin with `startswith`, chosen without a recorded reason, so a pin was satisfied by any rule whose issue extended it. The #498 completeness check, earlier the same day, had borrowed that guard's meaning outright — a `_RECORDED_DIFFS` key counts as "a winner is pinned" because the two rosters' keys are equal — and its review found the case where the borrowed meaning was hollow. Mechanics are owned by `_pinned_rule`'s docstring in tests/v2/test_ledger_guards.py; these are the decisions. Every figure was measured 2026-09-05 on master at `449a7bc`. - -Decisions that landed: - -- 2026-09-05 #508 — **a pin names exactly one rule.** The guard resolves each pin to one rule's issue — exact where a rule's issue equals the pin, else the single rule whose issue starts with it — and refuses a pin that resolves to none or to several; the winner comparison is then EQUALITY against the resolved issue. Prefix pins stay: a family prefix that reaches one rule names it as well as the full string does, and 17 of the 51 rows are written that way. Measured over the 51 rows before the change: 31 full issue strings reaching only themselves; 17 short prefixes reaching one rule; 2 full strings that are also a strict prefix of a sibling's — `'Bob Jones, author'` and `'Bob Jones, compositeur'`, pinned to the all-titles rule whose issue `... keeps the given/family split, the C1 example` extends; 1 short prefix reaching three rules — `'Andrews, M.D.'` on the bare `fix(comma-family)`. All three ambiguous rows are in `expected_since_1.4.0.toml`, the 2.x rosters have none, and each was latent: `classify()` returned the intended rule. `'Andrews, M.D.'` is spelled out to the lone-post-comma rule that wins it; the Bob Jones rows are unchanged, exact comparison being what their comment already rested on. A second guard, `test_every_pinned_winner_names_exactly_one_rule`, refuses an ambiguous pin on its own at pytest speed, so the failure names the pin rather than a classification. RECOMPUTE: for each ledger and each pin, count the `[[change]]` issues that `startswith` the pin and whether one equals it (the loop is in the #508 issue body). - -Found rather than decided: - -- **The #498 arc's "a key in `_RECORDED_DIFFS` means a winner is pinned" was true of 48 rows and hollow for 3.** Nothing was wrong today, because `classify()` happened to return the intended rule for all three. What the prefix comparison could not do was FAIL when it stopped: the `'Bob Jones, compositeur'` roster comment rested its pin on the C1 rule's regex being unable to reach the string, an argument no test executed — measured 2026-09-05, forcing `classify()` to answer the C1 string for that name passed the old guard and fails the new one. - -Declined: - -- Exact match everywhere (2026-09-05) — rewriting the 18 short pins to full strings and ending the family-pin idiom. The idiom is harmless where the prefix reaches one rule, which is 17 of the 18, and the resolver makes the eighteenth loud; a reworded rule breaking its pins is the same outcome under either reading. -- Prefix plus an allowlist (2026-09-05, #508's option 2) — a second list to keep honest, for three rows the resolver handles without one. -- A word-boundary prefix (2026-09-05, #508's option 3) — answers the two Bob Jones rows and not `'Andrews, M.D.'`, and adds a tokenising rule to a string comparison. -- Refusing two rules in one ledger from sharing a prefix (2026-09-05) — a property of the ledger, not of a pin; the ledger's rule families are deliberate (`fix(comma-family)`, `fix(#296)`), and the resolver already makes the only case that matters loud. - ## differential-ledger, retired for a snapshot (2026-10-13, #664) The ledgers' last arc. They were built to show that 2.0 changed nothing against 1.4 except fixes the release log lists, and they did; as a standing check every later behavior change had to be classified once per ledger, and the guards policing the ledgers outgrew the ledgers. @@ -295,10 +44,9 @@ Measured 2026-10-12, before the change, over the 40 first-parent commits ending Accepted costs: - No forced explanation per moved name. A reviewer can approve a large snapshot diff without reading it; the grouping by changed fields is the mitigation. -- No bound on what a change may reach. `_CORPUS_CLAIMS`, the dormancy checks and the contest pins existed only to police ledgers; mechanisms.md#LEDGER-RULE-SEPARATION and #CROSS-RULE-OUTCOME-PINS are retired with them. +- No bound on what a change may reach. `_CORPUS_CLAIMS`, the dormancy checks and the contest pins existed only to police ledgers; the two mechanisms entries that described them (LEDGER-RULE-SEPARATION and CROSS-RULE-OUTCOME-PINS) are retired with them. - The 1.4 compatibility promise is a question asked when needed (`compare.py --baseline 1.4.0`), not a gate at every release. The facade is deleted in 3.0. Declined: - Trimming the ledgers rather than retiring them (2026-10-13). The radar tier (#468, 2026-09-01) was that attempt: it stopped new corpora from failing the run, but every contract name still needed a rule in every ledger, and the cost grew by one ledger per release. - diff --git a/docs/design/decisions/generational-words.md b/docs/design/decisions/generational-words.md index e280b40c..1302dd84 100644 --- a/docs/design/decisions/generational-words.md +++ b/docs/design/decisions/generational-words.md @@ -5,8 +5,8 @@ Closes #661. 31 spelled-out generational words join SUFFIX_WORDS, and one predic - **CLDR is corroboration, not a source.** Its person-name samples write mlađi/млађи (hr, sr — on a woman's name too), младши/младша (bg), ifj. (hu) and जूनियर (hi), but its `generation` field also holds translations of the field LABEL rather than of the word: isizukulwane (zu), जेनेरेसन (ne), نسل (ps) and jẹnereṣọn (pcm) all mean "generation". A word is not added because CLDR has it. The wider roster (fils, yngri, الابن and the rest) is open at [#679](https://github.com/derek73/python-nameparser/issues/679), to be vetted word by word the same way. - **Excluded, each with its reason.** The refused words carry `excluded = { generation = … }` with the reason, in the file of the language whose reading is refused: `uv run python tools/vocab/query.py --excluded generation --issue 661` (kıdemli, sóisearach, sinsearach, ifj, id, млада, the CLDR label translation isizukulwane, and the abbreviations ml. and st. in the Latin-script languages and ст. in Russian; мл. shipped later, with #679). Óg and Mór, which Irish writes instead, are also inside given names (Seán Óg Ó hAilpín), so they are a separate decision and not taken here. - **Spaced only, and what that leaves.** bg, ru and uk write a modern Jr. hyphen-glued (Теодор Рузвелт-младши, Дональд Трамп-молодший), which no whole-token set reaches; for those three the gain is mainly the historical the Elder/the Younger names, written title case (Младший), which the lowercase fold matches. -- **The #602 run start reads the letters' script, not ASCII (Derek chose the Latin-script test, 2026-10-14). SUPERSEDED the same day by #679 (decisions/S8): no script is refused now, so the split this bullet accepts is gone.** rules/S2 lets "an unambiguous Latin suffix word of two or more letters" start the run, and `_pieces.starts_a_credential_run` had implemented Latin as `letters.isascii()`, exact while no accented Latin word was suffix vocabulary. This entry adds the first ones (mlađi, mlađa, mladší, starší), so without a change `Pieter Bruegel stariji Jones` would read suffix 'stariji Jones' while `Antonio Pavlović mlađi Jones` kept family Jones -- one language, one construction, split by the `đ`. The test now asks a non-ASCII word whether every letter's Unicode name is a LATIN one; the ASCII test stays first, so an ASCII word pays no frame for it. The same script test is what refuses the non-Latin honorifics (`Taro Tanaka さん Jones` keeps family Jones), and it refuses this entry's Cyrillic and Devanagari generational words with them, though the reason S2 gives for the refusal -- an honorific carries no credential mark -- is not true of them. So Serbian splits by script: `Ana Horvat stariji Jones` reads suffix 'stariji Jones' and `Ана Хорват старији Џонс` family Џонс, and after a comma C1 asks the same predicate, so `Ана Хорват, старији Џонс` reads given 'старији'. ACCEPTED for this entry (Derek, 2026-10-14) and stated at rules/S2's Accepted clause: the vocabulary does not mark which suffix words are honorifics, a script test is the only line available, and only a name word written after the generational word reaches it. Settling it is open at [#679](https://github.com/derek73/python-nameparser/issues/679), where the non-Latin generational words multiply. The first draft of this entry credited the ASCII test to S2's honorific exclusion instead and proposed deleting it, on the evidence that deleting it moved no test and no corpus name. The design-docs review found the attribution wrong and the evidence empty: nothing pinned the test, and deleting it reads `Taro Tanaka さん Jones` as suffix 'さん Jones' and `Гай Плиний Старший Иванов` as suffix 'Старший Иванов'. Both are rules/S2 boundary examples now. #678 was filed on the first draft and corrected. -- **Measurement (2026-10-14).** Six corpus names move, all in corpus_cldr.jsonl and all the CLDR hr/sr renderings carrying mlađi/млађи, each gaining suffix mlađi (or млађи) from a name field; cldr_agreement records 14 changes, every one toward CLDR's fields. The bg and hi renderings do not move because those locales' formats drop the generation field. The run-start change moves no name the corpus held before this entry; its one mover is S2's own `Antonio Pavlović mlađi Jones`. Of the 161 rows of tools/differential/cldr_fields.jsonl whose `generation` value `cldr_agreement.occurs()` in the text, none writing a non-ASCII generation word has a name word after it in the same part (two write given names after `II` across a comma, which ends the run's part). Recompute: `uv run python tools/differential/snapshot.py --since `. +- **The #602 run start reads the letters' script, not ASCII (Derek chose the Latin-script test, 2026-10-14). SUPERSEDED the same day by #679 (decisions/S8): no script is refused now, so the split this bullet accepts is gone.** It had replaced `_pieces.starts_a_credential_run`'s `letters.isascii()`, which would have split `Pieter Bruegel stariji Jones` from `Antonio Pavlović mlađi Jones` by the `đ`, with a test that every letter's Unicode name is LATIN, accepting that Serbian then split by script. +- **Measurement (2026-10-14).** Six corpus names move, all CLDR hr/sr renderings gaining suffix mlađi or млађи from a name field, and cldr_agreement records 14 changes, every one toward CLDR's fields; the run-start change moves no name the corpus held before (measured against the tree before PR #681; recipe in that PR). -- **2026-10-14 #679 — the rest of CLDR's roster, vetted word by word (Derek).** 76 more words join SUFFIX_WORDS (counted against master's set, none removed): the words carrying `issue = 679` (`uv run python tools/vocab/query.py --issue 679`), 30-odd languages. Vetting as above, run by three research passes over each language's Wikipedia (langlinks of the Jr./Sr./the Younger/the Elder articles, then the language's own search) and `tools/name_usage.py`. CLDR's `generation` field proved unreliable in about a third of languages, beyond the field-label translations: an adjective written BEFORE the name (ta இளைய, kn ಕಿರಿ, si බාල, az Kiçik, uz Kichik, tk kiçi, ba кесе, ky Кенже, kk кіші), an abbreviation nobody writes after a name (mr ज्यु., pa ਜੂ., kok धा.), a word that means "second" (chr ᏔᎵᏁ, hi द्वितीय, zh 二世 — regnal, and glued), or a word no Wikipedia uses where another is standard. Where the language's Wikipedia writes a different word after names, that word ships instead of CLDR's: lo ຈູເນຍ for ຮຸ່ນລູກ, ti ጁንየር (and am ጁኒየር) for ማንጁስ, or ଜୁନିଅର/ଜୁନିୟର୍ for ଜେଆର୍, ta இளையவர் and ஜூனியர், si ජූනියර්, mk помладиот for јуниор. EXCLUDED, each with its reason in the file of the language whose reading is refused: `uv run python tools/vocab/query.py --excluded generation --issue 679` (he סניור, borne as a surname, ja シニア beside it, English `senior`, the abbreviations, and the CLDR forms a given name or the initial shape collides with). Left out unnamed: the CLDR forms that are label translations or unattested (cv, chr, ak, blo, kok-Latin, kk-Arabic). PACK-SHAPED, NOT SHIPPED: fr fils/père and the Latin abbreviations ml./st. (cs, sk, sl, hr) are real and standard, but their collisions are with OTHER languages written in the same script — `Marie Fils` (767 US Census bearers of Fils) and `Jiří Dienstbier ml.` beside ML initials and St. as Saint — which is the case a locale pack exists for (the #604 `nic` precedent). No fr or cs pack exists; proposing them is open at [#684](https://github.com/derek73/python-nameparser/issues/684). The Cyrillic мл. does ship: ru, bg and sr all write it after names and nothing collides with it in Cyrillic, so a pack would protect nobody; ст. is barely attested after a name and stays out. Neither existing pack takes anything: tr and az put the junior word BEFORE the name or in a parenthesis, and Turkish baba collides inside Turkish. Measured 2026-10-14: 16 corpus names move, all CLDR renderings in is, ar, fa, gu, kn, ml, ru and cv (cv's rendering through мл.) gaining their generation word as the suffix out of a name field; cldr_agreement's default view records 4 suffix fields newly agreeing (cv, fa, is, ru) and no regression, and 40 still-wrong fields changing across the 16 renderings: 12 suffixes now the generation word alone where CLDR also holds the comma-led credential (`yngri` for `yngri þingmaður`), and 28 family, family_base and given_names fields that lost the word but still disagree. No rendering newly agrees on every role. The Burmese sample glues the Burmese comma ၊ (U+104A) to ဂျူနီယာ and is not reached, the tokenizer not splitting on it. +- **2026-10-14 #679 — the rest of CLDR's roster, vetted word by word (Derek).** 76 more words join SUFFIX_WORDS (counted against master's set, none removed): the words carrying `issue = 679` (`uv run python tools/vocab/query.py --issue 679`), 30-odd languages. Vetting as above, run by three research passes over each language's Wikipedia (langlinks of the Jr./Sr./the Younger/the Elder articles, then the language's own search) and `tools/name_usage.py`. CLDR's `generation` field proved unreliable in about a third of languages, beyond the field-label translations: an adjective written BEFORE the name (ta இளைய, kn ಕಿರಿ, si බාල, az Kiçik, uz Kichik, tk kiçi, ba кесе, ky Кенже, kk кіші), an abbreviation nobody writes after a name (mr ज्यु., pa ਜੂ., kok धा.), a word that means "second" (chr ᏔᎵᏁ, hi द्वितीय, zh 二世 — regnal, and glued), or a word no Wikipedia uses where another is standard. Where the language's Wikipedia writes a different word after names, that word ships instead of CLDR's: lo ຈູເນຍ for ຮຸ່ນລູກ, ti ጁንየር (and am ጁኒየር) for ማንጁስ, or ଜୁନିଅର/ଜୁନିୟର୍ for ଜେଆର୍, ta இளையவர் and ஜூனியர், si ජූනියර්, mk помладиот for јуниор. EXCLUDED, each with its reason in the file of the language whose reading is refused: `uv run python tools/vocab/query.py --excluded generation --issue 679` (he סניור, borne as a surname, ja シニア beside it, English `senior`, the abbreviations, and the CLDR forms a given name or the initial shape collides with). Left out unnamed: the CLDR forms that are label translations or unattested (cv, chr, ak, blo, kok-Latin, kk-Arabic). PACK-SHAPED, NOT SHIPPED: fr fils/père and the Latin abbreviations ml./st. (cs, sk, sl, hr) are real and standard, but their collisions are with OTHER languages written in the same script — `Marie Fils` (767 US Census bearers of Fils) and `Jiří Dienstbier ml.` beside ML initials and St. as Saint — which is the case a locale pack exists for (the #604 `nic` precedent). No fr or cs pack exists; proposing them is open at [#684](https://github.com/derek73/python-nameparser/issues/684). The Cyrillic мл. does ship: ru, bg and sr all write it after names and nothing collides with it in Cyrillic, so a pack would protect nobody; ст. is barely attested after a name and stays out. Neither existing pack takes anything: tr and az put the junior word BEFORE the name or in a parenthesis, and Turkish baba collides inside Turkish. Measured 2026-10-14 against the tree before PR #685: 16 corpus names move, all CLDR renderings gaining their generation word as the suffix out of a name field, and cldr_agreement's default view records 4 suffix fields newly agreeing, no regression, and 40 still-wrong fields changing. The Burmese sample glues the Burmese comma ၊ (U+104A) to ဂျူနီယာ and is not reached, the tokenizer not splitting on it. diff --git a/docs/design/decisions/phd-merge.md b/docs/design/decisions/phd-merge.md index 22beac86..44d13746 100644 --- a/docs/design/decisions/phd-merge.md +++ b/docs/design/decisions/phd-merge.md @@ -7,6 +7,6 @@ OPENING THE NAME IS NOT OPENING A PIECE LIST, and the first draft got that wrong: a credential run legitimately opens segment 1 after a family comma (`Smith, Ph. D. Jr.`), which C1 reads as a listing rather than a name beginning. Testing `k == 0` alone broke fifteen tests and took the gate red at all three baselines. The caller passes the distinction in, since `_group_segment` cannot see which segment it holds. ACCEPTED, and found by review rather than by design: a TITLE before the pair keeps the credential reading, because `k` is a piece index and a title is a piece. `Sir Ph. D. Van Johnson` still reads given 'Van Johnson' with an empty family — #371's own symptom, surviving one word to the left. It is 1.4.0 parity, and the rule states it as a boundary rather than deviating: "the first piece of the name" cannot be computed before this merge, since H2's abbreviation test is true of `Ph.` itself, so a scan that stepped over titles would step over the very piece being judged. rules/S2 therefore says OPENS THE STRING, which is what the code holds and what v1 held. THE FIRST DRAFT GOT "THE HEAD" WRONG IN THE OTHER DIRECTION, and the corpus could not see it. Testing the PIECE index alone declined the merge for `"Bob" Ph. D. John Smith`: `extract_delimited` removes a quoted or bracketed clause before grouping, so the pair reaches the merge at `k == 0` with a word standing before it in the INPUT. v1 merges there (its regex needed only a preceding space), and 112 measured names broke parity — every one opening with a quote or a bracket, and not one of them in any corpus. The test is now `a[0] == 0`, the first TOKEN, which is v1's boundary exactly. - ACCEPTED: `Parser.revise(suffix="Ph. D.")` now renders 'Ph., D.'. revise() runs a full sub-parse of the string it is given, and a field value has no head for a head-position rule to consult; a second draft carved that out by requiring a name to displace, and the carve-out made the head reading depend on what FOLLOWED it — appending a maiden clause changed whether 'Ph.' was a title, which `test_a_maiden_clause_changes_nothing_else` caught. Dropping the carve-out removed the inconsistency with it. The merge exists for a credential someone TYPED after a name; a caller who writes the spaced form into the suffix field is taken at their word (Derek's call, 2026-08-31). SUPERSEDED 2026-09-06 by #511, and half of the reasoning here still holds: the MERGE is a head-position rule and still does not fire in a field value. The other half — a caller who writes the spaced form is taken at their word — is what #511 reverses (Derek's call, 2026-09-06, choosing the entry rule over the accepted split when the round-trip limit was measured). What changed is that rules/R1's entry pass now runs inside `revise()` over the forced roles, and `Ph.` and `D.` share a comma bucket with nothing between them, so the ENTRY pass joins the pair for the reason it joins `MD` and `PhD` — a different rule reaching the same two words. `Parser.revise(suffix="Ph. D.")` renders 'Ph. D.' (decisions/R1, 2026-09-06 #511). + ACCEPTED: `Parser.revise(suffix="Ph. D.")` now renders 'Ph., D.'. revise() runs a full sub-parse of the string it is given, and a field value has no head for a head-position rule to consult; a second draft carved that out by requiring a name to displace, and the carve-out made the head reading depend on what FOLLOWED it — appending a maiden clause changed whether 'Ph.' was a title, which `test_a_maiden_clause_changes_nothing_else` caught. Dropping the carve-out removed the inconsistency with it. The merge exists for a credential someone TYPED after a name. SUPERSEDED 2026-09-06 by #511 in the half that took a caller who writes the spaced form into the suffix field at their word (Derek's call, 2026-08-31, reversed by his call of 2026-09-06): the MERGE is a head-position rule and still does not fire in a field value, but rules/R1's entry pass now runs inside `revise()` and joins the pair for the reason it joins `MD` and `PhD`, so `Parser.revise(suffix="Ph. D.")` renders 'Ph. D.' (decisions/R1, 2026-09-06 #511). What the leading `Ph.` becomes is H2's business, not this rule's: an abbreviation before a name is almost always a title, which is the same clause that reads `Esq. van Gogh` as title 'Esq.' — so the pair reads title 'Ph.', given 'D.'. The issue proposed given 'Ph.', middle 'D.'; H2 claims the first word before the positional read sees it, and the two answers differ only in which field holds `Ph.`. diff --git a/docs/design/decisions/suffix-acronym-collisions.md b/docs/design/decisions/suffix-acronym-collisions.md index f7b4a981..733cc95f 100644 --- a/docs/design/decisions/suffix-acronym-collisions.md +++ b/docs/design/decisions/suffix-acronym-collisions.md @@ -13,12 +13,12 @@ Closes #342 (a wordlist question) and #454 (a rules.md question) together, becau - **Parking lot: the shape-plus-position credential heuristic.** #490's idea without the "collides with vocabulary" part, and what would recover "John Smith RAI" without any wordlist at all. Two shapes: an ALL-CAPS acronym standing in the suffix position of a MIXED-CASE name ("John Smith XYZ", "John Smith, XYZ"), where the case contrast is the signal; and a DOTTED acronym in that position regardless of case ("John Smith X.Y.Z.", "john smith x.y.z."), where the periods are. Measured 2026-09-07, none of the four is read as a suffix today — the bare form gives family XYZ, the comma form given XYZ with family "John Smith", and the second two family X.Y.Z. — while the roman-numeral accident above reads three dotted forms as suffixes on a fork that is about generations, not credentials. That contrast is the evidence the dotted shape is unhandled: the parser already produces the wanted answer for X.Y.I. and R.A.V. by coincidence and the unwanted one for R.A.X. A follow-up issue carries it; nothing in this entry depends on it. - **esq left SUFFIX_ACRONYMS (2026-09-08, the #316/#489 bundle).** The same criterion asked of a word the frequency argument cannot reach: does the entry describe the WORD or the machinery. Esquire is a contraction, not an initialism, so the initialism set was the wrong home for it whatever its frequency — this is the criterion's third kind of answer, and it is the one rai and cha did not need. The entry arrived with the same 2019-12-11 bulk import (af5bdab, #93) that brought rai and cha and was never reviewed either; its only unique coverage was the multi-dot spelling `E.S.Q.`, the SUFFIX_WORDS membership carrying every single-token spelling. Measured over the corpus glob: one name moves, `John Smith E.S.Q.`, from suffix to family, at every baseline. What the removal buys is the invariant this entry's neighbours could not have: `SUFFIX_ACRONYMS ∩ SUFFIX_WORDS == ∅` is asserted at import (a vocabulary build check since 2026-10-18: one suffix kind per word, each an acronym or a word; decisions/vocabulary-model), esq having been the only member of that intersection and the reason the assert could not exist. Recompute the intersection with `L = Parser().lexicon; sorted(L.suffix_acronyms & L.suffix_words)`, which gives `[]` on 2026-09-08. The standing keep-out is the Excluded (SUFFIX_ACRONYMS — esq) block in decisions/vocabulary-exclusions. - **Not decided here:** sa stays where #296's audit put it (title and suffix dual, position decides); se and om stay unambiguous, there being no surname evidence worth standing behind and OM being the Order of Merit; the S2 comma-form boundary ("Smith, Ed" → given Ed) is not reopened. #342's own comment lists sa, se and om beside ba and cha as model recall rather than corpus-attested, and per the #360 lesson they need a human before they move. -- **Measurement (2026-09-07).** Five corpus names move, every one from the radar-tier corpus_issues.jsonl, in three diff shapes with one cause: Aishwarya Rai moves {family, suffix}, Lala Lajpat Rai and John Smith RAI move {middle, family, suffix}, and John Smith, RAI and Ahmad Jayadi, CHA move {given, family, suffix}. Recompute by parsing every name in the tools/differential/corpus*.jsonl glob twice — once with the shipped lexicon, once with `Lexicon.default().add(suffix_acronyms={"rai","cha"})`, which RESTORES the two entries this bundle removed so the diff read forwards is the removal's — and diffing the seven name fields plus `ambiguities`; the ba step of the same sweep moves nothing. Aishwarya Rai is the one name whose diff DISAPPEARS at 1.4.0, that release having read family Rai too, which is why the `fix(#342) NOT WANTED` ledger rule was deleted rather than rewritten and why the 1.4.0 gate lists four names under the replacement where the three 2.x ledgers list five. Read today's intentional counts off the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`; they rose by three at 1.4.0 and by five at each 2.x baseline. +- **Measurement (2026-09-07).** Five corpus names move, every one from the radar-tier corpus_issues.jsonl and with one cause (Aishwarya Rai, Lala Lajpat Rai, John Smith RAI, John Smith, RAI, Ahmad Jayadi, CHA), measured over the tools/differential/corpus*.jsonl glob against the shipped lexicon with the two entries restored (`Lexicon.default().add(suffix_acronyms={"rai","cha"})`); the ba step moves nothing, and Aishwarya Rai is 1.4.0's own reading (PR #515). - **2026-09-15 (#289/#516) — the parking lot is CLOSED, and this entry's "measured 2026-09-07, none of the four is read as a suffix today" sentence is superseded rather than rewritten.** The shape-plus-position heuristic this entry parked is what shipped: the DOTTED shape as `Policy.unlisted_dotted_suffixes`, default on, and the ALL-CAPS shape as `Policy.unlisted_caps_suffixes`, default off. Of the four names the bullet measured, `John Smith X.Y.Z.` and `john smith x.y.z.` now read suffix at the default, and `John Smith XYZ` / `John Smith, XYZ` read suffix only with the opt-in switch. See decisions/S9 for the whole decision and for why the two halves have different defaults. - **2026-09-15 (#289/#516) — the roman-numeral accident is RETIRED, and here is what narrowly means.** This entry's bullet recorded the accident as evidence rather than as behavior anyone relied on. It now fires no longer where every chunk the vocabulary matches is a single ASCII character — measured, `{i, v, 2}` — so `John Smith R.A.I.` and `John Smith X.Y.I.` reach the shape class instead and read by position, which gives the same fields for the first and 1.4.0's family reading for `Jack X.Y.I.`. `John Smith C.H.A.`, the bullet's other half, is unaffected: nothing claimed it before or after. The WIDE retirement was measured and rejected; decisions/S3 records the three names that pay for it. - **2026-09-25 (#540) — meng and lac joined SUFFIX_ACRONYMS_AMBIGUOUS (Derek: the marking over removal, and no masks).** #vocabulary-collisions C-i asked at the position the suffix claim acts on, the last word of a name: Meng (孟) is a common Chinese surname and given name, and Lac (Lạc) is a Vietnamese given name — the trailing word in native order, `Nguyễn Văn Lạc` — and a French surname; both are borne in that slot (`wang meng`, `tran lac`, `nguyen van lac`). MEng (Master of Engineering) and LAc (Licensed Acupuncturist) are live credentials, and neither reading is rare enough beside the other to give it the word — the rough balance this entry's criterion marks (where it is uncertain, C-i's default is the marking too). They are the set's first entries longer than two letters, which is the LENGTH clause of the criterion bullet applied, not an exception to it. Neither was checked against a surname when it arrived: `lac` with the af5bdab import (2019-12-11, #93), `meng` with 3e14ea20 (2026-07-01, the German/Dutch title and degree batch, first shipped in 1.3.0). No rule changes: rules/S2 reads the two words exactly as it reads `ma` and `ba`. Measured 2026-09-25 on this tree, the parent 120033b5 and the five released wheels (each wheel run as the Gotchas in AGENTS.md prescribe, from a directory outside the checkout under `PYTHONSAFEPATH=1`, asserting `nameparser.__file__`). THE FIX: `wang meng`, `li meng` and `tran lac` read given + family with a `suffix-or-name` report, where every release from 2.0.0 read suffix and no family name — reporting nothing at 2.0.0 through 2.2.0, and `given-or-family` at 2.3.0 and the parent, the one-word-name fork, which was the wrong fork — and 1.4.0 read the family, unflagged. The family-comma spelling returns to 1.4.0 with them, `Wang, Meng` reading given `Meng`, family `Wang` where 2.0.0 through 2.3.0 read suffix `Meng`; and under `FAMILY_FIRST` the bare `Wang Meng` reads given `Meng` where 2.0.0 through 2.3.0 and the parent read family `Wang`, suffix `Meng`, so the comma spelling under the default order and the bare one under `FAMILY_FIRST` still agree with each other. WITH WORDS TO SPARE the credential stays: `john smith meng`, `JOHN SMITH MENG` and `nguyen van lac` keep their suffix, now reported, and so does `john smith m.eng.`, which S2's period gate (rules/S3's since 2026-10-19) does not settle, that gate counting one period after each letter. DECIDED (Derek, 2026-09-25): the bare `Wang M.Eng.` reading — family `M.Eng.`, nothing to spare — is accepted and recorded rather than widening the period gate to cover a chunked dotted spelling; S2's period gate counts a member as unambiguous only when it is written one period per letter (`M.A.`), and MEng and LAc are the first members whose conventional dotted spelling is chunked instead. `John Smith M.Eng.` keeps the suffix by the words-to-spare count, unaffected. THE ACCEPTED COST is the one `ma` and `ba` already carry, and it falls on the conventional spellings: in a name written in more than one case S2 reads a member written neither in capitals nor all lower as the name even with words to spare, and MEng and LAc are written exactly that way. `john smith MEng` reads middle `smith`, family `MEng`, where every release from 1.4.0 read suffix `MEng`; `John Smith Meng` and `John Smith LAc` move the same way; and the Title-case `Nguyen Van Lac` reads family `Van Lac` where every release read family `Van`, suffix `Lac` (for this Vietnamese name the gain, the mechanism being the cost's) — only the all-lower `nguyen van lac` keeps its suffix, one case saying nothing. A CREDENTIAL RUN ending in one of them pays the same walk-stopping cost: `Mary Jones PhD MEng` reads middle `Jones PhD`, family `MEng`, and `Jane Doe MS LAc` reads middle `Doe MS`, family `LAc`, where every release read the whole run as a suffix (`PhD, MEng` and `MS, LAc` through 2.2, `PhD MEng` and `MS LAc` in 2.3); `John Smith MEng PhD` reads middle `Smith`, family `MEng`, suffix `PhD`, the unambiguous credential behind the declined pick still peeled; the mechanism is S2's own — the case signal costs a genuine suffix standing behind a name-leaning acronym, the walk stopping at the declined pick rather than continuing past it — pinned by `tests/v2/cases.py`'s `a_declined_ambiguous_pick_stops_the_walk` and, for the family-comma shape, `a_declined_member_ends_the_trailing_run` (`Doe, John MA Ma`). A QUIETER COST sits behind a suffix comma: once the run's last word is a bare ambiguous member, the run is no longer wholly suffix-shaped, so C1 reads the comma as a family comma instead and the run's own words fall into the usual post-comma name slots — `John Smith, PhD MEng` reads given `PhD`, middle `MEng`, family `John Smith`, and `john smith, phd meng` reads given `phd`, family `john smith`, suffix `meng` (the count then peels `meng` with a word to spare); neither case form saves it, `JOHN SMITH, PHD MENG` reading the same way. A title-listed word standing in front of the run reads as a TITLE instead, with no report: `Jane Doe, MS LAc` reads title `MS`, given `LAc`, family `Jane Doe`, unflagged, where every release read suffix `MS LAc`. The path is not new: `john smith, phd ma` and `Jane Doe, MS Ma` already read this way, `ma` having been ambiguous all along. Pinned by `tests/v2/cases.py`'s `comma_credential_run_ending_in_meng_re_reads_the_comma` and `comma_lower_credential_run_ending_in_meng_re_reads_the_comma`. The comma re-read (with its silent title-dual case) and whether a listed member's chunked dotted spelling should pass the period gate are [#544](https://github.com/derek73/python-nameparser/issues/544)'s questions. After a comma, `Smith, MEng` reads given `MEng` (1.4.0's reading; 2.0.0 through 2.3.0 read suffix) — and so does the all-lower `Smith, meng`, a restored reading rather than a cost — and `Smith, John MEng` middle `MEng`, pinned as case rows (`Smith, meng`, `Smith, MEng`, `Smith, John MEng`) and classified by the `fix(#540)` rules, while `John Smith, MEng` keeps the suffix on C1's name-word count (rules/C4 since #728); bracketed, `John Smith (MEng)` reads nickname `MEng` with a `suffix-or-nickname` report, S1's escape declining an ambiguous member, where every release read suffix `MEng`. `John Smith MENG` keeps the credential on the capitals lean. The judgment is the one behind rai and cha: a missed credential leaves the letters in a name field where a human can still see them, and a wrong credential reading destroys a surname on every record it appears in. edd and ded are NOT marked: both are borne as GIVEN names (`Edd Smith`, `Ded Gjo Luli`), which is not the position the suffix claim acts on, and no measurement shows either losing a family name — a trailing bearer turning up reopens them. Recomputed with #vocabulary-collisions' recipe on 2026-09-25: suffix_acronyms 608 (unmoved by this change) with 7 ambiguous, 5 before this change (particles 70 with 37 and titles 758, unmoved). - **Declined 2026-09-25 (#540), with measurement: removing the two words, and masking them.** REMOVAL, the rai/cha answer, measured with `Parser(lexicon=Lexicon.default().remove(suffix_acronyms={"meng", "lac"}, suffix_acronyms_ambiguous={"meng", "lac"}))`: `john smith meng` reads middle `smith`, family `meng`, and `JOHN SMITH MENG` family `MENG`, while `john smith m.eng.` stays a suffix by its dotted shape (S3), reported. Removal loses the bare credential outright where the marking keeps it with words to spare, and C-i's default under uncertainty is the marking. MASKS `meng → MEng` and `lac → LAc`, measured with both added to a private lexicon on top of the marking: the exceptions map is role-free (#R4, 2026-09-23), so `wang meng` renders `Wang MEng`, `tran lac` `Tran LAc` and `meng li` `MEng Li` — the mask re-spells the very surnames the marking restores. A suffix-gated mask would reverse the 2026-09-23 role-free decision for two words and is not taken. Without one, the default path renders `wang meng` as `Wang Meng` (the parent gave `Wang MENG`, the credential clause writing a suffix in capitals), `tran lac` as `Tran Lac`, `john smith meng` as `John Smith MENG` and `nguyen van lac` as `Nguyen Van LAC` by the acronym clause, and keeps a writer's `MEng` as written (#R5, 2026-09-24) — `john smith MEng` gives `john smith MEng`, and `John Smith Meng` under `force=True`, the word being the family name there. Pinned by `tests/v2/test_render.py::test_a_listed_acronym_that_is_a_name_word_gets_no_mask`, whose recorded negative control carries the two masks. -- **Measurement (2026-09-25, #540).** No corpus name written before this change carries a bare trailing `meng` or `lac` (the `tools/differential/corpus*.jsonl` glob at 120033b5), so the population that could move is this change's own shape-tagged case rows, and every mover is one of them. Classified by `fix(#540)` rules: at each 2.x baseline `wang meng` and `tran lac` move {family, suffix, _ambiguities}, `john smith MEng` and `John Smith MEng PhD` move {middle, family, suffix, _ambiguities}, and `john smith meng` and `nguyen van lac` `_ambiguities` alone; `Smith, John MEng` moves {middle, suffix, _ambiguities} at 2.x and {middle, suffix} at 1.4.0; `Smith, meng` and `Smith, MEng` move {given, suffix, _ambiguities} at 2.x and nothing at 1.4.0 (1.4.0's own reading); `John Smith, PhD MEng` moves {given, middle, family, suffix, _ambiguities} and `john smith, phd meng` {given, family, suffix, _ambiguities} at 2.x (without `_ambiguities` at 1.4.0); `Nguyen Van Lac` moves {family, suffix, _ambiguities} at 2.x and {family, suffix} at 1.4.0; and `Wang M.Eng.` moves {family, suffix, _ambiguities} at 2.x only, 1.4.0 reading the same family for an unrelated reason (its two-piece rule -- a lone word after the given name is the family -- not S2's gate). `nguyen van lac` also diffs at 1.4.0 on `_initials` (`n.` → `n. v.`) for fix(#385/#402)'s reason — its family `van` is a one-particle part — and joins that rule's literal list; the same case-insensitive regex also reaches `Nguyen Van Lac` at every baseline, without explaining anything there (its own family move, at 1.4.0 too, is explained by the Title-case Lac rule instead). Read today's intentional counts off the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`; this change raised them by seven at 1.4.0 and by thirteen at each 2.x baseline. +- **Measurement (2026-09-25, #540).** No corpus name written before this change carries a bare trailing `meng` or `lac` (the `tools/differential/corpus*.jsonl` glob at 120033b5), so every mover at every baseline is one of this change's own case rows, classified by `fix(#540)` rules (PR #543). - **2026-09-27 (#544) — #540's two pending questions are answered, and the #540 bullet stands as it landed.** The chunked dotted spelling passes S2's period gate (rules/S3's since 2026-10-19) — `Wang M.Eng.` reads suffix 'M.Eng.' as `Wang M.A.` does, and as 2.0.0 through 2.3.0 read it — and the comma is kept: a credential run after a full name that ends in a member is the credential run by C1's name-word count (rules/C4 since #728) (`John Smith, PhD MEng` suffix 'PhD MEng'; `Jane Doe, MS LAc` suffix 'MS LAc' where #540 left title 'MS', given 'LAc', at the cost, accepted in decisions/S7, of `John Smith, Ms Ma` reading suffix 'Ms Ma'). The walk-stopping cost that bullet lists for a run ENDING in a member goes with them, by S2's company clause: `Mary Jones PhD MEng` and `Jane Doe MS LAc` read the whole run as a suffix again. Where the member LEADS the run nothing speaks for it, so `John Smith MEng PhD` keeps its cost. The `fix(#540)` ledger rules for the comma run and the dotted spelling are retired. `fix(#544)` rules classify the comma-run names; `Wang M.Eng.`, whose roles every release from 2.0.0 read, is classified by `feat(#449)`'s given-or-family report rule at 2.0.0 through 2.2.0 and by the dotted `fix(suffix-routing)` rule beside `Jack M.A.` at 1.4.0, and differs from 2.3.0 not at all. The case rows the #540 bullet cites were renamed with the readings they pin: `a_declined_ambiguous_pick_stops_the_walk` is `a_credential_in_front_anchors_a_declined_pick` (`abdul Smith Jr Ma`, suffix 'Jr Ma'), and `comma_credential_run_ending_in_meng_re_reads_the_comma` and `comma_lower_credential_run_ending_in_meng_re_reads_the_comma` are `comma_credential_run_ending_in_meng_keeps_the_suffix_comma` and `comma_lower_credential_run_ending_in_meng_keeps_the_suffix_comma`. The decision and its measurements are decisions/S7's #544 entry. - **2026-10-18 — cva[22], gaee[14] and np[18] left SUFFIX_ACRONYMS: Wikipedia footnote marks, not credentials.** All three arrived with the `af5bdab` bulk import (2019-12-11), the bracketed number being the source page's reference mark copied along with the acronym; the 3.0 vocabulary spec's migration spike found them. They matched only a token written with the mark itself, so `John Smith NP[18]` read suffix `NP[18]` and now reads family `NP[18]`, and nothing else moves: `np` and `cva` have clean entries of their own, and `gaee` never had one (`John Smith GAEE` reads family `GAEE` before and after). Whether GAEE should be added as a clean entry is a separate vetting question, not taken here. A bulk import from a wiki page carries its reference marks: check a new list for brackets before it lands. Recompute that no vocabulary entry holds one: `uv run python -c "from nameparser import Lexicon; L = Lexicon.default(); print(sorted(w for f in ('titles', 'suffix_acronyms', 'suffix_words', 'particles', 'conjunctions', 'bound_given_names', 'maiden_markers') for w in getattr(L, f) if set(w) & set('[]')))"`. diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index 70d4c2ee..c9426213 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -2,8 +2,7 @@ How this codebase solves recurring problem shapes. Before proposing a design, check whether an entry already fits — the catalog is keyed by problem shape, because that is what you have in hand when searching. -Entry format: an `## UPPER-KEBAB-SLUG — one-line slogan` heading, then **Problem shape** (the situation that should trigger recall), -**Contract statement** (one or two sentences stating what the mechanism promises — the citable line), **How it works**, **Lives in**, and **Reach for it when** (the tell-tale sign you should be using this instead of inventing something). +Entry format: an `## UPPER-KEBAB-SLUG — one-line slogan` heading, then **Problem shape** (the situation that should trigger recall), **Contract statement** (one or two sentences stating what the mechanism promises — the citable line), **How it works**, **Lives in**, and **Reach for it when** (the tell-tale sign you should be using this instead of inventing something). Each label opens its own paragraph, and that is load-bearing: tests/v2/test_doc_citations.py reads a Contract statement to the end of its paragraph, so whatever shares the paragraph becomes quotable. An entry carries its contract and pointers; how an instance was found belongs in the decisions entry that decided it, or in the Field notes below when it is a trap rather than a decision. The Contract statement is citable under the same discipline as rules: a committed comment making a mechanism claim — including any claim about which stage or layer does something — cites the slug and quotes a verbatim excerpt (`# mechanisms.md#SPANS: ...`), checked by the citation-integrity test. Free-prose restatements of mechanism claims are how one wrong sentence has shipped six times. @@ -11,129 +10,331 @@ This catalog converts discovery into a one-time cost — it does not remove disc ## SPANS — position is identity, text is not -Problem shape. A later stage needs to refer to "that word." Contract statement. Every token carries its (start, end) character span in the original string, and stages refer to tokens by index, never by searching for matching text. How it works. v1 re-found pieces by value, so a name with a repeated word could rewrite the wrong occurrence (#100 and relatives); a position cannot be confused with a look-alike. Every parsed part is an exact slice of the input (rule T1's Background). Lives in. nameparser/_types.py (Span), threaded through every _pipeline/ stage. Reach for it when. New code is about to do `if token.text == ...` to LOCATE rather than to classify. +**Problem shape.** A later stage needs to refer to "that word." + +**Contract statement.** Every token carries its (start, end) character span in the original string, and stages refer to tokens by index, never by searching for matching text. + +**How it works.** v1 re-found pieces by value, so a name with a repeated word could rewrite the wrong occurrence (#100 and relatives); a position cannot be confused with a look-alike. Every parsed part is an exact slice of the input (rule T1's Background). + +**Lives in.** nameparser/_types.py (Span), threaded through every _pipeline/ stage. + +**Reach for it when.** New code is about to do `if token.text == ...` to LOCATE rather than to classify. ## FOLDED_TAG — reorder at render time, not parse time -Problem shape. A rule wants words to RENDER in a different order than they sit in the string. Contract statement. Tokens never move: a rule that needs different rendering order tags the token, and the rendering views consult the tag — EVERY view that renders the affected role, ordering folded tokens first, not the field views alone. How it works. Reordering the token tuple would break span math and reintroduce the #100 family. Parse state stays in string order; only the view reorders (rule R1, rule R3's order clause, rule O3's render clause). The consumer list is where this mechanism fails, in two opposite directions — and only ONE of them has ever shipped, which is worth keeping straight because a roster is walked differently when it is guarding against a defect than when it is guarding against a possibility. SHIPPED: a consumer that never read the tag (`initials()` walked written order through 2.0, 2.1 and 2.2 — #408, filed as an instance of RENDER-HONORS-THE-PARSE and fixed 2026-08-30). HAZARD, never released: a consumer that reads the tag where it should not (the revise strip below), which arrived with `Parser.revise` in the same commit as the test that pins it, so no version has gone out without it and no issue, decision entry or test names a defect of that shape. Adding a producer is cheap; adding a view is where the roster has to be walked. Lives in. nameparser/_types.py (FOLDED_TAG, `_text_for` — every role it renders, not FAMILY alone), nameparser/_render.py (`initials`, the same partition per role), nameparser/_facade.py (`_list_tokens_for`, which prepends the carriers the same way — it is the ONE walk behind both the v1 `*_list` views and the facade's initials since #528, and `_list_for` is the string view built on it and holds no ordering of its own, so a reader who follows this roster to the string builder lands one level below the partition), nameparser/_pipeline/_post_rules.py and _assign.py (its producers: O3's fold and P6's no-comma attachment in post_rules, P6's attachment after a family comma in assign since #613 -- every one of which re-roles to FAMILY — so FAMILY is the only role a parse can put the tag on today, and the views partitioning every role is uniformity with this contract rather than reachable behavior), and nameparser/_parser.py (`Parser.revise`, the one deliberate CONSUMER-side strip: it removes the tag from the tokens it harvests, a revised value not being allowed to inherit fold ordering) — losing that strip is this mechanism's measured hazard, a family rendering "García Gabriel Márquez". A module path and not a method name, because this roster is what a reader walks when adding a view, and a bare `Parser.revise` sends them looking in the wrong file. Reach for it when. A new rule needs "X renders before Y" and you are tempted to swap tokens. Don't swap. Tag. +**Problem shape.** A rule wants words to RENDER in a different order than they sit in the string. + +**Contract statement.** Tokens never move: a rule that needs different rendering order tags the token, and the rendering views consult the tag — EVERY view that renders the affected role, ordering folded tokens first, not the field views alone. + +**How it works.** Reordering the token tuple would break span math and reintroduce the #100 family. Parse state stays in string order; only the view reorders (rule R1, rule R3's order clause, rule O3's render clause). The consumer list is where this mechanism fails, in two opposite directions — and only ONE of them has ever shipped, which is worth keeping straight because a roster is walked differently when it is guarding against a defect than when it is guarding against a possibility. SHIPPED: a consumer that never read the tag (`initials()` walked written order through 2.0, 2.1 and 2.2 — #408, filed as an instance of RENDER-HONORS-THE-PARSE and fixed 2026-08-30). HAZARD, never released: a consumer that reads the tag where it should not (the revise strip below), which arrived with `Parser.revise` in the same commit as the test that pins it, so no version has gone out without it and no issue, decision entry or test names a defect of that shape. Adding a producer is cheap; adding a view is where the roster has to be walked. + +**Lives in.** nameparser/_types.py (FOLDED_TAG, `_text_for` — every role it renders, not FAMILY alone), nameparser/_render.py (`initials`, the same partition per role), nameparser/_facade.py (`_list_tokens_for`, which prepends the carriers the same way — it is the ONE walk behind both the v1 `*_list` views and the facade's initials since #528, and `_list_for` is the string view built on it and holds no ordering of its own, so a reader who follows this roster to the string builder lands one level below the partition), nameparser/_pipeline/_post_rules.py and _assign.py (its producers: O3's fold and P6's no-comma attachment in post_rules, P6's attachment after a family comma in assign since #613 -- every one of which re-roles to FAMILY — so FAMILY is the only role a parse can put the tag on today, and the views partitioning every role is uniformity with this contract rather than reachable behavior), and nameparser/_parser.py (`Parser.revise`, the one deliberate CONSUMER-side strip: it removes the tag from the tokens it harvests, a revised value not being allowed to inherit fold ordering) — losing that strip is this mechanism's measured hazard, a family rendering "García Gabriel Márquez". A module path and not a method name, because this roster is what a reader walks when adding a view, and a bare `Parser.revise` sends them looking in the wrong file. + +**Reach for it when.** A new rule needs "X renders before Y" and you are tempted to swap tokens. Don't swap. Tag. ## VOCAB-TAGS — the vocabulary layer speaks once -Problem shape. A later stage needs to know what the vocabulary knew about a word. Contract statement. classify tags every token with what the vocabulary knows about it, and later stages test tags — they never re-look a word up. How it works. One lookup site means one answer: a stage that re-derived vocabulary facts could disagree with the stage before it. Stable tags ("particle", "conjunction", "initial") are API; +**Problem shape.** A later stage needs to know what the vocabulary knew about a word. + +**Contract statement.** classify tags every token with what the vocabulary knows about it, and later stages test tags — they never re-look a word up. + +**How it works.** One lookup site means one answer: a stage that re-derived vocabulary facts could disagree with the stage before it. Stable tags ("particle", "conjunction", "initial") are API; "vocab:"-namespaced ones are not. -Lives in. nameparser/_pipeline/_classify.py (producer); consumers throughout _group/_assign/_post_rules, and nameparser/_render.py, which is not a stage but reads the same tags to decide what a view shows (#458 moved case repair's conjunction test onto the tag; initials() had read them since 2.0), and nameparser/_facade.py, whose `_token_is_conjunction` joined them in #528 — the v1 layer is a tag consumer too now, so a sweep that stops at `_render.py` is one module short. Reach for it when. A stage is about to import Lexicon to ask about a word classify already saw — or a view is, which is the harder one to notice, since a view legitimately holds a Lexicon for the questions classify never answered; the view side is RENDER-HONORS-THE-PARSE, which owns it and covers the decisions this entry does not produce. + +**Lives in.** nameparser/_pipeline/_classify.py (producer); consumers throughout _group/_assign/_post_rules, and nameparser/_render.py, which is not a stage but reads the same tags to decide what a view shows (#458 moved case repair's conjunction test onto the tag; initials() had read them since 2.0), and nameparser/_facade.py, whose `_token_is_conjunction` joined them in #528 — the v1 layer is a tag consumer too now, so a sweep that stops at `_render.py` is one module short. + +**Reach for it when.** A stage is about to import Lexicon to ask about a word classify already saw — or a view is, which is the harder one to notice, since a view legitimately holds a Lexicon for the questions classify never answered; the view side is RENDER-HONORS-THE-PARSE, which owns it and covers the decisions this entry does not produce. ## PIECES — joining structure survives assignment -Problem shape. A rule needs to know how words were JOINED (chained titles, particle groups), not just what roles they got. Contract statement. group records the joining structure as pieces — runs of token indices per segment — and that structure remains readable after roles are assigned. How it works. Roles alone lose the grouping ("who chained with whom"); #359's fix made the particle fold read the opening PIECE rather than the assigned role, which is what makes rule P1 hold under every name_order. Lives in. nameparser/_pipeline/_group.py (producer), _pipeline/_state.py (ParseState.pieces), _post_rules.py (reader). Reach for it when. A rule keyed on assigned roles behaves differently under different name_order values — the stable thing to read is usually the structure. Known limit, found by #395: the structure survives ASSIGNMENT, not group's own later merges. The prefix chain absorbs a joined piece into a longer one, so a rule reading pieces after that point can no longer see the join — see UNIT-PARTITION. +**Problem shape.** A rule needs to know how words were JOINED (chained titles, particle groups), not just what roles they got. + +**Contract statement.** group records the joining structure as pieces — runs of token indices per segment — and that structure remains readable after roles are assigned. + +**How it works.** Roles alone lose the grouping ("who chained with whom"); #359's fix made the particle fold read the opening PIECE rather than the assigned role, which is what makes rule P1 hold under every name_order. + +**Lives in.** nameparser/_pipeline/_group.py (producer), _pipeline/_state.py (ParseState.pieces), _post_rules.py (reader). + +**Reach for it when.** A rule keyed on assigned roles behaves differently under different name_order values — the stable thing to read is usually the structure. Known limit, found by #395: the structure survives ASSIGNMENT, not group's own later merges. The prefix chain absorbs a joined piece into a longer one, so a rule reading pieces after that point can no longer see the join — see UNIT-PARTITION. ## UNIT-PARTITION — count the units the joining rules built -Problem shape. A rule counts "one name word", but the input holds words that another rule has already joined into one name — and the joining structure it would read has been merged away. Contract statement. Three rules build multi-word units: a particle chain (P2), a conjunction join (P3), and a bound given-name pair (P5). A rule that counts name words counts those units, and takes each whole or not at all — with one stated exception, rules/C3's count of the words before a comma, which counts particle surnames alone (decisions/C3, #575). How it works. group builds each join as a piece, but its own prefix chain then merges the joined piece into a longer one, so PIECES no longer carries the boundary — the units are rebuilt from the tags the vocabulary layer left (`particle`, `conjunction`, `vocab:bound-given`). The rebuild is RECURSIVE: what a conjunction or a bound word joins is the next UNIT, not the next word, and absorbing a single token instead strands a particle severed from the words it chains. Note the two joins arrive here for opposite reasons — the conjunction join was built and then swallowed, while the bound-given join was never built at all (P5 joins only at the first non-title piece), so restoring piece boundaries in group would fix the first and silently split the second. Lives in. nameparser/_pipeline/_vocab.py (`unit_ends`, the walk, shared since #575) and its two readers: _post_rules.py (`_units`, over classify's tags, for P1's fold) and rules/C3's counts before a comma: `_comma._whole_name`, over classify's tags (`surname_unit_facts`) since #613 moved the decision after classify, and `_vocab.name_word_count`, which builds its facts from the vocabulary (`surname_unit_tags`, held to classify's tags by an agreement test) -- a view script_segment needed while it asked a hand copy of C1 before classify ran, until #613's /simplify had it ask `_comma.decide` itself on a classified copy. A fourth walk is S2's read (#614): `_pieces._chain_units` marks each piece by the unit P2's chain WILL put it in -- opening a unit, or a name word the chain joins -- asking the chain's own `chain_run_end`, which `_group`'s chain calls too; a word the reading weighs (`_weighed`) keeps a unit of its own, as a position always counted it. The particles straight behind each other are merged ahead of the read, into the piece the chain would open its run with (`_pieces.merge_particle_runs`, #625), so a particle run is one piece the read takes whole or not at all, and no mark stands for a particle inside it. The trailing peel's words to spare and #602's run start count in those marks, the peel stops at a word the chain joins, and everything else -- the numeral fork's look at the word in front, the H5 chain, the run's title test -- reads the pieces as written. It runs before the chain joins name words to a particle run, which is why it cannot read the merged tags `unit_ends` walks; #614 had rebuilt the chain as a merged COPY instead, whose three defects in one review -- words the reading weighs hidden inside a unit, indices drifting from the pieces' -- were each patched in that copy, and #620 replaced it with the marks (decisions/S2). C3 counts particle chains only, reaching one word as the fold does: a bound pair builds a given name, and whether P3 joins a connective depends on the whole name the count is helping to decide, so neither is one surname before a comma (`_vocab.SURNAME_UNIT_TAGS`, decisions/C3). rules/P1 and rules/C3 are the counting rules, P2/P3/P5 the joining ones. Reach for it when. A rule says "one name word" and the input can contain a join — enumerate the joining rules out of the rules rather than the ones you remember. +**Problem shape.** A rule counts "one name word", but the input holds words that another rule has already joined into one name — and the joining structure it would read has been merged away. + +**Contract statement.** Three rules build multi-word units: a particle chain (P2), a conjunction join (P3), and a bound given-name pair (P5). A rule that counts name words counts those units, and takes each whole or not at all — with one stated exception, rules/C3's count of the words before a comma, which counts particle surnames alone (decisions/C3, #575). + +**How it works.** group builds each join as a piece, but its own prefix chain then merges the joined piece into a longer one, so PIECES no longer carries the boundary — the units are rebuilt from the tags the vocabulary layer left (`particle`, `conjunction`, `vocab:bound-given`). The rebuild is RECURSIVE: what a conjunction or a bound word joins is the next UNIT, not the next word, and absorbing a single token instead strands a particle severed from the words it chains. Note the two joins arrive here for opposite reasons — the conjunction join was built and then swallowed, while the bound-given join was never built at all (P5 joins only at the first non-title piece), so restoring piece boundaries in group would fix the first and silently split the second. + +**Lives in.** nameparser/_pipeline/_vocab.py (`unit_ends`, the walk, shared since #575) and its two readers: _post_rules.py (`_units`, over classify's tags, for P1's fold) and rules/C3's counts before a comma: `_comma._whole_name`, over classify's tags (`surname_unit_facts`) since #613 moved the decision after classify, and `_vocab.name_word_count`, which builds its facts from the vocabulary (`surname_unit_tags`, held to classify's tags by an agreement test) -- a view script_segment needed while it asked a hand copy of C1 before classify ran, until #613's /simplify had it ask `_comma.decide` itself on a classified copy. A fourth walk is S2's read (#614): `_pieces._chain_units` marks each piece by the unit P2's chain WILL put it in -- opening a unit, or a name word the chain joins -- asking the chain's own `chain_run_end`, which `_group`'s chain calls too; a word the reading weighs (`_weighed`) keeps a unit of its own, as a position always counted it. The particles straight behind each other are merged ahead of the read, into the piece the chain would open its run with (`_pieces.merge_particle_runs`, #625), so a particle run is one piece the read takes whole or not at all, and no mark stands for a particle inside it. The trailing peel's words to spare and #602's run start count in those marks, the peel stops at a word the chain joins, and everything else -- the numeral fork's look at the word in front, the H5 chain, the run's title test -- reads the pieces as written. It runs before the chain joins name words to a particle run, which is why it cannot read the merged tags `unit_ends` walks; #614 had rebuilt the chain as a merged COPY instead, whose three defects in one review -- words the reading weighs hidden inside a unit, indices drifting from the pieces' -- were each patched in that copy, and #620 replaced it with the marks (decisions/S2). C3 counts particle chains only, reaching one word as the fold does: a bound pair builds a given name, and whether P3 joins a connective depends on the whole name the count is helping to decide, so neither is one surname before a comma (`_vocab.SURNAME_UNIT_TAGS`, decisions/C3). rules/P1 and rules/C3 are the counting rules, P2/P3/P5 the joining ones. + +**Reach for it when.** A rule says "one name word" and the input can contain a join — enumerate the joining rules out of the rules rather than the ones you remember. ## MARK-DONT-STRIP — record the decision, keep the fact -Problem shape. A stage concludes that a vocabulary fact is not operative for one occurrence — a particle that joins nothing, a middle that will render as family. Contract statement. It MARKS the token with a namespaced marker rather than removing the vocabulary tag: the tag says what the word IS, the marker says what was decided about it, and consumers ask for the pair. The cost, which has to be paid in the docs rather than avoided: where a stable tag was documented with a RECIPE — `particle` combined with Role.FAMILY "for actual family particles" — the marker makes the recipe diverge from the view it reproduced, so the recipe's documentation has to say so. How it works. Stripping is tempting because every downstream consumer becomes correct for free, with no predicate to change; it pays for that by destroying the record, and asymmetrically. `particle` is the only tag all but two of the never-given particles carry, so stripping it leaves nothing on the token, while every ambiguous one still carries `vocab:particle-ambiguous` (the two exceptions, `mc` and `vd`, carry `vocab:suffix` — the same dual membership P6's S2 precedence turns on) — the half where a later fork is most interesting is the half that loses its evidence, and ParsedName has no lexicon to re-derive from. Marking also leaves a POSITIVE predicate: "this token used to have a tag" is not testable, `UNJOINED_TAG in tags` is, which is what an ambiguity emitter needs (#405). And `particle` is documented stable API meaning "a word from the particle vocabulary wherever it lands", so removing it would be a breaking change as well as a lossy one. Lives in. nameparser/_types.py — FOLDED_TAG (O3's fold, P6's attachment), UNJOINED_TAG (R2's standalone particle) and UNJOINED_CONJUNCTION_TAG (R3's connective with nothing in its part to join, #461), all namespaced and internal, qualifying stable tags that stay put. Reach for it when. A rule decides a word is not doing the job its vocabulary implies. +**Problem shape.** A stage concludes that a vocabulary fact is not operative for one occurrence — a particle that joins nothing, a middle that will render as family. + +**Contract statement.** It MARKS the token with a namespaced marker rather than removing the vocabulary tag: the tag says what the word IS, the marker says what was decided about it, and consumers ask for the pair. The cost, which has to be paid in the docs rather than avoided: where a stable tag was documented with a RECIPE — `particle` combined with Role.FAMILY "for actual family particles" — the marker makes the recipe diverge from the view it reproduced, so the recipe's documentation has to say so. + +**How it works.** Stripping is tempting because every downstream consumer becomes correct for free, with no predicate to change; it pays for that by destroying the record, and asymmetrically. `particle` is the only tag all but two of the never-given particles carry, so stripping it leaves nothing on the token, while every ambiguous one still carries `vocab:particle-ambiguous` (the two exceptions, `mc` and `vd`, carry `vocab:suffix` — the same dual membership P6's S2 precedence turns on) — the half where a later fork is most interesting is the half that loses its evidence, and ParsedName has no lexicon to re-derive from. Marking also leaves a POSITIVE predicate: "this token used to have a tag" is not testable, `UNJOINED_TAG in tags` is, which is what an ambiguity emitter needs (#405). And `particle` is documented stable API meaning "a word from the particle vocabulary wherever it lands", so removing it would be a breaking change as well as a lossy one. + +**Lives in.** nameparser/_types.py — FOLDED_TAG (O3's fold, P6's attachment), UNJOINED_TAG (R2's standalone particle) and UNJOINED_CONJUNCTION_TAG (R3's connective with nothing in its part to join, #461), all namespaced and internal, qualifying stable tags that stay put. + +**Reach for it when.** A rule decides a word is not doing the job its vocabulary implies. ## STRUCTURE-GATES — comma shape as an explicit state -Problem shape. A rule should fire only under one comma convention. Contract statement. The comma structure (NO_COMMA, FAMILY_COMMA, SUFFIX_COMMA) is decided once and every later stage gates on that single decision rather than re-inspecting commas. segment parts the string and hands every comma form over as FAMILY_COMMA; group decides it at its head, once classify has tagged the words (#613), and script_segment, which runs between and reads the handover, asks its own narrower question where it needs more (`_postnominal_behind_a_whole_name`); classify, the other stage between, reads no structure since #613. Lives in. nameparser/_pipeline/_state.py (Structure), _pipeline/_segment.py (the parts), _pipeline/_comma.py (the one decider, called by group). Reach for it when. New code is about to count commas. +**Problem shape.** A rule should fire only under one comma convention. + +**Contract statement.** The comma structure (NO_COMMA, FAMILY_COMMA, SUFFIX_COMMA) is decided once and every later stage gates on that single decision rather than re-inspecting commas. segment parts the string and hands every comma form over as FAMILY_COMMA; group decides it at its head, once classify has tagged the words (#613), and script_segment, which runs between and reads the handover, asks its own narrower question where it needs more (`_postnominal_behind_a_whole_name`); classify, the other stage between, reads no structure since #613. + +**Lives in.** nameparser/_pipeline/_state.py (Structure), _pipeline/_segment.py (the parts), _pipeline/_comma.py (the one decider, called by group). + +**Reach for it when.** New code is about to count commas. ## TWO-LAYER-ASSIGN — vocabulary claims, position takes the rest -Problem shape. Where should a new "recognize X" behavior live? Contract statement. A vocabulary layer first claims words for what they ARE, wherever they sit; a positional layer then reads every unclaimed word by where it STANDS. Every rule belongs to exactly one layer — with one named exception: the leading-abbreviation shape (rule H2) fires before the suffix vocabulary is consulted, so +**Problem shape.** Where should a new "recognize X" behavior live? + +**Contract statement.** A vocabulary layer first claims words for what they ARE, wherever they sit; a positional layer then reads every unclaimed word by where it STANDS. Every rule belongs to exactly one layer — with one named exception: the leading-abbreviation shape (rule H2) fires before the suffix vocabulary is consulted, so "Esq. Smith" reads title, not suffix. -How it works. The two layers compose without ordering bugs because the positional layer never overrides a vocabulary claim (rule O4 is the positional layer's contract). The exception is still ONE, re-checked 2026-09-08 against the trailing title run (rule H5), which reads title vocabulary in the trailing slot and is NOT a second one: it runs after the suffix peel rather than before it, so a post-nominal keeps its claim — `John Smith Esq.` reads suffix and `John Smith Prof. Jr.` reads suffix `Jr.`, both measured. That rule is also the worked answer to the Reach-for-it line below: it wanted a word's identity and its position at once and was split by SLOT, the front asking about shape (H2) and the back about vocabulary (H5), two questions with two criteria rather than one rule fighting both layers. Lives in. _classify/_group (vocabulary side), _assign (positional side). Reach for it when. A proposed rule wants a word's identity AND its position at once — split it, or it will fight both layers. + +**How it works.** The two layers compose without ordering bugs because the positional layer never overrides a vocabulary claim (rule O4 is the positional layer's contract). The exception is still ONE, re-checked 2026-09-08 against the trailing title run (rule H5), which reads title vocabulary in the trailing slot and is NOT a second one: it runs after the suffix peel rather than before it, so a post-nominal keeps its claim — `John Smith Esq.` reads suffix and `John Smith Prof. Jr.` reads suffix `Jr.`, both measured. That rule is also the worked answer to the Reach-for-it line below: it wanted a word's identity and its position at once and was split by SLOT, the front asking about shape (H2) and the back about vocabulary (H5), two questions with two criteria rather than one rule fighting both layers. + +**Lives in.** _classify/_group (vocabulary side), _assign (positional side). + +**Reach for it when.** A proposed rule wants a word's identity AND its position at once — split it, or it will fight both layers. ## READ-WITHOUT-THEN-BIND — read the name without the construct, then fix that reading -Problem shape. An optional construct sits inside a name — a maiden clause, a trailing title — and the requirement is that writing it changes nothing else about the name. Checked after the fact, that invariant has to be defended at every place the construct can end, and each defense has to model what later stages would do to the words it gives up. Contract statement. Read the name as if the construct were not written, using the rule that reads that position, not the whole parse. The construct takes only what that reading leaves. Where later stages could still move what the reading decided, give those words their roles at the take so no later join can reach them. The invariant then holds by construction and is not a property each stop must defend. How it works. M2 before #601 scanned forward, stopped at suffix-looking words, and kept a release check asking at each stop whether the name left standing would read what was given up as post-nominals. That check modelled the P2 chain, the P5 bound join and P6's attachment, and #548 was it failing at the one stop that never asked. #601 replaced the stops and the check with the clause-free reading plus the bind: a net 689 lines out of `_group.py` (+210/−899 in the #601 commit), and #535's grids went from 15,586 and 8,713 invariant violations to 0 (prototype measurement, 2026-10-03, #601's comments). The first half already appears twice. H5's trailing title is TRANSPARENT: "what stands once the chain is taken reads exactly as it would read written without the title, plus the title". P3 and P8 ask the word count and the one-case question of the name's OWN words, so a clause beside the name changes neither. M2 is the first to bind as well, because its construct sits in front of the run it reads and the joins come later. C1 is the second (#613, rules/C7): the part after the comma is read once, after classify and before any join, by the reading a part holding no name word gets, and a postnominal part's roles are set there. segment had decided the structure before the tags existed, so it modelled group's particle chain (#562's pair) and assign's reading of a run the capitals settle, and assign re-decided #296's positional read with a second count; all three went. S2 is the third (#614): the trailing run of a main segment is read once, between P3's connective joins and the particle chain, and split off, so the joins that had re-asked the peel before and after themselves -- the chain's stop and its rollback, the bound-given reserve's two views -- answer from that one reading, and assign places the run off it. Where the read sits is the lesson. Joins the run does not depend on but whose units it counts come FIRST: read before P3's connectives, the H5 chain took the second title of a bare `Mr. and Mrs.` and left 'and' as the family name, and a joined pair counted as two words (#614's review). Joins the run does depend on come after, and the read counts what they will build instead -- the particle chain's run and the words it joins as one unit, marked on the pieces rather than built, with every word the reading weighs still a word of its own: #614 counted through a merged copy, whose review found it hiding a shape-only roman numeral and a title inside a credential run, each patched there, until #620 counted the chain's units over the pieces as written (decisions/S2). Two limits, both measured on #601. Read with the TRAILING rule, not the full parse: a head the full parse reads differently (H1's lone title, a P5 join) would otherwise move where the construct ends. And binding means the run can read differently from the same words with no construct (`JANE Q. DOE GEB. LE DO DO` gives suffix `DO DO`, while the full parse of `JANE Q. DOE DO DO` chains them into the family), which rules/M2 accepts in so many words. Lives in. nameparser/_pipeline/_group.py (`_maiden_take`: the view, `tail_reading` over it, and the role-tagged pieces `group()` applies); nameparser/_pipeline/_pieces.py (`tail_reading`, H5's chain); the P3 own-word test in `_group.py`; nameparser/_pipeline/_comma.py (C1's read-and-bind); `_pieces.read_trailing_run` and `_chain_units`, and the read after P3's joins in `_group._group_segment` (S2's). Reach for it when. A rule is growing stops, exceptions or a release check whose job is to keep one construct from disturbing the rest of the name, and above all when that check has started to model another stage. Ask what the name reads as without the construct, and make that the rule. +**Problem shape.** An optional construct sits inside a name — a maiden clause, a trailing title — and the requirement is that writing it changes nothing else about the name. Checked after the fact, that invariant has to be defended at every place the construct can end, and each defense has to model what later stages would do to the words it gives up. + +**Contract statement.** Read the name as if the construct were not written, using the rule that reads that position, not the whole parse. The construct takes only what that reading leaves. Where later stages could still move what the reading decided, give those words their roles at the take so no later join can reach them. The invariant then holds by construction and is not a property each stop must defend. + +**How it works.** M2 before #601 scanned forward, stopped at suffix-looking words, and kept a release check asking at each stop whether the name left standing would read what was given up as post-nominals. That check modelled the P2 chain, the P5 bound join and P6's attachment, and #548 was it failing at the one stop that never asked. #601 replaced the stops and the check with the clause-free reading plus the bind: a net 689 lines out of `_group.py` (+210/−899 in the #601 commit), and #535's grids went from 15,586 and 8,713 invariant violations to 0 (prototype measurement, 2026-10-03, #601's comments). The first half already appears twice. H5's trailing title is TRANSPARENT: "what stands once the chain is taken reads exactly as it would read written without the title, plus the title". P3 and P8 ask the word count and the one-case question of the name's OWN words, so a clause beside the name changes neither. M2 is the first to bind as well, because its construct sits in front of the run it reads and the joins come later. C1 is the second (#613, rules/C7): the part after the comma is read once, after classify and before any join, by the reading a part holding no name word gets, and a postnominal part's roles are set there. segment had decided the structure before the tags existed, so it modelled group's particle chain (#562's pair) and assign's reading of a run the capitals settle, and assign re-decided #296's positional read with a second count; all three went. S2 is the third (#614): the trailing run of a main segment is read once, between P3's connective joins and the particle chain, and split off, so the joins that had re-asked the peel before and after themselves -- the chain's stop and its rollback, the bound-given reserve's two views -- answer from that one reading, and assign places the run off it. Where the read sits is the lesson. Joins the run does not depend on but whose units it counts come FIRST: read before P3's connectives, the H5 chain took the second title of a bare `Mr. and Mrs.` and left 'and' as the family name, and a joined pair counted as two words (#614's review). Joins the run does depend on come after, and the read counts what they will build instead -- the particle chain's run and the words it joins as one unit, marked on the pieces rather than built, with every word the reading weighs still a word of its own: #614 counted through a merged copy, whose review found it hiding a shape-only roman numeral and a title inside a credential run, each patched there, until #620 counted the chain's units over the pieces as written (decisions/S2). Two limits, both measured on #601. Read with the TRAILING rule, not the full parse: a head the full parse reads differently (H1's lone title, a P5 join) would otherwise move where the construct ends. And binding means the run can read differently from the same words with no construct (`JANE Q. DOE GEB. LE DO DO` gives suffix `DO DO`, while the full parse of `JANE Q. DOE DO DO` chains them into the family), which rules/M2 accepts in so many words. + +**Lives in.** nameparser/_pipeline/_group.py (`_maiden_take`: the view, `tail_reading` over it, and the role-tagged pieces `group()` applies); nameparser/_pipeline/_pieces.py (`tail_reading`, H5's chain); the P3 own-word test in `_group.py`; nameparser/_pipeline/_comma.py (C1's read-and-bind); `_pieces.read_trailing_run` and `_chain_units`, and the read after P3's joins in `_group._group_segment` (S2's). + +**Reach for it when.** A rule is growing stops, exceptions or a release check whose job is to keep one construct from disturbing the rest of the name, and above all when that check has started to model another stage. Ask what the name reads as without the construct, and make that the rule. ## LICENSED-BY-NEIGHBOR — a structural word is structural only beside the right neighbor -Problem shape. A word in a structural vocabulary — a marker announcing a maiden name, a connective joining surnames — turns up where what it would announce makes no sense: behind a title alone, behind a credential, at the end of a name. Designing a reading for each such shape is endless, and the shapes are mostly junk. Contract statement. Make the vocabulary claim conditional on the neighbor: the word is structural only where the word beside it has the role the structure needs. Anywhere else it is an ordinary word and goes to the positional layer (TWO-LAYER-ASSIGN), with no reading designed for the junk. How it works. M2 (#601): a marker counts only behind a name word of the part holding the family name, not behind the leading title run, an unambiguous suffix word or a connective. So `Dr. nee Smith` gives first `nee`, last `Smith`, and `Jane Doe PhD nee Smith` gives suffix `PhD nee Smith`, neither designed. P3 is the same shape for the connectives that are also generational vocabulary: `i` joins only where "a name word stands on each side of it". The condition must be checked against real names, not just against the vocabulary. #601's first cut also refused a marker behind a particle and broke `Mai Le née Nguyen` and `Anh Do née Tran` — Le, Do and Van are particle vocabulary and among the commonest Vietnamese surnames. Lives in. nameparser/_pipeline/_group.py (`_maiden_take`'s head rule; `_between_name_words` for P3). Reach for it when. A structural word is producing bad readings in positions its structure does not fit. Make its claim depend on the neighbor before adding a reading or a stop for each position. +**Problem shape.** A word in a structural vocabulary — a marker announcing a maiden name, a connective joining surnames — turns up where what it would announce makes no sense: behind a title alone, behind a credential, at the end of a name. Designing a reading for each such shape is endless, and the shapes are mostly junk. + +**Contract statement.** Make the vocabulary claim conditional on the neighbor: the word is structural only where the word beside it has the role the structure needs. Anywhere else it is an ordinary word and goes to the positional layer (TWO-LAYER-ASSIGN), with no reading designed for the junk. + +**How it works.** M2 (#601): a marker counts only behind a name word of the part holding the family name, not behind the leading title run, an unambiguous suffix word or a connective. So `Dr. nee Smith` gives first `nee`, last `Smith`, and `Jane Doe PhD nee Smith` gives suffix `PhD nee Smith`, neither designed. P3 is the same shape for the connectives that are also generational vocabulary: `i` joins only where "a name word stands on each side of it". The condition must be checked against real names, not just against the vocabulary. #601's first cut also refused a marker behind a particle and broke `Mai Le née Nguyen` and `Anh Do née Tran` — Le, Do and Van are particle vocabulary and among the commonest Vietnamese surnames. + +**Lives in.** nameparser/_pipeline/_group.py (`_maiden_take`'s head rule; `_between_name_words` for P3). + +**Reach for it when.** A structural word is producing bad readings in positions its structure does not fit. Make its claim depend on the neighbor before adding a reading or a stop for each position. ## STATE-OFFSET-CHANNELS — early facts ride the state -Problem shape. A fact known during tokenization matters to a much later stage. Contract statement. A pre-token fact is recorded as offsets on the ParseState (comma_offsets, interpunct_offsets) and consulted later by position — or by presence alone where the fact is name-level, as both interpunct consumers do, or as a single boolean for the whole name, as `ParseState.one_case` is (#289/#516: the written-case fact three later sites read — the trailing suffix slot, the post-comma given slot, and the tail-segment reading — none of which may disagree about it) — rather than re-derived from text. How it works. The offsets survive every intermediate stage untouched; #298's transcription marker rides this channel from tokenize to order resolution (rules T3/W4). `one_case` differs from the offset channels in WHO writes it: it is computed once by whichever of segment and classify needs it first (classify always, segment only where a comma form could turn it on), recorded as `None` for "not asked yet", and never recomputed once set (decisions/S6). Lives in. nameparser/_pipeline/_state.py, produced in _tokenize (the offsets) or in segment/classify (`one_case`). Reach for it when. You are about to re-scan the original string in a late stage to rediscover something tokenize already knew. +**Problem shape.** A fact known during tokenization matters to a much later stage. + +**Contract statement.** A pre-token fact is recorded as offsets on the ParseState (comma_offsets, interpunct_offsets) and consulted later by position — or by presence alone where the fact is name-level, as both interpunct consumers do, or as a single boolean for the whole name, as `ParseState.one_case` is (#289/#516: the written-case fact three later sites read — the trailing suffix slot, the post-comma given slot, and the tail-segment reading — none of which may disagree about it) — rather than re-derived from text. + +**How it works.** The offsets survive every intermediate stage untouched; #298's transcription marker rides this channel from tokenize to order resolution (rules T3/W4). `one_case` differs from the offset channels in WHO writes it: it is computed once by whichever of segment and classify needs it first (classify always, segment only where a comma form could turn it on), recorded as `None` for "not asked yet", and never recomputed once set (decisions/S6). + +**Lives in.** nameparser/_pipeline/_state.py, produced in _tokenize (the offsets) or in segment/classify (`one_case`). + +**Reach for it when.** You are about to re-scan the original string in a late stage to rediscover something tokenize already knew. ## PIPELINE-STAGE-CONTRACTS — the ownership map -Problem shape. "Which stage does X?" — asked before attributing behavior in prose, comments, or fixes. Contract statement. Each stage's docstring header declares what it consumes, produces and reads, and ParseState's docstring holds the cross-stage map, pinned by tests/v2/pipeline/test_state.py. How it works. A claim about which stage or layer does something is CHECKABLE — `parse(s).tokens` prints every token's role and tags — so check it before writing it; one plausible attribution sentence once shipped six times wrong (AGENTS.md's stage-attribution note). Lives in. nameparser/_pipeline/_state.py and every stage header. Reach for it when. Writing any sentence of the form "X happens before Y sees it." +**Problem shape.** "Which stage does X?" — asked before attributing behavior in prose, comments, or fixes. + +**Contract statement.** Each stage's docstring header declares what it consumes, produces and reads, and ParseState's docstring holds the cross-stage map, pinned by tests/v2/pipeline/test_state.py. + +**How it works.** A claim about which stage or layer does something is CHECKABLE — `parse(s).tokens` prints every token's role and tags — so check it before writing it; one plausible attribution sentence once shipped six times wrong (AGENTS.md's stage-attribution note). + +**Lives in.** nameparser/_pipeline/_state.py and every stage header. + +**Reach for it when.** Writing any sentence of the form "X happens before Y sees it." ## ONE-PREDICATE-PER-QUESTION — one predicate answers it, and every other site calls that -Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix; is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage; and maiden_marker_run, the #434 instance and the clearest two-stage case, called by classify over token texts and by extract over a clause's whitespace words, with group reading the tags classify recorded because it runs later; and delimiter_cores, the #436/#437 instance, read by group where a tail segment DROPS a configured delimiter core and by post_rules where the suffix view's entry boundary asks whether a dropped token was one, with a third reader inside this same module, is_wholly_suffix, where a configured core counts as suffix-shaped; and in_initialless_script, the #322/#323 instance and the only one here that is a REPERTOIRE test rather than a vocabulary one — the script half of the #320 initial veto, read by is_initial one function away and by _pieces.is_leading_title, so "a script with no initials has no period abbreviations either" is one predicate over _policy._NO_INITIALS rather than a second reading of that table; it lost its leading underscore when the second caller arrived; and caps_shape_candidate, the #516 instance and the newest, called from the sites that each needed the identical question answered — classify's own tag emission and the comma decision's caps test (_segment.py's two comma tests until #613 moved the decision into _comma.py and classify's comma-run branch went with it), with its unit tests — where the usual reason for keeping such copies apart (a shared call costing every default-policy parse a frame it cannot use) does not hold: every caller asks first, in C, what the predicate would decline anyway — the setting and position, then alphabetic capitals and no listed suffix word — so only an unlisted all-caps word reaches the call, at the default as under any setting (decisions/S9)) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, leading_titles and peel_walk are called by both stages, as is is_title_piece since #602 and #603 gave assign two calls of its own (re-measured 2026-10-11), while is_leading_title and trailing_start are called by group alone (measured 2026-09-06 by call site: `is_leading_title` has no caller in `_assign.py`, which reads `leading_titles` instead — a first draft of this clause listed it among the shared ones) — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain stops at, and M2's walk wherever no trailing rule reads the clause (elsewhere, since #535, the walk stops where `tail_reading` says) — and segment_suffix_reading, by assign alone from #436/#437 and by `_comma.decide` alone since #613 moved the decision and its binding there, that last one being #430's instance, where THREE readers shared one answer until the render join, group's third, was replaced by a rule over the commas the writer typed (decisions/R1, 2026-09-06); it stays where it is, one call site being no reason to move a predicate that two sites will contest again. `trailing_titles` was that last shape for one day (2026-09-08, the #316/#489 bundle, rules/H5), and since the /simplify round of 2026-09-09 the SHARED predicate is `tail_reading` instead — the peel-and-chain fixed point that answers where the name pieces end (decisions/H5). Assign calls it at its main walk and group's bound-given reserve calls it twice, once per view the join compares, because that reserve reads the name words assign will leave and this walk is half of what leaves them (rules/P5; counting a trailing title word among them joined 'Prof. abdul rahman Prof.' where 'Prof. abdul rahman' does not). Since #535 group's maiden walk calls it as well, over the clause and over the view its take would leave, wherever a trailing rule reads the clause (rules/M2), for the same reason: the walk's stops must end the clause where assign's reading of the name will begin. `peel_trailing` and `trailing_titles` are what that fixed point is BUILT from, and neither is a two-stage question any longer: `peel_trailing` has no caller outside `_pieces.py`: `trailing_start` and `tail_reading`, its two callers in the leaf, fold the numeral's half and the acronym's half into one index. It had one until #601 — the maiden walk in `_group.py`, which asked the bare peel itself one half at a time (#424, #533) and from #535 only where no trailing rule read the clause; since #601 the maiden take reads `tail_reading` and `trailing_candidates` instead (re-measured 2026-10-11 by AST call-site census). `trailing_titles` has exactly one caller outside the leaf, assign's family-comma segment-1 walk, which reads the chain without the re-peel, and `_group.py` does not import it. The tail reading is in the leaf rather than inline because each assign site had been given a cheap frame-free gate written to match the walk's own first condition, which is a second implementation of the question and was removed in review; what the leaf costs is one frame per entry point, measured, and the walk's own first test is a compiled regex rather than a call, so an ordinary name pays a match and stops. The reserve's two calls cost the reference name nothing — it never enters that branch, having no bound given word — and the parse and facade frame counts did not move (measured 2026-09-09). Re-measured 2026-09-09 by an AST call-site census over `_pipeline/*.py` — every call node whose callee is one of these names, keyed by module and enclosing function, which is what caught the census claiming a share for `peel_trailing` that the round had just taken away — the rest of it holds unchanged: is_suffix_piece, leading_titles, peel_walk and now tail_reading shared, is_leading_title, is_title_piece and trailing_start group-only (is_title_piece has since joined the shared list, above) — assign still reads `leading_titles` and never `is_leading_title`, which is what keeps H2's shape inference out of the trailing slot. And nameparser/_pipeline/_post_rules.py over a state: suffix_entries, the #511 instance, the R1 entry pass as a function, the one instance living in a stage rather than in a leaf — it is a pass over a whole ParseState and no leaf takes one, and AGENTS.md names it as the exception — run by post_rules last in the stage (through its in-place worker) and by Parser.revise over a sub-parse whose roles it has forced, so a suffix value handed to revise() derives its entries by the rule a whole name uses rather than by a second reading of the value's commas (decisions/R1, 2026-09-06 #511). Two more since #610 (2026-10-04), both in `_pieces.py`: `particle_tail`, P6's walk, asked by post_rules' attachment and by assign's given-part credential run, which had been two walks agreeing only through the run's roles -- and since #613 asked once per parse, by assign's walk -- which both leaves the tail out of the run and attaches it -- or, behind a comma with nothing before it, by post_rules after H1 and M4; and `trailing_candidates`, the maiden take's choice of clause words, written beside the peel whose admissions it must cover, with a sweep test against `tail_reading` that found two shapes the hand copy had missed (decisions/M2). And `join_connectives` since #617, P3's connective joins, which `_comma.decide` had copied as `_title_chains` and which had already parted from group's three ways (no run merge, an unconditional single-letter skip, the title kind alone): one loop now, taking from each caller what genuinely differs between them -- the carve-out's answer, a titles-only mode, the frozen set. What the fold taught is the last of those: a parameter one caller had been filling with a guard is a question the other caller must answer too. The comma part had never been asked it -- its copy joined a generational connective of more letters as it joined any other (`og`, under caller vocabulary), and never met the default one-letter `i` at all -- and the shared run merge brought `i` into reach, where the property grid caught it (decisions/P3, 2026-10-06 #617). tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. +**Problem shape.** Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. + +**Contract statement.** Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. Stage order is this mechanism's limit: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. + +**How it works.** A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Every instance below was found as a defect before it was found as a pattern. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`, both leaves the stages sit on. The piece layer got its own module only in #439; until then those predicates collected in `_group`, the one place both stages could reach, so the sharing between group and assign predates the module (decisions/parse-cost measured #439 as a pure relocation). The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other — the right price there over plumbing a state field the two sites would not otherwise share. Folding two copies into one, take from each caller only what genuinely differs between them, and ask whether a parameter one caller had been filling with a guard is a question the other caller must answer too (#617, decisions/P3). The census of which stage calls which predicate, and how each instance was found, was kept here until #759 and is in git history (`git show fe85bb3d:docs/design/mechanisms.md`); the decisions entries carry each instance's argument. + +**Lives in.** nameparser/_pipeline/_vocab.py over token text: `is_wholly_suffix` (#319, decisions/W2), `is_trailing_numeral_suffix` (#401/#421, decisions/P5), `maiden_marker_run` (#434, decisions/M2), `delimiter_cores` (#436/#437, decisions/R1), `in_initialless_script` (#322/#323, decisions/cjk-full-stops), `caps_shape_candidate` (#516, decisions/S9). nameparser/_pipeline/_pieces.py over pieces and tags: `is_suffix_piece`, `leading_titles` and `is_leading_title` (#424), `is_title_piece`, `peel_walk`, `trailing_start`, `tail_reading` (the peel-and-chain fixed point that answers where the name pieces end, decisions/H5), `segment_suffix_reading` (#430, decisions/R1), `particle_tail` and `trailing_candidates` (#610, decisions/M2), `join_connectives` (#617, decisions/P3). nameparser/_pipeline/_post_rules.py: `suffix_entries` (#511, decisions/R1), the one instance in a stage rather than a leaf, being a pass over a whole ParseState. tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. + +**Reach for it when.** You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. ## RENDER-HONORS-THE-PARSE — the parse decides it, the views honor it -Problem shape. A render view needs a fact the parse already settled — whether a word is the conjunction or an initial, whether a particle is acting as one, which word renders first. Contract statement. The parse decides it; the render views honor those decisions and never re-evaluate them. Two directions break that, and each has been found here as a defect: a view RE-DERIVES the answer from the text, keeping its own copy of a pipeline predicate — that one shipped through 2.0 and 2.1 — or a view honors the record and then OVERRIDES it, readmitting what the decision excluded, which was filed and open here and is CLOSED (2026-09-20, #461). How it works. Re-deriving fails because the two copies stop being the same question long before anyone notices they are two: `_cap_word` re-ran the conjunction-versus-initial decision from the word's spelling against a hand-maintained copy of the pipeline's `_INITIAL` pattern while classify had already answered it and recorded it on the token, and `_classify.py` asks `is_initial()`, the shape test ANDed with a script-repertoire test since #320, where `_render.py` asked the bare pattern, and asked it per WORD of a token's text rather than per token, so `juan e-f smith` repaired to `Juan e-F Smith` (#458). Overriding fails more directly, and this entry's instance of it is now CLOSED (see the paragraph that closes it below): `initials()` honors the `conjunction` tag through `_SKIP_TAGS` and then readmits the token whenever it also carries UNJOINED_TAG, the mark of an all-particle part, where rules/R3 excludes a conjunction "even then" — so under a caller's vocabulary that puts a word in both sets the view readmits what the rule excluded, and under `Lexicon.default().add(particles={'y'})` — the lexicon those two readings need, and they hold under no other — `parse("Anh y Van")` initialed `A. y. V.` and `parse("Juan de y")` `J. d. y.` on 2026-08-29. Read those two values scoped to that lexicon or they are simply false: under the DEFAULT vocabulary the same strings gave `A. V.` and `J.` on that date, the second being a rules/R3 example line the doc runner asserts — and BOTH of those default readings have since moved, which the closing paragraph below carries. #461 is where that stands, and it is worth reading before re-fixing it: the narrowing was written, measured and BACKED OUT in the same PR, because honoring R3 there cost `initials()` its agreement with `family_base`, which reads that same `de y` as the base — this entry's other shape, arrived at from the render side, and the reason the question moved from the code to R3's clause (its decisions entry carries the argument). What to carry away is the DIRECTION and not its verdict here: a view that honors a record and then readmits what the record excluded is overriding a decision it never took, whichever way this one settles. Case repair reads that same token and does NOT readmit it — `capitalized(force=True)` on `Anh y Van` gives `Anh y Van` under that same lexicon, R4 carrying the carve-out in its own words and, since #461, on its own reason rather than on R3's authority: a connective that initials because it joins nothing is still not written the way a name is written. Until 2026-09-20 R4's text read "being no name word in any part — the carve-out R3 states for initials", so a change to R3's clause reached R4's TEXT; #461 cut that cross-reference, and nothing now ties the two rules' wording. They were never tied in BEHAVIOR either, though an earlier wording of this sentence asserted it: over the corpus names carrying a conjunction in the GIVEN group, `parse("john and jane smith").capitalized()` kept `and` lowercase while `.initials()` gave `j. a. j. s.` before #461 (decisions/R2 carries that population). Since #461 a connective with a name word beside it contributes no initial, so that name initials `j. j. s.` and the two views agree on it, while a connective joining nothing still initials and still repairs lowercase — `parse("Juan de y")` initials `J. y.` and repairs to `Juan de y` under force — which is where they part today, by decision (rules/R3 and #R4 each say so in their own words); only the FORCED call witnesses the repair half, R5's gate refusing a mixed-case name before any of this is consulted. This is the CONSUMER-side rule over the producer-side entries — VOCAB-TAGS records what the vocabulary knew, MARK-DONT-STRIP what a stage decided about it, FOLDED_TAG what order to render in — and a view reads what they recorded, whichever kind it is. It sits where ONE-PREDICATE-PER-QUESTION's stated limit leaves off: where two live sites need one answer they share a predicate, but a render view always comes AFTER the decider, so the answer is recorded rather than shared — on the TOKEN, which is the views' equivalent of that entry's `ParseState.order`, no view being able to see a ParseState at all. Known limit, and the half most easily got wrong next: a token the parse never saw carries no decision to honor, so a view falls back to the vocabulary — the tell is `UNCLASSIFIED_TAG`, which `ParsedName.replace()` stamps when it splices raw text into a field and the facade's v1 pickle load stamps when it rebuilds a name from `*_list` strings. It is NOT untaggedness, since an ordinary parsed name word carries no tags either; and it is NOT `span is None`, which was tried and is wrong in the other direction — span-less means SYNTHETIC, and `Parser.revise()` builds span-less tokens from a full sub-parse whose tags it keeps on purpose, so the span reading overrode exactly the tags `revise()` exists to preserve (`revise(middle='e-f')` repaired to `e-F` where the parse gave `E-F`). A hand-built span-less token is unmarked and therefore classified, which is the same tag-driven default every other view applies. A view can only fall back if it is HANDED a vocabulary, and TWO are: `capitalized(lexicon=...)`, and — since #528 — the v1 facade's `HumanName.initials()`, which holds the bound `Lexicon` its `Parser` was built from and so never has to guess one. `family_base` and `family_particles` are properties on ParsedName, whose fields are original/tokens/ambiguities and nothing else, so a spliced field empties the particles view and leaves the base the whole field, with `Parser.revise()` the crossing there too (docs/usage.rst says so where it documents the degradation). The CORE `ParsedName.initials()` is the near miss and the instructive one, and it is still the near miss: it is a METHOD, so it looks like it could ask, but its signature is `(spec, delimiter, separator)` and carries no lexicon — a fallback there was written and dropped because it had to GUESS `Lexicon.default()`, and the guess erased a whole field under a caller's own vocabulary (decisions/R4). Read the two initials views apart wherever this entry says "initials", because #528 made them differ exactly here: the core cannot fall back and the facade does. `capitalized()` guesses nothing either: it reads the lexicon it was handed, and only defaults to `Lexicon.default()` when the caller passes none, which is the documented meaning of omitting the argument rather than a fallback. Within a view that can fall back, the fallback is drawn per QUESTION (rules/R4's Accepted clause). Whether a word is the conjunction or an initial is a property of the word, which a vocabulary answers alone, so case repair asks it. Whether a part is wholly particles is a property of the whole PART, which the pipeline answers once and records as UNJOINED_TAG. What `_cap_word` cannot do is RE-DERIVE that answer where no word of the part carries a tag — it is handed the whole token's tags and gates the particle conjunct on UNJOINED_TAG, so it is not blind to the part, it simply has no evidence to reconstruct one from — so repair leaves that half to plain particle treatment and rules/R4's Accepted boundary records the consequence — a spliced field is not repaired as a parsed one is, with `Parser.revise()` as the supported crossing. A fallback is right only while it answers as the pipeline would, and THAT is held by hand rather than mechanically: test_regex_sync pins the two `_INITIAL` copies to each other and to config, while the repertoire half of the pipeline's predicate (#320) is deliberately not carried across, layering forbidding the import. What that divergence can reach was argued to be nothing observable, and #528 RETRACTED that for the facade view while leaving it standing for the core. The shape it needs is a caller-added conjunction written initial-SHAPED in a script that has no initials (`太.`, `씨.`), and the old argument was that case repair is the fallback's only reader, so the two paths differ by `lower()` versus `capitalize()` over a caseless script — the same string either way. The facade's `HumanName.initials()` is a second reader now, and it CAN show the difference, because a word the fallback calls a connective contributes no initial while one it does not contributes a letter. Witness, measured 2026-09-13 under `Constants()` with `conjunctions.add("太")`: `HumanName("Wang Chen 太. Li").initials()` is "W. C. L." — classify tags the `太.` `conjunction` in the family and the view honors it — while splicing the identical family text in, `h.last = "Chen 太. Li"`, gives "W. C. 太. L.", the fallback reading `太.` as initial-shaped and admitting it. Same string in the field, two answers, and `capitalized(force=True)` shows nothing at all on either. ACCEPTED rather than repaired, on this entry's own terms: the fallback path is only ever reached for text no parse read, a spliced field is the caller's own text, and it is answered by the SAME helper R4 hands case repair — so the remedy is the documented crossing, WHICH DIFFERS BY VIEW and must not be copied across: for the parsed name's views it is `Parser.revise()` (rules/R3's Accepted clause, decisions/R3), while the facade has no `revise` at all — its setters splice through `replace()` by decision — so the v1 remedy is the v1 one, parsing the whole string again by assigning `full_name`, which restores "W. C. L." on the name above (measured 2026-09-13 with the rest of this witness). What stays true for the core is the sentence that used to close this: `ParsedName.initials()` was once the reader that could witness it and no longer falls back at all, being handed no vocabulary. A second limit, recorded rather than closed: `_cap_word`'s PARTICLE conjunct still keys on the lexicon handed to the view rather than on the `particle` tag, so a repair run with a lexicon other than the parse's re-decides a word the parse already read — a name parsed under the default vocabulary, where `parse('juan smith vega')` reads `vega` as the family, repairs to `Juan Smith vega` when `capitalized()` is handed `Lexicon.default().add(particles={'vega'})` instead — the divergence needs the two lexicons to differ, and repairing under the parse's own lexicon gives `Juan Smith Vega`. Making it read the tag moves a boundary rules/R4 states in prose, so it is a separate decision and not a cleanup (decisions/R4, "NOT DONE"). CLOSED 2026-09-20 (#461), and what closed it was the RULE rather than the code: rules/R3 no longer excludes a connective unconditionally, so the readmission this entry called an override is what the rule now says, decided once over the whole part and recorded on the token as UNJOINED_CONJUNCTION_TAG for both initials views to read. The scoped values above STAY TRUE and are re-measured on this date: under `Lexicon.default().add(particles={'y'})`, `parse("Anh y Van")` initials `A. y. V.` and `parse("Juan de y")` `J. d. y.`, exactly as on 2026-08-29. What changed is the pair of DEFAULT-vocabulary readings quoted beside them, and both of them: `parse("Juan de y")` gives `J. y.` where it gave `J.` — the rules/R3 example line moved with it — and `parse("Anh y Van")` gives `A. y. V.` where it gave `A. V.`, its middle `y` holding that part alone. On THAT string the two lexicons now agree, which is the shape of the fix: the readmission was never about a caller's overlapping vocabulary, it was about a part with nothing left to join. `Juan de y` still differs between them and legitimately so — under the default vocabulary `de` is a working particle and `y` alone is the base, while under the overlap the whole part is particles and R2's own mark readmits both words. The disagreement with case repair is not closed and is not meant to be: it is narrowed to a connective the parse placed among the NAME words with nothing to join, and it is R4's own rule now rather than an oversight or a borrowing from R3 (decisions/R4, 2026-09-20). Narrowed twice, the second time by a defect the record itself turned up (2026-09-20): where the parse read such a word as the GENERATION it also spells, repair was reading classify's `conjunction` tag without asking the role and lower-casing a suffix, so `parse("John Quincy Smith i")` forced gave 'John Quincy Smith i'. A view honors what the parse DECIDED, and the role is that decision where the tag is only what the word could have been -- the same distinction this entry draws between re-deriving an answer and reading the record, one level up. The direction this entry teaches is unchanged and is why the paragraph is closed rather than deleted: a view that honors a record and then readmits what the record excluded is overriding a decision it never took — the remedy was to settle what the record should say, not to make the view louder. Lives in. nameparser/_render.py (`capitalized`/`_cap_word`, `_reads_as_conjunction`, and `initials`, the CORE view, which honors tags and never falls back) and nameparser/_types.py (`_text_for` and `UNCLASSIFIED_TAG`, with the `ParsedName.replace()` producer beside it) and nameparser/_facade.py, which is on this list TWICE and for opposite reasons: the v1 pickle load is the SECOND producer of that mark — named here because a change that follows the list into `_types.py` alone leaves it behind, which is the site test_a_restored_pickle_keeps_v1_conjunction_repair exists to protect — and since #528 `_token_is_conjunction`/`_process_initial` are a CONSUMER, the facade's initials view reading the tag and calling `_render._reads_as_conjunction` for the mark's own tokens. All of them read what nameparser/_pipeline/ recorded — the mark those views read is recomputed producer-side in `_remarked`, which is deliberately silent about text nobody classified and is right to be. Reach for it when. A view is about to consult a Lexicon, a regex or an exception list about a word the parse already saw — or a view and a field disagree about the same parse. #408 was that second shape, and is CLOSED (2026-08-30): `initials()` walked tokens in written order where the family field applies FOLDED_TAG's ordering, so `parse("der, y van")` gave family `van der` and initials `y. d. v.` on 2026-08-29 and gives `y. v. d.` now, the view reading the tag as `_text_for` does. Worth keeping as the worked instance of the shape rather than deleting with the fix, and worth two notes on how it read once measured. The disagreement was not a judgment call anyone had taken: the FACADE already ordered folded-first through its own `*_list` views, so the core view was out of step with the field, with v1, and with the facade at once, and nothing in 6125 tests touched it. And where the change has a v1 reference at all it RESTORES rather than deviates, which is not what this entry's other instances have been -- but read that SCOPED to the population it was measured over, because an unscoped version of this sentence stood here until 2026-08-30 and overstated in both directions. Only the two DEFAULT-ORDER policies have a v1 reference: v1 had `middle_name_as_last` and no general `name_order`, so 588 of the 660 moving parses -- the two family-first orders -- restore nothing and break nothing, there being no v1 answer to come into or leave (decisions/R3 says the same and carries the rest of the measurement). Where the reference does exist the claim is exact and worth keeping: over the 1094-name corpus at the default order, 71 names move under `middle_as_family`, of which 54 return to 1.4.0's answer and none leaves it. And it is a claim about THOSE 71 rather than about every name the fix touches -- this entry's own lead example is the counterexample, `parse("der, y van")` giving `y. d. v.` before and `y. v. d.` after where 1.4.0 gives `y.`, v1 contributing nothing at all for a family that is all particles (rules/R2's territory, and a divergence decisions/R2 has already decided in favor of). A view that stopped honoring a record had been quietly reproducing a v1 bug that v1 did not have. +**Problem shape.** A render view needs a fact the parse already settled — whether a word is the conjunction or an initial, whether a particle is acting as one, which word renders first. + +**Contract statement.** The parse decides it; the render views honor those decisions and never re-evaluate them. Two directions break that: a view RE-DERIVES the answer from the text, keeping its own copy of a pipeline predicate, or a view honors the record and then OVERRIDES it, readmitting what the decision excluded. Known limit: a token the parse never saw carries no decision to honor, so a view falls back to the vocabulary — the tell is `UNCLASSIFIED_TAG`, which `ParsedName.replace()` stamps when it splices raw text into a field and the facade's v1 pickle load stamps when it rebuilds a name from `*_list` strings. + +**How it works.** This is the CONSUMER-side rule over the producer-side entries — VOCAB-TAGS records what the vocabulary knew, MARK-DONT-STRIP what a stage decided about it, FOLDED_TAG what order to render in — and a view reads what they recorded. It sits where ONE-PREDICATE-PER-QUESTION's stated limit leaves off: a render view always comes AFTER the decider, so the answer is recorded on the TOKEN, no view being able to see a ParseState. The fallback's tell is the mark and nothing else: not untaggedness, since an ordinary parsed name word carries no tags, and not `span is None`, since `Parser.revise()` builds span-less tokens whose tags it keeps on purpose. A view can fall back only if it is HANDED a vocabulary — `capitalized(lexicon=...)`, and since #528 the facade's `HumanName.initials()` — and the core `ParsedName.initials()`, handed none, never guesses one (decisions/R4). Within a view that can fall back, the fallback is drawn per QUESTION (rules/R4's Accepted clause). Both directions have shipped as defects and both are closed: re-deriving in case repair (#458, decisions/R4) and in `initials()`'s written-order walk (#408, decisions/R3), overriding in `initials()`'s readmission of an all-particle part's connective, closed by settling what rules/R3 says rather than by making the view louder (#461, decisions/R2 and decisions/R3). Two divergences stand by decision: the facade's fallback can initial a spliced field differently from a parse of the same text (decisions/R3, 2026-09-13 #528), and case repair's PARTICLE conjunct keys on the lexicon it is handed rather than the `particle` tag (decisions/R4's NOT DONE bullet). The case history this entry carried until #759 is in git history (`git show fe85bb3d:docs/design/mechanisms.md`). + +**Lives in.** nameparser/_render.py (`capitalized`/`_cap_word`, `_reads_as_conjunction`, and `initials`, the CORE view, which honors tags and never falls back), nameparser/_types.py (`_text_for` and `UNCLASSIFIED_TAG`, with the `ParsedName.replace()` producer beside it), and nameparser/_facade.py, on this list twice and for opposite reasons: the v1 pickle load is the SECOND producer of the mark — a change that follows the list into `_types.py` alone leaves it behind, which is the site test_a_restored_pickle_keeps_v1_conjunction_repair protects — and since #528 `_token_is_conjunction`/`_process_initial` are a CONSUMER, reading the tag and calling `_render._reads_as_conjunction` for the mark's own tokens. + +**Reach for it when.** A view is about to consult a Lexicon, a regex or an exception list about a word the parse already saw — or a view and a field disagree about the same parse, which is the shape #408 was (`parse("der, y van")` gave family `van der` and initials `y. d. v.`; decisions/R3). ## CLAUSE-CONTENT-OVERRULES-DELIMITER — content wins -Problem shape. A bracketed clause should be treated as something other than what its delimiter pair says. Contract statement. extract may inspect a clause's content against the lexicon and, when it matches, mask only the two delimiter spans so the inner content rejoins the main token stream for ordinary downstream parsing. How it works. "Andrew Perkins (MBA)" is not a nickname (rule S1): the parens are masked away and MBA is classified by the normal machinery — reusing the downstream path, so the delimited and bare forms cannot drift. Lives in. nameparser/_pipeline/_extract.py (_suffix_shaped and the inner-span branch). Reach for it when. About to add a second code path that duplicates what the bare form already does — #335's fix is this shape (a _maiden_marked sibling predicate). +**Problem shape.** A bracketed clause should be treated as something other than what its delimiter pair says. + +**Contract statement.** extract may inspect a clause's content against the lexicon and, when it matches, mask only the two delimiter spans so the inner content rejoins the main token stream for ordinary downstream parsing. + +**How it works.** "Andrew Perkins (MBA)" is not a nickname (rule S1): the parens are masked away and MBA is classified by the normal machinery — reusing the downstream path, so the delimited and bare forms cannot drift. + +**Lives in.** nameparser/_pipeline/_extract.py (_suffix_shaped and the inner-span branch). + +**Reach for it when.** About to add a second code path that duplicates what the bare form already does — #335's fix is this shape (a _maiden_marked sibling predicate). ## CURATED-VOCABULARY-ALTERNATION — the config already splits them -Problem shape. Two string shapes collide — the same written form means two different things — and no predicate separates them. Contract statement. A curated vocabulary that OMITS the ambiguous entries is itself the separator: membership is the license, and the per-entry vetting reasons live beside the set. How it works. GLUED_HONORIFICS is the exemplar (rule W2): 씨 peels because it can never end a name; 양 stays out because 김지양 is a given name. The set, not a regex, draws the line. Lives in. nameparser/vocab/*.toml, where each word the peel may take carries `attached`. A word refused only the glued position carries `excluded = { attached = … }` on the reading it keeps (`uv run python tools/vocab/query.py --excluded attached`), and a word refused the honorific reading in both positions, such as 君 or bare 선생, carries a word-level `excluded` (`--excluded honorific`), each with its reason. Reach for it when. Arguing that "no regex can separate X from Y" — check whether a config set already splits them by listing one side. The second half of the pattern: to USE the set in a regex, hand-copy the alternation and pin the copy against the set (tests/v2/test_regex_sync.py holds the pipeline's copies). +**Problem shape.** Two string shapes collide — the same written form means two different things — and no predicate separates them. + +**Contract statement.** A curated vocabulary that OMITS the ambiguous entries is itself the separator: membership is the license, and the per-entry vetting reasons live beside the set. + +**How it works.** GLUED_HONORIFICS is the exemplar (rule W2): 씨 peels because it can never end a name; 양 stays out because 김지양 is a given name. The set, not a regex, draws the line. + +**Lives in.** nameparser/vocab/*.toml, where each word the peel may take carries `attached`. A word refused only the glued position carries `excluded = { attached = … }` on the reading it keeps (`uv run python tools/vocab/query.py --excluded attached`), and a word refused the honorific reading in both positions, such as 君 or bare 선생, carries a word-level `excluded` (`--excluded honorific`), each with its reason. + +**Reach for it when.** Arguing that "no regex can separate X from Y" — check whether a config set already splits them by listing one side. The second half of the pattern: to USE the set in a regex, hand-copy the alternation and pin the copy against the set (tests/v2/test_regex_sync.py holds the pipeline's copies). ## RECORDED-ROSTERS — record the answer, don't re-derive it -Problem shape. A guard needs to know what the answer WAS, so it can detect the answer changing. Contract statement. Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Known limit, and it is what the contract statement does NOT cover: a roster row can hold a recorded INPUT as well as a recorded expectation, and only the expectation half is asserted. The test to apply to a recorded input is not "is it checked" but "would the consumer answer DIFFERENTLY if it were wrong". The differential ledgers' rosters were this entry's worked cases (`_CROSS_RULE_WINNERS` carried wrong recorded shapes that agreed with themselves until #497 moved them where a run measured them), and were retired with the ledgers in #664; decisions/differential-ledger keeps their measurements. How it works. The purest case is a recorded expectation whose only consumer is the check that compares it, re-derived by nothing: tools/differential/snapshot.jsonl records every corpus name's reading, and tests/v2/test_snapshot.py compares the tree against it. Lives in. tools/differential/snapshot.jsonl, tests/v2/test_facade_cases.py (_CORE_ONLY_IDS), tests/v2/test_benchmark.py (_CALL_BASELINE), tests/v2/rules_budget.json (each rule's recorded size). Reach for it when. Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. And when a roster row carries anything the check CONSUMES rather than compares, ask what measures that half. +**Problem shape.** A guard needs to know what the answer WAS, so it can detect the answer changing. + +**Contract statement.** Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Known limit, and it is what the contract statement does NOT cover: a roster row can hold a recorded INPUT as well as a recorded expectation, and only the expectation half is asserted. The test to apply to a recorded input is not "is it checked" but "would the consumer answer DIFFERENTLY if it were wrong". The differential ledgers' rosters were this entry's worked cases (`_CROSS_RULE_WINNERS` carried wrong recorded shapes that agreed with themselves until #497 moved them where a run measured them), and were retired with the ledgers in #664; their measurements are in git, at the commit decisions/differential-ledger names. + +**How it works.** The purest case is a recorded expectation whose only consumer is the check that compares it, re-derived by nothing: tools/differential/snapshot.jsonl records every corpus name's reading, and tests/v2/test_snapshot.py compares the tree against it. + +**Lives in.** tools/differential/snapshot.jsonl, tests/v2/test_facade_cases.py (_CORE_ONLY_IDS), tests/v2/test_benchmark.py (_CALL_BASELINE), tests/v2/rules_budget.json (each rule's recorded size). + +**Reach for it when.** Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. And when a roster row carries anything the check CONSUMES rather than compares, ask what measures that half. ## COMPILED-DATA-COMMITTED — compile the data at build time, hold the artifact to a fresh compile -Problem shape. Data written for people (rich, commented, one fact per word) is the wrong thing for a runtime to parse on every import, and a generated runtime form can drift from its source. Contract statement. Compile the source into a committed, generated module that the runtime imports, and test that the committed module is byte-identical to a fresh compile, failing with the command that refreshes it; run the compiler's checks over the shipped source in the same test, with warnings as errors, and give each check a mutation it must refuse. How it works. The parser imports nameparser/config/_compiled.py, frozenset literals one entry per line so a one-word change is a one-line diff; tools/vocab/build.py writes it from nameparser/vocab/*.toml; tests/v2/test_vocab.py compiles in memory and compares. The source can grow descriptive data (notes, counts, expansions) that compiles to nothing and costs the runtime nothing. Cost: a data change touches two files and needs the tool run, which the staleness test turns from a silent drift into a CI failure carrying the fix. Lives in. tools/vocab/build.py, tests/v2/test_vocab.py (decisions/vocabulary-model). Reach for it when. A data file the runtime would otherwise parse, or a hand-kept Python copy of data that has a better home — and not for a measured expectation, which is RECORDED-ROSTERS. +**Problem shape.** Data written for people (rich, commented, one fact per word) is the wrong thing for a runtime to parse on every import, and a generated runtime form can drift from its source. + +**Contract statement.** Compile the source into a committed, generated module that the runtime imports, and test that the committed module is byte-identical to a fresh compile, failing with the command that refreshes it; run the compiler's checks over the shipped source in the same test, with warnings as errors, and give each check a mutation it must refuse. -## LEDGER-RULE-SEPARATION — file order decides, fields narrow by subset +**How it works.** The parser imports nameparser/config/_compiled.py, frozenset literals one entry per line so a one-word change is a one-line diff; tools/vocab/build.py writes it from nameparser/vocab/*.toml; tests/v2/test_vocab.py compiles in memory and compares. The source can grow descriptive data (notes, counts, expansions) that compiles to nothing and costs the runtime nothing. Cost: a data change touches two files and needs the tool run, which the staleness test turns from a silent drift into a CI failure carrying the fix. -Problem shape. Two differential-ledger rules claimed overlapping names. Contract statement. Retired with the ledgers (#664): the snapshot that replaced them classifies nothing, so no rule can claim a name and no order can decide a contest. How it works. Through 2.4, each `tools/differential/expected_since_*.toml` rule carried a `name_regex` and `fields`, file order decided every contest, and wide-first pairs had to declare themselves; decisions/differential-ledger records the arc. Lives in. Nothing; git history before #664. Reach for it when. Reading a decisions entry that cites it. +**Lives in.** tools/vocab/build.py, tests/v2/test_vocab.py (decisions/vocabulary-model). + +**Reach for it when.** A data file the runtime would otherwise parse, or a hand-kept Python copy of data that has a better home — and not for a measured expectation, which is RECORDED-ROSTERS. ## CANONICAL-VOCABULARY-AT-THE-BOUNDARY — one vocabulary at the comparison -Problem shape. Two surfaces name the same concept differently (first/last vs given/family), and a matcher needs to compare across them. Contract statement. Canonicalize at the point of comparison: every compared surface's output is converted to one vocabulary before matching, and the canonical choice is the one the codebase already derives everywhere else. How it works. Teaching the matcher both vocabularies doubles every comparison silently; canonicalizing means a second surface joins without new keys. The snapshot records the facade's v1 fields under Role's names, and compare.py's 1.4.0 rows do the same, so a 1.4 reading and a 2.x one compare field by field. Lives in. tools/differential/snapshot.py (row, V1_FOR). Reach for it when. A second surface joins an existing matcher — without this, every existing comparison silently stops matching the new surface, which looks like added coverage. +**Problem shape.** Two surfaces name the same concept differently (first/last vs given/family), and a matcher needs to compare across them. + +**Contract statement.** Canonicalize at the point of comparison: every compared surface's output is converted to one vocabulary before matching, and the canonical choice is the one the codebase already derives everywhere else. + +**How it works.** Teaching the matcher both vocabularies doubles every comparison silently; canonicalizing means a second surface joins without new keys. The snapshot records the facade's v1 fields under Role's names, and compare.py's 1.4.0 rows do the same, so a 1.4 reading and a 2.x one compare field by field. + +**Lives in.** tools/differential/snapshot.py (row, V1_FOR). + +**Reach for it when.** A second surface joins an existing matcher — without this, every existing comparison silently stops matching the new surface, which looks like added coverage. ## VOCABULARY-OVERLAP-AS-PRECONDITION — assert the intersection you stand on -Problem shape. A test's input depends on two config sets intersecting (a word that is both a title and a particle), and a vocabulary edit could quietly unground it. Contract statement. Assert the intersection as a precondition and distinguish the two failure modes: word moved (pick another member; the message prints what is left) versus intersection empty (the code under test is unreachable — delete it rather than repointing the test). Lives in. tests/v2/test_parser.py (test_the_chained_emitter_is_still_reachable). Reach for it when. Any config-coupled fixture — a test input chosen because of what a vocabulary happens to contain. +**Problem shape.** A test's input depends on two config sets intersecting (a word that is both a title and a particle), and a vocabulary edit could quietly unground it. + +**Contract statement.** Assert the intersection as a precondition and distinguish the two failure modes: word moved (pick another member; the message prints what is left) versus intersection empty (the code under test is unreachable — delete it rather than repointing the test). -## CROSS-RULE-OUTCOME-PINS — pin who wins the contest +**Lives in.** tests/v2/test_parser.py (test_the_chained_emitter_is_still_reachable). -Problem shape. Which differential-ledger rule won a contested name was measured by no total, and was exactly what a reorder or a narrowing changed. Contract statement. Retired with the ledgers (#664): with nothing classifying a diff there is no contest to pin. How it works. Through 2.4, `_CROSS_RULE_WINNERS` in tests/v2/test_ledger_guards.py pinned which rule classified each contested name, and `compare.unowned_contests` refused an unpinned one (#498, #508). Lives in. Nothing; git history before #664. Reach for it when. Reading a decisions entry that cites it. +**Reach for it when.** Any config-coupled fixture — a test input chosen because of what a vocabulary happens to contain. ## VOCABULARY-FEEDS-STRUCTURE — a wordlist edit can move the comma decision -Problem shape. Editing suffix vocabulary looks field-local, but the comma-structure decision (rule C1) reads that vocabulary, so a removal can flip which segment is the family name. Contract statement. Suffix-set membership is an input to the structure decision: removing a word from the suffix vocabulary can change a suffix-comma name into a listing-form name, relocating the family. How it works. Measured in the comma-suffix arc: dropping "dr" from the suffix vocabulary flips "John Smith, Dr." from family="Smith" suffix="Dr." to family="John Smith" — three individually correct documented facts composing into a family-name loss. Lives in. nameparser/_pipeline/_comma.py (the C1 decision, since #613) reading the suffix sets through _vocab.is_wholly_suffix. Reach for it when. Editing suffix membership for any word that occurs after a comma in real data — check the structure flip, not just the field. +**Problem shape.** Editing suffix vocabulary looks field-local, but the comma-structure decision (rule C1) reads that vocabulary, so a removal can flip which segment is the family name. + +**Contract statement.** Suffix-set membership is an input to the structure decision: removing a word from the suffix vocabulary can change a suffix-comma name into a listing-form name, relocating the family. + +**How it works.** Measured in the comma-suffix arc: dropping "dr" from the suffix vocabulary flips "John Smith, Dr." from family="Smith" suffix="Dr." to family="John Smith" — three individually correct documented facts composing into a family-name loss. + +**Lives in.** nameparser/_pipeline/_comma.py (the C1 decision, since #613) reading the suffix sets through _vocab.is_wholly_suffix. + +**Reach for it when.** Editing suffix membership for any word that occurs after a comma in real data — check the structure flip, not just the field. ## WARN-AT-THE-CALLER — walk out of the library, don't count frames -Problem shape. A warning should point at the caller's code, but the entry depth varies (constructor, add(), unpickle, dataclasses.replace, the shim snapshot), so any fixed stacklevel lands on library internals for most paths. Contract statement. The warner walks the stack outward until the first frame outside the library's own modules and warns there, instead of counting frames. Lives in. nameparser/_lexicon.py (_warn_dead_entry), and the related but distinct per-read-location choice is in the #293/#354 bridge (decisions/3-0-reevaluations). First measured instance of the hazard: the delegated bytes-deprecation warnings shipped with fixed stacklevel=2 and needed the follow-up fix (0044073). Reach for it when. Adding any warning reachable through more than one public entry point. +**Problem shape.** A warning should point at the caller's code, but the entry depth varies (constructor, add(), unpickle, dataclasses.replace, the shim snapshot), so any fixed stacklevel lands on library internals for most paths. + +**Contract statement.** The warner walks the stack outward until the first frame outside the library's own modules and warns there, instead of counting frames. + +**Lives in.** nameparser/_lexicon.py (_warn_dead_entry), and the related but distinct per-read-location choice is in the #293/#354 bridge (decisions/3-0-reevaluations). First measured instance of the hazard: the delegated bytes-deprecation warnings shipped with fixed stacklevel=2 and needed the follow-up fix (0044073). + +**Reach for it when.** Adding any warning reachable through more than one public entry point. ## LEGACY-STATE-SIGNATURE — subtract legacy defaults only as a complete set -Problem shape. A retired default rides in on old pickles, but a user may have deliberately re-added one of the same entries. Contract statement. Known-dead legacy defaults are subtracted from restored state only when the state carries ALL of them — the complete pre-retirement signature — so real legacy blobs clean up silently while a deliberate single re-add survives round-trips. Lives in. nameparser/_lexicon.py / _config_shim.py (_LEGACY_DEAD_ENTRIES). Reach for it when. Retiring any default vocabulary entry that existing pickles may carry. +**Problem shape.** A retired default rides in on old pickles, but a user may have deliberately re-added one of the same entries. + +**Contract statement.** Known-dead legacy defaults are subtracted from restored state only when the state carries ALL of them — the complete pre-retirement signature — so real legacy blobs clean up silently while a deliberate single re-add survives round-trips. + +**Lives in.** nameparser/_lexicon.py / _config_shim.py (_LEGACY_DEAD_ENTRIES). + +**Reach for it when.** Retiring any default vocabulary entry that existing pickles may carry. ## AMBIGUITY-AT-THE-DECISION-SITE — emit where the branch is taken -Problem shape. An ambiguity report should fire exactly when the parse chose between live readings — no more, no less — and the choosing happens in code, not in vocabulary. Contract statement. Emit at the site that takes the branch, not where an ambiguous tag sits; when a fork's two branches are decided in different stages, EVERY deciding stage carries an emitter; and a branch that runs but changes nothing is not a decision and must not report. The decision is the site's, but a FIELD the report names is not: a later rule can still move the word, so the emitter leaves the field to assemble (`PendingAmbiguity.field_tail`), which words it from the final role (#626). Segment's report on a part past the second comma names no field at all, since that part's words can land in two (#629). How it works. PARTICLE_OR_GIVEN fires from assignment for a lone leading particle and from grouping when a title shifts it off the front — for two years only the first site emitted. And keying on +**Problem shape.** An ambiguity report should fire exactly when the parse chose between live readings — no more, no less — and the choosing happens in code, not in vocabulary. + +**Contract statement.** Emit at the site that takes the branch, not where an ambiguous tag sits; when a fork's two branches are decided in different stages, EVERY deciding stage carries an emitter; and a branch that runs but changes nothing is not a decision and must not report. The decision is the site's, but a FIELD the report names is not: a later rule can still move the word, so the emitter leaves the field to assemble (`PendingAmbiguity.field_tail`), which words it from the final role (#626). Segment's report on a part past the second comma names no field at all, since that part's words can land in two (#629). + +**How it works.** PARTICLE_OR_GIVEN fires from assignment for a lone leading particle and from grouping when a title shifts it off the front — for two years only the first site emitted. And keying on "the code got here" instead of "the outcome differed" once reported -a fork for all ambiguous particles on "Dr. Van Jr.". Lives in. The AmbiguityKind emitters across _pipeline/ (rule A1 is the observable contract). Reach for it when. Adding any ambiguous vocabulary or any new fork — count the deciding sites, then count the emitters. +a fork for all ambiguous particles on "Dr. Van Jr.". + +**Lives in.** The AmbiguityKind emitters across _pipeline/ (rule A1 is the observable contract). + +**Reach for it when.** Adding any ambiguous vocabulary or any new fork — count the deciding sites, then count the emitters. ## MAKE-WRONG-STATES-UNREPRESENTABLE — the house meta-pattern -Problem shape. A convention keeps being violated no matter how clearly it is written down. Contract statement. Convert the consistency problem into a referential-integrity problem: make the wrong state impossible to express, or mechanically checked, rather than merely documented. How it works. Three instances built this documentation system: _CORE_ONLY_IDS replaced an enumerating comment; verbatim-excerpt citations replaced paraphrase; no-boundary: markers replaced silent omission. The executable examples in the rules are the same move at document scale. Lives in. tests/v2/test_doc_citations.py, tests/v2/test_rules_doc.py, and every recorded roster. Reach for it when. Tempted to write "remember to keep X in sync with Y." +**Problem shape.** A convention keeps being violated no matter how clearly it is written down. + +**Contract statement.** Convert the consistency problem into a referential-integrity problem: make the wrong state impossible to express, or mechanically checked, rather than merely documented. + +**How it works.** Three instances built this documentation system: _CORE_ONLY_IDS replaced an enumerating comment; verbatim-excerpt citations replaced paraphrase; no-boundary: markers replaced silent omission. The executable examples in the rules are the same move at document scale. + +**Lives in.** tests/v2/test_doc_citations.py, tests/v2/test_rules_doc.py, and every recorded roster. + +**Reach for it when.** Tempted to write "remember to keep X in sync with Y." ## LOCALE-PACKS-PURE-DATA — packs configure, they never compute -Problem shape. Language-specific behavior needs a home that cannot drift from the core. Contract statement. A locale pack is pure data — a Policy patch and Lexicon ADDITIONS, add-only: a pack unions vocabulary in and can never remove a base entry — applied by parser_for; packs contain no code paths of their own, and each pack's docstring declares its deviations from the defaults. How it works. What is policy for every language (an order constant) is policy, not pack data; packs are lowercase modules exposing uppercase constants; a pack error is wrapped with the pack's code (rule D2). Pure data means a pack can be audited by reading it — and add-only means a bad DEFAULT can never be routed around by a pack, which is why #342-class fixes must change shipped vocabulary. Lives in. nameparser/locales/ (packs), nameparser/_parser.py (parser_for, the one applier). Reach for it when. A language fix wants an if-statement — make it vocabulary or policy in a pack instead. +**Problem shape.** Language-specific behavior needs a home that cannot drift from the core. + +**Contract statement.** A locale pack is pure data — a Policy patch and Lexicon ADDITIONS, add-only: a pack unions vocabulary in and can never remove a base entry — applied by parser_for; packs contain no code paths of their own, and each pack's docstring declares its deviations from the defaults. + +**How it works.** What is policy for every language (an order constant) is policy, not pack data; packs are lowercase modules exposing uppercase constants; a pack error is wrapped with the pack's code (rule D2). Pure data means a pack can be audited by reading it — and add-only means a bad DEFAULT can never be routed around by a pack, which is why #342-class fixes must change shipped vocabulary. + +**Lives in.** nameparser/locales/ (packs), nameparser/_parser.py (parser_for, the one applier). + +**Reach for it when.** A language fix wants an if-statement — make it vocabulary or policy in a pack instead. ## FACADE-CONTRACT — HumanName wraps the core; 1.4-warning-free code keeps working -Problem shape. Where does v1-compatibility behavior live, and what may it do? Contract statement. HumanName is a mutable facade over the immutable core: code that runs warning-free on 1.4 keeps working through 2.x with identical results EXCEPT release-log-classified fixes, verified for 2.0 by the differential ledger and since #664 visible against 1.4.0 through `tools/differential/compare.py --baseline 1.4.0` (the RFC's compatibility promise carries the same carve-out) — via validating setters, dirty-tracked re-parses, and pickle round-trips — and the facade never calls the v1 parsing hooks it still carries. "Warning-free" describes the code's 1.4 behavior, not a promise it STAYS warning-free: 2.x adds deliberate warnings (the multi-word-vocabulary and field-assignment ones), each release-log-classified, always with identical results. Lives in. nameparser/_facade.py; v1 import paths preserved by nameparser/parser.py and nameparser/config/. Reach for it when. A core change needs a v1-visible behavior — the facade, not the core, is where parity lives (and cite decisions/3-0-reevaluations when parity is the only reason). +**Problem shape.** Where does v1-compatibility behavior live, and what may it do? + +**Contract statement.** HumanName is a mutable facade over the immutable core: code that runs warning-free on 1.4 keeps working through 2.x with identical results EXCEPT release-log-classified fixes, verified for 2.0 by the differential ledger and since #664 visible against 1.4.0 through `tools/differential/compare.py --baseline 1.4.0` (the RFC's compatibility promise carries the same carve-out) — via validating setters, dirty-tracked re-parses, and pickle round-trips — and the facade never calls the v1 parsing hooks it still carries. "Warning-free" describes the code's 1.4 behavior, not a promise it STAYS warning-free: 2.x adds deliberate warnings (the multi-word-vocabulary and field-assignment ones), each release-log-classified, always with identical results. + +**Lives in.** nameparser/_facade.py; v1 import paths preserved by nameparser/parser.py and nameparser/config/. + +**Reach for it when.** A core change needs a v1-visible behavior — the facade, not the core, is where parity lives (and cite decisions/3-0-reevaluations when parity is the only reason). ## CONFIG-SHIM-SNAPSHOT — v1 config mutations reach the core by snapshot -Problem shape. v1 code mutates CONSTANTS at runtime; the core is immutable. Contract statement. nameparser.config re-exports mutable managers whose contents are converted to immutable core objects through a dirty-tracked snapshot: mutations mark the snapshot stale, and the next parse rebuilds it, so v1 mutation semantics survive over an immutable core. Lives in. nameparser/_config_shim.py, nameparser/config/. Reach for it when. Wiring any new v1 config surface — it must go through the snapshot, never sideways into a Lexicon. +**Problem shape.** v1 code mutates CONSTANTS at runtime; the core is immutable. + +**Contract statement.** nameparser.config re-exports mutable managers whose contents are converted to immutable core objects through a dirty-tracked snapshot: mutations mark the snapshot stale, and the next parse rebuilds it, so v1 mutation semantics survive over an immutable core. + +**Lives in.** nameparser/_config_shim.py, nameparser/config/. + +**Reach for it when.** Wiring any new v1 config surface — it must go through the snapshot, never sideways into a Lexicon. ## Verification shapes @@ -141,19 +342,19 @@ How to measure in this codebase without fooling yourself. The inert-measurement ### VERSION-TELL — know who answered -Contract statement. A subprocess that speaks for a pinned version writes its __version__ and __file__ as its first output line, and the caller aborts before comparing anything if either half disagrees with what was requested. Both halves are load-bearing: an editable install reports the tree's version (agreement proves nothing when tree and baseline share a number); a genuine wheel at the wrong version passes any path check. Lives in tools/differential/compare.py (_check_tell). +**Contract statement.** A subprocess that speaks for a pinned version writes its __version__ and __file__ as its first output line, and the caller aborts before comparing anything if either half disagrees with what was requested. Both halves are load-bearing: an editable install reports the tree's version (agreement proves nothing when tree and baseline share a number); a genuine wheel at the wrong version passes any path check. Lives in tools/differential/compare.py (_check_tell). ### GENERATED-SCRIPT-OUTSIDE-THE-WORKTREE — escape the shadow -Contract statement. A worker that must run under a pinned dependency is rendered to a temp directory outside the worktree and spawned by absolute path, so sys.path[0] contains no copy of the package and the inline pin is genuine. Sentinel substitution (@@VERSION@@), not str.format — the worker body is mostly literal braces. Lives in tools/differential/compare.py (_worker_source, _run_worker). +**Contract statement.** A worker that must run under a pinned dependency is rendered to a temp directory outside the worktree and spawned by absolute path, so sys.path[0] contains no copy of the package and the inline pin is genuine. Sentinel substitution (@@VERSION@@), not str.format — the worker body is mostly literal braces. Lives in tools/differential/compare.py (_worker_source, _run_worker). ### SENTINEL-SET-OVER-MATCH-CHECK — catching match-everything -Contract statement. A user-supplied pattern is rejected as over-matching by probing it against a small set of inputs sharing no script, vocabulary or punctuation; matching all of them means it targets no behavior family. Measured: `.`, `.+`, `\b` and `[\s\S]` all decline the empty string — the naive probe — and still match every corpus name. Lived in tools/differential/compare.py (_SENTINELS) until the ledgers it checked were retired (#664); the shape stands for any user-supplied pattern. +**Contract statement.** A user-supplied pattern is rejected as over-matching by probing it against a small set of inputs sharing no script, vocabulary or punctuation; matching all of them means it targets no behavior family. Measured: `.`, `.+`, `\b` and `[\s\S]` all decline the empty string — the naive probe — and still match every corpus name. Lived in tools/differential/compare.py (_SENTINELS) until the ledgers it checked were retired (#664); the shape stands for any user-supplied pattern. ### VOCABULARY-EXERCISES-FORKS — pin the branch, not the wordlist -Contract statement. A vocabulary set a caller configures is data; the code that reads it is what has behavior, so a case row earns its place by exercising a FORK — a branch the parser takes for one INPUT and not another — never by covering one more member. The unit is the input, not the member, because most forks are keyed on the shape of the surrounding name rather than on which word appeared. Find them by reading the sites that consult the set and enumerating the branches they take, and expect that enumeration to come back empty: `maiden_markers` is read at exactly two sites (`_classify.py`'s tagging and `_extract.py`'s `_maiden_marked`), both plain whole-token membership after `_normalize`, and every shipped member parses to one structural class in bare, bracketed and lone-bracketed shapes with the marker text normalised out — zero member forks. What earns the maiden rows is clause SHAPE: marker first against marker later, a word after the marker against none, a marker a separator divides from the name against one glued to it. Two adjacent things are NOT forks of this set and do not want rows: the substring-versus-token split is a question tools/differential/README.md's coverage recipe asks wider than the parser does, and edge-period normalization lives in `_lexicon._normalize`, applying to every token of every vocabulary, so it discriminates spellings rather than members. Only "a member borne as an ordinary name" is member-keyed (decisions/vocabulary-collisions), and it has its own rows. The reduction does NOT carry across to delimiter pairs, and the difference is the line to hold: a pair the library SHIPS is caller-visible behavior, and vocabulary a caller configures is data — deleting `("«", "»")` from the defaults reddens three tests, so cases.py's one-row-per-shipped-pair convention is load-bearing and this criterion is not permission to delete those rows. Within the pairs the code does fork, on whether open and close are the same character (N2) and again inside that class, where `WORD_INTERNAL_DELIMITERS` carves the apostrophe out: `Jane Smith"` reports UNBALANCED_DELIMITER and `Jane Smith'` reports nothing. ASCII against fullwidth is not one of them — swapping `( )` for `( )` across every corpus name holding a paren changes no field and no ambiguity kind. A row may also earn its place a second way, with no fork behind it: pinning a name a release note advertises BY NAME, which is a promise to a reader rather than a branch in the parser. It follows that "N of M entries have no test" is not a finding; it reads as one because it carries a number, and it inflates with every entry added, which is the tell that it measures the data. The finding is "this fork has no test". State the argument without present-tense counts over a drifting wordlist — see docs/design/AGENTS.md — and where a count is genuinely wanted, ship the one-liner instead: +**Contract statement.** A vocabulary set a caller configures is data; the code that reads it is what has behavior, so a case row earns its place by exercising a FORK — a branch the parser takes for one INPUT and not another — never by covering one more member. The unit is the input, not the member, because most forks are keyed on the shape of the surrounding name rather than on which word appeared. Find them by reading the sites that consult the set and enumerating the branches they take, and expect that enumeration to come back empty: `maiden_markers` is read at exactly two sites (`_classify.py`'s tagging and `_extract.py`'s `_maiden_marked`), both plain whole-token membership after `_normalize`, and every shipped member parses to one structural class in bare, bracketed and lone-bracketed shapes with the marker text normalised out — zero member forks. What earns the maiden rows is clause SHAPE: marker first against marker later, a word after the marker against none, a marker a separator divides from the name against one glued to it. Two adjacent things are NOT forks of this set and do not want rows: the substring-versus-token split is a question tools/differential/README.md's coverage recipe asks wider than the parser does, and edge-period normalization lives in `_lexicon._normalize`, applying to every token of every vocabulary, so it discriminates spellings rather than members. Only "a member borne as an ordinary name" is member-keyed (decisions/vocabulary-collisions), and it has its own rows. The reduction does NOT carry across to delimiter pairs, and the difference is the line to hold: a pair the library SHIPS is caller-visible behavior, and vocabulary a caller configures is data — deleting `("«", "»")` from the defaults reddens three tests, so cases.py's one-row-per-shipped-pair convention is load-bearing and this criterion is not permission to delete those rows. Within the pairs the code does fork, on whether open and close are the same character (N2) and again inside that class, where `WORD_INTERNAL_DELIMITERS` carves the apostrophe out: `Jane Smith"` reports UNBALANCED_DELIMITER and `Jane Smith'` reports nothing. ASCII against fullwidth is not one of them — swapping `( )` for `( )` across every corpus name holding a paren changes no field and no ambiguity kind. A row may also earn its place a second way, with no fork behind it: pinning a name a release note advertises BY NAME, which is a promise to a reader rather than a branch in the parser. It follows that "N of M entries have no test" is not a finding; it reads as one because it carries a number, and it inflates with every entry added, which is the tell that it measures the data. The finding is "this fork has no test". State the argument without present-tense counts over a drifting wordlist — see docs/design/AGENTS.md — and where a count is genuinely wanted, ship the one-liner instead: ``` uv run python -c " @@ -171,19 +372,29 @@ print(len(MAIDEN_MARKERS), 'members ->', len(cls), 'structural classes')" ### FORCE-A-DECISION-TABLE — no silent defaults on growth -Contract statement. Where adding an enum member or a file must not silently inherit a default, a local table's key set is asserted equal to the population, so growth fails the suite until someone decides — against a local table, not the constant under test. Exemplar: tests/v2/pipeline/test_vocab.py's per-script initials check; reused for the pack registry (test_registry_is_the_pack_contract: every registered pack must ship DEVIATES and its rotator list or the suite fails structurally — decisions/deviates-registry). Known gap it exposes: DEFAULT_SCRIPT_ORDERS has no such guard. +**Contract statement.** Where adding an enum member or a file must not silently inherit a default, a local table's key set is asserted equal to the population, so growth fails the suite until someone decides — against a local table, not the constant under test. Exemplar: tests/v2/pipeline/test_vocab.py's per-script initials check; reused for the pack registry (test_registry_is_the_pack_contract: every registered pack must ship DEVIATES and its rotator list or the suite fails structurally — decisions/deviates-registry). Known gap it exposes: DEFAULT_SCRIPT_ORDERS has no such guard. ### SELF-EXPIRING-GUARD — a decline keyed to a measured defect -Contract statement. A workaround keyed to a third-party library's measured defect carries a canary test that pins the defect itself, so the workaround cannot outlive its reason: when the library fixes it, the canary fails and the decline gets revisited rather than fossilizing. Exemplar: tests/v2/test_locales.py's namedivider shime canary pins 0.4.x cutting 〆木太郎 at offset 1 with confidence 1.0, guarding the adapter's 〆 decline (#303 arc); its comment says to revisit the decline, not delete the test. +**Contract statement.** A workaround keyed to a third-party library's measured defect carries a canary test that pins the defect itself, so the workaround cannot outlive its reason: when the library fixes it, the canary fails and the decline gets revisited rather than fossilizing. Exemplar: tests/v2/test_locales.py's namedivider shime canary pins 0.4.x cutting 〆木太郎 at offset 1 with confidence 1.0, guarding the adapter's 〆 decline (#303 arc); its comment says to revisit the decline, not delete the test. ### SIMULATE-EVERY-OPTION — price the option space, not the chosen option -Contract statement. Where a fix has more than one plausible shape, every candidate is measured before one is chosen — simulated in-process against the real pipeline rather than implemented — and the choice cites the resulting table. How it works. The seams already exist, so simulating is cheap: `Parser(lexicon=dataclasses.replace(Lexicon.default(), ...))` for a vocabulary question, a pytest plugin recompiling one stage for a code question, a monkeypatched predicate for a guard. What the table buys is the discovery that the candidates differ in ways argument does not reveal — #400's obvious fix, adding `abd` to the wordlist, provably changed nothing on the common three-word shape, and the two vocabulary alternatives each silently lost a credential spelling; the winner was a fifth option the wordlist framing had hidden. The counterexample is the same PR's ledger `fields`, widened to include `suffix` on one reviewer's finding without pricing what else the widening claimed: it took `{family, suffix}` from a neighbouring rule and two further shapes from UNEXPLAINED, diffs in which the rule's own subject does not move, and was reverted. One table would have shown that before the commit rather than after. Reach for it when. A review names a defect and the fix looks obvious — that is when the option space is least examined, and the obvious fix is the candidate whose cost nobody has priced. Not a mechanism with a home module; the technique IS the seams above, which is why it sits here rather than in the catalog. +**Contract statement.** Where a fix has more than one plausible shape, every candidate is measured before one is chosen — simulated in-process against the real pipeline rather than implemented — and the choice cites the resulting table. + +**How it works.** The seams already exist, so simulating is cheap: `Parser(lexicon=dataclasses.replace(Lexicon.default(), ...))` for a vocabulary question, a pytest plugin recompiling one stage for a code question, a monkeypatched predicate for a guard. What the table buys is the discovery that the candidates differ in ways argument does not reveal — #400's obvious fix, adding `abd` to the wordlist, provably changed nothing on the common three-word shape, and the two vocabulary alternatives each silently lost a credential spelling; the winner was a fifth option the wordlist framing had hidden. The counterexample is the same PR's ledger `fields`, widened to include `suffix` on one reviewer's finding without pricing what else the widening claimed: it took `{family, suffix}` from a neighbouring rule and two further shapes from UNEXPLAINED, diffs in which the rule's own subject does not move, and was reverted. One table would have shown that before the commit rather than after. + +**Reach for it when.** A review names a defect and the fix looks obvious — that is when the option space is least examined, and the obvious fix is the candidate whose cost nobody has priced. Not a mechanism with a home module; the technique IS the seams above, which is why it sits here rather than in the catalog. ### TWO-ELEMENT-GROUPS — a partition's mutation check needs something to permute -Problem shape. A test pins an ordering, a sort, a dedup or a partition, and its fixture gives one of the affected groups a single element. Contract statement. Every group a test claims to cover carries at least TWO elements, because on a one-element group a partition is the identity — the assertion holds whether the code runs or not, and the mutation that should expose it passes. How it works. This is the inert-measurement class arriving through the FIXTURE rather than through the predicate, and it hides behind a mutation check that looks conclusive: #408's `initials()` partitions three roles, and its first test carried two GIVEN tokens and one MIDDLE, and its second carried two of each and no FAMILY token at all — a group of ZERO is the identity as surely as a group of one, and looks even less like a hole. Scoping the partition to FAMILY failed that test, which read as the arms being pinned — but skipping the partition for MIDDLE alone passed the entire suite (measured 2026-08-30, in review, at 6128 passed), because `initials("{middle}")` over one token is the same string either way. The tell is that the mutation which fails is not the mutation the test's docstring promises: a per-role claim wants a per-role mutation, one arm at a time, and only then does the fixture's arity show. Reach for it when. A test asserts on the ORDER or MEMBERSHIP of a rendered group and the fixture was written to be small. +**Problem shape.** A test pins an ordering, a sort, a dedup or a partition, and its fixture gives one of the affected groups a single element. + +**Contract statement.** Every group a test claims to cover carries at least TWO elements, because on a one-element group a partition is the identity — the assertion holds whether the code runs or not, and the mutation that should expose it passes. + +**How it works.** This is the inert-measurement class arriving through the FIXTURE rather than through the predicate, and it hides behind a mutation check that looks conclusive: #408's `initials()` partitions three roles, and its first test carried two GIVEN tokens and one MIDDLE, and its second carried two of each and no FAMILY token at all — a group of ZERO is the identity as surely as a group of one, and looks even less like a hole. Scoping the partition to FAMILY failed that test, which read as the arms being pinned — but skipping the partition for MIDDLE alone passed the entire suite (measured 2026-08-30, in review, at 6128 passed), because `initials("{middle}")` over one token is the same string either way. The tell is that the mutation which fails is not the mutation the test's docstring promises: a per-role claim wants a per-role mutation, one arm at a time, and only then does the fixture's arity show. + +**Reach for it when.** A test asserts on the ORDER or MEMBERSHIP of a rendered group and the fixture was written to be small. ### Field notes — the traps themselves