Trim the design docs: archive the differential ledger, tidy mechanisms.md, compact shipped measurements (#759)

docs/design/decisions/M3-marker-led-clause.md

@@ -5,7 +5,7 @@
 - 2026-08-26 #335 — the condition is a word AFTER the marker, not the marker alone. The reason is M1's: a one-word clause keeps its word because Nee is an attested surname (Irish Ní/Nee, and a Chinese romanization), and a marker with nothing after it marks nothing. Be precise about what pins it, because the obvious answer is wrong: M1's own boundary line carries the maiden-parens annotation, so it asserts the CONFIGURED reading and passes either way — built and measured 2026-08-26, a len(words) > 0 predicate still gives it maiden "Nee". What the loosened predicate actually breaks, measured by mutation 2026-08-26, is M3's own boundary example ("Jane Smith (née)" → nickname "née") and tests/v2/cases.py::maiden_marked_clause_one_word_stays_a_nickname in both runners — three pytest failures — AND the 2.1.0 differential gate, which goes red with three unexplained names: "Jane Smith (Nee)", "Jane Smith (Nee) (Jones)" and "Jane Smith (née)". The gate is the stronger signal and the one to look at first. The middle name is why: loosened, it reads maiden "Nee" and nickname "Jones", splitting one bracket pair into the maiden field and the next into the nickname field, which is the direct contradiction of M1's "two enclosures read as one maiden name". Anyone re-litigating the second-word test should check those, not M1's line. So the real division of labour is NOT markerless versus marker-led, which is how it was first written into rules.md and had to be corrected at review: it is one word versus a marker plus a word. "Jane Smith (Nee)" is marker-led and still reads nickname "Nee" by default, maiden "Nee" only with the pair configured (both measured 2026-08-26).
 - 2026-08-26 #335 — accepted consequence: the word taken after the marker is not tested for being a name word, so M3 does not stop at a suffix word where M2's bare take does. "Jane Smith (née V)" reads maiden "V" while the bare "Jane Smith née V" reads suffix "V" and family "née" (both measured). The reason is structural rather than an oversight: M3 hands the WHOLE clause to Role.MAIDEN and the bracket is its right boundary, so the trailing-suffix rule never sees the word as trailing the name at all. Pinned as M3's Accepted line in rules.md, which puts "Jane Smith (née V)" in the differential corpus and classified in all three ledgers.
 - 2026-08-26 #335 — rules/M3 lists implemented: nameparser/_pipeline/_extract.py alone, and that is deliberate rather than an omission. M3 decides a REGION'S ROLE; the marker drop it describes ("the marker itself dropped, as M1 drops it") is M1's mechanism in _group.py, which cites M1 and is governed by it. Adding _group.py to M3's list would mean inventing an M3 citation there for behavior M1 already owns — and test_doc_citations requires implemented: to equal the set of modules that cite the rule, so the two would have to move together. Do not re-open this.
-- 2026-08-26 #335 — the durable finding, and the one worth reading before touching either side: _maiden_marked asks a WHITESPACE-SPLIT, _normalize'd question of the clause's first word, while tokenize and classify ask a token-level one. _normalize strips a trailing period but not a comma, so 'née,' is not a marker to M3 while the tokenizer splits the comma off and still tags the token vocab:maiden-marker. That divergence is the only thing keeping the "role is not Role.MAIDEN" branch of _group.group's clause-scoped drop pass reachable at all: without it, a marker-led NICKNAME clause could no longer exist, since M3 would have converted every one to MAIDEN before group ran. Anyone "unifying" the two tests would orphan that filter silently. The pin is tests/v2/cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname, and it needs BOTH of its clauses — the drop pass is gated on the name holding a maiden region at all, so a marker-glued nickname clause standing alone leaves the branch unexercised (measured by mutation, 2026-08-26). The row that used to pin the filter, maiden_marker_delimited_beside_a_nickname_clause, lost the job to M3: both its clauses are marker-led, so both are maiden now and there is no contrast left in it. SUPERSEDED IN PART 2026-10-12 by #653: a clause's comma now stays on its word, so 'née,' is no marker to the tokenizer either and the comma row no longer reaches the branch; the divergence lives on through rules/T2's nakaguro, and the pin is cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname (see #N1's #653 bullet).
+- 2026-08-26 #335 — the durable finding, and the one worth reading before touching either side: _maiden_marked asks a WHITESPACE-SPLIT, _normalize'd question of the clause's first word, while tokenize and classify ask a token-level one. _normalize strips a trailing period but not a comma, so 'née,' is not a marker to M3 while the tokenizer splits the comma off and still tags the token vocab:maiden-marker. That divergence is the only thing keeping the "role is not Role.MAIDEN" branch of _group.group's clause-scoped drop pass reachable at all: without it, a marker-led NICKNAME clause could no longer exist, since M3 would have converted every one to MAIDEN before group ran. Anyone "unifying" the two tests would orphan that filter silently. Its pin was tests/v2/cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname, with a maiden clause beside the nickname one because the drop pass is gated on the name holding a maiden region at all. The row that used to pin the filter, maiden_marker_delimited_beside_a_nickname_clause, lost the job to M3: both its clauses are marker-led, so both are maiden now and there is no contrast left in it. SUPERSEDED IN PART 2026-10-12 by #653: a clause's comma now stays on its word, so 'née,' is no marker to the tokenizer either and the comma row no longer reaches the branch; the divergence lives on through rules/T2's nakaguro, and the pin is cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname (see #N1's #653 bullet).
 - 2026-08-26 #335 — M3 makes the marker vocabulary act in a THIRD position. M2's claim acts on the word after a name word; M1's on the content of a configured pair; M3's now on the first word of any bracketed clause, under the default policy. #vocabulary-collisions C-i is keyed on the position a claim acts on, so extending where a rule acts re-asks C-i of the whole set it reads, and nothing in the process prompts that — the rule was designed, reviewed and approved without the set being re-examined. Review afterwards found one entry that fails C-i, the Czech abbreviation roz, and its disposition is the Excluded (MAIDEN_MARKERS) block in decisions/vocabulary-exclusions, which is keyed to the vocabulary set rather than to any rule: it was removed. Two things worth separating, since the finding surfaced during this work and is easy to file under it — the defect was M2's and predates #335 (bare "Rosalind Roz Smith" lost its family name on master), and M3's own boundary already excluded the common one-word "(Roz)" spelling. What belongs to M3 is only the widening, and the lesson is the general one: re-ask C-i of a vocabulary set whenever a rule extends where its claim acts.
 - 2026-08-26 #335 — what configuring maiden_delimiters still buys, since M3 removes the commonest reason to reach for it: markerless clauses, and one-word clauses. "Cherice J. (Johnson) Williams" is a real US convention and a corpus name, and nothing in that clause says maiden, so it stays a nickname by default — only a caller who knows their data can say otherwise. The knob is a declaration about the DATA; M3 is a reading of what a clause declares about itself.
 

docs/design/decisions/P3-connectives.md

@@ -5,7 +5,7 @@
 - 2026-08-21 #418 (PR #420) — the three-word carve-out counts the name that remains once the maiden name leaves. The count was taken while the marker and the maiden name were still pieces, so `juan y garcia nee jones` counted five, 'y' joined, and the family was empty once the clause left — given 'juan y garcia', family ''. Nine corpus names show it under an appended clause. Closed by order, not by an exclusion: the marker pass now runs before the count (decisions/M2, 2026-08-21), and the three-piece gate ahead of the count, which had the same exposure, is closed the same way. rules/P3 now says the count is of the name's own words. The question #397 and #411 left open — whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another — is untouched: this was never a membership question, the words are simply not in the name.
 
 - 2026-09-20 #397 — DECIDED AND SHIPPED: `i` joins `conjunctions` and `conjunctions_ambiguous` together, which is what decisions/P8's 2026-09-13 #383 + #479 entry said would happen and which half of it the conjunctions Open block in decisions/vocabulary-exclusions called contested. `parse("Josep Carod i Rovira")` gives family "Carod i Rovira" where every release from 1.4.0 through 2.3.0 gave middle "Carod i" with family "Rovira"; `parse("Josep Lluis Carod i Rovira")` gives middle "Lluis" with that same family; and `parse("Carod i Rovira, Josep")` gives family "Carod i Rovira" where they read family "Carod Rovira" and took the link into `suffix` as a generation marker (all three measured 2026-09-20 on the released 1.4.0, 2.0.0, 2.1.0, 2.2.0 and 2.3.0 wheels from a throwaway environment, this tree shadowing a wheel otherwise).
-  THE OFF SWITCH is `Lexicon.default().remove(conjunctions={"i"})`, and on the v1 surface deleting `i` from `C.conjunctions`; both restore all three readings above byte for byte, measured 2026-09-20. That is a PROPERTY rather than an anecdote, and it is this change's durable statement, and it takes THREE exemptions rather than two once case repair is included — the earlier wording of this sentence named two and the release note named none. A name's FIELDS equal its reading with the letter out of the vocabulary unless (1) a link joined between two name words or (2) the two readings disagree about the letter being an initial; its REPAIR, plain and forced, equals the off switch's unless one of those two holds or (3) the parse placed the letter among the NAME words as a connective, where R4 keeps it lower-case and the off switch — for which it is no connective at all — title-cases it: `parse("Dr. John i Smith").capitalized(force=True)` keeps 'i' against the off switch's 'Dr. John I Smith', and `Carod i Rovira` and `Josep i Rovira` are the same shape. Measured 2026-09-20 over the shipped grid's 28,448 class-letter rows: 1,462 take the join exemption, 9,296 the initial-reading one, and 2,228 more reach the third, which is INV7's alone. `tests/v2/test_properties.py::test_a_link_that_joins_nothing_changes_no_field` runs it over a 16,576-text grid paired with the configurations that can read each text differently — 66,752 rows, of which 28,448 carry a class letter and are parsed twice — and carries the recorded control: 1,480 failing parses at `c8550b64`, the commit the first review was written against, 154 at `dc3bdf9c`, and 0 here. (Quoted over the full cross product of texts × lexicons × policies the earlier numbers were 2,360 and a 12,880-name grid times eight parsers; the shipped grid is trimmed by SHAPE and grew again with the second review, so the test's own docstring is the live figure and this is a dated snapshot.)
+  THE OFF SWITCH is `Lexicon.default().remove(conjunctions={"i"})`, and on the v1 surface deleting `i` from `C.conjunctions`; both restore all three readings above byte for byte, measured 2026-09-20. That is a PROPERTY rather than an anecdote, and it is this change's durable statement, and it takes THREE exemptions rather than two once case repair is included — the earlier wording of this sentence named two and the release note named none. A name's FIELDS equal its reading with the letter out of the vocabulary unless (1) a link joined between two name words or (2) the two readings disagree about the letter being an initial; its REPAIR, plain and forced, equals the off switch's unless one of those two holds or (3) the parse placed the letter among the NAME words as a connective, where R4 keeps it lower-case and the off switch — for which it is no connective at all — title-cases it: `parse("Dr. John i Smith").capitalized(force=True)` keeps 'i' against the off switch's 'Dr. John I Smith', and `Carod i Rovira` and `Josep i Rovira` are the same shape. Measured 2026-09-20 (PR #536) over the shipped grid's 28,448 class-letter rows: 1,462 take the join exemption, 9,296 the initial-reading one and 2,228 more the third, which is INV7's alone; `tests/v2/test_properties.py::test_a_link_that_joins_nothing_changes_no_field` holds it, its recorded control 1,480 failing parses at `c8550b64`, 154 at `dc3bdf9c` and 0 here, and its docstring is the live figure.
   THE COUNT REACHES THE CLASS (Derek, 2026-09-20), which ANSWERS the C-ii question the 2026-08-21 #418 bullet above left open — "whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another". It does not: a single-letter connective counts as a name word toward P3's three-word carve-out whatever else the vocabulary says it is, and `i` is also the roman numeral. The alternative was rejected on its own arithmetic rather than on taste — counting the link as the generation it also spells lets a connective raise the bar for ITS OWN join, so "Josep Carod i Rovira", the shortest linked name and the commonest one, would be the only shape that failed to link while the four-word spellings joined. The arithmetic runs the other way too, which is why the count is scoped to where the word is PLACED TO JOIN: a connective joining nothing would otherwise LOWER the bar for an unrelated one, and "Carod y Rovira i" is that shape — it counted the trailing generation and let the "y" join. It reads given "Carod", middle "y", family "Rovira", suffix "i" here, the same FIELDS the parent gave — its initials moved, but under R3's rule rather than this one.
   ACCEPTED CONSEQUENCE OF REACHING THE CLASS, and the one row the design's reading tables show it touching: under `Lexicon.default().add(particles={"y"})`, `Juan Velasquez y Garcia` goes from given "Juan" / middle "Velasquez" / family "y Garcia" to given "Juan" / family "Velasquez y Garcia" — the custom lexicon now AGREES with the default one, which is the right direction — and its `family_base` becomes "Velasquez Garcia", a pre-existing R2 limit (`family_base` drops a particle wherever it stands, not only leading) surfaced rather than created here. Measured 2026-09-20 on this branch.
   WIDER THAN ONE ROW, and the second review measured how much: the design's reading tables show one name because they hold one of this shape, not because one moves. Over the review's 43,925-name generated grid, `add(particles={"y"})` moves a word the parent roled SUFFIX into a name part on 130 names BEYOND what the default lexicon already moves there — "ABDUL Y ROVIRA MA" goes from given "ABDUL" / family "Y ROVIRA" / suffix "MA" to given "ABDUL Y ROVIRA" / family "MA" — and 71 of those 130 still report `suffix-or-name` on the moved word, so the reading is marked rather than silent. Four more lose a title, all four the same name in its four spellings ("Jr. y Soler Puig MA" and its casings). The direction is the one the row above describes: the custom lexicon converges on the default one, the particle joining what the connective already joined. Measured 2026-09-20.