Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/design/decisions/H6-title-before-a-family-comma.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@
MEASURED 2026-10-17 against master `eb2c1fa5`, py3.11. Snapshot (`uv run python tools/differential/snapshot.py --since master`, before this change's own rows entered the corpora): 20 entries move, all one shape — a title leaves the family for the title field — and 18 of their names are held only by corpus_cldr.jsonl. Several of those are names C1 reads as a family comma though the part after it is a credential CLDR's vocabulary lacks (`hr. Hans Henrik Sven Åge Møller, MF`); the title still comes out right there, and the family stays as wrong as before. CLDR agreement: 29 fields now agree and 28 more get closer to CLDR's value, with none regressing. Merging master after #705 listed `bapak` adds one more of the same shape, `Bapak Adinata, Dwi Putro bin MP` (title 'Bapak' out of family 'bin Bapak Adinata'), two more CLDR fields agreeing and one closer. Cost (`sys.setprofile` call events, one warm-up): `Dr. Juan Q. Xavier de la Vega III` 366 → 366 (the benchmark's `_REFERENCE` row, 307, is unmoved too), `Smith, John` 161 → 161, `Smith Jones, John` 203 → 203 (an inline tag test ahead of `is_title_piece` keeps a part no title opens free of frames), `Dr. Smith, John` 200 → 201 (one `is_title_piece`; the floor's `is_suffix_piece` on 'Smith' replaces the family loop's, which no longer asks the first family piece), `Prof. Dr. Brühl, Ada` 279 → 280. The family loop does not re-ask a piece at or past the floor, so a long post-nominal run costs one frame more than on master, not one per member: `Dr. Smith Jr. PhD MD, John` 306 → 307 (310 before the /simplify round), `Dr. Smith` plus twenty `PhD`, `, John` 903 → 904 (924 before). Those two changes moved no reading on a grid of 88,740 family-comma heads (one to four words from `Dr.`, `Prof`, `St`, `Freiherr`, `Sr.`, `Ms`, `Smith`, `van`, `Vd`, `PhD`, `Jr.`, `MA`, `V`, `de`, `Rev.`, `Mc`, `née`, before `, John`) under the default and `FAMILY_FIRST` orders, comparing fields, ambiguity kinds and `initials()`.
THE PR REVIEW (2026-10-17), three decisions. (a) `gov` JOINS TITLES (Derek). `Lt. Gov. Smith, John` read title 'Lt.', family 'Gov. Smith' — a compound split across two fields, where master kept it whole in the family and the comma-less `Lt. Gov. John Smith` reads title 'Lt. Gov.' by H2's shape. The run is vocabulary-only, so the fix is the vocabulary: `gov` is the abbreviation of the listed `governor`. Counts (`uv run python tools/name_usage.py Gov`, 2026-10-17): 236 US Census 2010 surname bearers, rank 80,419, 91.53% Asian/Pacific Islander; SSA given names under 5 a year; Wikidata 3 family-name bearers, all Israeli. The claim acts only in front, so `John Gov` keeps family 'Gov', and a surname written first loses it (`Gov Sokha` → title 'Gov'), the graf-class trade (#salutation-titles) at a smaller incidence. No corpus name moved. The other unlisted second words the review found (`insp`, `sen`, `supt`, `cmdr`, `atty`, `msgr`) still split; each is a vocabulary question on its own. (b) `Dr. PhD Smith, John` KEEPS the branch reading, family 'PhD Smith' (Derek): the first piece the run leaves is family whatever it is, as piece 0 is with no title (`PhD Smith, John`, read that way by 1.4.0 too); master and 1.4.0 gave `Dr. PhD Smith, John` family 'Dr. Smith', suffix 'PhD'. Garbage input, so no row. (c) `Graf Lambsdorff, Otto` reads title 'Graf' while `Freiherr von Berg, Ed` keeps the rank, and the clause first written for it gave the 1919 legal-surname argument as the reason — which holds for every German rank alike. The reason is the particle stop: `freiherr` is the one rank in `particles`. rules/H6's Accepted clause now says so, and the vocabulary question (should `graf`, `fürst`, `prinz` and the rest read as `freiherr` does, with and without a comma) is [#709](https://github.com/derek73/python-nameparser/issues/709).

- 2026-10-18 (Derek), #710 — POINT (1) KEEPS ITS VOCABULARY-ONLY READING, AND NOW HAS A NAMED COST BEHIND IT. Closing #710 raised it again: `Xyz. Smith, John` keeps family 'Xyz. Smith', and since no surname ends in a period, title 'Xyz.' looked like a fair reading. The 2026-10-17 entry gave only the doctrine (the slot is a surname's slot, as at #H5) and the absence of need (every CLDR case is a listed title). It named no surname the shape would break. There is one: a surname can OPEN with an abbreviated word. Filipino records write Santa and Santo as `Sta.` and `Sto.`, and H2's shape (two or more letters, one trailing period) claims both. So `Sta. Maria, Juan`, which reads family 'Sta. Maria' today, would read title 'Sta.', family 'Maria'. US Census 2010 surname bearers, measured 2026-10-18 with `uv run python tools/name_usage.py "Sta. Maria"` (and likewise for each): Sta. Ana 408, Sta. Maria 376, Sto. Domingo 163, Sta. Cruz 110, each over 90% Asian/Pacific Islander. The Census strips punctuation (it looks these up as `STAMARIA`), but the unabbreviated spelling is a separate entry (`Santa Maria`, 12,375, 77% Hispanic), so these counts are the bearers recorded with the abbreviation; what the fold hides is whether they wrote its period, which is what H2's shape needs. `ste` is a particle and no title, so `Ste. Marie, Jean` never enters H6's run today; under a shape peel, point (3)'s particle stop is what would keep it whole. `sta` and `sto` are in no vocabulary set, so nothing would stop the shape on them. `Mt.` and `Ft.` have the same shape and are rare as surnames (`Mt. Pleasant` and `Ft. Smith` each under 100 Census bearers, same date). rules/H6's rationale now names the class, and `Sta. Maria, Juan` is a boundary example there.
NOT FIXED BY THIS RULE: the comma-less writing splits the same surname. `Juan Sta. Maria` reads given 'Juan', middle 'Sta.', family 'Maria' (and `Juan Sto. Domingo` likewise), so it disagrees with `Sta. Maria, Juan` on the family; `Jean Ste. Marie` reads family 'Ste. Marie' because `ste` is a particle. Listing `sta`/`sto` as particles (measured with `Lexicon.default().add(particles={"sta", "sto"})`) fixes both comma-less readings and leaves `Sta. Maria, Juan` as it is. Added that way they are never-given, so P1's fold takes a bare leading one: `Sta Juan` moves from given 'Sta', family 'Juan' to family 'Sta Juan' (bare `Sta` has 120 US Census surname bearers). Listed as ambiguous instead (adding them to `particles_ambiguous` as well), the comma-less readings are fixed the same way and `Sta Juan` keeps given 'Sta', with a `PARTICLE_OR_GIVEN` report. Neither listing has been decided.
Declined: peeling by H2's shape before a family comma, again. Reopening it would need a stop for `sta`/`sto`. The particle listing above would supply one only if the peel kept point (3)'s particle stop, and that has to be done on purpose: H2's shape at the head of a name with no comma has no such stop, so `Ste. Marie Jean` reads title 'Ste.' today.
9 changes: 6 additions & 3 deletions docs/design/rules/H6-title-before-a-family-comma.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ H6. Rationale: the listing form moves the surname to the front and
ahead of the name ends up ahead of the surname — `Dr. Smith,
John` is `Dr. John Smith` indexed. The comma names the family,
and a title is no part of it. But the slot in front of a family
comma is a surname's slot as well, so only the vocabulary claims
a word there, never H2's shape, and a word that is a particle as
well as a title opens a surname rather than ranking a person.
comma is a surname's slot as well, and a surname can open with
an abbreviated word (Sta. Maria, Sto. Domingo), so only the
vocabulary claims a word there, never H2's shape, and a word
that is a particle as well as a title opens a surname rather
than ranking a person.
A run of title words opening the part before a family comma is
the title, joining the title field in input order with any title
after the comma. The run admits a title word only as vocabulary
Expand All @@ -28,6 +30,7 @@ H6. Rationale: the listing form moves the surname to the front and
"Prof. Smith, John, PhD" → family="Smith"
"Dr., John" → family="Dr." · boundary
"Dr. King, John" → family="King" · boundary
"Sta. Maria, Juan" → family="Sta. Maria" · boundary
"MD PhD Jr., John" → family="MD" · boundary
"St Pierre, Ed" → family="St Pierre" · boundary
"Dr. St. Pierre, Ed" → family="St. Pierre" · boundary
Expand Down
1 change: 1 addition & 0 deletions tools/differential/corpus_rules.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -463,6 +463,7 @@
"Smith, de Mesnil Jean"
"Smith. John"
"St Pierre, Ed"
"Sta. Maria, Juan"
"Stariji Bruegel"
"Steven Hardman, MD, DO, DDS"
"Steven Hardman, RN - CRNA"
Expand Down
1 change: 1 addition & 0 deletions tools/differential/snapshot.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -2918,6 +2918,7 @@
{"name": "Srta. Andréia da Silva", "title": "Srta.", "given": "Andréia", "family": "da Silva", "initials": "A. S."}
{"name": "St Clair", "title": "St", "family": "Clair", "initials": "C."}
{"name": "St Pierre, Ed", "given": "Ed", "family": "St Pierre", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "E. P."}
{"name": "Sta. Maria, Juan", "given": "Juan", "family": "Sta. Maria", "initials": "J. S. M."}
{"name": "Stander Zelda Herman", "order": "FAMILY_FIRST", "given": "Zelda", "middle": "Herman", "family": "Stander", "initials": "Z. H. S."}
{"name": "Stander, Zelda Herman", "given": "Zelda", "middle": "Herman", "family": "Stander", "initials": "Z. H. S."}
{"name": "Stariji Bruegel", "given": "Stariji", "family": "Bruegel", "initials": "S. B."}
Expand Down
Loading