From bec94c269fc818edc43e2778e6fc07f10bc57e9d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Tue, 13 Oct 2026 02:43:52 -0700 Subject: [PATCH 1/2] Remove shehu from TITLES: a Hausa given name, not a leading title (#660) `shehu` arrived with an early bulk import (1a51555f) and was never put to decisions.md#vocabulary-collisions C-i. It fails C-i where the title claim acts: `Shehu Shagari` read title Shehu, family Shagari in every release from 1.4.0 through 2.4.1, and now reads given Shehu. Three CLDR corpus names move, all toward CLDR's reading, and they leave cldr_agreement.json. The Excluded (TITLES) entry records the counts and the accepted cost (the honorific use reads as a given name). Co-Authored-By: Claude Opus 5.5 --- docs/design/decisions.md | 1 + docs/release_log.rst | 4 ++++ nameparser/config/titles.py | 1 - tools/differential/cldr_agreement.json | 33 -------------------------- tools/differential/snapshot.jsonl | 6 ++--- tools/name_usage.py | 6 ++--- 6 files changed, 11 insertions(+), 40 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index d12f1609..07c7d9d2 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -950,6 +950,7 @@ Excluded (TITLES): - cik, paron, tikin, janab (Latin spellings) — unverified, so not shipped. Latin janab follows the sri/shri pattern: Devanagari जनाब ships, and the Latin spelling needs its own check (2026-10-04, #606). - marshal — a given name (Marshal Yanda). The accepted cost is that `Field Marshal Bernard Montgomery` keeps reading title Field, given Marshal (2026-10-04, #606). - justice — a given name (Justice Smith). Likewise `Chief Justice John Roberts` keeps title Chief, given Justice (2026-10-04, #606). +- shehu (Hausa honorific) — a common Hausa given name and an Albanian and Nigerian surname, so the leading title claim took the given name of `Shehu Shagari`, a former president of Nigeria, who read title Shehu, family Shagari in every release from 1.4.0 through 2.4.1. It arrived with an early bulk import (1a51555f) and was never put to C-i; removed 2026-10-13 (#660). Counts from `tools/name_usage.py shehu` on 2026-10-13: 561 US Census surname bearers; on Wikidata 180 family-name bearers (Nigeria 20, Albania 11) and 1 given-name bearer (Nigeria), a lower bound, since Shagari's own entry has no given-name statement. Not the graf class (#salutation-titles): that class accepts a word RARELY borne in front, and Shehu leads a name as an ordinary given name. The accepted cost is the honorific use, `Shehu Abubakar` reading given Shehu as any untitled name does; it belongs in an opt-in pack if anyone asks for it. Moved three CLDR corpus names (`Shehu Muhammed`, `Muhammed, Shehu`, `Modibbo, Shehu Bello`), all three now agreeing with CLDR on every role, so they left `tools/differential/cldr_agreement.json` (moves: `uv run python tools/differential/snapshot.py --since v2.4.1`). - The trailing-position rule that must NOT be adopted, and 2026-09-08 says what the line is: TITLES holds hundreds of words in no suffix set, at least nineteen of them ordinary English surnames (king, judge, bishop, baron, sheriff, ...), so a blanket "vocabulary outranks position in the trailing slot" reading would turn "Mary Jane King" into title="King" with the family name gone. The leading half of this argument is AGENTS.md's "Dean is deliberately absent" gotcha; this is the trailing half, and it shadows the family name rather than the given (#316). What #316 settled is that the prohibition is on BARE words: "Mary Jane King" still reads family King, while "John Smith Prof." reads title Prof. **The second half of that argument is RETRACTED 2026-09-09**, in review of the docs commit: "a period-marked trailing word being one nobody writes as a surname" claimed the period keeps the collision out of the trailing slot, and it does not. Measured on the branch tree — "Mary Jane King." reads title "King.", given Mary, family Jane, the family name shadowed exactly as the bare reading would have shadowed it, and "John Smith Judge." reads title "Judge." The reach is the vocabulary rather than a handful: of the 746 titles in no suffix set, 627 read as a trailing title once a period is written behind them, and every one of the 119 that do not is kept out by the abbreviation SHAPE — a digit, a hyphen, an apostrophe, or a script whose letters carry combining marks (rules.md#H2 records that half) — no plain ASCII-letter title missing at all. So king, judge, bishop, baron and sheriff ARE reachable in the trailing slot; what the prohibition above buys is the bare spelling and nothing more. The period is a WRITING convention the rule can read, not evidence about the word, and reading it is an ACCEPTED COST under the premise that the input is a name (rules.md H Background): someone who ends a name with a period-marked word has written an abbreviation. rules.md#H5 carries it as an Accepted clause with "Mary Jane King." as the executable example. Recompute the reach with `L = Parser().lexicon; ws = [w for w in L.titles - L.suffix_acronyms - L.suffix_words if " " not in w]; len([w for w in ws if parse("Mary Jane %s." % w).title])` against `len(ws)` — 627 of 746 on 2026-09-09 — the digits move as the vocabulary grows and neither argument does. See #H5. Excluded (Policy.script_orders defaults): Script.KATAKANA is deliberately absent — a pure-katakana token is predominantly a transcribed foreign name kept in its source order, so nothing defaults on it (rule W4's boundary). Noted 2026-08-15: of the three Script-keyed axes, this is the one with no force-a-decision guard (mechanisms.md#FORCE-A-DECISION-TABLE), so a new Script member silently gets no order. diff --git a/docs/release_log.rst b/docs/release_log.rst index cc8f6620..0165857b 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -4,6 +4,10 @@ Release Log nameparser 2.5 is under development. + **Behavior Changes** + + - **Fix the Hausa given name Shehu being read as a title.** ``Shehu Shagari`` gives first ``Shehu``, last ``Shagari``, where 1.4.0 through 2.4.1 gave title ``Shehu`` and no first name. :issue:`660` + * 2.4.1 - October 13, 2026 nameparser 2.4.1 fixes two readings: an accented abbreviation written in decomposed form, and a comma inside a nickname. diff --git a/nameparser/config/titles.py b/nameparser/config/titles.py index 1430731a..8e5fd6d8 100644 --- a/nameparser/config/titles.py +++ b/nameparser/config/titles.py @@ -672,7 +672,6 @@ 'sgt', 'sgtmaj', 'sgtmajmc', - 'shehu', 'sheikh', 'sheriff', 'siddha', diff --git a/tools/differential/cldr_agreement.json b/tools/differential/cldr_agreement.json index b4fefc28..1cd6e0b6 100644 --- a/tools/differential/cldr_agreement.json +++ b/tools/differential/cldr_agreement.json @@ -3825,17 +3825,6 @@ "got": "" } }, - "ha/nativeFull/sorting": { - "given_names": { - "expected": "Shehu Bello", - "got": "Bello" - }, - "text": "Modibbo, Shehu Bello", - "title": { - "expected": "", - "got": "Shehu" - } - }, "ha/nativeFull/surnameFirst": { "given_names": { "expected": "Shehu Bello", @@ -3869,28 +3858,6 @@ }, "text": "Sule" }, - "ha/nativeGS/givenFirst": { - "given_names": { - "expected": "Shehu", - "got": "" - }, - "text": "Shehu Muhammed", - "title": { - "expected": "", - "got": "Shehu" - } - }, - "ha/nativeGS/sorting": { - "given_names": { - "expected": "Shehu", - "got": "" - }, - "text": "Muhammed, Shehu", - "title": { - "expected": "", - "got": "Shehu" - } - }, "he/foreignFull/givenFirst": { "family": { "expected": "פון ברוהל גונזלס דומינגו", diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 21c86923..1840ff9a 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -1362,7 +1362,7 @@ {"name": "Mkwawa John", "order": "FAMILY_FIRST", "given": "John", "family": "Mkwawa", "initials": "J. M."} {"name": "Mkwawa, John", "given": "John", "family": "Mkwawa", "initials": "J. M."} {"name": "Modibbo Malam Shehu Bello MP", "order": "FAMILY_FIRST", "given": "Malam", "middle": "Shehu Bello", "family": "Modibbo", "suffix": "MP", "initials": "M. S. B. M."} -{"name": "Modibbo, Shehu Bello", "title": "Shehu", "given": "Bello", "family": "Modibbo", "initials": "B. M."} +{"name": "Modibbo, Shehu Bello", "given": "Shehu", "middle": "Bello", "family": "Modibbo", "initials": "S. B. M."} {"name": "Mohamad Ahmad Ali Hassan", "given": "Mohamad", "middle": "Ahmad Ali", "family": "Hassan", "initials": "M. A. A. H."} {"name": "Mohamad Ali", "given": "Mohamad", "family": "Ali", "initials": "M. A."} {"name": "Mohamad Ali Khalil", "given": "Mohamad", "middle": "Ali", "family": "Khalil", "initials": "M. A. K."} @@ -1396,7 +1396,7 @@ {"name": "Mueller, Kathy", "given": "Kathy", "family": "Mueller", "initials": "K. M."} {"name": "Muhammad Yusuf Qodirov", "given": "Muhammad", "middle": "Yusuf", "family": "Qodirov", "initials": "M. Y. Q."} {"name": "Muhammed Shehu", "order": "FAMILY_FIRST", "given": "Shehu", "family": "Muhammed", "initials": "S. M."} -{"name": "Muhammed, Shehu", "title": "Shehu", "family": "Muhammed", "initials": "M."} +{"name": "Muhammed, Shehu", "given": "Shehu", "family": "Muhammed", "initials": "S. M."} {"name": "Mujanović Razija", "order": "FAMILY_FIRST", "given": "Razija", "family": "Mujanović", "initials": "R. M."} {"name": "Muller Kathe", "order": "FAMILY_FIRST", "given": "Kathe", "family": "Muller", "initials": "K. M."} {"name": "Muller Kathi", "order": "FAMILY_FIRST", "given": "Kathi", "family": "Muller", "initials": "K. M."} @@ -1632,7 +1632,7 @@ {"name": "Señor Carlos García", "title": "Señor", "given": "Carlos", "family": "García", "initials": "C. G."} {"name": "Señora María García", "title": "Señora", "given": "María", "family": "García", "initials": "M. G."} {"name": "Shanti Gaonkar", "given": "Shanti", "family": "Gaonkar", "initials": "S. G."} -{"name": "Shehu Muhammed", "title": "Shehu", "family": "Muhammed", "initials": "M."} +{"name": "Shehu Muhammed", "given": "Shehu", "family": "Muhammed", "initials": "S. M."} {"name": "Sheik Abu Bakar", "title": "Sheik", "given": "Abu Bakar", "initials": "A. B."} {"name": "Sheik abdul Jr Smith", "title": "Sheik", "given": "abdul", "middle": "Jr", "family": "Smith", "initials": "a. J. S."} {"name": "Sheik abdul salam", "title": "Sheik", "given": "abdul salam", "initials": "a. s."} diff --git a/tools/name_usage.py b/tools/name_usage.py index dc8492ce..760ec9a4 100644 --- a/tools/name_usage.py +++ b/tools/name_usage.py @@ -6,9 +6,9 @@ if it is not borne as a name IN THE POSITION THE CLAIM ACTS ON (C-i; decisions.md#salutation-titles records the one TITLES-only exception, decided on counts). That check was often run from memory, which misses -what the data shows at once: `shehu`, a title, is also an Albanian and -Nigerian surname, with 561 bearers in the 2010 US Census. This answers -the question with counts instead. +what the data shows at once: `shehu`, a title until #660, is also an +Albanian and Nigerian surname, with 561 bearers in the 2010 US Census. +This answers the question with counts instead. uv run python tools/name_usage.py shehu greve marshal uv run python tools/name_usage.py McDonald # pass names as written From 66a948947a516922ab0f33fc4a2826bf20906e75 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Tue, 13 Oct 2026 02:45:47 -0700 Subject: [PATCH 2/2] Ground shehu's graf-class exclusion in CLDR's data, not a count (#660) Review: the entry asserted Shehu leads a name, but its only leading-position count (1 Wikidata bearer) sits below herra's ~670, which the graf class accepted. Cite the evidence that does carry it: CLDR's six Hausa renderings file Shehu as the given name. Co-Authored-By: Claude Opus 5.5 --- docs/design/decisions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 07c7d9d2..64ffdd9e 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -950,7 +950,7 @@ Excluded (TITLES): - cik, paron, tikin, janab (Latin spellings) — unverified, so not shipped. Latin janab follows the sri/shri pattern: Devanagari जनाब ships, and the Latin spelling needs its own check (2026-10-04, #606). - marshal — a given name (Marshal Yanda). The accepted cost is that `Field Marshal Bernard Montgomery` keeps reading title Field, given Marshal (2026-10-04, #606). - justice — a given name (Justice Smith). Likewise `Chief Justice John Roberts` keeps title Chief, given Justice (2026-10-04, #606). -- shehu (Hausa honorific) — a common Hausa given name and an Albanian and Nigerian surname, so the leading title claim took the given name of `Shehu Shagari`, a former president of Nigeria, who read title Shehu, family Shagari in every release from 1.4.0 through 2.4.1. It arrived with an early bulk import (1a51555f) and was never put to C-i; removed 2026-10-13 (#660). Counts from `tools/name_usage.py shehu` on 2026-10-13: 561 US Census surname bearers; on Wikidata 180 family-name bearers (Nigeria 20, Albania 11) and 1 given-name bearer (Nigeria), a lower bound, since Shagari's own entry has no given-name statement. Not the graf class (#salutation-titles): that class accepts a word RARELY borne in front, and Shehu leads a name as an ordinary given name. The accepted cost is the honorific use, `Shehu Abubakar` reading given Shehu as any untitled name does; it belongs in an opt-in pack if anyone asks for it. Moved three CLDR corpus names (`Shehu Muhammed`, `Muhammed, Shehu`, `Modibbo, Shehu Bello`), all three now agreeing with CLDR on every role, so they left `tools/differential/cldr_agreement.json` (moves: `uv run python tools/differential/snapshot.py --since v2.4.1`). +- shehu (Hausa honorific) — a common Hausa given name and an Albanian and Nigerian surname, so the leading title claim took the given name of `Shehu Shagari`, a former president of Nigeria, who read title Shehu, family Shagari in every release from 1.4.0 through 2.4.1. It arrived with an early bulk import (1a51555f) and was never put to C-i; removed 2026-10-13 (#660). Counts from `tools/name_usage.py shehu` on 2026-10-13: 561 US Census surname bearers; on Wikidata 180 family-name bearers (Nigeria 20, Albania 11) and 1 given-name bearer (Nigeria), a lower bound, since Shagari's own entry has no given-name statement. Not weighed under the graf class (#salutation-titles), and the counts above would not settle that weighing: the one leading-position figure, the single Wikidata given-name bearer, is a lower bound and sits below herra's ~670, while the 561 and 180 are family-name bearers, a position C-i does not ask about. The evidence that Shehu leads a name is not a count. It is Shagari himself, and CLDR's Hausa sample data in this tree, whose six renderings carrying the word all file it as the given name (`grep '"given": "Shehu"' tools/differential/cldr_fields.jsonl`). Whoever re-argues the word should count Hausa given-name use first. The accepted cost is the honorific use, `Shehu Abubakar` reading given Shehu as any untitled name does; it belongs in an opt-in pack if anyone asks for it. Moved three CLDR corpus names (`Shehu Muhammed`, `Muhammed, Shehu`, `Modibbo, Shehu Bello`), all three now agreeing with CLDR on every role, so they left `tools/differential/cldr_agreement.json` (moves: `uv run python tools/differential/snapshot.py --since v2.4.1`). - The trailing-position rule that must NOT be adopted, and 2026-09-08 says what the line is: TITLES holds hundreds of words in no suffix set, at least nineteen of them ordinary English surnames (king, judge, bishop, baron, sheriff, ...), so a blanket "vocabulary outranks position in the trailing slot" reading would turn "Mary Jane King" into title="King" with the family name gone. The leading half of this argument is AGENTS.md's "Dean is deliberately absent" gotcha; this is the trailing half, and it shadows the family name rather than the given (#316). What #316 settled is that the prohibition is on BARE words: "Mary Jane King" still reads family King, while "John Smith Prof." reads title Prof. **The second half of that argument is RETRACTED 2026-09-09**, in review of the docs commit: "a period-marked trailing word being one nobody writes as a surname" claimed the period keeps the collision out of the trailing slot, and it does not. Measured on the branch tree — "Mary Jane King." reads title "King.", given Mary, family Jane, the family name shadowed exactly as the bare reading would have shadowed it, and "John Smith Judge." reads title "Judge." The reach is the vocabulary rather than a handful: of the 746 titles in no suffix set, 627 read as a trailing title once a period is written behind them, and every one of the 119 that do not is kept out by the abbreviation SHAPE — a digit, a hyphen, an apostrophe, or a script whose letters carry combining marks (rules.md#H2 records that half) — no plain ASCII-letter title missing at all. So king, judge, bishop, baron and sheriff ARE reachable in the trailing slot; what the prohibition above buys is the bare spelling and nothing more. The period is a WRITING convention the rule can read, not evidence about the word, and reading it is an ACCEPTED COST under the premise that the input is a name (rules.md H Background): someone who ends a name with a period-marked word has written an abbreviation. rules.md#H5 carries it as an Accepted clause with "Mary Jane King." as the executable example. Recompute the reach with `L = Parser().lexicon; ws = [w for w in L.titles - L.suffix_acronyms - L.suffix_words if " " not in w]; len([w for w in ws if parse("Mary Jane %s." % w).title])` against `len(ws)` — 627 of 746 on 2026-09-09 — the digits move as the vocabulary grows and neither argument does. See #H5. Excluded (Policy.script_orders defaults): Script.KATAKANA is deliberately absent — a pure-katakana token is predominantly a transcribed foreign name kept in its source order, so nothing defaults on it (rule W4's boundary). Noted 2026-08-15: of the three Script-keyed axes, this is the one with no force-a-decision guard (mechanisms.md#FORCE-A-DECISION-TABLE), so a new Script member silently gets no order.