diff --git a/AGENTS.md b/AGENTS.md index 9a1ce4a3..9ceda453 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -261,7 +261,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_ **Adding a flag-gated post-parse transform** (reorder/adjust) — add a `Constants` boolean (default `False`), implement a `handle_*()` method, and call it in `post_process()` after `handle_firstnames()` and before `handle_capitalization()`, gated on the flag. Default-off keeps existing parses byte-for-byte unchanged. Two shipped examples: `patronymic_name_order` gates both `handle_east_slavic_patronymic_name_order()` (#85) and `handle_turkic_patronymic_name_order()` (#185) — one flag driving two independent handlers, added in the same `post_process()` slot; `middle_name_as_last` gates `handle_middle_name_as_last()` (#133), which folds `middle_list` into `last_list`. -**Validating a new parsing rule** — before implementing, simulate it in a throwaway script against `TEST_NAMES` (plus a few target-language examples) to catch regressions/false-positives early. E.g. this surfaced the `last_base` `do`/`st`/`mc` empties and the patronymic `"David Michael Abramovich"` false-reorder. After the change, `tests/v2/test_snapshot.py` lists every corpus name it moved, grouped by changed field -- read them, and re-record with `uv run python tools/differential/snapshot.py --record` once every move is intended, so the PR diff of `snapshot.jsonl` shows the reviewer the same list. `uv run python tools/differential/cldr_agreement.py` reports how the tree's agreement with CLDR's stated name fields moved; `tests/v2/test_cldr_agreement.py` fails on any move, and `tools/differential/cldr_agreement.json` doubles as a measured list of names the parser and CLDR read differently. +**Validating a new parsing rule** — before implementing, simulate it in a throwaway script against `TEST_NAMES` (plus a few target-language examples) to catch regressions/false-positives early. E.g. this surfaced the `last_base` `do`/`st`/`mc` empties and the patronymic `"David Michael Abramovich"` false-reorder. After the change, `tests/v2/test_snapshot.py` lists every corpus name it moved, grouped by changed field -- read them, and re-record with `uv run python tools/differential/snapshot.py --record` once every move is intended, so the PR diff of `snapshot.jsonl` shows the reviewer the same list. `uv run python tools/differential/cldr_agreement.py` reports how the tree's agreement with CLDR's stated name fields moved; `tests/v2/test_cldr_agreement.py` fails on any move, and `tools/differential/cldr_agreement.json` doubles as a measured list of names the parser and CLDR read differently. A locale-pack change reads the second view the same command prints, per pack (`cldr_agreement_locales.json`, #672); its `JA` rows run only with the `ja` extra installed. ## Gotchas @@ -331,7 +331,7 @@ Don't use the bare `python3 -m doctest .rst` CLI (no `optionflags`) to che A parse-behavior test can still be a disguised constant-content test if it only exercises one or two hardcoded entries from a large list (e.g. guarding a specific missing-comma typo in `SUFFIX_ACRONYMS`) — it documents that one instance without catching the same bug class elsewhere in the list. Prefer a test on the structural property itself, or skip a dedicated test for narrow, unrepeatable typo fixes. -**A guard that decides its own scope can stop guarding without failing.** Two shapes hit in #329, both in live code. `_POLICY_SHAPES` in `test_benchmark.py` had one entry, and pytest turns an **empty parametrize into a skip, not a failure**, so deleting it would have dissolved the quadratic guard into the skip count. And `test_facade_cases.py::_constants_for` decides which case rows it can express: its gate had been structurally unsatisfiable since the maiden-wins canonicalization landed, silently skipping seven rows, while an unenumerated `Policy` field would instead let a row run MISTRANSLATED. Neither could fail, because a skip is indistinguishable from "correctly declined". Assert the table is non-empty, give each shape a reachability probe, and tie any hand-written field enumeration to `dataclasses.fields(...)` the way `test_policy.py`'s `test_policy_patch_mirrors_policy_field_names` does. **A component that filters its own input needs a test on the filter, not only on what survives it.** +**A guard that decides its own scope can stop guarding without failing.** Two shapes hit in #329, both in live code. `_POLICY_SHAPES` in `test_benchmark.py` had one entry, and pytest turns an **empty parametrize into a skip, not a failure**, so deleting it would have dissolved the quadratic guard into the skip count. And `test_facade_cases.py::_constants_for` decides which case rows it can express: its gate had been structurally unsatisfiable since the maiden-wins canonicalization landed, silently skipping seven rows, while an unenumerated `Policy` field would instead let a row run MISTRANSLATED. Neither could fail, because a skip is indistinguishable from "correctly declined". Assert the table is non-empty, give each shape a reachability probe, and tie any hand-written field enumeration to `dataclasses.fields(...)` the way `test_policy.py`'s `test_policy_patch_mirrors_policy_field_names` does. **A component that filters its own input needs a test on the filter, not only on what survives it.** The same holds for a record split by scope: a check that reads each scope's share of a record never sees an entry that falls in no scope, so test that every entry has one — #672's per-pack CLDR ratchet missed record entries for a file dropped from `LOCALE_FILES`, and its re-record carried them over as if their pack were merely unmeasured. **Deleting a test module: sort its tests by the code they exercise, not by their names.** A module named for a retired tool also holds tests of code that survives it. #664 deleted `test_ledger_guards.py` and rewrote `test_differential.py`, triaging both by test name, and review found five tests guarding live code in the discard pile: four of `build_issues_corpus.py`'s harvester, and the check that `compare.py` reports the entries it skips. For each test, grep its module-level handles (`load_tool("...")`, imports) and ask whether that code is also being deleted. diff --git a/tests/v2/test_cldr_agreement.py b/tests/v2/test_cldr_agreement.py index c38b4e06..2e963655 100644 --- a/tests/v2/test_cldr_agreement.py +++ b/tests/v2/test_cldr_agreement.py @@ -18,7 +18,7 @@ import pytest -from nameparser import parse +from nameparser import locales, parse _TOOLS = Path(__file__).parents[2] / "tools" / "differential" @@ -206,3 +206,72 @@ def test_an_unspaced_role_is_expected_in_text_order() -> None: {"given": "セシリア", "given2": "ローズ", "surname": "ブラウン"}, "ブラウン・ローズ・セシリア") assert agreement.same(roles["given_names"], "ローズ セシリア", "ブラウン・ローズ・セシリア") + + +# -- the locale view (#672) -------------------------------------------------- + +def test_every_locale_pack_is_measured_or_listed_as_having_no_cldr_file() -> None: + # A new pack must add its CLDR files to LOCALE_FILES or say it has + # none; otherwise it goes unmeasured by omission. + mapped = set(agreement.LOCALE_FILES.values()) + assert not mapped & agreement.NO_CLDR + assert mapped | agreement.NO_CLDR == {code.upper() for code in locales.available()} + + +def test_every_mapped_cldr_file_has_renderings() -> None: + # A misspelled file in the table would measure nothing and pass. + files = {row["file"] for row in agreement.load_rows()} + assert set(agreement.LOCALE_FILES) <= files + + +def test_the_locale_view_parses_with_the_pack() -> None: + # A view that fell back to the default parser would record the + # default's disagreements and still pass its own ratchet. 王小明 is + # one unspaced Han token: only ZH segments it. + row = {"file": "zh", "label": "nativeG", "order": "surnameFirst", + "text": "王小明", "shape": 4, "fields": {"surname": "王", "given": "小明"}} + assert agreement.disagreements([row], locale=True) == {} + assert agreement.disagreements([row]) != {} + + +def test_a_pack_parser_keeps_the_family_first_order_of_its_shape() -> None: + # Latin text: no script decides the order, so only the FAMILY_FIRST + # base the pack is applied over can read Ivanov as the family. + name = agreement.parse_rendering("Ivanov Ivan", 4, "RU") + assert (name.family, name.given) == ("Ivanov", "Ivan") + + +def test_a_recorded_entry_whose_file_left_the_table_is_reported_gone() -> None: + # No pack's check reads such an entry, so without this it would sit + # in the record unguarded -- and survive every re-record. + [message] = agreement.unmapped({"zh/nativeG/givenFirst": {}, "xx/nativeG/sorting": {}}) + assert message.startswith("gone: xx/nativeG/sorting") + assert agreement.unmapped(agreement.read_record(agreement.LOCALE_RECORD)) == [] + + +def test_a_re_record_drops_entries_whose_file_left_the_table( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # The carry-over that keeps an unmeasured pack's entries must not + # keep an unmapped file's too. + record = tmp_path / "locales.json" + agreement.write_record({"xx/nativeG/sorting": {"text": "X"}}, record) + monkeypatch.setattr(agreement, "LOCALE_RECORD", record) + monkeypatch.setattr(agreement, "RECORD", tmp_path / "default.json") + assert agreement.main(["--record"]) == 0 + assert "xx/nativeG/sorting" not in agreement.read_record(record) + + +@pytest.mark.parametrize("pack", [ + pytest.param(pack, marks=pytest.mark.skipif( + not agreement.pack_available(pack), + reason=f"needs {agreement._EXTRAS.get(pack)}; CI's ja-extra job runs it")) + for pack in sorted(set(agreement.LOCALE_FILES.values()))]) +def test_each_pack_still_agrees_with_cldr_where_it_did(pack: str) -> None: + problems = agreement.locale_changes( + agreement.load_rows(), pack, agreement.read_record(agreement.LOCALE_RECORD)) + assert not problems, "\n".join([ + *problems[:40], + f"{len(problems)} change(s) against " + f"tools/differential/cldr_agreement_locales.json for {pack}.", + f"If every one is intended, re-record with `{agreement.RECORD_COMMAND}` " + "and say why in the commit."]) diff --git a/tools/differential/README.md b/tools/differential/README.md index e1fddbba..f6e353b6 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -397,4 +397,8 @@ data can. and say why in the commit. Run it without `--record` for per-role agreement counts. Nothing is special-cased: an unnatural rendering or a modelling difference (CLDR's `さん` title is a suffix here) simply stays a recorded disagreement, which makes the record a measured worklist as well as a guard. +A second view measures the locale packs (#672). `LOCALE_FILES` in `cldr_agreement.py` maps each CLDR file to the pack for its language (`zh`, `zh_Hant`, `yue`, `yue_Hans` → `ZH`; `ja` → `JA`; `ru` → `RU`; `tr`, `az` → `TR_AZ`); those renderings are parsed with `parser_for(pack)`, over the same `FAMILY_FIRST` base where CLDR's order is surnameFirst, and their disagreements go to `cldr_agreement_locales.json`, ratcheted per pack by the same test. The summary prints, per pack, how many of its renderings agree in every role under the default parser and under the pack, so a pack PR shows what it moved on its own language. A new pack adds its files to `LOCALE_FILES`, or its name to `NO_CLDR`; the test fails on a pack in neither. The modelling differences stay recorded here too: a pack is not meant to agree with CLDR where the models differ by design. + +`--record` rewrites both records. `JA` needs the `ja` extra (`namedivider-python`), which only CI's `ja-extra` job installs: without it the `JA` ratchet skips, the summary says `not measured`, and `--record` keeps the recorded `ja` entries unchanged. Re-record with the extra installed (`uv sync --group dev --extra ja`) when a change moves Japanese. + `_CORPUS_NAMES` in `tests/v2/_differential_fixtures.py` leaves the file out (2026-10-12): the tests reading it carry counts measured over the names the other corpora held. `differential_corpus()` in `tests/v2/conftest.py`, which feeds the property tests, includes it since #652, so the decomposed-twin test walks its names. diff --git a/tools/differential/cldr_agreement.py b/tools/differential/cldr_agreement.py index 1617a226..ce81531c 100644 --- a/tools/differential/cldr_agreement.py +++ b/tools/differential/cldr_agreement.py @@ -17,6 +17,14 @@ uv run python tools/differential/cldr_agreement.py # summary uv run python tools/differential/cldr_agreement.py --record # re-record +A second view does the same with each locale pack (#672): the renderings +of every CLDR file in LOCALE_FILES are parsed with parser_for(pack), and +their disagreements are recorded in cldr_agreement_locales.json and +ratcheted the same way, so a pack change shows what it moved on its own +language. --record rewrites both records. A pack needing an optional +extra that is not installed (JA's namedivider) is not measured, and its +recorded entries are carried over unchanged. + Re-record deliberately, and say why in the commit, as with a watched diff. The record doubles as a measured worklist: every entry is a place the parser and CLDR read a name differently. @@ -28,25 +36,56 @@ from __future__ import annotations import argparse +import functools import json import sys from collections import Counter +from importlib.util import find_spec from pathlib import Path from typing import Any -from nameparser import FAMILY_FIRST, ParsedName, Parser, Policy +from nameparser import FAMILY_FIRST, ParsedName, Parser, Policy, locales, parser_for ROLES = ("title", "given_names", "family", "suffix", "family_particles", "family_base") HERE = Path(__file__).resolve().parent FIELDS = HERE / "cldr_fields.jsonl" RECORD = HERE / "cldr_agreement.json" +LOCALE_RECORD = HERE / "cldr_agreement_locales.json" RECORD_COMMAND = "uv run python tools/differential/cldr_agreement.py --record" #: shapes.py's FAMILY_FIRST shape, the only tag build_cldr_corpus.py writes. FAMILY_FIRST_SHAPE = 4 _PARSERS = {None: Parser(), FAMILY_FIRST_SHAPE: Parser(policy=Policy(name_order=FAMILY_FIRST))} +#: The locale view: CLDR file -> the nameparser.locales pack its +#: renderings are parsed with. One pack per file, so the locale record is +#: keyed like the default one. A new pack adds its files here, or its +#: name to NO_CLDR; test_cldr_agreement.py fails on a pack in neither. +LOCALE_FILES = {"zh": "ZH", "zh_Hant": "ZH", "yue": "ZH", "yue_Hans": "ZH", + "ja": "JA", "ru": "RU", "tr": "TR_AZ", "az": "TR_AZ"} +#: Packs with no CLDR file to measure them on. +NO_CLDR: frozenset[str] = frozenset() +#: Packs measured only where an optional extra is installed: pack -> the +#: module the extra provides. +_EXTRAS = {"JA": "namedivider"} + + +def pack_available(pack: str) -> bool: + """Whether the locale view can measure this pack here.""" + module = _EXTRAS.get(pack) + return module is None or find_spec(module) is not None + + +@functools.cache +def _parser(pack: str | None, shape: int | None) -> Parser: + if pack is None: + return _PARSERS[shape] + if pack == "JA": + return parser_for(locales.JA, base=_PARSERS[shape], + segmenter=locales.ja_segmenter()) + return parser_for(getattr(locales, pack), base=_PARSERS[shape]) + def _has_space(text: str) -> bool: return any(ch.isspace() for ch in text) @@ -124,10 +163,11 @@ def rendering_key(row: dict[str, Any]) -> str: return f"{row['file']}/{row['label']}/{row['order']}" -def parse_rendering(text: str, shape: int | None) -> ParsedName: +def parse_rendering(text: str, shape: int | None, pack: str | None = None) -> ParsedName: """The one place a rendering is parsed, so the recorder and the - checker cannot read it two ways.""" - return _PARSERS[shape].parse(text) + checker cannot read it two ways. `pack` names a nameparser.locales + pack for the locale view; its parser keeps the shape's name order.""" + return _parser(pack, shape).parse(text) def load_rows(path: Path = FIELDS) -> list[dict[str, Any]]: @@ -135,12 +175,20 @@ def load_rows(path: Path = FIELDS) -> list[dict[str, Any]]: if line.strip()] -def disagreements(rows: list[dict[str, Any]]) -> dict[str, dict[str, Any]]: +def locale_rows(rows: list[dict[str, Any]], pack: str) -> list[dict[str, Any]]: + """The renderings the locale view parses with `pack`.""" + return [row for row in rows if LOCALE_FILES.get(row["file"]) == pack] + + +def disagreements(rows: list[dict[str, Any]], + locale: bool = False) -> dict[str, dict[str, Any]]: """Every rendering where some role disagrees, with each disagreeing - role's expected and parsed value, sorted by key.""" + role's expected and parsed value, sorted by key. With `locale`, each + row is parsed with its file's pack, so every row must be mapped.""" found: dict[str, dict[str, Any]] = {} for row in rows: - got = got_roles(parse_rendering(row["text"], row["shape"])) + pack = LOCALE_FILES[row["file"]] if locale else None + got = got_roles(parse_rendering(row["text"], row["shape"], pack)) bad = {role: {"expected": expected, "got": got[role]} for role, expected in expected_roles(row["fields"], row["text"]).items() if not same(expected, got[role], row["text"])} @@ -185,16 +233,55 @@ def changes(recorded: dict[str, dict[str, Any]], current: dict[str, dict[str, An return out +def _file_of(key: str) -> str: + return key.split("/", 1)[0] + + +def locale_changes(rows: list[dict[str, Any]], pack: str, + recorded: dict[str, dict[str, Any]]) -> list[str]: + """changes() for one pack in the locale view: its renderings against + its share of the locale record.""" + mine = locale_rows(rows, pack) + return changes({k: v for k, v in recorded.items() + if LOCALE_FILES.get(_file_of(k)) == pack}, + disagreements(mine, locale=True), + {rendering_key(row) for row in mine}) + + +def unmapped(recorded: dict[str, dict[str, Any]]) -> list[str]: + """Locale-record entries whose file is no longer in LOCALE_FILES: no + pack's check reads them, so they are reported here as gone.""" + return [f"gone: {key} is recorded but its file is not in LOCALE_FILES" + for key in sorted(recorded) if _file_of(key) not in LOCALE_FILES] + + +def _packs() -> list[str]: + return sorted(set(LOCALE_FILES.values())) + + def main(argv: list[str] | None = None) -> int: ap = argparse.ArgumentParser(description=__doc__.splitlines()[0]) ap.add_argument("--record", action="store_true", - help=f"rewrite {RECORD.name} from the working tree") + help=f"rewrite {RECORD.name} and {LOCALE_RECORD.name} " + "from the working tree") args = ap.parse_args(argv) rows = load_rows() current = disagreements(rows) if args.record: write_record(current) print(f"recorded {len(current)} disagreeing renderings to {RECORD.name}") + old = read_record(LOCALE_RECORD) if LOCALE_RECORD.exists() else {} + measured = [p for p in _packs() if pack_available(p)] + unmeasured = set(_packs()) - set(measured) + locale = {k: v for k, v in old.items() + if LOCALE_FILES.get(_file_of(k)) in unmeasured} + for pack in sorted(unmeasured): + print(f"{pack} not measured (needs {_EXTRAS[pack]}); " + "its recorded entries are kept unchanged") + locale.update(disagreements( + [r for p in measured for r in locale_rows(rows, p)], locale=True)) + write_record(dict(sorted(locale.items())), LOCALE_RECORD) + print(f"recorded {len(locale)} disagreeing renderings to {LOCALE_RECORD.name}") return 0 totals: Counter[str] = Counter() for row in rows: @@ -204,8 +291,24 @@ def main(argv: list[str] | None = None) -> int: agree = totals[role] - wrong[role] print(f"{role:17} {agree:5}/{totals[role]:<5} agree") problems = changes(read_record(), current, {rendering_key(r) for r in rows}) - print("\n".join(problems) or "matches the record") - return 1 if problems else 0 + print("\n".join(problems) or "default view matches the record") + print("\nlocale view, renderings with every role agreeing:") + recorded = read_record(LOCALE_RECORD) + locale_problems = unmapped(recorded) + for pack in _packs(): + mine = locale_rows(rows, pack) + files = ", ".join(sorted({row["file"] for row in mine})) + if not pack_available(pack): + print(f"{pack:6} {files}: not measured (needs {_EXTRAS[pack]})") + continue + found = disagreements(mine, locale=True) + before = sum(rendering_key(row) not in current for row in mine) + after = len(mine) - len(found) + print(f"{pack:6} {files}: {before}/{len(mine)} default -> " + f"{after}/{len(mine)} with the pack") + locale_problems += locale_changes(rows, pack, recorded) + print("\n".join(locale_problems) or "locale view matches the record") + return 1 if problems or locale_problems else 0 if __name__ == "__main__": diff --git a/tools/differential/cldr_agreement_locales.json b/tools/differential/cldr_agreement_locales.json new file mode 100644 index 00000000..1dc5a87e --- /dev/null +++ b/tools/differential/cldr_agreement_locales.json @@ -0,0 +1,1333 @@ +{ + "az/foreignFull/surnameFirst": { + "family": { + "expected": "van den Volf", + "got": "van" + }, + "family_base": { + "expected": "Volf", + "got": "van" + }, + "family_particles": { + "expected": "van den", + "got": "" + }, + "given_names": { + "expected": "Ada Kornelia Sezar Martin", + "got": "den Volf Ada Kornelia Sezar Martin" + }, + "text": "van den Volf Ada Kornelia Sezar Martin M.D. Ph.D." + }, + "az/foreignG/sorting": { + "family": { + "expected": "", + "got": "Sinbad" + }, + "given_names": { + "expected": "Sinbad", + "got": "" + }, + "text": "Sinbad" + }, + "az/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "Sinbad" + }, + "given_names": { + "expected": "Sinbad", + "got": "" + }, + "text": "Sinbad" + }, + "az/nativeFull/givenFirst": { + "family": { + "expected": "Aydan", + "got": "Deputat" + }, + "family_base": { + "expected": "Aydan", + "got": "Deputat" + }, + "given_names": { + "expected": "Arif Aliyev İlvus Santak", + "got": "Cənab Arif Aliyev İlvus Santak Aydan" + }, + "suffix": { + "expected": "Deputat", + "got": "" + }, + "text": "Cənab Arif Aliyev İlvus Santak Aydan Deputat", + "title": { + "expected": "Cənab", + "got": "" + } + }, + "az/nativeFull/sorting": { + "given_names": { + "expected": "Arif Aliyev İlvus Santak", + "got": "Cənab Arif Aliyev İlvus Santak Deputat" + }, + "suffix": { + "expected": "Deputat", + "got": "" + }, + "text": "Aydan Cənab Arif Aliyev İlvus Santak Deputat", + "title": { + "expected": "Cənab", + "got": "" + } + }, + "az/nativeFull/surnameFirst": { + "given_names": { + "expected": "Arif Aliyev İlvus Santak", + "got": "Arif Aliyev İlvus Santak Deputat" + }, + "suffix": { + "expected": "Deputat", + "got": "" + }, + "text": "Aydan Arif Aliyev İlvus Santak Deputat" + }, + "az/nativeG/sorting": { + "family": { + "expected": "", + "got": "Sinbad" + }, + "given_names": { + "expected": "Sinbad", + "got": "" + }, + "text": "Sinbad" + }, + "az/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "Sinbad" + }, + "given_names": { + "expected": "Sinbad", + "got": "" + }, + "text": "Sinbad" + }, + "ja/foreignFull/givenFirst": { + "family": { + "expected": "フォン スミス", + "got": "ジェニファー" + }, + "family_base": { + "expected": "スミス", + "got": "ジェニファー" + }, + "family_particles": { + "expected": "フォン", + "got": "" + }, + "given_names": { + "expected": "ジェニファー ソフィア", + "got": "ソフィア フォン スミス ジュニア博士" + }, + "suffix": { + "expected": "ジュニア", + "got": "" + }, + "text": "ジェニファー・ソフィア・フォン・スミス・ジュニア博士", + "title": { + "expected": "博士", + "got": "" + } + }, + "ja/foreignFull/sorting": { + "family": { + "expected": "フォン スミス", + "got": "フォン" + }, + "family_base": { + "expected": "スミス", + "got": "フォン" + }, + "family_particles": { + "expected": "フォン", + "got": "" + }, + "given_names": { + "expected": "ソフィア ジェニファー", + "got": "スミス ソフィア ジェニファー" + }, + "text": "フォン・スミス・ソフィア・ジェニファー" + }, + "ja/foreignFull/surnameFirst": { + "family": { + "expected": "フォン スミス", + "got": "フォン" + }, + "family_base": { + "expected": "スミス", + "got": "フォン" + }, + "family_particles": { + "expected": "フォン", + "got": "" + }, + "given_names": { + "expected": "ソフィア ジェニファー", + "got": "スミス ソフィア ジェニファー博士" + }, + "text": "フォン・スミス・ソフィア・ジェニファー博士", + "title": { + "expected": "博士", + "got": "" + } + }, + "ja/foreignG/sorting": { + "family": { + "expected": "", + "got": "マイケル" + }, + "given_names": { + "expected": "マイケル", + "got": "" + }, + "text": "マイケル" + }, + "ja/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "マイケル" + }, + "given_names": { + "expected": "マイケル", + "got": "" + }, + "text": "マイケル" + }, + "ja/nativeFull/givenFirst": { + "family": { + "expected": "佐藤", + "got": "恵子" + }, + "family_base": { + "expected": "佐藤", + "got": "恵子" + }, + "given_names": { + "expected": "恵子 グレース", + "got": "グレース佐藤ジュニア" + }, + "suffix": { + "expected": "ジュニア", + "got": "さん" + }, + "text": "恵子グレース佐藤ジュニアさん", + "title": { + "expected": "さん", + "got": "" + } + }, + "ja/nativeFull/surnameFirst": { + "suffix": { + "expected": "", + "got": "さん" + }, + "text": "佐藤グレース恵子さん", + "title": { + "expected": "さん", + "got": "" + } + }, + "ja/nativeG/givenFirst": { + "family": { + "expected": "", + "got": "慎" + }, + "given_names": { + "expected": "慎太郎", + "got": "太郎" + }, + "text": "慎太郎" + }, + "ja/nativeG/sorting": { + "family": { + "expected": "", + "got": "慎" + }, + "given_names": { + "expected": "慎太郎", + "got": "太郎" + }, + "text": "慎太郎" + }, + "ja/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "慎" + }, + "given_names": { + "expected": "慎太郎", + "got": "太郎" + }, + "text": "慎太郎" + }, + "ja/nativeGGS/givenFirst": { + "family": { + "expected": "山田", + "got": "太郎" + }, + "given_names": { + "expected": "太郎 トーマス", + "got": "トーマス山田" + }, + "text": "太郎トーマス山田" + }, + "ja/nativeGS/givenFirst": { + "family": { + "expected": "安藤", + "got": "一郎" + }, + "given_names": { + "expected": "一郎", + "got": "安藤" + }, + "text": "一郎安藤" + }, + "ru/foreignFull/givenFirst": { + "family": { + "expected": "фон Брюль", + "got": "проф." + }, + "family_base": { + "expected": "Брюль", + "got": "проф." + }, + "family_particles": { + "expected": "фон", + "got": "" + }, + "given_names": { + "expected": "Ада Корнелия Сезар Мартин", + "got": "Ада Корнелия Сезар Мартин фон Брюль — мл." + }, + "suffix": { + "expected": "мл. к. м. н.", + "got": "к. м. н." + }, + "text": "проф., д-р Ада Корнелия Сезар Мартин фон Брюль — мл., к. м. н.", + "title": { + "expected": "проф., д-р", + "got": "д-р" + } + }, + "ru/foreignFull/sorting": { + "family": { + "expected": "Брюль фон", + "got": "Брюль" + }, + "family_particles": { + "expected": "фон", + "got": "" + }, + "given_names": { + "expected": "Ада Корнелия Сезар Мартин", + "got": "Ада Корнелия Сезар Мартин фон" + }, + "text": "Брюль, Ада Корнелия Сезар Мартин фон" + }, + "ru/foreignFull/surnameFirst": { + "family": { + "expected": "фон Брюль", + "got": "фон Брюль Ада Корнелия Сезар Мартин" + }, + "family_base": { + "expected": "Брюль", + "got": "фон Брюль Ада Корнелия Сезар Мартин" + }, + "family_particles": { + "expected": "фон", + "got": "" + }, + "given_names": { + "expected": "Ада Корнелия Сезар Мартин", + "got": "к. м. н." + }, + "suffix": { + "expected": "к. м. н.", + "got": "" + }, + "text": "фон Брюль Ада Корнелия Сезар Мартин, к. м. н." + }, + "ru/foreignG/sorting": { + "family": { + "expected": "", + "got": "Синдбад" + }, + "given_names": { + "expected": "Синдбад", + "got": "" + }, + "text": "Синдбад" + }, + "ru/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "Синдбад" + }, + "given_names": { + "expected": "Синдбад", + "got": "" + }, + "text": "Синдбад" + }, + "ru/nativeG/sorting": { + "family": { + "expected": "", + "got": "Сергей" + }, + "given_names": { + "expected": "Сергей", + "got": "" + }, + "text": "Сергей" + }, + "ru/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "Сергей" + }, + "given_names": { + "expected": "Сергей", + "got": "" + }, + "text": "Сергей" + }, + "tr/foreignFull/givenFirst": { + "given_names": { + "expected": "Ada Cornelia César Martín", + "got": "DDS Ada Cornelia César Martín" + }, + "suffix": { + "expected": "MD DDS Jr", + "got": "Jr" + }, + "text": "Prof. Dr. MD DDS Ada Cornelia César Martín von Brühl Jr", + "title": { + "expected": "Prof. Dr.", + "got": "Prof. Dr. MD" + } + }, + "tr/foreignFull/surnameFirst": { + "family": { + "expected": "von Brühl Plácido Domingo", + "got": "von" + }, + "family_base": { + "expected": "Brühl Plácido Domingo", + "got": "von" + }, + "family_particles": { + "expected": "von", + "got": "" + }, + "given_names": { + "expected": "Ada Cornelia César Martín", + "got": "Brühl Plácido Domingo Ada Cornelia César Martín" + }, + "text": "Prof. Dr. von Brühl Plácido Domingo Ada Cornelia César Martín, MD DDS" + }, + "tr/foreignG/sorting": { + "family": { + "expected": "", + "got": "Sinbad" + }, + "given_names": { + "expected": "Sinbad", + "got": "" + }, + "text": "Sinbad" + }, + "tr/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "Sinbad" + }, + "given_names": { + "expected": "Sinbad", + "got": "" + }, + "text": "Sinbad" + }, + "tr/nativeFull/givenFirst": { + "suffix": { + "expected": "Dr.", + "got": "" + }, + "text": "Sn. Dr. Fatma Su Yıldırım", + "title": { + "expected": "Sn.", + "got": "Sn. Dr." + } + }, + "tr/nativeFull/surnameFirst": { + "family": { + "expected": "Yıldırım Kaya", + "got": "Yıldırım" + }, + "family_base": { + "expected": "Yıldırım Kaya", + "got": "Yıldırım" + }, + "given_names": { + "expected": "Fatma Su", + "got": "Kaya Fatma Su" + }, + "suffix": { + "expected": "Dr.", + "got": "" + }, + "text": "Sn. Yıldırım Kaya Fatma Su, Dr.", + "title": { + "expected": "Sn.", + "got": "Sn. Dr." + } + }, + "tr/nativeG/sorting": { + "family": { + "expected": "", + "got": "Zeynep" + }, + "given_names": { + "expected": "Zeynep", + "got": "" + }, + "text": "Zeynep" + }, + "tr/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "Zeynep" + }, + "given_names": { + "expected": "Zeynep", + "got": "" + }, + "text": "Zeynep" + }, + "yue/foreignFull/givenFirst": { + "family": { + "expected": "陳 陳", + "got": "小" + }, + "family_base": { + "expected": "陳", + "got": "小" + }, + "family_particles": { + "expected": "陳", + "got": "" + }, + "given_names": { + "expected": "怡君 達印", + "got": "教授 怡君 達印 陳 陳" + }, + "suffix": { + "expected": "小 博士", + "got": "博士" + }, + "text": "教授·怡君·達印·陳·陳·小，博士", + "title": { + "expected": "教授", + "got": "" + } + }, + "yue/foreignFull/sorting": { + "family": { + "expected": "陳 陳", + "got": "陳" + }, + "family_particles": { + "expected": "陳", + "got": "" + }, + "text": "陳·怡君·達印" + }, + "yue/foreignFull/surnameFirst": { + "family": { + "expected": "陳 陳", + "got": "陳 陳 怡君" + }, + "family_base": { + "expected": "陳", + "got": "陳 陳 怡君" + }, + "family_particles": { + "expected": "陳", + "got": "" + }, + "given_names": { + "expected": "怡君 達印", + "got": "小 達印" + }, + "suffix": { + "expected": "小 博士", + "got": "教授, 博士" + }, + "text": "陳·陳·怡君·教授，小·達印·博士", + "title": { + "expected": "教授", + "got": "" + } + }, + "yue/foreignG/givenFirst": { + "family": { + "expected": "", + "got": "凱晴" + }, + "given_names": { + "expected": "凱晴", + "got": "" + }, + "text": "凱晴" + }, + "yue/foreignG/sorting": { + "family": { + "expected": "", + "got": "凱晴" + }, + "given_names": { + "expected": "凱晴", + "got": "" + }, + "text": "凱晴" + }, + "yue/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "凱晴" + }, + "given_names": { + "expected": "凱晴", + "got": "" + }, + "text": "凱晴" + }, + "yue/nativeFull/givenFirst": { + "family": { + "expected": "陳", + "got": "先生子媚惠儀陳小" + }, + "family_base": { + "expected": "陳", + "got": "先生子媚惠儀陳小" + }, + "given_names": { + "expected": "子媚 惠儀", + "got": "太平紳士" + }, + "suffix": { + "expected": "小 太平紳士", + "got": "" + }, + "text": "先生子媚惠儀陳小，太平紳士", + "title": { + "expected": "先生", + "got": "" + } + }, + "yue/nativeFull/surnameFirst": { + "family": { + "expected": "陳", + "got": "陳子媚先生" + }, + "family_base": { + "expected": "陳", + "got": "陳子媚先生" + }, + "given_names": { + "expected": "子媚 惠儀", + "got": "小惠儀太平紳士" + }, + "suffix": { + "expected": "小 太平紳士", + "got": "" + }, + "text": "陳子媚先生，小惠儀太平紳士", + "title": { + "expected": "先生", + "got": "" + } + }, + "yue/nativeG/givenFirst": { + "family": { + "expected": "", + "got": "文傑" + }, + "given_names": { + "expected": "文傑", + "got": "" + }, + "text": "文傑" + }, + "yue/nativeG/sorting": { + "family": { + "expected": "", + "got": "文傑" + }, + "given_names": { + "expected": "文傑", + "got": "" + }, + "text": "文傑" + }, + "yue/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "文傑" + }, + "given_names": { + "expected": "文傑", + "got": "" + }, + "text": "文傑" + }, + "yue/nativeGGS/givenFirst": { + "family": { + "expected": "林", + "got": "家豪明德林" + }, + "given_names": { + "expected": "家豪 明德", + "got": "" + }, + "text": "家豪明德林" + }, + "yue/nativeGS/givenFirst": { + "family": { + "expected": "張", + "got": "雅婷張" + }, + "given_names": { + "expected": "雅婷", + "got": "" + }, + "text": "雅婷張" + }, + "yue_Hans/foreignFull/givenFirst": { + "family": { + "expected": "陈 陈", + "got": "小" + }, + "family_base": { + "expected": "陈", + "got": "小" + }, + "family_particles": { + "expected": "陈", + "got": "" + }, + "given_names": { + "expected": "怡君 达印", + "got": "教授 怡君 达印 陈 陈" + }, + "suffix": { + "expected": "小 博士", + "got": "博士" + }, + "text": "教授·怡君·达印·陈·陈·小，博士", + "title": { + "expected": "教授", + "got": "" + } + }, + "yue_Hans/foreignFull/sorting": { + "family": { + "expected": "陈 陈", + "got": "陈" + }, + "family_particles": { + "expected": "陈", + "got": "" + }, + "text": "陈·怡君·达印" + }, + "yue_Hans/foreignFull/surnameFirst": { + "family": { + "expected": "陈 陈", + "got": "陈 陈 怡君" + }, + "family_base": { + "expected": "陈", + "got": "陈 陈 怡君" + }, + "family_particles": { + "expected": "陈", + "got": "" + }, + "given_names": { + "expected": "怡君 达印", + "got": "小 达印" + }, + "suffix": { + "expected": "小 博士", + "got": "教授, 博士" + }, + "text": "陈·陈·怡君·教授，小·达印·博士", + "title": { + "expected": "教授", + "got": "" + } + }, + "yue_Hans/foreignG/givenFirst": { + "family": { + "expected": "", + "got": "凯晴" + }, + "given_names": { + "expected": "凯晴", + "got": "" + }, + "text": "凯晴" + }, + "yue_Hans/foreignG/sorting": { + "family": { + "expected": "", + "got": "凯晴" + }, + "given_names": { + "expected": "凯晴", + "got": "" + }, + "text": "凯晴" + }, + "yue_Hans/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "凯晴" + }, + "given_names": { + "expected": "凯晴", + "got": "" + }, + "text": "凯晴" + }, + "yue_Hans/nativeFull/givenFirst": { + "family": { + "expected": "陈", + "got": "先生子媚惠仪陈小" + }, + "family_base": { + "expected": "陈", + "got": "先生子媚惠仪陈小" + }, + "given_names": { + "expected": "子媚 惠仪", + "got": "太平绅士" + }, + "suffix": { + "expected": "小 太平绅士", + "got": "" + }, + "text": "先生子媚惠仪陈小，太平绅士", + "title": { + "expected": "先生", + "got": "" + } + }, + "yue_Hans/nativeFull/surnameFirst": { + "family": { + "expected": "陈", + "got": "陈子媚先生" + }, + "family_base": { + "expected": "陈", + "got": "陈子媚先生" + }, + "given_names": { + "expected": "子媚 惠仪", + "got": "小惠仪太平绅士" + }, + "suffix": { + "expected": "小 太平绅士", + "got": "" + }, + "text": "陈子媚先生，小惠仪太平绅士", + "title": { + "expected": "先生", + "got": "" + } + }, + "yue_Hans/nativeG/givenFirst": { + "family": { + "expected": "", + "got": "文杰" + }, + "given_names": { + "expected": "文杰", + "got": "" + }, + "text": "文杰" + }, + "yue_Hans/nativeG/sorting": { + "family": { + "expected": "", + "got": "文杰" + }, + "given_names": { + "expected": "文杰", + "got": "" + }, + "text": "文杰" + }, + "yue_Hans/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "文杰" + }, + "given_names": { + "expected": "文杰", + "got": "" + }, + "text": "文杰" + }, + "yue_Hans/nativeGGS/givenFirst": { + "family": { + "expected": "林", + "got": "家豪明德林" + }, + "given_names": { + "expected": "家豪 明德", + "got": "" + }, + "text": "家豪明德林" + }, + "yue_Hans/nativeGS/givenFirst": { + "family": { + "expected": "张", + "got": "雅婷张" + }, + "given_names": { + "expected": "雅婷", + "got": "" + }, + "text": "雅婷张" + }, + "zh/foreignFull/givenFirst": { + "family": { + "expected": "冯 布鲁赫", + "got": "布鲁赫博士" + }, + "family_base": { + "expected": "布鲁赫", + "got": "布鲁赫博士" + }, + "family_particles": { + "expected": "冯", + "got": "" + }, + "given_names": { + "expected": "艾达·科妮莉亚 塞萨尔·马丁", + "got": "小艾达 科妮莉亚 塞萨尔 马丁 冯" + }, + "suffix": { + "expected": "小 博士", + "got": "教授" + }, + "text": "小艾达·科妮莉亚·塞萨尔·马丁·冯·布鲁赫博士教授", + "title": { + "expected": "教授", + "got": "" + } + }, + "zh/foreignFull/sorting": { + "family": { + "expected": "布鲁赫 冯", + "got": "布鲁赫" + }, + "family_particles": { + "expected": "冯", + "got": "" + }, + "given_names": { + "expected": "艾达·科妮莉亚 塞萨尔·马丁", + "got": "艾达 科妮莉亚 塞萨尔 马丁 冯" + }, + "text": "布鲁赫·艾达·科妮莉亚·塞萨尔·马丁·冯" + }, + "zh/foreignFull/surnameFirst": { + "family": { + "expected": "冯 布鲁赫", + "got": "小冯" + }, + "family_base": { + "expected": "布鲁赫", + "got": "小冯" + }, + "family_particles": { + "expected": "冯", + "got": "" + }, + "given_names": { + "expected": "艾达·科妮莉亚 塞萨尔·马丁", + "got": "布鲁赫 艾达 科妮莉亚 塞萨尔 马丁" + }, + "suffix": { + "expected": "小 博士", + "got": "博士 教授" + }, + "text": "小冯·布鲁赫·艾达·科妮莉亚·塞萨尔·马丁·博士教授", + "title": { + "expected": "教授", + "got": "" + } + }, + "zh/foreignG/givenFirst": { + "family": { + "expected": "", + "got": "辛巴达" + }, + "given_names": { + "expected": "辛巴达", + "got": "" + }, + "text": "辛巴达" + }, + "zh/foreignG/sorting": { + "family": { + "expected": "", + "got": "辛巴达" + }, + "given_names": { + "expected": "辛巴达", + "got": "" + }, + "text": "辛巴达" + }, + "zh/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "辛巴达" + }, + "given_names": { + "expected": "辛巴达", + "got": "" + }, + "text": "辛巴达" + }, + "zh/nativeFull/givenFirst": { + "family": { + "expected": "彭", + "got": "小德威东升彭议员" + }, + "family_base": { + "expected": "彭", + "got": "小德威东升彭议员" + }, + "given_names": { + "expected": "德威 东升", + "got": "" + }, + "suffix": { + "expected": "小 议员", + "got": "先生" + }, + "text": "小德威东升彭议员先生", + "title": { + "expected": "先生", + "got": "" + } + }, + "zh/nativeFull/surnameFirst": { + "family": { + "expected": "彭", + "got": "小彭德威东升议员" + }, + "family_base": { + "expected": "彭", + "got": "小彭德威东升议员" + }, + "given_names": { + "expected": "德威 东升", + "got": "" + }, + "suffix": { + "expected": "小 议员", + "got": "先生" + }, + "text": "小彭德威东升议员先生", + "title": { + "expected": "先生", + "got": "" + } + }, + "zh/nativeG/givenFirst": { + "family": { + "expected": "", + "got": "宇瀚" + }, + "given_names": { + "expected": "宇瀚", + "got": "" + }, + "text": "宇瀚" + }, + "zh/nativeG/sorting": { + "family": { + "expected": "", + "got": "宇瀚" + }, + "given_names": { + "expected": "宇瀚", + "got": "" + }, + "text": "宇瀚" + }, + "zh/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "宇瀚" + }, + "given_names": { + "expected": "宇瀚", + "got": "" + }, + "text": "宇瀚" + }, + "zh/nativeGGS/givenFirst": { + "family": { + "expected": "陈", + "got": "俊年杰思陈" + }, + "given_names": { + "expected": "俊年 杰思", + "got": "" + }, + "text": "俊年杰思陈" + }, + "zh/nativeGS/givenFirst": { + "family": { + "expected": "张", + "got": "安怡张" + }, + "given_names": { + "expected": "安怡", + "got": "" + }, + "text": "安怡张" + }, + "zh_Hant/foreignFull/givenFirst": { + "family": { + "expected": "馮 陳", + "got": "怡君達印馮 陳二世博士" + }, + "family_base": { + "expected": "陳", + "got": "怡君達印馮 陳二世博士" + }, + "family_particles": { + "expected": "馮", + "got": "" + }, + "given_names": { + "expected": "怡君 達印", + "got": "醫學博士" + }, + "suffix": { + "expected": "二世 醫學博士", + "got": "" + }, + "text": "怡君達印馮·陳二世博士，醫學博士", + "title": { + "expected": "博士", + "got": "" + } + }, + "zh_Hant/foreignFull/sorting": { + "family": { + "expected": "馮 陳", + "got": "馮" + }, + "family_base": { + "expected": "陳", + "got": "馮" + }, + "family_particles": { + "expected": "馮", + "got": "" + }, + "given_names": { + "expected": "怡君 達印", + "got": "陳 怡君 達印" + }, + "text": "馮·陳·怡君·達印" + }, + "zh_Hant/foreignFull/surnameFirst": { + "family": { + "expected": "馮 陳", + "got": "馮" + }, + "family_base": { + "expected": "陳", + "got": "馮" + }, + "family_particles": { + "expected": "馮", + "got": "" + }, + "given_names": { + "expected": "怡君 達印", + "got": "陳 怡君 達印 二世醫學博士" + }, + "suffix": { + "expected": "二世 醫學博士", + "got": "" + }, + "text": "馮·陳·怡君·達印·二世醫學博士", + "title": { + "expected": "博士", + "got": "" + } + }, + "zh_Hant/foreignG/givenFirst": { + "family": { + "expected": "", + "got": "雅婷" + }, + "given_names": { + "expected": "雅婷", + "got": "" + }, + "text": "雅婷" + }, + "zh_Hant/foreignG/sorting": { + "family": { + "expected": "", + "got": "雅婷" + }, + "given_names": { + "expected": "雅婷", + "got": "" + }, + "text": "雅婷" + }, + "zh_Hant/foreignG/surnameFirst": { + "family": { + "expected": "", + "got": "雅婷" + }, + "given_names": { + "expected": "雅婷", + "got": "" + }, + "text": "雅婷" + }, + "zh_Hant/foreignGGS/givenFirst": { + "family": { + "expected": "王", + "got": "雅婷婷婷王" + }, + "given_names": { + "expected": "雅婷 婷婷", + "got": "" + }, + "text": "雅婷婷婷王" + }, + "zh_Hant/foreignGS/givenFirst": { + "family": { + "expected": "王", + "got": "雅婷王" + }, + "given_names": { + "expected": "雅婷", + "got": "" + }, + "text": "雅婷王" + }, + "zh_Hant/nativeFull/givenFirst": { + "family": { + "expected": "王", + "got": "雅婷婷婷王二世先生" + }, + "family_base": { + "expected": "王", + "got": "雅婷婷婷王二世先生" + }, + "given_names": { + "expected": "雅婷 婷婷", + "got": "議員" + }, + "suffix": { + "expected": "二世 議員", + "got": "" + }, + "text": "雅婷婷婷王二世先生，議員", + "title": { + "expected": "先生", + "got": "" + } + }, + "zh_Hant/nativeFull/surnameFirst": { + "given_names": { + "expected": "雅婷 婷婷", + "got": "雅婷婷婷二世議員" + }, + "suffix": { + "expected": "二世 議員", + "got": "" + }, + "text": "王雅婷婷婷二世議員" + }, + "zh_Hant/nativeG/givenFirst": { + "family": { + "expected": "", + "got": "文傑" + }, + "given_names": { + "expected": "文傑", + "got": "" + }, + "text": "文傑" + }, + "zh_Hant/nativeG/sorting": { + "family": { + "expected": "", + "got": "文傑" + }, + "given_names": { + "expected": "文傑", + "got": "" + }, + "text": "文傑" + }, + "zh_Hant/nativeG/surnameFirst": { + "family": { + "expected": "", + "got": "文傑" + }, + "given_names": { + "expected": "文傑", + "got": "" + }, + "text": "文傑" + }, + "zh_Hant/nativeGGS/givenFirst": { + "family": { + "expected": "林", + "got": "家豪明德林" + }, + "given_names": { + "expected": "家豪 明德", + "got": "" + }, + "text": "家豪明德林" + }, + "zh_Hant/nativeGS/givenFirst": { + "family": { + "expected": "張", + "got": "雅婷張" + }, + "given_names": { + "expected": "雅婷", + "got": "" + }, + "text": "雅婷張" + } +}