Repository navigation
POST Wikidata queries from name_usage.py instead of a GET - #683
Merged
Merged
Conversation
The query's VALUES block repeats every spelling once per label language, so a seven-letter Cyrillic word (Младший, six bytes a letter percent-encoded) put the GET's URL past 7 KB and the endpoint answered HTTP 431. Measured: Младши's longer query is 6,909 bytes and succeeds, Младший's is 7,198 and failed. A form-encoded POST body has no such limit. Checked live: the seven words the issue names now answer, and shehu still reads 180 family as before. Closes #680 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #683 +/- ##
=======================================
Coverage 99.03% 99.03%
=======================================
Files 46 46
Lines 4541 4543 +2
=======================================
+ Hits 4497 4499 +2
Misses 44 44 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Review of #683: the body was correct UTF-8, but without a charset its decoding rested on the server's form default, and a misdecoded query still runs and matches nothing -- a false zero. Live control: Иван reads 17,119 given and Петров 964 family through the POST, with and without the declared charset. The test now also checks the response is read back from what urlopen returns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t's input class #683's first control for the POST was ASCII (shehu) and could not show the server decoding a Cyrillic body; review caught it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #680.
tools/name_usage.pysent its SPARQL query as a GET. The query'sVALUESblock holds one literal per spelling per label language (63 languages). Percent-encoded, a Cyrillic letter takes six bytes, so a seven-letter word likeМладшийmade the URL longer than the endpoint accepts and got HTTP 431. Measured: the largest URL that succeeded was 6,909 bytes (Младши, countries query) and the smallest that failed was 7,198 (Младший, totals query, which is sent first). A four-spelling word such asMcDonaldwritten in Cyrillic would also have failed.The query can't shrink: the comment at
WIKIDATA_LANGUAGESexplains that exact literals keep it on the label index, while aFILTERtimes out. So this PR changes the transport._sparqlnow POSTs a form-encodedquery=body declaredcharset=utf-8, which has no length limit.test_sparql_sends_the_query_in_the_body_not_the_urlcaptures theRequestand asserts POST, a bare endpoint URL, and a body that decodes back to the query. It also checks that the response is read from whaturlopenreturns. It failed before the fix.Иванgives 17,119 given andПетров964 family, and김also gives counts. So the server decodes the body as UTF-8, and the zeros for the seven words the issue names are genuine answers, not failures to match. The control is run in Cyrillic because a misdecoded body would still run and match nothing, which looks like a false zero. An ASCII control (shehu, 180 family) cannot show that.🤖 Generated with Claude Code