Capture the COPY ... FROM STDIN payload verbatim

[LucaCappelletti94]

Parsing COPY t (a, b) FROM stdin; with two rows of two columns produced six values with nothing marking where the first row ended, so the rows could not be recovered even by counting columns. Printing the statement did not reproduce its input either, and every print and re-parse cycle inserted one more tab before each field, without bound.

The cause is that the payload was being split on tabs and newlines with \N interpreted, which is a partial implementation of the PostgreSQL text format. This removes that rather than completing it. Statement::Copy now carries payload: Option<String>, holding the text between the command and the \. terminator exactly as written, and Display writes it back unchanged. What a tab or a \N means is left to whoever consumes the statement. None means the statement carried no inline data, so a bare COPY t FROM STDIN; still prints without an invented terminator, while Some("") is an explicitly empty payload.

This breaks Statement::Copy, and Parser::parse_tsv and Parser::parse_tab_value are removed along with the format handling. Every other COPY sub-parser was already private.

My suggestion here is that either we go full csv parsing (which I agree might be a bit much), or no parsing at all (which is what I am suggesting we do in this PR), as the current impl was neither and broken.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 97.16981% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.40%. Comparing base (d32825d) to head (8b2d212).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/tokenizer.rs	97.86%	1 Missing and 3 partials ⚠️
src/ast/mod.rs	60.00%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2428      +/-   ##
==========================================
+ Coverage   81.29%   81.40%   +0.11%     
==========================================
  Files          42       42              
  Lines       33950    34120     +170     
  Branches    33950    34120     +170     
==========================================
+ Hits        27601    27777     +176     
+ Misses       2785     2775      -10     
- Partials     3564     3568       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sabir-akhadov-localstack]

Three non-blocking comments inline.

[sabir-akhadov-localstack]

The payload is re-rendered from tokens rather than sliced from the source, so it is not byte-for-byte: 0x1F comes back as X'1F', E'\x41' as E'A', CRLF as LF, and 'a''b' as 'a'b', which no longer re-parses. Worth softening "as written" here and in the Statement::Copy doc unless the tokenizer hands over the raw text.

[LucaCappelletti94]

Rendering tokens could never be exact, and data such as O'Brien did not even tokenize.

The tokenizer now cuts the payload out of the source before tokenizing, so it comes back byte for byte, including 0x1F, E'\x41', CRLF and 'a''b'.

Objectively, the tokenizer needs a complete overhaul, but I will tackle that after most of the fuzzer bugs are squashed or the new tokenizer will be a mess to fuzz.

[sabir-akhadov-localstack]

PostgreSQL only ends the data at a line containing just \., so 1\tfoo\.bar (literal foo.bar in text format, plain data in CSV) terminates the payload here and then fails on bar. Requiring the previous token to be a newline (or payload start) and the next to be newline/EOF would match the docs; the same spot could also treat \. as the statement terminator, since \.\nSELECT 1; currently fails with "Expected: end of statement".

[LucaCappelletti94]

The payload now ends only at a line holding just \., and a statement after it parses.

That said, while your claim is very reasonable, PostgreSQL 16.15 rejects 1\tfoo\.bar in text format with end-of-copy marker corrupt, so a mid-line \. stays in the payload and lets the server decides.

[sabir-akhadov-localstack]

COPY t FROM STDIN;\n\n yields Some("\n") and prints a \., while ;\n yields None; a whitespace-only unterminated trail should probably be None too.

[LucaCappelletti94]

psql loads that blank line as a row. Running COPY t FROM STDIN; followed by an empty line at end of file through psql 16.15 inserts one row holding '', so the payload keeps it. It is recorded as unterminated and prints without an invented \..