jackylee-ch opened a new pull request, #9938: URL: https://github.com/apache/paimon/pull/9938
### Purpose `CsvFormatWriter` quoted a value containing the row separator, but neither line reader tracks quotes: `StandardLineReader` splits on CR/LF and `CustomLineReader` is a leftmost-match KMP over the delimiter bytes. `CsvParser.parse` takes one line at a time with no continuation state, so `(1, "hello\nworld")` came back as `(1, null)` and `(null, null)` — `COUNT(*)` changed. Quoting cannot fix it without giving up splittability: a split boundary may fall inside the value. The writer now refuses such a value, naming the column. CR/LF are rejected only when they are the separator — under a custom `csv.line-delimiter` they are ordinary bytes, the documented way to carry a line break inside a value, and that still round-trips. `needsQuoting` still quotes a value that merely *begins* a delimiter match, since leftmost-match means the appended delimiter would complete a match started by the value's trailing bytes. `escapable` is gone: since #9904 the escape character is exactly one character. Set `csv.line-delimiter` to a sequence absent from the data, or strip the separator upstream. ### Tests `CsvFileFormatTest#testValueContainingRowSeparatorIsRejected`. `paimon-format`: 658 run, 0 failures. Written with Claude Code; verification is mine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
