LuciferYang opened a new pull request, #9556: URL: https://github.com/apache/paimon/pull/9556
### Purpose close #9555 `CsvParser.parse` walks the projected fields in a `for` loop. In PERMISSIVE mode the branch that handles a malformed field broke out of that loop, so every later field of the row kept the null it was initialized with, even when those fields parse fine. `csv.mode` defaults to PERMISSIVE and the option is documented as setting the malformed field to null, so a plain `SELECT` over a CSV table returned wrong data with nothing thrown and nothing logged. Only a row whose malformed field was the last projected column read correctly. This regressed in #6856. Before it the mode was handled by a `switch`, where `case PERMISSIVE: break;` left the switch and the loop went on to the next field; that commit turned the switch into an if-else chain and the same `break` now leaves the loop. This nulls only the offending field. ### Tests `CsvFileFormatTest.testCsvPermissiveKeepsFieldsAfterMalformed` reads two files. The first covers a malformed field in the first position, `x,Alice,1.5` read as `(INT, STRING, DOUBLE)`, followed by a clean row so the per-row state reset stays pinned. The second is built from `new Options()`, since PERMISSIVE is the default mode, and covers a malformed field in the middle (`1,oops,world`) plus two malformed fields in one row (`y,bad,keep`). That last row goes through both ways a field can fail: `y` makes `parseInt` return null, and `bad` makes `Double.parseDouble` throw. Against the unfixed parser the first block fails with `expected: Alice but was: null` and the second block fails on the same assertion for `world`. A malformed field in the last projected column behaves identically before and after this change, and `testCsvModeWriteRead` and `testSpecialCases` already cover that shape, so the new test does not repeat it. `mvn -pl paimon-format test` on JDK 8: 597 tests, 0 failures. `spotless:check` and `checkstyle:check` are clean. One note on the shape of the fix: the PERMISSIVE branch keeps an explicit `row.setField(i, null)`. On a freshly allocated `GenericRow` that is a no-op, so the behavior change comes from dropping the `break`; the statement is there so the branch states the documented contract instead of reading as an unhandled case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
