thswlsqls opened a new pull request, #9065:
URL: https://github.com/apache/paimon/pull/9065

   
   ### Purpose
   
   fix #9063
   
   - `CsvFileReader#setupReading` skipped a line on every split, but the header 
only exists at byte 0, so a split with `offset > 0` dropped its own first data 
row.
   - A header-bearing CSV larger than `source.split.target-size` lost one row 
per extra split.
   - The boundary record is already dropped by 
`StandardLineReader#skipFirstLine` and read in full by the previous split, so 
guarding on `offset == 0` cannot duplicate rows and leaves the single-split 
path unchanged; a comment records this.
   
   ### Tests
   
   - Added `CsvFileFormatTest#testHeaderSkippedOnlyInFirstSplit`: reads a 
50-row file as two segments via `createReader(context, offset, length)` and 
asserts no row is lost or duplicated, with `csv.include-header` true and false. 
Fails on master (49 of 50 rows).
   - `mvn -pl paimon-format -DfailIfNoTests=false clean install` — 530 tests 
passed.
   - No existing test changes result: the existing `read()` helper only uses 
`createReader(context)`.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to