thswlsqls opened a new issue, #9063: URL: https://github.com/apache/paimon/issues/9063
**Search before asking** - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. **Paimon version** master @ a5e87ebf6 (2.1-SNAPSHOT) **Compute Engine** Engine-agnostic (format table read path) **Minimal reproduce step** 1. Create a CSV format table with `'csv.include-header' = 'true'`. 2. Write one uncompressed CSV file larger than `source.split.target-size` (default 128MB). 3. `SELECT COUNT(*)`. **What doesn't meet your expectations?** Expected: the count equals the number of data rows. Actual: it is short by (number of splits - 1). `SplitEnumerator.preferToSplitFile()` splits such a file into `(offset, length)` segments, each read by a `CsvFileReader`. `CsvFileReader#setupReading` (line 62-69) calls `readLine()` on every split, ignoring `offset`. The header only exists at byte 0, so where `offset > 0` the discarded line is that split's first data row. **Anything else?** The record straddling a boundary is already dropped by `StandardLineReader#skipFirstLine` and read in full by the previous split, so this extra `readLine()` is pure over-discard. A header plus 20 data rows, read as two segments, returns 19 rows. **Are you willing to submit a PR?** - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
