sundapeng opened a new pull request, #8703: URL: https://github.com/apache/paimon/pull/8703
### Purpose Format table scan planning currently treats a pushed row `LIMIT` as a maximum split count. However, `FormatDataSplit` does not expose row counts, and a split may produce no rows—for example, a header-only CSV file, a dropped malformed file, or an ignored corrupt/lost file. If such a split is ordered first, `LIMIT 1` can plan only that split and silently return no rows even though a later split contains data. This change keeps the zero-limit fast path but stops pruning positive limits by split count. The existing reader-side limit remains in place, so readers can still stop early without discarding later splits during planning. The scan already enumerates all files before the old truncation, so this does not add directory-listing work; it may schedule more splits when needed to preserve correctness. This is the format-table counterpart of #7665. ### Brief change log - Keep all planned format-table splits when the positive row count of each split is unknown. - Add a regression test with a header-only first split and a later split containing the requested row. ### Tests - RED on the pre-fix code: `testPositiveLimitDoesNotPruneUnknownRowCountSplits` returned `[]` instead of `["42,later"]` in both parameterized runs. - GREEN after the fix: all 74 `FormatTableScanTest` runs passed. - `mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=none -Dtest=FormatTableScanTest test` - `mvn -pl paimon-core -am -DskipTests compile` - `git diff --check` ### API and Format - [x] This PR does not change any public API. - [x] This PR does not change the storage format. ### Documentation - [x] No documentation change is needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
