skyyyy955 opened a new pull request, #11167: URL: https://github.com/apache/paimon/pull/11167
### Purpose Fixes #11165. Data Evolution scans can discard matching rows when a row-range selection is combined with LIMIT or TopN. For example, selecting one row from each of two 256-row files and requesting LIMIT 2 returns only the first selected row. The lower scan counts the whole first file toward the limit before the row-range selection is applied. TopN can similarly discard the file containing the smallest selected value because its statistics include unselected rows. Skip these two file-level pushdowns in `DataEvolutionBatchScan.planEager` when `rowRangeIndex` is present. Keep the existing row-range pruning, deferred-filter checks, and global-index TopN handling. Ordinary LIMIT and TopN without row ranges retain their existing fast paths. The change touches one production class and adds one real-table regression class. It changes no public API, storage format, or write/commit behavior. Queries with row ranges may read more candidate files or metadata to produce correct results; no performance improvement is claimed. ### Tests Against master `9b6c997198ea31584579a4bbefb239f052f40bbd`, four of the eight new test invocations fail. The same eight pass with the fix. They cover explicit row ranges with LIMIT, a supplied `GlobalIndexResult` with LIMIT, TopN with and without a separate LIMIT, ordinary LIMIT/TopN pruning, a data predicate, and empty/out-of-range selections. Normal Maven validation on Windows with JDK 8u451 and Maven 3.8.9 passed **26 tests, with no failures, errors, or skips**: - `DataEvolutionRowRangeLimitTest`: 8. - `DataEvolutionBatchScanTest`: 9. - `AppendOnlyLimitManifestReadTest`: 9. Checkstyle, Spotless, and Enforcer passed. After building the core dependencies, the focused test command is: ```sh mvn -B -ntp -pl paimon-core -DwildcardSuites=none -Dflink.forkCount=1 -Dtest=DataEvolutionRowRangeLimitTest,DataEvolutionBatchScanTest,AppendOnlyLimitManifestReadTest test ``` This is local focused validation, not the full CI matrix. The new tests write and read real Parquet tables; the supplied-index case constructs row IDs without an external service. Linux, engine integrations, deletion vectors, and partial-column updates have not been validated for this patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
