LuciferYang opened a new pull request, #10020: URL: https://github.com/apache/paimon/pull/10020
### Purpose close #10019 `TopNDataSplitEvaluator` supports Spark single-key TopN pushdown (`ORDER BY col [ASC|DESC] [NULLS FIRST|LAST] LIMIT n` on an append-only table, a min/max-comparable sort type, limit ≤ 100). It orders the splits by the sort column's aggregate min/max and keeps the first `limit`. In the NULLS LAST branches the comparison falls to `ascCompare` / `descCompare`, which treat a null min/max as the smallest value, so a split whose sort column is entirely NULL sorts to the front, takes a kept slot, and displaces the split holding the real top value. With `LIMIT 1` and two or more splits the reader keeps only the all-null split and emits a NULL row instead of the true top-N. This keeps a provably-all-null split (its tracked null count equals its row count) last under NULLS LAST. `RichSplit` gains an `allNull` flag, and the NULLS LAST comparators order by it first, then fall back to the existing min/max and null-count comparison. A split whose bound is merely unknown (null min/max but not provably all-null, for example a `stats.mode=counts` file) is not all-null, so it still sorts first and is read conservatively. NULLS FIRST ordering is unchanged. It also fixes a `Comparator` contract violation in the same file: `ascCompare` / `descCompare` / `nullsFirstCompare` / `nullsLastCompare` returned `-1` when the left operand was null without checking the right, so two nulls compared as `-1` in both directions. They now return `0` when both operands are null. ### Tests `TableScanTest.testPushDownTopNNullsLastSortsAllNullSplitLast`: with an all-null split and a real split under ASC and DESC NULLS LAST at `LIMIT 1`, asserts the real split is kept; without the fix the all-null split takes the slot and the query returns a NULL row. It also asserts NULLS FIRST still treats the null-containing split as the best candidate, and that two tied all-null splits are both kept. `TableScanTest.testPushDownTopNNullsLastKeepsStatsUnknownSplitFirst`: a split with unknown bounds (null min/max, null count below row count) stays first under NULLS LAST, so an unknown-stats split is read rather than dropped. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
