discivigour opened a new pull request, #8991:
URL: https://github.com/apache/paimon/pull/8991
## What changed
- add a TopN result type that keeps sortable keys and merges candidates
globally across BTree readers
- support forward and reverse BTree iteration for ascending and descending
TopN, including null ordering
- prune dominated BTree index files before lookup while retaining the
complete file set for coverage calculation
- group candidate row IDs by key and pass the resulting row ranges into
data-evolution scans
- add unit and table-level coverage for result merging, iterator direction,
file selection, cross-file candidates, diagnostics, and fallback behavior
## Why
BTree global indexes could evaluate predicates, but TopN planning could not
use the index ordering to reduce index and data reads. Evaluating each file
independently is insufficient because the final candidates must be compared
globally, and coverage handling must not treat intentionally pruned index files
as unindexed data.
This change computes a conservative global candidate set for supported
single-column TopN queries. The execution engine still retains the final sort
and limit, so the pushdown remains a pruning optimization rather than the
source of final result ordering.
## Impact
Data-evolution tables with a compatible BTree global index can reduce
index-file opens, index-entry scans, and downstream row reads for ascending or
descending TopN queries. Unsupported shapes and incomplete metadata continue to
fall back conservatively.
## Validation
```bash
mvn -pl paimon-core -am \
-Dmaven.repo.local=/opt/homebrew/opt/maven/repository \
-DwildcardSuites=none \
-DfailIfNoTests=false \
-Dtest=GlobalIndexEvaluatorTest,TopNGlobalIndexResultTest,BTreeIndexReaderTest,LazyFilteredBTreeIndexReaderTest,BlockIteratorTest,BTreeTopNIndexFileSelectorTest,BtreeGlobalIndexTableTest
\
test
```
- paimon-common: 229 tests passed
- paimon-core: 27 tests passed
- Checkstyle, Spotless, and Maven Enforcer passed
- `git diff --check` passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
