hhhizzz opened a new pull request, #470:
URL: https://github.com/apache/paimon-rust/pull/470
### Purpose
Linked issue: close #469
Push safe LIMIT planning into split construction so limited scans can stop
building candidate splits once known split-level row counts cover the requested
limit.
This keeps manifest planning complete and preserves the existing safety
gates: LIMIT early-stop is only applied when split-level `merged_row_count()`
remains conservative, so scans with data predicates or effective row ranges
still build the full candidate set.
### Brief change log
- Add an internal LIMIT pushdown accumulator that runs while scan planning
constructs `DataSplit`s.
- Stop split construction once known `merged_row_count()` values cover the
requested LIMIT.
- Preserve fallback behavior when the LIMIT is not covered by known row
counts, so the original candidate split set is returned instead of being
unsafely pruned.
- Return an empty plan for `LIMIT 0` without building split candidates.
- Extend scan trace output with `split_candidates_built` and
`limit_early_stopped` so tests and diagnostics can distinguish final split
pruning from construction-time early stop.
- Add/extend data-evolution integration coverage for successful early stop
and row-range safety.
### Tests
Targeted tests run:
- `rtk cargo test -p paimon test_incremental_limit_accumulator --
--nocapture`
- `2 passed`
- `rtk cargo test -p paimon-integration-tests test_limit_pushdown --
--nocapture`
- `3 passed`
Full `cargo test` was not run locally.
### API and Format
No storage format changes.
The public scan trace surface is extended with additive fields for LIMIT
planning diagnostics:
- `split_candidates_built`
- `limit_early_stopped`
Existing read results and LIMIT semantics are unchanged; callers still need
to enforce the final LIMIT at the query/read layer.
### Documentation
No user-facing documentation update is required. This is an internal
scan-planning optimization with trace diagnostics covered by tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
