hhhizzz opened a new issue, #469: URL: https://github.com/apache/paimon-rust/issues/469
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-rust/issues) and found nothing similar. ### Description ### Description `ReadBuilder::with_limit()` can pass a row-count hint into paimon-core scan planning, but scan planning can still build every candidate split before the final limited plan is returned. For small LIMIT reads on tables with many candidate split groups, especially data-evolution tables, this means the final plan may contain only a few splits while planning still pays the cost of constructing the rest. The optimization should allow scan planning to stop split construction once known split-level row counts already cover the requested limit. The limit hint must remain conservative. It should only be used when split-level `merged_row_count()` is safe as a planning hint. Cases where rows may be reduced after planning, such as data predicates or effective row ranges, should continue to build the complete candidate split set. Expected direction: - Keep manifest planning complete for now. - Push safe LIMIT planning into split construction. - Stop building candidate splits once known `merged_row_count()` values cover the requested limit. - Preserve fail-open behavior when known row counts do not cover the limit. - Preserve safety gates for predicates and row ranges. - Treat `LIMIT 0` as an empty plan. - Expose scan trace state so tests and diagnostics can distinguish construction-time early stop from final split pruning. This is related to, but separate from, #220. That issue covered the DataFusion fetch/limit contract; this issue is about paimon-core scan planning reducing split construction work after the limit hint has already reached core. ### Willingness to contribute - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
