XiaoHongbo-Hope opened a new pull request, #699: URL: https://github.com/apache/paimon-rust/pull/699
## Purpose Reduce unnecessary object-store IO and memory use while planning scans with sorted global indexes. ## Changes - prune sorted global-index manifest entries with the scan partition filter before evaluating index shards - use the file size already stored in `IndexFileMeta`, falling back to metadata lookup for legacy or invalid sizes - bound the BTree reader cache by `global-index.thread-num` - keep deletion-vector selection unchanged ## Performance On a partitioned data-evolution table with an equality lookup and a partition predicate: - native FAST planning: 8.18s -> 2.93s - retained index entries: 762 -> 176 - indexed candidate ranges: 4 -> 1 - peak RSS in a plan-plus-explain benchmark: about 2.45 GiB -> 586 MiB The result split and row range were unchanged. ## Tests - `cargo test -p paimon global_index_scanner --lib` (47 passed) - `cargo test -p paimon table_scan::tests --lib` (84 passed) - `cargo fmt --all -- --check` - `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
