XiaoHongbo-Hope opened a new pull request, #597: URL: https://github.com/apache/paimon-rust/pull/597
### Purpose A row selection without predicates did not load the Parquet OffsetIndex. Arrow-rs could filter rows, but still fetched unselected data pages, including pages for nested columns. This is independent of #590: that PR prunes files, while this change prunes pages inside a retained Parquet file. It also improves selected-range reads introduced by #592. ### Changes - Load ColumnIndex for predicate page pruning. - Load OffsetIndex for predicates or row selections. - Add a nested List regression test that verifies sparse selection reduces page I/O. ### Tests - cargo fmt --all -- --check - Parquet module tests: 41 passed - paimon library tests: 1,839 passed, 1 ignored - clippy with warnings denied -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
