JingsongLi opened a new pull request, #549: URL: https://github.com/apache/paimon-rust/pull/549
## Summary Add a pruning-only predicate channel for file readers. These predicates may skip Parquet row groups or pages whose statistics prove they cannot match, but they never remove individual rows from emitted batches. This is the core prerequisite for a follow-up DataFusion runtime-filter PR, where the parent operator may still own exact evaluation. ## Changes - carry exact and pruning-only predicates separately through `TableRead`, format-table reads, and `DataFileReader` - use the union only for Parquet row-group and page selection - keep Arrow row filters and residual evaluation restricted to exact predicates - remap pruning predicates safely across file schema evolution - reject `_ROW_ID` projection when pruning could skip physical positions - keep pruning predicates out of PK merge and data-evolution pre-merge readers - update file-format predicate contexts for the new explicit channel ## Testing - `cargo test -p paimon --lib parquet_pruning_predicate_skips_row_groups_without_filtering_rows` - `cargo test -p paimon --lib` - `cargo test -p paimon --all-targets --features fulltext,vortex` - `cargo check -p paimon --all-targets --features vortex` - `cargo clippy --all-targets --workspace --features fulltext,vortex -- -D warnings` - `cargo fmt --all -- --check` - `git diff --check` The regression test verifies that `id > 100` skips a `[1, 2]` row group while retaining both rows from `[100, 101]`, including the non-matching boundary row `100`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
