JingsongLi commented on code in PR #448:
URL: https://github.com/apache/paimon-rust/pull/448#discussion_r3523397822
##########
crates/paimon/src/table/read_builder.rs:
##########
@@ -233,10 +232,13 @@ impl<'a> ReadBuilder<'a> {
Some(projected) => self.resolve_projected_fields(projected)?,
};
+ // Pass the FULL data predicate through (including `And`/`Or`/`Not`).
+ // Pushdown/stats skip compound nodes; the residual pass enforces the
full
+ // predicate exactly. Pruning here would drop compound predicates.
Ok(TableRead::new(
self.table,
read_type,
- reader_pruning_predicates(self.filter.data_predicates.clone()),
+ self.filter.data_predicates.clone(),
Review Comment:
This still does not make the public read path exact for
`data_evolution_enabled` tables. `new_read()` now preserves the full data
predicate here, but `TableRead::read_with_evolution()` builds
`DataEvolutionReader` without passing `self.data_predicates`, and
`DataEvolutionReader` creates every `DataFileReader` with `Vec::new()`.
`TableScan` also disables normal data-predicate pushdown for data-evolution
tables (`pushdown_data_predicates = &[]`), so a filtered scan without a usable
global index, or with only coarse row ranges, will still return non-matching
rows from the data-evolution reader. This regresses the PR goal that scan
batches are exactly filtered; the evolution reader needs to receive and apply
the same residual predicates after column merge (or before when it is safe).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
