JingsongLi commented on code in PR #672:
URL: https://github.com/apache/paimon-rust/pull/672#discussion_r3712874870
##########
crates/paimon/src/table/read_builder.rs:
##########
@@ -56,6 +60,15 @@ fn is_exact_filter_pushdown_for_schema(
let (_, data_predicates) =
split_partition_and_data_predicates(filter.clone(), fields,
partition_keys);
data_predicates.is_empty()
+ || data_predicates.iter().all(|predicate| {
+ matches!(
+ predicate,
+ Predicate::Leaf {
+ op: PredicateOperator::Eq | PredicateOperator::Like,
Review Comment:
[P1] Keep filtered row counts inexact
Marking data equality and `LIKE` predicates as exact is valid for filter
execution, but this same classification becomes `PaimonTableScan.filter_exact`.
With no limit, `partition_statistics` then publishes
`sum(split.merged_row_count())` as `Precision::Exact`, even though those counts
are computed before this data predicate. DataFusion AggregateStatistics can
therefore rewrite `SELECT COUNT(*) FROM t WHERE name = "alice"` or a residual
`LIKE` to the unfiltered file count without executing the scan. Please separate
"the connector enforces this filter exactly" from "the pre-filter split row
count is exact", or keep `num_rows` inexact whenever data predicates are
present. A `COUNT(*)` regression for equality and residual LIKE would catch
this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
