2010YOUY01 opened a new issue, #18856: URL: https://github.com/apache/datafusion/issues/18856
### Is your feature request related to a problem or challenge? Original PR discussion: https://github.com/apache/datafusion/pull/18644#discussion_r2520068776 ## Backgrounds - Backgrounds for dynamic filter: https://datafusion.apache.org/blog/2025/09/10/dynamic-filters/ - Backgrounds for parquet filter pushdown: Parquet accepts a pushed-down filter for two purposes: 1) pruning partitions using statistics 2) Do row-level filtering for late materialization - Current APIs TLDR: - When pushdown predicates, we don't differentiate is it for only stats pruning, or both stats and row-level filters, and the parquet operator will try to apply both (unless turned off by an option) Reference: https://github.com/apache/datafusion/blob/f1ecaccd183367086ecb5b7736d93b3aba109e01/datafusion/physical-plan/src/execution_plan.rs#L563 - When child reply back the pushdown result in the optimizer pass, it said either [Yes/No](https://github.com/apache/datafusion/blob/f1ecaccd183367086ecb5b7736d93b3aba109e01/datafusion/physical-plan/src/filter_pushdown.rs#L121-L128) (I think the semantics now is Yes -> stat+row, No->maybe-stat) ### Describe the solution you'd like When one `ExecutionPlan` tries to pushdown a predicate, it should be able to indicate "this predicate is only for stats pruning", etc. The reason is some predicate might be too expensive and not worth evaluate at row level. When the child propagate back pushdown result, it should also indicate how do they accept the filter (only for stat, or for both stat and row-level), so that the source node for the predicate can respond accordingly. ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
