kosiew commented on code in PR #19884:
URL: https://github.com/apache/datafusion/pull/19884#discussion_r2704403935
##########
datafusion/core/src/physical_planner.rs:
##########
@@ -1912,24 +1907,48 @@ fn get_physical_expr_pair(
}
/// Extract filter predicates from a DML input plan (DELETE/UPDATE).
-/// Walks the logical plan tree and collects Filter predicates,
-/// splitting AND conjunctions into individual expressions.
+/// Walks the logical plan tree and collects Filter predicates and any filters
+/// pushed down into TableScan nodes, splitting AND conjunctions into
individual expressions.
/// Column qualifiers are stripped so expressions can be evaluated against
-/// the TableProvider's schema.
+/// the TableProvider's schema. Deduplicates filters to avoid passing the same
+/// predicate twice when filters appear in both Filter and TableScan nodes.
///
fn extract_dml_filters(input: &Arc<LogicalPlan>) -> Result<Vec<Expr>> {
let mut filters = Vec::new();
input.apply(|node| {
- if let LogicalPlan::Filter(filter) = node {
- // Split AND predicates into individual expressions
-
filters.extend(split_conjunction(&filter.predicate).into_iter().cloned());
+ match node {
+ LogicalPlan::Filter(filter) => {
+ // Split AND predicates into individual expressions
+
filters.extend(split_conjunction(&filter.predicate).into_iter().cloned());
+ }
+ LogicalPlan::TableScan(TableScan {
+ filters: scan_filters,
+ ..
+ }) => {
+ for filter in scan_filters {
+
filters.extend(split_conjunction(filter).into_iter().cloned());
+ }
+ }
+ _ => {}
Review Comment:
Good point!
The DML input we build for DELETE/UPDATE currently only carries predicates
in `Filter` nodes or in `TableScan.filters` after pushdown, so the match over
those two variants is exhaustive for today's plan shapes.
We also have regression tests for pushdown
paths--`test_delete_filter_pushdown_extracts_table_scan_filters` and
`test_delete_compound_filters_with_pushdown which should catch any future
optimizer change that relocates predicates away from these nodes.
If we ever add a new logical node that can hold filters, we'll extend
`extract_dml_filters` instead of relying on the default case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
