thinkharderdev opened a new issue, #5424:
URL: https://github.com/apache/arrow-datafusion/issues/5424
**Describe the bug**
A clear and concise description of what the bug is.
The filter pushdown optimizer will not push down predicates through
extension plans even if the extension plan allows it.
**To Reproduce**
Steps to reproduce the behavior:
```
#[derive(Debug)]
struct NoopPlan {
input: Vec<LogicalPlan>,
schema: DFSchemaRef,
}
impl UserDefinedLogicalNode for NoopPlan {
fn as_any(&self) -> &dyn Any {
self
}
fn inputs(&self) -> Vec<&LogicalPlan> {
self.input.iter().collect()
}
fn schema(&self) -> &DFSchemaRef {
&self.schema
}
fn expressions(&self) -> Vec<Expr> {
self.input
.iter()
.flat_map(|child| child.expressions())
.collect()
}
fn prevent_predicate_push_down_columns(&self) -> HashSet<String> {
HashSet::from_iter(vec!["c".to_string()])
}
fn fmt_for_explain(&self, f: &mut Formatter) -> std::fmt::Result {
write!(f, "NoopPlan")
}
fn from_template(
&self,
_exprs: &[Expr],
inputs: &[LogicalPlan],
) -> Arc<dyn UserDefinedLogicalNode> {
Arc::new(Self {
input: inputs.iter().cloned().collect(),
schema: self.schema.clone(),
})
}
}
#[test]
fn user_defined_plan() -> Result<()> {
let table_scan = test_table_scan()?;
let custom_plan = LogicalPlan::Extension(Extension {
node: Arc::new(NoopPlan {
input: vec![table_scan.clone()],
schema: table_scan.schema().clone(),
}),
});
let plan = LogicalPlanBuilder::from(custom_plan)
.filter(col("a").eq(lit(1i64)))?
.build()?;
// Push filter below NoopPlan
let expected = "\
NoopPlan\
\n Filter: test.a = Int64(1)\
\n TableScan: test";
assert_optimized_plan_eq(&plan, expected)?;
}
```
**Expected behavior**
A clear and concise description of what you expected to happen.
If a predicate does not contain any columns in
`UserDefinedLogicalNode::prevent_predicate_push_down_columns` it should get
pushed down
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
