[I] PushDownFilter optimizer pushes down filters through distinct on [datafusion]

[via GitHub]

epsio-banay opened a new issue, #12942:
URL: https://github.com/apache/datafusion/issues/12942

   ### Describe the bug
   
   a filter should not be pushed down through a distinct on because it will 
change the results.
   The PushDownFilter optimizer rule does not behave correctly.
   
   ### To Reproduce
   
   Add this test to push_down_filter.rs:
   
   ```
   #[test]
       fn distinct_on() -> Result<()> {
           let table_scan = test_table_scan()?;
           let plan = LogicalPlanBuilder::from(table_scan)
               .distinct_on(vec![col("a")], vec![col("a")], None)?
               .filter(col("a").eq(lit(1i64)))?
               .build()?;
           // filter appears below Union
           let expected = "\
           Filter: a = Int64(1)\
           \n  DistinctOn: on_expr=[[test.a]], select_expr=[[a]], 
sort_expr=[[]]\
           \n    TableScan: test";
           assert_optimized_plan_eq(plan, expected)
       }
   ```
   
   you will get the logical plan:
   ```
   DistinctOn: on_expr=[[test.a]], select_expr=[[a]], sort_expr=[[]]
     TableScan: test, full_filters=[a = Int64(1)]
   ```
   
   ### Expected behavior
   
   Filters should not be pushed down through distinct on.
   the logical plan should be as expected in the above test.
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org