xudong963 commented on a change in pull request #1339:
URL: https://github.com/apache/arrow-datafusion/pull/1339#discussion_r753654701
##########
File path: datafusion/src/optimizer/filter_push_down.rs
##########
@@ -408,10 +412,20 @@ fn optimize(plan: &LogicalPlan, mut state: State) ->
Result<LogicalPlan> {
//
// Join clauses with `Using` constraints also take advantage of
this logic to make sure
// predicates reference the shared join columns are pushed to both
sides.
+ let mut null_unsafe_predicates = vec![];
let join_side_filters = state
.filters
.iter()
.filter_map(|(predicate, columns)| {
+ if join_type == &JoinType::Left
+ || join_type == &JoinType::Right
+ || join_type == &JoinType::Full
+ {
+ if let Expr::IsNull(..) | Expr::IsNotNull(..) =
predicate {
Review comment:
> I don't think we can push _ANY_ filers from the `ON` clause down the
non preserved side(s) of an outer join -- in other words, this is not a problem
with `IS NULL` and `IS NOT NULL`.
Yes, I agree. Luckily, we don't implement the case.
https://github.com/apache/arrow-datafusion/blob/0facd4d483e8c289ee4e3a89487d0cd1ede1a110/datafusion/src/sql/planner.rs#L564
However, I think there is a problem with `IS NULL` and `IS NOT NULL` in
`WHERE` clause with an outer join, we shouldn't push them down.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
