mustafasrepo commented on code in PR #8626:
URL: https://github.com/apache/arrow-datafusion/pull/8626#discussion_r1435993847
##########
datafusion/optimizer/src/push_down_filter.rs:
##########
@@ -955,6 +965,36 @@ impl PushDownFilter {
}
}
+/// Convert cross join to join by pushing down filter predicate to the join
condition
+fn convert_cross_join_to_inner_join(cross_join: CrossJoin) -> Result<Join> {
+ let CrossJoin { left, right, .. } = cross_join;
+ let join_schema = build_join_schema(left.schema(), right.schema(),
&JoinType::Inner)?;
+ // predicate is given
+ Ok(Join {
+ left,
+ right,
+ join_type: JoinType::Inner,
+ join_constraint: JoinConstraint::On,
+ on: vec![],
+ filter: None,
+ schema: DFSchemaRef::new(join_schema),
+ null_equals_null: true,
+ })
+}
+
+/// Converts the inner join with empty equality predicate and empty filter
condition to the cross join
+fn convert_to_cross_join_if_beneficial(plan: LogicalPlan) ->
Result<LogicalPlan> {
+ if let LogicalPlan::Join(join) = &plan {
+ // Can be converted back to cross join
+ if join.on.is_empty() && join.filter.is_none() {
+ return LogicalPlanBuilder::from(join.left.as_ref().clone())
+ .cross_join(join.right.as_ref().clone())?
+ .build();
+ }
+ }
+ Ok(plan)
+}
Review Comment:
To be more precise filter can be pushed down below the join completely. In
this case we may end up with joins empty equality predicate and empty filter
condition.
As an example
```
Filter(l.a>l.b AND r.a>r.b)
--Join (on=[], filter=None)
----LeftTable(a, b)
----RightTable(a, b)
```
will be converted to the plan below after `push_down_all_join`
```
Join (on=[], filter=None)
--Filter(l.a>l.b)
----LeftTable(a, b)
--Filter(r.a>r.b)
----RightTable(a, b)
```
this util ensures that plan above is converted to the plan below
```
CrossJoin
--Filter(l.a>l.b)
----LeftTable(a, b)
--Filter(r.a>r.b)
----RightTable(a, b)
```
However, if the original plan were
```
Filter(l.a>r.b)
--Join (on=[], filter=None)
----LeftTable(a, b)
----RightTable(a, b)
```
after `push_down_all_join` we will end up with following plan
```
Join (on=[], filter=Some(l.a>r.b))
--LeftTable(a, b)
--RightTable(a, b)
```
in this case join top cannot be converted to the cross join.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
