viirya commented on code in PR #8626:
URL: https://github.com/apache/arrow-datafusion/pull/8626#discussion_r1436018505
##########
datafusion/optimizer/src/push_down_filter.rs:
##########
@@ -955,6 +965,36 @@ impl PushDownFilter {
}
}
+/// Convert cross join to join by pushing down filter predicate to the join
condition
+fn convert_cross_join_to_inner_join(cross_join: CrossJoin) -> Result<Join> {
+ let CrossJoin { left, right, .. } = cross_join;
+ let join_schema = build_join_schema(left.schema(), right.schema(),
&JoinType::Inner)?;
+ // predicate is given
+ Ok(Join {
+ left,
+ right,
+ join_type: JoinType::Inner,
+ join_constraint: JoinConstraint::On,
+ on: vec![],
+ filter: None,
+ schema: DFSchemaRef::new(join_schema),
+ null_equals_null: true,
+ })
+}
+
+/// Converts the inner join with empty equality predicate and empty filter
condition to the cross join
+fn convert_to_cross_join_if_beneficial(plan: LogicalPlan) ->
Result<LogicalPlan> {
+ if let LogicalPlan::Join(join) = &plan {
+ // Can be converted back to cross join
+ if join.on.is_empty() && join.filter.is_none() {
+ return LogicalPlanBuilder::from(join.left.as_ref().clone())
+ .cross_join(join.right.as_ref().clone())?
+ .build();
+ }
+ }
+ Ok(plan)
+}
Review Comment:
I mean if after `push_down_all_join` there are predicates remaining in
Filter, but Join has empty `on` and empty `filter`, because you match the plan
by `if let LogicalPlan::Join(join) = &plan`, the Join won't be converted to a
Cross Join even looks like it should be (based on the logic here).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
