Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18692#discussion_r153060551
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +152,99 @@ object EliminateOuterJoin extends Rule[LogicalPlan]
with PredicateHelper {
if (j.joinType == newJoinType) f else Filter(condition,
j.copy(joinType = newJoinType))
}
}
+
+/**
+ * A rule that eliminates CROSS joins by inferring join conditions from
propagated constraints.
+ *
+ * The optimization is applicable only to CROSS joins. For other join
types, adding inferred join
+ * conditions would potentially shuffle children as child node's
partitioning won't satisfy the JOIN
+ * node's requirements which otherwise could have.
+ *
+ * For instance, if there is a CROSS join, where the left relation has 'a
= 1' and the right
+ * relation has 'b = 1', the rule infers 'a = b' as a join predicate.
--- End diff --
> For instance, given a CROSS join with the constraint 'a = 1' from the
left child and the constraint 'b = 1' from the right child, this rule infers a
new join predicate 'a = b' and convert it to an Inner join.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
