Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/18692#discussion_r136868330
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +152,71 @@ object EliminateOuterJoin extends Rule[LogicalPlan]
with PredicateHelper {
if (j.joinType == newJoinType) f else Filter(condition,
j.copy(joinType = newJoinType))
}
}
+
+/**
+ * A rule that uses propagated constraints to infer join conditions. The
optimization is applicable
+ * only to CROSS joins.
--- End diff --
Can you also mention the reason why we are restricting this to cross joins
only ?
```
For other join types, adding inferred join conditions would potentially
shuffle children as child node's partitioning won't satisfying the JOIN node's
requirements which otherwise could have.
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
