Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22008#discussion_r208352309
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,45 @@ object EliminateOuterJoin extends Rule[LogicalPlan]
with PredicateHelper {
if (j.joinType == newJoinType) f else Filter(condition,
j.copy(joinType = newJoinType))
}
}
+
+/**
+ * Swaps right and left logical plans of a join when left is bigger than
right. This is useful
+ * because underlying cartesian product performs a nested loop, thus if
the outer table is
+ * smaller there are less iterator initialization.
--- End diff --
I think this only makes sense when building left iterator and right
iterator are the same cost. When building right iterator is less costly than
building left, swapping them might be performance regression.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
