Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/10566#discussion_r49676639
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -769,6 +770,107 @@ object ReorderJoin extends Rule[LogicalPlan] with
PredicateHelper {
}
/**
+ * Elimination of Outer Join by Parent Join Condition
+ *
+ * Given an outer join is involved in another join (called parent join),
when the join type of the
+ * parent join is inner, left-semi, left-outer and right-outer, checking
if the join condition of
+ * the parent join satisfies the following two conditions:
+ *
+ * 1) there exist null filtering predicates against the columns in the
null-supplying side of
+ * parent join.
+ * 2) these columns are from the child join.
+ *
+ * If having such join predicates, execute the elimination rules:
+ * - full outer -> inner if both sides of the child join have such
predicates
+ * - left outer -> inner if the right side of the child join has such
predicates
+ * - right outer -> inner if the left side of the child join has such
predicates
+ * - full outer -> left outer if only the left side of the child join has
such predicates
+ * - full outer -> right outer if only the right side of the child join
has such predicates
+ *
+ */
+object OuterJoinElimination extends Rule[LogicalPlan] with PredicateHelper
{
+
+ private def containsAttr(plan: LogicalPlan, attr: Attribute): Boolean =
+ plan.outputSet.exists(_.semanticEquals(attr))
+
+ private def hasNullFilteringPredicate(predicate: Expression, plan:
LogicalPlan): Boolean = {
--- End diff --
Sure, will do. Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
