Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1015#discussion_r13582164
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -254,28 +257,74 @@ object PushPredicateThroughProject extends
Rule[LogicalPlan] {
/**
* Pushes down [[catalyst.plans.logical.Filter Filter]] operators where
the `condition` can be
- * evaluated using only the attributes of the left or right side of an
inner join. Other
+ * evaluated using only the attributes of the left or right side of a
join. Other
* [[catalyst.plans.logical.Filter Filter]] conditions are moved into the
`condition` of the
* [[catalyst.plans.logical.Join Join]].
+ *
+ * The basic condition (JoinCondition & Filter) push down should conform
to the rule:
+ * https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior
*/
-object PushPredicateThroughInnerJoin extends Rule[LogicalPlan] with
PredicateHelper {
+object PushPredicateThroughJoin extends Rule[LogicalPlan] with
PredicateHelper {
+ // split the condition expression into 3 parts,
+ // (canEvaluateInLeft, canEvaluateInRight, haveToEvaluateInBoth)
--- End diff --
How about?
```
/**
* Splits join condition expressions into three categories based on the
attributes required
* to evaluate them.
* @returns (canEvaluateInLeft, canEvaluateInRight, haveToEvaluateInBoth)
*/
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
