GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/10542
[SPARK-12594] [SQL] Outer Join Conversion: Outer/Right/Left to Inner and
Outer to Left/Right
Conversion of outer joins, if the local predicates can restrict the result
sets so that all null-supplying rows are eliminated.
- full outer -> inner if both sides have such local predicates
- left outer -> inner if the right side has such local predicates
- right outer -> inner if the left side has such local predicates
- full outer -> left outer if only the left side has such local predicates
- full outer -> right outer if only the right side has such local predicates
If applicable, this can greatly improve the performance, since outer join
is much slower than inner join, full outer join is much slower than left/right
outer join.
TODO: add extra test cases to Optimizer buckets.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark outerJoinConversion
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10542.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10542
----
commit 2f6c80dc57f10171cb97e22f6fb3257f9b4e308d
Author: gatorsmile <[email protected]>
Date: 2015-12-31T23:48:32Z
Merge remote-tracking branch 'upstream/master' into outerJoinElimination
commit 90576aacb085e9192360505ac5687a3df5e54b6f
Author: gatorsmile <[email protected]>
Date: 2016-01-01T00:23:03Z
outer join conversion
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
