GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/10567
[SPARK-12594] [SQL] Outer Join Elimination by Filter Conditions
Conversion of outer joins, if the predicates in filter conditions can
restrict the result sets so that all null-supplying rows are eliminated.
- `full outer` -> `inner` if both sides have such predicates
- `left outer` -> `inner` if the right side has such predicates
- `right outer` -> `inner` if the left side has such predicates
- `full outer` -> `left outer` if only the left side has such predicates
- `full outer` -> `right outer` if only the right side has such predicates
If applicable, this can greatly improve the performance, since outer join
is much slower than inner join, full outer join is much slower than left/right
outer join.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark
outerJoinEliminationByFilterCond
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10567.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10567
----
commit d516ed443b043261005c39917922f79094385358
Author: gatorsmile <[email protected]>
Date: 2016-01-04T03:32:39Z
outer join elimination by Filter condition
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
