GitHub user davies opened a pull request:
https://github.com/apache/spark/pull/8579
[SPARK-9730] [SQL] Add Full Outer Join support for SortMergeJoin
This PR is based on #8383 , thanks to @viirya
JIRA: https://issues.apache.org/jira/browse/SPARK-9730
This patch adds the Full Outer Join support for SortMergeJoin. A new class
SortMergeFullJoinScanner is added to scan rows from left and right iterators.
FullOuterIterator is simply a wrapper of type RowIterator to consume joined
rows from SortMergeFullJoinScanner.
Closes #8383
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/davies/spark smj_fullouter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8579.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8579
----
commit 5703d5685ab66742db22f9f6b826fa9cee8f39d5
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-23T17:06:00Z
Add Full Outer Join support for SortMergeJoin.
commit 3cc1cb810d2a8ac63fb72be48066dd41ed16f9b4
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-24T05:58:33Z
Merge remote-tracking branch 'upstream/master' into smj-fullouter
commit ddc5f825439dcc480d4b05beca9b9ab844da1797
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-24T06:12:19Z
Update ExtractEquiJoinKeys for SortMergeJoin.
commit 4a3178b8231825d25d29cdd530f6b55b6775774a
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-24T09:27:47Z
Fix outputPartitioning and NPE bug. Fix a test.
commit 06e0f74c489e9f0bdb8b889ca8e1802aa4c0a91e
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-24T16:17:24Z
Can not compare two rows when there are only nulls.
commit 0559a40e0202de5a4886672d422812d1a86d0576
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-25T17:28:18Z
Fix UnsafeRow.allNull.
commit 27b6044a388aa62eb18baf7b859099c489f0df26
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-26T04:43:27Z
Should use anyNull instead of allNull.
commit 74a601b3260837ec3c2111e90e5055d9b483472a
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-09-02T15:23:36Z
Merge remote-tracking branch 'upstream/master' into smj-fullouter
commit addf5fe04c667d3e57a776eebcddf0b82de69b3e
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-09-02T15:26:58Z
For comment.
commit 8a81df4eac38ec17be81c15a0a0cd51e340c7a04
Author: Davies Liu <[email protected]>
Date: 2015-09-02T21:35:19Z
refactor
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
