GitHub user DoingDone9 opened a pull request:
https://github.com/apache/spark/pull/5610
[SPARK-7026][SQL] make LeftSemiJoin work when it has both equal condition
and not equal condition
when leftsemijoin has both equal condition and not equal condition, it can
not work. sql like that
```
SELECT * FROM testData2 x
LEFT SEMI JOIN testData2 y
ON x.b = y.b
AND x.a >= y.a + 2
```
Reason ï¼
when leftsemijoin has both equal condition and not equal condition, it will
be tranformed to below PhysicalPlan. The output of LeftSemiJoinHash is
left.output, but the input of Filter shoule be left.output++right.output
```
== Physical Plan ==
Project [a#600,b#601]
!Filter (a#600 >= (a#602 + 2))
LeftSemiJoinHash [b#601], [b#603], LeftSemi
Exchange (HashPartitioning [b#601], 200)
HiveTableScan [a#600,b#601], (MetastoreRelation
tpcds_bin_partitioned_orc_3, testdata2, Some(x)), None
Exchange (HashPartitioning [b#603], 200)
HiveTableScan [a#602,b#603], (MetastoreRelation
tpcds_bin_partitioned_orc_3, testdata2, Some(y)), None
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DoingDone9/spark leftsemijoin
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5610.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5610
----
commit c3f046f8de7c418d4aa7e74afea9968a8baf9231
Author: DoingDone9 <[email protected]>
Date: 2015-03-02T02:11:18Z
Merge pull request #1 from apache/master
merge lastest spark
commit cb1852d14f62adbd194b1edda4ec639ba942a8ba
Author: DoingDone9 <[email protected]>
Date: 2015-03-05T07:05:10Z
Merge pull request #2 from apache/master
merge lastest spark
commit c87e8b6d8cb433376a7d14778915006c31f6c01c
Author: DoingDone9 <[email protected]>
Date: 2015-03-10T07:46:12Z
Merge pull request #3 from apache/master
merge lastest spark
commit 161cae3a29951d793ce721f9904888bd9529de72
Author: DoingDone9 <[email protected]>
Date: 2015-03-12T06:46:28Z
Merge pull request #4 from apache/master
merge lastest spark
commit 98b134f39ca57f11a5b761c7b9e5f8a7477bd069
Author: DoingDone9 <[email protected]>
Date: 2015-03-19T09:00:07Z
Merge pull request #5 from apache/master
merge lastest spark
commit d00303b7af9436b9bd6d6d27d411a5c8a2e2294d
Author: DoingDone9 <[email protected]>
Date: 2015-03-24T08:43:44Z
Merge pull request #6 from apache/master
merge lastest spark
commit 802261c043f56bd5ebe9e46b15e33cdc7c212176
Author: DoingDone9 <[email protected]>
Date: 2015-03-26T02:21:24Z
Merge pull request #7 from apache/master
merge lastest spark
commit 34b1a9a8a30f689b41fd52b8a10c08666c2ff2b5
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-08T07:55:24Z
Merge pull request #8 from apache/master
merge lastest spark
commit f61210c03f693a266969e06c52c23ccd1bfe3e1b
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-17T09:10:48Z
Merge pull request #9 from apache/master
merge lastest spark
commit f12fa50a675ed490be728c2b0389db47a8a0f990
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-21T06:23:53Z
Merge pull request #10 from apache/master
merge lastest spark
commit f42fa376a521152c419bed0ebd83d9989a14709b
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-21T11:16:14Z
Update basicOperators.scala
commit a331b2b3152875d79856c4ac8fc323ade7e1b4b0
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-21T11:17:17Z
Update SparkStrategies.scala
commit 03d59e18d946f3e0b4be9ecbecabf7979dda0143
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-21T11:19:01Z
Update SQLQuerySuite.scala
commit 8fa077934389938db277ce6c80c70bb5c7719052
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-21T11:31:19Z
Update SparkStrategies.scala
commit f3bdb2a9480691967a52c508dd3629b6db7603fc
Author: Zhongshuai Pei <[email protected]>
Date: 2015-04-21T11:38:08Z
Update SQLQuerySuite.scala
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
