GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/12306
[SPARK-4226][SQL] Support Correlated Sub-queries
### What changes were proposed in this pull request?
This PR adds support for correlated subqueries to Spark. Correlated
sub-queries are typically used as a filtering condition in a query (this is the
only supported use case). A correlated sub-query comes in two forms:
- `[NOT] EXISTS(subquery)`
- `[NOT] IN (subquery)`
This PR is (loosely) based on the work of @davies
(https://github.com/apache/spark/pull/10706) and @chenghao-intel
(https://github.com/apache/spark/pull/9055). They should be credited for the
work they did.
### How was this patch tested?
Modified parsing unit tests.
Added tests to `org.apache.spark.sql.SQLQuerySuite`
cc @rxin, @davies & @chenghao-intel
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hvanhovell/spark SPARK-4226
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12306.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12306
----
commit b2108a99b6595a6ac5bf617827e4d81b3076e755
Author: Herman van Hovell <[email protected]>
Date: 2016-04-05T12:09:37Z
Enable Subqueries in parser.
commit 91107e0db752074b487a0a789b551720aa084242
Author: Herman van Hovell <[email protected]>
Date: 2016-04-07T05:38:00Z
Merge remote-tracking branch 'apache-github/master' into SPARK-4226
# Conflicts:
#
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
commit 3ac4f4f9f7f917dd9e24c1bc65c7e4af1801d5c9
Author: Herman van Hovell <[email protected]>
Date: 2016-04-11T20:46:11Z
Add subquery planning
commit 4d3e2659dd9256b39e468f25b16b31e0aeae1f3a
Author: Herman van Hovell <[email protected]>
Date: 2016-04-11T20:54:23Z
Merge remote-tracking branch 'apache-github/master' into SPARK-4226
# Conflicts:
#
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
commit 082bb861b5b76b956dbf7bb6d9ca544bc87c85a7
Author: Herman van Hovell <[email protected]>
Date: 2016-04-11T21:04:35Z
Touch-up comment.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
