GitHub user saucam opened a pull request:
https://github.com/apache/spark/pull/3618
SPARK-4762: Add support for tuples in 'where in' clause query
Currently, in the where in clause the filter is applied only on a single
column. We can enhance it to accept filter on multiple columns.
So current support is for queries like :
Select * from table where c1 in (value1,value2,...value n);
This added support for queries like :
Select * from table where (c1,c2,... cn) in ((value1,value2...value n),
(value1' , value2' ... ,value n').... )
Also, added optimized version of where in clause of tuples , where we
create a hashset of the filter tuples for matching rows.
This also requires a change in the hive parser since currently there is no
support for multiple columns in IN clause.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/saucam/spark tuple_where_clause
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3618.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3618
----
commit c877926c64c7c6f2048d31759f35446c9cec1cdc
Author: Yash Datta <[email protected]>
Date: 2014-12-05T08:55:29Z
SPARK-4762: 1. Add support for tuples in 'where in' clause query
2. Also adds optimized version of the same, which uses hashset
to filter rows
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
