GitHub user ron8hu opened a pull request:
https://github.com/apache/spark/pull/17415
[SPARK-19408][SQL] filter estimation on two columns of same table
## What changes were proposed in this pull request?
In SQL queries, we also see predicate expressions involving two columns
such as "column-1 (op) column-2" where column-1 and column-2 belong to same
table. Note that, if column-1 and column-2 belong to different tables, then it
is a join operator's work, NOT a filter operator's work. This PR estimates
filter selectivity on two columns of same table.
## How was this patch tested?
We added 6 new test cases to test various logical predicates involving two
columns of same table.
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ron8hu/spark filterTwoColumns
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17415.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17415
----
commit 893066905b690c78a127eae58b908dff1dabf7cf
Author: Ron Hu <[email protected]>
Date: 2017-03-24T20:31:35Z
filter estimation on two columns of same table
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
