GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/15761
[SPARK-17337][SQL] Do not pushdown predicates through filters with
predicate subqueries [WIP]
## What changes were proposed in this pull request?
The `PushDownPredicate` rule can create a wrong result if we try to push a
filter containing a predicate subquery through a project when the subquery and
the project share attributes (have the same source).
The current PR fixes this by such a pushdown, this is however a crude fix.
I'd like to spend a little more time to see if I can come up with something
more subtle.
## How was this patch tested?
Added a test to `SubquerySuite`. @nsyca has done previous work this. I have
taken test from his initial PR.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hvanhovell/spark SPARK-17337
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15761.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15761
----
commit 6993c50974ba456593d7d38f63ebb04af02e8182
Author: Herman van Hovell <[email protected]>
Date: 2016-11-03T23:44:26Z
Do not pushdown predicates through filters containing predicate subqueries.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
