GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/15772
[SPARK-17337][SQL] Do not pushdown predicates through filters with
predicate subqueries [BRANCH-2.0]
## What changes were proposed in this pull request?
The `PushDownPredicate` rule can create a wrong result if we try to push a
filter containing a predicate subquery through a project when the subquery and
the project share attributes (have the same source).
The current PR fixes this by making sure that we do not push down when
there is a predicate subquery that outputs the same attributes as the filters
new child plan.
## How was this patch tested?
Added a test to `SubquerySuite`. @nsyca has done previous work on this. I
have taken the test from his initial PR.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hvanhovell/spark SPARK-17337-2.0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15772.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15772
----
commit 898f7228af9b08eeebe285047f8126631102fbcb
Author: Herman van Hovell <[email protected]>
Date: 2016-11-04T20:18:13Z
[SPARK-17337][SQL] Do not pushdown predicates through filters with
predicate subqueries
## What changes were proposed in this pull request?
The `PushDownPredicate` rule can create a wrong result if we try to push a
filter containing a predicate subquery through a project when the subquery and
the project share attributes (have the same source).
The current PR fixes this by making sure that we do not push down when
there is a predicate subquery that outputs the same attributes as the filters
new child plan.
## How was this patch tested?
Added a test to `SubquerySuite`. nsyca has done previous work this. I have
taken test from his initial PR.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
