GitHub user wzhfy opened a pull request:
https://github.com/apache/spark/pull/17428
[SPARK-20094][SQL] Don't put predicate with subquery into join condition in
ReorderJoin because it fails RewritePredicateSubquery.rewriteExistentialExpr
## What changes were proposed in this pull request?
`ReorderJoin` collects all predicates and try to put them into join
condition when creating ordered join. If a predicate with a subquery is in a
join condition instead of a filter condition,
`RewritePredicateSubquery.rewriteExistentialExpr` would fail to convert the
subquery to an `ExistenceJoin`, and thus result in error.
## How was this patch tested?
Add a new test case in `JoinOptimizationSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wzhfy/spark noSubqueryInJoinCond
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17428.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17428
----
commit bd91947aaaa49050da59fb6caf704e572f842822
Author: wangzhenhua <[email protected]>
Date: 2017-03-25T07:19:42Z
don't put predicate with subquery into join condition in ReorderJoin
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
