GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/13884
[SPARK-16181][SQL: outer join with isNull filter may return wrong result
## What changes were proposed in this pull request?
The root cause is: the output attributes of outer join are derived from its
children, while they are actually different attributes(outer join can return
null).
We have already added some special logic to handle it, e.g.
`PushPredicateThroughJoin` won't push down predicates through outer join side,
`FixNullability`.
This PR adds one more special logic in `FoldablePropagation`.
## How was this patch tested?
new test in `DataFrameSuite`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark bug
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13884.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13884
----
commit 9316d7f0baec6d59e8a5a88cd872eca3e6720f9d
Author: Wenchen Fan <[email protected]>
Date: 2016-06-24T04:48:58Z
fix bug
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
