GitHub user marmbrus opened a pull request:
https://github.com/apache/spark/pull/5062
[SPARK-6247][SQL] Fix resolution of ambiguous joins caused by new aliases
We need to handle ambiguous `exprId`s that are produced by new aliases as
well as those caused by leaf nodes (`MultiInstanceRelation`).
Attempting to fix this revealed a bug in `equals` for `Alias` as these
objects were comparing equal even when the expression ids did not match.
Additionally, `LocalRelation` did not correctly provide statistics, and some
tests in `catalyst` were not using the helper functions for comparing plans.
Based on #4991 by @chenghao-intel
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marmbrus/spark selfJoins
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5062.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5062
----
commit 898af7313156b0bcdec4cd4ea9ef3344cba4c602
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T04:40:20Z
Fix Alias equality.
commit f9c67c292366b414cf12292b001eb98942d3ad8b
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T04:40:57Z
Check for duplicate attributes in join resolution.
commit d8caa40f12dcca41657518bf80475cd905c8cf81
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T04:47:18Z
test case: SPARK-6247
commit b022ef7df91303a629f9db01d8eb84e552f49cf0
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T04:48:01Z
Handle project aliases.
commit a925d08d79bea2053566393db9b7ec0d884fb4af
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T04:48:25Z
check for conflicting attributes in join resolution
commit 725f1ab7ea669663273c1985e871a5d1288e8567
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T04:51:57Z
add statistics
commit c3c574b52ab77af4556122b91b4ba3a9f7188d9d
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T04:54:06Z
revert change.
commit 0b9c6879a69a6b2615781d25f21f7f2f0fc69922
Author: Michael Armbrust <[email protected]>
Date: 2015-03-17T05:24:28Z
fix more tests
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
