GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/16757
[SPARK-18609][SQL] Fix redundant Alias removal in the optimizer
## What changes were proposed in this pull request?
The optimizer tries to remove redundant alias only projections from the
query plan using the `RemoveAliasOnlyProject` rule. The current rule identifies
removes such a project and rewrites the project's attributes in the **entire**
tree. This causes problems when parts of the tree are duplicated (for instance
a self join on a temporary view/CTE) and the duplicated part contains the
alias only project, in this case the rewrite will break the tree.
[Solution] TODO
It was difficult to control both the blacklisted attributes, the
transformation of the tree, and the to keep the rewrite local to a node's
parents. I have made a few changes to `TreeNode`, `QueryPlan` and
`LogicalPlan` to open up the transformation logic which allows us to have (the
needed) more fine grained control over tree transformations.
## How was this patch tested?
Added a test to `RemoveRedundantAliasAndProjectSuite` and existing tests. I
will add some more integration tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hvanhovell/spark SPARK-18609
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16757.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16757
----
commit 6c89a15ed8eb868b23237bba07498fb2053f4643
Author: Herman van Hovell <[email protected]>
Date: 2017-01-30T12:11:46Z
Open-up TreeNode's transform logic.
commit dac7ec99075ce98ebea92e108ad66b05537de396
Author: Herman van Hovell <[email protected]>
Date: 2017-01-31T16:03:57Z
Split RemoveAliasOnlyProject into RemoveRedundantAliases and
RemoveRedundantProject.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
