GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/23035
[SPARK-26054][SQL] Transform also analyzed plans when dedup references
## What changes were proposed in this pull request?
In SPARK-24865 `AnalysisBarrier` was removed and in order to improve
resolution speed, the `analyzed` flag was (re-)introduced in order to process
only plans which are not yet analyzed. This should not be the case when
performing attribute deduplication as in that case we need to transform also
the plans which were already analyzed, otherwise we can miss to rewrite some
attributes leading to invalid plans.
## How was this patch tested?
added UT
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/spark SPARK-26057
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/23035.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #23035
----
commit 62a895febb8a91ce578133528a31060fe44981f9
Author: Marco Gaido <marcogaido91@...>
Date: 2018-11-14T15:16:37Z
[SPARK-26054][SQL] Trasform also analyzed plans when dedup references
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
