This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push: new f96fc06ca4a [SPARK-39887][SQL][FOLLOW-UP] Do not exclude Union's first child attributes when traversing other children in RemoveRedundantAliases f96fc06ca4a is described below commit f96fc06ca4a5a1bbc154aca1c7a9b96e7fc42c7e Author: Peter Toth <pt...@cloudera.com> AuthorDate: Wed Aug 17 14:57:35 2022 +0800 [SPARK-39887][SQL][FOLLOW-UP] Do not exclude Union's first child attributes when traversing other children in RemoveRedundantAliases ### What changes were proposed in this pull request? Do not exclude `Union`'s first child attributes when traversing other children in `RemoveRedundantAliases`. ### Why are the changes needed? We don't need to exclude those attributes that `Union` inherits from its first child. See discussion here: https://github.com/apache/spark/pull/37496#discussion_r944509115 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing UTs. Closes #37534 from peter-toth/SPARK-39887-keep-attributes-of-unions-first-child-follow-up. Authored-by: Peter Toth <pt...@cloudera.com> Signed-off-by: Wenchen Fan <wenc...@databricks.com> (cherry picked from commit e732232dac420826af269d8cf5efacb52933f59a) Signed-off-by: Wenchen Fan <wenc...@databricks.com> --- .../scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala index 7993829f1f6..6797bbfc714 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala @@ -536,7 +536,7 @@ object RemoveRedundantAliases extends Rule[LogicalPlan] { }) Join(newLeft, newRight, joinType, newCondition, hint) - case _: Union => + case u: Union => var first = true plan.mapChildren { child => if (first) { @@ -547,7 +547,8 @@ object RemoveRedundantAliases extends Rule[LogicalPlan] { // output attributes could return incorrect result. removeRedundantAliases(child, excluded ++ child.outputSet) } else { - removeRedundantAliases(child, excluded) + // We don't need to exclude those attributes that `Union` inherits from its first child. + removeRedundantAliases(child, excluded -- u.children.head.outputSet) } } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org