AMC-hawk commented on code in PR #58760:
URL: https://github.com/apache/spark/pull/58760#discussion_r4057435519
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##########
@@ -2085,6 +2086,45 @@ class Analyzer(
}.map(_.asInstanceOf[NamedExpression])
}
+ /**
+ * The SQL pipe SET operator is implemented as a star expansion that
excludes the assigned
+ * column and appends a replacement of the same name. That drops the
original attribute from
+ * the project list, which would also make it unreachable through its
table alias, contradicting
+ * the documented behavior that table aliases keep referring to the
original row values after an
+ * assignment. Retain the excluded attributes as hidden output instead,
the same way USING joins
+ * hide their duplicated join keys (SPARK-59146).
+ *
+ * The hidden output holds the whole qualified source row in its original
order, not just the
+ * excluded attributes, because a qualified star expands hidden output
ahead of the visible
+ * output. Retaining only the excluded attributes would move them to the
front, so that
+ * `VALUES (1, 2, 3) AS t(a, b, c) |> SET b = 20 |> SELECT t.*` returned
`(b, a, c)`. Attributes
+ * that are also visible are emitted once by the star expansion, in their
hidden position.
+ */
+ private def retainExceptedColumnsAsHiddenOutput(original: Project,
expanded: Project): Unit = {
+ val retain = original.projectList.exists {
+ case s: UnresolvedStarExceptOrReplace =>
s.retainExceptedColumnsAsHidden
+ case _ => false
+ }
+ if (retain) {
+ val child = expanded.child
+ if (child.output.exists(!expanded.outputSet.contains(_))) {
+ // The row a qualified star sees on the child: its qualified-only
hidden output first,
+ // then its output. Only qualified attributes are reachable through
a qualified star.
+ val sourceRow = (child.metadataOutput.filter(_.qualifiedAccessOnly)
++ child.output)
+ .filter(_.qualifier.nonEmpty)
+ .distinctBy(_.exprId)
Review Comment:
@cloud-fan Done, Fixed in `262b18bbe7d`
`distinctBy(_.exprId)` and the set-wide visible suppression are gone. t.*
now returns `(a, a, b) / 1 1 2`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
