AMC-hawk commented on code in PR #58760:
URL: https://github.com/apache/spark/pull/58760#discussion_r4057443964
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala:
##########
@@ -687,9 +687,12 @@ trait ColumnResolutionHelper extends Logging with
DataTypeErrorsBase {
// ancestor plan (e.g. a natural/USING join wrapper that hides a join key
// via `Project.hiddenOutputTag`). We accept that here but tag the
candidate
// as `hidden` so the top-level merge in `resolveDataFrameColumn` can
prefer
- // a regular (p.output) match over hidden (p.metadataOutput) ones.
+ // a regular (p.output) match over hidden (p.metadataOutput) ones. An
+ // attribute can be in both, e.g. under the SQL pipe SET operator, and then
+ // counts as a regular match.
val filtered = candidates.flatMap { c =>
- val hidden = c.hidden ||
c.expr.references.subsetOf(AttributeSet(p.metadataOutput))
+ val hidden = c.hidden || (!c.expr.references.subsetOf(p.outputSet) &&
Review Comment:
Agreed, the earlier test was redundant. Replaced in `262b18bbe7d`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
