vranes commented on code in PR #59591:
URL: https://github.com/apache/spark/pull/59591#discussion_r4315738431
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/NaturalAndUsingJoinResolution.scala:
##########
@@ -84,6 +85,22 @@ object NaturalAndUsingJoinResolution extends
DataTypeErrorsBase with SQLConfHelp
(output, hiddenOutput, newCondition)
}
+ /**
+ * Wraps a NATURAL or USING join in a [[Project]] that hides `hiddenList`
(the duplicate join
+ * keys). It also passes up the hidden columns of nested NATURAL and USING
joins.
+ */
+ def buildUsingJoinProject(
+ projectList: Seq[NamedExpression],
+ hiddenList: Seq[Attribute],
+ join: LogicalPlan): Project = {
+ val project = Project(projectList, join)
+ project.setTagValue(
+ Project.hiddenOutputTag,
+ hiddenList.map(_.markAsQualifiedAccessOnly()) ++
+ join.metadataOutput.filter(_.qualifiedAccessOnly))
Review Comment:
nit: `join.metadataOutput` recurses into the child plans, and the
single-pass resolver also uses this object.
Can we take these columns as a parameter instead, like
`computeJoinOutputsAndNewCondition` takes `leftOutput`? The fixed-point callers
can then pass `join.metadataOutput`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
