cloud-fan commented on a change in pull request #27105: [SPARK-30433][SQL] Make
conflict attributes resolution more scalable in ResolveReferences
URL: https://github.com/apache/spark/pull/27105#discussion_r363265208
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1089,27 +1089,31 @@ class Analyzer(
.nonEmpty =>
(oldVersion, oldVersion.copy(windowExpressions =
newAliases(windowExpressions)))
}
- // Only handle first case, others will be fixed on the next pass.
- .headOption match {
- case None =>
- /*
- * No result implies that there is a logical plan node that produces
new references
- * that this rule cannot handle. When that is the case, there must
be another rule
- * that resolves these conflicts. Otherwise, the analysis will fail.
- */
- right
- case Some((oldRelation, newRelation)) =>
- val attributeRewrites =
AttributeMap(oldRelation.output.zip(newRelation.output))
- right transformUp {
- case r if r == oldRelation => newRelation
- } transformUp {
- case other => other transformExpressions {
- case a: Attribute =>
- dedupAttr(a, attributeRewrites)
- case s: SubqueryExpression =>
- s.withNewPlan(dedupOuterReferencesInSubquery(s.plan,
attributeRewrites))
- }
+
+ /*
+ * Note that it's possible for conflictPlans to be empty while it
implies that there
Review comment:
\`conflictPlans\` can be empty which implies that ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
