viirya commented on a change in pull request #28898:
URL: https://github.com/apache/spark/pull/28898#discussion_r448799188
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala
##########
@@ -152,7 +168,12 @@ object NestedColumnAliasing {
val exclusiveAttrSet = AttributeSet(exclusiveAttrs ++ otherRootReferences)
val aliasSub = nestedFieldReferences.asInstanceOf[Seq[ExtractValue]]
.filter(!_.references.subsetOf(exclusiveAttrSet))
- .groupBy(_.references.head)
+ .groupBy(t => (t.references.head.exprId, t.references.head.dataType))
+ // The above groupBy is to avoid situation like the following situation.
+ // For example, `exprIdA -> List(a, b)` and `exprIdA -> List(c, d)`
Review comment:
I'm not sure about what `exprIdA -> List(a, b)` means? I think we need
to explain that attribute might have different qualifiers so we cannot well
group up semantically equal attributes here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
