AngersZhuuuu commented on a change in pull request #28490:
URL: https://github.com/apache/spark/pull/28490#discussion_r480231511
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1583,6 +1611,35 @@ class Analyzer(
failAnalysis(s"Invalid usage of '*' in expression '${o.prettyName}'")
}
}
+
+ // For aggregateExpression we need NamedExpression, so for top level
Struct field
+ // Alias, we won't trim it, for non top level Struct field Alias, we need
to trim
+ // it since we have trimmed groupByExpressions.
+ def trimNonTopLevelStructFieldAlias(expr: Expression): NamedExpression = {
+ expr match {
+ case alias @ Alias(_: GetStructField, _) => alias
+ case e => trimStructFieldAlias(e).asInstanceOf[NamedExpression]
+ }
+ }
+
+ // For struct field, it will be resolve as Alias(GetStructField, name),
+ // In Aggregate/GroupingSets this behavior will cause the same struct
fields
+ // in aggExprs/groupExprs/selectedGroupByExprs be treated as different
ones due to different
+ // ExprIds in Alias, and stops us finding the grouping expressions in
aggExprs. Here we
+ // will use this method to remove Alias with different ExprId of
GetStructField
+ // in groupByAlias/selectedGroupByExprs and aggregateExpressions
+ def trimStructFieldAlias(e: Expression): Expression = {
+ var fixed = ArrayBuffer[Alias]()
+ e.transformDown {
+ case a @ Alias(Alias(struct, _), name) if
struct.isInstanceOf[GetStructField] =>
Review comment:
> ok things get complicated. How about we add a new flag to
resolveExpressionTopDown, so that we can strip alias only for resolved column?
> and Add top-level Alias manually after calling resolveExpressionTopDown.
Updated..really complicated...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
