cloud-fan commented on code in PR #39773:
URL: https://github.com/apache/spark/pull/39773#discussion_r1091609918
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveLateralColumnAliasReference.scala:
##########
@@ -177,8 +195,18 @@ object ResolveLateralColumnAliasReference extends
Rule[LogicalPlan] {
val newAggExprs = collection.mutable.Set.empty[NamedExpression]
val expressionMap =
collection.mutable.LinkedHashMap.empty[Expression, NamedExpression]
- val projectExprs = aggregateExpressions.map { exp =>
- exp.transformDown {
+ // Extract the expressions to keep in the Aggregate. Return the
transformed expression
+ // fully substituted with the attribute reference to the extracted
expressions.
+ def extractExpressions(expr: Expression): Expression = {
+ expr match {
+ case w @ WindowExpression(function, spec) =>
+ // Manually skip the handling on the function itself but
iterate on its children
+ // instead. This is to avoid extracting a window expression's
aggregate functions.
+ // For example, for `sum(sum(col1)) over (..)`, we should
extract sum(col1) but not
Review Comment:
Maybe we should be more explicit: `WindowExpression.function` can be
`AggregateExpression`, but we don't want to extract that `AggregateExpression`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
