Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19488#discussion_r145281452
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
---
@@ -235,8 +236,16 @@ object PhysicalAggregation {
expr.transformDown {
case ae: AggregateExpression =>
// The final aggregation buffer's attributes will be
`finalAggregationAttributes`,
- // so replace each aggregate expression by its corresponding
attribute in the set:
- ae.resultAttribute
+ // so replace each aggregate expression by its corresponding
attribute in the set.
+ // Note that non-deterministic aggregate expressions should
not be deduplicated and
+ // should be handled differently.
+ val newAe = if (ae.deterministic) {
+ equivalentAggregateExpressions.getEquivalentExprs(ae)
+ .head.asInstanceOf[AggregateExpression]
+ } else {
+ ae
+ }
--- End diff --
->
`equiExprs.getEquivalentExprs(ae).headOption.getOrElse(ae).asInstanceOf[AggregateExpression]`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
