Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20965#discussion_r183092814
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
---
@@ -266,28 +342,53 @@ case class HashAggregateExec(
e.aggregateFunction.asInstanceOf[DeclarativeAggregate].mergeExpressions
}
}
- ctx.currentVars = bufVars ++ input
+
+ // We need to copy the aggregation buffer to local variables first
because each aggregate
+ // function directly updates the buffer when it finishes.
+ val localBufVars = bufVars.zip(updateExpr).map { case (ev, e) =>
--- End diff --
but this may introduce a perf regression, doesn't it? in this case I am not
sure if we should do it or not, WDYT @cloud-fan @gatorsmile @hvanhovell ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
