Github user juliuszsompolski commented on a diff in the pull request:
https://github.com/apache/spark/pull/19324#discussion_r141037588
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
---
@@ -462,18 +464,36 @@ case class HashAggregateExec(
$evaluateAggResults
${consume(ctx, resultVars)}
"""
-
} else if (modes.contains(Partial) || modes.contains(PartialMerge)) {
- // This should be the last operator in a stage, we should output
UnsafeRow directly
- val joinerTerm = ctx.freshName("unsafeRowJoiner")
- ctx.addMutableState(classOf[UnsafeRowJoiner].getName, joinerTerm,
- s"$joinerTerm = $plan.createUnsafeJoiner();")
- val resultRow = ctx.freshName("resultRow")
+ // resultExpressions are Attributes of groupingExpressions and
aggregateBufferAttributes.
+ assert(resultExpressions.forall(_.isInstanceOf[Attribute]))
+ assert(resultExpressions.length ==
+ groupingExpressions.length + aggregateBufferAttributes.length)
--- End diff --
Final/Complete aggregations can have arbitrary projections in their
`resultExpressions`, while partial aggregations are always constructed with
only the grouping keys and aggregate expressions. The code that was here before
with the UnsafeRowJoiner was using this assumption, so now I put it into
assertion.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
