karuppayya commented on a change in pull request #28804:
URL: https://github.com/apache/spark/pull/28804#discussion_r466749506
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
##########
@@ -750,18 +768,19 @@ case class HashAggregateExec(
finishRegularHashMap
}
+ outputFunc = generateResultFunction(ctx)
val doAggFuncName = ctx.addNewFunction(doAgg,
s"""
|private void $doAgg() throws java.io.IOException {
| ${child.asInstanceOf[CodegenSupport].produce(ctx, this)}
+ | $childrenConsumed = true;
Review comment:
Before my change, All the records of the the dataset were processed and
HashMap populated(+ spilt to sorter) at once.
With my change, all the records are not processed at once due to the change
at line#878
Hence we need to know if all the records of the dataset is processed,
otherwise process them.
This variable serves the purpose.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
