cloud-fan commented on code in PR #59758:
URL: https://github.com/apache/spark/pull/59758#discussion_r4366321770
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala:
##########
@@ -918,11 +918,22 @@ case class HashAggregateExec(
// For adaptive partial aggregation pass-through, each bypassed row is
emitted as a single-row
// partial buffer: start from the initial aggregation buffer, apply the
update expressions once,
// and output `key ++ buffer` for the Final aggregate to merge. This
projects the initial
- // buffer.
- val emptyAggBufferCode = if (adaptivePartialAggEnabled) {
- GenerateUnsafeProjection.createCode(ctx, declFunctions.flatMap(f =>
f.initialValues))
+ // buffer, in its own method: the projection's code grows with the number
of buffer slots, and
Review Comment:
Updated the PR description. It no longer claims a JIT-limit effect; it
describes the change as a constant-size reset that matches the interpreted
path, without one copy per Union child.
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala:
##########
@@ -918,11 +918,22 @@ case class HashAggregateExec(
// For adaptive partial aggregation pass-through, each bypassed row is
emitted as a single-row
// partial buffer: start from the initial aggregation buffer, apply the
update expressions once,
// and output `key ++ buffer` for the Final aggregate to merge. This
projects the initial
- // buffer.
- val emptyAggBufferCode = if (adaptivePartialAggEnabled) {
- GenerateUnsafeProjection.createCode(ctx, declFunctions.flatMap(f =>
f.initialValues))
+ // buffer, in its own method: the projection's code grows with the number
of buffer slots, and
+ // inlining it into the per-row consume method could push a wide
aggregation's consume method
+ // past the JIT's huge-method limit.
+ val makeEmptyAggBufferFunc = if (adaptivePartialAggEnabled) {
Review Comment:
No longer applies: the generated method is gone, and the two new fields
follow the file's `*Term` naming for mutable states.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
