cloud-fan opened a new pull request, #59764: URL: https://github.com/apache/spark/pull/59764
### What changes were proposed in this pull request? This is a followup of https://github.com/apache/spark/pull/57742. This PR makes two changes to adaptive partial aggregation: - Renames the `HashAggregateExec` SQL metric `numBypassingRows` ("number of bypassing rows") to `numPassthroughRows` ("passthrough output rows"). - Stops timing pass-through rows into `aggTime` ("time in aggregation build"), on both the codegen path (`HashAggregateExec`) and the interpreted path (`TungstenAggregationIterator`). `aggTime` again covers only the build up to the point where pass-through starts. ### Why are the changes needed? Passthrough is a better name than bypass for this metric. It counts the rows that the operator passes through as single-row partial buffers, which is also what the implementation calls them. The pass-through timers cost more than they measure: - On the codegen path, when an exchange splits the plan, every passed-through row returns from `processNext` and re-enters the resumed build, so the timer around it calls `System.nanoTime()` twice per row. Each slice is far below a millisecond, and the integer millisecond division truncates it to 0, so the timer adds nothing to the metric. - On the interpreted path, rows are pulled lazily through `next()`, so timing from the first to the last pass-through row also counts the downstream work between the calls, such as the shuffle write. ### Does this PR introduce _any_ user-facing change? No. Adaptive partial aggregation is new in the unreleased 4.4.0 and disabled by default. Within the unreleased branches, when the feature is enabled, the SQL UI shows the metric as "passthrough output rows", and "time in aggregation build" no longer includes pass-through rows. ### How was this patch tested? Existing tests. `AdaptivePartialAggregationSuite` now reads the renamed metric, and `build/sbt "sql/testOnly *AdaptivePartialAggregationSuite"` passes. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
