Smallfu666 commented on issue #5098: URL: https://github.com/apache/datafusion-comet/issues/5098#issuecomment-5168874368
I benchmarked the straightforward implementation proposed here: cast each input batch to `Int64`, then use Arrow's sum kernel for scalar aggregation and a per-element loop over the widened `Int64Array` to accumulate per-group sums for grouped aggregation. Across Int8/Int16/Int32/Int64, scalar/grouped, and Legacy/ANSI/Try configurations (Criterion, 100 samples, repeated in two independent runs), the widened-array approach consistently regressed narrow-integer performance: approximately 75–322% for scalar aggregation and 40–97% for grouped aggregation. Int64 was roughly neutral. The main cost appears to be materializing and writing a widened `Int64Array` for every batch, which the current per-width loops avoid. Given that this trades a substantial performance regression for dispatch cleanup, I am dropping this implementation and unassigning myself. A viable follow-up would likely need either a zero-allocation widening reduction that accumulates narrow inputs directly into `i64`, or a dispatch-deduplication refactor that retains the current per-width loops. I can provide the benchmark matrix if useful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
