sunchao opened a new issue, #7116: URL: https://github.com/apache/datafusion-comet/issues/7116
### What problem does this solve? Keep high-precision ungrouped decimal AVG/TRY_AVG and ever-expanding decimal AVG windows native where Spark retains a wide temporary sum until division. PR #5420 repairs empty partials, sticky overflow, and division scaling, but conservatively falls back when the input precision is at least 28. Aggregate fallback also moves sibling MIN/MAX/COUNT/SUM expressions to Spark. This is the follow-up requested in https://github.com/apache/datafusion-comet/pull/5420#discussion_r4189976294. #6041 established a similar design for decimal SUM: its scalar accumulator keeps an i256 running sum and checks precision at state export/evaluation, while the planner declines ungrouped execution without codegen because Spark's UnsafeRow buffer latches overflow earlier. ### Proposed work - Give scalar AvgDecimalAccumulator a wide running sum, checking overflow at the boundaries required by Spark's AVG partial state and final division. - Keep ungrouped maximum-precision AVG in Spark when codegen is disabled; preserve restrictions for grouped object/sort aggregation and unsafe decimal mixed-engine buffers. - Revisit the ever-expanding window guard once the scalar accumulator matches Spark for cancellation after an intermediate overflow. - Keep the grouped accumulator's sticky overflow behavior, which models Spark's UnsafeRow aggregation buffer. ### Validation Cover empty/all-null input, cancellation such as DECIMAL(38,38) values 0.6, 0.6, -0.4, partial-state export/merge, ANSI/legacy/TRY_AVG, codegen on/off, shuffled and unshuffled global aggregates, and ever-expanding windows. Include sibling aggregate route assertions and a benchmark comparing the current fallback with the native path. Verify the arithmetic bounds of i256 intermediates, including division scaling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
