dwsmith1983 commented on code in PR #6041: URL: https://github.com/apache/datafusion-comet/pull/6041#discussion_r4053650871
########## docs/source/user-guide/latest/compatibility/index.md: ########## @@ -145,6 +145,12 @@ so users hunting an unexpected value have a single place to check: - Native `RANGE` window frames with an explicit `PRECEDING` / `FOLLOWING` offset diverge from Spark when the boundary arithmetic overflows for `DATE` or `DECIMAL` `ORDER BY` columns ([#5022](https://github.com/apache/datafusion-comet/issues/5022)). +- Ungrouped decimal `SUM` keeps an unbounded intermediate and checks the result precision only + when a partial is written out or the sum is evaluated, which matches Spark's whole-stage codegen + path. With `spark.sql.codegen.wholeStage=false` Spark's ungrouped aggregate buffers in an + `UnsafeRow` and latches as soon as a running sum leaves the precision, so an intermediate + overflow that later cancels out returns `NULL` (or raises under ANSI) in Spark but the recovered + value in Comet. Spark's own decimal overflow tests accept either outcome. Review Comment: > Could we remove this claim? Spark's `assertDecimalSumOverflow` accepts two overflow error messages under ANSI; it still requires NULL in legacy mode. Removed in e5bd4cc, and the bullet now describes the narrower remaining divergence after the codegen guard. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
