skambha commented on a change in pull request #27627: [WIP][SPARK-28067][SQL]
Fix incorrect results for decimal aggregate sum by returning null on decimal
overflow
URL: https://github.com/apache/spark/pull/27627#discussion_r380939010
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Sum.scala
##########
@@ -60,38 +60,104 @@ case class Sum(child: Expression) extends
DeclarativeAggregate with ImplicitCast
private lazy val sumDataType = resultType
private lazy val sum = AttributeReference("sum", sumDataType)()
+ private lazy val overflow = AttributeReference("overflow", BooleanType,
false)()
private lazy val zero = Literal.default(resultType)
- override lazy val aggBufferAttributes = sum :: Nil
+ override lazy val aggBufferAttributes = sum :: overflow :: Nil
override lazy val initialValues: Seq[Expression] = Seq(
- /* sum = */ Literal.create(null, sumDataType)
+ /* sum = */ Literal.create(null, sumDataType),
+ /* overflow = */ Literal.create(false, BooleanType)
)
override lazy val updateExpressions: Seq[Expression] = {
- if (child.nullable) {
+ if (!SQLConf.get.ansiEnabled) {
+ if (child.nullable) {
+ Seq(
+ /* sum = */
+ coalesce(coalesce(sum, zero) + child.cast(sumDataType), sum),
Review comment:
The changes in the Sum are mostly to check if overflow has occurred when we
do the different additions in the updateExpressions and mergeExpressions. The
actual addition operations are all the same. Reading the diff may not show
that easily so wanted to make a note here on that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
