mgaido91 commented on a change in pull request #25035: [SPARK-28235][SQL] Sum
of decimals should return a decimal with MAX_PRECISION
URL: https://github.com/apache/spark/pull/25035#discussion_r300079905
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecisionSuite.scala
##########
@@ -86,7 +86,7 @@ class DecimalPrecisionSuite extends AnalysisTest with
BeforeAndAfter {
checkType(Divide(d2, d1), DecimalType(10, 6))
checkType(Remainder(d1, d2), DecimalType(3, 2))
checkType(Remainder(d2, d1), DecimalType(3, 2))
- checkType(Sum(d1), DecimalType(12, 1))
+ checkType(Sum(d1), DecimalType(DecimalType.MAX_PRECISION, 1))
Review comment:
Actually the UT failures pointed out an issue. The `DecimalAggregates`
optimization works in case the resulting type has a precision lower than the
maximum number of precision digits for a long. Keeping the rule as is, just
returning a wider data type means that applying that rule can cause a
difference in the output of the query, which means that it is not a valid
optimization rule anymore.
It is true, though, that we wouldn't have regressions, just some cases which
were returning `null` would still return `null` unless that rule is excluded.
What do you think on this? Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
