mgaido91 opened a new pull request #25347: [SPARK-28610][SQL] Allow having a decimal buffer for long sum URL: https://github.com/apache/spark/pull/25347 ## What changes were proposed in this pull request? If `spark.sql.arithmeticOperations.failOnOverFlow` is `true` temporary overflows in sum of long values cause an exception. This is against what other RDBMS do. The root cause is that we use a long attribute to store the intermediate result of a sum of longs: so if there is an intermediate value which is our of the range representable by long, an exception is raised. The PR introduces a flag which allows to control the datatype of the intermediate attribute of sum of long. When the flag is set to `true`, intermediate buffer is a decimal, so temporary overflows don't cause any issue. The flag has been introduces on @cloud-fan 's suggestion because as he pointed out, using always a decimal as intermediate buffer would cause a performance regression. ## How was this patch tested? Added UT
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
