[ https://issues.apache.org/jira/browse/SPARK-28224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takeshi Yamamuro resolved SPARK-28224. -------------------------------------- Fix Version/s: 3.0.0 Assignee: Mick Jermsurawong Resolution: Fixed Resolved by [https://github.com/apache/spark/pull/25033|https://github.com/apache/spark/pull/25033#] > Check overflow in decimal Sum aggregate > --------------------------------------- > > Key: SPARK-28224 > URL: https://issues.apache.org/jira/browse/SPARK-28224 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Mick Jermsurawong > Assignee: Mick Jermsurawong > Priority: Major > Fix For: 3.0.0 > > > To reproduce: > {code:java} > import spark.implicits._ > val ds = spark > .createDataset(Seq(BigDecimal("1" * 20), BigDecimal("9" * 20))) > .agg(sum("value")) > .as[BigDecimal] > ds.collect shouldEqual Seq(null){code} > Given the option to throw exception on overflow on, sum aggregation of > overflowing bigdecimal still remain null. {{DecimalAggregates}} is only > invoked when expression of the sum (not the elements to be operated) has > sufficiently small precision. The fix seems to be in Sum expression itself. > -- This message was sent by Atlassian Jira (v8.3.2#803003) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org