[
https://issues.apache.org/jira/browse/SPARK-8800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-8800:
-----------------------------------
Assignee: (was: Apache Spark)
> Spark SQL Decimal Division operation loss of precision/scale when type is
> defined as DecimalType.Unlimited
> ----------------------------------------------------------------------------------------------------------
>
> Key: SPARK-8800
> URL: https://issues.apache.org/jira/browse/SPARK-8800
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Jihong MA
>
> According to specification defined in Java doc over BigDecimal :
> http://docs.oracle.com/javase/1.5.0/docs/api/java/math/BigDecimal.html
> When a MathContext object is supplied with a precision setting of 0 (for
> example, MathContext.UNLIMITED), arithmetic operations are exact, as are the
> arithmetic methods which take no MathContext object. (This is the only
> behavior that was supported in releases prior to 5.) As a corollary of
> computing the exact result, the rounding mode setting of a MathContext object
> with a precision setting of 0 is not used and thus irrelevant. In the case of
> divide, the exact quotient could have an infinitely long decimal expansion;
> for example, 1 divided by 3. If the quotient has a nonterminating decimal
> expansion and the operation is specified to return an exact result, an
> ArithmeticException is thrown. Otherwise, the exact result of the division is
> returned, as done for other operations.
> when Decimal data is defined as DecimalType.Unlimited in Spark SQL, the exact
> result of the division should be returned or truncated to precision = 38
> which is in align with what Hive supports. the current behavior is as shown
> following, which cause we lose the accuracy of Decimal division operation.
> scala> val aa = Decimal(2) / Decimal(3);
> aa: org.apache.spark.sql.types.Decimal = 1
> here is another example where we should return 0.125 instead of 0
> scala> val aa = Decimal(1) /Decimal(8)
> aa: org.apache.spark.sql.types.Decimal = 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
