Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/6814#discussion_r32810151
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala ---
@@ -261,7 +261,7 @@ final class Decimal extends Ordered[Decimal] with
Serializable {
def - (that: Decimal): Decimal = Decimal(toBigDecimal -
that.toBigDecimal)
- def * (that: Decimal): Decimal = Decimal(toBigDecimal *
that.toBigDecimal)
+ def * (that: Decimal): Decimal =
Decimal(toJavaBigDecimal.multiply(that.toJavaBigDecimal))
--- End diff --
`toBigDecimal` just creates scala `BigDecimal` with its information. I
think it is correct. The problem looks like the scala `BigDecimal` produces
wrong result, compared with java `BigDecimal` when doing this multiplication.
To show that, we can create a scala `BigDecimal`. We find that it has
correct precision as same as its underlying java `BigDecimal`:
scala> val d = BigDecimal(Long.MaxValue, 0)
d: scala.math.BigDecimal = 9223372036854775807
scala> d.precision
res16: Int = 19
scala> d.underlying.precision
res17: Int = 19
scala> d.scale
res18: Int = 0
scala> d.underlying.scale
res19: Int = 0
When we multiply two scala `BigDecimal` carrying `Long.MaxValue`, we get
wrong result:
scala> val t = BigDecimal(Long.MaxValue, 0) *
BigDecimal(Long.MaxValue, 0)
t: scala.math.BigDecimal = 8.507059173023461584739690778423250E+37
scala> t.precision
res20: Int = 34
scala> t.scale
res21: Int = -4
scala> t.underlying.unscaledValue.toString
res22: String = 8507059173023461584739690778423250
When we multiply two java `BigDecimal` carrying `Long.MaxValue`, the result
is correct:
scala> val j = d.underlying.multiply(d.underlying)
j: java.math.BigDecimal = 85070591730234615847396907784232501249
scala> j.precision
res23: Int = 38
scala> j.scale
res24: Int = 0
scala> j.toString
res25: String = 85070591730234615847396907784232501249
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
