Varadharajan created SPARK-13612:
------------------------------------
Summary: Multiplication of BigDecimal columns not working as
expected
Key: SPARK-13612
URL: https://issues.apache.org/jira/browse/SPARK-13612
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.0
Reporter: Varadharajan
Please consider the below snippet:
{code}
case class AM(id: Int, a: BigDecimal)
case class AX(id: Int, b: BigDecimal)
val x = sc.parallelize(List(AM(1, 10))).toDF
val y = sc.parallelize(List(AX(1, 10))).toDF
x.join(y, x("id") === y("id")).withColumn("z", x("a") * y("b")).show
{code}
output:
{code}
| id| a| id| b| z|
| 1|10.00000000000000...| 1|10.00000000000000...|null|
{code}
Here the multiplication of the columns ("z") return null instead of 100.
As of now we are using the below workaround, but definitely looks like a
serious issue.
{code}
x.join(y, x("id") === y("id")).withColumn("z", x("a") / (expr("1") /
y("b"))).show
{code}
{code}
| id| a| id| b| z|
| 1|10.00000000000000...| 1|10.00000000000000...|100.0000000000000...|
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
