Philip Dodds created SPARK-12128: ------------------------------------ Summary: Multiplication on decimals in dataframe returns null Key: SPARK-12128 URL: https://issues.apache.org/jira/browse/SPARK-12128 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.5.2, 1.5.1, 1.5.0 Environment: Scala 2.11/Spark 1.5.0/1.5.1/1.5.2 Reporter: Philip Dodds
I hit a weird issue when I tried to multiply to decimals in a select (either in scala or as SQL), and Im assuming I must be missing the point. The issue is fairly easy to recreate with something like the following: {code:java} val sqlContext = new org.apache.spark.sql.SQLContext(sc) import sqlContext.implicits._ import org.apache.spark.sql.types.Decimal case class Trade(quantity: Decimal,price: Decimal) val data = Seq.fill(100) { val price = Decimal(20+scala.util.Random.nextInt(10)) val quantity = Decimal(20+scala.util.Random.nextInt(10)) Trade(quantity, price) } val trades = sc.parallelize(data).toDF() trades.registerTempTable("trades") trades.select(trades("price")*trades("quantity")).show sqlContext.sql("select price/quantity,price*quantity,price+quantity,price-quantity from trades").show {code} The odd part is if you run it you will see that the addition/division and subtraction works but the multiplication returns a null. Tested on 1.5.1/1.5.2 (Scala 2.10 and 2.11) ie. {code} +------------------+ |(price * quantity)| +------------------+ | null| | null| | null| | null| | null| +------------------+ +--------------------+----+--------------------+--------------------+ | _c0| _c1| _c2| _c3| +--------------------+----+--------------------+--------------------+ |0.952380952380952381|null|41.00000000000000...|-1.00000000000000...| |1.380952380952380952|null|50.00000000000000...|8.000000000000000000| |1.272727272727272727|null|50.00000000000000...|6.000000000000000000| |0.833333333333333333|null|44.00000000000000...|-4.00000000000000...| |1.000000000000000000|null|58.00000000000000...| 0E-18| +--------------------+----+--------------------+--------------------+ {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org