[
https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618892#comment-16618892
]
Marco Gaido commented on SPARK-22036:
-------------------------------------
[~bersprockets] first of all thank you for reporting this and sorry for my
mistake on this.
I think the solution you are suggesting isn't the right one. Also the result in
the case allowPrecisionLoss=true should not have any truncation here. The
problem is the way we handle negative scale. So this issue I think is related
to SPARK-24468. The problem is that Hive and MSSQL we are taking our rules from
are not allowing negative scale, while we do. So this has to be revisited. May
you please submit a new JIRA for this? Meanwhile I am starting working on it
and I'll submit a fix ASAP. Sorry for the trouble. Thanks.
> BigDecimal multiplication sometimes returns null
> ------------------------------------------------
>
> Key: SPARK-22036
> URL: https://issues.apache.org/jira/browse/SPARK-22036
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Olivier Blanvillain
> Assignee: Marco Gaido
> Priority: Major
> Fix For: 2.3.0
>
>
> The multiplication of two BigDecimal numbers sometimes returns null. Here is
> a minimal reproduction:
> {code:java}
> object Main extends App {
> import org.apache.spark.{SparkConf, SparkContext}
> import org.apache.spark.sql.SparkSession
> import spark.implicits._
> val conf = new
> SparkConf().setMaster("local[*]").setAppName("REPL").set("spark.ui.enabled",
> "false")
> val spark =
> SparkSession.builder().config(conf).appName("REPL").getOrCreate()
> implicit val sqlContext = spark.sqlContext
> case class X2(a: BigDecimal, b: BigDecimal)
> val ds = sqlContext.createDataset(List(X2(BigDecimal(-0.1267333984375),
> BigDecimal(-1000.1))))
> val result = ds.select(ds("a") * ds("b")).collect.head
> println(result) // [null]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
