[
https://issues.apache.org/jira/browse/SPARK-41219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637216#comment-17637216
]
XiDuo You edited comment on SPARK-41219 at 11/22/22 11:59 AM:
--------------------------------------------------------------
it seems the root reason is decimal.toPrecision will break when change to
decimal(0, 0)
{code:java}
val df = Seq(0).toDF("a")
// return 0
df.selectExpr("cast(0 as decimal(0,0))").show
// reutrn 0
df.select(lit(BigDecimal(0)) as "c").show
// return null
df.select(lit(BigDecimal(0)) as "c").selectExpr("cast(c as decimal(0,0))").show
{code}
was (Author: ulysses):
it seems the root reason is decimal.toPrecision will break when change to
decimal(0, 0)
{code:java}
val df = Seq(0).toDF("a")
// return 0
df.selectExpr("cast(0 as decimal(0,0))").show
// return null
df.select(lit(BigDecimal(0)) as "c").selectExpr("cast(c as decimal(0,0))").show
{code}
> Regression in IntegralDivide returning null instead of 0
> --------------------------------------------------------
>
> Key: SPARK-41219
> URL: https://issues.apache.org/jira/browse/SPARK-41219
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: Raza Jafri
> Priority: Major
>
> There seems to be a regression in Spark 3.4 Integral Divide
>
> {code:java}
> scala> val df = Seq("0.5944910","0.3314242").toDF("a")
> df: org.apache.spark.sql.DataFrame = [a: string]
> scala> df.selectExpr("cast(a as decimal(7,7)) div 100").show
> +---------------------------------+
> |(CAST(a AS DECIMAL(7,7)) div 100)|
> +---------------------------------+
> | null|
> | null|
> +---------------------------------+
> {code}
>
> While in Spark 3.3.0
> {code:java}
> scala> val df = Seq("0.5944910","0.3314242").toDF("a")
> df: org.apache.spark.sql.DataFrame = [a: string]
> scala> df.selectExpr("cast(a as decimal(7,7)) div 100").show
> +---------------------------------+
> |(CAST(a AS DECIMAL(7,7)) div 100)|
> +---------------------------------+
> | 0|
> | 0|
> +---------------------------------+
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
