Sławomir Bogutyn created SPARK-19727:
----------------------------------------
Summary: Spark SQL round function modifies original column
Key: SPARK-19727
URL: https://issues.apache.org/jira/browse/SPARK-19727
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.1.0
Reporter: Sławomir Bogutyn
Priority: Minor
{code:java}
import org.apache.spark.sql.functions
case class MyRow(value : BigDecimal)
val values = List(MyRow(BigDecimal.valueOf(1.23456789)))
val dataFrame = spark.createDataFrame(values)
dataFrame.show()
dataFrame.withColumn("value_rounded",
functions.round(dataFrame.col("value"))).show()
{code}
This produces output:
{noformat}
+--------------------+
| value|
+--------------------+
|1.234567890000000000|
+--------------------+
+--------------------+-------------+
| value|value_rounded|
+--------------------+-------------+
|1.000000000000000000| 1|
+--------------------+-------------+
{noformat}
Same problem occurs when I use round function to filter dataFrame.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
