Lior Regev created SPARK-15528:
----------------------------------
Summary: conv function returns inconsistent result for the same
data
Key: SPARK-15528
URL: https://issues.apache.org/jira/browse/SPARK-15528
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.1
Reporter: Lior Regev
When using F.conv to convert a column from a hexadecimal string to an integer,
the results are inconsistent
val col = F.conv(df("some_col"), 16, 10)
val a = df.select(F.countDistinct("some_col"), F.countDistinct(col)).collect()
val b = df.select(F.countDistinct("some_col"), F.countDistinct(col)).collect()
returns:
a: Array[org.apache.spark.sql.Row] = Array([59776,1941936])
b: Array[org.apache.spark.sql.Row] = Array([59776,1965154])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
