Arun Jijo created SPARK-30646:
---------------------------------
Summary: transform_keys function throws exception as "Cannot use
null as map key", but there isn't any null key in the map
Key: SPARK-30646
URL: https://issues.apache.org/jira/browse/SPARK-30646
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.0
Reporter: Arun Jijo
Have started experimenting Spark 3.0 new SQL functions and along the way found
an issue with the *transform_keys* function. It is raising "Cannot use null as
map key" exception but the Map actually doesn't hold any Null values.
Find my spark code below to reproduce the error.
{code:java}
val df = Seq(Map("EID_1"->10000,"EID_2"->25000)).toDF("employees")
df.withColumn("employees",transform_keys($"employees",(k,v)=>lit(k.+("XYX"))))
.show
{code}
Exception in thread "main" java.lang.RuntimeException: *Cannot use null as map
key*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
