[
https://issues.apache.org/jira/browse/SPARK-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931691#comment-15931691
]
roncenzhao commented on SPARK-20017:
------------------------------------
The root cause of this exception is:
The `StringToMap`'s `dataType` is `MapType(StringType, StringType,
valueContainsNull = false)` and the result of `str_to_map('')` is `"" -> null`,
so in the function `explode`, it will throw NPE exception.
I think we should change the `dataType` of `StringToMap` to
`MapType(StringType, StringType)`, the default value of `valueContainsNull` is
`true`.
> Functions "str_to_map" and "explode" throws NPE exceptioin
> ----------------------------------------------------------
>
> Key: SPARK-20017
> URL: https://issues.apache.org/jira/browse/SPARK-20017
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.0
> Reporter: roncenzhao
> Attachments: screenshot-1.png
>
>
> ```
> val sqlDf = spark.sql("select k,v from (select str_to_map('') as map_col from
> range(2)) tbl lateral view explode(map_col) as k,v")
> sqlDf.show
> ```
> The sql throws NPE exception.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
