[
https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187537#comment-14187537
]
Michael Armbrust commented on SPARK-3683:
-----------------------------------------
[~davies] there used to be some explicit code that checked for "NULL", but I
can't find it anymore, so you are right this problem might exist in scala too.
However, I can't reproduce it as most serdes seem to store null as "\N". Some
sample code to reproduce the issue would be helpful.
> PySpark Hive query generates "NULL" instead of None
> ---------------------------------------------------
>
> Key: SPARK-3683
> URL: https://issues.apache.org/jira/browse/SPARK-3683
> Project: Spark
> Issue Type: Bug
> Components: PySpark, SQL
> Affects Versions: 1.1.0
> Reporter: Tamas Jambor
> Assignee: Davies Liu
>
> When I run a Hive query in Spark SQL, I get the new Row object, where it does
> not convert Hive NULL into Python None instead it keeps it string 'NULL'.
> It's only an issue with String type, works with other types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
