GitHub user jmchung opened a pull request:
https://github.com/apache/spark/pull/18930
Spark 21677
## What changes were proposed in this pull request?
``` scala
scala> Seq(("""{"Hyukjin": 224, "John":
1225}""")).toDS.selectExpr("json_tuple(value, trim(null))").show()
...
java.lang.NullPointerException
at ...
```
Currently the `null` field name will throw NullPointException. As a given
field name null can't be matched with any field names in json, we just output
null as its column value. This PR achieves it by returning a very unlikely
column name `__NullFieldName` in evaluation of the field names.
## How was this patch tested?
Added unit test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jmchung/spark SPARK-21677
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18930.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18930
----
commit 596c2804265e457c378cec433ba2b7fe64f4efdd
Author: Jen-Ming Chung <[email protected]>
Date: 2017-08-12T08:37:04Z
fix null field name and add corresponding test
commit 796041f0f84eac70a2528c37e02dbe4163ddf3aa
Author: Jen-Ming Chung <[email protected]>
Date: 2017-08-12T08:39:56Z
add description
commit f07a9f7e73a17ab5447945d8139a40a1ba8372c5
Author: Jen-Ming Chung <[email protected]>
Date: 2017-08-13T01:48:31Z
modify the comment about nullFieldName
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
