GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/20068
SPARK-17916: Fix empty string being parsed as null when nullValue is set.
## What changes were proposed in this pull request?
When the option `nullValue` is set, the empty value is also set to the same
value. Therefore empty strings get parsed as `null`, which should not happen.
This PR explicitly changes this to be an empty string.
## How was this patch tested?
Tests were added without the fix. It was tested that they failed. Then the
fix was added and the tests have been ensured to pass.
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aa8y/spark csvEmptyValue
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20068.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20068
----
commit 1c3d2216380c9cc89ea829588305b5f31c71d6d5
Author: Jeff Zhang <zjffdu@...>
Date: 2016-04-29T17:42:52Z
Rebase with master.
commit b4eddd67234637feb1b255811d8d018b28894095
Author: Arun Allamsetty <arun@...>
Date: 2017-10-14T19:46:53Z
Merge remote-tracking branch 'upstream/master'
commit f406de9fe13f96b0ee615d496c283b21f415fd2b
Author: Arun Allamsetty <arun.allamsetty@...>
Date: 2017-12-12T00:44:15Z
Merge remote-tracking branch 'upstream/master'
commit 762c14487c762a193fd4f4359c51aaba71eca3f9
Author: Arun Allamsetty <arun.allamsetty@...>
Date: 2017-12-21T21:49:50Z
Merge remote-tracking branch 'upstream/master'
commit ebe2900aadd3af0114ed71506088c6a736dd5002
Author: Arun Allamsetty <arun.allamsetty@...>
Date: 2017-12-21T22:52:15Z
SPARK-17916: Fix empty string being parsed as null when nullValue is set.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
