GitHub user mmolimar opened a pull request:
https://github.com/apache/spark/pull/22234
[SPARK-25241][SQL] Configurable empty values when reading/writing CSV files
## What changes were proposed in this pull request?
There is an option in the CSV parser to set values when we have empty
values in the CSV files or in our dataframes.
Currently, this option cannot be configured and always sets a default value
(empty string for reading and `""` for writing).
This PR is about enabling a new CSV option in the reader/writer to set
custom empty values when reading/writing CSV files.
## How was this patch tested?
The changes were tested by CSVSuite adding two unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mmolimar/spark SPARK-25241
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22234.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22234
----
commit 8b5180021d246ab2fdf0824c01b9f180136837ce
Author: Mario Molina <mmolimar@...>
Date: 2018-08-25T17:42:03Z
Configurable empty values when reading/writing CSV files
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
