Github user aa8y commented on the issue:
https://github.com/apache/spark/pull/20068
@HyukjinKwon I made code changes based on your suggestions. I also changed
the tests to use the data mentioned in the ticket. However, you're right, the
tests no longer pass. But that is because the Univocity `CsvParser`, when it
encounters an empty string while parsing the data, replaces it with the
`nullValue` we set (see
[setNullValue()](http://docs.univocity.com/parsers/2.5.9/com/univocity/parsers/common/CommonSettings.html#setNullValue(java.lang.String))).
And the `emptyValue` is only effective when the _empty string_ being read has
quotes around it (see
[setEmptyValue()](http://docs.univocity.com/parsers/2.5.9/com/univocity/parsers/csv/CsvParserSettings.html#setEmptyValue(java.lang.String))).
So I believe, at this point, the issue needs to be fixed in the underlying
library being used.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
