Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20125
Yup, I was thinking of SparkSQL only feature.
For more details, the original intention was to support multiple values for
`nullValue` but I realised such option support can be generallised - there were
several issues about this since CSV is thirdparty library (I will find and give
some links if requested). Also, there is one reference in R too:
```R
> d <- "col1,col2
+ 1,3
+ 2,4"
> df <- read.csv(text=d, na.strings=c("3", "2"))
> df
```
```
col1 col2
1 1 NA
2 NA 4
```
For more context, original proposal (Scala/SQL/Python/Java) here -
https://github.com/apache/spark/pull/16611 touched many files and I received an
advice to make this smaller, which I liked.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
