Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14118
@rxin Please let me leave my though why I thought it looks good to me in
case it is helpful.
Yes, but we should set `nullValue` for writing `null`. So, I think, setting
`""` for `nullValue` means treating `""` as null.
For example, if we have the dataframe as below:
```
+------+
| a|
+------+
| abc|
| null|
+------+
```
with `nullValue` set to `"abc"`, this will writes
```
abc
abc
```
Here, we ended up with no diff between `null` and `abc`. but since users
set `nullValue` to `abc` for output, users would understand this behaviour.
I mean.. there is no expression for actual `null` so we are explicitly
giving the representation for this and so, I thought it is okay even if we can
differentiate `nullValue` from actual `null`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
