wayneguow commented on pull request #34853:
URL: https://github.com/apache/spark/pull/34853#issuecomment-992615673
@MaxGekk Thank you for your careful reviews and providing a significant test
case.
But in my opinion, the root cause is that the parameter `skipEmptyLines` is
set to true by default.
Before the 2.4 version, when a row has only one column with null values,
this row is also not to be saved in the final csv files.
Since 2.4 version, we save null values and empty strings both as quoted
empty strings "" _by mistake_, so the row which has only one null values column
can be saved in files.
The change what I made this time is just to distinguish null values and
empty strings in saved in csv files really and it's stiil to keep the behavior
with before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
