HyukjinKwon commented on issue #23323: [SPARK-26372][SQL] Don't reuse value from previous row when parsing bad CSV input field URL: https://github.com/apache/spark/pull/23323#issuecomment-447621229 Ah, hm. NVM. We don't produce partial records during conversion (string to appropriate typed value in a row) in branch-2.4. If we failed to parse because of the different length of tokens, we will explicitly set `null` for them. So, I don't think the same thing happens in lower Spark versions.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
