HyukjinKwon commented on issue #23323: [SPARK-26372][SQL] Don't reuse value from previous row when parsing bad CSV input field URL: https://github.com/apache/spark/pull/23323#issuecomment-447613903 Merged to master. @bersprockets, mind checking and making a backport at branch-2.4 please? Looks the similar issue will be persistent at branch-2.4: https://github.com/apache/spark/blob/branch-2.4/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala#L247-L262 and looks we should set the row all nulls.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
