MaxGekk commented on issue #23325: [SPARK-26376][SQL] Skip inputs without tokens by JSON datasource URL: https://github.com/apache/spark/pull/23325#issuecomment-450823391 > then how about from_json always return null for corrupted record if mode is PERMISSIVE? Does `always` mean we should return `null`s for structs, right? So, it actually means, need to revert commits related to partial results #23253 & #23120 and for sure #22938. Frankly speaking, I would avoid this. Just to be clear, the main motivation for rejection of this PR and reverting 3 above is the `columnNameOfCorruptRecord` option cannot be supported for maps/arrays in the `PERMISSIVE` mode for `from_json`, am I right?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
