MaxGekk opened a new pull request #23325: [SPARK-26376][SQL] Skip inputs without tokens by JSON datasource URL: https://github.com/apache/spark/pull/23325 ## What changes were proposed in this pull request? Added new flag for `JacksonParser` - `skipInputWithoutTokens` to control parser's behaviour when its input doesn't contain any valid JSON tokens. The flag is set to `true` for JSON datasource and enables the same behaviour of the datasource as it has in Spark 2.4 and earlier. The flag is set to `false` for JSON functions like `from_json`. As a consequence of that, `from_json` produces bad records in the `PERMISSIVE` mode for strings without JSON tokens. ## How was this patch tested? It was tested by `JsonSuite`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
