Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r132688380 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat extends TextBasedFileFormat with DataSourceRegister { } (file: PartitionedFile) => { - val parser = new JacksonParser(actualSchema, parsedOptions) + // SPARK-21610: when the `requiredSchema` only contains `_corrupt_record`, --- End diff -- @cloud-fan @HyukjinKwon Because the results in those queries are pretty weird if you don't know the details of it. If we are going to agree that `_corrupted_record` is set to null if no columns are selected, we must clearly document this behavior. We need to let users know that `_corrupted_record` is a derived column from other columns and cannot be selected alone in a query. If you are all agreed that, I think we can revert this change and go to update all documents related to `_corrupted_record` here. What do you think? It makes sense to you?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org