Github user viirya commented on a diff in the pull request:
    --- Diff: 
    @@ -114,7 +114,16 @@ class JsonFileFormat extends TextBasedFileFormat with 
DataSourceRegister {
         (file: PartitionedFile) => {
    -      val parser = new JacksonParser(actualSchema, parsedOptions)
    +      // SPARK-21610: when the `requiredSchema` only contains 
    --- End diff --
    @cloud-fan @HyukjinKwon Because the results in those queries are pretty 
weird if you don't know the details of it. If we are going to agree that 
`_corrupted_record` is set to null if no columns are selected, we must clearly 
document this behavior. 
    We need to let users know that `_corrupted_record` is a derived column from 
other columns and cannot be selected alone in a query.
    If you are all agreed that, I think we can revert this change and go to 
update all documents related to `_corrupted_record` here.
    What do you think? It makes sense to you? 

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to