gengliangwang opened a new pull request #23639: [SPARK-26716][SQL] FileFormat: the supported types of read/write should be consistent URL: https://github.com/apache/spark/pull/23639 ## What changes were proposed in this pull request? 1. Remove parameter `isReadPath`. The supported types of read/write should be the same. 2. rename to `supportsDataType`, which is more consistent with other data source API(e.g. `SupportsBatchRead`, `SupportsBatchWrite`). 3. Disallow reading `NullType` for ORC data source. In #21667 and #21389, it was supposed that ORC supports reading `NullType`, but can't write it. This doesn't make sense. I read docs and did some tests. ORC doesn't support `NullType`. 4. Fix the write path of JSON data source for `NullType`. It will output `{"col": null}`, and the output file can be read back. ## How was this patch tested? Unit tset
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
