dongjoon-hyun commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-474188179 Hi, @viirya , @maropu , @dbtsai , @cloud-fan . For the question about the other data sources, I made the following PR for that. I think the reduced schema has no harm on all file-based data sources in this cases. Please review the following PR. - [SPARK-27197 Add ReadNestedSchemaTest for file-based data sources](https://github.com/apache/spark/pull/24139) And, if there is another concerns or example, please let me know. We can narrow down more specifically for our purpose. Thank you in advance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
