maomaodev commented on issue #6943: URL: https://github.com/apache/kyuubi/issues/6943#issuecomment-4393682142
@pan3793 Hi! After merging https://github.com/apache/kyuubi/pull/7130 and https://github.com/apache/kyuubi/pull/7123, for PARQUET/ORC Hive tables, KSHC will use `ParquetScan`/`ORCScan` from Spark datasource v2 by default. However, `ParquetScan`/`ORCScan` still do not support DPP. The same author, @flaming-archer, has opened PRs in the Spark community (see https://github.com/apache/spark/pull/52180 and https://github.com/apache/spark/pull/52009), but they have not been merged yet. I would like to ask your thoughts and suggestions: 1. Is it still necessary for KSHC `HiveScan` to support DPP? 2. For `ParquetScan`/`ORCScan`, should we keep pushing the fixes in the Spark community, or implement the functionality by wrapping `ParquetScan`/`ORCScan` inside KSHC? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
