yihua opened a new issue, #19673: URL: https://github.com/apache/hudi/issues/19673
### Problem Hudi has no bundle that ships a native execution engine. Anyone wanting [Apache DataFusion Comet](https://datafusion.apache.org/comet/) alongside Hudi has to assemble and version-match the jars themselves, against a Comet release matrix that does not line up with Hudi's Spark profiles: Comet publishes one artifact per Spark minor version and, at 1.0.0, covers only Spark 3.4, 3.5, 4.0 and 4.1. ### Proposal Publish `hudi-native-spark-bundle`, the contents of `hudi-spark-bundle` plus Comet, for the Spark versions Comet actually releases for. Scope note: Comet does not read Hudi tables natively (it rejects Hudi's file format with `Unsupported file format HoodieFileGroupReaderBasedFileFormat`), so the Parquet decode stays on Spark. What the bundle enables is native execution of the rest of the plan: joins, shuffles, sorts and aggregates over Hudi tables. Making the Hudi scan itself native is separate work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
