yihua commented on code in PR #19674: URL: https://github.com/apache/hudi/pull/19674#discussion_r3833871241
########## README.md: ########## @@ -139,6 +139,24 @@ Refer to the table below for building with different Spark and Scala versions. | `-Dspark4.2` | hudi-spark4.2-bundle_2.13 | For Spark 4.2 and Scala 2.13 (Needs java 17) | | `-Dspark3` | hudi-spark3-bundle_2.12 (legacy bundle name) | For Spark 3.5.x and Scala 2.12 | +### Native Spark bundle + +`hudi-native-spark-bundle` is `hudi-spark-bundle` plus [Apache DataFusion Comet](https://datafusion.apache.org/comet/) +for native vectorized execution. It is built alongside the Spark bundle for the Spark versions Comet +publishes artifacts for, and produces nothing for the others. + +| Maven build options | Expected native bundle jar name | Notes | +|:--------------------------|:--------------------------------|:-----------------------------------------------| +| `-Dspark3.4` | hudi-native-spark3.4-bundle_2.12 | For Spark 3.4.x and Scala 2.12 | +| `-Dspark3.5 -Dscala-2.12` | hudi-native-spark3.5-bundle_2.12 | For Spark 3.5.x and Scala 2.12 | +| `-Dspark3.5 -Dscala-2.13` | hudi-native-spark3.5-bundle_2.13 | For Spark 3.5.x and Scala 2.13 | +| `-Dspark4.0` | hudi-native-spark4.0-bundle_2.13 | For Spark 4.0 and Scala 2.13 | +| `-Dspark4.1` | hudi-native-spark4.1-bundle_2.13 | For Spark 4.1 and Scala 2.13 | +| `-Dspark3.3`, `-Dspark4.2` | none | Comet publishes no artifact for these versions | Review Comment: `-Dspark3` is deprecated, so there is no need to surface that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
