yihua opened a new issue, #20427: URL: https://github.com/apache/hudi/issues/20427
Hudi compiles against Spark 3.5.9 (since #19884), but most bundle validation lanes still run the bundles on a Spark 3.5.1 runtime. Since the base image moved to glibc (#20417), the Spark native bundle validation runs in the Java 17 pass of those lanes and fails on master: Apache DataFusion Comet 1.1.0 references `org.apache.spark.sql.execution.columnar.InMemoryTableScanLike`, which first ships in Spark 3.5.2, so Comet fails to initialize with `NoClassDefFoundError` and the copy-on-write query check fails (`validate-bundles (scala-2.13, flink1.20, 1.11.4, 1.13.1, spark3.5, spark3.5.1)`). The bundle validation runtime should be the latest 3.5 patch release (3.5.9), matching the compile-time version, in the PR CI lanes and in the release candidate and Maven artifact validation workflows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
