LuciferYang opened a new pull request, #56337: URL: https://github.com/apache/spark/pull/56337
### What changes were proposed in this pull request? This PR adds Hive `4.2.0` as a supported metastore client version, following 4.0 (SPARK-45265) and 4.1 (SPARK-53095). - Add `hive.v4_2` with the `extraDeps` taken from the Hive 4.2 POM. A few datanucleus/jdo deps are actually lower than 4.1 (`datanucleus-api-jdo` 6.0.3 vs 6.0.5, `datanucleus-core` 6.0.10 vs 6.0.11, `javax.jdo` 3.2.0 vs 3.2.1), while Derby is bumped to `10.17.1.0` for Java 21. There is a note in `package.scala` so these don't get "fixed" upward later. - `Shim_v4_2` extends `Shim_v4_1`. The shimmed method signatures are unchanged between 4.1 and 4.2, so the body is empty. - Hive 4.2 is compiled with `maven.compiler.target=21`, so its jars cannot load on an older JVM. When a 4.2 client is constructed on Java < 21, it now fails with `UNSUPPORTED_HIVE_METASTORE_VERSION_FOR_JAVA` instead of a raw `UnsupportedClassVersionError`. The check lives on the client-construction path rather than in `hiveVersion()`, so config validation still resolves `4.2.0` normally. - `HiveClientVersions` includes `4.2` in the test sweep only on Java 21+. - Update the supported metastore version range in the docs. ### Why are the changes needed? Hive 4.2.0 is released and supports JDK 21. Users on Java 21 should be able to connect Spark to a Hive 4.2 metastore via `spark.sql.hive.metastore.version=4.2.0`. ### Does this PR introduce _any_ user-facing change? Yes. `4.2.0` is now a valid value for `spark.sql.hive.metastore.version` (Java 21+). On an older JVM, setting it fails fast with a clear message. ### How was this patch tested? - Adding `4.2` to `HiveClientVersions` runs it through `HiveClientSuites`, which loads the client via the isolated classloader (requires network access to download the 4.2 jars). - Checked the shimmed methods in `Hive.java` and `IMetaStoreClient.java` against Hive 4.2 source; no differences from 4.1. - `build/sbt 'core/testOnly *SparkThrowableSuite'` for the new error condition. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
