Denys, As I have mentioned before, Spark uses a built-in version of the Hive metastore (currently 2.3.10), but can be configured to use a different version and be pointed to a path containing Hive metastore jars for the different version. However, the highest Hive version that can be configured for Spark 3.5 is 3.1.3.
Many Spark users use the built-in version of the Hive metastore support. For that reason, I think Iceberg should still support Hive 2 for now. The proposal here is to provide iceberg-hive-metastore jars built against Hive 2.3.10, 3.1.3, and 4.0. (Note that we can build them against a single source set.) We can remove the iceberg-hive-metastore classes from engine runtime jars, and engine users are free to use the version of iceberg-hive-metastore jar corresponding to their HMS, if they are using the HiveCatalog. - Wing Yew On Tue, May 20, 2025 at 1:10 AM Denys Kuzmenko <dkuzme...@apache.org> wrote: > Wing Yew, how about Hive 3.x HMS? Is it compatible with Spark 3.x? If yes, > why keep support for v2? > FYI Hive 2.x has many unresolved security vulnerabilities. >
