Peter and Denys, thank you for your responses. First, I'd like to get agreement on producing separate iceberg-hive-metastore jars for Hive 2, 3 and 4. Peter, do you agree with that? Second, if we do that, I listed three options for the runtime jars of engines, and it appears that you prefer option a. (remove iceberg-hive-metastore classes from the runtime jars).
Denys, I do know that Hive 2 and 3 are EOL. However, we support Spark 3 in Iceberg, and Spark 3 is not able to work with a Hive 4 HMS. Spark 4.0 is about to be released, and I'm happy to see efforts for Hive 4 support in Spark. I think there is still a lot of work to be done before Spark can use Hive 4 in its built-in classes (this won't be in Spark 4.0). The bottom line is that for now, we should support older versions of HMS. - Wing Yew ps. I want to thank Peter for all his help in the work so far! On Mon, May 19, 2025 at 1:54 AM Denys Kuzmenko <dkuzme...@apache.org> wrote: > Hi Wing Yew, > > Hive-2.x and Hive 3.x are EOL. We encourage everyone to upgrade instead of > keeping compatibility with the legacy versions. > > Note: Spark started migration to Hive-4.0, and we are helping here: > https://github.com/apache/spark/pull/50213 > > Regards, > Denys >
