Also to clarify -- It is my understanding that removing the hive-metastore module is NOT under consideration; is that correct? We still need a Hive version to depend on for the hive-metastore module. In https://github.com/apache/iceberg/pull/10996, this is Hive 3. Does this present any problem?
On Tue, Nov 19, 2024 at 10:26 PM Manu Zhang <owenzhang1...@gmail.com> wrote: > To clarify, the changes discussed here don't affect hive connectors in > engines, which either use the built-in hive version (Spark) or can be > upgraded to hive 3 (Flink). > > On Wed, Nov 20, 2024 at 2:19 PM Manu Zhang <owenzhang1...@gmail.com> > wrote: > >> Okay, let me add this option >> >> D. Drop Hive 2 & 3 support and suggest to use built-in Iceberg support of >> Hive 4 >> >> On Wed, Nov 20, 2024 at 2:00 PM Cheng Pan <pan3...@gmail.com> wrote: >> >>> Hive 4 brings built-in support for Iceberg format, duplicated >>> implementation in both sides look a redundant stuff. >>> >>> As Hive 2 and 3 do not support Java 11+, and Iceberg 1.8 requires Java >>> 11+, the combination is invalid. How about simply dropping support for Hive >>> 2&3 and suggesting the Hive user upgrade Hive 4 to gain the built-in >>> Iceberg support? >>> >>> Thanks, >>> Cheng Pan >>> >>> >>> >>> On Nov 20, 2024, at 12:47, Manu Zhang <owenzhang1...@gmail.com> wrote: >>> >>> Hi all, >>> >>> We previously reached consensus[1] to deprecate Hive 2 in 1.7 and drop >>> in 1.8. However, when working on the removal PR[2], multiple tests failed >>> in Hive 3 due to not supporting JDK11[3]. The fix has been back-ported to >>> branch-3.1[4] but not released yet. As announced on Hive website, Hive 3.x >>> is declared as End of Life so there will be no more Hive 3 release. >>> Peter(@pvary) suggested upgrading to Hive 4 instead. On the other hand, >>> iceberg-hive3 tests are already broken after we dropped JDK 8 support. It's >>> not caught previously due to tests not running[6]. >>> >>> Based on the current situation, here are the options I can think of to >>> move forward >>> >>> A. Continue to remove Hive 2 in the current PR and upgrade to Hive 4 in >>> a separate PR. >>> B. Hold on removing Hive 2 until we upgrade to Hive 4 >>> C. Add source dependency[7] on Hive branch-3.1 or make a Hive 3.1 >>> release from a forked repo. >>> >>> 1. https://lists.apache.org/thread/zg14b8cor4lnbyd3t4n1297y2bwb1fsg >>> 2. https://github.com/apache/iceberg/pull/10996 >>> 3. https://issues.apache.org/jira/browse/HIVE-21584 >>> 4. https://github.com/apache/hive/commits/branch-3.1/ >>> 5. https://hive.apache.org/general/downloads/ >>> 6. https://github.com/apache/iceberg/pull/11584 >>> 7. https://blog.gradle.org/introducing-source-dependencies >>> >>> Which option do you prefer? Any better alternative? >>> >>> Thanks, >>> Manu >>> >>> >>>
