+1 in favor of adding java 21 support +1 in favor of removing java 8 support
I am currently working on Spark 4.0 / Iceberg integration <https://github.com/apache/iceberg/pull/10622>. Spark 4.0 runs on Java 17/21. On Fri, Jul 19, 2024 at 4:58 AM Piotr Findeisen <piotr.findei...@gmail.com> wrote: > Hi, > > We recently started to test Hive3 with Java 11 and 17 > <https://github.com/apache/iceberg/pull/10482> and the tests pass. > So dropping Java 8 doesn't technically require removing the Hive 3 related > modules, unless users cannot do anything useful with them (because e.g. > they can only run Hive runtime with Java 8 for some reason). > Peter, can you please confirm this is not the case? > Then it seems we could proceed with JDK 8 drop and discuss what to do with > Hive modules *separately*. > > re original question of adding JDK 21 support -- we seem to have strong > consensus to add it. > Eduard plans to merge the PR once 1.6.0 is out. So I think we no longer > need to debate this topic, unless there are any new objections to be raised. > > > Best > Piotr > > > > > > > > On Fri, 19 Jul 2024 at 13:49, Péter Váry <peter.vary.apa...@gmail.com> > wrote: > >> Back to the main topic: >> >> Removing java8 support still allows us to use the old HMS libraries, so >> iceberg-hive-metastore module is not affected in this sense. We still need >> to run the tests, to ensure that there is no changes in the behaviour, but >> it is unlikely. >> >> +1 in favor of adding java 21 support >> +1 in favor of removing java 8 support - especially if it is not bound to >> removing Hive support as well >> >> Thanks, >> Peter >> >> Cheng Pan <pan3...@gmail.com> ezt írta (időpont: 2024. júl. 19., P, >> 4:56): >> >>> A basic question, is iceberg-hive-metastore considered part of the >>> "Hive module"? >>> >>> I suppose that HMS 2.x is still widely used. AFAIK, the current >>> iceberg-hive-metastore is compatible with HMS 2.1+, based on Iceberg >>> and Spark CI, I also suppose it works well with Java 8 to 21. >>> >>> Thanks, >>> Cheng Pan >>> >>> On Fri, Jul 19, 2024 at 4:44 AM Ryan Blue <b...@databricks.com.invalid> >>> wrote: >>> > >>> > Thanks for the context, Denys and Peter. Sounds like there's a good >>> question here about where the Hive integration should live and the most >>> recent decision was to maintain that support in Hive. I definitely hear the >>> point about Hive 3 users depending on the Iceberg modules. I'm also glad to >>> hear that some of the issues are expected to be fixed with the release of >>> Hive 4.0.x. >>> > >>> > I think that we have two separate questions for how to move forward >>> with Hive support, depending on the Hive version. There is a question about >>> what we do with the current Hive modules and what to do with the Hive 4 >>> support that has been developed externally. >>> > >>> > For Hive 2.x and 3.x, we have code in the Iceberg repo that is not >>> being developed. Hive 2 is fairly easy since it is EOL. While Hive 3 is >>> still used, I don't think it makes sense to keep releasing versions of it >>> if it requires Java 8, which has not been publicly maintained for 5 years. >>> We need to upgrade and that is at odds with keeping support for Hive 3. As >>> Fokko and I both pointed out, people can still use older releases. >>> > >>> > For the question of how to maintain support for Hive 4, I think it's >>> worth having a separate discussion (probably not on the thread about JDK >>> versions) about where to maintain it. I think that it is best to maintain >>> integration in engines and not in the Iceberg project; there are few >>> implementations here and I think that it is a hard problem for Iceberg to >>> maintain support for multiple versions (as you can see with support for so >>> many different Flink, Hive, and Spark versions). >>> > >>> > Ryan >>> > >>> > On Thu, Jul 18, 2024 at 7:25 AM Denys Kuzmenko <dkuzme...@apache.org> >>> wrote: >>> >> >>> >> In the following 1-2 months we plan to release HIVE-4.0.1 which >>> includes bug fixes and then focus on HIVE-4.1.0 release with jdk17. >>> > >>> > >>> > >>> > -- >>> > Ryan Blue >>> > Databricks >>> >>
