I was traveling the last few weeks, so just a follow-up from my end.
Fokko, can you elaborate on the discussions held in other OSS projects to
> drop Java <17? How did they weigh the benefits/drawbacks for dropping both
> Java 8 and 11 LTS versions? I'd also be curious if other projects plan to
> support older branches with security patches.
So, the ones that I'm involved with (including a TLDR):
- Avro:
- (April 2024: Consensus on moving to 11+, +1 for moving to 17+)
https://lists.apache.org/thread/6vbd3w5qk7mpb5lyrfyf2s0z1cymjt5w
- (Jan 2024: Consensus on dropping 8)
https://lists.apache.org/thread/bd39zhk655pgzfctq763vp3z4xrjpx58
- Iceberg:
- (Jan 2023: Concerns about Hive):
https://lists.apache.org/thread/hr7rdxvddw3fklfyg3dfbqbsy81hzhyk
- (Feb 2024: Concensus to drop Hadoop 2.x, and move to JDK11+,
also +1's for moving to 17+):
https://lists.apache.org/thread/ntrk2thvsg9tdccwd4flsdz9gg743368
I think the most noteworthy (slow-moving in general):
- Spark 4 supports JDK 17+
- Hive 4 is still on Java 8
<https://github.com/apache/hive?tab=readme-ov-file#java>
It looks like most of the projects are looking at each other. Keep in mind,
that projects that still support older versions of Java, can still use
older versions of Arrow.
[image: spiderman-pointing-at-spiderman.jpeg]
(in case the image doesn't come through, that's Spiderman pointing at
Spiderman)
Concerning the Java 11 support, some data:
- Oracle 11: support until January 2032 (extended fee has been waived)
- Cornetto 11: September 2027
- Adoptium 11: At least Oct 2027
- Zulu 11: Jan 2032
- OpenJDK11: October 2024
I think it is fair to support 11 for the time being, but at some point, we
also have to move on and start exploiting the new features and make sure
that we keep up to date. For example, Java 8 also has extended support
until 2030. Dependabot on the Iceberg project
<https://github.com/apache/iceberg/pulls?q=is%3Aopen+is%3Apr+label%3Adependencies>
nicely shows which projects are already at JDK11+ :)
Thanks Dane for driving this!
Kind regards,
Fokko
Op vr 17 mei 2024 om 07:44 schreef Jean-Baptiste Onofré <j...@nanthrax.net>:
> Hi Dane
>
> Do we have a formal consensus about Java version in regards of arrow
> version ?
> I agree with the plan but just wondering if it’s ok from everyone with the
> community.
>
> Regards
> JB
>
> Le jeu. 16 mai 2024 à 18:05, Dane Pitkin <dpit...@apache.org> a écrit :
>
> > To wrap up this thread on Java 8 deprecation, here is my current plan of
> > action:
> >
> > 1) Arrow v17 will be the last version supporting Java 8 and the release
> > notes will warn of its impending deprecation.
> > 2) Arrow v18 will be the first release supporting min version Java 11.
> >
> > I have updated the GH issue[1] to reflect this.
> >
> > [1]https://github.com/apache/arrow/issues/38051
> >
> > On Wed, May 8, 2024 at 5:46 PM Dane Pitkin <d...@voltrondata.com.invalid
> >
> > wrote:
> >
> > > Thank you all for your valuable input. The consensus from my
> > understanding
> > > is that dropping Java 8 is not contentious, so we will move forward
> here.
> > >
> > > We won't drop Java 11 yet, but there's a chance it will happen sooner
> > than
> > > later. I brought up Java 8 & 11 deprecation in the community sync again
> > > today. The summary is that the ASF could be enforcing stricter security
> > > practices in the near future. Arrow Java may be forced to drop Java 11
> if
> > > any of its dependencies no longer support Java 11. This is something
> > we'll
> > > have to investigate and monitor. When the time is right, we should
> start
> > a
> > > new thread on the mailing list to discuss.
> > >
> > > Thanks,
> > > Dane
> > >
> > > On Sat, May 4, 2024 at 2:51 AM <martin.trave...@icloud.com.invalid>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > We were originally expecting to keep Java 11 to the 2026 EOL date for
> > > > extended support, but now that date is moved to 2032 which feels like
> > > more
> > > > time than we need. The issue for us is that getting technology
> approved
> > > for
> > > > use in an enterprise can have ridiculously long lead times, so
> having a
> > > > minimum supported version that is only 2 years old, while probably ok
> > in
> > > > most case, would be a bit aggressive. We use optional dependencies
> > where
> > > we
> > > > can, so e.g. the Java 17 dependency for Spark 4 would only affect
> > clients
> > > > using Spark 4, and they could wait to upgrade. But we chose to use
> > Arrow
> > > in
> > > > the core of our product, it is the internal format everything else
> goes
> > > > through. On the compliance side we have to keep current with security
> > > > updates, so there is no option to stick on an old version.
> > > >
> > > > If we were to drop Java 11 after the next LTS comes out, i.e. 2025 /
> > > 2026,
> > > > then the three latest LTS versions would be supported and the minimum
> > > > version would have been available for 4 - 5 years. I think it would
> be
> > > very
> > > > hard to argue 17 can’t be made available at that point. If Arrow
> forces
> > > our
> > > > hand then obviously we’ll have to go sooner, but it wouldn’t be ideal
> > for
> > > > us.
> > > >
> > > > Lastly just on language capabilities, the only things we’re really
> > > > interested in are performance related, probably virtual threads and
> > > foreign
> > > > memory would be the main ones. Both of the those could be optional
> > > > dependencies, in the case of FFM we’d rely on either yourselves or
> > Netty
> > > > anyway to provide an allocator. So in fact there is very little
> benefit
> > > for
> > > > us to drop Java 11 early, all it costs us is one extra CI job.
> > > >
> > > > Hope some of this is helpful - apologies for the high latency, busy
> as
> > > > always!!
> > > >
> > > > Martin.
> > > >
> > > >
> > > > > On 1 May 2024, at 22:38, Dane Pitkin <d...@voltrondata.com.INVALID
> >
> > > > wrote:
> > > > >
> > > > > Thanks, Martin. It's great to hear of real-world use cases. Do you
> > > > > anticipate any timeline for dropping Java 11 for your product? If
> > Arrow
> > > > did
> > > > > drop Java 11, then it sounds like pinning Arrow Java to an older
> > > version
> > > > > wouldn't be an ideal option if security patches are not backported.
> > > > >
> > > > > Fokko, can you elaborate on the discussions held in other OSS
> > projects
> > > to
> > > > > drop Java <17? How did they weigh the benefits/drawbacks for
> dropping
> > > > both
> > > > > Java 8 and 11 LTS versions? I'd also be curious if other projects
> > plan
> > > to
> > > > > support older branches with security patches.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Apr 30, 2024 at 4:14 PM <martin.trave...@icloud.com
> .invalid>
> > > > wrote:
> > > > >
> > > > >> Speaking for my own product we would like to see Java 11 support,
> we
> > > > rely
> > > > >> heavily on Arrow and have Java 11 as our minimum supported
> version.
> > > We’d
> > > > >> like to keep doing that if possible. Our clients are big
> enterprises
> > > > with
> > > > >> notoriously sluggish update cycles, so we want to offer maximum
> > > > >> compatibility. Once security patches are no longer available on
> the
> > > > regular
> > > > >> public channels then there is a compliance issue, so we generally
> > > follow
> > > > >> the EOL schedule of our dependencies.
> > > > >>
> > > > >> Corretto, Adoptium and Zulu all have recent public builds of both
> 8
> > > and
> > > > 11
> > > > >> and look set to support them with public builds for many years to
> > > come.
> > > > >> Several organisations I have worked with switched away from Oracle
> > > when
> > > > >> they made their licensing blunder with Java 8 and although that is
> > > > >> rectified now, the change seems to have stuck in quite a few
> places
> > > (at
> > > > >> least in my anecdotal experience).
> > > > >>
> > > > >> A major practical difference to me in Java 17 is the strong
> > > > encapsulation
> > > > >> of internals. Since that affects the majority of serious Java
> > > > applications
> > > > >> then perhaps most people have figured out by now to add the JVM
> > params
> > > > that
> > > > >> let Java continue working. Still, it could be a consideration, if
> > > > Java17
> > > > >> is the baseline supported version.
> > > > >>
> > > > >> Regards,
> > > > >> Martin.
> > > > >>
> > > > >> - In case anyone is curious why we don’t support Java 8 per our
> own
> > > > >> policy, it’s because of the “var” keyword - seriously, why did
> Java
> > > > take so
> > > > >> long with that, even C++ got there sooner!
> > > > >>
> > > > >>> On 30 Apr 2024, at 16:20, Jacob Wujciak <assignu...@apache.org>
> > > wrote:
> > > > >>>
> > > > >>> Hello everyone!
> > > > >>> Great to see this move forward!
> > > > >>> +1 on dropping both 8 and 11 unless there is very good reason to
> > keep
> > > > 11
> > > > >>> around.
> > > > >>> Otherwise people will just move to 11 and then have the pain of
> > > > migration
> > > > >>> again when we drop that (which will happen soon regardless imo).
> > > > >>>
> > > > >>> Am Di., 30. Apr. 2024 um 16:18 Uhr schrieb Dane Pitkin
> > > > >>> <d...@voltrondata.com.invalid>:
> > > > >>>
> > > > >>>> Thanks, JB. Are we aware of any downstream dependencies that
> would
> > > > >> benefit
> > > > >>>> from maintaining Java 11 support? Apache Spark jumped straight
> to
> > > Java
> > > > >> 17.
> > > > >>>> It seems other projects are dropping both 8 and 11 at the same
> > time
> > > as
> > > > >>>> mentioned by Fokko. From a maintenance perspective, it would be
> > nice
> > > > to
> > > > >>>> drop both.
> > > > >>>>
> > > > >>>> On Mon, Apr 29, 2024 at 11:20 AM Jean-Baptiste Onofré <
> > > > j...@nanthrax.net>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Hi
> > > > >>>>>
> > > > >>>>> I think it's time to drop JDK8 support. I would say that we
> > should
> > > > >>>>> keep Java11 (jumping directly to Java17 would be problematic
> > > > >>>>> potentially for some users I guess).
> > > > >>>>>
> > > > >>>>> Regards
> > > > >>>>> JB
> > > > >>>>>
> > > > >>>>> On Thu, Apr 25, 2024 at 10:21 PM James Duong
> > > > >>>>> <james.du...@improving.com.invalid> wrote:
> > > > >>>>>>
> > > > >>>>>> If we dropped JDK 8, we could use the JDK to compile
> > > > module-info.java
> > > > >>>>> files. Then we could remove the custom maven plugin we’re using
> > for
> > > > >>>>> compiling module-info.java files for JPMS support and get
> better
> > > IDE
> > > > >>>>> integration (as what we’re doing currently somewhat shoe-horns
> > > module
> > > > >>>>> information alongside JDK8 bytecode).
> > > > >>>>>>
> > > > >>>>>> From: Dane Pitkin <d...@voltrondata.com.INVALID>
> > > > >>>>>> Date: Thursday, April 25, 2024 at 1:02 PM
> > > > >>>>>> To: dev@arrow.apache.org <dev@arrow.apache.org>
> > > > >>>>>> Subject: [DISCUSS] Drop Java 8 support
> > > > >>>>>> Hi all,
> > > > >>>>>>
> > > > >>>>>> I would like to revisit the discussion of dropping Java 8 (and
> > > maybe
> > > > >>>> 11)
> > > > >>>>>> from Arrow's Java implementation. See GH issue[1] below. This
> > was
> > > > also
> > > > >>>>>> discussed in the last Arrow community sync meeting on
> > 2024-04-24.
> > > > >>>>>>
> > > > >>>>>> For context, this was discussed[2] last year on this mailing
> > list.
> > > > We
> > > > >>>>>> decided to revisit the discussion around the June 2024 release
> > > > (Arrow
> > > > >>>>> v17).
> > > > >>>>>> The timing coincides with the initial release of Apache Spark
> > > 4.0.0,
> > > > >>>>> which
> > > > >>>>>> drops both Java 8 and 11 support.
> > > > >>>>>>
> > > > >>>>>> For background, we chose not to drop Java 8 support last year
> > > > because
> > > > >>>>> Arrow
> > > > >>>>>> is seen as a low level library that should support as many
> > > > >> environments
> > > > >>>>> as
> > > > >>>>>> possible. Nowadays, we see more enthusiasm for dropping Java 8
> > > (and
> > > > >> 11)
> > > > >>>>> as
> > > > >>>>>> exemplified by Apache Spark as well as Apache Iceberg[3].
> > > > >>>>>>
> > > > >>>>>> Is it time to consider dropping Java 8? Should we drop Java 11
> > and
> > > > >> skip
> > > > >>>>>> straight to Java 17 as our minimum version? What implications
> do
> > > we
> > > > >>>> need
> > > > >>>>> to
> > > > >>>>>> be aware of?
> > > > >>>>>>
> > > > >>>>>> Thanks,
> > > > >>>>>> Dane
> > > > >>>>>>
> > > > >>>>>> [1]https://github.com/apache/arrow/issues/38051
> > > > >>>>>> [2]
> > > https://lists.apache.org/thread/s07jx58yw4mkl54t3bkggnyg0sftcrr8
> > > > >>>>>> [3]
> > > https://lists.apache.org/thread/ntrk2thvsg9tdccwd4flsdz9gg743368
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> >
>
