Team "if anybody runs a Hadoop cluster on Java 11, and Nutch is compiled for Java 17, you cannot run it anymore, at least, not without extra work, for example, dockerizing [1]."
Folks, it is time to get with the future, or the past. Java 11 is EOL. Java 11 (LTS) End of Life (EOL) varies by vendor, with most public updates concluding in 2023-2024 I worked at a financial institution, Java 11 is a non-starter. Spring boot is min java 17. Anyone shipping spring boot (which is everyone) thus needs java 17. You can't even find a JVM 11. If you install one the moment you do your vulnerability scanner throws up. If your company has a team managing the build server, THEY wont even install java 11 even in a jenkins agent. Because then their build server shows OSS issues. Everyone has a build bot with security scanners now, it is barely possible to even "merge" things with java 11. Its not worth it to maintain functionality with eol JVM. You can't use language features and no one is EVERY running that JVM in an audited environment. It just creates all types of dumb problems, no one solves problems into the past. Need a protobuf compiler? The only actual supported one doesnt support java 8 or 11. Who benefits? someone on an 2018 version of log4j or jboss? They need to update and that is the reality :) Edward On Wed, Feb 4, 2026 at 3:08 PM Sebastian Nagel <[email protected]> wrote: > Hi Isabelle, > > thanks for asking. Yes, it's complex. But maybe I'm also a bit > overcautious. Let > me explain... > > If anybody runs a Hadoop cluster on Java 11, and Nutch is compiled for > Java 17, > you cannot run it anymore, at least, not without extra work, for example, > dockerizing [1]. > > The Hadoop services (HDFS name and data node, YARN resource and node > manager, > etc.) and the Nutch tasks all run in their own JVM. But JAVA_HOME and > parts of the class path are shared. > > Hadoop distributions are usually conservative and many support Java 8 and > 11, > few Java 17. > > I know about it, Apache Bigtop [2] is still on Java 8 and 11. But while I > might > be able to adapt Nutch, others may not. > > I want to avoid that a forced upgrade to Java 17 breaks the Nutch setups > on > Hadoop for users. > > In local mode this isn't an issue: just install a second Java version. > > > > I've had occasion to run a few small crawls using the default embedded > > Hadoop, both with JVM 17 and 21. > > Thanks, good to now. > > > > Actually, at my former employer, Nutch is running on Java 21 > > But is this locally or deployed on a Hadoop cluster? > > > > Otherwise, if it's compile-time, we don't have to wait for Hadoop to > > compile with Java 17. > > Of course. But again: for distributed mode, the usual way is to compile > Nutch > yourself: the job file in runtime/deploy includes the modified > nutch-site.xml > and other configuration files. This adds additional complexity in > documenting > deployment. And it would only help for some upgrades, e.g. JUnit 6 > (NUTCH-3145), > others would require also a Java 17 runtime (index-geoip / NUTCH-3064). > > > Thanks for the feedback! > > More is very welcome! > Especially, any notes ahead if an upgrade to Java 17 would break your > setup. > Thanks! > > > Best, > Sebastian > > > [1] > > https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/DockerContainers.html > [2] https://bigtop.apache.org/ > > > > On 2/4/26 15:52, Isabelle Giguere wrote: > > Hi; > > I'm trying to understand exactly what the issue is. > > > > I've had occasion to run a few small crawls using the default embedded > > Hadoop, both with JVM 17 and 21. Actually, at my former employer, Nutch > is > > running on Java 21, last I saw. > > > > In a full production environment, if Nutch is using an external Hadoop, > and > > if Hadoop has difficulties with Java 17+ at runtime, then, Nutch and > > Hadoop can each run on their own JVM. > > > > Otherwise, if it's compile-time, we don't have to wait for Hadoop to > > compile with Java 17. > > > > Or am I completely off-track ? > > > > Isabelle Giguère > > > > > > Le mar. 3 févr. 2026 à 16:52, Sebastian Nagel < > [email protected]> > > a écrit : > > > >> Hi everybody, > >> > >> the current Nutch development is ready for Java 17 > >> with NUTCH-2971 fixed - thanks Isabelle! > >> > >> By now Nutch does not require Java 17 at compile or > >> run time. Java 11 is still sufficient. > >> > >> This is good because Hadoop still does not guarantee > >> full support of Java 17 [1]. > >> > >> However, staying compatible with Java 11 becomes a burden > >> because more and more dependencies require an upgrade to Java 17. > >> We have already two PRs open which are great improvements > >> (thanks to Lewis!) but would require Java 17: > >> - index-geoip NUTCH-3064 / PR #825 [2] > >> - JUnit 6 NUTCH-3145 / PR #883 [3] > >> > >> We now have the following options for the next release (1.22): > >> > >> 1. stay on Java 11 > >> > >> 2. require Java 17 at compile time, but compile using "-target 11" > >> to stay compatible with Java 11 at runtime > >> > >> 3. drop support for Java 11 and switch to Java 17 > >> > >> > >> I'm leaning in favor of option 1 and try to release Nutch 1.22 > >> during the next weeks. After the release go to option 3. > >> > >> > >> Please share your thoughts and opinions! > >> > >> > >> ~Sebastian > >> > >> > >> [1] https://issues.apache.org/jira/browse/HADOOP-17177 > >> [2] https://github.com/apache/nutch/pull/825 > >> [3] https://github.com/apache/nutch/pull/883 > >> > > > >
