> > Dear community, I'd like to initiate a discussion on the future release > and support strategy for Hadoop. This includes a few important topics that > I believe we should align on: 1. Ongoing Support for the Hadoop 3.4 Branch > As it stands, Hadoop 3.4 is the last release line that supports Java 8. It > is expected that Java 8 compilation support will be officially removed from > the trunk branch after a community vote. At the same time, the trunk branch > is increasingly diverging from the 3.4 line due to the adoption of JUnit 5 > and new features based on JDK17 syntax. As a result, future bug fixes and > backports will become significantly more difficult. Given this, I’d like to > ask: *after the release of hadoop-3.4.2, does the community plan to > continue maintaining and releasing hadoop-3.4.3 or other 3.4-series > versions?* 2. Preparation for Hadoop 3.5 Release With the JDK17 support > work nearing completion, it’s time to start planning for the Hadoop 3.5 > release. Specific questions for discussion include: - Should we set up a > new CI pipeline for JDK17 on the trunk branch? -
Agree > Should we also add support for JDK21 in CI? Agree > - If JDK21 support is added, can we remove the existing JDK11 pipeline? Agree Also Java 8 if we still have that > 3. Operating System Support Strategy Several major OS distributions—CentOS > 7, CentOS 8, and Debian 10—have already reached or are approaching > end-of-life (EOL). Should we adjust our support policy for these systems? > For example: - Officially deprecate EOL systems; - Provide > documentation-only support for compilation; - Remove outdated base images > from Dockerfiles. While CentOS 8 is EOL, RHEL 8 is still very much in support. Instead of simply removing Centos 8 support, I propose switching to a maintained RHEL clone like Rocky Linux instead. (It would also be nice to have support for RHEL/Rocky 9 and 10, but that's a different discussion) > 4. Release Strategy Under Multi-JDK Builds As we prepare to support JDK11, > JDK17, and JDK21, we also need a clear strategy for future releases. Key > questions include: I think we should also drop Java 11 support, and go straight to Java 17. JDK11 is already also EOL. For example, HBase is also going to require JDK 17 for 3.0. > - Should we publish multiple build variants like: hadoop-3.5.0-jdk11, > hadoop-3.5.0-jdk17, hadoop-3.5.0-jdk21? - I don't think that's necessary. Building for the lowest supported Java should be enough (as long as we test with all supported JDKs) What naming convention should we adopt for these variants? - For Maven > Central artifacts, should we distinguish by JDK version or consider a > unified approach? Unified. > Additional Context and Suggestions from My Side: 1. The JDK17 > compatibility work is being tracked under HADOOP-17177 < > https://issues.apache.org/jira/browse/HADOOP-17177>. I am in the process > of reviewing and linking all relevant JIRAs—both completed and pending—to > this umbrella JIRA, and will share an overview in the coming week. I think it's basically done as part of my ongoing Java 24/25 work. > 2. Regarding the JUnit 5 migration: Based on our experience, JUnit 5 > introduces significant differences compared to JUnit 4. It fundamentally > changes how tests are written and run. My proposed upgrade path is as > follows: - Complete the migration to *JUnit 5.8.2* (we are still working on > HDFS and Hadoop-Azure and aim to finish in the next two weeks); - Remove > JUnit 4 usage module by module (e.g., Yarn, MapReduce, HDFS, Common, etc.); > - Upgrade to *JUnit 5.13.2* (as Steve suggested); - I think we should start with this. There are several completely broken tests which are masked by the ancient Surefire + JUnit 5 versions we use. Upgrade the *Maven Surefire Plugin*. Earlier issues were due to the mixed > use of JUnit 4 and 5, which will no longer be a concern once JUnit 5 is > fully adopted; I did not follow these discussions. During my tests, I only saw broken tests which are masked by the old Surefire + Junit 5 versions. These need to be fixed anyway, better sooner than later. > - Upgrade the *Maven version* to 3.9.0 for better compatibility and plugin > support. 3. Once the JUnit 5 migration is complete, we will conduct a > thorough test audit to ensure everything runs correctly and mark unstable > tests for future attention. 4. Regarding legacy OS support, I recommend a > "lightweight" approach: - Remove unsupported base images from our > Dockerfiles or upgrade them to newer versions; See my suggestion regarding Centos8 above > - Provide documentation to help users build on older systems; - But > *formally discontinue* full support for EOL systems. Agree. > I’d appreciate hearing your thoughts on these topics. Looking forward to a > fruitful discussion! Best regards, Shilun Fan.
