Hi! I've been working on JDK23 support for a few weeks, and now I have a mostly working patch (a few test failures may still happen).
The existing HADOOP-19212 JIRA and its original PR is very incomplete, and does not deal at all with the removal of Subject propagation into new threads (which Hadoop heavily relies on) I have opened a new ticket HADOOP-19486 for the increased scope of the JDK23 work, and linked my 99% working rough draft PR to it. I'm aware that getting such a huge change reviewed and committed is a huge task, so I'd like to use this forum to discuss the changes, and how to structure the work. The major changes in the patch: * Adds SubjectUtils as a JDK compatibility layer, (similar to the original patch) * Replaces deprecated direct deprecated Subject method calls in code (including UGI) * Adds callAs() to UserGroupInformation, and deprecates the old doAs() calls. * Adds a new HadoopThread wrapper class (and patches Daemon) to preserve the pre-JEP411 Subject propagation semantics. * Replaces all Thread class usage with HadoopThread. Unfortunately, the only way I could get this to work was adding a new HadoopThread.work() method which wraps the Thread.run(), and HadoopThread subclasses have to override work() instead of run() (Runnable works without changes) Other than that there are a lot of smaller changes, like disabling tests if they require JavaScript support and JDK does not have it, disabling tests using SecurityManager if it is not available, adapting to toString() output changes, not using 0.0.0.0 as target address, etc. The current patch is huge, a bit rough and incomplete, as I haven't replaced all deprecated UGI calls. The test suite can be run with JDK23 by running *start-build-env.sh* and setting *JAVA_HOME* to */usr/lib/jvm/temurin-23-jdk-amd64/*. (I have also added JDK23 to the ARM image, but I could not test that) I am prepared to split this up to make reviewing more palatable, but I would first like to get approval on how to do that. My current plan is: - Add JDK23 support to Docker images - Add all the small changes not directly related to UGI and HadoopThread/Daemon - Add the Subject and thread propagation related changes. (the difficult parts) - Replace newly deprecated calls in non-test code - Replace newly deprecated calls in test code - Add JDK23 or 24 runs to CI This should group the high impact changes into a single patch that is testable and as small as possible Please review the *UserGroupInformation*, *SubjectUtil*, *HadoopThread* and *Daemon* classes in the draft patch as those are the possibly controversial changes, and indicate if the general solution is acceptable, or if you have suggestions to make it better. Please also comment on the plan for the final patches. Is it acceptable ? Do you have suggestions for improvements ? Istvan
