If you're upgrading an environment without doing any additional testing - sure, it can be helpful to isolate the issue.
However, outside of this scenario, where you actually test your upgrade process and vet the functionality, I don't see it as a big gain - certainly not enough of one to hold the project from moving forward. If there are bugs in the JVM itself, they should have been found already. We're almost 2 years behind in the LTS release, we have plenty of testing, this stuff should be caught long before it's time to upgrade. The way I see it, we should do one of the following: * support multiple versions and only limited overlap. For example, we support 11, 17, 21, 24 in 6.0, then 21, 24, 27 in 7.0. * or do the two version thing and not bother with overlap. The main reason I can think of to continue to support older versions is dependency compatibility. If we use C*-all within the bulk reader and that requires supporting older JVMs for Spark itself. The other reasons (A/B JVM testing & debugging upgrade bugs) are pretty weak in comparison to the gains to be had from moving forward. For example, dropping older versions means we can: * Use generational ZGC (jdk 21+) as standard. No more long pauses. * Move all allocation to per-thread arenas (leveraging memory layouts & structured memory) to avoid allocation on the heap (21+) * Ability to use virtual threads (I think 24+) With our current policy, this is our timetable: * 6.0 : 17 + 21. 2025 * 70: 21 + 24. 2026 * 8.0 24 + 27: 2027 We're a year away from having a release that can use any of the "newer" JVM features and 2 years away from having the ability to do some intelligent memory management. That's a long time for an entire community to wait because of the users who want to do A/B tests against JVM versions, or want the ability to debug potential java release issues in production without a staging environment. Jon On Tue, May 20, 2025 at 9:07 AM Brandon Williams <dri...@gmail.com> wrote: > On Tue, May 20, 2025 at 10:59 AM Jon Haddad <j...@rustyrazorblade.com> > wrote: > > > There is also that recommendation that I keep on hearing - don’t do C* > major upgrade and JDK upgrade simultaneously. I believe that was one of the > reasons for overlap too > > > > There's no practical reason for this today. Maybe in the Java 6 or 8 > days, sure. But now, it's a useless requirement. > > If I'm going to encounter a strange bug after upgrading, I'd like the > surface area to be limited to one of C* or the JVM if possible. > > Kind Regards, > Brandon >
