In my experience, there are a few notable areas where core refactoring is happening. Most contributions don’t happen in those areas, and as a result could be cleanly backported if not for gotchas like HBaseTestingUtil rename.
Anyway I agree that just adding a compile check is easier. That said, I would still advocate for not diverging the jdk version from branch-2. In my opinion almost all commits should be backported to branch-2. The only exceptions are for specific incompatibile/unsafe 3.x features. The reason for that is we don’t do major releases nearly often enough, so backporting to branch-2 is the only way to get changes into users hands. So if this change is going to make that much more difficult then personally I’d prefer a more aggressive approach of bumping jdk for branch-2, or a more conservative approach of not allowing new language features in branch-3. Overall I think more frequent smaller major releases would help us be more agile, and aligns more with other modern projects I’ve seen. On Tue, May 7, 2024 at 10:00 AM Istvan Toth <st...@cloudera.com.invalid> wrote: > I'd expect the automated backporting process to only work for fairly > trivial patches which do not use protobuf, etc. > More involved patches would need manual work anyway. > > If we want to make sure that everything compiles with JDK8, it's easier to > just compile the master branch with JDK8 (along with 11/17), > and fail the CI check if it doesn't. > > We need to find a balance between using the new Java features and keeping > the workload manageable. > We could keep compiling master with JDK8 for a year or two, and when > activity on the 2.x branches tapers off, we could remove that restriction. > > > On Tue, May 7, 2024 at 3:56 PM Andrew Purtell <andrew.purt...@gmail.com> > wrote: > > > I also like the suggestion to have CI help us here too. > > > > > On May 7, 2024, at 9:42 AM, Bryan Beaudreault <bbeaudrea...@apache.org > > > > wrote: > > > > > > I'm nervous about creating more big long-term divergences between the > > > branches. Already I sometimes get caught up on HBaseTestingUtil vs > > > HBaseTestingUtility. And we all know the burden of maintaining the old > > > HTable impl. > > > > > > I'm not sure if this is a useful suggestion since it would require > > someone > > > to do a good deal of work, but I wonder if we could automate backport > > > testing a bit. Our yetus checks already check the patch, maybe it could > > > apply the patch to branch-2. This would increase the cost of master > > branch > > > PRs but maybe speed us up overall. > > > > > >> On Tue, May 7, 2024 at 9:21 AM 张铎(Duo Zhang) <palomino...@gmail.com> > > wrote: > > >> > > >> The problem is that, if we only compile and run tests on JDK11+, the > > >> contributors may implicitly use some JDK11+ only features and > > >> introduce difference when backporting to branch-2.x. > > >> > > >> Maybe a possible policy is that, once a patch should go into > > >> branch-2.x too, before mering the master PR, we should make sure the > > >> contributor open a PR for branch-2.x too, so we can catch the > > >> differences between the 2 PRs, and whether to align them. > > >> > > >> WDYT? > > >> > > >> Thanks. > > >> > > >> Andrew Purtell <apurt...@apache.org> 于2024年5月7日周二 20:20写道： > > >>> > > >>> I don’t expect 2.x to wind down for up to several more years. We will > > be > > >>> still using it in production at my employer for a long time and I > would > > >>> continue my role as RM for 2.x as needed. HBase 3 is great but not GA > > yet > > >>> and then some users will want to wait one to a couple years before > > >> adopting > > >>> the new major version, especially if migration is not seamless. (We > > even > > >>> faced breaking changes in a minor upgrade from 2.4 to 2.5 that > brought > > >> down > > >>> a cluster during a rolling upgrade, so there should be no expectation > > of > > >> a > > >>> seamless upgrade.) My plan is to continue releasing 2.x until, like > > with > > >>> 1.x, the commits to branch-2 essentially stop, or until the PMC stops > > >>> allowing release of the candidates. > > >>> > > >>> Perhaps we do not need to do a total ban on use of 11 features. We > > should > > >>> allow a case by case discussion. We can minimize their scope and even > > >>> potentially offer multiversion support like we do with Unsafe access > > >>> utility classes in hbase-thirdparty. There are no planned uses of new > > 11+ > > >>> APIs and features now anyhow. > > >>> > > >>> > > >>> On Tue, May 7, 2024 at 7:40 AM 张铎(Duo Zhang) <palomino...@gmail.com> > > >> wrote: > > >>> > > >>>> For me I think Istvan's plan is also acceptable. > > >>>> > > >>>> So in conclusion, we should > > >>>> > > >>>> 1. Jump to JDK11/JDK17(we could start a new thread to discuss this, > > >>>> maybe also on the user mailing list) > > >>>> 2. Claim and also make sure 3.x does not work with JDK8 > > >>>> 3. Introduce a policy to only allow JDK8 features on master and > > >>>> branch-3.x for a while(maybe still keep the release version as 8?) > > >>>> > > >>>> Any other suggestions? > > >>>> > > >>>> Thanks. > > >>>> > > >>>> Istvan Toth <st...@cloudera.com.invalid> 于2024年4月30日周二 12:45写道： > > >>>>> > > >>>>> Spring is a good argument for JDK17. > > >>>>> > > >>>>> Duo's suggestion is a great step forward, firmly stating that JDK8 > > >> is not > > >>>>> officially supported solves most of our expected future CVE > problems. > > >>>>> > > >>>>> However, I think that ripping off the bandaid, and making sure that > > >>>> HBase 3 > > >>>>> does not work with Java 8 would be better. > > >>>>> It's easier to accept such a change in a major version than in a > > >> minor > > >>>>> version. > > >>>>> > > >>>>> IMO users that are so conservative that they are still using Java 8 > > >> are > > >>>>> unlikely to be first movers to a new major release anyway. > > >>>>> > > >>>>> I think that the following upgrade path would optimal: > > >>>>> > > >>>>> - User stays on (supported) Hbase 2.x until ready to upgrade Java > > >>>>> - User upgrades to Java 11/17 with the same HBase > > >>>>> - User upgrades to Hbase 3.x > > >>>>> > > >>>>> As noted, we will need to support 2.x for some time anyway (just > > >> like 1.x > > >>>>> was supported for a long time). > > >>>>> > > >>>>> As for the backporting issues: > > >>>>> We could make it a policy to avoid using Java 11+ features in Hbase > > >> code > > >>>>> until 2.x supports winds down. > > >>>>> This has worked quite well for Phoenix with Java 7 / Java 8. > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Tue, Apr 30, 2024 at 3:59 AM 张铎(Duo Zhang) < > palomino...@gmail.com > > >>> > > >>>> wrote: > > >>>>> > > >>>>>> AFAIK spring 6 and spring-boot 3 have jumped to java17 directly, > > >> so if > > >>>> we > > >>>>>> want to upgrade, I also suggest that we jump to java 17 directly. > > >>>>>> > > >>>>>> While upgrading to java 17 can reduce our compatibility work on > > >>>> branch-3+, > > >>>>>> but consider the widely usage for java 8, I think we still need to > > >>>> support > > >>>>>> branch-2 for several years, then this will increase the > > >> compatibility > > >>>> work > > >>>>>> as the code between branch-3+ and branch-2.x will be more and more > > >>>>>> different. > > >>>>>> > > >>>>>> So for me, a workable solution is > > >>>>>> > > >>>>>> 1. We first claim that branch-3+ will move minimum java support to > > >> 11 > > >>>> or > > >>>>>> 17. > > >>>>>> 2. Start to move the compilation to java 11 or 17, but still keep > > >>>> release > > >>>>>> version 8, and still keep the pre commit pipeline to run java 8, > > >> 11, > > >>>> 17, to > > >>>>>> minimum our compatibility work before we have the first 3.0.0 > > >> release. > > >>>>>> 3. Cut branch-3.0 and release 3.0.0, so we have a 3.0.0 release, > > >>>> actually > > >>>>>> which can still run on java 8, so it will be easier for our users > > >> to > > >>>>>> upgrade to 3.x and reduce our pressure on maintaining branch-2, > > >>>> especially > > >>>>>> do not need to back port new features there. > > >>>>>> 4. Start to move the release version to 11 or 17 on branch-3+, and > > >>>> prepare > > >>>>>> for 3.1.0 release, which will be the real 11 or 17 only release. > > >>>>>> > > >>>>>> Thanks. > > >>>>>> > > >>>>>> Bryan Beaudreault <bbeaudrea...@apache.org>于2024年4月30日 周二02:54写道： > > >>>>>> > > >>>>>>> I am a huge +1 for dropping java8. > > >>>>>>> > > >>>>>>> One reason I would suggest going to 17 is that it seems so hard > > >> to > > >>>> change > > >>>>>>> these things given our long development cycle on major releases. > > >>>> There > > >>>>>> are > > >>>>>>> some nice language features in 17, but more importantly is that > > >> the > > >>>>>> initial > > >>>>>>> release of java11 was released 6 years ago and java17 released 3 > > >>>> years. > > >>>>>>> Java21 is already released as well. So I could see java17 being > > >>>> widely > > >>>>>>> available enough that we could jump "in the middle" rather than > > >> to > > >>>> the > > >>>>>>> oldest LTS. > > >>>>>>> > > >>>>>>> I will say that we're already running java 21 on all of our > > >>>> hbase/hadoop > > >>>>>> in > > >>>>>>> prod (70 clusters, 7k regionservers). I know not every > > >> organization > > >>>> can > > >>>>>> be > > >>>>>>> that aggressive, and I wouldn't suggest jumping to 21 in the > > >>>> codebase. > > >>>>>> Just > > >>>>>>> pointing it out in terms of basic support already existing and > > >> being > > >>>>>>> stable. > > >>>>>>> > > >>>>>>> On Mon, Apr 29, 2024 at 2:33 PM Andrew Purtell < > > >>>> andrew.purt...@gmail.com > > >>>>>>> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> I also agree that mitigation of security problems in > > >> dependencies > > >>>> will > > >>>>>> be > > >>>>>>>> increasingly difficult, as we cannot expect our dependencies to > > >>>>>> continue > > >>>>>>> to > > >>>>>>>> support Java 8. They might, but as time goes on it is less > > >> likely. > > >>>>>>>> > > >>>>>>>> A minimum of Java 11 makes a lot of sense. This is where the > > >>>> center of > > >>>>>>>> gravity of the Java ecosystem is, probably. > > >>>>>>>> > > >>>>>>>> A minimum of 17 is aggressive and I don’t see the point unless > > >>>> there > > >>>>>> is a > > >>>>>>>> feature in 17 that we would like to base an improvement on. > > >>>>>>>> > > >>>>>>>>> On Apr 29, 2024, at 1:23 PM, chrajeshbab...@gmail.com wrote: > > >>>>>>>>> > > >>>>>>>>> Hi! > > >>>>>>>>> > > >>>>>>>>> With 3.0 on the horizon, we could look into bumping the > > >> minimum > > >>>>>>> required > > >>>>>>>>> Java version for HBase. > > >>>>>>>>> > > >>>>>>>>> The last discussion I could find was four years ago, when > > >>>> dropping > > >>>>>> 8.0 > > >>>>>>>>> support was rejected. > > >>>>>>>>> > > >>>>>>>>> > > >> https://lists.apache.org/thread/ph8xry0x37cvjj89fp2jk1k48yb7gs46 > > >>>>>>>>> > > >>>>>>>>> Now it's four years later, and the end of OpenJDK support for > > >>>> Java 8 > > >>>>>>> and > > >>>>>>>> 11 > > >>>>>>>>> are much closer. > > >>>>>>>>> (Oracle public support is so short that I consider that > > >>>> irrelevant) > > >>>>>>>>> > > >>>>>>>>> Some critical dependencies (like Jetty) have ended even > > >> regular > > >>>>>>> security > > >>>>>>>>> support for Java 8. > > >>>>>>>>> > > >>>>>>>>> By supporting Java 8 we are alse limiting ourselves to using > > >> an > > >>>>>> already > > >>>>>>>> 10 > > >>>>>>>>> year old Java release, ignoring any developments in the > > >> language. > > >>>>>>>>> > > >>>>>>>>> My take is that with the current dogmatic emphasis on CVE > > >>>> mitigation > > >>>>>>> the > > >>>>>>>>> benefits of bumping the required JDK version outweigh the > > >>>> benefits > > >>>>>> even > > >>>>>>>> for > > >>>>>>>>> the legacy install base, especially it's getting harder and > > >>>> harder to > > >>>>>>> be > > >>>>>>>>> CVE free with Java 8. > > >>>>>>>>> > > >>>>>>>>> Furthermore, with RedHat dropping JDK11 support this year, I > > >>>> think we > > >>>>>>>> could > > >>>>>>>>> also consider bumping the minimum requirement straight to > > >> JDK 17. > > >>>>>>>>> > > >>>>>>>>> Hadoop is still on Java 8, but previously it has dropped > > >> Java 7 > > >>>>>> support > > >>>>>>>> in > > >>>>>>>>> a patch release, and I wouldn't be surprised if it dropped > > >> Java > > >>>> 8 in > > >>>>>> a > > >>>>>>>>> similar manner, so I would not put too much stock in that. > > >>>>>>>>> > > >>>>>>>>> What do you think ? > > >>>>>>>>> > > >>>>>>>>> Thanks, > > >>>>>>>>> Rajeshbabu. > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> *István Tóth* | Sr. Staff Software Engineer > > >>>>> *Email*: st...@cloudera.com > > >>>>> cloudera.com <https://www.cloudera.com> > > >>>>> [image: Cloudera] <https://www.cloudera.com/> > > >>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> > [image: > > >>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > > >>>> Cloudera > > >>>>> on LinkedIn] <https://www.linkedin.com/company/cloudera> > > >>>>> ------------------------------ > > >>>>> ------------------------------ > > >>>> > > >> > > > > > -- > *István Tóth* | Sr. Staff Software Engineer > *Email*: st...@cloudera.com > cloudera.com <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera > on LinkedIn] <https://www.linkedin.com/company/cloudera> > ------------------------------ > ------------------------------ >
