Thanks! I will backport the test changes, but keep the default Hadoop version.
We will have more information then. Istvan On Wed, Oct 30, 2024 at 10:22 AM Nick Dimiduk <ndimi...@apache.org> wrote: > On Mon, Oct 28, 2024 at 11:00 AM Istvan Toth <st...@cloudera.com.invalid> > wrote: > > > > I have looked at branch-2.5, but the nightly looks off there, as it runs > > the packaging tests with Hadoop 3.1.1, which it doesn't even officially > > support anymore. > > > > What should we do with branch-2.5 ? > > > > I think that it would not be a lot of extra work to backport everything, > > both the backwards compatibility tests and defaulting Hadoop to 3.4.1. > > We just have to update the version in the pom, and add 3.2.4 to the list > of > > versions to test for backwards compatibility and integration (and remove > > 3.1.1). > > > > I would prefer to have uniform tests and default to Hadoop 3.4.1 on all > > active branches. > > Having a (few) final 2.5.x release(s) with tested Hadoop 3.4.x support > may > > be useful for users for migration and CVE mitigation purposes. > > > > WDYT ? > > branch-2.5's default hadoop3 version is 3.2.4. That's a big dependency > to change for a patch release. I don't think that we can get away with > that change and maintain our compatibility obligations. I'm not up to > speed on the current state of CVEs for this older (EOL?) version, so > we have that dimension to consider. If the newer version is "drop-in" > compatible (and only if), then I have no issue with moving that > release line forward. Ultimately it's the release manager for 2.5 to > make a determination, so I defer to Andrew's assessment. > > I am in favor of backing-porting the improved testing coverage you've > added to branch-2.5. It would be great to understand if branch-2.5 (1) > compiled against 3.2.4 will run on Hadoop 3.4.1 and (2) builds and > tests out on 3.4.1. That will give the more security-minded users > additional confidence in bumping their hadoop dependency on their own. > > Thanks, > Nick > > > On Mon, Oct 21, 2024 at 6:54 PM Istvan Toth <st...@cloudera.com> wrote: > > > > > We could also move the default to 3.4.1 directly. > > > We already test for 3.4.0 in the nightly job. > > > > > > On Mon, Oct 21, 2024 at 3:49 PM 张铎(Duo Zhang) <palomino...@gmail.com> > > > wrote: > > > > > >> And seems hadoop 3.4.1 is out. we could see whether to bump to this > > >> version later? > > >> > > >> Istvan Toth <st...@cloudera.com.invalid> 于2024年10月21日周一 20:56写道： > > >> > > > >> > I have merged the new tests to the nightly Jenkins runs on master. > > >> > > > >> > They have identified another 3.4.0 incompatibility: > > >> > HBASE-28929 <https://issues.apache.org/jira/browse/HBASE-28929> > > >> > > > >> > I will hold off backporting the test changes until HBASE-28929 is > > >> resolved. > > >> > > > > > > > > > -- > > > *István Tóth* | Sr. Staff Software Engineer > > > *Email*: st...@cloudera.com > > > cloudera.com <https://www.cloudera.com> > > > [image: Cloudera] <https://www.cloudera.com/> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > > > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> > > > ------------------------------ > > > ------------------------------ > > > > > > > > > -- > > *István Tóth* | Sr. Staff Software Engineer > > *Email*: st...@cloudera.com > > cloudera.com <https://www.cloudera.com> > > [image: Cloudera] <https://www.cloudera.com/> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > Cloudera > > on LinkedIn] <https://www.linkedin.com/company/cloudera> > > ------------------------------ > > ------------------------------ > -- *István Tóth* | Sr. Staff Software Engineer *Email*: st...@cloudera.com cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------ ------------------------------
