Yes, that's exactly that I wanted. Thank you. Do you have input on the branch-2.5 Hadoop version changes ?
On Tue, Oct 29, 2024 at 10:57 AM Nick Dimiduk <ndimi...@apache.org> wrote: > Hi Istvan, > > Thank you so much for continuing to dig in here. > > We do save off test run logs, they are too big for jenkins so we ship > them off to nightlies.a.o. For example, you can find logs from the > last nightly run of the master branch on > https://nightlies.apache.org/hbase/HBase%20Nightly/master/1194/. I > haven't cracked open one of these archives in a while, so I don't know > if the logs you need will be present, but maybe it's worth taking a > look. > > Thanks, > Nick > > On Tue, Oct 29, 2024 at 8:35 AM Istvan Toth <st...@cloudera.com.invalid> > wrote: > > > > Looking at the nightly tests, the test stability has decreased. > > AFAICT there is not much of a pattern to the test failures, and most of > the > > failures seem to be > > minicluster startup failures. > > Some of the failures are for Hadoop2, and Hadoop 3.3.5 and 3.3.6 seems as > > prone to the failures as > > 3.4.0 and 3.4.1, so I don't think these are directly caused by the > upgrade. > > Some of this may be explained by us running much more tests now, and thus > > having a higher chance of triggering > > random/transient errors in them. > > Unfortunately, debugging the minicluster failures would require the full > > debug log from the failing tests, which none of the jobs save. > > > > > > > > On Mon, Oct 28, 2024 at 10:58 AM Istvan Toth <st...@cloudera.com> wrote: > > > > > I have backported the compatibility checks into the nightlies down to > > > branch-2.6. > > > As soon as the tests are stable, I will also backport the change for > > > bumping the default version to 3.4.1 in branch-2.6 and later. > > > > > > I have looked at branch-2.5, but the nightly looks off there, as it > runs > > > the packaging tests with Hadoop 3.1.1, which it doesn't even > officially > > > support anymore. > > > > > > What should we do with branch-2.5 ? > > > > > > I think that it would not be a lot of extra work to backport > everything, > > > both the backwards compatibility tests and defaulting Hadoop to 3.4.1. > > > We just have to update the version in the pom, and add 3.2.4 to the > list > > > of versions to test for backwards compatibility and integration (and > remove > > > 3.1.1). > > > > > > I would prefer to have uniform tests and default to Hadoop 3.4.1 on all > > > active branches. > > > Having a (few) final 2.5.x release(s) with tested Hadoop 3.4.x support > may > > > be useful for users for migration and CVE mitigation purposes. > > > > > > WDYT ? > > > > > > On Mon, Oct 21, 2024 at 6:54 PM Istvan Toth <st...@cloudera.com> > wrote: > > > > > >> We could also move the default to 3.4.1 directly. > > >> We already test for 3.4.0 in the nightly job. > > >> > > >> On Mon, Oct 21, 2024 at 3:49 PM 张铎(Duo Zhang) <palomino...@gmail.com> > > >> wrote: > > >> > > >>> And seems hadoop 3.4.1 is out. we could see whether to bump to this > > >>> version later? > > >>> > > >>> Istvan Toth <st...@cloudera.com.invalid> 于2024年10月21日周一 20:56写道： > > >>> > > > >>> > I have merged the new tests to the nightly Jenkins runs on master. > > >>> > > > >>> > They have identified another 3.4.0 incompatibility: > > >>> > HBASE-28929 <https://issues.apache.org/jira/browse/HBASE-28929> > > >>> > > > >>> > I will hold off backporting the test changes until HBASE-28929 is > > >>> resolved. > > >>> > > >> > > >> > > >> -- > > >> *István Tóth* | Sr. Staff Software Engineer > > >> *Email*: st...@cloudera.com > > >> cloudera.com <https://www.cloudera.com> > > >> [image: Cloudera] <https://www.cloudera.com/> > > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > > >> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> > > >> ------------------------------ > > >> ------------------------------ > > >> > > > > > > > > > -- > > > *István Tóth* | Sr. Staff Software Engineer > > > *Email*: st...@cloudera.com > > > cloudera.com <https://www.cloudera.com> > > > [image: Cloudera] <https://www.cloudera.com/> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > > > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> > > > ------------------------------ > > > ------------------------------ > > > > > > > > > -- > > *István Tóth* | Sr. Staff Software Engineer > > *Email*: st...@cloudera.com > > cloudera.com <https://www.cloudera.com> > > [image: Cloudera] <https://www.cloudera.com/> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > Cloudera > > on LinkedIn] <https://www.linkedin.com/company/cloudera> > > ------------------------------ > > ------------------------------ > -- *István Tóth* | Sr. Staff Software Engineer *Email*: st...@cloudera.com cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------ ------------------------------
