We could further restrict branch-2/2.5/2.6 precommits to testing only Hadoop 3 and JDK 8 plus whichever later JDK we consider the leading edge. My understanding is that is currently 17. That would reduce that workload by 50%?
We could even drop JDK 8 in precommits for further savings. The periodic tests or RM activity will catch JDK specific issues, although it may be harder to quickly determine the change causing the issue by then. > On Jan 9, 2025, at 9:44 PM, Istvan Toth <st...@cloudera.com.invalid> wrote: > I'm afraid that wouldn't be enough. > branch-2.x tests are one of the main causes of overload, as they currently > consume thrice the resources (JDK and Hadoop combinations) of branch-3 and > master, where we only test for JDK17+Hadoop3. > > I've suggested some more heavy-handed changes in a new DISCUSS thread to > get the queue under control until we get more resources. > > Some of the current queue overload is caused by David testing his web ui > backports on each 2.x branch separately, which we usually don't do, > but even without that most capacity seems to be eaten up by the nightlies > alone. > > Istvan > > > On Thu, Jan 9, 2025 at 10:12 PM Andrew Purtell <andrew.purt...@gmail.com> > wrote: > >> Should we limit precommit for the time being to just master and branch-2.5 >> and branch-2.6, given master-first commit policy and the ultimate goal of >> most changes to get them into the GA releases? The rest are in pre-alpha or >> beta state. I’m going to pursue a targeted donation to improve resources >> but that will take time. >> >>> On Jan 9, 2025, at 8:55 AM, Nick Dimiduk <ndimi...@gmail.com> wrote: >>> We lost a fair number of build workers at the end of the year. It now >>> looks like our builds are back-logged significantly. Your build failed >>> due to timeout waiting for a build worker to make itself available. >>> https://ci-hbase.apache.org/label/hbase/load-statistics >>>> On Thu, Jan 9, 2025 at 5:33 PM Daniel Roudnitsky (BLOOMBERG/ 919 3RD >>>> A) <droudnits...@bloomberg.net> wrote: >>>> Hello all, >>>> I would like to get HBASE-27781 in this release if possible, which >> addresses a bug which can cause the client to throw an unchecked exception >> that can kill a caller thread that invoked a client operation which should >> have been timed out, I have a PR up for review if someone could kindly >> review. >>>> Checks passed before I pushed a refactor which did not change any >> logic, after which checks failed due to what looks like a lack of jenkins >> resources, I will push an empty commit now to retrigger checks (is there an >> easier way to retrigger checks?). >>>> Thank you, >>>> Daniel >>>> From: dev@hbase.apache.org At: 01/06/25 12:38:43 UTC-5:00To: >> dev@hbase.apache.org >>>> Subject: Time for 2.5.11 >>>> We are overdue for a maintenance release of 2.5. >>>> If you have any pending work that should go in to such a release, please >>>> get it committed in the next couple of days. Please let me know if you >> have >>>> any blocking issues preventing that. >>>> -- >>>> Best regards, >>>> Andrew >>>> Unrest, ignorance distilled, nihilistic imbeciles - >>>> It's what we’ve earned >>>> Welcome, apocalypse, what’s taken you so long? >>>> Bring us the fitting end that we’ve been counting on >>>> - A23, Welcome, Apocalypse > > > -- > *István Tóth* | Sr. Staff Software Engineer > *Email*: st...@cloudera.com > cloudera.com <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera > on LinkedIn] <https://www.linkedin.com/company/cloudera> > ------------------------------ > ------------------------------
