Yes, we're missing a bunch of H nodes as they being upgraded. Those nodes are all donated, so infra has to wait for them to be handed over. While having those nodes back will be a huge help, having hourly builds and a nightly run every 6 hours is a bit excessive. As you can see, if there is any hiccup or change in the Jenkins infrastructure, these frequent builds pile up and have to be killed/paused until everything catches up.
Thanks again for the quick response on rescheduling those builds! -Chris T. On Fri, 2018-11-02 at 07:14 +0800, 张铎(Duo Zhang) wrote: > Flakey is only part of all the UTs so usually it should be finished > soon and should not queue lots of jobs. > > And recently I found that half of the machines under the Hadoop label > are gone, and filed an infra issue. The guy claimed that the machines > are being reinstalled and will be back soon. Maybe this is the root > cause that why the resources are not enough? > > Thanks. > > Chris Thistlethwaite <chr...@apache.org>于2018年11月2日 周五00:41写道： > > Awesome! Thanks for the quick response. > > > > -Chris T. > > > > On Thu, 2018-11-01 at 09:38 -0700, Stack wrote: > > > Thank you for the notice. I'm making the change now. You should > > > notice difference in an hour or so. > > > https://issues.apache.org/jira/browse/HBASE-21424 is our issue to > > > address. > > > Thanks, > > > S > > > > > > On Thu, Nov 1, 2018 at 9:09 AM Chris Thistlethwaite < > > > chr...@apache.org> wrote: > > > > Greetings! > > > > > > > > During the Jenkins outage yesterday I noticed a ton of builds > > from > > > > HBase-Flaky-Tests > > > > > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/ > > > > in > > > > the queue. Turns out this runs a bunch of pipeline builds every > > > > hour > > > > which clogs up Jenkins, both for you and other projects. For > > > > example, > > > > branch-2.0 is currently queuing 3 builds, waiting on the 4th to > > > > finish, > > > > and it's also behind the HBase Nightly. > > > > > > > > That brings me to HBase Nightly > > > > > > https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/ > > > > it > > > > runs every 6 hours, which is a bit excessive for a nightly > > build > > > > which > > > > by definition should be once a day. Especially as it gets > > > > dangerously > > > > close to running into itself as builds currently around 4-5 > > hours > > > > of > > > > build time. > > > > > > > > I suggest something more like Flaky-Tests every 6 hours and the > > > > Nightly > > > > once a day. If you agree to these changes, feel free to update > > > > Jenkins. > > > > Otherwise, I'll update the jobs in the next few days if there > > is no > > > > response. > > > > > > > > Please add t...@infra.apache.org and/or my address to any > > replies > > > > as > > > > we're not subbed to your dev list. > > > > > > > > Thank you, > > > > Chris T. > > > > > > > > > >
