Since branch-2.3 has been cut, plan to do the final 2.1.10 recently and EOM branch-2.1, so we do not need to test branch-2.1 any more.
Stack <st...@duboce.net> 于2020年3月19日周四 上午7:09写道： > This dashboard on executors helps: > https://builds.apache.org/label/Hadoop/load-statistics > > Hard to figure who is occupying the executors other than by clicking on > each machine in turn along the top: e.g H0, H1, H10, etc.. When I do, I see > lots of hbase at the moment. > > For flakey runs, I changed master, branch-2, and branch-2.1 to twice a day > instead of every four hours. I left branch-2.3, branch-2.2, and branch-1 at > every 4 hours since these are probably the most closely watched. I changed > branch-1.4 and branch-1.3 to @daily (1.3 was running every hour). > https://issues.apache.org/jira/browse/HBASE-24017 > > I disabled this job though was running infrequently as probably not being > looked at (though it was set to run only on change): > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3-IT > > Will keep an eye on it. > S > > > > On Wed, Mar 18, 2020 at 2:54 PM Stack <st...@duboce.net> wrote: > > > On Wed, Mar 18, 2020 at 11:34 AM Sean Busbey <bus...@apache.org> wrote: > > > >> I agree we should be triggering on SCM poll. Whomever updates that > please > >> monitor the results to make sure we weren't avoiding that before due to > a > >> bug in jenkins polling. > >> > >> > > I changed all branches to poll once a day and only run nightly if change > > (s/cron/pollSCM/). https://issues.apache.org/jira/browse/HBASE-24016. > > Will keep an eye on how it plays out going forward > > > > > > > >> If we switch the various builds to rely on flaky lists from their minor > >> release that could be fine, but we'll have to be more proactive in > >> backporting things that stabilize tests and EOM branches where we don't > do > >> those backports. That sounds great to me personally. > >> > >> > > Let me just down frequency on the less active branches for now. Will > > report back. If someone is game for Sean's prescription, they can go > after > > I'm done. > > > > Can we switch our jobs to do sequential steps instead of parallel? that > >> will make them take a lot longer but it will mean we don't overwhelm the > >> executors all at once. > >> > >> > > Suggest we wait on result of above work before we go all serial. By all > > means if we are still hogs when above is done, lets go in series. I am > > reluctant too because then a nightly will take a good part of a day > instead > > of 3-4 hours. > > > > S > > > > > >> On Wed, Mar 18, 2020 at 12:45 AM Stack <st...@duboce.net> wrote: > >> > >> > On Tue, Mar 17, 2020 at 3:36 PM Nick Dimiduk <ndimi...@apache.org> > >> wrote: > >> > > >> > > "Poll SCM: daily" sounds like a good default for everything. > >> > > > >> > > > >> > This sounds good to me. Let me try it for: 1.3, 1.4, 2.1 (unless > >> > objection). > >> > > >> > There are two other branches that run nightly: > >> > > >> > > >> > > >> > https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/HBASE-22114-branch-1/configure > >> > > >> > > >> > https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/HBASE-23162-branch-1/ > >> > > >> > The top-most I changed today seemingly successfully from @daily to > >> @weekly > >> > (at Sean's request). I should change it and HBASE-23162-branch-1 to > >> 'Poll > >> > SCM: daily' too (If I can figure out how to do it). > >> > > >> > On flakeys, if they are not being looked at/addressed, they just > burning > >> > CPU. Up on slack, he asked that we not turn off flakeys. I could turn > >> down > >> > frequency on the old builds (I think I know how to do this...). > >> > > >> > Will mess w/ this starting tomorrow (out loud on slack) time > permitting. > >> > S > >> > > >> > > >> > > >> > > >> > > >> > > On Tue, Mar 17, 2020 at 11:44 AM Bharath Vissapragada < > >> > bhara...@apache.org > >> > > > > >> > > wrote: > >> > > > >> > > > Switch the build trigger for branches from "Build periodically: > >> daily" > >> > to > >> > > > "Poll SCM: daily" ? This may not help master much because commits > >> land > >> > > > almost every day but for branches like branch-2.2 where the > commits > >> are > >> > > not > >> > > > that often the nightlies are triggered only when there are new > >> commits > >> > > > < > >> > > > > >> > > > >> > > >> > https://stackoverflow.com/questions/41114251/what-is-the-difference-between-poll-scm-build-periodically-in-jenkins/41114307 > >> > > > > > >> > > > . > >> > > > > >> > > > Also +1 to reducing the frequency of 1.3/1.4/2.1 and feature > >> branches. > >> > > > > >> > > > On Tue, Mar 17, 2020 at 10:58 AM Stack <st...@duboce.net> wrote: > >> > > > > >> > > > > On Tue, Mar 17, 2020 at 10:35 AM Nick Dimiduk < > >> ndimi...@apache.org> > >> > > > wrote: > >> > > > > > >> > > > > > Heya, > >> > > > > > > >> > > > > > We've been dinged on HBASE-24001 for our unneighborly Jenkins > >> > usage. > >> > > I > >> > > > > > haven't dug into the report to verify or diagnose, I hear it > as > >> a > >> > > > > > reasonable accusation. I recently made things worse during the > >> > JDK11 > >> > > > work > >> > > > > > by using parallel stages. I'm about to make it worse by adding > >> > > > branch-2.3 > >> > > > > > to the nightly matrix. > >> > > > > > > >> > > > > > How should we approach this? > >> > > > > > > >> > > > > > I would suggest we reduce our flakey build runner to only look > >> at > >> > > > master, > >> > > > > > branch-2, and branch-1. We'd need to update the nightly and PR > >> jobs > >> > > to > >> > > > > use > >> > > > > > their respective branch's flakey list as well. > >> > > > > > > >> > > > > > > >> > > > > Could turn down or off the frequency at which some of the > >> nightlies > >> > > run. > >> > > > Do > >> > > > > we need 1.3 and 1.4 run every night? 2.1? > >> > > > > On flakies, could at least turn off 2.1, 1.3, and 1.4 runs > unless > >> > they > >> > > > are > >> > > > > being studied? > >> > > > > > >> > > > > S > >> > > > > > >> > > > > > >> > > > > > >> > > > > > Other idea? > >> > > > > > > >> > > > > > Thanks, > >> > > > > > Nick > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
