It seems that precommit-admin job which triggers the precommit-hbase job is not running as frequently as it used to be. Only once every ~6 hours or so.
https://builds.apache.org/job/PreCommit-Admin/ The job is starved for the slaves. Enis On Fri, Jun 19, 2015 at 5:23 PM, Enis Söztutar <enis....@gmail.com> wrote: > hadoopqa is acting weird for the last couple of days. It does not report > back the results. One example is: > https://builds.apache.org/job/PreCommit-HBASE-Build/14475/console. > > I've excluded ubuntu3 from the list because it does not have protoc-2.5. > > > On Mon, Jun 15, 2015 at 9:55 PM, Sean Busbey <bus...@cloudera.com> wrote: > >> Added new HBase-1.3 build or branch-1 and changed the existing 1.2 builds >> to use branch-1.2. (ref HBASE-13912) >> >> On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bus...@cloudera.com> wrote: >> >> > I've add a new build test to run the ITs under sunny day conditions >> using >> > minicluster for both java 7 and java 8 on the 1.2 line. >> > >> > https://builds.apache.org/job/HBase-1.2-IT/ >> > >> > I haven't turned on notifications yet, because BulkIngest and TestIngest >> > are read. details on HBASE-13750 >> > >> > On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bus...@cloudera.com> >> wrote: >> > >> >> sigh. that should have been "is now a matrix build". >> >> >> >> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bus...@cloudera.com> >> wrote: >> >> >> >>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 >> in >> >>> parallel. >> >>> >> >>> I saved the old job for now and left it disabled[2]. >> >>> >> >>> >> >>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/ >> >>> [2]: >> >>> >> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/ >> >>> >> >>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bus...@cloudera.com> >> >>> wrote: >> >>> >> >>>> Sorry for the noise. I also updated the checkstyle step on >> HBase-TRUNK >> >>>> >> >>>> from >> >>>> mvn checkstyle:checkstyle >> >>>> to >> >>>> mvn -DskipTests package checkstyle:checkstyle >> >>>> >> >>>> to deal with the same issue. Looks all clear now. >> >>>> >> >>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bus...@cloudera.com> >> >>>> wrote: >> >>>> >> >>>>> Updated the following builds: >> >>>>> >> >>>>> * HBase-TRUNK >> >>>>> >> >>>>> moved from >> >>>>> >> >>>>> mvn clean compile findbugs:findbugs >> >>>>> >> >>>>> to >> >>>>> >> >>>>> mvn clean -DskipTests package findbugs:findbugs >> >>>>> >> >>>>> To work around the known issue where we can't do a bootstrap compile >> >>>>> without previous install or remote SNAPSHOT artifacts. (recently >> triggered >> >>>>> by the addition of the procedure module on master) >> >>>>> >> >>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <palomino...@gmail.com> wrote: >> >>>>> >> >>>>>> Make findbugs and checkstyle plugins always run. >> >>>>>> The default behavior only publishes result for stable builds, but >> >>>>>> master is >> >>>>>> not always stable and sometimes we introduce new warnings in >> unstable >> >>>>>> builds(see builds 6310-6314). >> >>>>>> And findbugs and checkstyle will not fail unless we abort the >> building >> >>>>>> process I think, so it is safe to turn it on every time. >> >>>>>> >> >>>>>> Thanks. >> >>>>>> >> >>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: >> >>>>>> >> >>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report. >> >>>>>> > >> >>>>>> > Cheers >> >>>>>> > >> >>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <palomino...@gmail.com> >> wrote: >> >>>>>> > >> >>>>>> > > Going to change the config of HBase-TRUNK jenkins to show >> >>>>>> findbugs, >> >>>>>> > > checkstyle and jacoco coverage report. >> >>>>>> > > The config has been tested on >> >>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 >> >>>>>> times. >> >>>>>> > Not >> >>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the >> >>>>>> overhead >> >>>>>> > of >> >>>>>> > > collecting information for code coverage). >> >>>>>> > > Thanks. >> >>>>>> > > >> >>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurt...@apache.org >> >: >> >>>>>> > > >> >>>>>> > > > +1, thanks a lot for improving our build hygiene. >> >>>>>> > > > >> >>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar < >> >>>>>> enis....@gmail.com> >> >>>>>> > > wrote: >> >>>>>> > > > >> >>>>>> > > > > Thanks Sean. This is great. >> >>>>>> > > > > >> >>>>>> > > > > Enis >> >>>>>> > > > > >> >>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey < >> >>>>>> bus...@cloudera.com> >> >>>>>> > > > wrote: >> >>>>>> > > > > >> >>>>>> > > > > > FYI I just finished chasing down the breakage for mvn >> site >> >>>>>> on all >> >>>>>> > > patch >> >>>>>> > > > > > builds. >> >>>>>> > > > > > >> >>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch >> >>>>>> code >> >>>>>> > where >> >>>>>> > > we >> >>>>>> > > > > > hard coded MAVEN_OPTS. >> >>>>>> > > > > > >> >>>>>> > > > > > If you look at the PreCommit job now, we use the "set >> >>>>>> environment >> >>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything >> else >> >>>>>> > respects >> >>>>>> > > > > that >> >>>>>> > > > > > setting. >> >>>>>> > > > > > >> >>>>>> > > > > > I set the initial value to be a combination of the memory >> >>>>>> > limitations >> >>>>>> > > > > we've >> >>>>>> > > > > > been actually running with (the ~6G was getting ignored) >> >>>>>> and the >> >>>>>> > > > permgen >> >>>>>> > > > > > needed for site. >> >>>>>> > > > > > >> >>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData >> -XX:MaxPermSize=256m >> >>>>>> > > > > > >> >>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack < >> st...@duboce.net> >> >>>>>> wrote: >> >>>>>> > > > > > >> >>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as >> >>>>>> patch-build >> >>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS >> to >> >>>>>> be the >> >>>>>> > > same >> >>>>>> > > > as >> >>>>>> > > > > > > those of trunk build too, setting >> >>>>>> MAVEN_OPTS="-Xmx6100m"... it >> >>>>>> > had >> >>>>>> > > > been >> >>>>>> > > > > > > 3000. >> >>>>>> > > > > > > >> >>>>>> > > > > > > Yours, >> >>>>>> > > > > > > St.Ack >> >>>>>> > > > > > > >> >>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack < >> st...@duboce.net> >> >>>>>> wrote: >> >>>>>> > > > > > > >> >>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds >> and >> >>>>>> or last >> >>>>>> > 7 >> >>>>>> > > > > days, >> >>>>>> > > > > > > > whichever comes first. >> >>>>>> > > > > > > > St.Ack >> >>>>>> > > > > > > > >> >>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack < >> st...@duboce.net >> >>>>>> > >> >>>>>> > wrote: >> >>>>>> > > > > > > > >> >>>>>> > > > > > > >> Branch-1 and master have stabilized and now run >> mostly >> >>>>>> blue >> >>>>>> > > (give >> >>>>>> > > > or >> >>>>>> > > > > > > take >> >>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue >> branch-1 >> >>>>>> has >> >>>>>> > > helped >> >>>>>> > > > us >> >>>>>> > > > > > > >> identify at least one destabilizing commit in the >> last >> >>>>>> few >> >>>>>> > days, >> >>>>>> > > > > maybe >> >>>>>> > > > > > > two; >> >>>>>> > > > > > > >> this is as it should be (smile). >> >>>>>> > > > > > > >> >> >>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, >> make >> >>>>>> sure >> >>>>>> > > > > subsequent >> >>>>>> > > > > > > >> builds stay blue. You can subscribe to >> >>>>>> > bui...@hbase.apache.org >> >>>>>> > > to >> >>>>>> > > > > get >> >>>>>> > > > > > > >> notice of failures if not already subscribed. >> >>>>>> > > > > > > >> >> >>>>>> > > > > > > >> Thanks, >> >>>>>> > > > > > > >> St.Ack >> >>>>>> > > > > > > >> >> >>>>>> > > > > > > >> 1. >> >>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/ >> >>>>>> > > > > > > >> 2. >> >>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/ >> >>>>>> > > > > > > >> >> >>>>>> > > > > > > >> >> >>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack < >> >>>>>> st...@duboce.net> >> >>>>>> > > wrote: >> >>>>>> > > > > > > >> >> >>>>>> > > > > > > >>> A few notes on testing. >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> Too long to read, infra is more capable now and >> after >> >>>>>> some >> >>>>>> > > work, >> >>>>>> > > > we >> >>>>>> > > > > > are >> >>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets >> >>>>>> try and >> >>>>>> > > keep >> >>>>>> > > > it >> >>>>>> > > > > > > this >> >>>>>> > > > > > > >>> way going forward. >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> Apache Infra has new, more capable hardware. >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> A recent spurt of test fixing combined with more >> >>>>>> capable >> >>>>>> > > hardware >> >>>>>> > > > > > seems >> >>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly >> >>>>>> passing >> >>>>>> > now >> >>>>>> > > on >> >>>>>> > > > > > > branch-1 >> >>>>>> > > > > > > >>> and master. Lets try and keep it this way and >> start >> >>>>>> to trust >> >>>>>> > > our >> >>>>>> > > > > > test >> >>>>>> > > > > > > runs >> >>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and >> nail >> >>>>>> them. >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> Our tests now run in parallel with other test >> suites >> >>>>>> where >> >>>>>> > > > previous >> >>>>>> > > > > > we >> >>>>>> > > > > > > >>> ran alone. You can see this sometimes when our >> zombie >> >>>>>> > detector >> >>>>>> > > > > > reports >> >>>>>> > > > > > > >>> tests from another project altogether as lingerers >> >>>>>> (To be >> >>>>>> > > fixed). >> >>>>>> > > > > > > Some of >> >>>>>> > > > > > > >>> our tests are failing because a concurrent hbase >> run >> >>>>>> is >> >>>>>> > undoing >> >>>>>> > > > > > > classes and >> >>>>>> > > > > > > >>> data from under it. Also, lets fix. >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them >> to >> >>>>>> > complete. >> >>>>>> > > > > Many >> >>>>>> > > > > > > >>> are heavy-duty integration tests starting up >> multiple >> >>>>>> > clusters >> >>>>>> > > > and >> >>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they >> >>>>>> pass at >> >>>>>> > all. >> >>>>>> > > > > > > Usually >> >>>>>> > > > > > > >>> integration tests have been cast as unit tests >> >>>>>> because there >> >>>>>> > > was >> >>>>>> > > > no >> >>>>>> > > > > > > where >> >>>>>> > > > > > > >>> else for them to get an airing. We have the >> hbase-it >> >>>>>> suite >> >>>>>> > now >> >>>>>> > > > > which >> >>>>>> > > > > > > would >> >>>>>> > > > > > > >>> be a more apt place but until these are run on a >> >>>>>> regular >> >>>>>> > basis >> >>>>>> > > in >> >>>>>> > > > > > > public >> >>>>>> > > > > > > >>> for all to see, the fat integration tests disguised >> >>>>>> as unit >> >>>>>> > > tests >> >>>>>> > > > > > will >> >>>>>> > > > > > > >>> remain. A review of our current unit tests weeding >> >>>>>> the old >> >>>>>> > > cruft >> >>>>>> > > > > and >> >>>>>> > > > > > > the >> >>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice >> >>>>>> undertaking >> >>>>>> > if >> >>>>>> > > > > > > someone is >> >>>>>> > > > > > > >>> looking to contribute. >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> Alex Newman has been working on making our tests >> work >> >>>>>> up on >> >>>>>> > > > travis >> >>>>>> > > > > > and >> >>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes >> >>>>>> end-to-end. He >> >>>>>> > also >> >>>>>> > > > > added >> >>>>>> > > > > > in >> >>>>>> > > > > > > >>> some "type" categorizations -- client, filter, >> >>>>>> mapreduce -- >> >>>>>> > > > > alongside >> >>>>>> > > > > > > our >> >>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large. >> >>>>>> His >> >>>>>> > > thinking >> >>>>>> > > > > is >> >>>>>> > > > > > > that >> >>>>>> > > > > > > >>> we can run these categorizations in parallel so we >> >>>>>> could run >> >>>>>> > > the >> >>>>>> > > > > > total >> >>>>>> > > > > > > >>> suite in about the time of the longest test, say >> >>>>>> > 20-30minutes? >> >>>>>> > > > We >> >>>>>> > > > > > > could >> >>>>>> > > > > > > >>> even change Apache to run them this way. >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> FYI, >> >>>>>> > > > > > > >>> St.Ack >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >>> >> >>>>>> > > > > > > >> >> >>>>>> > > > > > > > >> >>>>>> > > > > > > >> >>>>>> > > > > > >> >>>>>> > > > > > >> >>>>>> > > > > > >> >>>>>> > > > > > -- >> >>>>>> > > > > > Sean >> >>>>>> > > > > > >> >>>>>> > > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> > > > -- >> >>>>>> > > > Best regards, >> >>>>>> > > > >> >>>>>> > > > - Andy >> >>>>>> > > > >> >>>>>> > > > Problems worthy of attack prove their worth by hitting back. >> - >> >>>>>> Piet >> >>>>>> > Hein >> >>>>>> > > > (via Tom White) >> >>>>>> > > > >> >>>>>> > > >> >>>>>> > >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> Sean >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Sean >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Sean >> >>> >> >> >> >> >> >> >> >> -- >> >> Sean >> >> >> > >> > >> > >> > -- >> > Sean >> > >> >> >> >> -- >> Sean >> > >
