hadoopqa is acting weird for the last couple of days. It does not report back the results. One example is: https://builds.apache.org/job/PreCommit-HBASE-Build/14475/console.
I've excluded ubuntu3 from the list because it does not have protoc-2.5. On Mon, Jun 15, 2015 at 9:55 PM, Sean Busbey <bus...@cloudera.com> wrote: > Added new HBase-1.3 build or branch-1 and changed the existing 1.2 builds > to use branch-1.2. (ref HBASE-13912) > > On Sun, Jun 14, 2015 at 1:11 AM, Sean Busbey <bus...@cloudera.com> wrote: > > > I've add a new build test to run the ITs under sunny day conditions using > > minicluster for both java 7 and java 8 on the 1.2 line. > > > > https://builds.apache.org/job/HBase-1.2-IT/ > > > > I haven't turned on notifications yet, because BulkIngest and TestIngest > > are read. details on HBASE-13750 > > > > On Sat, Jun 13, 2015 at 8:28 PM, Sean Busbey <bus...@cloudera.com> > wrote: > > > >> sigh. that should have been "is now a matrix build". > >> > >> On Sat, Jun 13, 2015 at 7:31 PM, Sean Busbey <bus...@cloudera.com> > wrote: > >> > >>> The HBase-1.2 build[1] is not a matrix build that runs JDK7 and JDK8 in > >>> parallel. > >>> > >>> I saved the old job for now and left it disabled[2]. > >>> > >>> > >>> [1]: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2/ > >>> [2]: > >>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.2%20non-matrix/ > >>> > >>> On Fri, Apr 10, 2015 at 3:42 PM, Sean Busbey <bus...@cloudera.com> > >>> wrote: > >>> > >>>> Sorry for the noise. I also updated the checkstyle step on HBase-TRUNK > >>>> > >>>> from > >>>> mvn checkstyle:checkstyle > >>>> to > >>>> mvn -DskipTests package checkstyle:checkstyle > >>>> > >>>> to deal with the same issue. Looks all clear now. > >>>> > >>>> On Fri, Apr 10, 2015 at 3:03 PM, Sean Busbey <bus...@cloudera.com> > >>>> wrote: > >>>> > >>>>> Updated the following builds: > >>>>> > >>>>> * HBase-TRUNK > >>>>> > >>>>> moved from > >>>>> > >>>>> mvn clean compile findbugs:findbugs > >>>>> > >>>>> to > >>>>> > >>>>> mvn clean -DskipTests package findbugs:findbugs > >>>>> > >>>>> To work around the known issue where we can't do a bootstrap compile > >>>>> without previous install or remote SNAPSHOT artifacts. (recently > triggered > >>>>> by the addition of the procedure module on master) > >>>>> > >>>>> On Mon, Mar 30, 2015 at 8:49 PM, 张铎 <palomino...@gmail.com> wrote: > >>>>> > >>>>>> Make findbugs and checkstyle plugins always run. > >>>>>> The default behavior only publishes result for stable builds, but > >>>>>> master is > >>>>>> not always stable and sometimes we introduce new warnings in > unstable > >>>>>> builds(see builds 6310-6314). > >>>>>> And findbugs and checkstyle will not fail unless we abort the > building > >>>>>> process I think, so it is safe to turn it on every time. > >>>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> 2015-03-22 22:44 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > >>>>>> > >>>>>> > +1 on letting HBase-TRUNK jenkins show coverage report. > >>>>>> > > >>>>>> > Cheers > >>>>>> > > >>>>>> > On Sun, Mar 22, 2015 at 5:51 AM, 张铎 <palomino...@gmail.com> > wrote: > >>>>>> > > >>>>>> > > Going to change the config of HBase-TRUNK jenkins to show > >>>>>> findbugs, > >>>>>> > > checkstyle and jacoco coverage report. > >>>>>> > > The config has been tested on > >>>>>> > > https://builds.apache.org/job/HBase-TRUNK-jacoco for nearly 30 > >>>>>> times. > >>>>>> > Not > >>>>>> > > much different from HBase-TRUNK unless it runs ~30% slower(the > >>>>>> overhead > >>>>>> > of > >>>>>> > > collecting information for code coverage). > >>>>>> > > Thanks. > >>>>>> > > > >>>>>> > > 2015-03-12 5:08 GMT+08:00 Andrew Purtell <apurt...@apache.org>: > >>>>>> > > > >>>>>> > > > +1, thanks a lot for improving our build hygiene. > >>>>>> > > > > >>>>>> > > > On Wed, Mar 11, 2015 at 2:01 PM, Enis Söztutar < > >>>>>> enis....@gmail.com> > >>>>>> > > wrote: > >>>>>> > > > > >>>>>> > > > > Thanks Sean. This is great. > >>>>>> > > > > > >>>>>> > > > > Enis > >>>>>> > > > > > >>>>>> > > > > On Wed, Mar 11, 2015 at 1:54 PM, Sean Busbey < > >>>>>> bus...@cloudera.com> > >>>>>> > > > wrote: > >>>>>> > > > > > >>>>>> > > > > > FYI I just finished chasing down the breakage for mvn site > >>>>>> on all > >>>>>> > > patch > >>>>>> > > > > > builds. > >>>>>> > > > > > > >>>>>> > > > > > HBASE-13191 consolidates the few places in the test-patch > >>>>>> code > >>>>>> > where > >>>>>> > > we > >>>>>> > > > > > hard coded MAVEN_OPTS. > >>>>>> > > > > > > >>>>>> > > > > > If you look at the PreCommit job now, we use the "set > >>>>>> environment > >>>>>> > > > > > variables" option to set MAVEN_OPTS and then everything > else > >>>>>> > respects > >>>>>> > > > > that > >>>>>> > > > > > setting. > >>>>>> > > > > > > >>>>>> > > > > > I set the initial value to be a combination of the memory > >>>>>> > limitations > >>>>>> > > > > we've > >>>>>> > > > > > been actually running with (the ~6G was getting ignored) > >>>>>> and the > >>>>>> > > > permgen > >>>>>> > > > > > needed for site. > >>>>>> > > > > > > >>>>>> > > > > > MAVEN_OPTS=-Xmx3100M -XX:-UsePerfData -XX:MaxPermSize=256m > >>>>>> > > > > > > >>>>>> > > > > > On Mon, Feb 23, 2015 at 11:46 PM, Stack <st...@duboce.net > > > >>>>>> wrote: > >>>>>> > > > > > > >>>>>> > > > > > > I just made TRUNK and branch-1 builds use same jvm as > >>>>>> patch-build > >>>>>> > > > > > > (hadoopqa) -- i.e. jdku51 -- and I set the MAVEN_OPTS to > >>>>>> be the > >>>>>> > > same > >>>>>> > > > as > >>>>>> > > > > > > those of trunk build too, setting > >>>>>> MAVEN_OPTS="-Xmx6100m"... it > >>>>>> > had > >>>>>> > > > been > >>>>>> > > > > > > 3000. > >>>>>> > > > > > > > >>>>>> > > > > > > Yours, > >>>>>> > > > > > > St.Ack > >>>>>> > > > > > > > >>>>>> > > > > > > On Wed, Dec 31, 2014 at 4:22 PM, Stack < > st...@duboce.net> > >>>>>> wrote: > >>>>>> > > > > > > > >>>>>> > > > > > > > I upped hadoopqa retention to keep last 100 builds and > >>>>>> or last > >>>>>> > 7 > >>>>>> > > > > days, > >>>>>> > > > > > > > whichever comes first. > >>>>>> > > > > > > > St.Ack > >>>>>> > > > > > > > > >>>>>> > > > > > > > On Tue, Nov 4, 2014 at 9:38 AM, Stack < > st...@duboce.net > >>>>>> > > >>>>>> > wrote: > >>>>>> > > > > > > > > >>>>>> > > > > > > >> Branch-1 and master have stabilized and now run > mostly > >>>>>> blue > >>>>>> > > (give > >>>>>> > > > or > >>>>>> > > > > > > take > >>>>>> > > > > > > >> the odd failure) [1][2]. Having a mostly blue > branch-1 > >>>>>> has > >>>>>> > > helped > >>>>>> > > > us > >>>>>> > > > > > > >> identify at least one destabilizing commit in the > last > >>>>>> few > >>>>>> > days, > >>>>>> > > > > maybe > >>>>>> > > > > > > two; > >>>>>> > > > > > > >> this is as it should be (smile). > >>>>>> > > > > > > >> > >>>>>> > > > > > > >> Lets keep our builds blue. If you commit a patch, > make > >>>>>> sure > >>>>>> > > > > subsequent > >>>>>> > > > > > > >> builds stay blue. You can subscribe to > >>>>>> > bui...@hbase.apache.org > >>>>>> > > to > >>>>>> > > > > get > >>>>>> > > > > > > >> notice of failures if not already subscribed. > >>>>>> > > > > > > >> > >>>>>> > > > > > > >> Thanks, > >>>>>> > > > > > > >> St.Ack > >>>>>> > > > > > > >> > >>>>>> > > > > > > >> 1. > >>>>>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/ > >>>>>> > > > > > > >> 2. > >>>>>> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/ > >>>>>> > > > > > > >> > >>>>>> > > > > > > >> > >>>>>> > > > > > > >> On Mon, Oct 13, 2014 at 4:41 PM, Stack < > >>>>>> st...@duboce.net> > >>>>>> > > wrote: > >>>>>> > > > > > > >> > >>>>>> > > > > > > >>> A few notes on testing. > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> Too long to read, infra is more capable now and > after > >>>>>> some > >>>>>> > > work, > >>>>>> > > > we > >>>>>> > > > > > are > >>>>>> > > > > > > >>> seeing branch-1 and trunk mostly running blue. Lets > >>>>>> try and > >>>>>> > > keep > >>>>>> > > > it > >>>>>> > > > > > > this > >>>>>> > > > > > > >>> way going forward. > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> Apache Infra has new, more capable hardware. > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> A recent spurt of test fixing combined with more > >>>>>> capable > >>>>>> > > hardware > >>>>>> > > > > > seems > >>>>>> > > > > > > >>> to have gotten us to a new place; tests are mostly > >>>>>> passing > >>>>>> > now > >>>>>> > > on > >>>>>> > > > > > > branch-1 > >>>>>> > > > > > > >>> and master. Lets try and keep it this way and start > >>>>>> to trust > >>>>>> > > our > >>>>>> > > > > > test > >>>>>> > > > > > > runs > >>>>>> > > > > > > >>> again. Just a few flakies remain. Lets try and > nail > >>>>>> them. > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> Our tests now run in parallel with other test suites > >>>>>> where > >>>>>> > > > previous > >>>>>> > > > > > we > >>>>>> > > > > > > >>> ran alone. You can see this sometimes when our > zombie > >>>>>> > detector > >>>>>> > > > > > reports > >>>>>> > > > > > > >>> tests from another project altogether as lingerers > >>>>>> (To be > >>>>>> > > fixed). > >>>>>> > > > > > > Some of > >>>>>> > > > > > > >>> our tests are failing because a concurrent hbase run > >>>>>> is > >>>>>> > undoing > >>>>>> > > > > > > classes and > >>>>>> > > > > > > >>> data from under it. Also, lets fix. > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> Our tests are brittle. It takes 75minutes for them > to > >>>>>> > complete. > >>>>>> > > > > Many > >>>>>> > > > > > > >>> are heavy-duty integration tests starting up > multiple > >>>>>> > clusters > >>>>>> > > > and > >>>>>> > > > > > > >>> mapreduce all in the one JVM. It is a miracle they > >>>>>> pass at > >>>>>> > all. > >>>>>> > > > > > > Usually > >>>>>> > > > > > > >>> integration tests have been cast as unit tests > >>>>>> because there > >>>>>> > > was > >>>>>> > > > no > >>>>>> > > > > > > where > >>>>>> > > > > > > >>> else for them to get an airing. We have the > hbase-it > >>>>>> suite > >>>>>> > now > >>>>>> > > > > which > >>>>>> > > > > > > would > >>>>>> > > > > > > >>> be a more apt place but until these are run on a > >>>>>> regular > >>>>>> > basis > >>>>>> > > in > >>>>>> > > > > > > public > >>>>>> > > > > > > >>> for all to see, the fat integration tests disguised > >>>>>> as unit > >>>>>> > > tests > >>>>>> > > > > > will > >>>>>> > > > > > > >>> remain. A review of our current unit tests weeding > >>>>>> the old > >>>>>> > > cruft > >>>>>> > > > > and > >>>>>> > > > > > > the > >>>>>> > > > > > > >>> no longer relevant or duplicates would be a nice > >>>>>> undertaking > >>>>>> > if > >>>>>> > > > > > > someone is > >>>>>> > > > > > > >>> looking to contribute. > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> Alex Newman has been working on making our tests > work > >>>>>> up on > >>>>>> > > > travis > >>>>>> > > > > > and > >>>>>> > > > > > > >>> circle-ci. That'll be sweet when it goes > >>>>>> end-to-end. He > >>>>>> > also > >>>>>> > > > > added > >>>>>> > > > > > in > >>>>>> > > > > > > >>> some "type" categorizations -- client, filter, > >>>>>> mapreduce -- > >>>>>> > > > > alongside > >>>>>> > > > > > > our > >>>>>> > > > > > > >>> old "sizing" categorizations of small/medium/large. > >>>>>> His > >>>>>> > > thinking > >>>>>> > > > > is > >>>>>> > > > > > > that > >>>>>> > > > > > > >>> we can run these categorizations in parallel so we > >>>>>> could run > >>>>>> > > the > >>>>>> > > > > > total > >>>>>> > > > > > > >>> suite in about the time of the longest test, say > >>>>>> > 20-30minutes? > >>>>>> > > > We > >>>>>> > > > > > > could > >>>>>> > > > > > > >>> even change Apache to run them this way. > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> FYI, > >>>>>> > > > > > > >>> St.Ack > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >>> > >>>>>> > > > > > > >> > >>>>>> > > > > > > > > >>>>>> > > > > > > > >>>>>> > > > > > > >>>>>> > > > > > > >>>>>> > > > > > > >>>>>> > > > > > -- > >>>>>> > > > > > Sean > >>>>>> > > > > > > >>>>>> > > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > -- > >>>>>> > > > Best regards, > >>>>>> > > > > >>>>>> > > > - Andy > >>>>>> > > > > >>>>>> > > > Problems worthy of attack prove their worth by hitting back. - > >>>>>> Piet > >>>>>> > Hein > >>>>>> > > > (via Tom White) > >>>>>> > > > > >>>>>> > > > >>>>>> > > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Sean > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Sean > >>>> > >>> > >>> > >>> > >>> -- > >>> Sean > >>> > >> > >> > >> > >> -- > >> Sean > >> > > > > > > > > -- > > Sean > > > > > > -- > Sean >
