Nice. Should we check it into ./dev-tools? Thanks, St.Ack
On Tue, Mar 5, 2013 at 7:12 PM, Andrew Purtell <[email protected]> wrote: > This is handy. Thanks so much for sharing it. > > On Wednesday, March 6, 2013, Jeffrey Zhong wrote: > > > Hey, > > > > Recently I'm working on some hbase test case failures and I think it's > > useful if we can see a report of all failed test cases from most recent > > runs. So we can easily see how flaky a test case is. I wrote a tool some > > time back and below are some reports against different branches from > > today's > > run. You can get the tools' source at > > https://github.com/jeffreyz88/jenkins-tools. If we can run it daily and > > send > > out an email then we can quickly notice what may break after recent > > check-ins. > > > > Notes: > > 1) " 873 874 875 876 877 878 879 880 881" are builds which has > > failed test cases in current or previous runs > > 2) 1 means "PASSED", 0 means "NOT RUN AT ALL", -1 means "FAILED" > > > > HBase-0.95 (from last 10 runs configurable) > > > > Failed Test Cases 21 22 23 24 25 27 > > ======================================================== > > > > > org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h > > adoop.hbase.catalog.testmetamigrationconvertingtopb 0 0 0 0 > > -1 > > -1 > > > > > org.apache.hadoop.hbase.coprocessor.example.testbulkdeleteprotocol.testbulkd > > eletecolumn -1 0 0 0 0 0 > > > > > org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint.org.apache. > > hadoop.hbase.coprocessor.example.testrowcountendpoint -1 0 0 0 > > 0 0 > > > > > org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver. > > > org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver > > -1 0 0 0 0 0 > > > > > org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock > > edritondeadrs 1 -1 -1 0 1 1 > > > > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul > > > dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback > > 1 1 1 1 1 -1 > > > > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd > > ownfixupwhendaughterhassplit 1 1 1 -1 -1 -1 > > org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning 0 > 1 > > 1 1 -1 0 > > > > > org.apache.hadoop.hbase.replication.testmasterreplication.testcyclicreplicat > > ion 1 1 1 1 1 -1 > > > > > org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov > > er 1 -1 0 1 -1 0 > > > > > org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q > > ueuefailover 0 1 -1 0 -1 0 > > > > > org.apache.hadoop.hbase.security.access.testaccesscontroller.org.apache.hado > > op.hbase.security.access.testaccesscontroller 0 -1 0 0 0 > 0 > > > > As you can see we have a few test cases don't run successfully at all or > > recently. > > > > HBase-TRUNK (from last 10 runs) > > > > Failed Test Cases 3908 3909 3910 3912 3913 3914 3915 3916 > > ======================================================== > > > > > org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h > > adoop.hbase.catalog.testmetamigrationconvertingtopb 0 -1 0 0 > > 0 > > 0 -1 -1 > > > > > org.apache.hadoop.hbase.client.testadmin.testcloseregionwhenservernameisempt > > y 1 1 1 1 1 1 -1 0 > > org.apache.hadoop.hbase.client.testscannertimeout.test3686a 1 1 > -1 > > 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testofflinesnap > > shotregionoperationsindependent 0 1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testonlinesnaps > > hotregionoperationsindependent 1 1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.master.testassignmentmanageroncluster.testmoveregion > > 1 1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.master.testdistributedlogsplitting.testdelayeddelete > > onfailure 1 1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock > > edritondeadrs 1 1 1 1 1 -1 0 1 > > org.apache.hadoop.hbase.master.testtablelockmanager.testdelete 1 1 > > -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.procedure.testzkprocedurecontrollers.testsimplezkcoh > > ortmembercontroller 1 1 1 1 1 1 -1 0 > > > > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testexist > > ingznodeblockssplitandwerollback 1 1 1 1 1 -1 0 1 > > > > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul > > > dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback > > 1 1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd > > ownfixupwhendaughterhassplit -1 -1 -1 0 -1 0 -1 0 > > org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning 0 > -1 > > -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.replication.testmultislavereplication.testmultislave > > replication 1 1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov > > er 1 1 1 -1 0 -1 0 1 > > > > > org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q > > ueuefailover 1 1 1 -1 0 -1 0 1 > > > > > org.apache.hadoop.hbase.security.access.testaccesscontrolfilter.testqualifie > > raccess -1 0 1 -1 0 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testdegenerateregions 1 > -1 > > 0 1 1 1 1 1 > > > > > org.apache.hadoop.hbase.util.testhbasefsck.testfixassignmentsandnohdfschecki > > ng 1 -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testfixbytable 1 -1 0 > > 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testhbckthreadpooling 1 > -1 > > 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testlingeringreferencefile > 1 > > -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testmissingfirstregion 1 > -1 > > 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testmissinglastregion 1 > -1 > > 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testnotinhdfs 1 -1 0 > > 1 > > 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testnotinmetaordeployedhole > 1 > > -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testoverlapandorphan 1 -1 > > 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testregionhole 1 -1 0 > > 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testregionshouldnotbedeployed > > 1 -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testsidelineoverlapregion 1 > > -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testsplitdaughtersnotinmeta > 1 > > -1 0 1 1 1 1 1 > > > > More test cases are not in good shape. > > > > HBase-0.94(from last 10 runs) > > > > Failed Test Cases 873 874 875 876 877 878 879 880 > 881 > > ======================================================== > > org.apache.hadoop.hbase.backup.testhfilearchiving.testcleaningrace -1 > > 0 > > 1 1 1 1 1 1 1 > > org.apache.hadoop.hbase.catalog.testmetareadereditor.testgetregion 1 > > 1 > > -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.catalog.testmetareadereditor.testretrying 1 > 1 > > -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.catalog.testmetareadereditor.testscanmetafortable > > 1 1 -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.catalog.testmetareadereditor.testtableexists 1 > > 1 -1 0 1 1 1 1 1 > > > > > org.apache.hadoop.hbase.client.testmetamigrationremovinghtd.org.apache.hadoo > > p.hbase.client.testmetamigrationremovinghtd 0 -1 0 0 0 0 > > 0 0 0 > > org.apache.hadoop.hbase.client.testmultiparallel.testflushcommitsnoabort > > 1 1 1 1 1 -1 0 1 1 > > > org.apache.hadoop.hbase.client.testmultiparallel.testflushcommitswithabort > > 1 1 1 1 1 -1 0 1 1 > > org.apache.hadoop.hbase.client.testshell.testrunshelltests 1 1 1 > > 1 1 1 -1 0 1 > > > > > org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock > > edritondeadrs 1 1 -1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.regionserver.testatomicoperation.testmultirowmutatio > > nmultithreads 1 1 1 -1 0 1 1 1 1 > > > > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd > > ownfixupwhendaughterhassplit 1 -1 0 1 1 1 1 1 1 > > > > As you can see 0.94 is stable and testmetamigrationremovinghtd hasn't > been > > run for a while either being removed or due to other failures. Some test > > cases aren't stable but not too bad. > > > > Thanks, > > -Jeffrey > > > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
