This is handy. Thanks so much for sharing it. On Wednesday, March 6, 2013, Jeffrey Zhong wrote:
> Hey, > > Recently I'm working on some hbase test case failures and I think it's > useful if we can see a report of all failed test cases from most recent > runs. So we can easily see how flaky a test case is. I wrote a tool some > time back and below are some reports against different branches from > today's > run. You can get the tools' source at > https://github.com/jeffreyz88/jenkins-tools. If we can run it daily and > send > out an email then we can quickly notice what may break after recent > check-ins. > > Notes: > 1) " 873 874 875 876 877 878 879 880 881" are builds which has > failed test cases in current or previous runs > 2) 1 means "PASSED", 0 means "NOT RUN AT ALL", -1 means "FAILED" > > HBase-0.95 (from last 10 runs configurable) > > Failed Test Cases 21 22 23 24 25 27 > ======================================================== > > org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h > adoop.hbase.catalog.testmetamigrationconvertingtopb 0 0 0 0 > -1 > -1 > > org.apache.hadoop.hbase.coprocessor.example.testbulkdeleteprotocol.testbulkd > eletecolumn -1 0 0 0 0 0 > > org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint.org.apache. > hadoop.hbase.coprocessor.example.testrowcountendpoint -1 0 0 0 > 0 0 > > org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver. > org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver > -1 0 0 0 0 0 > > org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock > edritondeadrs 1 -1 -1 0 1 1 > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul > dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback > 1 1 1 1 1 -1 > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd > ownfixupwhendaughterhassplit 1 1 1 -1 -1 -1 > org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning 0 1 > 1 1 -1 0 > > org.apache.hadoop.hbase.replication.testmasterreplication.testcyclicreplicat > ion 1 1 1 1 1 -1 > > org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov > er 1 -1 0 1 -1 0 > > org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q > ueuefailover 0 1 -1 0 -1 0 > > org.apache.hadoop.hbase.security.access.testaccesscontroller.org.apache.hado > op.hbase.security.access.testaccesscontroller 0 -1 0 0 0 0 > > As you can see we have a few test cases don't run successfully at all or > recently. > > HBase-TRUNK (from last 10 runs) > > Failed Test Cases 3908 3909 3910 3912 3913 3914 3915 3916 > ======================================================== > > org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h > adoop.hbase.catalog.testmetamigrationconvertingtopb 0 -1 0 0 > 0 > 0 -1 -1 > > org.apache.hadoop.hbase.client.testadmin.testcloseregionwhenservernameisempt > y 1 1 1 1 1 1 -1 0 > org.apache.hadoop.hbase.client.testscannertimeout.test3686a 1 1 -1 > 0 1 1 1 1 > > org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testofflinesnap > shotregionoperationsindependent 0 1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testonlinesnaps > hotregionoperationsindependent 1 1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.master.testassignmentmanageroncluster.testmoveregion > 1 1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.master.testdistributedlogsplitting.testdelayeddelete > onfailure 1 1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock > edritondeadrs 1 1 1 1 1 -1 0 1 > org.apache.hadoop.hbase.master.testtablelockmanager.testdelete 1 1 > -1 0 1 1 1 1 > > org.apache.hadoop.hbase.procedure.testzkprocedurecontrollers.testsimplezkcoh > ortmembercontroller 1 1 1 1 1 1 -1 0 > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testexist > ingznodeblockssplitandwerollback 1 1 1 1 1 -1 0 1 > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul > dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback > 1 1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd > ownfixupwhendaughterhassplit -1 -1 -1 0 -1 0 -1 0 > org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning 0 -1 > -1 0 1 1 1 1 > > org.apache.hadoop.hbase.replication.testmultislavereplication.testmultislave > replication 1 1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov > er 1 1 1 -1 0 -1 0 1 > > org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q > ueuefailover 1 1 1 -1 0 -1 0 1 > > org.apache.hadoop.hbase.security.access.testaccesscontrolfilter.testqualifie > raccess -1 0 1 -1 0 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testdegenerateregions 1 -1 > 0 1 1 1 1 1 > > org.apache.hadoop.hbase.util.testhbasefsck.testfixassignmentsandnohdfschecki > ng 1 -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testfixbytable 1 -1 0 > 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testhbckthreadpooling 1 -1 > 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testlingeringreferencefile 1 > -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testmissingfirstregion 1 -1 > 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testmissinglastregion 1 -1 > 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testnotinhdfs 1 -1 0 > 1 > 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testnotinmetaordeployedhole 1 > -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testoverlapandorphan 1 -1 > 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testregionhole 1 -1 0 > 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testregionshouldnotbedeployed > 1 -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testsidelineoverlapregion 1 > -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.util.testhbasefsck.testsplitdaughtersnotinmeta 1 > -1 0 1 1 1 1 1 > > More test cases are not in good shape. > > HBase-0.94(from last 10 runs) > > Failed Test Cases 873 874 875 876 877 878 879 880 881 > ======================================================== > org.apache.hadoop.hbase.backup.testhfilearchiving.testcleaningrace -1 > 0 > 1 1 1 1 1 1 1 > org.apache.hadoop.hbase.catalog.testmetareadereditor.testgetregion 1 > 1 > -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.catalog.testmetareadereditor.testretrying 1 1 > -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.catalog.testmetareadereditor.testscanmetafortable > 1 1 -1 0 1 1 1 1 1 > org.apache.hadoop.hbase.catalog.testmetareadereditor.testtableexists 1 > 1 -1 0 1 1 1 1 1 > > org.apache.hadoop.hbase.client.testmetamigrationremovinghtd.org.apache.hadoo > p.hbase.client.testmetamigrationremovinghtd 0 -1 0 0 0 0 > 0 0 0 > org.apache.hadoop.hbase.client.testmultiparallel.testflushcommitsnoabort > 1 1 1 1 1 -1 0 1 1 > org.apache.hadoop.hbase.client.testmultiparallel.testflushcommitswithabort > 1 1 1 1 1 -1 0 1 1 > org.apache.hadoop.hbase.client.testshell.testrunshelltests 1 1 1 > 1 1 1 -1 0 1 > > org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock > edritondeadrs 1 1 -1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.regionserver.testatomicoperation.testmultirowmutatio > nmultithreads 1 1 1 -1 0 1 1 1 1 > > org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd > ownfixupwhendaughterhassplit 1 -1 0 1 1 1 1 1 1 > > As you can see 0.94 is stable and testmetamigrationremovinghtd hasn't been > run for a while either being removed or due to other failures. Some test > cases aren't stable but not too bad. > > Thanks, > -Jeffrey > > > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
