See <https://builds.apache.org/job/hbase-0.96-hadoop2/27/changes>
Changes: [jxiang] HBASE-9480 Regions are unexpectedly made offline in certain failure conditions ------------------------------------------ [...truncated 4332 lines...] Running org.apache.hadoop.hbase.TestGlobalMemStoreSize Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.245 sec Running org.apache.hadoop.hbase.security.access.TestAccessControlFilter Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.481 sec Running org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.443 sec Running org.apache.hadoop.hbase.security.token.TestZKSecretWatcher Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.027 sec Running org.apache.hadoop.hbase.TestMultiVersions Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.62 sec Running org.apache.hadoop.hbase.security.access.TestTablePermissions Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.58 sec Running org.apache.hadoop.hbase.TestHBaseTestingUtility Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.966 sec Running org.apache.hadoop.hbase.replication.TestReplicationTrackerZKImpl Tests run: 4, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 1.793 sec Running org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.751 sec Running org.apache.hadoop.hbase.TestRegionRebalancing Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.044 sec Running org.apache.hadoop.hbase.replication.TestReplicationStateZKImpl Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.719 sec Running org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.087 sec Running org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.543 sec Running org.apache.hadoop.hbase.security.access.TestAccessController Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.242 sec Running org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.73 sec Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.085 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.894 sec Running org.apache.hadoop.hbase.replication.TestReplicationSource Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.687 sec Running org.apache.hadoop.hbase.replication.TestMasterReplication Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 69.89 sec Running org.apache.hadoop.hbase.replication.TestMultiSlaveReplication Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.413 sec Running org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.691 sec Running org.apache.hadoop.hbase.TestLocalHBaseCluster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.207 sec Running org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.325 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.515 sec Running org.apache.hadoop.hbase.fs.TestBlockReorder Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.313 sec Running org.apache.hadoop.hbase.TestAcidGuarantees Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 94.399 sec Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.075 sec Running org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.437 sec Running org.apache.hadoop.hbase.coprocessor.TestMasterObserver Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.661 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 82.825 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.208 sec Running org.apache.hadoop.hbase.coprocessor.TestWALObserver Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.81 sec Running org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.089 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.97 sec Running org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.764 sec Running org.apache.hadoop.hbase.coprocessor.TestOpenTableInCoprocessor Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.769 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.905 sec Running org.apache.hadoop.hbase.coprocessor.TestClassLoading Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.428 sec Running org.apache.hadoop.hbase.procedure.TestZKProcedureControllers Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.974 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationHLogReaderManager Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 224.376 sec Running org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.287 sec Running org.apache.hadoop.hbase.snapshot.TestSnapshotDescriptionUtils Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.25 sec Running org.apache.hadoop.hbase.procedure.TestZKProcedure Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.536 sec Running org.apache.hadoop.hbase.constraint.TestConstraint Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.43 sec Running org.apache.hadoop.hbase.mapred.TestTableInputFormat Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 117.992 sec Running org.apache.hadoop.hbase.snapshot.TestExportSnapshot Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 164.803 sec Running org.apache.hadoop.hbase.master.TestAssignmentManager Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.831 sec Running org.apache.hadoop.hbase.master.TestOpenedRegionHandler Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.684 sec Running org.apache.hadoop.hbase.master.TestActiveMasterManager Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.645 sec Running org.apache.hadoop.hbase.mapred.TestTableMapReduce Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 214.507 sec Running org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.676 sec Running org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.583 sec Running org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 242.219 sec Running org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 261.782 sec Running org.apache.hadoop.hbase.master.TestMasterTransitions Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 26.02 sec Running org.apache.hadoop.hbase.master.TestMasterFailover Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.402 sec Running org.apache.hadoop.hbase.master.TestRollingRestart Running org.apache.hadoop.hbase.master.TestMasterStatusServlet Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.354 sec Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.989 sec Running org.apache.hadoop.hbase.master.TestHMasterRPCException Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.382 sec Running org.apache.hadoop.hbase.master.TestRestartCluster Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 42.674 sec Running org.apache.hadoop.hbase.master.TestMasterMetrics Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.949 sec Running org.apache.hadoop.hbase.master.balancer.TestBaseLoadBalancer Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.035 sec Running org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.611 sec Running org.apache.hadoop.hbase.master.balancer.TestDefaultLoadBalancer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.336 sec Running org.apache.hadoop.hbase.master.TestMaster Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.167 sec Running org.apache.hadoop.hbase.master.TestTableLockManager Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.315 sec Running org.apache.hadoop.hbase.master.TestRegionPlacement Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.016 sec Running org.apache.hadoop.hbase.master.cleaner.TestHFileCleaner Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.392 sec Running org.apache.hadoop.hbase.master.cleaner.TestLogsCleaner Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.347 sec Running org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.904 sec Running org.apache.hadoop.hbase.master.TestMasterShutdown Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.947 sec Running org.apache.hadoop.hbase.master.TestMasterFileSystem Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.359 sec Running org.apache.hadoop.hbase.master.handler.TestTableDeleteFamilyHandler Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.802 sec Running org.apache.hadoop.hbase.master.handler.TestTableDescriptorModification Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.423 sec Running org.apache.hadoop.hbase.master.TestClusterStatusPublisher Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 sec Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.38 sec Running org.apache.hadoop.hbase.master.handler.TestCreateTableHandler Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.255 sec Running org.apache.hadoop.hbase.master.TestMasterNoCluster Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.855 sec Running org.apache.hadoop.hbase.master.TestDeadServer Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.441 sec Running org.apache.hadoop.hbase.master.TestMasterMetricsWrapper Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.238 sec Running org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.284 sec Running org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 110.637 sec Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting Running org.apache.hadoop.hbase.master.TestSplitLogManager Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.536 sec Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 155.818 sec Results : Tests run: 1526, Failures: 0, Errors: 0, Skipped: 19 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping HBase [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] HBase ............................................. SUCCESS [6.125s] [INFO] HBase - Common .................................... SUCCESS [22.939s] [INFO] HBase - Protocol .................................. SUCCESS [9.225s] [INFO] HBase - Client .................................... SUCCESS [22.965s] [INFO] HBase - Hadoop Compatibility ...................... SUCCESS [5.374s] [INFO] HBase - Hadoop Two Compatibility .................. SUCCESS [4.675s] [INFO] HBase - Prefix Tree ............................... SUCCESS [3.512s] [INFO] HBase - Server .................................... FAILURE [45:18.738s] [INFO] HBase - Integration Tests ......................... SKIPPED [INFO] HBase - Examples .................................. SKIPPED [INFO] HBase - Assembly .................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 46:36.623s [INFO] Finished at: Sat Sep 14 23:18:33 UTC 2013 [INFO] Final Memory: 71M/855M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test (secondPartTestsExecution) on project hbase-server: Failure or timeout -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hbase-server Build step 'Invoke top-level Maven targets' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : ZOMBIE_TESTS_COUNT=`jps | grep surefirebooter | wc -l` if [[ $ZOMBIE_TESTS_COUNT != 0 ]] ; then #It seems sometimes the tests are not dying immediately. Let's give them 10s echo "Suspicious java process found - waiting 10s to see if there are just slow to stop" sleep 10 ZOMBIE_TESTS_COUNT=`jps | grep surefirebooter | wc -l` if [[ $ZOMBIE_TESTS_COUNT != 0 ]] ; then echo "There are $ZOMBIE_TESTS_COUNT zombie tests, they should have been killed by surefire but survived" echo "************ BEGIN zombies jstack extract" ZB_STACK=`jps | grep surefirebooter | cut -d ' ' -f 1 | xargs -n 1 jstack | grep ".test" | grep "\.java"` jps | grep surefirebooter | cut -d ' ' -f 1 | xargs -n 1 jstack echo "************ END zombies jstack extract" JIRA_COMMENT="$JIRA_COMMENT {color:red}-1 core zombie tests{color}. There are ${ZOMBIE_TESTS_COUNT} zombie test(s): ${ZB_STACK}" BAD=1 jps | grep surefirebooter | cut -d ' ' -f 1 | xargs kill -9 else echo "We're ok: there is no zombie test, but some tests took some time to stop" fi else echo "We're ok: there is no zombie test" fi [hbase-0.96-hadoop2] $ /bin/bash -xe /tmp/hudson7410634716653729382.sh ++ jps ++ grep surefirebooter ++ wc -l + ZOMBIE_TESTS_COUNT=0 + [[ 0 != 0 ]] + echo 'We'\''re ok: there is no zombie test' We're ok: there is no zombie test POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Archiving artifacts Recording test results Updating HBASE-9480
