TestImportExport hanging is HBASE-14915 St.Ack On Sat, Dec 5, 2015 at 6:03 PM, Stack <st...@duboce.net> wrote:
> Two hanging tests: > > Hanging test : org.apache.hadoop.hbase.mapreduce.TestImportExport > Hanging test : org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot > > St.Ack > > > On Sat, Dec 5, 2015 at 3:03 PM, Apache Jenkins Server < > jenk...@builds.apache.org> wrote: > >> See < >> https://builds.apache.org/job/HBase-1.2/jdk=latest1.7,label=Hadoop/426/changes >> > >> >> Changes: >> >> [stack] HBASE-1422 Delayed flush doesn't work causing flush storms; >> addendum by >> >> ------------------------------------------ >> [...truncated 44637 lines...] >> Running org.apache.hadoop.hbase.io.hfile.TestSeekBeforeWithInlineBlocks >> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.867 >> sec - in org.apache.hadoop.hbase.io.hfile.TestHFileBlockIndex >> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.05 >> sec - in org.apache.hadoop.hbase.io.hfile.TestHFileBlock >> Running org.apache.hadoop.hbase.io.hfile.TestCacheConfig >> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.968 sec >> - in org.apache.hadoop.hbase.io.hfile.TestCacheConfig >> Running org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks >> Running org.apache.hadoop.hbase.io.TestFileLink >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.869 >> sec - in org.apache.hadoop.hbase.io.hfile.TestSeekBeforeWithInlineBlocks >> Running org.apache.hadoop.hbase.procedure.TestZKProcedure >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.073 sec >> - in org.apache.hadoop.hbase.procedure.TestZKProcedure >> Running org.apache.hadoop.hbase.procedure.TestZKProcedureControllers >> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.854 >> sec - in org.apache.hadoop.hbase.io.TestFileLink >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.914 sec >> - in org.apache.hadoop.hbase.procedure.TestZKProcedureControllers >> Running org.apache.hadoop.hbase.TestHBaseOnOtherDfsCluster >> Running org.apache.hadoop.hbase.ipc.TestHBaseClient >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.594 sec >> - in org.apache.hadoop.hbase.ipc.TestHBaseClient >> Running org.apache.hadoop.hbase.ipc.TestDelayedRpc >> Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 332.951 >> sec - in org.apache.hadoop.hbase.io.encoding.TestEncodedSeekers >> Running org.apache.hadoop.hbase.ipc.TestProtoBufRpc >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.376 sec >> - in org.apache.hadoop.hbase.ipc.TestDelayedRpc >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.671 >> sec - in org.apache.hadoop.hbase.TestHBaseOnOtherDfsCluster >> Running org.apache.hadoop.hbase.TestAcidGuarantees >> Running org.apache.hadoop.hbase.TestLocalHBaseCluster >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.053 sec >> - in org.apache.hadoop.hbase.ipc.TestProtoBufRpc >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.433 >> sec - in org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.788 >> sec - in org.apache.hadoop.hbase.TestLocalHBaseCluster >> Running org.apache.hadoop.hbase.zookeeper.TestRecoverableZooKeeper >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.949 sec >> - in org.apache.hadoop.hbase.zookeeper.TestRecoverableZooKeeper >> Running org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide >> Running org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker >> Running org.apache.hadoop.hbase.zookeeper.TestZooKeeperACL >> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.526 sec >> - in org.apache.hadoop.hbase.zookeeper.TestZooKeeperACL >> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.721 sec >> - in org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker >> Running org.apache.hadoop.hbase.zookeeper.TestZKTableStateManager >> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.992 >> sec - in org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide >> Running org.apache.hadoop.hbase.zookeeper.TestHQuorumPeer >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.778 sec >> - in org.apache.hadoop.hbase.zookeeper.TestZKTableStateManager >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.627 sec >> - in org.apache.hadoop.hbase.zookeeper.TestHQuorumPeer >> Running org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.516 sec >> - in org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager >> Running org.apache.hadoop.hbase.zookeeper.TestZKMulti >> Running >> org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock >> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.526 >> sec - in org.apache.hadoop.hbase.zookeeper.TestZKMulti >> Running org.apache.hadoop.hbase.TestMetaTableAccessorNoCluster >> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.532 sec >> - in org.apache.hadoop.hbase.TestMetaTableAccessorNoCluster >> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.644 >> sec - in >> org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock >> Running org.apache.hadoop.hbase.TestJMXListener >> Running org.apache.hadoop.hbase.TestZooKeeper >> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.685 >> sec - in org.apache.hadoop.hbase.TestJMXListener >> Running >> org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager >> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 110.495 >> sec - in org.apache.hadoop.hbase.TestAcidGuarantees >> Running >> org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint >> Running >> org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster >> Tests run: 72, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 366.144 >> sec - in org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite >> Running >> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.977 >> sec - in >> org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster >> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.44 sec >> - in org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink >> Running org.apache.hadoop.hbase.replication.TestReplicationTrackerZKImpl >> Running >> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager >> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.559 sec >> - in org.apache.hadoop.hbase.replication.TestReplicationTrackerZKImpl >> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.785 sec >> - in >> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager >> Running org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS >> Running >> org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed >> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 158.836 >> sec - in >> org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint >> Running org.apache.hadoop.hbase.replication.TestMultiSlaveReplication >> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 210.102 >> sec - in org.apache.hadoop.hbase.TestZooKeeper >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.841 >> sec - in org.apache.hadoop.hbase.replication.TestMultiSlaveReplication >> Running >> org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.37 sec >> - in >> org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 115.445 >> sec - in >> org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed >> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 297.904 >> sec - in >> org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 119.934 >> sec - in org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS >> Running org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS >> Running >> org.apache.hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleWAL >> Running org.apache.hadoop.hbase.replication.TestPerTableCFReplication >> Running >> org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL >> Running >> org.apache.hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL >> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.544 >> sec - in org.apache.hadoop.hbase.replication.TestPerTableCFReplication >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.333 >> sec - in >> org.apache.hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL >> Running org.apache.hadoop.hbase.replication.TestMasterReplication >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.021 >> sec - in >> org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 135.471 >> sec - in org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 132.053 >> sec - in >> org.apache.hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleWAL >> Running >> org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.311 >> sec - in >> org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer >> Running org.apache.hadoop.hbase.replication.TestReplicationWithTags >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.406 sec >> - in org.apache.hadoop.hbase.replication.TestReplicationWithTags >> Running org.apache.hadoop.hbase.replication.TestReplicationSource >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.893 sec >> - in org.apache.hadoop.hbase.replication.TestReplicationSource >> Running org.apache.hadoop.hbase.replication.TestReplicationEndpoint >> Running org.apache.hadoop.hbase.replication.TestReplicationStateZKImpl >> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.533 sec >> - in org.apache.hadoop.hbase.replication.TestReplicationStateZKImpl >> Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests >> Running org.apache.hadoop.hbase.fs.TestBlockReorder >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 136.018 >> sec - in org.apache.hadoop.hbase.replication.TestMasterReplication >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.501 >> sec - in org.apache.hadoop.hbase.replication.TestReplicationEndpoint >> Running org.apache.hadoop.hbase.TestMovedRegionsCleaner >> Running org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.605 >> sec - in org.apache.hadoop.hbase.TestMovedRegionsCleaner >> Running org.apache.hadoop.hbase.trace.TestHTraceHooks >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.124 sec >> - in org.apache.hadoop.hbase.trace.TestHTraceHooks >> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 62.585 >> sec - in org.apache.hadoop.hbase.fs.TestBlockReorder >> Running org.apache.hadoop.hbase.TestMetaTableLocator >> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.464 sec >> - in org.apache.hadoop.hbase.TestMetaTableLocator >> Running org.apache.hadoop.hbase.namespace.TestNamespaceAuditor >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 74.735 >> sec - in org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool >> Running org.apache.hadoop.hbase.mapred.TestTableMapReduceUtil >> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 129.209 >> sec - in org.apache.hadoop.hbase.replication.TestReplicationSmallTests >> Running org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat >> Running org.apache.hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat >> Running org.apache.hadoop.hbase.mapred.TestTableInputFormat >> Tests run: 12, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 114.177 >> sec - in org.apache.hadoop.hbase.namespace.TestNamespaceAuditor >> Running org.apache.hadoop.hbase.mapred.TestTableMapReduce >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 238.115 >> sec - in org.apache.hadoop.hbase.mapred.TestTableMapReduceUtil >> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 215.444 >> sec - in org.apache.hadoop.hbase.mapred.TestTableInputFormat >> Running org.apache.hadoop.hbase.TestMetaMigrationConvertingToPB >> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.185 sec >> - in org.apache.hadoop.hbase.TestMetaMigrationConvertingToPB >> Running org.apache.hadoop.hbase.TestMultiVersions >> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.552 >> sec - in org.apache.hadoop.hbase.TestMultiVersions >> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 266.167 >> sec - in org.apache.hadoop.hbase.mapred.TestTableMapReduce >> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 432.74 >> sec - in org.apache.hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat >> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 604.913 >> sec - in org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat >> >> Results : >> >> Flaked tests: >> >> org.apache.hadoop.hbase.master.cleaner.TestLogsCleaner.testZnodeCversionChange(org.apache.hadoop.hbase.master.cleaner.TestLogsCleaner) >> Run 1: TestLogsCleaner.testZnodeCversionChange:156 » TestTimedOut test >> timed out afte... >> Run 2: PASS >> >> >> Tests run: 2328, Failures: 0, Errors: 0, Skipped: 40, Flakes: 1 >> >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] Reactor Summary: >> [INFO] >> [INFO] Apache HBase ...................................... SUCCESS >> [1:48.639s] >> [INFO] Apache HBase - Checkstyle ......................... SUCCESS >> [9.778s] >> [INFO] Apache HBase - Resource Bundle .................... SUCCESS >> [0.557s] >> [INFO] Apache HBase - Annotations ........................ SUCCESS >> [2.207s] >> [INFO] Apache HBase - Protocol ........................... SUCCESS >> [37.461s] >> [INFO] Apache HBase - Common ............................. SUCCESS >> [6:42.305s] >> [INFO] Apache HBase - Procedure .......................... SUCCESS >> [5:07.741s] >> [INFO] Apache HBase - Client ............................. SUCCESS >> [2:17.762s] >> [INFO] Apache HBase - Hadoop Compatibility ............... SUCCESS >> [13.054s] >> [INFO] Apache HBase - Hadoop Two Compatibility ........... SUCCESS >> [18.704s] >> [INFO] Apache HBase - Prefix Tree ........................ SUCCESS >> [22.761s] >> [INFO] Apache HBase - Server ............................. FAILURE >> [2:57:23.015s] >> [INFO] Apache HBase - Testing Util ....................... SKIPPED >> [INFO] Apache HBase - Thrift ............................. SKIPPED >> [INFO] Apache HBase - Rest ............................... SKIPPED >> [INFO] Apache HBase - Shell .............................. SKIPPED >> [INFO] Apache HBase - Integration Tests .................. SKIPPED >> [INFO] Apache HBase - Examples ........................... SKIPPED >> [INFO] Apache HBase - External Block Cache ............... SKIPPED >> [INFO] Apache HBase - Assembly ........................... SKIPPED >> [INFO] Apache HBase - Shaded ............................. SKIPPED >> [INFO] Apache HBase - Shaded - Client .................... SKIPPED >> [INFO] Apache HBase - Shaded - Server .................... SKIPPED >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] BUILD FAILURE >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] Total time: 3:15:39.480s >> [INFO] Finished at: Sat Dec 05 22:58:21 UTC 2015 >> [INFO] Final Memory: 435M/661M >> [INFO] >> ------------------------------------------------------------------------ >> [ERROR] Failed to execute goal >> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test >> (secondPartTestsExecution) on project hbase-server: There was a timeout or >> other error in the fork -> [Help 1] >> [ERROR] >> [ERROR] To see the full stack trace of the errors, re-run Maven with the >> -e switch. >> [ERROR] Re-run Maven using the -X switch to enable full debug logging. >> [ERROR] >> [ERROR] For more information about the errors and possible solutions, >> please read the following articles: >> [ERROR] [Help 1] >> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException >> [ERROR] >> [ERROR] After correcting the problems, you can resume the build with the >> command >> [ERROR] mvn <goals> -rf :hbase-server >> Build step 'Invoke top-level Maven targets' marked build as failure >> Performing Post build task... >> Match found for :.* : True >> Logical operation result is TRUE >> Running script : # Post-build task script. TODO: Check this in and have >> all builds reference check-in. >> pwd && ls >> # NOTE!!!! The below code has been copied and pasted from >> ./dev-tools/run-test.sh >> # Do not change here without syncing there and vice-versa. >> ZOMBIE_TESTS_COUNT=`jps -v | grep surefirebooter | grep -e '-Dhbase.test' >> | wc -l` >> if [[ $ZOMBIE_TESTS_COUNT != 0 ]] ; then >> echo "Suspicious java process found - waiting 30s to see if there are >> just slow to stop" >> sleep 30 >> ZOMBIE_TESTS_COUNT=`jps -v | grep surefirebooter | grep -e >> '-Dhbase.test' | wc -l` >> if [[ $ZOMBIE_TESTS_COUNT != 0 ]] ; then >> echo " {color:red}There appear to be $ZOMBIE_TESTS_COUNT zombie >> tests{color}, they should have been killed by surefire but survived" >> jps -v | grep surefirebooter | grep -e '-Dhbase.test' >> jps -v | grep surefirebooter | grep -e '-Dhbase.test' | cut -d ' ' -f >> 1 | xargs -n 1 jstack >> # Exit with error >> exit 1 >> else >> echo "We're ok: there is no zombie test, but some tests took some time >> to stop" >> fi >> else >> echo "We're ok: there is no zombie test" >> fi >> [Hadoop] $ /bin/bash -xe /tmp/hudson3971431945764947670.sh >> + pwd >> <https://builds.apache.org/job/HBase-1.2/jdk=latest1.7,label=Hadoop/ws/> >> + ls >> bin >> CHANGES.txt >> conf >> dev-support >> hbase-annotations >> hbase-assembly >> hbase-checkstyle >> hbase-client >> hbase-common >> hbase-examples >> hbase-external-blockcache >> hbase-hadoop2-compat >> hbase-hadoop-compat >> hbase-it >> hbase-native-client >> hbase-prefix-tree >> hbase-procedure >> hbase-protocol >> hbase-resource-bundle >> hbase-rest >> hbase-server >> hbase-shaded >> hbase-shell >> hbase-spark >> hbase-testing-util >> hbase-thrift >> LICENSE.txt >> NOTICE.txt >> pom.xml >> README.txt >> src >> target >> ++ jps -v >> ++ grep surefirebooter >> ++ grep -e -Dhbase.test >> ++ wc -l >> + ZOMBIE_TESTS_COUNT=0 >> + [[ 0 != 0 ]] >> + echo 'We'\''re ok: there is no zombie test' >> We're ok: there is no zombie test >> POST BUILD TASK : SUCCESS >> END OF POST BUILD TASK : 0 >> Archiving artifacts >> Recording test results >> > >
