See <https://builds.apache.org/job/HBase-TRUNK/2526/changes>
Changes: [nspiegelberg] [jira] [HBASE-4908] HBase cluster test tool (port from 0.89-fb) Summary: Porting one of our HBase cluster test tools (a single-process multi-threaded load generator and verifier) from 0.89-fb to trunk. I cleaned up the code a bit compared to what's in 0.89-fb, and discovered that it has some features that I have not tried yet (some kind of a kill test, and some way to run HBase as multiple processes on one machine). The main utility of this piece of code for us has been the HBaseClusterTest command-line tool (called HBaseTest in 0.89-fb), which we usually invoke as a load test in our five-node dev cluster testing, e.g.: hbase org.apache.hadoop.hbase.util.LoadTestTool -write 50:100:20 -tn loadtest4 -read 100:10 -zk <zk_quorum_node> -bloom ROWCOL -compression LZO -key_window 5 -max_read_errors 10000 -num_keys 10000000000 -start_key 0 Test Plan: Run this on a dev cluster. Run all unit tests. Reviewers: stack, Karthik, Kannan, nspiegelberg, JIRA Reviewed By: nspiegelberg CC: stack, nspiegelberg, mbautin, Karthik Differential Revision: 549 [stack] HBASE-4976 Add compaction/flush queue size metrics mistakenly removed by HFile v2 [tedyu] HBASE-4610 Addendum removes break statement [tedyu] HBASE-4610 Port HBASE-3380 (Master failover can split logs of live servers) to 92/trunk ------------------------------------------ [...truncated 1902 lines...] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.156 sec Running org.apache.hadoop.hbase.client.TestInstantSchemaChange Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 231.037 sec Running org.apache.hadoop.hbase.master.TestActiveMasterManager Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.373 sec Running org.apache.hadoop.hbase.master.TestMasterStatusServlet Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.693 sec Running org.apache.hadoop.hbase.master.TestHMasterRPCException Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.555 sec Running org.apache.hadoop.hbase.master.TestDeadServer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.083 sec Running org.apache.hadoop.hbase.master.TestRollingRestart Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.61 sec Running org.apache.hadoop.hbase.master.TestLogsCleaner Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.563 sec Running org.apache.hadoop.hbase.master.TestOpenedRegionHandler Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.321 sec Running org.apache.hadoop.hbase.master.TestMasterTransitions Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0 sec Running org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.818 sec Running org.apache.hadoop.hbase.master.TestRestartCluster Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.133 sec Running org.apache.hadoop.hbase.master.TestDefaultLoadBalancer Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.231 sec Running org.apache.hadoop.hbase.master.TestMasterFailover Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 69.532 sec Running org.apache.hadoop.hbase.master.TestClockSkewDetection Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.125 sec Running org.apache.hadoop.hbase.master.TestMaster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.714 sec Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 127.414 sec Running org.apache.hadoop.hbase.master.TestSplitLogManager Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.943 sec Running org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.049 sec Running org.apache.hadoop.hbase.TestFullLogReconstruction Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.467 sec Running org.apache.hadoop.hbase.metrics.TestMetricsMBeanBase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.115 sec Running org.apache.hadoop.hbase.TestGlobalMemStoreSize Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.751 sec Running org.apache.hadoop.hbase.TestZooKeeper Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.52 sec Running org.apache.hadoop.hbase.replication.TestMasterReplication Running org.apache.hadoop.hbase.replication.TestReplicationSource Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.556 sec Running org.apache.hadoop.hbase.replication.TestReplication Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.084 sec Running org.apache.hadoop.hbase.replication.TestReplicationPeer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.049 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.252 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.191 sec Running org.apache.hadoop.hbase.replication.TestMultiSlaveReplication Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.385 sec Running org.apache.hadoop.hbase.TestHBaseTestingUtility Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 90.342 sec Running org.apache.hadoop.hbase.TestMultiVersions Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.503 sec Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditorNoCluster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.069 sec Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditor Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.306 sec Running org.apache.hadoop.hbase.catalog.TestCatalogTracker Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 2.959 sec Running org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.636 sec Running org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol Tests run: 44, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.006 sec Running org.apache.hadoop.hbase.coprocessor.TestMasterObserver Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.848 sec Running org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.432 sec Running org.apache.hadoop.hbase.coprocessor.TestClassLoading Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.907 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.106 sec Running org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.11 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.743 sec Running org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.021 sec Running org.apache.hadoop.hbase.coprocessor.TestWALObserver Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.587 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.484 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.056 sec Running org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.235 sec Running org.apache.hadoop.hbase.avro.TestAvroServer Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.527 sec Running org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 75.838 sec Running org.apache.hadoop.hbase.thrift.TestThriftServer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.272 sec Running org.apache.hadoop.hbase.monitoring.TestTaskMonitor Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.167 sec Running org.apache.hadoop.hbase.monitoring.TestMemoryBoundedLogMessageBuffer Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 sec Running org.apache.hadoop.hbase.regionserver.TestHRegion Tests run: 60, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.71 sec Running org.apache.hadoop.hbase.regionserver.TestMasterAddressManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.111 sec Running org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.294 sec Running org.apache.hadoop.hbase.regionserver.TestSplitLogWorker Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.018 sec Running org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.978 sec Running org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.553 sec Running org.apache.hadoop.hbase.regionserver.TestStore Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.253 sec Running org.apache.hadoop.hbase.regionserver.TestAtomicOperation Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.842 sec Running org.apache.hadoop.hbase.regionserver.TestBlocksRead Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.84 sec Running org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.334 sec Running org.apache.hadoop.hbase.regionserver.TestSeekOptimizations Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.011 sec Running org.apache.hadoop.hbase.regionserver.metrics.TestSchemaMetrics Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.849 sec Running org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.793 sec Running org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.621 sec Running org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.079 sec Running org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.367 sec Running org.apache.hadoop.hbase.regionserver.TestParallelPut Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.294 sec Running org.apache.hadoop.hbase.regionserver.wal.TestHLogBench Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.882 sec Running org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.872 sec Running org.apache.hadoop.hbase.regionserver.wal.TestLogRolling Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 303.797 sec Running org.apache.hadoop.hbase.regionserver.wal.TestHLog Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.598 sec Running org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit Tests run: 30, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 239.366 sec Running org.apache.hadoop.hbase.regionserver.wal.TestWALReplay Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.774 sec Running org.apache.hadoop.hbase.regionserver.TestStoreScanner Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec Running org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.294 sec Running org.apache.hadoop.hbase.regionserver.TestRpcMetrics Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.638 sec Running org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.545 sec Running org.apache.hadoop.hbase.mapred.TestTableMapReduce Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.94 sec Running org.apache.hadoop.hbase.mapred.TestTableInputFormat Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.548 sec Running org.apache.hadoop.hbase.TestDrainingServer Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.164 sec Running org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.257 sec Running org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 273.944 sec Running org.apache.hadoop.hbase.mapreduce.TestImportTsv Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.151 sec Running org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 428.891 sec Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.064 sec Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.665 sec Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.504 sec Results : Tests in error: testHundredsOfTable(org.apache.hadoop.hbase.client.TestAdmin): Call to juno.apache.org/67.195.138.61:41217 failed on socket timeout exception: java.net.SocketTimeoutException: 1500 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/67.195.138.61:57167 remote=juno.apache.org/67.195.138.61:41217] Tests run: 777, Failures: 0, Errors: 1, Skipped: 9 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1:55:20.083s [INFO] Finished at: Thu Dec 08 08:43:00 UTC 2011 [INFO] Final Memory: 21M/1030M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.11-TRUNK-HBASE-2:test (secondPartTestsExecution) on project hbase: There are test failures. [ERROR] [ERROR] Please refer to <https://builds.apache.org/job/HBase-TRUNK/ws/trunk/target/surefire-reports> for the individual test results. [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.11-TRUNK-HBASE-2:test (secondPartTestsExecution) on project hbase: There are test failures. Please refer to <https://builds.apache.org/job/HBase-TRUNK/ws/trunk/target/surefire-reports> for the individual test results. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.MojoExecutor.executeForkedExecutions(MojoExecutor.java:365) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196) at org.apache.maven.cli.MavenCli.main(MavenCli.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352) Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures. Please refer to <https://builds.apache.org/job/HBase-TRUNK/ws/trunk/target/surefire-reports> for the individual test results. at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution(SurefireHelper.java:87) at org.apache.maven.plugin.surefire.SurefirePlugin.writeSummary(SurefirePlugin.java:641) at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary(SurefirePlugin.java:615) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:137) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:98) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) ... 23 more [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Build step 'Invoke top-level Maven targets' marked build as failure [locks-and-latches] Releasing all the locks [locks-and-latches] All the locks released Archiving artifacts Recording test results Updating HBASE-4610 Updating HBASE-4976 Updating HBASE-4908 Updating HBASE-3380
