See <https://builds.apache.org/job/HBase-TRUNK/2402/changes>
Changes: [dmeil] HBASE-4730 book.xml, ops_mgt.xml, performance.xml - handful of changes [nspiegelberg] HBASE-2312 [jira] Possible data loss when RS goes into GC pause while rolling HLog Summary: There is a very corner case when bad things could happen(ie data loss): 1) RS #1 is going to roll its HLog - not yet created the new one, old one will get no more writes 2) RS #1 enters GC Pause of Death 3) Master lists HLog files of RS#1 that is has to split as RS#1 is dead, starts splitting 4) RS #1 wakes up, created the new HLog (previous one was rolled) and appends an edit - which is lost The following seems like a possible solution: 1) Master detects RS#1 is dead 2) The master renames the /hbase/.logs/<regionserver name> directory to something else (say /hbase/.logs/<regionserver name>-dead) 3) Add mkdir support (as opposed to mkdirs) to HDFS - so that a file create fails if the directory doesn't exist. Dhruba tells me this is very doable. 4) RS#1 comes back up and is not able create the new hlog. It restarts itself. Test Plan: EMPTY Reviewers: JIRA, stack, khemani Reviewed By: khemani CC: tedyu, nspiegelberg, stack, Kannan, khemani, jgray Differential Revision: 99 [nspiegelberg] [jira] [HBASE-4627] Ability to specify a custom start/end to RegionSplitter Summary: [HBASE-4627] added a custom start/end row to RegionSplitter. Also solved an off-by-one error because the end row is prefix-inclusive and not exclusive. <a href="https://issues.apache.org/jira/browse/HBASE-4489"; title="Better key splitting in RegionSplitter"><del>HBASE-4489</del></a> changed the default endKey on HexStringSplit from 7FFF... to FFFF... While this is correct, existing users of 0.90 RegionSplitter have 7FFF as the end key in their schema and the last region will not split properly under this new code. We need to let the user specify a custom start/end key range for when situations like this arise. Optimally, we should also write the start/end key in META so we could figure this out implicitly instead of requiring the user to explicitly specify it. Test Plan: - mvn test -Dtest=TestRegionSplitter CC: JIRA Reviewers: DUMMY_REVIEWER Differential Revision: 39 [tedyu] HBASE-4716 Addendum Use Preconditions and remove enum ------------------------------------------ [...truncated 1749 lines...] Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditorNoCluster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.889 sec Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditor Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.897 sec Running org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.932 sec Running org.apache.hadoop.hbase.catalog.TestCatalogTracker Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.178 sec Running org.apache.hadoop.hbase.client.TestResult Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.142 sec Running org.apache.hadoop.hbase.client.TestTimestampsFilter Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.058 sec Running org.apache.hadoop.hbase.client.TestShell Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.649 sec Running org.apache.hadoop.hbase.client.TestFromClientSide Tests run: 49, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 156.366 sec Running org.apache.hadoop.hbase.client.TestHCM Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.333 sec Running org.apache.hadoop.hbase.client.TestAttributes Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.138 sec Running org.apache.hadoop.hbase.client.TestScannerTimeout Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.709 sec Running org.apache.hadoop.hbase.client.TestMetaScanner Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.368 sec Running org.apache.hadoop.hbase.client.TestHTableUtil Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.113 sec Running org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.849 sec Running org.apache.hadoop.hbase.client.TestOperation Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.573 sec Running org.apache.hadoop.hbase.client.TestHTablePool Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.166 sec Running org.apache.hadoop.hbase.client.TestGet Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.135 sec Running org.apache.hadoop.hbase.client.TestMultipleTimestamps Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.049 sec Running org.apache.hadoop.hbase.client.replication.TestReplicationAdmin Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.759 sec Running org.apache.hadoop.hbase.client.TestMultiParallel Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.261 sec Running org.apache.hadoop.hbase.client.TestAdmin Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 393.55 sec Running org.apache.hadoop.hbase.client.TestScan Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.138 sec Running org.apache.hadoop.hbase.TestHServerInfo Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.146 sec Running org.apache.hadoop.hbase.metrics.TestMetricsMBeanBase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.129 sec Running org.apache.hadoop.hbase.executor.TestExecutorService Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.438 sec Running org.apache.hadoop.hbase.rest.TestGzipFilter Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.311 sec Running org.apache.hadoop.hbase.rest.TestMultiRowResource Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.063 sec Running org.apache.hadoop.hbase.rest.TestScannersWithFilters Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.756 sec Running org.apache.hadoop.hbase.rest.TestRowResource Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.838 sec Running org.apache.hadoop.hbase.rest.model.TestCellModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.37 sec Running org.apache.hadoop.hbase.rest.model.TestCellSetModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406 sec Running org.apache.hadoop.hbase.rest.model.TestTableListModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.385 sec Running org.apache.hadoop.hbase.rest.model.TestVersionModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.367 sec Running org.apache.hadoop.hbase.rest.model.TestColumnSchemaModel Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.275 sec Running org.apache.hadoop.hbase.rest.model.TestTableInfoModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.401 sec Running org.apache.hadoop.hbase.rest.model.TestRowModel Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.37 sec Running org.apache.hadoop.hbase.rest.model.TestTableSchemaModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.404 sec Running org.apache.hadoop.hbase.rest.model.TestStorageClusterVersionModel Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.271 sec Running org.apache.hadoop.hbase.rest.model.TestTableRegionModel Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471 sec Running org.apache.hadoop.hbase.rest.model.TestStorageClusterStatusModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.447 sec Running org.apache.hadoop.hbase.rest.model.TestScannerModel Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.407 sec Running org.apache.hadoop.hbase.rest.client.TestRemoteAdmin Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.508 sec Running org.apache.hadoop.hbase.rest.client.TestRemoteTable Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.228 sec Running org.apache.hadoop.hbase.rest.TestVersionResource Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.542 sec Running org.apache.hadoop.hbase.rest.TestScannerResource Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.331 sec Running org.apache.hadoop.hbase.rest.TestTransform Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.512 sec Running org.apache.hadoop.hbase.rest.TestTableResource Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.43 sec Running org.apache.hadoop.hbase.rest.TestStatusResource Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.575 sec Running org.apache.hadoop.hbase.rest.TestSchemaResource Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.314 sec Running org.apache.hadoop.hbase.thrift.TestThriftServer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.838 sec Running org.apache.hadoop.hbase.io.TestHeapSize Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.158 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileSeek Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.74 sec Running org.apache.hadoop.hbase.io.hfile.slab.TestSlab Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.143 sec Running org.apache.hadoop.hbase.io.hfile.slab.TestSlabCache Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.77 sec Running org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.768 sec Running org.apache.hadoop.hbase.io.hfile.TestCachedBlockQueue Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.104 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileBlockIndex Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.147 sec Running org.apache.hadoop.hbase.io.hfile.TestLruBlockCache Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.343 sec Running org.apache.hadoop.hbase.io.hfile.TestHFilePerformance Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.841 sec Running org.apache.hadoop.hbase.io.hfile.TestFixedFileTrailer Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.559 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.962 sec Running org.apache.hadoop.hbase.io.hfile.TestBlockCacheColumnFamilySummary Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.056 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileReaderV1 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.58 sec Running org.apache.hadoop.hbase.io.hfile.TestHFile Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.866 sec Running org.apache.hadoop.hbase.io.hfile.TestReseekTo Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.652 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileBlock Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.214 sec Running org.apache.hadoop.hbase.io.hfile.TestSeekTo Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.682 sec Running org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.414 sec Running org.apache.hadoop.hbase.io.TestHbaseObjectWritable Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.467 sec Running org.apache.hadoop.hbase.io.TestImmutableBytesWritable Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec Running org.apache.hadoop.hbase.io.TestHalfStoreFileReader Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.706 sec Running org.apache.hadoop.hbase.migration.TestMigrationFrom090To092 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.269 sec Running org.apache.hadoop.hbase.TestFSTableDescriptorForceCreation Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.522 sec Running org.apache.hadoop.hbase.avro.TestAvroUtil Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.63 sec Running org.apache.hadoop.hbase.avro.TestAvroServer Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.143 sec Running org.apache.hadoop.hbase.TestKeyValue Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.13 sec Running org.apache.hadoop.hbase.replication.TestMultiSlaveReplication Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.917 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.897 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.408 sec Running org.apache.hadoop.hbase.replication.TestMasterReplication Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 75.124 sec Running org.apache.hadoop.hbase.replication.TestReplicationSource Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.933 sec Running org.apache.hadoop.hbase.replication.TestReplicationPeer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.516 sec Running org.apache.hadoop.hbase.replication.TestReplication Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 125.053 sec Running org.apache.hadoop.hbase.ipc.TestDelayedRpc Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.525 sec Running org.apache.hadoop.hbase.TestCompare Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.238 sec Running org.apache.hadoop.hbase.TestZooKeeper Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 81.357 sec Running org.apache.hadoop.hbase.TestHBaseTestingUtility Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 74.664 sec Results : Failed tests: testGlobalMemStore(org.apache.hadoop.hbase.TestGlobalMemStoreSize): Server=janus.apache.org,45000,1320271047103, i=1 expected:<0> but was:<496> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): LoadIncrementalHFiles should put expected data in table expected:<1024> but was:<746> Tests in error: testMasterFailoverWithMockedRITOnDeadRS(org.apache.hadoop.hbase.master.TestMasterFailover): test timed out after 180000 milliseconds Tests run: 1089, Failures: 2, Errors: 1, Skipped: 10 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1:38:39.433s [INFO] Finished at: Wed Nov 02 22:44:54 UTC 2011 [INFO] Final Memory: 66M/1463M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.9:test (default-test) on project hbase: There are test failures. [ERROR] [ERROR] Please refer to <https://builds.apache.org/job/HBase-TRUNK/ws/trunk/target/surefire-reports> for the individual test results. [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.9:test (default-test) on project hbase: There are test failures. Please refer to <https://builds.apache.org/job/HBase-TRUNK/ws/trunk/target/surefire-reports> for the individual test results. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.MojoExecutor.executeForkedExecutions(MojoExecutor.java:365) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196) at org.apache.maven.cli.MavenCli.main(MavenCli.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352) Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures. Please refer to <https://builds.apache.org/job/HBase-TRUNK/ws/trunk/target/surefire-reports> for the individual test results. at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution(SurefireHelper.java:87) at org.apache.maven.plugin.surefire.SurefirePlugin.writeSummary(SurefirePlugin.java:631) at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary(SurefirePlugin.java:605) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:136) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:97) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) ... 23 more [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Build step 'Invoke top-level Maven targets' marked build as failure [locks-and-latches] Releasing all the locks [locks-and-latches] All the locks released Archiving artifacts Recording test results Updating HBASE-4489 Updating HBASE-4716 Updating HBASE-4730 Updating HBASE-2312 Updating HBASE-4627
