[
https://issues.apache.org/jira/browse/HBASE-10977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-10977.
-----------------------------------
Resolution: Cannot Reproduce
> TestHBaseFsck.testQuarantineMissingHFile fails missing files test
> -----------------------------------------------------------------
>
> Key: HBASE-10977
> URL: https://issues.apache.org/jira/browse/HBASE-10977
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.15
> Reporter: stack
> Priority: Minor
>
> On our internal rig, ran into this failure:
> {code}
> java.lang.AssertionError: expected:<2> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.doQuarantineTest(TestHBaseFsck.java:1737)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingHFile(TestHBaseFsck.java:1781)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {code}
> This is what is failing:
> assertEquals(hfcc.getMissing().size(), missing);
> The file remove has not run yet or, else, this complaint is related:
> {code}
> 2014-04-12 23:24:57,638 WARN [IPC Server handler 4 on 50919]
> security.UserGroupInformation(1551): PriviledgedActionException as:jenkins
> (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist:
> /user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc
> {code}
> Below is full log. Will come back and add logging....
> {code}
> Regression
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingHFile
> Failing for the past 1 build (Since Failed#3 )
> Took 10 ms.
> add description
> Error Message
> expected:<2> but was:<1>
> Stacktrace
> java.lang.AssertionError: expected:<2> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.doQuarantineTest(TestHBaseFsck.java:1737)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingHFile(TestHBaseFsck.java:1781)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Standard Output
> Allow checking/fixes for table: testQuarantineMissingHFile
> Checked 4 hfile for corruption
> HFiles corrupted: 0
> HFiles successfully quarantined: 0
> HFiles failed quarantine: 0
> HFiles moved while checking: 2
>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc
>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/fam/9dd79f30f29e4cfeaa46f6e20b32e078
> Summary: OK => OK
> Version: 0.96.1.1-cdh5.0.1-SNAPSHOT
> ---- Table 'testQuarantineMissingHFile': region split map
> : [ { meta => null, hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e,
> deployed => }, A]
> A: [ { meta => null, hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef,
> deployed => }, B]
> B: [ { meta => null, hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351,
> deployed => }, C]
> C: [ { meta => null, hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d,
> deployed => }, ]
> null:
> ---- Table 'testQuarantineMissingHFile': overlap groups
> There are 0 overlap groups with 0 overlapping regions
> ---- Table 'hbase:meta': region split map
> : [ { meta => null, hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/hbase/meta/1588230740,
> deployed => }, ]
> null:
> ---- Table 'hbase:meta': overlap groups
> There are 0 overlap groups with 0 overlapping regions
> Number of live region servers: 3
> p0419.mtv.cloudera.com,41017,1397370264982
> p0419.mtv.cloudera.com,39010,1397370264935
> p0419.mtv.cloudera.com,32877,1397370265023
> Number of dead region servers: 0
> Master: p0419.mtv.cloudera.com,33001,1397370264247
> Number of backup masters: 0
> Average load: 2.0
> Number of requests: 655
> Number of regions: 6
> Number of regions in transition: 0
> RegionServer: p0419.mtv.cloudera.com,32877,1397370265023 number of regions: 0
> RegionServer: p0419.mtv.cloudera.com,39010,1397370264935 number of regions: 0
> RegionServer: p0419.mtv.cloudera.com,41017,1397370264982 number of regions: 1
> hbase:meta,,1.1588230740 id: 1 encoded_name: 1588230740 start: end:
> Number of empty REGIONINFO_QUALIFIER rows in hbase:meta: 0
> Number of Tables: 0
> Number of Tables in flux: 1
> ---- Table 'hbase:meta': region split map
> : [ { meta => hbase:meta,,1.1588230740, hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/hbase/meta/1588230740,
> deployed =>
> p0419.mtv.cloudera.com,41017,1397370264982;hbase:meta,,1.1588230740 }, ]
>
> null:
> ---- Table 'hbase:meta': overlap groups
> There are 0 overlap groups with 0 overlapping regions
> Summary:
> testQuarantineMissingHFile is okay.
> Number of regions: 0
> Deployed on:
> hbase:meta is okay.
> Number of regions: 1
> Deployed on: p0419.mtv.cloudera.com,41017,1397370264982
> 0 inconsistencies detected.
> Status: OK
> Standard Error
> 2014-04-12 23:24:54,802 INFO [pool-1-thread-1] hbase.ResourceChecker(147):
> before: util.TestHBaseFsck#testQuarantineMissingHFile Thread=410,
> OpenFileDescriptor=691, MaxFileDescriptor=32768, SystemLoadAverage=422,
> ProcessCount=485, AvailableMemoryMB=3139, ConnectionCount=5
> 2014-04-12 23:24:54,809 INFO [RpcServer.handler=0,port=33001]
> master.HMaster(1748): Client=jenkins//172.29.122.11 create
> 'testQuarantineMissingHFile', {NAME => 'fam', DATA_BLOCK_ENCODING => 'NONE',
> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION
> => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
> 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
> 2014-04-12 23:24:54,886 DEBUG [RpcServer.handler=0,port=33001]
> lock.ZKInterProcessLockBase(226): Acquired a lock for
> /hbase/table-lock/testQuarantineMissingHFile/write-master:330010000000000
> 2014-04-12 23:24:54,960 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.CreateTableHandler(148): Create table testQuarantineMissingHFile
> 2014-04-12 23:24:55,060 INFO [IPC Server handler 6 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741867_1043{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,061 INFO [IPC Server handler 3 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to
> blk_1073741867_1043{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,062 INFO [IPC Server handler 5 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to blk_1073741867_1043 size 305
> 2014-04-12 23:24:55,185 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> util.FSTableDescriptors(640): Wrote descriptor into:
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/.tabledesc/.tableinfo.0000000001
> 2014-04-12 23:24:55,188 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-1]
> regionserver.HRegion(3969): creating HRegion testQuarantineMissingHFile HTD
> == 'testQuarantineMissingHFile', {NAME => 'fam', DATA_BLOCK_ENCODING =>
> 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1',
> COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false',
> BLOCKCACHE => 'true'} RootDir =
> hdfs://localhost:50919/user/jenkins/hbase/.tmp Table name ==
> testQuarantineMissingHFile
> 2014-04-12 23:24:55,188 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-2]
> regionserver.HRegion(3969): creating HRegion testQuarantineMissingHFile HTD
> == 'testQuarantineMissingHFile', {NAME => 'fam', DATA_BLOCK_ENCODING =>
> 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1',
> COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false',
> BLOCKCACHE => 'true'} RootDir =
> hdfs://localhost:50919/user/jenkins/hbase/.tmp Table name ==
> testQuarantineMissingHFile
> 2014-04-12 23:24:55,188 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-3]
> regionserver.HRegion(3969): creating HRegion testQuarantineMissingHFile HTD
> == 'testQuarantineMissingHFile', {NAME => 'fam', DATA_BLOCK_ENCODING =>
> 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1',
> COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false',
> BLOCKCACHE => 'true'} RootDir =
> hdfs://localhost:50919/user/jenkins/hbase/.tmp Table name ==
> testQuarantineMissingHFile
> 2014-04-12 23:24:55,188 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-4]
> regionserver.HRegion(3969): creating HRegion testQuarantineMissingHFile HTD
> == 'testQuarantineMissingHFile', {NAME => 'fam', DATA_BLOCK_ENCODING =>
> 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1',
> COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647',
> KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false',
> BLOCKCACHE => 'true'} RootDir =
> hdfs://localhost:50919/user/jenkins/hbase/.tmp Table name ==
> testQuarantineMissingHFile
> 2014-04-12 23:24:55,378 INFO [IPC Server handler 2 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741869_1045{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,379 INFO [IPC Server handler 0 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to
> blk_1073741869_1045{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,379 INFO [IPC Server handler 1 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to
> blk_1073741869_1045{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,386 INFO [IPC Server handler 9 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to
> blk_1073741868_1044{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,387 INFO [IPC Server handler 2 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741868_1044{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,388 INFO [IPC Server handler 1 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to blk_1073741868_1044 size 61
> 2014-04-12 23:24:55,410 INFO [IPC Server handler 9 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741871_1047{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,410 INFO [IPC Server handler 2 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to
> blk_1073741871_1047{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,411 INFO [IPC Server handler 4 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to blk_1073741871_1047 size 61
> 2014-04-12 23:24:55,418 INFO [IPC Server handler 8 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741870_1046{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,418 INFO [IPC Server handler 9 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to
> blk_1073741870_1046{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:55,419 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-3]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:55,419 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-3]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.:
> disabling compactions & flushes
> 2014-04-12 23:24:55,419 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-3]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:55,419 INFO [IPC Server handler 4 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to blk_1073741870_1046 size 60
> 2014-04-12 23:24:55,419 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-3]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:55,458 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-1]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:55,458 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-2]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:55,459 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-1]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.:
> disabling compactions & flushes
> 2014-04-12 23:24:55,459 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-2]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.:
> disabling compactions & flushes
> 2014-04-12 23:24:55,459 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-1]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:55,459 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-2]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:55,459 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-1]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:55,459 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-2]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:55,460 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-4]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:55,461 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-4]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.:
> disabling compactions & flushes
> 2014-04-12 23:24:55,461 DEBUG
> [RegionOpenAndInitThread-testQuarantineMissingHFile-4]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:55,461 INFO
> [RegionOpenAndInitThread-testQuarantineMissingHFile-4]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:55,502 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> catalog.MetaEditor(278): Added 4
> 2014-04-12 23:24:55,503 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> master.AssignmentManager(2506): Bulk assigning 4 region(s) across 3
> server(s), round-robin=true
> 2014-04-12 23:24:55,503 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-0]
> master.AssignmentManager(1451): Assigning 1 region(s) to
> p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,503 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-1]
> master.AssignmentManager(1451): Assigning 1 region(s) to
> p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,503 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-2]
> master.AssignmentManager(1451): Assigning 2 region(s) to
> p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,504 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-1]
> zookeeper.ZKAssign(175): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Async create of unassigned node
> a8a68d998d21b00499aca60887ae5aef with OFFLINE state
> 2014-04-12 23:24:55,504 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-2]
> zookeeper.ZKAssign(175): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Async create of unassigned node
> d383980be98665b638fd56bfac97a351 with OFFLINE state
> 2014-04-12 23:24:55,503 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-0]
> zookeeper.ZKAssign(175): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Async create of unassigned node
> 63cdcba1fc55ae6463463ae16f4e454e with OFFLINE state
> 2014-04-12 23:24:55,503 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> master.GeneralBulkAssigner(177): Timeout-on-RIT=152000
> 2014-04-12 23:24:55,504 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-2]
> zookeeper.ZKAssign(175): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Async create of unassigned node
> f9c185520bca999a753ee3ce0a244f6d with OFFLINE state
> 2014-04-12 23:24:55,510 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-04-12 23:24:55,510 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback(69): rs={a8a68d998d21b00499aca60887ae5aef
> state=OFFLINE, ts=1397370295502, server=null},
> server=p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,511 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback(69): rs={d383980be98665b638fd56bfac97a351
> state=OFFLINE, ts=1397370295502, server=null},
> server=p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,518 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback(69): rs={63cdcba1fc55ae6463463ae16f4e454e
> state=OFFLINE, ts=1397370295502, server=null},
> server=p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,519 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback(69): rs={f9c185520bca999a753ee3ce0a244f6d
> state=OFFLINE, ts=1397370295502, server=null},
> server=p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,519 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback$ExistCallback(106):
> rs={a8a68d998d21b00499aca60887ae5aef state=OFFLINE, ts=1397370295502,
> server=null}, server=p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,519 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback$ExistCallback(106):
> rs={d383980be98665b638fd56bfac97a351 state=OFFLINE, ts=1397370295502,
> server=null}, server=p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,519 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-1]
> master.AssignmentManager(1502): p0419.mtv.cloudera.com,39010,1397370264935
> unassigned znodes=1 of total=1
> 2014-04-12 23:24:55,519 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback$ExistCallback(106):
> rs={63cdcba1fc55ae6463463ae16f4e454e state=OFFLINE, ts=1397370295502,
> server=null}, server=p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,519 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-1]
> master.RegionStates(316): Transitioned {a8a68d998d21b00499aca60887ae5aef
> state=OFFLINE, ts=1397370295504, server=null} to
> {a8a68d998d21b00499aca60887ae5aef state=PENDING_OPEN, ts=1397370295519,
> server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:55,519 DEBUG [pool-1-thread-1-EventThread]
> master.OfflineCallback$ExistCallback(106):
> rs={f9c185520bca999a753ee3ce0a244f6d state=OFFLINE, ts=1397370295502,
> server=null}, server=p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,519 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-0]
> master.AssignmentManager(1502): p0419.mtv.cloudera.com,32877,1397370265023
> unassigned znodes=1 of total=1
> 2014-04-12 23:24:55,520 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-2]
> master.AssignmentManager(1502): p0419.mtv.cloudera.com,41017,1397370264982
> unassigned znodes=2 of total=2
> 2014-04-12 23:24:55,520 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-0]
> master.RegionStates(316): Transitioned {63cdcba1fc55ae6463463ae16f4e454e
> state=OFFLINE, ts=1397370295503, server=null} to
> {63cdcba1fc55ae6463463ae16f4e454e state=PENDING_OPEN, ts=1397370295520,
> server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:55,520 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-2]
> master.RegionStates(316): Transitioned {d383980be98665b638fd56bfac97a351
> state=OFFLINE, ts=1397370295504, server=null} to
> {d383980be98665b638fd56bfac97a351 state=PENDING_OPEN, ts=1397370295520,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,520 INFO [Priority.RpcServer.handler=1,port=39010]
> regionserver.HRegionServer(3513): Open
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:55,520 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-2]
> master.RegionStates(316): Transitioned {f9c185520bca999a753ee3ce0a244f6d
> state=OFFLINE, ts=1397370295504, server=null} to
> {f9c185520bca999a753ee3ce0a244f6d state=PENDING_OPEN, ts=1397370295520,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,521 INFO [Priority.RpcServer.handler=1,port=32877]
> regionserver.HRegionServer(3513): Open
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:55,521 INFO [Priority.RpcServer.handler=1,port=41017]
> regionserver.HRegionServer(3513): Open
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:55,525 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> zookeeper.ZKAssign(832): regionserver:39010-0x14559c215740001,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> a8a68d998d21b00499aca60887ae5aef from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,526 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-1]
> master.AssignmentManager(1626): Bulk assigning done for
> p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,526 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> zookeeper.ZKAssign(832): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> d383980be98665b638fd56bfac97a351 from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,526 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-0]
> master.AssignmentManager(1626): Bulk assigning done for
> p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,526 INFO [Priority.RpcServer.handler=1,port=41017]
> regionserver.HRegionServer(3513): Open
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:55,526 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> zookeeper.ZKAssign(832): regionserver:32877-0x14559c215740003,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> 63cdcba1fc55ae6463463ae16f4e454e from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,527 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> zookeeper.ZKAssign(832): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> f9c185520bca999a753ee3ce0a244f6d from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,527 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-GeneralBulkAssigner-2]
> master.AssignmentManager(1626): Bulk assigning done for
> p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,527 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> master.GeneralBulkAssigner(153): bulk assigning total 4 regions to 3 servers,
> took 23ms, with 4 regions still in transition
> 2014-04-12 23:24:55,527 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> master.AssignmentManager(2513): Bulk assigning done
> 2014-04-12 23:24:55,535 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> zookeeper.ZKAssign(907): regionserver:39010-0x14559c215740001,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> a8a68d998d21b00499aca60887ae5aef from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,535 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:55,536 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> regionserver.HRegion(4153): Opening region: {ENCODED =>
> a8a68d998d21b00499aca60887ae5aef, NAME =>
> 'testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.',
> STARTKEY => 'A', ENDKEY => 'B'}
> 2014-04-12 23:24:55,536 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> regionserver.MetricsRegionSourceImpl(64): Creating new
> MetricsRegionSourceImpl for table testQuarantineMissingHFile
> a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:55,536 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:55,577 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:55,577 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> zookeeper.ZKAssign(907): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> d383980be98665b638fd56bfac97a351 from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,578 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> regionserver.HRegion(4153): Opening region: {ENCODED =>
> d383980be98665b638fd56bfac97a351, NAME =>
> 'testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.',
> STARTKEY => 'B', ENDKEY => 'C'}
> 2014-04-12 23:24:55,578 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> regionserver.MetricsRegionSourceImpl(64): Creating new
> MetricsRegionSourceImpl for table testQuarantineMissingHFile
> d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:55,579 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:55,594 INFO
> [StoreOpener-a8a68d998d21b00499aca60887ae5aef-1]
> compactions.CompactionConfiguration(85): size [134217728,
> 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000;
> throttle point 2684354560; delete expired; major period 604800000, major
> jitter 0.500000
> 2014-04-12 23:24:55,598 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> regionserver.HRegion(2822): Found 0 recovered edits file(s) under
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:55,600 INFO [RS_OPEN_REGION-p0419:39010-1]
> regionserver.HRegion(637): Onlined a8a68d998d21b00499aca60887ae5aef; next
> sequenceid=1
> 2014-04-12 23:24:55,600 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> zookeeper.ZKAssign(644): regionserver:39010-0x14559c215740001,
> quorum=localhost:53570, baseZNode=/hbase Attempting to retransition opening
> state of node a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:55,644 INFO
> [StoreOpener-d383980be98665b638fd56bfac97a351-1]
> compactions.CompactionConfiguration(85): size [134217728,
> 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000;
> throttle point 2684354560; delete expired; major period 604800000, major
> jitter 0.500000
> 2014-04-12 23:24:55,648 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> regionserver.HRegion(2822): Found 0 recovered edits file(s) under
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:55,650 INFO [RS_OPEN_REGION-p0419:41017-0]
> regionserver.HRegion(637): Onlined d383980be98665b638fd56bfac97a351; next
> sequenceid=1
> 2014-04-12 23:24:55,650 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> zookeeper.ZKAssign(644): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Attempting to retransition opening
> state of node d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:55,657 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:55,657 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> zookeeper.ZKAssign(907): regionserver:32877-0x14559c215740003,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> 63cdcba1fc55ae6463463ae16f4e454e from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,658 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:55,658 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> zookeeper.ZKAssign(907): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> f9c185520bca999a753ee3ce0a244f6d from M_ZK_REGION_OFFLINE to
> RS_ZK_REGION_OPENING
> 2014-04-12 23:24:55,658 DEBUG [AM.ZK.Worker-pool2-t11]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENING,
> server=p0419.mtv.cloudera.com,39010,1397370264935,
> region=a8a68d998d21b00499aca60887ae5aef,
> current_state={a8a68d998d21b00499aca60887ae5aef state=PENDING_OPEN,
> ts=1397370295519, server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:55,658 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> regionserver.HRegion(4153): Opening region: {ENCODED =>
> 63cdcba1fc55ae6463463ae16f4e454e, NAME =>
> 'testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.',
> STARTKEY => '', ENDKEY => 'A'}
> 2014-04-12 23:24:55,658 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> regionserver.HRegion(4153): Opening region: {ENCODED =>
> f9c185520bca999a753ee3ce0a244f6d, NAME =>
> 'testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.',
> STARTKEY => 'C', ENDKEY => ''}
> 2014-04-12 23:24:55,658 INFO [AM.ZK.Worker-pool2-t11]
> master.RegionStates(316): Transitioned {a8a68d998d21b00499aca60887ae5aef
> state=PENDING_OPEN, ts=1397370295519,
> server=p0419.mtv.cloudera.com,39010,1397370264935} to
> {a8a68d998d21b00499aca60887ae5aef state=OPENING, ts=1397370295658,
> server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:55,659 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> regionserver.MetricsRegionSourceImpl(64): Creating new
> MetricsRegionSourceImpl for table testQuarantineMissingHFile
> 63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:55,659 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> regionserver.MetricsRegionSourceImpl(64): Creating new
> MetricsRegionSourceImpl for table testQuarantineMissingHFile
> f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:55,659 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:55,659 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> regionserver.HRegion(542): Instantiated
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:55,694 DEBUG [AM.ZK.Worker-pool2-t13]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENING,
> server=p0419.mtv.cloudera.com,41017,1397370264982,
> region=d383980be98665b638fd56bfac97a351,
> current_state={d383980be98665b638fd56bfac97a351 state=PENDING_OPEN,
> ts=1397370295520, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,694 INFO [AM.ZK.Worker-pool2-t13]
> master.RegionStates(316): Transitioned {d383980be98665b638fd56bfac97a351
> state=PENDING_OPEN, ts=1397370295520,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {d383980be98665b638fd56bfac97a351 state=OPENING, ts=1397370295694,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,694 DEBUG [AM.ZK.Worker-pool2-t18]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENING,
> server=p0419.mtv.cloudera.com,32877,1397370265023,
> region=63cdcba1fc55ae6463463ae16f4e454e,
> current_state={63cdcba1fc55ae6463463ae16f4e454e state=PENDING_OPEN,
> ts=1397370295520, server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:55,694 DEBUG [AM.ZK.Worker-pool2-t10]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENING,
> server=p0419.mtv.cloudera.com,41017,1397370264982,
> region=f9c185520bca999a753ee3ce0a244f6d,
> current_state={f9c185520bca999a753ee3ce0a244f6d state=PENDING_OPEN,
> ts=1397370295520, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,694 INFO
> [PostOpenDeployTasks:d383980be98665b638fd56bfac97a351]
> regionserver.HRegionServer(1688): Post open deploy tasks for
> region=testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:55,694 INFO [AM.ZK.Worker-pool2-t18]
> master.RegionStates(316): Transitioned {63cdcba1fc55ae6463463ae16f4e454e
> state=PENDING_OPEN, ts=1397370295520,
> server=p0419.mtv.cloudera.com,32877,1397370265023} to
> {63cdcba1fc55ae6463463ae16f4e454e state=OPENING, ts=1397370295694,
> server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:55,694 INFO
> [PostOpenDeployTasks:a8a68d998d21b00499aca60887ae5aef]
> regionserver.HRegionServer(1688): Post open deploy tasks for
> region=testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:55,695 INFO [AM.ZK.Worker-pool2-t10]
> master.RegionStates(316): Transitioned {f9c185520bca999a753ee3ce0a244f6d
> state=PENDING_OPEN, ts=1397370295520,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {f9c185520bca999a753ee3ce0a244f6d state=OPENING, ts=1397370295695,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,718 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> lock.ZKInterProcessLockBase(328): Released
> /hbase/table-lock/testQuarantineMissingHFile/write-master:330010000000000
> 2014-04-12 23:24:55,723 INFO
> [PostOpenDeployTasks:d383980be98665b638fd56bfac97a351]
> catalog.MetaEditor(464): Updated row
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> with server=p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,723 INFO
> [PostOpenDeployTasks:d383980be98665b638fd56bfac97a351]
> regionserver.HRegionServer(1713): Finished post open deploy task for
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:55,726 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> zookeeper.ZKAssign(832): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> d383980be98665b638fd56bfac97a351 from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,726 INFO
> [PostOpenDeployTasks:a8a68d998d21b00499aca60887ae5aef]
> catalog.MetaEditor(464): Updated row
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> with server=p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,726 INFO
> [PostOpenDeployTasks:a8a68d998d21b00499aca60887ae5aef]
> regionserver.HRegionServer(1713): Finished post open deploy task for
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:55,727 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> zookeeper.ZKAssign(832): regionserver:39010-0x14559c215740001,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> a8a68d998d21b00499aca60887ae5aef from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,728 INFO
> [StoreOpener-f9c185520bca999a753ee3ce0a244f6d-1]
> compactions.CompactionConfiguration(85): size [134217728,
> 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000;
> throttle point 2684354560; delete expired; major period 604800000, major
> jitter 0.500000
> 2014-04-12 23:24:55,731 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> regionserver.HRegion(2822): Found 0 recovered edits file(s) under
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:55,732 INFO [RS_OPEN_REGION-p0419:41017-2]
> regionserver.HRegion(637): Onlined f9c185520bca999a753ee3ce0a244f6d; next
> sequenceid=1
> 2014-04-12 23:24:55,733 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> zookeeper.ZKAssign(644): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Attempting to retransition opening
> state of node f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:55,777 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:55,777 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> zookeeper.ZKAssign(907): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> d383980be98665b638fd56bfac97a351 from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,777 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> handler.OpenRegionHandler(389): Transitioned d383980be98665b638fd56bfac97a351
> to OPENED in zk on p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,777 DEBUG [RS_OPEN_REGION-p0419:41017-0]
> handler.OpenRegionHandler(189): Opened
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> on p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,786 INFO
> [StoreOpener-63cdcba1fc55ae6463463ae16f4e454e-1]
> compactions.CompactionConfiguration(85): size [134217728,
> 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000;
> throttle point 2684354560; delete expired; major period 604800000, major
> jitter 0.500000
> 2014-04-12 23:24:55,789 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> regionserver.HRegion(2822): Found 0 recovered edits file(s) under
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:55,792 INFO [RS_OPEN_REGION-p0419:32877-0]
> regionserver.HRegion(637): Onlined 63cdcba1fc55ae6463463ae16f4e454e; next
> sequenceid=1
> 2014-04-12 23:24:55,792 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> zookeeper.ZKAssign(644): regionserver:32877-0x14559c215740003,
> quorum=localhost:53570, baseZNode=/hbase Attempting to retransition opening
> state of node 63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:55,827 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:55,827 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> zookeeper.ZKAssign(907): regionserver:39010-0x14559c215740001,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> a8a68d998d21b00499aca60887ae5aef from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,827 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> handler.OpenRegionHandler(389): Transitioned a8a68d998d21b00499aca60887ae5aef
> to OPENED in zk on p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,827 DEBUG [AM.ZK.Worker-pool2-t15]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENED,
> server=p0419.mtv.cloudera.com,41017,1397370264982,
> region=d383980be98665b638fd56bfac97a351,
> current_state={d383980be98665b638fd56bfac97a351 state=OPENING,
> ts=1397370295694, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,827 DEBUG [RS_OPEN_REGION-p0419:39010-1]
> handler.OpenRegionHandler(189): Opened
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> on p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,827 INFO [AM.ZK.Worker-pool2-t15]
> master.RegionStates(316): Transitioned {d383980be98665b638fd56bfac97a351
> state=OPENING, ts=1397370295694,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {d383980be98665b638fd56bfac97a351 state=OPEN, ts=1397370295827,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,828 INFO
> [PostOpenDeployTasks:f9c185520bca999a753ee3ce0a244f6d]
> regionserver.HRegionServer(1688): Post open deploy tasks for
> region=testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:55,828 DEBUG [AM.ZK.Worker-pool2-t15]
> handler.OpenedRegionHandler(149): Handling OPENED of
> d383980be98665b638fd56bfac97a351 from
> p0419.mtv.cloudera.com,41017,1397370264982; deleting unassigned node
> 2014-04-12 23:24:55,828 INFO
> [PostOpenDeployTasks:63cdcba1fc55ae6463463ae16f4e454e]
> regionserver.HRegionServer(1688): Post open deploy tasks for
> region=testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:55,828 DEBUG [AM.ZK.Worker-pool2-t1]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENED,
> server=p0419.mtv.cloudera.com,39010,1397370264935,
> region=a8a68d998d21b00499aca60887ae5aef,
> current_state={a8a68d998d21b00499aca60887ae5aef state=OPENING,
> ts=1397370295658, server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:55,828 INFO [AM.ZK.Worker-pool2-t1]
> master.RegionStates(316): Transitioned {a8a68d998d21b00499aca60887ae5aef
> state=OPENING, ts=1397370295658,
> server=p0419.mtv.cloudera.com,39010,1397370264935} to
> {a8a68d998d21b00499aca60887ae5aef state=OPEN, ts=1397370295828,
> server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:55,829 DEBUG [AM.ZK.Worker-pool2-t1]
> handler.OpenedRegionHandler(149): Handling OPENED of
> a8a68d998d21b00499aca60887ae5aef from
> p0419.mtv.cloudera.com,39010,1397370264935; deleting unassigned node
> 2014-04-12 23:24:55,844 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:55,844 DEBUG [AM.ZK.Worker-pool2-t15]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> d383980be98665b638fd56bfac97a351 in expected state RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,844 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-04-12 23:24:55,844 DEBUG [AM.ZK.Worker-pool2-t15]
> master.AssignmentManager$4(1173): Znode
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> deleted, state: {d383980be98665b638fd56bfac97a351 state=OPEN,
> ts=1397370295827, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,844 INFO [AM.ZK.Worker-pool2-t15]
> master.RegionStates(377): Onlined d383980be98665b638fd56bfac97a351 on
> p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,848 INFO
> [PostOpenDeployTasks:f9c185520bca999a753ee3ce0a244f6d]
> catalog.MetaEditor(464): Updated row
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> with server=p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,848 INFO
> [PostOpenDeployTasks:63cdcba1fc55ae6463463ae16f4e454e]
> catalog.MetaEditor(464): Updated row
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> with server=p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,849 INFO
> [PostOpenDeployTasks:63cdcba1fc55ae6463463ae16f4e454e]
> regionserver.HRegionServer(1713): Finished post open deploy task for
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:55,849 INFO
> [PostOpenDeployTasks:f9c185520bca999a753ee3ce0a244f6d]
> regionserver.HRegionServer(1713): Finished post open deploy task for
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:55,850 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> zookeeper.ZKAssign(832): regionserver:32877-0x14559c215740003,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> 63cdcba1fc55ae6463463ae16f4e454e from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,850 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> zookeeper.ZKAssign(832): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> f9c185520bca999a753ee3ce0a244f6d from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,869 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:55,869 DEBUG [AM.ZK.Worker-pool2-t1]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> a8a68d998d21b00499aca60887ae5aef in expected state RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,869 DEBUG [AM.ZK.Worker-pool2-t1]
> master.AssignmentManager$4(1173): Znode
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> deleted, state: {a8a68d998d21b00499aca60887ae5aef state=OPEN,
> ts=1397370295828, server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:55,869 INFO [AM.ZK.Worker-pool2-t1]
> master.RegionStates(377): Onlined a8a68d998d21b00499aca60887ae5aef on
> p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:55,910 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> zookeeper.ZKAssign(907): regionserver:32877-0x14559c215740003,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> 63cdcba1fc55ae6463463ae16f4e454e from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,910 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> zookeeper.ZKAssign(907): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> f9c185520bca999a753ee3ce0a244f6d from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,911 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> handler.OpenRegionHandler(389): Transitioned f9c185520bca999a753ee3ce0a244f6d
> to OPENED in zk on p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,910 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:55,911 DEBUG [RS_OPEN_REGION-p0419:41017-2]
> handler.OpenRegionHandler(189): Opened
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> on p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:55,910 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> handler.OpenRegionHandler(389): Transitioned 63cdcba1fc55ae6463463ae16f4e454e
> to OPENED in zk on p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,911 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:55,911 DEBUG [RS_OPEN_REGION-p0419:32877-0]
> handler.OpenRegionHandler(189): Opened
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> on p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,912 DEBUG [AM.ZK.Worker-pool2-t7]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENED,
> server=p0419.mtv.cloudera.com,32877,1397370265023,
> region=63cdcba1fc55ae6463463ae16f4e454e,
> current_state={63cdcba1fc55ae6463463ae16f4e454e state=OPENING,
> ts=1397370295694, server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:55,912 INFO [AM.ZK.Worker-pool2-t7]
> master.RegionStates(316): Transitioned {63cdcba1fc55ae6463463ae16f4e454e
> state=OPENING, ts=1397370295694,
> server=p0419.mtv.cloudera.com,32877,1397370265023} to
> {63cdcba1fc55ae6463463ae16f4e454e state=OPEN, ts=1397370295912,
> server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:55,912 DEBUG [AM.ZK.Worker-pool2-t20]
> master.AssignmentManager(790): Handling RS_ZK_REGION_OPENED,
> server=p0419.mtv.cloudera.com,41017,1397370264982,
> region=f9c185520bca999a753ee3ce0a244f6d,
> current_state={f9c185520bca999a753ee3ce0a244f6d state=OPENING,
> ts=1397370295695, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,912 DEBUG [AM.ZK.Worker-pool2-t7]
> handler.OpenedRegionHandler(149): Handling OPENED of
> 63cdcba1fc55ae6463463ae16f4e454e from
> p0419.mtv.cloudera.com,32877,1397370265023; deleting unassigned node
> 2014-04-12 23:24:55,912 INFO [AM.ZK.Worker-pool2-t20]
> master.RegionStates(316): Transitioned {f9c185520bca999a753ee3ce0a244f6d
> state=OPENING, ts=1397370295695,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {f9c185520bca999a753ee3ce0a244f6d state=OPEN, ts=1397370295912,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,913 DEBUG [AM.ZK.Worker-pool2-t20]
> handler.OpenedRegionHandler(149): Handling OPENED of
> f9c185520bca999a753ee3ce0a244f6d from
> p0419.mtv.cloudera.com,41017,1397370264982; deleting unassigned node
> 2014-04-12 23:24:55,927 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:55,927 DEBUG [AM.ZK.Worker-pool2-t7]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> 63cdcba1fc55ae6463463ae16f4e454e in expected state RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,927 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-04-12 23:24:55,927 DEBUG [AM.ZK.Worker-pool2-t7]
> master.AssignmentManager$4(1173): Znode
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> deleted, state: {63cdcba1fc55ae6463463ae16f4e454e state=OPEN,
> ts=1397370295912, server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:55,927 INFO [AM.ZK.Worker-pool2-t7]
> master.RegionStates(377): Onlined 63cdcba1fc55ae6463463ae16f4e454e on
> p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:55,935 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:55,935 DEBUG [AM.ZK.Worker-pool2-t20]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> f9c185520bca999a753ee3ce0a244f6d in expected state RS_ZK_REGION_OPENED
> 2014-04-12 23:24:55,936 DEBUG [AM.ZK.Worker-pool2-t20]
> master.AssignmentManager$4(1173): Znode
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> deleted, state: {f9c185520bca999a753ee3ce0a244f6d state=OPEN,
> ts=1397370295912, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:55,936 INFO [AM.ZK.Worker-pool2-t20]
> master.RegionStates(377): Onlined f9c185520bca999a753ee3ce0a244f6d on
> p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:56,206 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process
> identifier=catalogtracker-on-hconnection-0x1ae9e1aa connecting to ZooKeeper
> ensemble=localhost:53570
> 2014-04-12 23:24:56,206 DEBUG [Thread-587] catalog.CatalogTracker(193):
> Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@55a56f72
> 2014-04-12 23:24:56,219 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x1ae9e1aa,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=SyncConnected, path=null
> 2014-04-12 23:24:56,219 DEBUG [Thread-587] zookeeper.ZKUtil(428):
> catalogtracker-on-hconnection-0x1ae9e1aa, quorum=localhost:53570,
> baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
> 2014-04-12 23:24:56,220 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387):
> catalogtracker-on-hconnection-0x1ae9e1aa-0x14559c215740041 connected
> 2014-04-12 23:24:56,225 DEBUG [Thread-587] catalog.CatalogTracker(209):
> Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@55a56f72
> 2014-04-12 23:24:56,263 DEBUG [Thread-587] client.ClientScanner(257):
> Advancing internal scanner to startKey at 'A'
> 2014-04-12 23:24:56,267 DEBUG [Thread-587] client.ClientScanner(257):
> Advancing internal scanner to startKey at 'B'
> 2014-04-12 23:24:56,270 DEBUG [Thread-587] client.ClientScanner(257):
> Advancing internal scanner to startKey at 'C'
> 2014-04-12 23:24:56,273 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process
> identifier=catalogtracker-on-hconnection-0x1ae9e1aa connecting to ZooKeeper
> ensemble=localhost:53570
> 2014-04-12 23:24:56,274 DEBUG [Thread-587] catalog.CatalogTracker(193):
> Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@7c28bed5
> 2014-04-12 23:24:56,294 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x1ae9e1aa,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=SyncConnected, path=null
> 2014-04-12 23:24:56,294 DEBUG [Thread-587] zookeeper.ZKUtil(428):
> catalogtracker-on-hconnection-0x1ae9e1aa, quorum=localhost:53570,
> baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
> 2014-04-12 23:24:56,295 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387):
> catalogtracker-on-hconnection-0x1ae9e1aa-0x14559c215740042 connected
> 2014-04-12 23:24:56,320 INFO [Priority.RpcServer.handler=0,port=32877]
> regionserver.HRegionServer(3661): Flushing
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:56,322 DEBUG [Priority.RpcServer.handler=0,port=32877]
> regionserver.HRegion(1513): Started memstore flush for
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.,
> current region memstore size 320
> 2014-04-12 23:24:56,460 INFO [IPC Server handler 1 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741872_1048{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,461 INFO [IPC Server handler 0 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to
> blk_1073741872_1048{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,462 INFO [IPC Server handler 7 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to blk_1073741872_1048 size 1042
> 2014-04-12 23:24:56,502 INFO [Priority.RpcServer.handler=0,port=32877]
> regionserver.DefaultStoreFlusher(88): Flushed, sequenceid=15, memsize=320,
> hasBloomFilter=true, into tmp file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/.tmp/57a07eaac97e4acd8dc04e08d1950adc
> 2014-04-12 23:24:56,514 DEBUG [Priority.RpcServer.handler=0,port=32877]
> regionserver.HRegionFileSystem(338): Committing store file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/.tmp/57a07eaac97e4acd8dc04e08d1950adc
> as
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc
> 2014-04-12 23:24:56,584 INFO [Priority.RpcServer.handler=0,port=32877]
> regionserver.HStore(764): Added
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc,
> entries=2, sequenceid=15, filesize=1.0 K
> 2014-04-12 23:24:56,584 INFO [Priority.RpcServer.handler=0,port=32877]
> regionserver.HRegion(1660): Finished memstore flush of ~320/320,
> currentsize=0/0 for region
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> in 262ms, sequenceid=15, compaction requested=false
> 2014-04-12 23:24:56,586 INFO [Priority.RpcServer.handler=0,port=39010]
> regionserver.HRegionServer(3661): Flushing
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:56,589 DEBUG [Priority.RpcServer.handler=0,port=39010]
> regionserver.HRegion(1513): Started memstore flush for
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.,
> current region memstore size 320
> 2014-04-12 23:24:56,668 INFO [IPC Server handler 7 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741873_1049{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,669 INFO [IPC Server handler 2 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to
> blk_1073741873_1049{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,669 INFO [IPC Server handler 8 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to
> blk_1073741873_1049{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,677 INFO [Priority.RpcServer.handler=0,port=39010]
> regionserver.DefaultStoreFlusher(88): Flushed, sequenceid=10, memsize=320,
> hasBloomFilter=true, into tmp file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef/.tmp/0ce0a00435f64a02856bfe0fb779093b
> 2014-04-12 23:24:56,688 DEBUG [Priority.RpcServer.handler=0,port=39010]
> regionserver.HRegionFileSystem(338): Committing store file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef/.tmp/0ce0a00435f64a02856bfe0fb779093b
> as
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef/fam/0ce0a00435f64a02856bfe0fb779093b
> 2014-04-12 23:24:56,701 INFO [Priority.RpcServer.handler=0,port=39010]
> regionserver.HStore(764): Added
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef/fam/0ce0a00435f64a02856bfe0fb779093b,
> entries=2, sequenceid=10, filesize=1.0 K
> 2014-04-12 23:24:56,701 INFO [Priority.RpcServer.handler=0,port=39010]
> regionserver.HRegion(1660): Finished memstore flush of ~320/320,
> currentsize=0/0 for region
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> in 112ms, sequenceid=10, compaction requested=false
> 2014-04-12 23:24:56,702 INFO [Priority.RpcServer.handler=1,port=41017]
> regionserver.HRegionServer(3661): Flushing
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:56,702 DEBUG [Priority.RpcServer.handler=1,port=41017]
> regionserver.HRegion(1513): Started memstore flush for
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.,
> current region memstore size 320
> 2014-04-12 23:24:56,734 INFO [IPC Server handler 8 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to
> blk_1073741874_1050{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,734 INFO [IPC Server handler 5 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741874_1050{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,735 INFO [IPC Server handler 9 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to
> blk_1073741874_1050{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-45b6fa92-7966-4203-8e5a-f8115f53e1ae:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-34a4a47f-8a95-461c-8633-2a7fd583b8d9:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-80866ac4-36df-41f3-a856-da7636fa89a3:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:56,744 INFO [Priority.RpcServer.handler=1,port=41017]
> regionserver.DefaultStoreFlusher(88): Flushed, sequenceid=14, memsize=320,
> hasBloomFilter=true, into tmp file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/.tmp/9dd79f30f29e4cfeaa46f6e20b32e078
> 2014-04-12 23:24:56,754 DEBUG [Priority.RpcServer.handler=1,port=41017]
> regionserver.HRegionFileSystem(338): Committing store file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/.tmp/9dd79f30f29e4cfeaa46f6e20b32e078
> as
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/fam/9dd79f30f29e4cfeaa46f6e20b32e078
> 2014-04-12 23:24:56,786 INFO [Priority.RpcServer.handler=1,port=41017]
> regionserver.HStore(764): Added
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/fam/9dd79f30f29e4cfeaa46f6e20b32e078,
> entries=2, sequenceid=14, filesize=1.0 K
> 2014-04-12 23:24:56,786 INFO [Priority.RpcServer.handler=1,port=41017]
> regionserver.HRegion(1660): Finished memstore flush of ~320/320,
> currentsize=0/0 for region
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> in 84ms, sequenceid=14, compaction requested=false
> 2014-04-12 23:24:56,787 INFO [Priority.RpcServer.handler=0,port=41017]
> regionserver.HRegionServer(3661): Flushing
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:56,787 DEBUG [Priority.RpcServer.handler=0,port=41017]
> regionserver.HRegion(1513): Started memstore flush for
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.,
> current region memstore size 320
> 2014-04-12 23:24:57,027 INFO [IPC Server handler 9 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:53997 is added to
> blk_1073741875_1051{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[[DISK]DS-0cf6df80-0db5-43f5-bbc4-a7092d736c08:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-a192707c-27de-4afa-9948-d58a0f0df6bf:NORMAL|RBW],
>
> ReplicaUnderConstruction[[DISK]DS-b2725947-b15b-4c3a-be84-aedc2a57e25f:NORMAL|RBW]]}
> size 0
> 2014-04-12 23:24:57,028 INFO [IPC Server handler 6 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:49422 is added to blk_1073741875_1051 size 1042
> 2014-04-12 23:24:57,029 INFO [IPC Server handler 4 on 50919]
> blockmanagement.BlockManager(2339): BLOCK* addStoredBlock: blockMap updated:
> 127.0.0.1:33353 is added to blk_1073741875_1051 size 1042
> 2014-04-12 23:24:57,077 INFO [Priority.RpcServer.handler=0,port=41017]
> regionserver.DefaultStoreFlusher(88): Flushed, sequenceid=15, memsize=320,
> hasBloomFilter=true, into tmp file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d/.tmp/404e9e4c06b54305a0831dc4749d9a5f
> 2014-04-12 23:24:57,089 DEBUG [Priority.RpcServer.handler=0,port=41017]
> regionserver.HRegionFileSystem(338): Committing store file
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d/.tmp/404e9e4c06b54305a0831dc4749d9a5f
> as
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d/fam/404e9e4c06b54305a0831dc4749d9a5f
> 2014-04-12 23:24:57,153 INFO [Priority.RpcServer.handler=0,port=41017]
> regionserver.HStore(764): Added
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d/fam/404e9e4c06b54305a0831dc4749d9a5f,
> entries=2, sequenceid=15, filesize=1.0 K
> 2014-04-12 23:24:57,154 INFO [Priority.RpcServer.handler=0,port=41017]
> regionserver.HRegion(1660): Finished memstore flush of ~320/320,
> currentsize=0/0 for region
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> in 367ms, sequenceid=15, compaction requested=false
> 2014-04-12 23:24:57,154 DEBUG [Thread-587] catalog.CatalogTracker(209):
> Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@7c28bed5
> 2014-04-12 23:24:57,176 INFO [Thread-587] client.HBaseAdmin$5(909): Started
> disable of testQuarantineMissingHFile
> 2014-04-12 23:24:57,178 INFO [RpcServer.handler=0,port=33001]
> master.HMaster(1980): Client=jenkins//172.29.122.11 disable
> testQuarantineMissingHFile
> 2014-04-12 23:24:57,219 DEBUG [RpcServer.handler=0,port=33001]
> lock.ZKInterProcessLockBase(226): Acquired a lock for
> /hbase/table-lock/testQuarantineMissingHFile/write-master:330010000000001
> 2014-04-12 23:24:57,252 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DisableTableHandler(130): Attempting to disable table
> testQuarantineMissingHFile
> 2014-04-12 23:24:57,253 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process
> identifier=catalogtracker-on-hconnection-0x1ae9e1aa connecting to ZooKeeper
> ensemble=localhost:53570
> 2014-04-12 23:24:57,254 DEBUG [Thread-587] catalog.CatalogTracker(193):
> Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@17dc9229
> 2014-04-12 23:24:57,277 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DisableTableHandler(174): Offlining 4 regions.
> 2014-04-12 23:24:57,278 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0]
> master.AssignmentManager(2232): Starting unassign of
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> (offlining), current state: {63cdcba1fc55ae6463463ae16f4e454e state=OPEN,
> ts=1397370295927, server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:57,278 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-2]
> master.AssignmentManager(2232): Starting unassign of
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> (offlining), current state: {d383980be98665b638fd56bfac97a351 state=OPEN,
> ts=1397370295844, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,278 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-2]
> zookeeper.ZKAssign(527): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Creating unassigned node
> d383980be98665b638fd56bfac97a351 in a CLOSING state
> 2014-04-12 23:24:57,278 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1]
> master.AssignmentManager(2232): Starting unassign of
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> (offlining), current state: {a8a68d998d21b00499aca60887ae5aef state=OPEN,
> ts=1397370295869, server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:57,278 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1]
> zookeeper.ZKAssign(527): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Creating unassigned node
> a8a68d998d21b00499aca60887ae5aef in a CLOSING state
> 2014-04-12 23:24:57,278 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-3]
> master.AssignmentManager(2232): Starting unassign of
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> (offlining), current state: {f9c185520bca999a753ee3ce0a244f6d state=OPEN,
> ts=1397370295936, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,278 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0]
> zookeeper.ZKAssign(527): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Creating unassigned node
> 63cdcba1fc55ae6463463ae16f4e454e in a CLOSING state
> 2014-04-12 23:24:57,279 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-3]
> zookeeper.ZKAssign(527): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Creating unassigned node
> f9c185520bca999a753ee3ce0a244f6d in a CLOSING state
> 2014-04-12 23:24:57,311 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x1ae9e1aa,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=SyncConnected, path=null
> 2014-04-12 23:24:57,312 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387):
> catalogtracker-on-hconnection-0x1ae9e1aa-0x14559c215740043 connected
> 2014-04-12 23:24:57,319 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-04-12 23:24:57,319 DEBUG [Thread-587] zookeeper.ZKUtil(428):
> catalogtracker-on-hconnection-0x1ae9e1aa-0x14559c215740043,
> quorum=localhost:53570, baseZNode=/hbase Set watcher on existing
> znode=/hbase/meta-region-server
> 2014-04-12 23:24:57,320 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-2]
> master.RegionStates(316): Transitioned {d383980be98665b638fd56bfac97a351
> state=OPEN, ts=1397370295844,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {d383980be98665b638fd56bfac97a351 state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,320 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0]
> master.RegionStates(316): Transitioned {63cdcba1fc55ae6463463ae16f4e454e
> state=OPEN, ts=1397370295927,
> server=p0419.mtv.cloudera.com,32877,1397370265023} to
> {63cdcba1fc55ae6463463ae16f4e454e state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:57,320 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1]
> master.RegionStates(316): Transitioned {a8a68d998d21b00499aca60887ae5aef
> state=OPEN, ts=1397370295869,
> server=p0419.mtv.cloudera.com,39010,1397370264935} to
> {a8a68d998d21b00499aca60887ae5aef state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:57,320 INFO
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-3]
> master.RegionStates(316): Transitioned {f9c185520bca999a753ee3ce0a244f6d
> state=OPEN, ts=1397370295936,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {f9c185520bca999a753ee3ce0a244f6d state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,320 INFO [Priority.RpcServer.handler=0,port=41017]
> regionserver.HRegionServer(3635): Close d383980be98665b638fd56bfac97a351, via
> zk=yes, znode version=0, on null
> 2014-04-12 23:24:57,320 INFO [Priority.RpcServer.handler=1,port=32877]
> regionserver.HRegionServer(3635): Close 63cdcba1fc55ae6463463ae16f4e454e, via
> zk=yes, znode version=0, on null
> 2014-04-12 23:24:57,321 INFO [Priority.RpcServer.handler=1,port=39010]
> regionserver.HRegionServer(3635): Close a8a68d998d21b00499aca60887ae5aef, via
> zk=yes, znode version=0, on null
> 2014-04-12 23:24:57,321 INFO [Priority.RpcServer.handler=1,port=41017]
> regionserver.HRegionServer(3635): Close f9c185520bca999a753ee3ce0a244f6d, via
> zk=yes, znode version=0, on null
> 2014-04-12 23:24:57,321 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-2]
> master.AssignmentManager(1674): Sent CLOSE to
> p0419.mtv.cloudera.com,41017,1397370264982 for region
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:57,321 DEBUG [RS_CLOSE_REGION-p0419:41017-1]
> handler.CloseRegionHandler(125): Processing close of
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:57,321 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1]
> master.AssignmentManager(1674): Sent CLOSE to
> p0419.mtv.cloudera.com,39010,1397370264935 for region
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:57,321 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0]
> master.AssignmentManager(1674): Sent CLOSE to
> p0419.mtv.cloudera.com,32877,1397370265023 for region
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:57,321 DEBUG [RS_CLOSE_REGION-p0419:39010-1]
> handler.CloseRegionHandler(125): Processing close of
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:57,321 DEBUG [RS_CLOSE_REGION-p0419:41017-2]
> handler.CloseRegionHandler(125): Processing close of
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:57,321 DEBUG [RS_CLOSE_REGION-p0419:32877-1]
> handler.CloseRegionHandler(125): Processing close of
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:57,321 DEBUG
> [p0419.mtv.cloudera.com,33001,1397370264247-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-3]
> master.AssignmentManager(1674): Sent CLOSE to
> p0419.mtv.cloudera.com,41017,1397370264982 for region
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:57,322 DEBUG [RS_CLOSE_REGION-p0419:41017-1]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.:
> disabling compactions & flushes
> 2014-04-12 23:24:57,323 DEBUG [RS_CLOSE_REGION-p0419:41017-1]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:57,323 DEBUG [RS_CLOSE_REGION-p0419:39010-1]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.:
> disabling compactions & flushes
> 2014-04-12 23:24:57,323 DEBUG [RS_CLOSE_REGION-p0419:39010-1]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:57,323 DEBUG [RS_CLOSE_REGION-p0419:41017-2]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.:
> disabling compactions & flushes
> 2014-04-12 23:24:57,323 DEBUG [RS_CLOSE_REGION-p0419:32877-1]
> regionserver.HRegion(988): Closing
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.:
> disabling compactions & flushes
> 2014-04-12 23:24:57,323 DEBUG [RS_CLOSE_REGION-p0419:41017-2]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:57,323 DEBUG [RS_CLOSE_REGION-p0419:32877-1]
> regionserver.HRegion(1010): Updates disabled for region
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:57,327 INFO
> [StoreCloserThread-testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.-1]
> regionserver.HStore(661): Closed fam
> 2014-04-12 23:24:57,327 INFO
> [StoreCloserThread-testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.-1]
> regionserver.HStore(661): Closed fam
> 2014-04-12 23:24:57,328 INFO [RS_CLOSE_REGION-p0419:39010-1]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:57,328 DEBUG [RS_CLOSE_REGION-p0419:39010-1]
> zookeeper.ZKAssign(832): regionserver:39010-0x14559c215740001,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> a8a68d998d21b00499aca60887ae5aef from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,328 INFO [RS_CLOSE_REGION-p0419:41017-1]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:57,330 DEBUG [RS_CLOSE_REGION-p0419:41017-1]
> zookeeper.ZKAssign(832): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> f9c185520bca999a753ee3ce0a244f6d from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,330 INFO
> [StoreCloserThread-testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.-1]
> regionserver.HStore(661): Closed fam
> 2014-04-12 23:24:57,330 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DisableTableHandler$BulkDisabler(230): Disable waiting until done;
> 300000 ms remaining; [{ENCODED => 63cdcba1fc55ae6463463ae16f4e454e, NAME =>
> 'testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.',
> STARTKEY => '', ENDKEY => 'A'}, {ENCODED =>
> a8a68d998d21b00499aca60887ae5aef, NAME =>
> 'testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.',
> STARTKEY => 'A', ENDKEY => 'B'}, {ENCODED =>
> d383980be98665b638fd56bfac97a351, NAME =>
> 'testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.',
> STARTKEY => 'B', ENDKEY => 'C'}, {ENCODED =>
> f9c185520bca999a753ee3ce0a244f6d, NAME =>
> 'testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.',
> STARTKEY => 'C', ENDKEY => ''}]
> 2014-04-12 23:24:57,330 INFO [RS_CLOSE_REGION-p0419:32877-1]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:57,330 INFO
> [StoreCloserThread-testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.-1]
> regionserver.HStore(661): Closed fam
> 2014-04-12 23:24:57,331 DEBUG [RS_CLOSE_REGION-p0419:32877-1]
> zookeeper.ZKAssign(832): regionserver:32877-0x14559c215740003,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> 63cdcba1fc55ae6463463ae16f4e454e from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,331 DEBUG [Thread-587] catalog.CatalogTracker(209):
> Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@17dc9229
> 2014-04-12 23:24:57,331 INFO [RS_CLOSE_REGION-p0419:41017-2]
> regionserver.HRegion(1068): Closed
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:57,331 DEBUG [RS_CLOSE_REGION-p0419:41017-2]
> zookeeper.ZKAssign(832): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioning
> d383980be98665b638fd56bfac97a351 from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,381 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DisableTableHandler$BulkDisabler(230): Disable waiting until done;
> 299947 ms remaining; [{ENCODED => 63cdcba1fc55ae6463463ae16f4e454e, NAME =>
> 'testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.',
> STARTKEY => '', ENDKEY => 'A'}, {ENCODED =>
> a8a68d998d21b00499aca60887ae5aef, NAME =>
> 'testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.',
> STARTKEY => 'A', ENDKEY => 'B'}, {ENCODED =>
> d383980be98665b638fd56bfac97a351, NAME =>
> 'testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.',
> STARTKEY => 'B', ENDKEY => 'C'}, {ENCODED =>
> f9c185520bca999a753ee3ce0a244f6d, NAME =>
> 'testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.',
> STARTKEY => 'C', ENDKEY => ''}]
> 2014-04-12 23:24:57,385 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:57,385 DEBUG [RS_CLOSE_REGION-p0419:39010-1]
> zookeeper.ZKAssign(907): regionserver:39010-0x14559c215740001,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> a8a68d998d21b00499aca60887ae5aef from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,386 DEBUG [RS_CLOSE_REGION-p0419:39010-1]
> handler.CloseRegionHandler(168): Set closed state in zk for
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> on p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:57,386 DEBUG [RS_CLOSE_REGION-p0419:39010-1]
> handler.CloseRegionHandler(176): Closed
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:57,402 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:57,402 DEBUG [RS_CLOSE_REGION-p0419:41017-1]
> zookeeper.ZKAssign(907): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> f9c185520bca999a753ee3ce0a244f6d from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,402 DEBUG [RS_CLOSE_REGION-p0419:41017-1]
> handler.CloseRegionHandler(168): Set closed state in zk for
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> on p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:57,402 DEBUG [RS_CLOSE_REGION-p0419:41017-1]
> handler.CloseRegionHandler(176): Closed
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:57,402 DEBUG [AM.ZK.Worker-pool2-t2]
> master.AssignmentManager(790): Handling RS_ZK_REGION_CLOSED,
> server=p0419.mtv.cloudera.com,39010,1397370264935,
> region=a8a68d998d21b00499aca60887ae5aef,
> current_state={a8a68d998d21b00499aca60887ae5aef state=PENDING_CLOSE,
> ts=1397370297320, server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:57,403 DEBUG [AM.ZK.Worker-pool2-t2]
> handler.ClosedRegionHandler(92): Handling CLOSED event for
> a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:57,403 DEBUG [Thread-587] client.HBaseAdmin(948): Sleeping=
> 100ms, waiting for all regions to be disabled in testQuarantineMissingHFile
> 2014-04-12 23:24:57,403 DEBUG [AM.ZK.Worker-pool2-t2]
> master.AssignmentManager(1374): Table being disabled so deleting ZK node and
> removing from regions in transition, skipping assignment of region
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> 2014-04-12 23:24:57,410 DEBUG [RS_CLOSE_REGION-p0419:32877-1]
> zookeeper.ZKAssign(907): regionserver:32877-0x14559c215740003,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> 63cdcba1fc55ae6463463ae16f4e454e from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,410 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:57,410 DEBUG [AM.ZK.Worker-pool2-t4]
> master.AssignmentManager(790): Handling RS_ZK_REGION_CLOSED,
> server=p0419.mtv.cloudera.com,41017,1397370264982,
> region=f9c185520bca999a753ee3ce0a244f6d,
> current_state={f9c185520bca999a753ee3ce0a244f6d state=PENDING_CLOSE,
> ts=1397370297320, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,410 DEBUG [RS_CLOSE_REGION-p0419:32877-1]
> handler.CloseRegionHandler(168): Set closed state in zk for
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> on p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:57,411 DEBUG [AM.ZK.Worker-pool2-t4]
> handler.ClosedRegionHandler(92): Handling CLOSED event for
> f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:57,411 DEBUG [RS_CLOSE_REGION-p0419:32877-1]
> handler.CloseRegionHandler(176): Closed
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:57,411 DEBUG [AM.ZK.Worker-pool2-t4]
> master.AssignmentManager(1374): Table being disabled so deleting ZK node and
> removing from regions in transition, skipping assignment of region
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> 2014-04-12 23:24:57,418 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected,
> path=/hbase/region-in-transition/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:57,418 DEBUG [RS_CLOSE_REGION-p0419:41017-2]
> zookeeper.ZKAssign(907): regionserver:41017-0x14559c215740002,
> quorum=localhost:53570, baseZNode=/hbase Transitioned node
> d383980be98665b638fd56bfac97a351 from M_ZK_REGION_CLOSING to
> RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,419 DEBUG [AM.ZK.Worker-pool2-t12]
> master.AssignmentManager(790): Handling RS_ZK_REGION_CLOSED,
> server=p0419.mtv.cloudera.com,32877,1397370265023,
> region=63cdcba1fc55ae6463463ae16f4e454e,
> current_state={63cdcba1fc55ae6463463ae16f4e454e state=PENDING_CLOSE,
> ts=1397370297320, server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:57,419 DEBUG [RS_CLOSE_REGION-p0419:41017-2]
> handler.CloseRegionHandler(168): Set closed state in zk for
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> on p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:57,419 DEBUG [AM.ZK.Worker-pool2-t12]
> handler.ClosedRegionHandler(92): Handling CLOSED event for
> 63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:57,419 DEBUG [RS_CLOSE_REGION-p0419:41017-2]
> handler.CloseRegionHandler(176): Closed
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:57,419 DEBUG [AM.ZK.Worker-pool2-t12]
> master.AssignmentManager(1374): Table being disabled so deleting ZK node and
> removing from regions in transition, skipping assignment of region
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> 2014-04-12 23:24:57,427 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:57,427 DEBUG [AM.ZK.Worker-pool2-t2]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> a8a68d998d21b00499aca60887ae5aef in expected state RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,427 DEBUG [AM.ZK.Worker-pool2-t14]
> master.AssignmentManager(790): Handling RS_ZK_REGION_CLOSED,
> server=p0419.mtv.cloudera.com,41017,1397370264982,
> region=d383980be98665b638fd56bfac97a351,
> current_state={d383980be98665b638fd56bfac97a351 state=PENDING_CLOSE,
> ts=1397370297320, server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,427 INFO [AM.ZK.Worker-pool2-t2]
> master.RegionStates(316): Transitioned {a8a68d998d21b00499aca60887ae5aef
> state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,39010,1397370264935} to
> {a8a68d998d21b00499aca60887ae5aef state=OFFLINE, ts=1397370297427,
> server=p0419.mtv.cloudera.com,39010,1397370264935}
> 2014-04-12 23:24:57,427 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-04-12 23:24:57,427 INFO [AM.ZK.Worker-pool2-t2]
> master.RegionStates(457): Offlined a8a68d998d21b00499aca60887ae5aef from
> p0419.mtv.cloudera.com,39010,1397370264935
> 2014-04-12 23:24:57,428 DEBUG [AM.ZK.Worker-pool2-t14]
> handler.ClosedRegionHandler(92): Handling CLOSED event for
> d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:57,428 DEBUG [AM.ZK.Worker-pool2-t14]
> master.AssignmentManager(1374): Table being disabled so deleting ZK node and
> removing from regions in transition, skipping assignment of region
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> 2014-04-12 23:24:57,431 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DisableTableHandler$BulkDisabler(230): Disable waiting until done;
> 299897 ms remaining; [{ENCODED => 63cdcba1fc55ae6463463ae16f4e454e, NAME =>
> 'testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.',
> STARTKEY => '', ENDKEY => 'A'}, {ENCODED =>
> d383980be98665b638fd56bfac97a351, NAME =>
> 'testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.',
> STARTKEY => 'B', ENDKEY => 'C'}, {ENCODED =>
> f9c185520bca999a753ee3ce0a244f6d, NAME =>
> 'testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.',
> STARTKEY => 'C', ENDKEY => ''}]
> 2014-04-12 23:24:57,435 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:57,435 DEBUG [AM.ZK.Worker-pool2-t4]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> f9c185520bca999a753ee3ce0a244f6d in expected state RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,435 INFO [AM.ZK.Worker-pool2-t4]
> master.RegionStates(316): Transitioned {f9c185520bca999a753ee3ce0a244f6d
> state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {f9c185520bca999a753ee3ce0a244f6d state=OFFLINE, ts=1397370297435,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,436 INFO [AM.ZK.Worker-pool2-t4]
> master.RegionStates(457): Offlined f9c185520bca999a753ee3ce0a244f6d from
> p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:57,444 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:57,444 DEBUG [AM.ZK.Worker-pool2-t12]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> 63cdcba1fc55ae6463463ae16f4e454e in expected state RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,444 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-04-12 23:24:57,444 INFO [AM.ZK.Worker-pool2-t12]
> master.RegionStates(316): Transitioned {63cdcba1fc55ae6463463ae16f4e454e
> state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,32877,1397370265023} to
> {63cdcba1fc55ae6463463ae16f4e454e state=OFFLINE, ts=1397370297444,
> server=p0419.mtv.cloudera.com,32877,1397370265023}
> 2014-04-12 23:24:57,444 INFO [AM.ZK.Worker-pool2-t12]
> master.RegionStates(457): Offlined 63cdcba1fc55ae6463463ae16f4e454e from
> p0419.mtv.cloudera.com,32877,1397370265023
> 2014-04-12 23:24:57,452 DEBUG [AM.ZK.Worker-pool2-t14]
> zookeeper.ZKAssign(480): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Deleted unassigned node
> d383980be98665b638fd56bfac97a351 in expected state RS_ZK_REGION_CLOSED
> 2014-04-12 23:24:57,452 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:57,452 INFO [AM.ZK.Worker-pool2-t14]
> master.RegionStates(316): Transitioned {d383980be98665b638fd56bfac97a351
> state=PENDING_CLOSE, ts=1397370297320,
> server=p0419.mtv.cloudera.com,41017,1397370264982} to
> {d383980be98665b638fd56bfac97a351 state=OFFLINE, ts=1397370297452,
> server=p0419.mtv.cloudera.com,41017,1397370264982}
> 2014-04-12 23:24:57,452 INFO [AM.ZK.Worker-pool2-t14]
> master.RegionStates(457): Offlined d383980be98665b638fd56bfac97a351 from
> p0419.mtv.cloudera.com,41017,1397370264982
> 2014-04-12 23:24:57,482 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DisableTableHandler$BulkDisabler(230): Disable waiting until done;
> 299846 ms remaining; []
> 2014-04-12 23:24:57,493 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DisableTableHandler(190): Disabled table is done=true
> 2014-04-12 23:24:57,502 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> lock.ZKInterProcessLockBase(328): Released
> /hbase/table-lock/testQuarantineMissingHFile/write-master:330010000000001
> 2014-04-12 23:24:57,504 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process
> identifier=catalogtracker-on-hconnection-0x1ae9e1aa connecting to ZooKeeper
> ensemble=localhost:53570
> 2014-04-12 23:24:57,505 DEBUG [Thread-587] catalog.CatalogTracker(193):
> Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@6c7f76bb
> 2014-04-12 23:24:57,519 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x1ae9e1aa,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=SyncConnected, path=null
> 2014-04-12 23:24:57,519 DEBUG [Thread-587] zookeeper.ZKUtil(428):
> catalogtracker-on-hconnection-0x1ae9e1aa, quorum=localhost:53570,
> baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
> 2014-04-12 23:24:57,520 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387):
> catalogtracker-on-hconnection-0x1ae9e1aa-0x14559c215740044 connected
> 2014-04-12 23:24:57,525 DEBUG [Thread-587] catalog.CatalogTracker(209):
> Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@6c7f76bb
> 2014-04-12 23:24:57,536 INFO [Thread-587] client.HBaseAdmin(964): Disabled
> testQuarantineMissingHFile
> 2014-04-12 23:24:57,540 INFO [Thread-587] util.HBaseFsck(3774): Checking all
> hfiles for corruption
> 2014-04-12 23:24:57,602 INFO [IPC Server handler 7 on 50919]
> blockmanagement.BlockManager(1053): BLOCK* addToInvalidates:
> blk_1073741872_1048 127.0.0.1:49422 127.0.0.1:53997 127.0.0.1:33353
> 2014-04-12 23:24:57,638 WARN [IPC Server handler 4 on 50919]
> security.UserGroupInformation(1551): PriviledgedActionException as:jenkins
> (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist:
> /user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc
> 2014-04-12 23:24:57,640 WARN [pool-115-thread-1]
> hbck.HFileCorruptionChecker(111): HFile
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam/57a07eaac97e4acd8dc04e08d1950adc
> was missing. Likely removed due to compaction/split?
> 2014-04-12 23:24:57,652 INFO [IPC Server handler 2 on 50919]
> blockmanagement.BlockManager(1053): BLOCK* addToInvalidates:
> blk_1073741874_1050 127.0.0.1:33353 127.0.0.1:53997 127.0.0.1:49422
> 2014-04-12 23:24:57,679 WARN [IPC Server handler 3 on 50919]
> security.UserGroupInformation(1551): PriviledgedActionException as:jenkins
> (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist:
> /user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/fam/9dd79f30f29e4cfeaa46f6e20b32e078
> 2014-04-12 23:24:57,680 WARN [pool-115-thread-3]
> hbck.HFileCorruptionChecker(111): HFile
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/fam/9dd79f30f29e4cfeaa46f6e20b32e078
> was missing. Likely removed due to compaction/split?
> 2014-04-12 23:24:57,682 INFO [Thread-587] util.HBaseFsck(375): Loading
> regioninfos HDFS
> 2014-04-12 23:24:57,682 INFO [Thread-587] util.HBaseFsck(608): Loading HBase
> regioninfo from HDFS...
> 2014-04-12 23:24:57,691 DEBUG [Thread-587] util.HBaseFsck(1334): Loading
> region dirs from
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile
> 2014-04-12 23:24:57,691 DEBUG [Thread-587] util.HBaseFsck(1334): Loading
> region dirs from hdfs://localhost:50919/user/jenkins/hbase/data/hbase/meta
> 2014-04-12 23:24:57,693 DEBUG [pool-115-thread-2]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:57,693 DEBUG [pool-115-thread-1]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/hbase/meta/1588230740
> 2014-04-12 23:24:57,696 DEBUG [pool-115-thread-2]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:57,699 DEBUG [pool-115-thread-2]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:57,701 DEBUG [pool-115-thread-2]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:57,714 DEBUG [pool-115-thread-4] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY
> => '', ENDKEY => ''}
> 2014-04-12 23:24:57,721 DEBUG [pool-115-thread-3] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => 63cdcba1fc55ae6463463ae16f4e454e, NAME =>
> 'testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.',
> STARTKEY => '', ENDKEY => 'A'}
> 2014-04-12 23:24:57,755 DEBUG [pool-115-thread-6] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => d383980be98665b638fd56bfac97a351, NAME =>
> 'testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.',
> STARTKEY => 'B', ENDKEY => 'C'}
> 2014-04-12 23:24:57,755 DEBUG [pool-115-thread-1] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => f9c185520bca999a753ee3ce0a244f6d, NAME =>
> 'testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.',
> STARTKEY => 'C', ENDKEY => ''}
> 2014-04-12 23:24:57,756 DEBUG [pool-115-thread-5] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => a8a68d998d21b00499aca60887ae5aef, NAME =>
> 'testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.',
> STARTKEY => 'A', ENDKEY => 'B'}
> 2014-04-12 23:24:57,782 INFO [Thread-587] util.HBaseFsck(1107): Checking
> HBase region split map from HDFS data...
> 2014-04-12 23:24:57,782 INFO [Thread-587] util.HBaseFsck(617): No integrity
> errors. We are done with this phase. Glorious.
> 2014-04-12 23:24:57,835 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer
> 2014-04-12 23:24:57,836 INFO [RpcServer.handler=0,port=33001]
> master.HMaster(1550): Client=jenkins//172.29.122.11 set balanceSwitch=false
> 2014-04-12 23:24:57,840 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process identifier=hbase Fsck connecting
> to ZooKeeper ensemble=localhost:53570
> 2014-04-12 23:24:57,869 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): hbase Fsck, quorum=localhost:53570,
> baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected,
> path=null
> 2014-04-12 23:24:57,870 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387): hbase Fsck-0x14559c215740045 connected
> 2014-04-12 23:24:57,902 INFO [Thread-587] util.HBaseFsck(431): Loading
> regionsinfo from the hbase:meta table
> 2014-04-12 23:24:57,912 INFO [Thread-587] util.HBaseFsck(2521):
> getHTableDescriptors == tableNames => []
> 2014-04-12 23:24:57,922 DEBUG [Thread-587] util.HBaseFsck(1334): Loading
> region dirs from
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile
> 2014-04-12 23:24:57,922 DEBUG [Thread-587] util.HBaseFsck(1334): Loading
> region dirs from hdfs://localhost:50919/user/jenkins/hbase/data/hbase/meta
> 2014-04-12 23:24:57,923 DEBUG [pool-115-thread-10]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/hbase/meta/1588230740
> 2014-04-12 23:24:57,924 DEBUG [pool-115-thread-9]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:57,926 DEBUG [pool-115-thread-9]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:57,929 DEBUG [pool-115-thread-9]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:57,931 DEBUG [pool-115-thread-9]
> util.HBaseFsck$WorkItemHdfsDir(3211): Loading region info from
> hdfs:hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:57,936 DEBUG [pool-115-thread-4] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY
> => '', ENDKEY => ''}
> 2014-04-12 23:24:57,937 DEBUG [pool-115-thread-1] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => d383980be98665b638fd56bfac97a351, NAME =>
> 'testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.',
> STARTKEY => 'B', ENDKEY => 'C'}
> 2014-04-12 23:24:57,977 DEBUG [pool-115-thread-6] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => a8a68d998d21b00499aca60887ae5aef, NAME =>
> 'testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.',
> STARTKEY => 'A', ENDKEY => 'B'}
> 2014-04-12 23:24:57,977 DEBUG [pool-115-thread-5] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => f9c185520bca999a753ee3ce0a244f6d, NAME =>
> 'testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.',
> STARTKEY => 'C', ENDKEY => ''}
> 2014-04-12 23:24:57,978 DEBUG [pool-115-thread-3] util.HBaseFsck(762):
> HRegionInfo read: {ENCODED => 63cdcba1fc55ae6463463ae16f4e454e, NAME =>
> 'testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.',
> STARTKEY => '', ENDKEY => 'A'}
> 2014-04-12 23:24:58,004 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process identifier=hbase Fsck connecting
> to ZooKeeper ensemble=localhost:53570
> 2014-04-12 23:24:58,036 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): hbase Fsck, quorum=localhost:53570,
> baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected,
> path=null
> 2014-04-12 23:24:58,037 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387): hbase Fsck-0x14559c215740046 connected
> 2014-04-12 23:24:58,069 INFO [Thread-587] util.HBaseFsck(1650): Region {
> meta =>
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.,
> hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e,
> deployed => } is in META, and in a disabled tabled that is not deployed
> 2014-04-12 23:24:58,069 INFO [Thread-587] util.HBaseFsck(1650): Region {
> meta =>
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.,
> hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef,
> deployed => } is in META, and in a disabled tabled that is not deployed
> 2014-04-12 23:24:58,069 INFO [Thread-587] util.HBaseFsck(1650): Region {
> meta =>
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.,
> hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351,
> deployed => } is in META, and in a disabled tabled that is not deployed
> 2014-04-12 23:24:58,069 INFO [Thread-587] util.HBaseFsck(1650): Region {
> meta =>
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.,
> hdfs =>
> hdfs://localhost:50919/user/jenkins/hbase/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d,
> deployed => } is in META, and in a disabled tabled that is not deployed
> 2014-04-12 23:24:58,070 DEBUG [Thread-587] util.HBaseFsck(1804): There are 5
> region info entries
> 2014-04-12 23:24:58,110 DEBUG [pool-1-thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:33001-0x14559c215740000,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer
> 2014-04-12 23:24:58,111 INFO [RpcServer.handler=0,port=33001]
> master.HMaster(1550): Client=jenkins//172.29.122.11 set balanceSwitch=true
> 2014-04-12 23:24:58,152 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process identifier=hbase Fsck connecting
> to ZooKeeper ensemble=localhost:53570
> 2014-04-12 23:24:58,177 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): hbase Fsck, quorum=localhost:53570,
> baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected,
> path=null
> 2014-04-12 23:24:58,179 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387): hbase Fsck-0x14559c215740047 connected
> 2014-04-12 23:24:58,182 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process
> identifier=catalogtracker-on-hconnection-0x1ae9e1aa connecting to ZooKeeper
> ensemble=localhost:53570
> 2014-04-12 23:24:58,182 DEBUG [Thread-587] catalog.CatalogTracker(193):
> Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@73b62d67
> 2014-04-12 23:24:58,219 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x1ae9e1aa,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=SyncConnected, path=null
> 2014-04-12 23:24:58,219 DEBUG [Thread-587] zookeeper.ZKUtil(428):
> catalogtracker-on-hconnection-0x1ae9e1aa, quorum=localhost:53570,
> baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
> 2014-04-12 23:24:58,220 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387):
> catalogtracker-on-hconnection-0x1ae9e1aa-0x14559c215740048 connected
> 2014-04-12 23:24:58,226 DEBUG [Thread-587] catalog.CatalogTracker(209):
> Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@73b62d67
> 2014-04-12 23:24:58,279 INFO [Thread-587]
> zookeeper.RecoverableZooKeeper(120): Process
> identifier=catalogtracker-on-hconnection-0x1ae9e1aa connecting to ZooKeeper
> ensemble=localhost:53570
> 2014-04-12 23:24:58,279 DEBUG [Thread-587] catalog.CatalogTracker(193):
> Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@490d9812
> 2014-04-12 23:24:58,302 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x1ae9e1aa,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=SyncConnected, path=null
> 2014-04-12 23:24:58,303 DEBUG [Thread-587] zookeeper.ZKUtil(428):
> catalogtracker-on-hconnection-0x1ae9e1aa, quorum=localhost:53570,
> baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
> 2014-04-12 23:24:58,303 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(387):
> catalogtracker-on-hconnection-0x1ae9e1aa-0x14559c215740049 connected
> 2014-04-12 23:24:58,309 DEBUG [Thread-587] catalog.CatalogTracker(209):
> Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@490d9812
> 2014-04-12 23:24:58,354 INFO [RpcServer.handler=0,port=33001]
> master.HMaster(1817): Client=jenkins//172.29.122.11 delete
> testQuarantineMissingHFile
> 2014-04-12 23:24:58,420 DEBUG [RpcServer.handler=0,port=33001]
> lock.ZKInterProcessLockBase(226): Acquired a lock for
> /hbase/table-lock/testQuarantineMissingHFile/write-master:330010000000002
> 2014-04-12 23:24:58,432 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.TableEventHandler(123): Handling table operation C_M_DELETE_TABLE on
> table testQuarantineMissingHFile
> 2014-04-12 23:24:58,438 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(93): Deleting regions from META
> 2014-04-12 23:24:58,449 INFO [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> catalog.MetaEditor(495): Deleted [{ENCODED =>
> 63cdcba1fc55ae6463463ae16f4e454e, NAME =>
> 'testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.',
> STARTKEY => '', ENDKEY => 'A'}, {ENCODED =>
> a8a68d998d21b00499aca60887ae5aef, NAME =>
> 'testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.',
> STARTKEY => 'A', ENDKEY => 'B'}, {ENCODED =>
> d383980be98665b638fd56bfac97a351, NAME =>
> 'testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.',
> STARTKEY => 'B', ENDKEY => 'C'}, {ENCODED =>
> f9c185520bca999a753ee3ce0a244f6d, NAME =>
> 'testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.',
> STARTKEY => 'C', ENDKEY => ''}]
> 2014-04-12 23:24:58,510 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(104): Archiving region
> testQuarantineMissingHFile,,1397370294808.63cdcba1fc55ae6463463ae16f4e454e.
> from FS
> 2014-04-12 23:24:58,511 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(93): ARCHIVING
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:58,515 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(134): Archiving [class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath,
> file:hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e/fam]
> 2014-04-12 23:24:58,635 INFO [IPC Server handler 2 on 50919]
> blockmanagement.BlockManager(1053): BLOCK* addToInvalidates:
> blk_1073741869_1045 127.0.0.1:49422 127.0.0.1:33353 127.0.0.1:53997
> 2014-04-12 23:24:58,636 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(453): Deleted all region files in:
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/63cdcba1fc55ae6463463ae16f4e454e
> 2014-04-12 23:24:58,636 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(104): Archiving region
> testQuarantineMissingHFile,A,1397370294808.a8a68d998d21b00499aca60887ae5aef.
> from FS
> 2014-04-12 23:24:58,636 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(93): ARCHIVING
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:58,641 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(134): Archiving [class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath,
> file:hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef/fam]
> 2014-04-12 23:24:58,802 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(438): Finished archiving from class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath,
> file:hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef/fam/0ce0a00435f64a02856bfe0fb779093b,
> to
> hdfs://localhost:50919/user/jenkins/hbase/archive/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef/fam/0ce0a00435f64a02856bfe0fb779093b
> 2014-04-12 23:24:58,835 INFO [IPC Server handler 7 on 50919]
> blockmanagement.BlockManager(1053): BLOCK* addToInvalidates:
> blk_1073741871_1047 127.0.0.1:33353 127.0.0.1:53997 127.0.0.1:49422
> 2014-04-12 23:24:58,836 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(453): Deleted all region files in:
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/a8a68d998d21b00499aca60887ae5aef
> 2014-04-12 23:24:58,836 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(104): Archiving region
> testQuarantineMissingHFile,B,1397370294808.d383980be98665b638fd56bfac97a351.
> from FS
> 2014-04-12 23:24:58,836 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(93): ARCHIVING
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:58,840 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(134): Archiving [class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath,
> file:hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351/fam]
> 2014-04-12 23:24:58,860 INFO [IPC Server handler 7 on 50919]
> blockmanagement.BlockManager(1053): BLOCK* addToInvalidates:
> blk_1073741868_1044 127.0.0.1:53997 127.0.0.1:49422 127.0.0.1:33353
> 2014-04-12 23:24:58,861 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(453): Deleted all region files in:
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/d383980be98665b638fd56bfac97a351
> 2014-04-12 23:24:58,861 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(104): Archiving region
> testQuarantineMissingHFile,C,1397370294808.f9c185520bca999a753ee3ce0a244f6d.
> from FS
> 2014-04-12 23:24:58,861 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(93): ARCHIVING
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:58,865 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(134): Archiving [class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath,
> file:hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d/fam]
> 2014-04-12 23:24:58,893 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(438): Finished archiving from class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath,
> file:hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d/fam/404e9e4c06b54305a0831dc4749d9a5f,
> to
> hdfs://localhost:50919/user/jenkins/hbase/archive/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d/fam/404e9e4c06b54305a0831dc4749d9a5f
> 2014-04-12 23:24:58,902 INFO [IPC Server handler 8 on 50919]
> blockmanagement.BlockManager(1053): BLOCK* addToInvalidates:
> blk_1073741870_1046 127.0.0.1:33353 127.0.0.1:53997 127.0.0.1:49422
> 2014-04-12 23:24:58,902 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> backup.HFileArchiver(453): Deleted all region files in:
> hdfs://localhost:50919/user/jenkins/hbase/.tmp/data/default/testQuarantineMissingHFile/f9c185520bca999a753ee3ce0a244f6d
> 2014-04-12 23:24:58,910 INFO [IPC Server handler 5 on 50919]
> blockmanagement.BlockManager(1053): BLOCK* addToInvalidates:
> blk_1073741867_1043 127.0.0.1:33353 127.0.0.1:53997 127.0.0.1:49422
> 2014-04-12 23:24:58,910 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(114): Table 'testQuarantineMissingHFile' archived!
> 2014-04-12 23:24:58,911 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(117): Removing 'testQuarantineMissingHFile'
> descriptor.
> 2014-04-12 23:24:58,912 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(121): Removing 'testQuarantineMissingHFile' from
> region states.
> 2014-04-12 23:24:58,912 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> handler.DeleteTableHandler(125): Marking 'testQuarantineMissingHFile' as
> deleted.
> 2014-04-12 23:24:58,927 DEBUG [MASTER_TABLE_OPERATIONS-p0419:33001-0]
> lock.ZKInterProcessLockBase(328): Released
> /hbase/table-lock/testQuarantineMissingHFile/write-master:330010000000002
> 2014-04-12 23:24:59,068 INFO [Thread-587] client.HBaseAdmin(696): Deleted
> testQuarantineMissingHFile
> 2014-04-12 23:24:59,115 INFO [pool-1-thread-1] hbase.ResourceChecker(171):
> after: util.TestHBaseFsck#testQuarantineMissingHFile Thread=420 (was 410)
> Potentially hanging thread: Thread-587-SendThread(localhost:53570)
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1075)
> Potentially hanging thread: pool-115-thread-1
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-6
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: DataXceiver for client
> DFSClient_NONMAPREDUCE_-1456019779_8 at /127.0.0.1:34032 [Waiting for
> operation #5]
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> java.io.DataInputStream.readShort(DataInputStream.java:312)
>
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:55)
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:206)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-4
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-3
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-5
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: Thread-587-EventThread
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
> Potentially hanging thread: pool-115-thread-9
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: DataXceiver for client
> DFSClient_NONMAPREDUCE_-1456019779_8 at /127.0.0.1:34026 [Waiting for
> operation #7]
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> java.io.DataInputStream.readShort(DataInputStream.java:312)
>
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:55)
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:206)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: DataXceiver for client
> DFSClient_NONMAPREDUCE_-1456019779_8 at /127.0.0.1:33883 [Waiting for
> operation #12]
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> java.io.DataInputStream.readShort(DataInputStream.java:312)
>
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:55)
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:206)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-8
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-10
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: DataXceiver for client
> DFSClient_NONMAPREDUCE_-1456019779_8 at /127.0.0.1:36692 [Waiting for
> operation #7]
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> java.io.DataInputStream.readShort(DataInputStream.java:312)
>
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:55)
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:206)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: DataXceiver for client
> DFSClient_NONMAPREDUCE_-1456019779_8 at /127.0.0.1:44587 [Waiting for
> operation #12]
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> java.io.DataInputStream.readShort(DataInputStream.java:312)
>
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:55)
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:206)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: DataXceiver for client
> DFSClient_NONMAPREDUCE_-1456019779_8 at /127.0.0.1:36707 [Waiting for
> operation #6]
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> java.io.DataInputStream.readShort(DataInputStream.java:312)
>
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:55)
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:206)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-2
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: DataXceiver for client
> DFSClient_NONMAPREDUCE_-1456019779_8 at /127.0.0.1:44585 [Waiting for
> operation #14]
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
>
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> java.io.DataInputStream.readShort(DataInputStream.java:312)
>
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:55)
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:206)
> java.lang.Thread.run(Thread.java:724)
> Potentially hanging thread: pool-115-thread-7
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
>
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
>
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:724)
> - Thread LEAK? -, OpenFileDescriptor=692 (was 691) - OpenFileDescriptor
> LEAK? -, MaxFileDescriptor=32768 (was 32768), SystemLoadAverage=453 (was 422)
> - SystemLoadAverage LEAK? -, ProcessCount=485 (was 485),
> AvailableMemoryMB=3026 (was 3139), ConnectionCount=5 (was 5)
> 2014-04-12 23:28:53,025 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(310): hbase Fsck-0x14559c215740047,
> quorum=localhost:53570, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=Disconnected, path=null
> 2014-04-12 23:28:53,025 DEBUG [Thread-587-EventThread]
> zookeeper.ZooKeeperWatcher(392): hbase Fsck-0x14559c215740047,
> quorum=localhost:53570, baseZNode=/hbase Received Disconnected from
> ZooKeeper, ignoring
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
