[
https://issues.apache.org/jira/browse/HBASE-13405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395212#comment-14395212
]
Mikhail Antonov commented on HBASE-13405:
-----------------------------------------
In fact I can see very similar failures in other test cases in this class, e.g.
{code}
2015-04-03 15:48:31,330 WARN [Thread-5524] util.HBaseFsck(1759): Could not
process regionserver 10.1.4.219:50028
org.apache.hadoop.hbase.ipc.CallTimeoutException: callId: 7 methodName:
GetOnlineRegion param {TODO: class
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$GetOnlineRegionRequest}
at
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:235)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.getOnlineRegion(AdminProtos.java:22869)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getOnlineRegions(ProtobufUtil.java:1784)
at
org.apache.hadoop.hbase.util.HBaseFsck$WorkItemRegion.call(HBaseFsck.java:3895)
at
org.apache.hadoop.hbase.util.HBaseFsck$WorkItemRegion.call(HBaseFsck.java:3874)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
2015-04-03 15:48:29,727 DEBUG [RS_CLOSE_META-10.1.4.219:50028-0]
regionserver.HRegionFileSystem(381): Committing store file
hdfs://localhost:50002/user/mantonov/test-data/539995fc-b1fe-452f-9f50-42d07acc78ac/data/hbase/meta/1588230740/.tmp/e468fb98da9241e2aeaf47d0b92e2cc7
as
hdfs://localhost:50002/user/mantonov/test-data/539995fc-b1fe-452f-9f50-42d07acc78ac/data/hbase/meta/1588230740/table/e468fb98da9241e2aeaf47d0b92e2cc7
2015-04-03 15:48:29,726 INFO
[RpcServer.reader=1,bindAddress=10.1.4.219,port=50028]
ipc.RpcServer$Connection(1640): Connection from 10.1.4.219 port: 53374 with
version info: version: "2.0.0-SNAPSHOT" url:
"git://Mikhails-MacBook-Pro.local/wandisco/source/hbase" revision:
"d8b10656d00779e194c3caca118995136babce99" user: "mantonov" date: "Fri Apr 3
15:22:53 PDT 2015" src_checksum: "e261fe7369b6136e325d351242d32004"
java.lang.AssertionError:
Expected :[UNKNOWN, NO_META_REGION, NULL_META_REGION]
Actual :[UNKNOWN, NO_META_REGION, NULL_META_REGION, RS_CONNECT_FAILURE,
RS_CONNECT_FAILURE]
<Click to see difference>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertErrors(HbckTestingUtil.java:100)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testFixAssignmentsWhenMETAinTransition(TestHBaseFsck.java:278)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
> TestHBaseFsck is flaky
> ----------------------
>
> Key: HBASE-13405
> URL: https://issues.apache.org/jira/browse/HBASE-13405
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0
> Reporter: Mikhail Antonov
>
> Once in a while I'm seeing the following, running #testContainedRegionOverlap
> test in IDE after clean install (mac osx, hbase master):
> {code}
> regionserver.HRegionServer(1863): Post open deploy tasks for
> tableContainedRegionOverlap,A,1428099123733.03a139b02119e99ef08149addd9a7996.
> 2015-04-03 15:12:11,695 INFO
> [PostOpenDeployTasks:03a139b02119e99ef08149addd9a7996]
> regionserver.HRegionServer(1956): Failed to report region transition, will
> retry
> java.io.InterruptedIOException: Origin: InterruptedException
> at
> org.apache.hadoop.hbase.util.ExceptionUtil.asInterrupt(ExceptionUtil.java:65)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:313)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.reportRegionStateTransition(HRegionServer.java:1955)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1882)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:241)
> Caused by: java.lang.InterruptedException: callId: 158 methodName:
> ReportRegionStateTransition param {TODO: class
> org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$ReportRegionStateTransitionRequest}
> at
> io.netty.util.concurrent.DefaultPromise.await0(DefaultPromise.java:333)
> at
> io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:266)
> at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:42)
> at
> org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:226)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.reportRegionStateTransition(RegionServerStatusProtos.java:9030)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.reportRegionStateTransition(HRegionServer.java:1946)
> ... 2 more
> 2015-04-03 15:12:11,696 INFO
> [B.defaultRpcServer.handler=1,queue=0,port=51217]
> master.MasterRpcServices(237): Client=mantonov//10.1.4.219 set
> balanceSwitch=false
> 2015-04-03 15:12:11,696 DEBUG [main-EventThread]
> zookeeper.ZooKeeperWatcher(388): maste
> {code}
> and then:
> {code}
> 015-04-03 15:12:11,796 INFO [Thread-3019] client.HBaseAdmin$10(981): Started
> disable of tableContainedRegionOverlap
> 2015-04-03 15:12:21,641 INFO
> [B.defaultRpcServer.handler=1,queue=0,port=51217] master.HMaster(1645):
> Client=mantonov//10.1.4.219 disable tableContainedRegionOverlap
> java.lang.AssertionError:
> Expected :[]
> Actual :[NOT_DEPLOYED, HOLE_IN_REGION_CHAIN]
> <Click to see difference>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertNoErrors(HbckTestingUtil.java:92)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testContainedRegionOverlap(TestHBaseFsck.java:941)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
