Suggest keeping client version the same as the version on server. Can you check log of the region server where hbase:meta was hosted around the time this happened ? There may be some clue.
Thanks On Tue, Oct 11, 2016 at 2:21 AM, Shoubo Li <[email protected]> wrote: > Hi， I encountered a question bothering me for several days. > > my environment: > > 1. HBase Server: 1.0.11 > 2. HBase Client: 1.0.5-SNAPSHOT > > The region server always print warn log as follow: > > 2016-10-11 16:52:33,489 WARN > [RW.default.writeRpcServer.handler=46,queue=4,port=16020] > regionserver.HRegion: Failed getting lock in batch put, > row=0799\x00\x00\x01W\x90\x90\xE7&\x0A\x02m2=\x1E\x00\x00\x01 > org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested > row out of range for row lock on HRegion > hzop:vtrace,7280,1466058715841.f8db60758f6905bc0d81c23d03aba360., > startKey='7280', getEndKey()='7290', > row='0799\x00\x00\x01W\x90\x90\xE7&\x0A\x02m2=\x1E\x00\x00\x01' > at org.apache.hadoop.hbase.regionserver.HRegion.checkRow( > HRegion.java:3853) > at org.apache.hadoop.hbase.regionserver.HRegion. > getRowLockInternal(HRegion.java:3884) > at org.apache.hadoop.hbase.regionserver.HRegion. > doMiniBatchMutation(HRegion.java:2614) > at org.apache.hadoop.hbase.regionserver.HRegion. > batchMutate(HRegion.java:2478) > at org.apache.hadoop.hbase.regionserver.HRegion. > batchMutate(HRegion.java:2432) > at org.apache.hadoop.hbase.regionserver.HRegion. > batchMutate(HRegion.java:2436) > at org.apache.hadoop.hbase.regionserver.RSRpcServices. > doBatchOp(RSRpcServices.java:642) > at org.apache.hadoop.hbase.regionserver.RSRpcServices. > doNonAtomicRegionMutation(RSRpcServices.java:606) > at org.apache.hadoop.hbase.regionserver.RSRpcServices. > multi(RSRpcServices.java:1836) > ^Cat org.apache.hadoop.hbase.protobuf.generated. > ClientProtos$ClientService$2.callBlockingMethod > ClientProtos.java:31451) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop( > RpcExecutor.java:130) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > at java.lang.Thread.run(Thread.java:745) > > There are two questions: > > 1. why WrongRegionException happen? I known old hbase client version has > bugs, but latter version fixed it. > 2. > > why region server always print warn log? I also has read the source > code, I found every time RpcExcecutor will call > > consumerLoop(callQueues.get(index)); > > so I stop the hbase client job, so that no data would be written to this > table. however, the warn log always looped print. I wonder that the > wrong > row data is still in the queue, so every loop will fetch from the queue > and > handle it. > > any idea? >
