Hi, It seems similar. The infinite loop sounds right because the region server doesn't seem to want to come down cleanly.
On Wed, Sep 9, 2015 at 2:23 PM, Rendon, Carlos (KBB - Irvine) < [email protected]> wrote: > We've been seeing a similar issue. > Take a look at https://issues.apache.org/jira/browse/HBASE-13471 and see > if it matches what you are seeing. > > -Carlos > > -----Original Message----- > From: Hbase Janitor [mailto:[email protected]] > Sent: Monday, September 07, 2015 11:05 AM > To: [email protected] > Subject: Re: strange replication problem > > I wanted to add that the message just happen once or twice, it's flooding > the logs. > > We had to stop replication and stop the region server to stop it. > > On Mon, Sep 7, 2015 at 2:00 PM, Ted Yu <[email protected]> wrote: > > > WrongRegionException is retriable. Meaning the client would retry upon > > receiving the exception. > > > > Did you observe any failed job(s) ? > > > > > > Cheers > > > > On Mon, Sep 7, 2015 at 10:54 AM, Hbase Janitor > > <[email protected]> > > wrote: > > > > > Ted, thank you for your response. > > > > > > Yes, there was. Looks like it moved yesterday morning. > > > > > > On Mon, Sep 7, 2015 at 1:47 PM, Ted Yu <[email protected]> wrote: > > > > > > > Was there region movement prior to 11:24:00 (on the region server > > > > where WrongRegionException was observed) ? > > > > > > > > Cheers > > > > > > > > On Mon, Sep 7, 2015 at 9:59 AM, Hbase Janitor > > > > <[email protected]> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > We've recently upgraded to hbase 1.0 and we are seeing a strange > > error > > > in > > > > > the region logs: > > > > > > > > > > 2015-09-07 11:24:00,960 WARN > > > > org.apache.hadoop.hbase.regionserver.HRegion: > > > > > Failed getting lock in batch put, row=HMV14395619228 > > > > > > > > > > org.apache.hadoop.hbase.regionserver.WrongRegionException: > > > > > Requested > > > row > > > > > out of range for row lock on HRegion region-name., > > > > startKey='ZMA178741312', > > > > > getEndKey()='ZMY253622718', row='HMV14395619228' > > > > > > > > > > at > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:4 > > > 731) > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegion.getRowLockInternal(HRegio > > n.java:4762) > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegi > > on.java:2877) > > > > > > > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java: > > 2741) > > > > > > > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java: > > 2676) > > > > > > > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java: > > 2680) > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServ > > ices.java:653) > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMu > > tation(RSRpcServices.java:615) > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices > > .java:1901) > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$ > > 2.callBlockingMethod(ClientProtos.java:31451) > > > > > > > > > > at > > > > org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035) > > > > > > > > > > at > > > > org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java: > > 130) > > > > > > > > > > at > > > > > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:1 > > > > > 07) > > > > > > > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > > > > > > > Any idea what causes it? > > > > > > > > > > > > > > >
