Re: WrongRegionException

2018-01-31 Thread Anoop John 
Or else you should do below in prePut
Create the new Put and get the COnnection from the CP environment and
do a put op on the connection. (Not directly on Region)  And then call
bypass() to bypass this Put op with old RK.  Just suggesting. But I
some how feel that why not a middle client layer can do this convert

-Anoop-

On Thu, Feb 1, 2018 at 1:07 PM, Anoop John  wrote:
> Theoretically I dont think what you are trying to do is correct. I
> mean the incoming keys are fully ignored and new one is been made at
> CP layers. When CP is been contacted already the mutations has reached
> upto that Region.  The new RK might be outside the boundary of this
> region. (As the split happens).   Dont know your use case. But I feel
> like you should handle this conversion of RKs and create new Put in a
> client tier not at CP.
>
> -Anoop-
>
> On Mon, Jan 29, 2018 at 9:44 PM, Ted Yu  wrote:
>> w.r.t. region split, do you verify that the new rowkey is in the same
>> region as the rowkey from incoming Put ?
>>
>> If not, there is a chance that the new rowkey is in different region which
>> is going thru split.
>>
>> FYI
>>
>> On Mon, Jan 29, 2018 at 6:40 AM, Yang Zhang  wrote:
>>
>>> Both are the same question.
>>> I want to prevent the incoming puts and then copy it as one or more new
>>> puts with different rowkey.
>>> So when there is only one region, my rowkey will belong to it. But when
>>> region splited, some rowkeys may not belong to the new region.
>>> I used to thought HBase will stop new coming puts, finish all of the puts
>>> in the batch, and then try to split.
>>> But this maybe not right according to the exception that I got.
>>>
>>> BTY , It seems that I can't add put
>>> to MiniBatchOperationInProgress miniBatchOp. There are only some
>>> functions for get.
>>>
>>> Thank you very much for your help
>>>
>>> 2018-01-29 18:46 GMT+08:00 Anoop John :
>>>
>>> > Another related Q was also there..  Can you tell the actual
>>> > requirement?  So the incoming puts you want to change the RKs of that?
>>> > Or you want to insert those as well as some new cells with a changed
>>> > RK?
>>> >
>>> > -Anoop-
>>> >
>>> > On Mon, Jan 29, 2018 at 3:49 PM, Yang Zhang 
>>> > wrote:
>>> > > Hello Everyone
>>> > >
>>> > > I am using coprocesser to prevent the normal put and replace it
>>> > > with another rowkey, The method is HRegion.put(). It works fine, but
>>> when
>>> > > the region splited, There will be an WrongRegionException.
>>> > >
>>> > > 2018-01-28 09:32:51,528 WARN
>>> > > [B.DefaultRpcServer.handler=21,queue=3,port=60020]
>>> regionserver.HRegion:
>>> > > Failed getting lock in batch put,
>>> > > row=\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r
>>> > > org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested
>>> row
>>> > > out of range for row lock on HRegion
>>> > > GISdoop_GeoKey,,1517085124215.341534e84727245f1c67f345c3e467ac.,
>>> > > startKey='', getEndKey()='\xE6G8\x00\x00\x00\x00\x00\x00\x00\x00\x00',
>>> > > row='\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r'
>>> > > at org.apache.hadoop.hbase.regionserver.HRegion.checkRow(
>>> > HRegion.java:4677)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> > getRowLock(HRegion.java:4695)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> > doMiniBatchMutation(HRegion.java:2786)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> > batchMutate(HRegion.java:2653)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> > batchMutate(HRegion.java:2589)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> doBatchMutate(HRegion.java:
>>> > 3192)
>>> > > at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:2459)
>>> > > at site.luoyu.Core.Index.JavaTreeMap.insertPoint(JavaTreeMap.java:287)
>>> > > at site.luoyu.Core.Index.JavaTreeMap.insertRecord(
>>> JavaTreeMap.java:256)
>>> > > at site.luoyu.Core.Observer.IndexCopressor.prePut(
>>> > IndexCopressor.java:130)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(
>>> > RegionCoprocessorHost.java:1122)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> doPreMutationHook(HRegion.
>>> > java:2674)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> > batchMutate(HRegion.java:2649)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> > batchMutate(HRegion.java:2589)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>>> > batchMutate(HRegion.java:2593)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegionServer.
>>> > doBatchOp(HRegionServer.java:4402)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegionServer.
>>> > doNonAtomicRegionMutation(HRegionServer.java:3584)
>>> > > at
>>> > > org.apache.hadoop.hbase.regionserver.HRegionServer.
>>> >

Re: WrongRegionException

2018-01-31 Thread Anoop John 
Theoretically I dont think what you are trying to do is correct. I
mean the incoming keys are fully ignored and new one is been made at
CP layers. When CP is been contacted already the mutations has reached
upto that Region.  The new RK might be outside the boundary of this
region. (As the split happens).   Dont know your use case. But I feel
like you should handle this conversion of RKs and create new Put in a
client tier not at CP.

-Anoop-

On Mon, Jan 29, 2018 at 9:44 PM, Ted Yu  wrote:
> w.r.t. region split, do you verify that the new rowkey is in the same
> region as the rowkey from incoming Put ?
>
> If not, there is a chance that the new rowkey is in different region which
> is going thru split.
>
> FYI
>
> On Mon, Jan 29, 2018 at 6:40 AM, Yang Zhang  wrote:
>
>> Both are the same question.
>> I want to prevent the incoming puts and then copy it as one or more new
>> puts with different rowkey.
>> So when there is only one region, my rowkey will belong to it. But when
>> region splited, some rowkeys may not belong to the new region.
>> I used to thought HBase will stop new coming puts, finish all of the puts
>> in the batch, and then try to split.
>> But this maybe not right according to the exception that I got.
>>
>> BTY , It seems that I can't add put
>> to MiniBatchOperationInProgress miniBatchOp. There are only some
>> functions for get.
>>
>> Thank you very much for your help
>>
>> 2018-01-29 18:46 GMT+08:00 Anoop John :
>>
>> > Another related Q was also there..  Can you tell the actual
>> > requirement?  So the incoming puts you want to change the RKs of that?
>> > Or you want to insert those as well as some new cells with a changed
>> > RK?
>> >
>> > -Anoop-
>> >
>> > On Mon, Jan 29, 2018 at 3:49 PM, Yang Zhang 
>> > wrote:
>> > > Hello Everyone
>> > >
>> > > I am using coprocesser to prevent the normal put and replace it
>> > > with another rowkey, The method is HRegion.put(). It works fine, but
>> when
>> > > the region splited, There will be an WrongRegionException.
>> > >
>> > > 2018-01-28 09:32:51,528 WARN
>> > > [B.DefaultRpcServer.handler=21,queue=3,port=60020]
>> regionserver.HRegion:
>> > > Failed getting lock in batch put,
>> > > row=\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r
>> > > org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested
>> row
>> > > out of range for row lock on HRegion
>> > > GISdoop_GeoKey,,1517085124215.341534e84727245f1c67f345c3e467ac.,
>> > > startKey='', getEndKey()='\xE6G8\x00\x00\x00\x00\x00\x00\x00\x00\x00',
>> > > row='\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r'
>> > > at org.apache.hadoop.hbase.regionserver.HRegion.checkRow(
>> > HRegion.java:4677)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> > getRowLock(HRegion.java:4695)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> > doMiniBatchMutation(HRegion.java:2786)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> > batchMutate(HRegion.java:2653)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> > batchMutate(HRegion.java:2589)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> doBatchMutate(HRegion.java:
>> > 3192)
>> > > at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:2459)
>> > > at site.luoyu.Core.Index.JavaTreeMap.insertPoint(JavaTreeMap.java:287)
>> > > at site.luoyu.Core.Index.JavaTreeMap.insertRecord(
>> JavaTreeMap.java:256)
>> > > at site.luoyu.Core.Observer.IndexCopressor.prePut(
>> > IndexCopressor.java:130)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(
>> > RegionCoprocessorHost.java:1122)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> doPreMutationHook(HRegion.
>> > java:2674)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> > batchMutate(HRegion.java:2649)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> > batchMutate(HRegion.java:2589)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegion.
>> > batchMutate(HRegion.java:2593)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegionServer.
>> > doBatchOp(HRegionServer.java:4402)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegionServer.
>> > doNonAtomicRegionMutation(HRegionServer.java:3584)
>> > > at
>> > > org.apache.hadoop.hbase.regionserver.HRegionServer.
>> > multi(HRegionServer.java:3474)
>> > > at
>> > > org.apache.hadoop.hbase.protobuf.generated.
>> ClientProtos$ClientService$2.
>> > callBlockingMethod(ClientProtos.java:3)
>> > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078)
>> > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
>> > > at
>> > > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(
>> > RpcExecutor.java:114)
>> > > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
>> > > at