the operation is taking longer then the rpc timeout to be executed (probably due to the large number of columns). the client will send the operation again to the region server, and you get that "may have already completed" exception. I think the property to increase the operation timeout is "hbase.client.operation.timeout", but someone can correct me if i'm wrong.
Matteo On Tue, Sep 8, 2015 at 11:02 AM, Sameer Babu K K < [email protected]> wrote: > I am sporadically getting OperationConflictException for individual > increment operations. The version of HBase is 0.98.6-cdh5.2.1. Around 200 > threads are writing to the same table. Some of the threads may be trying to > put many values and increment columns. > > Tried different combinations like separating puts and increments in > different batches, sending individual increments etc. But none of these > helps. Also noticed that whenever puts contain many columns (> 30) it > happens. I used to have 4 columns before and didn't see this issue then. > > org.apache.hadoop.hbase.exceptions.OperationConflictException: > org.apache.hadoop.hbase.exceptions.OperationConflictException: The > operation with nonce {6276654920751931456, 1349555467801158147} on row > [aDM^@^@^@^@^@^@^@a^@^@^@^@^@'?^@201408080000008||????D?A > ?] may have already completed > at > org.apache.hadoop.hbase.regionserver.HRegionServer.startNonceOperation(HRegionServer.java:4331) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.increment(HRegionServer.java:4295) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.mutate(HRegionServer.java:2986) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29585) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > at java.lang.Thread.run(Thread.java:744) > > >
