[
https://issues.apache.org/jira/browse/HBASE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868280#comment-16868280
]
ramkrishna.s.vasudevan commented on HBASE-22608:
------------------------------------------------
Seems an size accounting issue with in-memory compaction - due to some
threading issues? Or some wrong accounting.
> MVCC's writeEntry didn't complete and make MVCC stuck
> -----------------------------------------------------
>
> Key: HBASE-22608
> URL: https://issues.apache.org/jira/browse/HBASE-22608
> Project: HBase
> Issue Type: Bug
> Reporter: Guanghao Zhang
> Priority: Critical
>
> {code:java}
> 2019-06-20,05:03:44,917 ERROR
> [RpcServer.default.RWQ.Fifo.write.handler=61,queue=1,port=22600]
> org.apache.hadoop.hbase.regionserver.HRegion: Asked to modify this region's
> (xxxxxx,,1560481375170.10b01c12d58ce75c9aaf1ac15cc2a7f3.) memStoreSizing to a
> negative value which is incorrect. Current memStoreSizing=-1686222,
> delta=1489930
> java.lang.Exception
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkNegativeMemStoreDataSize(HRegion.java:1317)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.incMemStoreSize(HRegion.java:1295)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.writeMiniBatchOperationsToMemStore(HRegion.java:3316)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.writeMiniBatchOperationsToMemStore(HRegion.java:3821)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4248)
> at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4179)
> at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4109)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:1059)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicBatchOp(RSRpcServices.java:991)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:954)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2833)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42014)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> {code}
> See
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3728]
> {code:java}
> @Override
> public WriteEntry writeMiniBatchOperationsToMemStore(
> final MiniBatchOperationInProgress<Mutation> miniBatchOp, @Nullable
> WriteEntry writeEntry)
> throws IOException {
> if (writeEntry == null) {
> writeEntry = region.mvcc.begin();
> }
> super.writeMiniBatchOperationsToMemStore(miniBatchOp,
> writeEntry.getWriteNumber());
> return writeEntry;
> }
> {code}
> super.writeMiniBatchOperationsToMemStore throw a exception and the new
> writeEntry cannot be complete and make the MVCC stuck.
>
> And we meet this problem when enable in-memory compaction. But that should be
> another issue and need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
