[ 
https://issues.apache.org/jira/browse/HBASE-22422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840955#comment-16840955
 ] 

Zheng Hu commented on HBASE-22422:
----------------------------------

Another risk is here, LruBlockCache#evictBlock,  we should move the 
previous.getBuffer().release() to the last line before return because once the 
release decrease the refCnt to zero then nobody can access the buf (such as 
victimHandler).
{code}
  protected long evictBlock(LruCachedBlock block, boolean 
evictedByEvictionProcess) {
    LruCachedBlock previous = map.remove(block.getCacheKey());
    if (previous == null) {
      return 0;
    }
    // Decrease the block's reference count, and if refCount is 0, then it'll 
auto-deallocate.
    previous.getBuffer().release();
    updateSizeMetrics(block, true);
    long val = elements.decrementAndGet();
    if (LOG.isTraceEnabled()) {
      long size = map.size();
      assertCounterSanity(size, val);
    }
    if (block.getBuffer().getBlockType().isData()) {
       dataBlockElements.decrement();
    }
    if (evictedByEvictionProcess) {
      // When the eviction of the block happened because of invalidation of 
HFiles, no need to
      // update the stats counter.
      stats.evicted(block.getCachedTime(), block.getCacheKey().isPrimary());
      if (victimHandler != null) {
        victimHandler.cacheBlock(block.getCacheKey(), block.getBuffer());
      }
    }
    return block.heapSize();
  }
{code}

> Retain an ByteBuff with refCnt=0 when getBlock from LRUCache
> ------------------------------------------------------------
>
>                 Key: HBASE-22422
>                 URL: https://issues.apache.org/jira/browse/HBASE-22422
>             Project: HBase
>          Issue Type: Sub-task
>          Components: BlockCache
>            Reporter: Zheng Hu
>            Assignee: Zheng Hu
>            Priority: Major
>         Attachments: LRUBlockCache-getBlock.png, 
> image-2019-05-15-12-00-03-641.png
>
>
> After runing YCSB scan/get benchmark in our XiaoMi cluster,  we found the get 
> QPS dropped from  25000/s to hunderds per second in a cluster with five 
> nodes.  
> After enable the debug log at YCSB client side,  I found the following 
> stacktrace , see 
> https://issues.apache.org/jira/secure/attachment/12968745/image-2019-05-15-12-00-03-641.png
> Need a patch to fix this. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to