[
https://issues.apache.org/jira/browse/HBASE-22463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848724#comment-16848724
]
Zheng Hu edited comment on HBASE-22463 at 5/27/19 8:35 AM:
-----------------------------------------------------------
Upload the patch.v2 to address the unreleased block issue:
We have two kind of block type now: SHARED or EXCLUSIVE. for exclusive
hfileBlock, we won't do the block#release because we thought it's a heap block
( not from BucketCache), so no need to do that. but now when we read a block
from HFile, though it's a exclusive block but its memory was allocated from
ByteBuffAllocator, and MUST do a block#release if shipped to client. Otherwise,
the memory leak issue happen, say the ByteBuffAllocator will be full and
heapAllocationRatio will be 100%.
We introduced the SHARED or EXCLUSIVE type before, because we handling the heap
block and off-heap block as two diff path. After HBASE-21879, we did an
uniform, which means both heap block and offheap block will have a reference
count, the difference is heapHFileBlock#Recycler will do nothing and offheap
HFileBlock#Recyler will put the ByteBuffers back to allocator. So I mean the
memoryType can be removed now After HBASE-21879.
was (Author: openinx):
Upload the patch.v2 to address the unreleased block issue.
> Some paths in HFileScannerImpl did not consider block#release which will
> exhaust the ByteBuffAllocator
> --------------------------------------------------------------------------------------------------------
>
> Key: HBASE-22463
> URL: https://issues.apache.org/jira/browse/HBASE-22463
> Project: HBase
> Issue Type: Sub-task
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Major
> Attachments: HBASE-22463.HBASE-21879.v1.patch,
> HBASE-22463.HBASE-21879.v1.patch, HBASE-22463.HBASE-21879.v1.patch,
> HBASE-22463.HBASE-21879.v2.patch
>
>
> When I debug the issue HBASE-22422, I observed that the
> ByteBuffAllocator#usedBufCount will was always increasing and all direct
> ByteBuffers would be exhausted, which lead to may heap allocation happen.
> The comment here [1] is also related to this problem.
> Check the code path, the HFileScannerImpl is the biggest suspect, so create
> issue to address this.
> 1.
> https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16838446&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16838446
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
