[
https://issues.apache.org/jira/browse/HBASE-22127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807492#comment-16807492
]
Zheng Hu commented on HBASE-22127:
----------------------------------
Got it, made an mistake here:
{code}
+ if (buf instanceof HFileBlock) {
+ HFileBlock blk = ((HFileBlock) buf);
+ if (!blk.getBufferReadOnly().hasArray()) {
+ buf = blk.deepCloneOnHeap();
+ }
+ } else {
+ // The block will be referenced by this LRUBlockCache, so should
increase its refCnt here.
+ buf.retain();
+ }
{code}
When block is an heap HFileBlock, it won't retain now while it should do that.
> Ensure that the block cached in the LRUBlockCache offheap is allocated from
> heap
> --------------------------------------------------------------------------------
>
> Key: HBASE-22127
> URL: https://issues.apache.org/jira/browse/HBASE-22127
> Project: HBase
> Issue Type: Sub-task
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Major
> Attachments: HBASE-22127.HBASE-21879.v1.patch
>
>
> In here [1], [~anoop.hbase] pointed out an crtial problem , I pasted here:
> bq. So if we read from HDFS into a pooled BB and then give to LRU cache for
> caching (ya mostly cache on read might be true) we will cache the block which
> is backed by this pooled DBB? Unless the block is evicted , this BB wont go
> back to pool. I think this is some thing we can not livw with !! For LRU
> cache the sizing itself is based on what % of heap size we can grow. But here
> in effect we are occupying the off heap space for the cached blocks. All the
> sizing assumptions and calc going out of control !
> It's indeed an big problem here. so we can only make the block ref to an heap
> area if we use LRUCache (both LruBlockCache and CombinedBlockCache case). Or
> we can also make the lru cache offheap ?
> I think we can introduce an switch indicate that whether the lru block cache
> offheap or not, if heap, then coping those bytes from ByteBuff to heap.
> https://reviews.apache.org/r/70153/diff/6?file=2133545#file2133545line398
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
