Zheng Hu created HBASE-22491:
--------------------------------
Summary: Separate the heap HFileBlock and offheap HFileBlock
because the heap block won't need refCnt and save into prevBlocks list before
shipping
Key: HBASE-22491
URL: https://issues.apache.org/jira/browse/HBASE-22491
Project: HBase
Issue Type: Sub-task
Reporter: Zheng Hu
Assignee: Zheng Hu
In here [1], [~anoop.hbase] has a comment:
bq. There is a concern here. Even if the block is on an exclusive heap memory
area, we will keep ref to that in this list. In a Phoenix Aggregation kind of
use case where many blocks might get fetched and not immediately shipped, we
are keeping the ref unwantedly here for longer time. This makes the GC not able
to reclaim the heap memory area for the blocks. This might be a hidden bomb
IMO. Its not good to remove the MemType. Lets create the block with memory type
as EXCLUSIVE when the block data is on heap. The block might be coming from LRU
cache or by fetching the block data from HDFS into heap memory area. When the
block comes from off heap BC or if it is backed by a BB from the pool (While
reading from HDFS, read into pooled BB) lets create the block with mem type as
SHARED. Every block can have the retain and release method but let the
EXCLUSIVE types do a noop here.
We've a discussion about this, say need to address two thing in this jira:
1. separate the HFileBlock into shared or non-shared;
2. Make the retain/release of non-shared block as a noop, say don't do
reference count change for heap block.
1. https://github.com/apache/hbase/pull/257/files
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
