[
https://issues.apache.org/jira/browse/HBASE-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15653107#comment-15653107
]
Anoop Sam John commented on HBASE-15786:
----------------------------------------
Oh ya.. I have added high level description but forgot to change the title of
the jira. It is not purely Off heap memstore. The memstore as such is same..
It is a way to make Off heap MSLAB pool. You can see that is the major change.
The patch is big because I have made 2 renaming.
HeapMemorySizeUtil -> MemorySizeUtil. Slight changes in methods here and
some new methods. The diff is this calc the off heap memory related stuff also.
The max off heap memory for global memstore.
HeapMSLAB -> MSLABImpl. This is because this deal with MSLAB on heap/off heap
now. Thought of making a new class OffheapMSLAB. The major change is when one
say off heap MSLAB, we think the chunks allocated by it is always off heap.
But we dont want create on demand DBBs which will NOT get pooled. Only those
from the pool is DBB. When pool is not having free chunk, we will make new
from MSLAB but that will be on heap in any case. Is it good to call that MSLAB
as OffheapMALAB? I am thinking not.
A cell is getting copied into the chunk area while adding to memstore. We have
write() API in ExtendedCell. So the cell is responsible for writing itself
into the MSLAB chunk area. Only diff is this was taking byte[] as the chunk
backing was that. Now I changed chunk backing to be BB. So all the methods
in CellUtil/KVUtil is copy of byte[] based methods. That is why confusing
names. Did not do any renaming or refactor way. Instead of cell doing the
write, we pull fragments of cell and write in KVUtil, this is for the Cells
NOT of type ExtendedCell.. Actually speaking all cells in server side is
supposed to be ExtendedCell type any way. Still for the completion of the
impl, we consider other also.
bq.AN HRegion needs reference to RegionServerServices memory management?
This is existing ref. I did not add in this patch.
bq.Should the DefaultHeapMemoryTuner be renamed now you've renamed the class as
MemorySizeUtil?
No. This tuner tune only Heap memory. When MSLAB pool is off heap we should
not be tuning that area. That may be another tuner. As of now we will keep it
simple. Though we need some changes. Ram is working on a patch for that. Will
come soon.
> Create Offheap Memstore
> -----------------------
>
> Key: HBASE-15786
> URL: https://issues.apache.org/jira/browse/HBASE-15786
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: ramkrishna.s.vasudevan
> Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-15786.patch
>
>
> We can make use of MSLAB pool for this off heap memstore.
> Right now one can specify the global memstore size (heap size) as a % of max
> memory using a config. We will add another config with which one can specify
> the global off heap memstore size. This will be exact size not as %. When off
> heap memstore in use, we will give this entire area for the MSLAB pool and
> that will create off heap chunks. So when cells are added to memstore, the
> cell data gets copied into the off heap MSLAB chunk spaces. Note that when
> the pool size is not really enough and we need additional chunk creation, we
> wont use off heap area for that. We dony want to create so many on demand
> DBBs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
