[
https://issues.apache.org/jira/browse/HBASE-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289314#comment-16289314
]
Anastasia Braginsky commented on HBASE-19282:
---------------------------------------------
bq. My intention for asking that was to do in a PoC way and measure the diff..
Ya when MSLAB is on heap all are on heap only.
I fully understand what you are saying and in HBASE-19506 I have said that the
JIRA is opened to check the issue. It is not a one line fix to check, and is
not related to this JIRA, therefore can not be done as part of this JIRA. But
we will do it on separate JIRA. You are welcome to use this JIRA as well.
bq. The diff is whether u want to waste the 2 MB chunks for much smaller sized
index. The index is going to stick there for some more time until flushes. Or
else the MSLAB should have been gives more space total considering the index
chunk needs also.
Clearly it sounds not reasonable to waste the 2 MB chunks. However the index is
not there till the flush-to-disk, but till next merge; and the more merges we
do the bigger the index is. One chunk index serves 2^19 cells (about half
million), therefore it is not likely to have more then one index chunk per
MSLAB. From here, I do not think MSLAB should have more space total considering
the index chunk needs.
bq. Say for index we need a size of 0.5 MB. We use the 2 MB chunk for this. Any
way the total chunks in MSLAB pool is fixed. So this might cause a request for
chunk for storing the cells, to create an on demand one.
bq. So will end up making a 2 MB on heap chunk instead we would have just
created a 0.5 MB earlier.
This is very clear, that fixed-size-chunks can be underutilized. I am with you
for creating a variable-size chunks. However, all chunks need to be created via
ChunkCreator. Please pay attention that to allocate an on-demand memory
off-heap is expensive. And you can not reuse those variable-size chunks (put
them into pool). So this is not a trivial answer whether it 100% worth to make
a variable-sized chunks.
> CellChunkMap Benchmarking and User Interface
> --------------------------------------------
>
> Key: HBASE-19282
> URL: https://issues.apache.org/jira/browse/HBASE-19282
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Attachments: CCM Benchmarking.pdf
>
>
> We have made some experiments how working with CellChunkMap (CCM) influences
> the performance when running on-heap and off-heap. Based on those results it
> is suggested to tie the MSLAB usage (off-heap or on-heap) with CCM index
> usage.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
