[
https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614876#comment-15614876
]
Eshcar Hillel commented on HBASE-16417:
---------------------------------------
First I plan to experiment with what we implemented so far without any
additional code, to have the full picture including mixed workloads, so we'll
know what is working well and what we want to improve on.
This would also be a baseline for comparing enhancement that can be achieved by
"smart" policies, including the ones you suggested that only merges segments
upon flush to disk (but this would need to wait to the 3rd round of experiments
;) )
I started with PE tool -- nothing surprising with the results we got here. PE
limits us to write only workload which is important but does not reveal the
true gain of this in-memory feature. For this reason I plan to start using the
YCSB tool, and as a first step to verify I can re-produce the results we got
from PE for 100% writes. The only difference is that YCSB does not have an easy
way to set WAL off, so I expect the absolute numbers to be different but the
trends to remain the same.
bq. We avoid copying for the data case? Is that unreal?
What I meant (and obviously wasn't clear enough) is that I turn off MSLAB in
the *configuration* not by changing the code to avoid MSLABs.
When MSLAB is on segments use chunks and whenever creating a new segment while
executing data compaction the retained cells are being copied to the new chunks
in the new segments, so that the GC can reclaim the old chunks.
When MSLAB is off, segments do not use chunks and therefore don't need to copy
the data to new chunks; retained cells are being pointed by the flat index
(cell array map) and the rest are collected by the GC.
bq. Now what happens is we will avoid frequent merges (by upping the #segments
for merge) so reduce merge cost but that will have an impact on flush size.
[~anoop.hbase] I agree with what you say, a bit more patient, the solution you
are after can be captured as one of the policies. We'll get to that after we
complete the next evaluation round.
> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
> Key: HBASE-16417
> URL: https://issues.apache.org/jira/browse/HBASE-16417
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
