Anastasia Braginsky commented on HBASE-16608:

Hi [~anoop.hbase] and [~ram_krish],

I first want to thank you for your unstoppable interest and support for this 
big CompactingMemStore issue! You are getting everything very fast and see it 
all in the deep details. Really pleasure to work with such smart people as you 

So you are saying that you want not to use any merge in order to have the 
things simpler. Am I getting you right? I do not see how merges (especially if 
done once in a while) can affect GC. The size of CellArrayMap is negligible 
relative to the data size. It is reasonable to think that GC is busy with 
releasing the chunks that are not accessible after flush to disk.

When we hold more data in the memory either due to index or data compaction we 
flush bigger files to the disk and bigger chunks of memory are need to be freed 
upon each flush to disk. This sounds like a reason for making GC to work 
harder. If you see how merges directly affect garbage collection please 
explain. All this makes me think that your suggestion (not to merge) will 
result in the same GC behavior. We already had an implementation with 
flattening only, and there (under stress) we have seen tens of segments in the 
compaction pipeline. I wonder the performance of the gets and scans in this 
structure. The process of flushing multiple segments to disk should also 
prolong the flushing to disk, which is undesirable. When we had flushes waiting 
for merge Ram had seen lots of blocking writes till the system became 
unresponsive. So I do not clearly see why not-to-merge-at-all is a better 

I am not attached to anything, but really trying to understand why one way is 
better then the other. I mean can we measure the better quality of not merging 
relative to intermediate merging?

Regarding the issue of N-MSLABs merge, raised in the RB, it looks irrelevant 
now, till we decide whether to merge or not to merge. Thus let's discuss it 
later. I also believe this is not such a big issue and we can arrange it this 
way or another. 


> Introducing the ability to merge ImmutableSegments without copy-compaction or 
> SQM usage
> ---------------------------------------------------------------------------------------
>                 Key: HBASE-16608
>                 URL: https://issues.apache.org/jira/browse/HBASE-16608
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>         Attachments: HBASE-16417-V02.patch, HBASE-16417-V04.patch, 
> HBASE-16417-V06.patch, HBASE-16417-V07.patch, HBASE-16417-V08.patch, 
> HBASE-16417-V10.patch, HBASE-16608-V01.patch, HBASE-16608-V03.patch, 
> HBASE-16608-V04.patch, HBASE-16608-V08.patch

This message was sent by Atlassian JIRA

Reply via email to