[
https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451211#comment-15451211
]
ramkrishna.s.vasudevan commented on HBASE-16417:
------------------------------------------------
I could not add some points here all these days. Sorry about that.
In your tests you were saying that the scanning of the segments was taking some
time and so you felt that should be avoided.
In our opinion I think not only the scanning part but the compaction part of
combining all segments into one is also consuming memory because till the older
ones in the pipeline are discarded you have two copies of the data. So this
requires you to ensure that the GC is tuned properly because the active working
set is double the required one atleast till the time the original copies are
removed. Have you seen this behaviour in your tests? I think that is one
reason why you were not able to run with PE with the config that we had
specified.
Making this scan simpler will definitely help but am just saying with memory
perspective and heavy GC.
> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
> Key: HBASE-16417
> URL: https://issues.apache.org/jira/browse/HBASE-16417
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Assignee: Anastasia Braginsky
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
