[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424241#comment-15424241
]
ramkrishna.s.vasudevan commented on HBASE-14921:
------------------------------------------------
I did a test with the latest patch - V10 version.
With 40G heap space and the below PE configs,
{code}
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50
--size=150 --columns=50 --valueSize=200 --writeToWAL=false --bloomFilter=NONE
--inmemoryCompaction=false/true randomWrite 50
{code}
Am still not able to run with InMemoryCompaction enabled and we get a FullGC.
With default memstore am able to run.
Good news is that in that 2 min that the sytem ran there was no negative flush
size happening. So it may be we are good here.
The reason should be because,
>From the code I think though flattening is enabled, we see if the number of
>segments is > 3 and so we do compaction. I think it goes under that logic.
>Speculative scan is default false.
Since compaction is happening I think we have duplicated segments till the new
one is updated in the pipeline. This is what I was saying in the last test
also.
You said you were able to run with YCSB. With YCSB what were the number of cols
and the data size?
As said I won't block the check in - we need to identify the reason for the
Full GC and add some policy or do the all segment flush thing that we had done
based on the policy. Lets discuss that in other JIRA.
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
> HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch,
> HBASE-14921-V06-CAO.patch, HBASE-14921-V08-CAO.patch,
> HBASE-14921-V09-CAO.patch, HBASE-14921-V10-CAO.patch,
> HBASE-14921-V11-CAO.patch, InitialCellArrayMapEvaluation.pdf,
> IntroductiontoNewFlatandCompactMemStore.pdf, MemStoreSizes.pdf,
> MemstoreItrCountissue.patch, NewCompactingMemStoreFlow.pptx
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
