[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339264#comment-15339264
]
Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------
Hey!
I am happy you are taking a punctilious look on the results :)
Generally, we didn't intend to do a stress or a real system test, at least not
now.
We plan to show the *relative* behavior between different MemStore variants.
>From here, we test on a single Region Server and the limit on the MemStore is
>not so much relevant.
Let's say the limit on the MemStore size is X, we are interested to see how the
space of X is used by Default/Skip-List/CellArayMap.
What happens if we flatten without compaction? How much frequent compaction
"cost" us?
This is what is interesting for now. Later you can replace X by any number.
bq. 1 GB heap for RS seems very minimal. How you arrived at only 30% being
given as Memstore size where 40% is default?
I agree that this is not enough, but we want to keep the MemStore limit small,
in order to see the flush to disk (of Default) earlier.
Otherwise you just need to wait longer before some interesting activity starts.
But I can put Memstore size to be 40% and rerun the tests.
bq. You wanted to make use of MSLAB chunk pool. But in the listed configs am
not able to see "hbase.hregion.memstore.chunkpool.maxsize". This defaults to 0
and so no pooling of MALAB chunks !!
Actually, in the resent patch we removed the CellChunkMap, so we do not care
about MemStoreChunkPool, as far as all implementations behave the same with
allocations.
CellArrayMap doesn't care where everything is allocated, but we use chunks, but
without pooling.
Again, if you care, and if already rerunning the tests, I'll set the chunk
pooling on.
bq. You wanted this lower size to be 10% ie. 0.1? 1.0 is an invalid value for
this config
First, thank you for paying my attention! I took a closer look and found that
global.memstore.lowerLimit is depricated and I need to use
global.memstore.lower.limit, will fix that.
Second, after fixing the name, it looks like 100% is a valid value.
Copy-pasting below from hbase-default.xml about
hbase.regionserver.global.memstore.size.lower.limit:
"A 100% value for this value causes the minimum possible flushing to occur when
updates are blocked due to memstore limiting."
Anyway, will soon publish the newer results :)
Thanks,
Anastasia
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA.patch,
> InitialCellArrayMapEvaluation.pdf, IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
