[
https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15725746#comment-15725746
]
Eshcar Hillel commented on HBASE-16417:
---------------------------------------
Thanks [~ram_krish].
AFAIK 2.0 is not released yet.
I advocate to experiment with the final default configuration before releasing
it, and comparing it to the previous default configuration -- make sure
performance remains the same or improves in *all* common workloads, including
mixed workload.
If what is planned for 2.0 is 40% memstore, 40% BC, chunk pool on by default,
and we assume G1GC will be used by many application, then yes definitely I
advocate to revert.
When running 3 node cluster the system pays much more for each compaction,
since now network is also involved in writing new files. Having less
compactions/writing less MB to HDFS with eager (and also with basic) means
avoiding part of this cost. This is emphasized with SSDs, since just writing
data to disk doesn't cost much. But once you also need to pay for network
traffic the advantage is more pronounced.
We are running zipfian distribution by YCSB. This is pretty much standard
distribution for KV-stores benchmarks, which generates duplication.
We do plan to play a bit with the alpha parameter to check performance under
less/more heavy-head distributions. A workload that accesses 10-20% of the keys
80-90% of the time is considered valid.
The amount of flushes depends on duplication ratio, and also on what the policy
decides to flush. Currently basic flushes the entire pipeline, eager only
flushes the tail. Other policies might decide differently.
> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
> Key: HBASE-16417
> URL: https://issues.apache.org/jira/browse/HBASE-16417
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-16417-benchmarkresults-20161101.pdf,
> HBASE-16417-benchmarkresults-20161110.pdf,
> HBASE-16417-benchmarkresults-20161123.pdf,
> HBASE-16417-benchmarkresults-20161205.pdf
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
