[
https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15724919#comment-15724919
]
Eshcar Hillel commented on HBASE-16417:
---------------------------------------
Thanks for the comments [~anoop.hbase].
The scope of the current Jira is to explore performance of different in-memory
compaction policies, and therefore all other parameters are used with their
default/recommended values.
Finding the sweat spot for mslab and chunk pool usage vs the size allocated for
memstore and BC vs the parameters for the GC should be explored in a different
Jira; there are too many parameters to optimize in a single round and finding
the global maximum in terms of performance (total throughput, read and write
latency, etc.) may require multiple experiments.
IMHO, these must be explored before setting chunk pool to on by default in 2.0.
Changing some parameters without re-tuning other parameters can cause major
degradation in performance. But it is not the ticket of the current Jira.
In my experiments IHOP is set to 60%. I can change it to 45% or 50% for future
experiments but I don't think it will make much difference.
The policy that is named none refers to using default memstore, no compaction
whatsoever.
Batching size at client side: some applications don't use any batching.
Optimally to have a clean measure of the write latency the experiments should
use zero batching, but this would make the experiment infinitely long. In
striking the balance between valid measurements and reasonable run time having
buffer of size 10KB (which is approximately 100 put operations) was the optimal
point for me.
Scan memstore-only first optimization: this is not a correct solution yet since
it does not handle all the issues that you raised regarding TS etc. It is only
for demonstrating potential gain of the optimization. I can publish the patch
in a different Jira after we clean up some open Jira's we have here :).
> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
> Key: HBASE-16417
> URL: https://issues.apache.org/jira/browse/HBASE-16417
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-16417-benchmarkresults-20161101.pdf,
> HBASE-16417-benchmarkresults-20161110.pdf,
> HBASE-16417-benchmarkresults-20161123.pdf,
> HBASE-16417-benchmarkresults-20161205.pdf
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
