[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16412807#comment-16412807
]
Eshcar Hillel commented on HBASE-20188:
---------------------------------------
Something to consider:
When I ran the experiments to determine the optimal default values for
in-memory compaction (HBASE-16417), I ran them with CAM setting and no mslab.
My findings for the optimal setting (for SSD) were a threshold of 2% for the
active segment (hbase.memstore.inmemoryflush.threshold.factor=0.02) and
pipeline length of 4 (hbase.hregion.compacting.pipeline.segments.limit=4), and
3 for hdd.
*However*, the current settings are different:
1) using mslab (which means CCM setting and not CAM)
2) the active segment threshold was changed and is now
hbase.memstore.inmemoryflush.threshold.factor=0.1
But the pipeline length is still set to 4.
This doesn't make much sense. If we flush "only" after aggregating 10% of the
data in the active segment we can use a pipeline of length 1, this would be
equivalent to a longer pipeline (of length 4) with smaller segments (consisting
of 2% of the data).
I hope what I am saying here makes sense.
Bottom line, given that the threshold was changed to 10%, I would recommend
changing the pipeline length to 1. This should mainly have a positive affect on
read latency. And regardless, presuming mslab is default (which is not what we
recommend) may require an additional round of parameter tuning.
> [TESTING] Performance
> ---------------------
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
> Reporter: stack
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: ITBLL2.5B_1.2.7vs2.0.0_cpu.png,
> ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png,
> ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png,
> ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png,
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png,
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png,
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
