[
https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098476#comment-15098476
]
Yu Li commented on HBASE-14969:
-------------------------------
[~tedyu] and [~Apache9],
Sorry for the lag, here are the testing results:
Test env (to simulate high flush pressure by ingesting 100GB data into a single
RS cluster):
* 1 single RegionServer, 4 datanodes
* HBase version: commit c8b9754a5e, w/o and w/ patch here
* Test command: {{bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
--nomapred --presplit=16 --rows=10485760 randomWrite 10}}
Overall time (Average of 10 threads), milliseconds:
* no-throttling: 2909600
* throttling: 2876955
Latency (Average of 10 threads), microseconds:
||Flush-mode||Max||99.999th||99.9th||75th||50th||Min||Avg||StdDev
|no-throttling|1818356.5|1124412.62|21.8999|3.0|2.2|2.0|273.25|11455.52|
|throttling|1467822.8|1084914.03|22.0998|3.0|2.0|2.0|273.12|11590.58|
Notice that we've observed much higher disk utils causing bigger spike in our
online cluster during flush than the testing env, so I think this feature will
be more useful on real environment.
> Add throughput controller for flush
> -----------------------------------
>
> Key: HBASE-14969
> URL: https://issues.apache.org/jira/browse/HBASE-14969
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Yu Li
> Assignee: Yu Li
> Attachments: HBASE-14969.patch, HBASE-14969_v2.patch
>
>
> In HBASE-8329 we added a throughput controller for compaction, to avoid spike
> caused by huge IO pressure like network/disk overflow. However, even with
> this control on, we are still observing disk utils near 100%, and by analysis
> we think this is caused by flush, especially when we increase the setting of
> {{hbase.hstore.flusher.count}}
> In this JIRA, we propose to add throughput control feature for flush, as a
> supplement of HBASE-8329 to better control IO pressure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
