[
https://issues.apache.org/jira/browse/HBASE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14251252#comment-14251252
]
zhangduo commented on HBASE-8329:
---------------------------------
I met the problem recently. Sometimes "Compaction progress" report the rate is
about 40MB/sec，so if we have two compaction in progess and other operation
together then we can easily exceed a 1000Mbit/sec network card's max bandwith,
and the regionserver will be unresponsive.
I guess the reason is that we have some large rows(several hundreds KBs to
several MBs), so the compaction speed can easily burst high. I can do a test to
examine how does row size effect the compaction speed. And our cluster is
small(7 regionserver. A machine can have 30TB disk capacity today, and IT
department calculate the number of regionservers by disk capacity, not by
memory and network:( ) so the total bandwith is small.
I apply the patch to 0.98.9RC1 to limit the compaction speed. It works for now
but I don't think it should be the final solution. At least to we need to
calculate the limitation automatically I think.
> Limit compaction speed
> ----------------------
>
> Key: HBASE-8329
> URL: https://issues.apache.org/jira/browse/HBASE-8329
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
> Reporter: binlijin
> Assignee: Sergey Shelukhin
> Fix For: 2.0.0, 1.1.0
>
> Attachments: HBASE-8329-10.patch, HBASE-8329-11.patch,
> HBASE-8329-12.patch, HBASE-8329-2-trunk.patch, HBASE-8329-3-trunk.patch,
> HBASE-8329-4-trunk.patch, HBASE-8329-5-trunk.patch, HBASE-8329-6-trunk.patch,
> HBASE-8329-7-trunk.patch, HBASE-8329-8-trunk.patch, HBASE-8329-9-trunk.patch,
> HBASE-8329-trunk.patch
>
>
> There is no speed or resource limit for compaction，I think we should add this
> feature especially when request burst.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
