[
https://issues.apache.org/jira/browse/HBASE-12609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233017#comment-14233017
]
cuijianwei commented on HBASE-12609:
------------------------------------
[~apurtell], I agree that a statically configured value is not enough to
control the replication bandwidth accurately. If we do replication from a big
cluster (contains hundreds of region servers) to a small cluster(contains less
than ten region servers), the regions of table need to be replicated may locate
in different number of region servers(might locate in only one region server or
all the region servers), it is hard to define a statically bandwidth throttling
value for region server to control the whole bandwidth used by replication
between cluster. I think we may need global control and more dedicate
algorithm.
> Consider bandwidth throttling in
> ReplicationSource.readAllEntriesToReplicateOrNextFile
> --------------------------------------------------------------------------------------
>
> Key: HBASE-12609
> URL: https://issues.apache.org/jira/browse/HBASE-12609
> Project: HBase
> Issue Type: Improvement
> Components: Replication
> Affects Versions: 0.98.8
> Reporter: cuijianwei
> Priority: Minor
>
> The push size in ReplicationSource.shipEdits is controlled by
> 'replicationQueueSizeCapacity' and 'replicationQueueNbCapacity'. When the
> push size is much bigger than throttler's bandwidth(replication throttler is
> enabled), ReplicationSource will have a outgoing bandwidth peak followed by a
> period of sleep controlled by the throttler. How about considering bandwidth
> throttling in ReplicationSource.readAllEntriesToReplicateOrNextFile? such as
> the following code:
> {code}
> // Stop if too many entries or too big
> long bandwidth =
> this.conf.getLong("replication.source.per.peer.node.bandwidth", 0);
> if ((bandwidth > 0 && currentSize > bandwidth) || // ==> also consider
> bandwidth throttling
> currentSize >= this.replicationQueueSizeCapacity ||
> entries.size() >= this.replicationQueueNbCapacity) {
> break;
> }
> {code}
> Then, ReplicationSource will have more steady outgoing bandwidth.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
