[
https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123956#comment-15123956
]
Jiangjie Qin commented on KAFKA-1464:
-------------------------------------
It looks our purpose is to minimize user impact during replica catching up.
From broker point of view, as long as client request latency is acceptable we
should fully utilize the bandwidth we have to let replicas keep up. We should
be able to measure the user experience by checking Queuing time of requests
from and response to clients.
If that is the case, maybe we can let user set an SLA for latency. And we will
not throttle replication as long as the user ProduceRequest / FetchRequest
queuing time. Otherwise, we will throttle the fetching from out of sync replica
(We probably don't want to throttle in-sync replicas).
> Add a throttling option to the Kafka replication tool
> -----------------------------------------------------
>
> Key: KAFKA-1464
> URL: https://issues.apache.org/jira/browse/KAFKA-1464
> Project: Kafka
> Issue Type: New Feature
> Components: replication
> Affects Versions: 0.8.0
> Reporter: mjuarez
> Assignee: Ismael Juma
> Priority: Minor
> Labels: replication, replication-tools
> Fix For: 0.9.1.0
>
>
> When performing replication on new nodes of a Kafka cluster, the replication
> process will use all available resources to replicate as fast as possible.
> This causes performance issues (mostly disk IO and sometimes network
> bandwidth) when doing this in a production environment, in which you're
> trying to serve downstream applications, at the same time you're performing
> maintenance on the Kafka cluster.
> An option to throttle the replication to a specific rate (in either MB/s or
> activities/second) would help production systems to better handle maintenance
> tasks while still serving downstream applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
