[
https://issues.apache.org/jira/browse/KAFKA-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-1677:
---------------------------------
Assignee: Geoffrey Anderson (was: Neha Narkhede)
> Governor on concurrent replica reassignments
> --------------------------------------------
>
> Key: KAFKA-1677
> URL: https://issues.apache.org/jira/browse/KAFKA-1677
> Project: Kafka
> Issue Type: New Feature
> Components: controller
> Reporter: Ryan Berdeen
> Assignee: Geoffrey Anderson
> Labels: newbie++
>
> We have seen a cluster be killed via too many concurrent partition transfers.
> An ideal solution is a configuration setting to limit the number of
> concurrent transfers per host (dynamically tunable). (eg: transfer_limit
> defined in
> http://docs.basho.com/riak/latest/ops/advanced/configs/configuration-files/#Ring).
> To work around this, we generate our assignments, then use a tool to feed the
> reassignments in small batches.
> The size of the batch is based on either
> * *the number partitions*, e.g., reassign all replicas for the first 2
> partitions that have any moves
> * *the number of individual replica moves*, e.g. when reassigning \[1,2,3,4]
> to \[5,6,7,8], first reassign to \[5,6,3,4] then reassign to \[5,6,7,8]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
