Hi Folks, I've created a KIP about an improvement of the reassignment algorithm we have. It aims to enable partition-wise incremental reassignment. The motivation for this is to avoid excess load that the current replication algorithm implicitly carries as in that case there are points in the algorithm where both the new and old replica set could be online and replicating which puts double (or almost double) pressure on the brokers which could cause problems. Instead my proposal would slice this up into several steps where each step is calculated based on the final target replicas and the current replica assignment taking into account scenarios where brokers could be offline and when there are not enough replicas to fulfil the min.insync.replica requirement.
The link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-435%3A+Incremental+Partition+Reassignment I'd be happy to receive any feedback. An important note is that this KIP and another one, KIP-236 that is about interruptible reassignment ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-236%3A+Interruptible+Partition+Reassignment) should be compatible. Thanks, Viktor
