Damian Guy updated KAFKA-3438:
    Fix Version/s:     (was: 1.1.0)

> Rack Aware Replica Reassignment should warn of overloaded brokers
> -----------------------------------------------------------------
>                 Key: KAFKA-3438
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3438
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions:
>            Reporter: Ben Stopford
>            Assignee: Vahid Hashemian
>            Priority: Major
>             Fix For: 2.0.0
> We've changed the replica reassignment code to be rack aware.
> One problem that might catch users out would be that they rebalance the 
> cluster using kafka-reassign-partitions.sh but their rack configuration means 
> that some high proportion of replicas are pushed onto a single, or small 
> number of, brokers. 
> This should be an easy problem to avoid, by changing the rack assignment 
> information, but we should probably warn users if they are going to create 
> something that is unbalanced. 
> So imagine I have a Kafka cluster of 12 nodes spread over two racks with rack 
> awareness enabled. If I add a 13th machine, on a new rack, and run the 
> rebalance tool, that new machine will get ~6x as many replicas as the least 
> loaded broker. 
> Suggest a warning  be added to the tool output when --generate is called. 
> "The most loaded broker has 2.3x as many replicas as the the least loaded 
> broker. This is likely due to an uneven distribution of brokers across racks. 
> You're advised to alter the rack config so there are approximately the same 
> number of brokers per rack" and displays the individual rack→#brokers and 
> broker→#replicas data for the proposed move.  

This message was sent by Atlassian JIRA

Reply via email to