[
https://issues.apache.org/jira/browse/KAFKA-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358216#comment-16358216
]
Andy Coates commented on KAFKA-3438:
------------------------------------
Morning.
Any reason why you’re email thing to me?
On 8 Feb 2018, at 14:22, Damian Guy (JIRA) <j...@apache.org> wrote:
[
https://issues.apache.org/jira/browse/KAFKA-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damian Guy updated KAFKA-3438:
------------------------------
Fix Version/s: (was: 1.1.0)
2.0.0
Rack Aware Replica Reassignment should warn of overloaded brokers
-----------------------------------------------------------------
Key: KAFKA-3438
URL: https://issues.apache.org/jira/browse/KAFKA-3438
Project: Kafka
Issue Type: Improvement
Affects Versions: 0.10.0.0
Reporter: Ben Stopford
Assignee: Vahid Hashemian
Priority: Major
Fix For: 2.0.0
We've changed the replica reassignment code to be rack aware.
One problem that might catch users out would be that they rebalance the cluster
using kafka-reassign-partitions.sh but their rack configuration means that some
high proportion of replicas are pushed onto a single, or small number of,
brokers.
This should be an easy problem to avoid, by changing the rack assignment
information, but we should probably warn users if they are going to create
something that is unbalanced.
So imagine I have a Kafka cluster of 12 nodes spread over two racks with rack
awareness enabled. If I add a 13th machine, on a new rack, and run the
rebalance tool, that new machine will get ~6x as many replicas as the least
loaded broker.
Suggest a warning be added to the tool output when --generate is called. "The
most loaded broker has 2.3x as many replicas as the the least loaded broker.
This is likely due to an uneven distribution of brokers across racks. You're
advised to alter the rack config so there are approximately the same number of
brokers per rack" and displays the individual rack→#brokers and
broker→#replicas data for the proposed move.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
> Rack Aware Replica Reassignment should warn of overloaded brokers
> -----------------------------------------------------------------
>
> Key: KAFKA-3438
> URL: https://issues.apache.org/jira/browse/KAFKA-3438
> Project: Kafka
> Issue Type: Improvement
> Affects Versions: 0.10.0.0
> Reporter: Ben Stopford
> Assignee: Vahid Hashemian
> Priority: Major
> Fix For: 2.0.0
>
>
> We've changed the replica reassignment code to be rack aware.
> One problem that might catch users out would be that they rebalance the
> cluster using kafka-reassign-partitions.sh but their rack configuration means
> that some high proportion of replicas are pushed onto a single, or small
> number of, brokers.
> This should be an easy problem to avoid, by changing the rack assignment
> information, but we should probably warn users if they are going to create
> something that is unbalanced.
> So imagine I have a Kafka cluster of 12 nodes spread over two racks with rack
> awareness enabled. If I add a 13th machine, on a new rack, and run the
> rebalance tool, that new machine will get ~6x as many replicas as the least
> loaded broker.
> Suggest a warning be added to the tool output when --generate is called.
> "The most loaded broker has 2.3x as many replicas as the the least loaded
> broker. This is likely due to an uneven distribution of brokers across racks.
> You're advised to alter the rack config so there are approximately the same
> number of brokers per rack" and displays the individual rack→#brokers and
> broker→#replicas data for the proposed move.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
