[
https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-3297:
---------------------------------
*Reminder to the contributor / reviewer of the PR*: please note that the code
deadline for 1.0.0 is less than 2 weeks away (Oct. 4th). Please re-evaluate
your JIRA and see if it still makes sense to be merged into 1.0.0 or it could
be pushed out to 1.1.0, or be closed directly if the JIRA itself is not valid
any more, or re-assign yourself as contributor / committer if you are no longer
working on the JIRA.
> More optimally balanced partition assignment strategy (new consumer)
> --------------------------------------------------------------------
>
> Key: KAFKA-3297
> URL: https://issues.apache.org/jira/browse/KAFKA-3297
> Project: Kafka
> Issue Type: Improvement
> Components: consumer
> Reporter: Andrew Olson
> Assignee: Andrew Olson
> Fix For: 1.0.0
>
>
> While the roundrobin partition assignment strategy is an improvement over the
> range strategy, when the consumer topic subscriptions are not identical
> (previously disallowed but will be possible as of KAFKA-2172) it can produce
> heavily skewed assignments. As suggested
> [here|https://issues.apache.org/jira/browse/KAFKA-2172?focusedCommentId=14530767&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14530767]
> it would be nice to have a strategy that attempts to assign an equal number
> of partitions to each consumer in a group, regardless of how similar their
> individual topic subscriptions are. We can accomplish this by tracking the
> number of partitions assigned to each consumer, and having the partition
> assignment loop assign each partition to a consumer interested in that topic
> with the least number of partitions assigned.
> Additionally, we can optimize the distribution fairness by adjusting the
> partition assignment order:
> * Topics with fewer consumers are assigned first.
> * In the event of a tie for least consumers, the topic with more partitions
> is assigned first.
> The general idea behind these two rules is to keep the most flexible
> assignment choices available as long as possible by starting with the most
> constrained partitions/consumers.
> This JIRA addresses the new consumer. For the original high-level consumer,
> see KAFKA-2435.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
