[
https://issues.apache.org/jira/browse/KAFKA-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16548341#comment-16548341
]
ASF GitHub Bot commented on KAFKA-7144:
---------------------------------------
bbejeck opened a new pull request #5390: KAFKA-7144: Fix task assignment to be
even
URL: https://github.com/apache/kafka/pull/5390
For this PR the following has been done:
1. Sort all tasks before attempting to do an assignment.
2. Remove the previous two loops (assign tasks to clients previously
assigned as active and assign standby tasks to clients already assigned.
3. Now all assignments occur in one loop checking first for reassigning
previous active tasks, then previous standby tasks, then finally assign the
task if the last two checks don't perform any assignment.
The net effect of this change is we are now favoring load balancing over
stickiness from previously assigned tasks.
Updated current tests and ran locally
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Kafka Streams doesn't properly balance partition assignment
> -----------------------------------------------------------
>
> Key: KAFKA-7144
> URL: https://issues.apache.org/jira/browse/KAFKA-7144
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 1.1.0
> Reporter: James Cheng
> Priority: Major
> Attachments: OneThenTwelve.java
>
>
> Kafka Streams doesn't always spread the tasks across all available
> instances/threads
> I have a topology which consumes a single partition topic and goes .through()
> a 12 partition topic. The makes 13 partitions.
>
> I then started 2 instances of the application. I would have expected the 13
> partitions to be split across the 2 instances roughly evenly (7 partitions on
> one, 6 partitions on the other).
> Instead, one instance gets 12 partitions, and the other instance gets 1
> partition.
>
> Repro case attached. I ran it a couple times, and it was fairly repeatable.
> Setup for the repro:
> {code:java}
> $ ./bin/kafka-topics.sh --zookeeper localhost --create --topic one
> --partitions 1 --replication-factor 1
> $ ./bin/kafka-topics.sh --zookeeper localhost --create --topic twelve
> --partitions 12 --replication-factor 1
> $ echo foo | kafkacat -P -b 127.0.0.1 -t one
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
