dajac opened a new pull request, #23735:
URL: https://github.com/apache/kafka/pull/23735
Depends on #23734: only the last commit belongs to this PR.
uniform2 decides how many partitions of every topic each member gets in
three steps. The keep and hand out steps give every extra partition to
a distinct subscriber of its topic, but they can leave a member with an
assignment much larger than another subscriber of one of its topics.
This patch adds the balance step, `ShareBalancer`, and runs it after
the other two in `Shares.of`. It applies a single rule until it no
longer applies: the member with the largest assignment among those
having an extra partition that can move gives its cheapest one to the
smallest subscriber of the topics of its extra partitions that has no
extra partition of the topic and is at least two partitions smaller.
Every move lowers the sum of the squares of the sizes, so the step
ends, and it ends exactly when no extra partition can move to a
subscriber of its topic at least two partitions smaller.
Changes:
- The givers wait in a heap, the largest first, and a giver without a
receiver is parked until a move may give it one. A cohort sharing no
topic with another is done at its first giver without a receiver.
- On equal sizes, a giver with a free extra partition gives first, and
a receiver owning more than the base partitions of a topic with extra
partitions receives first, since only it can save a move.
- `Shares.of` takes the ways of counting of `ExtraPartitionMoves`, and
a test checks that they give the same shares on random groups.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
