geertbrit opened a new pull request, #3374:
URL: https://github.com/apache/iggy/pull/3374
## Summary
- Fix cooperative rebalancing to distribute partitions evenly across idle
members
- Add 6 new integration tests verifying balanced distribution in various
scenarios
## Problem
When multiple consumers joined a consumer group, the first idle member
received most excess partitions while others starved. Example: 16 partitions
held by 1 member + 15 new members → uneven distribution.
## Solution
Two-pass distribution approach:
1. **Pass 1:** Collect ALL excess partitions from over-assigned members
2. **Pass 2:** Distribute round-robin using `idle_slab_ids[i %
idle_slab_ids.len()]`
Example: 16 partitions held by 1 member, 15 idle members join
- Before: first idle member gets many, others get 0-1
- After: each of 16 members gets exactly 1 partition
## Test plan
- [x] All 95 existing consumer group partition assignment tests pass
- [x] 6 new tests added covering balanced distribution scenarios:
- 16 partitions → 16 consumers (exact benchmark scenario)
- 12 partitions → 4 concurrent joins
- 10 partitions → 4 members (uneven ratio with remainder)
- 16 partitions with 2 over-assigned → 8 total members
- 24 partitions → 8 members (large scale)
- Member churn (leave + rejoin)
Closes #3334
🤖 Generated with [Claude Code](https://claude.ai/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
