ramesh-muthusamy opened a new pull request #9319: URL: https://github.com/apache/kafka/pull/9319
Allow even distribution of lost/new tasks when more than one worker joins the group at the same time Issue description: Existing issue 1 description : When more than one worker joins the consumer group the incremental co operative assignor revokes and re assigns atmost average number of tasks per worker. Issue: This results in the additional workers joining the group stay idle and would require more future rebalances to happen to have even distribution of tasks. Fix: As part of task assignment calculation following a deployment, the reassignment of tasks are calculated by revoking all the tasks above ceil(average) number of tasks. Existing issue 2 description: When more than one worker is lost and rejoins the group at most one worker will be re assigned with the lost tasks from all the workers that left the group. Issue: In scenarios where more than one worker is lost and rejoins the group only one among them gets assigned all the partitions that were lost in the past. The additional workers that have joined would not get any task assigned to them until a rebalance that happens in future. Fix: As part fo lost task re assignment all the new workers that have joined the group would be considered for task assignment and would be assigned in a round robin fashion with the new tasks. Testing strategy : System testing in a Kube environment completed. UT : updated to UT ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
