Sean Quah created KAFKA-21584:
---------------------------------

             Summary: ConsumerGroupHeartbeat returns member epoch 0 after 
upgrading a generation-0 classic group when the assignor is offloaded
                 Key: KAFKA-21584
                 URL: https://issues.apache.org/jira/browse/KAFKA-21584
             Project: Kafka
          Issue Type: Sub-task
          Components: group-coordinator
            Reporter: Sean Quah
            Assignee: Sean Quah


When a classic group is upgraded to a consumer group before completing its 
first rebalance, the upgraded group's target assignment epoch is 0, because the 
members hold no assignment yet. When the assignor is offloaded, the 
ConsumerGroupHeartbeat that triggers the upgrade schedules a new target 
assignment and replies using the current target assignment. The member is 
reconciled to target assignment epoch 0, and the response returns member epoch 
0. Epoch 0 is reserved for members joining a group, so the coordinator must 
never return it.

This happens for every upgrade at generation 0, whether the upgrade is 
triggered by a new member or by static member replacement. Without offload, 
only static member replacement is affected (KAFKA-21581), because a new 
member's join computes the new target assignment synchronously.

KAFKA-21581 and KAFKA-21582 do not cover this. Their fix bumps the group epoch 
when a rebalancing classic group is upgraded, so that a new target assignment 
is computed. Without offload, the new target assignment is computed in the same 
heartbeat, and the member is reconciled to it at epoch 1. With offload, the new 
target assignment is computed later, and until it is installed the member can 
only be reconciled to the existing target assignment. At generation 0, that 
target assignment has epoch 0.

*Steps to reproduce*
# Set {{group.consumer.migration.policy}} to {{upgrade}} and 
{{group.consumer.assignor.offload.enable}} to {{true}}. Set 
{{group.initial.rebalance.delay.ms}} high enough that the classic group's 
initial rebalance doesn't complete during the test.
# A classic member joins a new group G. The group goes to PreparingRebalance at 
generation 0, and the JoinGroup stays pending.
# Before the initial rebalance completes, a consumer-protocol member sends 
ConsumerGroupHeartbeat to G with member epoch 0.
# The coordinator upgrades G to a consumer group with target assignment epoch 
0. The heartbeat bumps the group epoch and offloads the new target assignment.
# The heartbeat replies with the current target assignment, at epoch 0.
# The response returns member epoch 0.

*Expected:* the member gets an epoch greater than 0. New consumer groups avoid 
this by starting at target assignment epoch 1 with an empty assignment 
(KAFKA-20268). An upgraded generation-0 group should start the same way.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to