Sean Quah created KAFKA-21584:
---------------------------------
Summary: ConsumerGroupHeartbeat returns member epoch 0 after
upgrading a generation-0 classic group when the assignor is offloaded
Key: KAFKA-21584
URL: https://issues.apache.org/jira/browse/KAFKA-21584
Project: Kafka
Issue Type: Sub-task
Components: group-coordinator
Reporter: Sean Quah
Assignee: Sean Quah
When a classic group is upgraded to a consumer group before completing its
first rebalance, the upgraded group's target assignment epoch is 0, because the
members hold no assignment yet. When the assignor is offloaded, the
ConsumerGroupHeartbeat that triggers the upgrade schedules a new target
assignment and replies using the current target assignment. The member is
reconciled to target assignment epoch 0, and the response returns member epoch
0. Epoch 0 is reserved for members joining a group, so the coordinator must
never return it.
This happens for every upgrade at generation 0, whether the upgrade is
triggered by a new member or by static member replacement. Without offload,
only static member replacement is affected (KAFKA-21581), because a new
member's join computes the new target assignment synchronously.
KAFKA-21581 and KAFKA-21582 do not cover this. Their fix bumps the group epoch
when a rebalancing classic group is upgraded, so that a new target assignment
is computed. Without offload, the new target assignment is computed in the same
heartbeat, and the member is reconciled to it at epoch 1. With offload, the new
target assignment is computed later, and until it is installed the member can
only be reconciled to the existing target assignment. At generation 0, that
target assignment has epoch 0.
*Steps to reproduce*
# Set {{group.consumer.migration.policy}} to {{upgrade}} and
{{group.consumer.assignor.offload.enable}} to {{true}}. Set
{{group.initial.rebalance.delay.ms}} high enough that the classic group's
initial rebalance doesn't complete during the test.
# A classic member joins a new group G. The group goes to PreparingRebalance at
generation 0, and the JoinGroup stays pending.
# Before the initial rebalance completes, a consumer-protocol member sends
ConsumerGroupHeartbeat to G with member epoch 0.
# The coordinator upgrades G to a consumer group with target assignment epoch
0. The heartbeat bumps the group epoch and offloads the new target assignment.
# The heartbeat replies with the current target assignment, at epoch 0.
# The response returns member epoch 0.
*Expected:* the member gets an epoch greater than 0. New consumer groups avoid
this by starting at target assignment epoch 1 with an empty assignment
(KAFKA-20268). An upgraded generation-0 group should start the same way.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)