David Jacot created KAFKA-21467:
-----------------------------------
Summary: Classic to consumer group conversion fails when no
protocol is selected or a member does not support it
Key: KAFKA-21467
URL: https://issues.apache.org/jira/browse/KAFKA-21467
Project: Kafka
Issue Type: Bug
Affects Versions: 4.0.0
Reporter: David Jacot
Assignee: David Jacot
When the first member using the consumer rebalance protocol joins a classic
group, the group coordinator converts the group to a consumer group in
{{ConsumerGroup.fromClassicGroup}}. The conversion parses the subscription of
every classic member with the selected protocol of the classic group. This
fails in two states that the classic group can be in while it rebalances:
* No protocol is selected yet, for instance because the first rebalance of the
group has not completed. {{classicGroup.protocolName().get()}} throws
{{NoSuchElementException}}.
* A member that does not support the selected protocol has joined the
rebalance. The next generation would select another protocol, but the member
has no metadata for the current one, so {{ClassicGroupMember.metadata}} throws
{{IllegalArgumentException}}.
In both cases, the ConsumerGroupHeartbeat request that triggers the conversion
fails with UNKNOWN_SERVER_ERROR. The consumer treats this error as fatal, so
the upgraded consumer fails instead of joining the group. Every upgraded
consumer that joins fails the same way until the classic rebalance completes.
Steps to reproduce the first state, with
group.consumer.migration.policy=upgrade:
# A classic consumer joins the group and waits for the initial rebalance to
complete (group.initial.rebalance.delay.ms).
# Before it completes, a consumer with group.protocol=consumer joins the same
group.
# The heartbeat of the second consumer fails with UNKNOWN_SERVER_ERROR.
Steps to reproduce the second state:
# A classic consumer that supports the range and cooperative-sticky assignors
joins the group, which becomes stable with range.
# A classic consumer that only supports cooperative-sticky joins the group,
which starts a rebalance.
# Before the rebalance completes, a consumer with group.protocol=consumer joins
the same group. Its heartbeat fails with UNKNOWN_SERVER_ERROR.
The fix converts the members with the selected protocol when every member
supports it, and otherwise with the protocol that the next generation would
select.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)