David Jacot created KAFKA-21467:
-----------------------------------

             Summary: Classic to consumer group conversion fails when no 
protocol is selected or a member does not support it
                 Key: KAFKA-21467
                 URL: https://issues.apache.org/jira/browse/KAFKA-21467
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 4.0.0
            Reporter: David Jacot
            Assignee: David Jacot


When the first member using the consumer rebalance protocol joins a classic 
group, the group coordinator converts the group to a consumer group in 
{{ConsumerGroup.fromClassicGroup}}. The conversion parses the subscription of 
every classic member with the selected protocol of the classic group. This 
fails in two states that the classic group can be in while it rebalances:

* No protocol is selected yet, for instance because the first rebalance of the 
group has not completed. {{classicGroup.protocolName().get()}} throws 
{{NoSuchElementException}}.
* A member that does not support the selected protocol has joined the 
rebalance. The next generation would select another protocol, but the member 
has no metadata for the current one, so {{ClassicGroupMember.metadata}} throws 
{{IllegalArgumentException}}.

In both cases, the ConsumerGroupHeartbeat request that triggers the conversion 
fails with UNKNOWN_SERVER_ERROR. The consumer treats this error as fatal, so 
the upgraded consumer fails instead of joining the group. Every upgraded 
consumer that joins fails the same way until the classic rebalance completes.

Steps to reproduce the first state, with 
group.consumer.migration.policy=upgrade:
# A classic consumer joins the group and waits for the initial rebalance to 
complete (group.initial.rebalance.delay.ms).
# Before it completes, a consumer with group.protocol=consumer joins the same 
group.
# The heartbeat of the second consumer fails with UNKNOWN_SERVER_ERROR.

Steps to reproduce the second state:
# A classic consumer that supports the range and cooperative-sticky assignors 
joins the group, which becomes stable with range.
# A classic consumer that only supports cooperative-sticky joins the group, 
which starts a rebalance.
# Before the rebalance completes, a consumer with group.protocol=consumer joins 
the same group. Its heartbeat fails with UNKNOWN_SERVER_ERROR.

The fix converts the members with the selected protocol when every member 
supports it, and otherwise with the protocol that the next generation would 
select.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to