[ 
https://issues.apache.org/jira/browse/KAFKA-21468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Jacot resolved KAFKA-21468.
---------------------------------
    Fix Version/s: 4.1.3
                   4.0.3
                   4.5.0
                   4.4.1
                   4.3.3
                   4.2.3
       Resolution: Fixed

> SyncGroup returns COORDINATOR_LOAD_IN_PROGRESS, which classic clients do not 
> handle
> -----------------------------------------------------------------------------------
>
>                 Key: KAFKA-21468
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21468
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 4.0.0
>            Reporter: David Jacot
>            Assignee: David Jacot
>            Priority: Major
>             Fix For: 4.1.3, 4.0.3, 4.5.0, 4.4.1, 4.3.3, 4.2.3
>
>
> When a SyncGroup request reaches the group coordinator while its 
> __consumer_offsets partition is loading, 
> {{GroupCoordinatorService#syncGroup}} answers with 
> COORDINATOR_LOAD_IN_PROGRESS. Classic clients do not expect this error in a 
> SyncGroup response. {{AbstractCoordinator.SyncGroupResponseHandler}} has no 
> branch for it and raises a non-retriable {{KafkaException("Unexpected error 
> from SyncGroup: The coordinator is loading and hence can't process 
> requests.")}}. As a result:
> * {{KafkaConsumer#poll}} of a classic consumer throws instead of rejoining 
> the group.
> * A Connect worker exits. The exception escapes {{DistributedHerder#tick}} 
> and reaches the catch in {{DistributedHerder#run}}, which logs "Uncaught 
> exception in herder work thread, exiting" and calls {{Exit.exit(1)}}.
> The old group coordinator answered REBALANCE_IN_PROGRESS in this case. Its 
> comment said: "By returning rebalance in progress, the consumer will attempt 
> to rejoin without needing to rediscover the coordinator. Note that we cannot 
> return COORDINATOR_LOAD_IN_PROGRESS since older clients do not expect the 
> error." The new coordinator never added that mapping. It already maps the 
> same error to NONE for Heartbeat.
> The window opens when a member has received its JoinGroup response and the 
> coordinator partition then moves away and back to the same broker, which 
> reloads it before the member's SyncGroup arrives. The rebalance state is lost 
> on reload, so the member has to rejoin anyway. Consumers using the consumer 
> rebalance protocol are not affected.
> The fix maps COORDINATOR_LOAD_IN_PROGRESS to REBALANCE_IN_PROGRESS in 
> {{GroupCoordinatorService#syncGroup}}. The member then rejoins, and its 
> JoinGroup retries the load error until the load completes, which the client 
> already handles.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to