[
https://issues.apache.org/jira/browse/KAFKA-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553811#comment-17553811
]
Guozhang Wang commented on KAFKA-12639:
---------------------------------------
This is a separate issue, as `joinGroupIfNeeded` is not called by the heartbeat
thread at all.
I checked the code in trunk and compared with 2.7, I can confirm the situation
still persists there. But this may not be a real issue, just in
[~matiss.gutmanis]'s testing environment.
The key is that inside the while loop we only check `while
(rejoinNeededOrPending()) {` and is only existing due to timed out if the
response cannot be received in time. This is relied on the fact that if the
timer has already elapsed, after sending the join-group request we should be
effectively triggering `select(0)` in which we should rarely receive the
response immediately, and hence at that time we would be exiting out of the
while loop. I suspect in this testing the cluster failure is shortcut-mocked by
returning immediately the response (again, which should not be a common case in
practice) which caused the calling loop to never exit.
All that being said, we can still augment the condition as `while
(rejoinNeededOrPending() && timer.notElapsed()) {`, but doing so has a
potential caveat that we would not send out the request before exiting the
while loop --- and this is also the intention of the current code to always
make sure we have the inflight request before exiting, to optimize the
pipelining latency slightly --- cc [~pnee] to chime in here.
> AbstractCoordinator ignores backoff timeout when joining the consumer group
> ---------------------------------------------------------------------------
>
> Key: KAFKA-12639
> URL: https://issues.apache.org/jira/browse/KAFKA-12639
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 2.7.0
> Reporter: Matiss Gutmanis
> Assignee: Philip Nee
> Priority: Major
>
> We observed heavy logging while trying to join consumer group during partial
> unavailability of Kafka cluster (it's part of our testing process). Seems
> that {{rebalanceConfig.retryBackoffMs}} used in {{
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator#joinGroupIfNeeded}}
> is not respected. Debugging revealed that {{Timer}} instance technically is
> expired thus using sleep of 0 milliseconds which defeats the purpose of
> backoff timeout.
> Minimal backoff timeout should be respected.
>
> {code:java}
> 2021-03-30 08:30:24,488 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] JoinGroup failed: Coordinator
> 127.0.0.1:9092 (id: 2147483634 rack: null) is loading the group.
> 2021-03-30 08:30:24,488 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] Rebalance failed.
> org.apache.kafka.common.errors.CoordinatorLoadInProgressException: The
> coordinator is loading and hence can't process requests.
> 2021-03-30 08:30:24,488 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] (Re-)joining group
> 2021-03-30 08:30:24,489 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] JoinGroup failed: Coordinator
> 127.0.0.1:9092 (id: 2147483634 rack: null) is loading the group.
> 2021-03-30 08:30:24,489 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] Rebalance failed.
> org.apache.kafka.common.errors.CoordinatorLoadInProgressException: The
> coordinator is loading and hence can't process requests.
> 2021-03-30 08:30:24,489 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] (Re-)joining group
> 2021-03-30 08:30:24,490 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] JoinGroup failed: Coordinator
> 127.0.0.1:9092 (id: 2147483634 rack: null) is loading the group.
> 2021-03-30 08:30:24,490 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] Rebalance failed.
> org.apache.kafka.common.errors.CoordinatorLoadInProgressException: The
> coordinator is loading and hence can't process requests.
> 2021-03-30 08:30:24,490 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] (Re-)joining group
> 2021-03-30 08:30:24,491 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] JoinGroup failed: Coordinator
> 127.0.0.1:9092 (id: 2147483634 rack: null) is loading the group.
> 2021-03-30 08:30:24,491 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] Rebalance failed.
> org.apache.kafka.common.errors.CoordinatorLoadInProgressException: The
> coordinator is loading and hence can't process requests.
> 2021-03-30 08:30:24,491 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] (Re-)joining group
> 2021-03-30 08:30:24,492 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] JoinGroup failed: Coordinator
> 127.0.0.1:9092 (id: 2147483634 rack: null) is loading the group.
> 2021-03-30 08:30:24,492 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] Rebalance failed.
> org.apache.kafka.common.errors.CoordinatorLoadInProgressException: The
> coordinator is loading and hence can't process requests.
> 2021-03-30 08:30:24,492 INFO
> [fs2-kafka-consumer-41][o.a.k.c.c.i.AbstractCoordinator] [Consumer
> clientId=app_clientid, groupId=consumer-group] (Re-)joining group
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
