Chris Egerton created KAFKA-17115:
-------------------------------------
Summary: Closing newly-created consumers during rebalance can
cause rebalances to hang
Key: KAFKA-17115
URL: https://issues.apache.org/jira/browse/KAFKA-17115
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 3.9.0
Reporter: Chris Egerton
Assignee: Chris Egerton
When a dynamic consumer (i.e., one with no group instance ID configured) first
tries to join a group, the group coordinator normally responds with the
MEMBER_ID_REQUIRED error, under the assumption that the member will retry soon
after. During this step, the group coordinator will also generate a new member
ID for the consumer, include it in the error response for the initial join
group request, and expect that a member with that ID will participate in future
rebalances.
If a consumer is closed in between the time that it sends the JoinGroup request
and the time that it receives the response from the group coordinator, it will
not attempt to leave the group, since it doesn't have a member ID to include in
that request.
This will cause future rebalances to hang, since the group coordinator will
still expect a member with the ID for the now-closed consumer to join.
Eventually, the group coordinator may remove the closed consumer from the
group, but with default configuration settings, this could take as long as five
minutes.
One possible fix is to send a LeaveGroup response with the member ID if the
consumer receives a JoinGroup response with a member ID after it has been
closed.
This applies to the legacy consumer; I have not verified yet with the new async
consumer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
