Lianet Magrans created KAFKA-21574:
--------------------------------------
Summary: Async consumer can apply a HB response from before a
poll timer expiration leave + rejoin
Key: KAFKA-21574
URL: https://issues.apache.org/jira/browse/KAFKA-21574
Project: Kafka
Issue Type: Bug
Components: clients, consumer
Reporter: Lianet Magrans
When the poll timer expires the async consumer sends a leave HB (even if there
was one in-flight) and clears the in-flight flag. The earlier HB response will
still be received (AbstractMembershipManager#onHeartbeatSuccess), and that
logic already considers it has to drop responses that are not relevant anymore,
which is if the member is LEAVING, not in the group etc.
But if the situation above combines with the app calling poll again before that
response arrives, the member is already JOINING, so the old response would be
accepted.
This is probably hard to trigger in real life scenarios: needs poll timer
expiration while a heartbeat is in flight + the application to call poll before
the response to the first in-flight arrives, and (not sure but) I guess it
needs the response to include an assignment change for this to be harmful,
otherwise the impact it's just unneeded processing on the consumer. But if it
has assignment change it could end up reconciliing and triggering callbacks for
the old assignment when the member is already rejoining.
We should fix to not apply HB responses for requests sent before the leave,
even if the member is not LEAVING (may be joining). Let's also look at how the
in-flight flag is cleared in this case. And this probably needs fixing
consistently for the consumer and streams membership managers.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)