Ravi Gupta created KAFKA-18143:
----------------------------------
Summary: Kafka consumer keeps getting records on poll after
eviction from group
Key: KAFKA-18143
URL: https://issues.apache.org/jira/browse/KAFKA-18143
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 3.9.0, 3.7.1
Reporter: Ravi Gupta
My application polls records from a MSK Kafka cluster. The application
maintains offset of each partition and hence has disabled autocommit. It
actually never commits offset as it persists offset to internal data store.
In production, we observed duplicate records. {*}The duplicate records don't
stop until we restart the instance with zombie consumer (evicted from group but
keeps polling){*}. A consumer turns to zombie when it fails to send heartbeats.
This typically happens due to IAM authentication issues in MSK which sometime
lasts for longer time.
On further digging, I found that the partitions that are assigned to zombie
consumer are assigned to other active consumers, but the zombie consumers poll
continue to return the records.
The question is - *should a zombie consumer get records on poll?*
I have been able to reproduce it locally. Here is my local setup with issue
reproduced:
* A single broker (docker image) with three different external port.
* Create one topic with two partitions
* [Toxiproxy|https://github.com/Shopify/toxiproxy] proxies these port.
* Two consumers (subscribed to the topic) each connecting one of the proxied
port with session timeout set to 10 seconds
* Introduce latency in one of the proxy to evict one of the consumer from the
group
* Produce some messages for each partition
* Both the consumer keep getting the messages
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
