[
https://issues.apache.org/jira/browse/KAFKA-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Jacot updated KAFKA-17182:
--------------------------------
Priority: Blocker (was: Critical)
> Consumer fetch sessions are evicted too quickly with AsyncKafkaConsumer
> -----------------------------------------------------------------------
>
> Key: KAFKA-17182
> URL: https://issues.apache.org/jira/browse/KAFKA-17182
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.8.0
> Reporter: Kirk True
> Assignee: Kirk True
> Priority: Blocker
> Labels: consumer-threading-refactor
> Fix For: 4.0.0
>
>
> In stress testing the new consumer, the new consumer is evicting fetch
> sessions on the broker much more frequently than expected. There is an
> ongoing investigation into this behavior, but it appears to stem from a race
> condition due to the design of the new consumer.
> In the background thread, fetch requests are sent in a near continuous
> fashion for partitions that are "fetchable." A timing bug appears to cause
> partitions to be "unfetchable," which then causes them to end up in the
> "removed" set of partitions. The broker then removes them from the fetch
> session, which causes the number of remaining partitions for that session to
> drop below a threshold that allows it to be evicted by another competing
> session. Within a few milliseconds, though, the partitions become "fetchable"
> again, and are added to the "added" set of partitions on the next fetch
> request. This causes thrashing on both the client and broker sides as both
> are handling a steady stream of evictions, which negatively affects
> consumption throughput.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
