[
https://issues.apache.org/jira/browse/KAFKA-15173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirk True updated KAFKA-15173:
------------------------------
Fix Version/s: 3.8.0
(was: 4.0.0)
> Consumer event queues should be bounded
> ---------------------------------------
>
> Key: KAFKA-15173
> URL: https://issues.apache.org/jira/browse/KAFKA-15173
> Project: Kafka
> Issue Type: Improvement
> Components: clients, consumer
> Reporter: Philip Nee
> Assignee: Kirk True
> Priority: Major
> Labels: consumer-threading-refactor
> Fix For: 3.8.0
>
>
> The async consumer uses ApplicationEventQueue and BackgroundEventQueue to
> facilitate message passing between the application thread and the background
> thread. The current implementation is boundless, which can potentially cause
> OOM and other performance-related issues.
> I think the queues need a finite bound, and we need to decide how to handle
> the situation when the bound is reached. In particular, I would like to
> answer these questions:
>
> # What should the upper limit be for both queues: Can this be a
> configurable, memory-based bound? Or just an arbitrary number of events as
> the bound.
> # What should happen when the application event queue is filled up? It
> seems like we should introduce a new exception type and notify the user that
> the consumer is full.
> # What should happen when the background event queue is filled up? This
> seems less likely to happen, but I imagine it could happen when the user
> stops polling the consumer, causing the queue to be filled.
> # Is it necessary to introduce a public configuration for the queue? I think
> initially we would select an arbitrary constant number and see the community
> feedback to make a forward plan accordingly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
