Lianet Magrans created KAFKA-20915:
--------------------------------------
Summary: Unneeded app thread wakeup when a fetch already in flight
and nothing has changed in AsyncConsumer
Key: KAFKA-20915
URL: https://issues.apache.org/jira/browse/KAFKA-20915
Project: Kafka
Issue Type: Bug
Components: clients, consumer
Reporter: Lianet Magrans
Assignee: Lianet Magrans
Fix For: 4.4.0
On the async consumer, the fetch req manager has logic to wake up the fetch
buffer if it doesn't generate any fetch request to send.
[https://github.com/apache/kafka/blob/dacff9c85a29806fcafe617feb2db0c9677e6de7/clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchRequestManager.java#L149-L151]
That wake up is needed in some cases (to unblock app thread if the case was no
fetch req generated because partitions were paused, not subscribed etc)., but
the wakeup is useless and harmful if the case was that no requests was
generated because they were already in-flight.
* Useless because nothing has changed yet regarding data (it is the fetch
response for the inflights that should wake up the app thread)
* harmful because it leads to a spin (high CPU) between the app thread and the
background (background wakes ups app thread when nothing really changed
regarding data, we're just waiting for in-flight requests that will themselves
wake up the buffer then complete).
This wakeup behaviour was introduced in 4.2, but it's impact on CPU due to
unneeded app thread - background thread spin was revealed with a recent
correctness fix https://issues.apache.org/jira/browse/KAFKA-20780
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
