[
https://issues.apache.org/jira/browse/KAFKA-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702176#comment-16702176
]
Jun Rao commented on KAFKA-7681:
--------------------------------
Hi, [~mgharat],
The Broker Topic metrics only collect the byte read/write rate. This is a bit
limiting since it only covers produce/fetch requests. Sometimes, other types of
requests (e.g. metadata, joinGroup) could be hogging the request handling
threads. Also, the byte rate doesn't tell us how much the request handler
thread is being used. For example, the serving of a fetch request is mostly
done in the network thread, instead of the request handling thread.
What I am thinking is the following. Each request handler thread is being used
from the time that it takes a request from the request queue, until the local
processing of the request is done (KafkaApis.handle() returns), which is the
request localTime. If we aggregate the localTime per request type (e.g.
producer/ fetch/metadata/joinGroup, etc) and calculate a rate of that value, it
gives us the fraction of request handler usage by request type. This will tell
us which type of request is using the requests handler threads the most and can
be useful for debugging.
> new metric for request thread utilization by request type
> ---------------------------------------------------------
>
> Key: KAFKA-7681
> URL: https://issues.apache.org/jira/browse/KAFKA-7681
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jun Rao
> Priority: Major
>
> When the request thread pool is saturated, it's often useful to know which
> type request is using the thread pool the most. It would be useful to add a
> metric that tracks the fraction of request thread pool usage by request type.
> This would be equivalent to (request rate) * (request local time ms) / 1000,
> but will be more direct. This would require a new KIP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
