[jira] [Commented] (KAFKA-7681) new metric for request thread utilization by request type
[ https://issues.apache.org/jira/browse/KAFKA-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764927#comment-16764927 ] Chris Bogan commented on KAFKA-7681: I'm trying to join gropg sorry a Little confused > new metric for request thread utilization by request type > - > > Key: KAFKA-7681 > URL: https://issues.apache.org/jira/browse/KAFKA-7681 > Project: Kafka > Issue Type: Improvement >Reporter: Jun Rao >Assignee: Mayuresh Gharat >Priority: Major > > When the request thread pool is saturated, it's often useful to know which > type request is using the thread pool the most. It would be useful to add a > metric that tracks the fraction of request thread pool usage by request type. > This would be equivalent to (request rate) * (request local time ms) / 1000, > but will be more direct. This would require a new KIP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7681) new metric for request thread utilization by request type
[ https://issues.apache.org/jira/browse/KAFKA-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721607#comment-16721607 ] Jun Rao commented on KAFKA-7681: [~mgharat], Meter is calculated by (accumulated values / time of the accumulation). So, if you do 1), essentially you get the ratio of the time a particular request type spending on the request handler. We just need to name the metric accordingly. > new metric for request thread utilization by request type > - > > Key: KAFKA-7681 > URL: https://issues.apache.org/jira/browse/KAFKA-7681 > Project: Kafka > Issue Type: Improvement >Reporter: Jun Rao >Assignee: Mayuresh Gharat >Priority: Major > > When the request thread pool is saturated, it's often useful to know which > type request is using the thread pool the most. It would be useful to add a > metric that tracks the fraction of request thread pool usage by request type. > This would be equivalent to (request rate) * (request local time ms) / 1000, > but will be more direct. This would require a new KIP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7681) new metric for request thread utilization by request type
[
https://issues.apache.org/jira/browse/KAFKA-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718138#comment-16718138
]
Mayuresh Gharat commented on KAFKA-7681:
Hi [~junrao],
# IIUC, After inspecting the code and your suggestion above, this seems doable
by adding a new metric type like "localTimeRate" or
"RequestHandlerThreadpoolUitlizationRate" or similar and having it as a Meter()
like we have for {code:java} totalProduceRequestRate {code}.
This will give us the rate of local time for each request which ~ usage of the
RequestHandlerThreadPool.
# I was thinking more on the lines of having a ratio, that would give us
instantaneous value (Guage) like we have for "NetworkProcessorAvgIdlePercent",
but for each request type wherein the
{code:java}
value = (Total sampled Local Time of A Request) / (Total of sampled local times
of all the requests)
{code}
> new metric for request thread utilization by request type
> -
>
> Key: KAFKA-7681
> URL: https://issues.apache.org/jira/browse/KAFKA-7681
> Project: Kafka
> Issue Type: Improvement
>Reporter: Jun Rao
>Assignee: Mayuresh Gharat
>Priority: Major
>
> When the request thread pool is saturated, it's often useful to know which
> type request is using the thread pool the most. It would be useful to add a
> metric that tracks the fraction of request thread pool usage by request type.
> This would be equivalent to (request rate) * (request local time ms) / 1000,
> but will be more direct. This would require a new KIP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-7681) new metric for request thread utilization by request type
[ https://issues.apache.org/jira/browse/KAFKA-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715264#comment-16715264 ] Mayuresh Gharat commented on KAFKA-7681: [~junrao] will try to give a shot at this. > new metric for request thread utilization by request type > - > > Key: KAFKA-7681 > URL: https://issues.apache.org/jira/browse/KAFKA-7681 > Project: Kafka > Issue Type: Improvement >Reporter: Jun Rao >Assignee: Mayuresh Gharat >Priority: Major > > When the request thread pool is saturated, it's often useful to know which > type request is using the thread pool the most. It would be useful to add a > metric that tracks the fraction of request thread pool usage by request type. > This would be equivalent to (request rate) * (request local time ms) / 1000, > but will be more direct. This would require a new KIP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7681) new metric for request thread utilization by request type
[ https://issues.apache.org/jira/browse/KAFKA-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702176#comment-16702176 ] Jun Rao commented on KAFKA-7681: Hi, [~mgharat], The Broker Topic metrics only collect the byte read/write rate. This is a bit limiting since it only covers produce/fetch requests. Sometimes, other types of requests (e.g. metadata, joinGroup) could be hogging the request handling threads. Also, the byte rate doesn't tell us how much the request handler thread is being used. For example, the serving of a fetch request is mostly done in the network thread, instead of the request handling thread. What I am thinking is the following. Each request handler thread is being used from the time that it takes a request from the request queue, until the local processing of the request is done (KafkaApis.handle() returns), which is the request localTime. If we aggregate the localTime per request type (e.g. producer/ fetch/metadata/joinGroup, etc) and calculate a rate of that value, it gives us the fraction of request handler usage by request type. This will tell us which type of request is using the requests handler threads the most and can be useful for debugging. > new metric for request thread utilization by request type > - > > Key: KAFKA-7681 > URL: https://issues.apache.org/jira/browse/KAFKA-7681 > Project: Kafka > Issue Type: Improvement >Reporter: Jun Rao >Priority: Major > > When the request thread pool is saturated, it's often useful to know which > type request is using the thread pool the most. It would be useful to add a > metric that tracks the fraction of request thread pool usage by request type. > This would be equivalent to (request rate) * (request local time ms) / 1000, > but will be more direct. This would require a new KIP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7681) new metric for request thread utilization by request type
[ https://issues.apache.org/jira/browse/KAFKA-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701387#comment-16701387 ] Mayuresh Gharat commented on KAFKA-7681: Hi [~junrao], This seems more like the Broker Topic Metrics that we have today but at the RequestHandler level and not for a specific topic. Is my understanding correct? > new metric for request thread utilization by request type > - > > Key: KAFKA-7681 > URL: https://issues.apache.org/jira/browse/KAFKA-7681 > Project: Kafka > Issue Type: Improvement >Reporter: Jun Rao >Priority: Major > > When the request thread pool is saturated, it's often useful to know which > type request is using the thread pool the most. It would be useful to add a > metric that tracks the fraction of request thread pool usage by request type. > This would be equivalent to (request rate) * (request local time ms) / 1000, > but will be more direct. This would require a new KIP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
