John Roesler created KAFKA-9557:
-----------------------------------
Summary: Thread-level "process" metrics are computed incorrectly
Key: KAFKA-9557
URL: https://issues.apache.org/jira/browse/KAFKA-9557
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: John Roesler
Assignee: John Roesler
Among others, Streams reports the following two thread-level "process" metrics:
"process-rate": The average number of process calls per second.
"process-total": The total number of process calls across all tasks.
See the docs:
https://kafka.apache.org/documentation/#kafka_streams_thread_monitoring
There's some surprising ambiguity in these definitions that has led to Streams
actually reporting something different than what most people would probably
expect. Specifically, it's not defined what a "process call" is.
A reasonable definition of a "process call" is processing a record or
processing a task (both of which are publicly facing concepts, and both of
which are the same, since tasks process records one at a time). However, we
currently measure number of invocations to a private, internal `process()`
method, which would actually process more than one record at a time. Thus, the
current metric is under-counting the throughput, in an esoteric and confusing
way.
Instead, we should simply change the rate and total metrics to measure the
(rate and total) of _record_ processing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
