John Roesler created KAFKA-9557:

             Summary: Thread-level "process" metrics are computed incorrectly
                 Key: KAFKA-9557
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: John Roesler
            Assignee: John Roesler

Among others, Streams reports the following two thread-level "process" metrics:

"process-rate": The average number of process calls per second.
"process-total": The total number of process calls across all tasks.

See the docs:

There's some surprising ambiguity in these definitions that has led to Streams 
actually reporting something different than what most people would probably 
expect. Specifically, it's not defined what a "process call" is.

A reasonable definition of a "process call" is processing a record or 
processing a task (both of which are publicly facing concepts, and both of 
which are the same, since tasks process records one at a time). However, we 
currently measure number of invocations to a private, internal `process()` 
method, which would actually process more than one record at a time. Thus, the 
current metric is under-counting the throughput, in an esoteric and confusing 

Instead, we should simply change the rate and total metrics to measure the 
(rate and total) of _record_ processing.

This message was sent by Atlassian Jira

Reply via email to