[
https://issues.apache.org/jira/browse/KAFKA-18369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rafał Sumisławski updated KAFKA-18369:
--------------------------------------
Summary: State updater's *-ratio metrics are incorrect (was: State
updater's `*-ratio` metrics are incorrect)
> State updater's *-ratio metrics are incorrect
> ---------------------------------------------
>
> Key: KAFKA-18369
> URL: https://issues.apache.org/jira/browse/KAFKA-18369
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.8.1
> Reporter: Rafał Sumisławski
> Priority: Major
>
> h2. Background
> {{DefaultStateUpdater}} defines {{idle-ratio}}, {{active-restore-ratio}},
> {{standby-update-ratio}}, {{checkpoint-ratio}} metrics here:
> https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java#L1101-L1115
> These metrics are averages, that are supposed to indicate "The fraction of
> time the thread spent on \{action}". But the metrics don't actually do that.
> h2. Issue
> Let me explain this with an example:
> For simplicity's sake, let's consider the following example involving just
> {{standby-update-ratio}}, {{checkpoint-ratio}} and ignoring the existence of
> the other two metrics.
> Let's say the thread did:
> * {{999}} iterations with {{standby-update}} taking {{1ms}} in each iteration
> and no {{checkpoint}} happening ({{0ms}}).
> * {{1}} iteration with {{standby-update}} taking {{1ms}} and {{checkpoint}}
> taking {{9000ms}}
> The thread spent {{10s}} working, of which it spent {{1s}} on
> {{standby-updates}} and {{9s}} on checkpoint, so the fraction of time it
> spent on checkpoint (checkpoint-ratio) is {{~0.001 (0.1%)}}. Or at least that
> is what the metrics will say. I would instead argue that it spent {{9s/10s ==
> 0.9 == 90%}} on checkpoint. If you agree with my logic, then you agree that
> this metrics is incorrect.
> The problem is that the code computes a ratio for each iteration, and then
> averages those ratios out, producing a number devoid of statistical meaning
> or practical application. It ignores the fact that the one iteration that
> took {{9s}}, should have a much higher weight than those quick 1ms iterations.
> {{(999*(0ms/1ms) + 1*(9000ms/9001ms))/1000 ~= 0.001}}
> h2. Solution
> What we would like to see instead is either a ratio of average/total times,
> not an average of ratios. I don't think this can be easily realised within
> the existing metrics system. So instead, what I propose as a solution is to
> report {{duration-total}} and/or {{duration-rate}} (with the unit of seconds
> per second) metric for each of {{idle}}, {{active-restore}},
> {{standby-restore}}, {{checkpoint}}. The observers of these metrics, when
> needed, could then derive the actual ratio of time spent on each operation
> for example as {{checkpoint-ratio = checkpoint-duration-rate /
> (idle-duration-rate + active-restore-duration-rate +
> standby-restore-duration-rate + checkpoint-duration-rate)}}. Or by performing
> an analogical calculation on deltas of the {{total}} metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
