Chi-Hsuan Huang created HDDS-16611:
--------------------------------------
Summary: Use MutableGaugeLong for numVolumes, numBuckets and
numKeys in OMMetrics
Key: HDDS-16611
URL: https://issues.apache.org/jira/browse/HDDS-16611
Project: Apache Ozone
Issue Type: Bug
Components: Ozone Manager
Reporter: Chi-Hsuan Huang
Assignee: Chi-Hsuan Huang
{{OMMetrics}} declares {{numVolumes}}, {{numBuckets}} and {{numKeys}} as
{{MutableCounterLong}}, but they behave like gauges. They are decremented with
{{incr\(\-1\)}}, and {{setNum\*\(\)}} emulates a set via {{incr\(val \-
oldVal\)}}. This has been the case since HDDS\-816.
The JMX values are correct, but {{/prom}} exports these metrics as {{# TYPE
counter}}. Prometheus treats any decrease in a counter as a reset. So the _Key
Creation Rate_ panel in the Memory Consumption dashboard, which uses
{{rate\(om\_metrics\_num\_keys\[1m\]\)}}, reports inflated values whenever keys
are deleted.
HDDS\-10597 fixed the same pattern in {{SafeModeMetrics}}.
Proposed changes:
\- Switch the three fields to {{MutableGaugeLong}} and use {{set\(\)}} in
{{setNumVolumes}}, {{setNumBuckets}} and {{setNumKeys}}.
\- Update {{TestOmMetrics}} to read these values with {{getLongGauge}} instead
of {{getLongCounter}}.
\- Update the Key Creation Rate panel to use {{deriv\(\)}}, or
{{rate\(om\_metrics\_num\_key\_commits\[1m\]\)}}.
Metric names in {{/jmx}} and {{/prom}} stay the same.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
