[
https://issues.apache.org/jira/browse/IGNITE-29047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18123239#comment-18123239
]
Ignite TC Bot commented on IGNITE-29047:
----------------------------------------
Monitoring: Added failure.ignored metrics that count each failure type
suppressed by the configured failure handler.
The Release Note field has been filled with the wording above. Please review it
and edit it if needed.
> Add ignored failure count metric to FailureProcessor
> ----------------------------------------------------
>
> Key: IGNITE-29047
> URL: https://issues.apache.org/jira/browse/IGNITE-29047
> Project: Ignite
> Issue Type: Task
> Reporter: Dmitry Werner
> Assignee: Dmitry Werner
> Priority: Major
> Labels: from-log-to-metric
> Fix For: 2.19
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> *Problem*
> When a FailureHandler is configured to ignore certain failure types (via
> AbstractFailureHandler#setIgnoredFailureTypes), the corresponding failures
> are silently suppressed — only a warning is logged. There is no way to
> observe, through the metrics system, how many failures of each type have been
> suppressed on a node. Operators monitoring a cluster have no visibility into
> the rate/volume of ignored failures, which makes it hard to detect recurring
> critical conditions (e.g. a repeatedly blocked system worker) that are being
> deliberately tolerated.
> *Proposed Solution*
> Expose one long counter per ignored failure type, registered under a
> dedicated metrics registry failure.ignored. Each counter is incremented every
> time FailureProcessor suppresses a failure of the matching type. Counters are
> created only for the failure types the configured handler actually ignores,
> so no metrics are registered when nothing is ignored.
> *Proposed change*
> - Register a new metrics registry "failure.ignored" in FailureProcessor.
> - Add a long counter per ignored failure type, incremented every time a
> failure is suppressed (ignored) by the configured failure handler — i.e.
> exactly where IGNORED_FAILURE_LOG_MSG is printed.
> - Keep the existing log message unchanged.
> - Document the new metric in docs/_docs/monitoring-metrics/new-metrics.adoc.
> - Add FailureProcessorMetricsTest covering per-type counting and the
> no-metrics-when-nothing-ignored case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
