The alert formation doesn't seem to be a problem here, because it happens
for different alerts randomly. Below is the alert for Exporter being down
for which it has happened thrice today.
- alert: ExporterDown
expr: up == 0
for: 10m
labels:
severity: "CRITICAL"
annotations:
summary: "Exporter down on *{{ $labels.instance }}*"
description: "Not able to fetch application metrics from *{{
$labels.instance }}*"
- the ALERTS metric shows what is pending or firing over time
>> But the problem is that one of my ExporterDown alerts is active since
the past 10 days, there is no genuine reason for the alert to go to a
resolved state.
- evaluate the alert expression in Prometheus for the given time period.
Are there gaps or does e.g. a label change between before and after the gap?
>> No gaps in the Prometheus GUI Console for the time period. The value for
UP is zero constantly for last 6 hours but still the alert when to resolved
state during that time and went to firing again.
On Wednesday, November 25, 2020 at 5:03:42 PM UTC+5:30
[email protected] wrote:
> This could be many things, likely it has to do with the formulation of the
> alert. What does it look like in Prometheus? Specifically
>
> - the ALERTS metric shows what is pending or firing over time
> - evaluate the alert expression in Prometheus for the given time period.
> Are there gaps or does e.g. a label change between before and after the gap?
>
> /MR
>
> On Wed, Nov 25, 2020 at 11:01 AM [email protected] <
> [email protected]> wrote:
>
>>
>> Hi. I am using Alertmanager 0.21.0. Occasionally, the active alerts go to
>> resolved state for a second and then come back to firing state immediately.
>> There is no pattern of this happening, it happens randomly. Haven't been
>> able to identify why this is happening.
>> Any thoughts here? Where should I start to look for this? Checked
>> Alertmanager's logs, everything seems normal.
>>
>> Thanks in advance!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-users/5dc917e0-d35f-4666-b61b-36afa7851d15n%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/prometheus-users/5dc917e0-d35f-4666-b61b-36afa7851d15n%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/d6e8ccd5-2042-4320-be9d-7bf835f98c33n%40googlegroups.com.
