What type of metric are we talking about? A gauge? Prometheus scrapes it's targets continuously so there should be a continuous stream of data points forming a series, not just a single data point when a event occurs. This sounds more like a log to me. It would be helpful if you could post your rule. It sounds like a gauge that goes up to 1 and than back to 0 directly, flapping around like that. For events I recommend you to switch to a counter instead.
Since you have mentioned the alert going into a pending state (yellow) and than going back within 15 seconds I assume that you evaluation interval is above 15s. If it was below that, the alert should have fired. You can try putting the rule into it's own group with a low evaluation interval (5s for example) just to try it out. But in the long term you should do something about the rule and / or the metric itself [email protected] schrieb am Montag, 19. Oktober 2020 um 23:02:39 UTC+2: > Hi, > > I have a metric which sends data only in case of a specific event. In > other words, it will not have regular continuous telemetry every 15 > seconds, but it will be there for a single data point. Now, I configured an > alert to check if the metric exists (did not include any 'for' condition in > the rule as there is no need to wait), an alert should trigger. What is > happening is that when the metric becomes available in Prometheus, the rule > gets activated to yellow, but instead of firing, it automatically goes > away within 15 seconds. Is there some additional setting I have to do to > alert on event based metric which lasts for only 1 single datapoint or 15s > ? > > Thanks, > Arnav > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d6784e4b-cb9c-4bce-b25c-f7bdbad754b4n%40googlegroups.com.
