For a very long time, Prometheus did not store apeet state across restarts, so the alert startsAt would update even though the condition had not changed.
I don't think we ever considered this time to be very meaningful or stable, partially due to the originally stateless implementation, but also due to the HA synchronization issue you mentioned. Can you explain more what the scenario is where the current label-based identity doesn't work? If I am reading it right, this is the first time someone asks for the alerts to be more responsive to flapping, more typically the desire is to reduce that, identifying successive alerts as being the same thing even if the alert condition wasn't held for a short period of time. /MR On Tue, 20 Jun 2023, 15:14 'George Robinson' via Prometheus Developers, < prometheus-developers@googlegroups.com> wrote: > In prometheus/common the fingerprint of an alert is calculated as an > fnv64a hash of it's labels. The labels are first sorted, and then the label > name, separator, label value, and another separator for each label is added > to the hash before the final sum is calculated.period of time > > I noticed that something missing from the fingerprint is the alert's > StartsAt time. You could argue that an alert with labels a₁a₂aₙ that > started at time t₁ and then resolved at time t₂ is a different alert than > one also with the labels a₁a₂aₙ but started at time t₃ - and so these two > alerts should have different fingerprints. > > The fact that the fingerprint is constant over its labels has proven > interesting while debugging cases of flapping alerts in Alertmanager. > > However, while I would like to add StartsAt to the fingerprint, I am > concerned that adding the StartsAt timestamp to the fingerprint will break > Prometheus rules when run in HA as I do not believe the StartsAt time is > synchronised across rulers. > > I was wondering if there is some historical context for this? Perhaps the > reasons mentioned above, but there could be others that I am also unaware > of? > > Best regards > > George > > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to prometheus-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/b722e00e-bfd8-4ff4-bbef-e5e0836280bbn%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-developers/b722e00e-bfd8-4ff4-bbef-e5e0836280bbn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAMV%3D_gYviO9Ad%3DJXrHKHdZzMDgnu545rTEtPi_hnALvyWzaxAA%40mail.gmail.com.
