Hi Brian, An alert is identified by its unique set of labels, so the labels really serve as an identification, and changing / adding / removing any label will semantically mean that it's a different alert than before (with all the effects that has on routing and grouping in Alertmanager). Annotations are non-identifying pieces of extra information on alerts that do not affect routing or grouping at all, so those are indeed likely what you want. What you want to do might work with just labels if your metadata has a stable relationship to existing alerts that wouldn't cause frequent changes to their label set identities (e.g. if it's just a few extra attributes that rarely change). What is the metadata you want to attach?
External labels are applied to every outgoing alert of a Prometheus server equally (not just for some alerts), is that what you want? Regards, Julius On Fri, Jan 15, 2021 at 4:38 PM Brian Gottfried <[email protected]> wrote: > We're forwarding our alerts out to an external system that requires some > additional metadata and I'm trying to understand the effect of providing > this metadata as annotations vs labels on each alert. > > Context: We're utilizing both custom alerts and the default alerts that > come with Openshift. The wrinkle is that those Openshift default alerts are > Operator-managed, so we can't modify them directly to add the metadata as > annotations (which from what I've found is the correct way to add metadata > that you want to be the same across all instances of an alert). However, we > can override the Openshift Prometheus' settings to add the values via the > external labels field. Will this cause any issues with alerts being grouped > together oddly or not being sent? > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/14d57178-e2ff-4902-a270-655f4cb5e04fn%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/14d57178-e2ff-4902-a270-655f4cb5e04fn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- Julius Volz PromLabs - promlabs.com -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAObpH5z1Y1xcBrJFDmrDyY8H0N5AaHLFPri_g0euFOjNuhOUeg%40mail.gmail.com.
