Or slightly weirder:
absent(jenkins_up) and absent(absent(up{job="jenkins"}))
absent(absent(...)) being a way to get the RHS to have no labels, to match
the LHS.
On Sunday, 3 October 2021 at 20:14:59 UTC+1 Brian Candler wrote:
> This might be an XY problem, because it is often better to have a defined
> "up/down" metric (with value 1/0), which tells you whether something worked
> or not, rather than alerting on presence or absence of a metric.
>
> However, to answer your question directly, I think you would need to
> include some condition saying whether that metric *should* be there or not
> - which is the presence of some other metric. The "up" metric added by all
> scrape jobs can be useful for this. In this case, I expect
> *up{job="jenkins"}* will exist, if and only if you have a 'jenkins'
> scrape job in that region. Therefore maybe something like this will do
> what you want:
>
> absent(jenkins_up{job="jenkins"}) unless on (job) absent(up{job="jenkins"})
>
> which I think may simplify, if the 'jenkins_up' metric is only scraped by
> the 'jenkins' job, to this (not sure):
>
> absent(jenkins_up) unless on () absent(up{job="jenkins"})
>
> On Sunday, 3 October 2021 at 19:40:23 UTC+1 Dan S wrote:
>
>> Hi,
>>
>> Looking for some general advice about shared prom alert rules between
>> regions. We currently push the same alert rules to all regions, and
>> sometimes we run into situations where we have a specific job in region X
>> but not Y.
>>
>> This is fine for basic cases, such as *up{job="jenkins"} == 0* which
>> will be ignored in regions where there's no jenkins job present (or could
>> easily specify region="X").
>>
>> But in some situations I'd like to use absent on a metric that often has
>> gaps for example
>> *absent(jenkins_up{job="jenkins"})*
>> This would trigger in all regions, whether or not there's a job "jenkins"
>> (obviously because it's triggering on the missing metrics) even if I try to
>> be more specific: *absent(jenkins_up{job="jenkins", region="US"}).*
>>
>> Any suggestions how I can craft an alert query using absent() in on
>> metrics that don't appear in all regions? So that if region="US" has
>> job="jenkins" and I watch to catch gaps here, it won't also fire in
>> region="EU" which never has job="jenkins".... ?
>>
>> Thanks for any advice.
>>
>> Dan
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/8533225b-267c-4dd1-af32-5967966d5156n%40googlegroups.com.
