Or slightly weirder:
absent(jenkins_up) and absent(absent(up{job="jenkins"}))
absent(absent(...)) being a way to get the RHS to have no labels, to match
the LHS.
On Sunday, 3 October 2021 at 20:14:59 UTC+1 Brian Candler wrote:
> This might be an XY problem, because it is often better to have a defined
> "up/down" metric (with value 1/0), which tells you whether something worked
> or not, rather than alerting on presence or absence of a metric.
>
> However, to answer your question directly, I think you would need to
> include some condition saying whether that metric *should* be there or not
> - which is the presence of some other metric. The "up" metric added by all
> scrape jobs can be useful for this. In this case, I expect
> *up{job="jenkins"}* will exist, if and only if you have a 'jenkins'
> scrape job in that region. Therefore maybe something like this will do
> what you want:
>
> absent(jenkins_up{job="jenkins"}) unless on (job) absent(up{job="jenkins"})
>
> which I think may simplify, if the 'jenkins_up' metric is only scraped by
> the 'jenkins' job, to this (not sure):
>
> absent(jenkins_up) unless on () absent(up{job="jenkins"})
>
> On Sunday, 3 October 2021 at 19:40:23 UTC+1 Dan S wrote:
>
>> Hi,
>>
>> Looking for some general advice about shared prom alert rules between
>> regions. We currently push the same alert rules to all regions, and
>> sometimes we run into situations where we have a specific job in region X
>> but not Y.
>>
>> This is fine for basic cases, such as *up{job="jenkins"} == 0* which
>> will be ignored in regions where there's no jenkins job present (or could
>> easily specify region="X").
>>
>> But in some situations I'd like to use absent on a metric that often has
>> gaps for example
>> *absent(jenkins_up{job="jenkins"})*
>> This would trigger in all regions, whether or not there's a job "jenkins"
>> (obviously because it's triggering on the missing metrics) even if I try to
>> be more specific: *absent(jenkins_up{job="jenkins", region="US"}).*
>>
>> Any suggestions how I can craft an alert query using absent() in on
>> metrics that don't appear in all regions? So that if region="US" has
>> job="jenkins" and I watch to catch gaps here, it won't also fire in
>> region="EU" which never has job="jenkins".... ?
>>
>> Thanks for any advice.
>>
>> Dan
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/8533225b-267c-4dd1-af32-5967966d5156n%40googlegroups.com.
