Thank you for your reply.
Let me clarify what we are going to do more.
We have the following alert rules for a container. But we still want to
include the CPU usage of the corresponding host which host this container.
That is:
- when CPU usage for container is larger than 85%, and CPU usage for host
is larger than 85%, then the alert is triggered.
#rules for container
- alert: ContainerCpuUsage
expr: (sum(rate(container_cpu_usage_seconds_total{name=~".+"}[3m]))
BY (instance, name) * 100) > 85
for: 10m
labels:
severity: warning
annotations:
summary: "Container CPU usage (instance {{ $labels.instance }})"
description: "Container CPU usage is above 80%\n VALUE = {{
$value }}\n LABELS: {{ $labels }}"
#rules for host
- alert: HighCpuLoad
expr: 100 - (avg by(instance)
(irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
for: 5m
labels:
severity: critical
annotations:
summary: "High CPU load (instance {{ $labels.instance }})"
description: "[TEST] CPU load is > 80%\n VALUE = {{ $value
}}\n LABELS: {{ $labels }}"
#rules for
On Monday, December 6, 2021 at 4:04:50 PM UTC+8 Brian Candler wrote:
> Yes, but you need to be really clear on what you're doing. What you need
> to realise is that this is a *vector* expression:
>
> expr: containerCPUusage > 80
>
> This may return zero, one, or many results in a vector: that is, a set of
> all timeseries with metric name "containerCPUusage" that meet the
> condition. In general, a given metric can refer to multiple timeseries,
> distinguished by having different label sets.
>
> The expression "serverCPUusage>80" is the same, but it could return a
> vector with a different set of labels. So the clever part is being able to
> associated containerCPUusage with serverCPUusage, by finding a subset of
> labels which allows you to do the matching in a unique way, for example N
> containers link to 1 server. It's documented here
> <https://prometheus.io/docs/prometheus/latest/querying/operators/#many-to-one-and-one-to-many-vector-matches>
> .
>
> If you show real examples of the complete metrics, with their full sets of
> labels, we may be able to give more specific help.
>
> On Monday, 6 December 2021 at 07:58:01 UTC [email protected] wrote:
>
>> Hello guys,
>>
>> Is there a way to report an alert like this:
>>
>> expr: if containerCPUusage>80 and serverCPUusge>80
>>
>> Then trigger the alert.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/5b43a958-e011-4e98-becf-729424ce5eefn%40googlegroups.com.
