Great...Thankyou very much Brian..
After I fixed labels and annotations, everything works fine :)
On Thursday, April 13, 2023 at 2:21:39 AM UTC-7 Brian Candler wrote:
> > And always alert is getting triggered even if I keep input series value
> as 0.
>
> No, the alert is *not* being triggered. That's what "got: []" means: "I
> got an empty list of alerts" (although your test is written to *expect* a
> non-empty list of alerts, so the reality doesn't match expectations)
>
> > Any help on this please?
>
> Well, to start with:
>
> 100000000/1024/1024/1024 *100 = 9.31322574
>
> So you need to add a zero to your test values.
>
> Secondly, your alerting rule has "for: 5m". If you keep this, then the
> alert condition has to have been true for 5 minutes, and has to be
> evaluated at 5 minutes. So you need:
>
> values: '1000000000 1000000000 1000000000 1000000000 1000000000
> 1000000000'
> and uncomment "eval_time: 5m"
>
> Alternatively, you can comment out "for: 5m" in your alerting rule, in
> which case you only need
> values: '1000000000'
>
> After this, your alert triggers (it "got" an alert), but you'll find the
> labels and annotations don't match exactly what you'd put in the
> expectations:
>
> FAILED:
> alertname: ServiceMemoryUsage90Percent, time: 5m,
>
> exp:[
> 0:
> Labels:{alert_owner="alerts-infra",
> alertname="ServiceMemoryUsage90Percent", severity="Critical"}
> Annotations:{description="Service memory has been at over
> 90% for 1 minute. Service: (), DnsName: ()."}
> ],
> got:[
> 0:
> Labels:{alert_owner="alerts-infra",
> alertname="ServiceMemoryUsage90Percent",
> container_label_io_rancher_stack_name="network-service",
> severity="Critical"}
> Annotations:{description="Service memory has been at over
> 90% for 1 minute. Service: network-service, DnsName: . \n"}
> ]
>
> You can then modify your expectations to make them match what the alerting
> rule actually generates - or you can modify the alerting rule itself to
> generate a different set of labels and annotations.
>
> On Wednesday, 12 April 2023 at 18:47:10 UTC+1 radhamani...@gmail.com
> wrote:
>
>> Hi,
>> I need help in this alert test.
>> This is the alert rule I have,
>>
>>
>> - alert: ServiceMemoryUsage90Percent
>> expr:
>> sum(container_memory_usage_bytes{job="cadvisor",container_label_io_rancher_stack_name!="",
>>
>> image=~"rancher.*"}) by (container_label_io_rancher_stack_name, instance,
>> dns_name) /1024/1024/1024 *100 > 90
>> for: 5m
>> labels:
>> severity: Critical
>> alert_owner: alerts-infra
>> annotations:
>> description: "Service memory has been at over 90% for 1 minute. Service:
>> {{ $labels.container_label_io_rancher_stack_name }}, DnsName: {{
>> $labels.dns_name }}. \n"
>>
>>
>>
>> I am trying to write a test for this,
>> tests:
>> # Infra Alert Tests
>> - input_series:
>> - series:
>> 'container_memory_usage_bytes{job="cadvisor",container_label_io_rancher_stack_name="network-service",image="rancher.test"}'
>> values: '0 100000000 0 100000000 100000000'
>>
>> alert_rule_test:
>> - alertname: ServiceMemoryUsage90Percent
>> # eval_time: 5m
>> exp_alerts:
>> - exp_labels:
>> alertname: ServiceMemoryUsage90Percent
>> severity: Critical
>> alert_owner: alerts-infra
>> exp_annotations:
>> description: "Service memory has been at over 90% for 1 minute. Service:
>> (), DnsName: ()."
>>
>> But I am always getting ,
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Unit Testing: tests/infra_db_alerts-tests.yml FAILED: alertname:
>> ServiceMemoryUsage90Percent, time: 0s, exp:[ 0:
>> Labels:{alert_owner="alerts-infra",
>> alertname="ServiceMemoryUsage90Percent", severity="Critical"}
>> Annotations:{description="Service memory has been at over 90% for 1 minute.
>> Service: (), DnsName: ()."} ], got:[]*
>>
>>
>> I tried many options in alert rule testing. But nothing helped.
>> Any help on this please?
>> not sure why I am getting got as empty. And always alert is getting
>> triggered even if I keep input series value as 0.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/7ac30d8c-d60e-4e66-b70b-84c6d390caadn%40googlegroups.com.
