maybe I found some usefull information in alertmanger debug logs, it shows
sometime the resolved alert messages didn't send to webhook , I feel a
little confused, could you give me some ideas? thank you.
for details, see below：
level=debug ts=2020-03-11T09:08:58.366265467Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][active]
level=debug ts=2020-03-11T09:09:24.672978327Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:09:58.365912476Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:11:24.675325088Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:11:58.365464319Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:13:24.673909146Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:13:58.36588506Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:15:24.678054827Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:15:58.366574391Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:17:24.674620753Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:17:58.365941732Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:19:24.673374598Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:19:58.370509047Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:21:24.675464534Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:21:58.366838423Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:23:24.674355197Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:23:58.366254181Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[65e7146][resolved]
level=debug ts=2020-03-11T09:58:24.674017138Z caller=dispatch.go:104
component=dispatcher msg="Received alert"
alert=IstioService_429_Error[4a3874a][active]
level=debug ts=2020-03-11T09:58:54.67426069Z caller=dispatch.go:343
component=dispatcher aggrGroup="{}:{alertname=\"IstioService_429_Error\"}"
msg=flushing alerts=[IstioService_429_Error[4a3874a][active]]
level=debug ts=2020-03-11T09:58:54.814196814Z caller=nflog.go:538
component=nflog msg="gossiping new entry"
entry="entry:<group_key:\"{}:{alertname=\\\"IstioService_429_Error\\\"}\"
receiver:<group_name:\"webhook\" integration:\"webhook\" idx:2 >
timestamp:<seconds:1583920734 nanos:772370937 >
firing_alerts:15719501330978136231 > expires_at:<seconds:1584352734
nanos:772370937 > "
在 2020年3月11日星期三 UTC+8下午8:45:03，zhang windy写道：
>
> Hi, buddies, I met a problem, while the alert in it had a listed end
> time of 2020-03-11T09:59:24, the actual webhook was only sent at
> 2020-03-11T10:13,
> it looks like 15 minutes interval, I want to decrease the period time to
> 5s, so I carefully look for the configuration about alertmanager, but
> cannot found anything about it, how can I set it? thank you very much.
>
> below is configuration of alertmanger:
>
> "global":
> "resolve_timeout": "5m"
> smtp_smarthost: 'XXXXXXX'
> smtp_from: 'XXXXXXX'
> smtp_auth_username: 'XXXXXXX
> smtp_auth_password: 'XXXXXXX'
> smtp_require_tls: false
> "receivers":
> - "name": "mail"
> email_configs:
> - to: '[email protected]'
> - "name": "webhook"
> webhook_configs:
> - url: 'http://flask-alert-service.monitoring:5000/send'
> #- url: 'http://flask-alert-service-2.monitoring:5000/send'
> - url: 'http://aipaas-alert-service.aipaas:10019/alert/message'
> - url: '
> http://aipaas-serving-model-service.aipaas:10014/serving/alert/message'
> "route":
> "group_by": ['alertname', 'cluster','service']
> #- "job"
> "group_interval": "1h"
> "group_wait": "30s"
> "receiver": "webhook"
> "repeat_interval": "12h"
> "routes":
> - "match":
> "alertname": "DeadMansSwitch"
> "receiver": "webhook"
> - "match":
> severity: warning
> "receiver": "webhook"
>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/6cf7473c-efcb-4660-8de1-b5a9f63c953e%40googlegroups.com.
