hi, buddies, I met a promblem, sometimes I just received a resolved email but not firing email. Do you kown the problem? so I want to comfirm the relevant logs, I have configed the prometheus log level to debug, but I can't find the relevant alert’s log in prometheus, I can see some relevant logs in alertmanager, how can see a firing alert that was sent to alertmanager in prometheus's log? is there somethng wrong I missed? could you give me some ideas? thank you very much.
the prometheus's deug log below, I can't find the relevant log about alert: level=info ts=2020-02-15T07:00:03.047720487Z caller=head.go:488 component=tsdb msg="head GC completed" duration=117.297372ms level=info ts=2020-02-15T07:00:08.727679818Z caller=head.go:535 component=tsdb msg="WAL checkpoint complete" low=53 high=65 duration=5. 679902199s level=debug ts=2020-02-15T07:35:46.342968643Z caller=scrape.go:826 component="scrape manager" scrape_pool=monitoring/kube-state-metrics/0 target=https://10.222.2.31:8443/metricsmsg="Scrape failed" err="context deadline exceeded" level=debug ts=2020-02-15T08:00:14.342979729Z caller=scrape.go:826 component="scrape manager" scrape_pool=monitoring/kube-state-metrics/0 target=https://10.222.2.31:8443/metricsmsg="Scrape failed" err="context deadline exceeded" level=info ts=2020-02-15T09:00:02.184258444Z caller=compact.go:398 component=tsdb msg="write block" mint=1581746400000 maxt=1581753600000 ulid=01E1416XN899QYYF84S7KE2Y5V level=info ts=2020-02-15T09:00:02.472066954Z caller=head.go:488 component=tsdb msg="head GC completed" duration=102.889478ms level=info ts=2020-02-15T09:00:08.099244646Z caller=head.go:535 component=tsdb msg="WAL checkpoint complete" low=66 high=78 duration=5. 627118986s level=debug ts=2020-02-15T10:10:10.343090205Z caller=scrape.go:826 component="scrape manager" scrape_pool=monitoring/kube-state-metrics/0 target=https://10.222.2.31:8443/metricsmsg="Scrape failed" err="context deadline exceeded" level=info ts=2020-02-15T11:00:02.363236579Z caller=compact.go:398 component=tsdb msg="write block" mint=1581753600000 maxt=1581760800000 ulid=01E1482MXB0Q5QD94MG17PPEXQ level=info ts=2020-02-15T11:00:02.648408131Z caller=head.go:488 component=tsdb msg="head GC completed" duration=108.614406ms level=info ts=2020-02-15T11:00:08.250280453Z caller=head.go:535 component=tsdb msg="WAL checkpoint complete" low=79 high=91 duration=5. 601822028s -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6538a366-5d20-45dd-a40c-79b74cebd283%40googlegroups.com.
