Just basic debugging: 1. Use the Prometheus web interface (usually x.x.x.:9090). It has an "Alerts" tab. It will tell you if the alerts exist, and whether any of them are firing.
If not, find out why. Did you HUP prometheus to re-read the rules after changing config? Does your prometheus.yml reference the rules file? Try: promtool check config /etc/prometheus/prometheus.yml (it will tell you which rules files it read, and how many rules read in from each) If this is all OK, then: 2. Use the Alertmanager web interface (usually x.x.x.x:9093). It will tell you if the alerts are active. If not, find out why. (Did you HUP alertmanager to re-read the config?) Similarly, check the configuration: amtool check-config /etc/alertmanager/alertmanager.yml If this is all OK, then: 3. Look at prometheus and alertmanager logs, e.g. if you're running under systemd: journalctl -eu prometheus journalctl -eu alertmanager Increase log verbosity if required. Run the binaries with "--help" flag to see the available options. Should be something like --log.level=debug 4. Use tcpdump to investigate traffic between prometheus and alertmanager, and between alertmanager and postfix. My guess: you might need to set "smtp_require_tls: false" under global, since it defaults to true: https://prometheus.io/docs/alerting/latest/configuration/#configuration-file -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/e819bea8-85a3-4862-b61a-9022c91df132o%40googlegroups.com.
