My apologies if this has been answered already, but I've looked through the
configs for a setting that would allow me to define how many targets can be
scraped at once and came up empty. Essentially, what I've got going on
here is my prometheus is being blocked by my checkpoint firewalls (for
between 10-20 minutes) due to the number of targets that it's scraping at
once ( because of the Suspicious Activity Monitoring module.)
My configuration:
- Central Prometheus server
- Multiple Data Centers
- SNMP monitored by local SNMP Exporters local to each datacenter
- Windows / Linux boxes monitored via Telegraf scraping
- Various other exporters (generally on the Prometheus server itself
unless large number of targets in remote datacenter)
Unfortunately, I've already talked to Checkpoint and made all of the
changes they recommend without any improvement. I've also already
increased the scrape interval (currently sitting at 4m) but the scrapes
appear to all be happening within say a minute of each other. This results
in the checkpoints blocking the activity and the targets appearing to be
down.
My only other idea to resolve this is to increase the time in the alert
configuration to give additional time so that while the firewall is still
blocking the traffic, we don't get the alerts. This feels moronic though,
and I'm holding it back as a "just keep my mailbox empty" route.
Has anyone come up with a clever way to work around this?
Thanks,
Andy
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/1d24070e-eda2-4c1a-b5f3-e747e920ba82%40googlegroups.com.
