I agree, the best practice here is to have Prometheus as close to the targets as possible. This way any WAN links, VPNs, or proxies are not monitored as a side effect.
The remote write feature is designed for this. It can stream data to a central service like Cortex or Thanos. Or a hosted service like Grafana Cloud. On Mon, Dec 14, 2020 at 11:28 AM Stuart Clark <[email protected]> wrote: > On 2020-12-14 10:24, Patrick Macdonald wrote: > > This might be more of a general networking question than a Prometheus > > question, but I thought I'd ask anyway. > > > > If we have a Prometheus server on site-A and an arbitrary number of > > client sites, each with hardware we might want monitor, is there best > > practice for how to achieve this? > > I'm assuming pushgateway isn't the correct use-case here? Is the only > > way to create a tunnel in to each client site so that the Prometheus > > server can access each target directly? > > I'm hoping there might be some way to achieve this without much work > > on the client IT side. > > > > Generally you'd want Prometheus servers at each client site. The reason > is that you want to be able to monitor the end devices/services > themselves, rather than also be monitoring/affected by the network. For > example, if the connection is down you'd lose all visibility for the > whole period, even once the connection is back. > > Another useful tool is Push Prox: > https://github.com/prometheus-community/PushProx > > But you are otherwise correct that the Push Gateway isn't something that > you should be using at all for this use case (unless you also have short > lived cron style jobs that you are wanting to monitor). > > -- > Stuart Clark > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/a1ff2424c9506887e9bb3cf29a77dd15%40Jahingo.com > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmoThhDg8KRLrEHR0j-KBokj%3Dn8nEtjdKCBFndOcN3Vg7Q%40mail.gmail.com.
