This is probably a case where you would want to look into Thanos or Cortex to provide a larger aggregation layer on top of multiple Prometheus servers.
On Sun, May 17, 2020 at 11:53 AM Rajesh Reddy Nachireddi < [email protected]> wrote: > Hi, > > Basically, we have large networking setup with 10k devices. we are hitting > 1M metrics every second from 20 % of devices itself, so we have 5 prom > instances and one global proemtheus which uses remote read to handle alert > rule evaluations and thanos querier for visualisation on grafana. > > We have segregated devices with specific device ip ranges to each > Prometheus instances. > > So, we have one aggregator which is using remote read from all the > individual prom instances through remote read > > 1. will the remote read cause an issue w.r.t loading the large time series > over wire every 1 min ? > 2. Is it CPU or memory intensive ? > > What is best design strategy to handle these scale and alerting across the > devices or metrics ? > > Regards, > > Rajesh > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/CAEyhnp%2BfG8YvciR4-30D%2BzsDzg_kF%2BKkJUavdbyGCxoz-97q_A%40mail.gmail.com > <https://groups.google.com/d/msgid/prometheus-users/CAEyhnp%2BfG8YvciR4-30D%2BzsDzg_kF%2BKkJUavdbyGCxoz-97q_A%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmpN%3DuLhdNe_7v%3Dj53WJtfBdxSYA9XyZL2kXHsMqT%2BF9JA%40mail.gmail.com.
