On 17/11/2020 03:42, Andy Pan wrote:
I think you might misunderstand what I said, the load-balancer is not
for Prometheus collection metrics (Prometheus collects metrics from
all push gateways) but for business servers pushing metrics.
In this case, would this be the workaround?
If you have 2 push gateways and one thing pushing metrics (for example)
as you say you could easily scrape both push gateways. In steady state
you now have 2 copies of every metric (differing just by the instance
label).
If you have a load balancer between the pushing app & the two push
gateways whenever you push it will update one of the two push gateways.
You now have one push gateway containing the old metrics and one
containing the new.
You continue to scrape both push gateways and now Prometheus has
ingested those metrics.
How would you query this data?
If you just ignore the 2nd push gateway (e.g. by adding a instance label
filter in your query) then you will miss some of the changes (because
the push gateway you are ignoring was the one that got updated)
If you don't ignore one, you'd end up with two sets of answers to your
query, one based on the data from push gateway 1 & the other from push
gateway 2. At various points in time different answers will be correct,
but you wouldn't know which one.
So you are now in a situation where you have twice the number of metrics
that you actually want, while also not knowing which values are correct.
So not a good situation to be in.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/9f0c3bf0-afe0-19fe-6194-b250667c6f52%40Jahingo.com.
