On Wed, Feb 26, 2020, at 14:01, Riyan Shaik wrote: > a) With a HA pair, the prometheus data will be local to both the prometheus > instances. Is it a good idea to have these 2 prometheus instances write to > some sort of a network mounted filesystem like NFS / GlusterFS filesystem,so > that the data is identical for both the prometheus instances ? Has anyone > tried this ?
As has been discussed elsewhere, two Prometheus instances cannot share the same data store. I'd also add that using NFS introduces extra, unnecessary failure modes. > b) AFAIK, with both the ha pairs scraping the same targets, how do i build a > global view of these local prometheus instances? Is federation the only way > with another Prometheus instance scraping the ha pair ? If the two instances are both scraping the same targets you don't need a global view. I just point my Grafana instance at the load balancer sitting in front of my parallel Prometheus instances; I've never noticed any display glitches caused by the load balancer switching between instances. The reality is that while their datasets may not be identical, they'll be "close enough". > c) When the 2 ha pair scrape the same targets, are the metric values > identical or slightly different, due to time offsets between the scrapes ? > What happens if my scrape interval for prometheus A is 15sec and Prometheus B > is 16 sec, then do i still need to dedup, since the values will be different > ? What's the right strategy to dedup the metrics ? You would only need deduplication at all if you were sending the data on to another system (Prometheus federation, Thanos, Victoriametrics, etc.). I'm currently experimenting with sending data from all of my instances (DEV, paired TEST, paired PROD) to a single Victoriametrics server. I haven't yet played with de-duplication though. -- Harald [email protected] -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/a7f2c52f-203b-4afe-becc-68f762505d80%40www.fastmail.com.
