Hello, We have 4 stand-alone prometheus instances spread across different data-centers.
We have one of them federating the other 3, with externalLabels set to "dc:<datacenter name>". This setup is mostly for the ease of graphing, querying and alerting. For example, if I want to see the different latencies to an application from different data-centers, I'm able to do it all from the primary prometheus instance. The issue we're hitting is that the "dc" label is only available for the 3 federated instances, not the "primary" instance. This makes querying/alerting harder, because the "dc" label is not consistently available. We're essentially having to add a separate query for the "primary" instance metrics, in order to see all 4. We're not sure what the best solution is to this. We could add a 5th node, and federate the other 4 there, but it seems a little overkill. An ideal solution would be to have the "dc" label added to the metrics from the primary node, but that would require modifying all the scrape configs. It's possible, but again, seems like there should be a better way. Any suggestions greatly appreciated. Thanks, Joe -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/917ec937-6d27-4a75-8abc-e013ae9e70bcn%40googlegroups.com.
