I would probably skip federation and remote write with that setup and use Thanos to create a single pane view of all of them.
On Mon, Jul 18, 2022, 7:00 PM tejaswini vadlamudi <[email protected]> wrote: > Hello Stuart, > > I have the 4 Prometheus instances in the same cluster. > > - Instance-1, monitoring k8s & cadvisor > - Instance-2, monitoring workload-1 in namespace-1 > - Instance-3, monitoring workload-2 in namespace-2 > - Instance-4 is the central one collecting metrics from all 3 > instances (for global querying and alerting). not sure if the federation is > a good fit for this sort of deployment pattern. > > > Thanks, Teja > > > On Monday, July 18, 2022 at 6:49:45 PM UTC+2 Stuart Clark wrote: > >> On 18/07/2022 17:21, tejaswini vadlamudi wrote: >> > Can someone point me to the advantages of using remote-write over >> > federation? >> > I understand that remote-write is more of a standard interface in the >> > monitoring domain. >> > Are there any handy performance measurements that were >> observed/recorded? >> > >> They are really quite different. >> >> Federation is a way of pulling data from a remote Prometheus into >> (generally) a local one. The puller gets to choose how often to pull >> data and what data to fetch. If the puller can't fetch the data for any >> reason (local/remote outage, network issues, etc.) there will be gaps. >> >> Remote write is a way of pushing data from a Prometheus server to >> "something else", which could be another Prometheus or one of the many >> things which implement the API (e.g. various databases, Thanos, custom >> analytics tools, etc.). For these you get all the data (basically as >> soon as it has been scraped) with the ability to do filtering via >> relabling. If there is an outage/disconnect data will be queued for a >> while (too long and things will get lost) so small issues can be handled >> transparently. >> >> So you have a difference in what data you get - either all (filtered) >> data or data on a schedule (so in effect a form of built-in >> downsampling), and who controls that - either the data source Prometheus >> or the destination. >> >> Which is "better" depends on what you are trying to achieve and the >> constraints you might have (for example difficulties with accepting >> network connections or data storage/transfer limits). Don't forget the >> organisation differences too - for remote write adding/changing a >> destination (or filter rules) needs changes to every data source >> Prometheus where federation is purely controlled at the other end, which >> might be a good or bad thing depending on team responsibilities/timings. >> >> -- >> Stuart Clark >> >> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/a174584b-8f1d-4ab8-bcda-bfae9401af0en%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/a174584b-8f1d-4ab8-bcda-bfae9401af0en%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmorLg2%3DOGzKP8ydVBy715FC%3D4ZAZ-1DJ4U3EL2EQnGk5g%40mail.gmail.com.
