@Stuart: I agree with most of the ideas you say :-) I see remote-write as
the most appropriate metrics forwarding for my deployment use case.
Using federation is not good in terms of interface
standardization, HA of monitoring stack, and feature support.
For the above case, I have functions and a dedicated set of
engineers who own such workload to query individual instances, and the
global instance is used as centralized monitoring.
I was looking at this
<https://github.com/prometheus/prometheus/issues/5666>closed bug, raised on
Prometheus in the 2019 Summer. To my understanding, there are performance
issues with remote-write but most of them are resolved and the community
sees remote-write to perform better when compared to the federation. Am I
thinking correctly?
Could you clarify the performance comparison between
remote-write and federation?
/Teja
On Tuesday, July 19, 2022 at 5:02:11 PM UTC+2 Stuart Clark wrote:
> On 19/07/2022 13:24, tejaswini vadlamudi wrote:
> > @Ben: Makes a point, but getting Thanos or Cortex into the picture
> > could be a way forward after some time. For now, do you think it is
> > good enough to use remote-write instead of federation? From a
> > performance and resource consumption POV, do you see remote-write as
> > the way-forward?
> >
> With remote write you could use agent mode, so you don't have to have
> local storage other than for the destination instance.
>
> However again it depends what you are trying to achieve and why you have
> suggested having four instances. Are you wanting to query all four
> instances or only the "global" one? Are you wanting to copy all data to
> the "global" instance or only some metrics? Every data point, or only at
> a lower frequency?
>
> If you are intending to copy all data (both metrics & data points) that
> leans towards remote write as federation works differently. But in that
> case there doesn't seem to be any advantage in having the extra three
> instances at all (unless you are intending on doing local querying,
> alerting or recording rules) - so I'd just have a single instance that
> scrapes all namespaces.
>
> Alternatively if you are needing to have separate instances with local
> storage/querying then I'd probably not look to copy all the data to the
> "global" instance (which just doubles storage and memory usage) and
> either use remote write for a much smaller subset of metrics, federation
> with a slower scrape rate/reduced set of metrics, or as Ben suggested
> something like Thanos (other options exist as well) to do away with the
> fourth instance entirely and distribute the queries to the individual
> instances instead.
>
> Maybe if you could explain a bit about what the design is hoping to
> achieve it would help us advise better?
>
> --
> Stuart Clark
>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/e58b5287-609d-48ab-8f20-4a6744d29bedn%40googlegroups.com.
