The thing is we can implement multiple prometheus instances as well, thats no issue and will probably happen anyway. The thing is, we tried using Thanos to manage multiple prometheus servers and do deduplication, but the deduplication does not work for the collectd exporters. The problem its having i think is that we need to filter out duplicate data by exported instance, but this is not possible for Thanos since it needs predefined external labels within prometheus.
Regards, Op donderdag 25 maart 2021 om 14:15:24 UTC+1 schreef Julien Pivotto: > The way this is usually solved is by duplicating prometheus - it seems > that now you have moved the SPOF from the exporter to prometheus. > > Regards, > > On 25 Mar 06:01, robbe vaes wrote: > > Okay so, we want to have an environment using Prometheus, where we can > > monitor our servers etc with a push method rather then pull due to > network > > security aspects. As of now, we managed to set up collectd together with > > collectd exporter for prometheus. This way we can have the clients or > > servers push their data to the exporter and have prometheus scrape the > data > > from the exporter to minimize the amount of pulling that has to be done. > > The issue now is, we want to have multiple collectd exporters to improve > > HA, but the problem now is that in prometheus, when scraping both > > exporters, it takes the data from both. It shows for example for server > A > > the cpu load but it shows it twice instead of only once because its > being > > pulled from both exporters. We want to have something like this but it > > needs to have a form of deduplication so we dont have the same data > twice. > > > > > > Op donderdag 25 maart 2021 om 13:27:44 UTC+1 schreef [email protected]: > > > > > Can you describe more about what your network topology is exactly? > There > > > are a number of solutions for dealing with distributed monitoring. > > > > > > On Thu, Mar 25, 2021 at 12:45 PM robbe vaes <[email protected]> > wrote: > > > > > >> Hi, > > >> > > >> I am trying to setup a monitoring environment with Prometheus, but it > has > > >> to be using a push mechanism instead of the standard pull mechanism > > >> Prometheus uses. I was wondering what options there are to create an > > >> environment like this. It would also have to perfom data > deduplication. The > > >> main issue is that I don't want Prometheus to scrape the clients > itself, > > >> but rather that it scrapes a certain location for all the metrics and > that > > >> the clients push their metrics to that location automatically. > > >> > > >> Suggestions are very welcome! > > >> > > >> -- > > >> You received this message because you are subscribed to the Google > Groups > > >> "Prometheus Users" group. > > >> To unsubscribe from this group and stop receiving emails from it, > send an > > >> email to [email protected]. > > >> To view this discussion on the web visit > > >> > https://groups.google.com/d/msgid/prometheus-users/44d081b2-90c1-451f-ac94-efb143fdd0c0n%40googlegroups.com > > > >> < > https://groups.google.com/d/msgid/prometheus-users/44d081b2-90c1-451f-ac94-efb143fdd0c0n%40googlegroups.com?utm_medium=email&utm_source=footer > > > > >> . > > >> > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/3325df03-3ad7-4d24-be81-f0e6c4c5b07fn%40googlegroups.com > . > > > -- > Julien Pivotto > @roidelapluie > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/634b223b-467e-4356-a556-513675d79b9fn%40googlegroups.com.
