> If you have the Prometheus Operator, you have a my understanding is that > you can make your "generic sd" by creating Probe CRD's (e.g. from within > the kuma controllers). Did you consider that option too? If you are not > in kube then you don't have shared mountpoints issues.
Just for clarification, Prometheus Operator Probes are only meant for automating the setup and specialties around making blackbox probes more accessible. They are not a general-purpose way to scrape targets. On Sat, 10 Oct 2020 at 01:09, Julien Pivotto <[email protected]> wrote: > On 09 Oct 14:48, Austin Cawley-Edwards wrote: > > Hey all, > > > > Trying to bring this thread back to life and give some updates since it > > petered out. > > > > At Kuma[0], we've gone down the suggested `file_sd` route by building a > > custom binary that runs in a Prometheus sidecar to do the polling and > > writing of the sd file, which is then shared w/ Prometheus via volume > > mounts[1]. This works very well once it is set up correctly, but has > proven > > difficult for many users to get up and running as it involves a good > deal > > of low-level, non-prometheus config. It's even more difficult if they > are > > not on the team managing Prometheus infra. > > > > For those reasons, implementing another generic SD would help purely > with > > adoption, as configuring a few lines of yaml is a dramatically lower > > starting place compared to configuring a sidecar with shared volume > mounts. > > I understand and agree with the general hesitation here, especially when > it > > comes to defining entirely new "generic" SDs, and even the hesitation to > > adopt a widely-used protocol like envoy's xDS, but with the primary goal > of > > increasing adoption, what do you all think about exposing a grpc_sd that > > uses the same format as the file_sd? I'm sure much of the implementation > > could be shared and it would remove the burden of setting up the network > > hop and file sharing themselves. > > > > Lastly, in terms of continued support, we at Kuma are committed to our > > Prometheus integration and are happy to contribute and maintain this SD > > with the Promtheus community. > > > > Other related issues: > > - #6484 [2], gRPC proposal > > - #7919 [3], Kuma SD proposal > > As you might have seen, we are slowly getting new SD's into prometheus > core. We had 4 new SD's in the last weeks. > > When the issue for Kuma arrived in te bug tracker, there were some > leftover questions, like did you try the DNS-SD? Then the issue was > directly closed. > > If you have the Prometheus Operator, you have a my understanding is that > you can make your "generic sd" by creating Probe CRD's (e.g. from within > the kuma controllers). Did you consider that option too? If you are not > in kube then you don't have shared mountpoints issues. > > Should we have a generic SD, I would not like that much GRPC as would > bring many dependencies, that we already had issues with. Additionally, > we have more chances to use a more-established standard like HTTP (with > long poll). Especially since the communications only need to be one-way. > > I also think that such SD's should be limited to what they do, and that > it should be less flexible than the current file_sd. I think that any > generic remote file_sd should have constraints, such as all the labels > should be prefixes with __meta unless the __address__ label. We should > try to enforce on those SD's the same rules we have kept for the SD's > that are implemented now. > > > > > Thanks for hearing this out, > > Austin > > > > [0]: https://kuma.io/ > > [1]: > > > https://kuma.io/docs/0.5.0/policies/traffic-metrics/#configure-dataplane-discovery-by-prometheus-2 > > [2]: https://github.com/prometheus/prometheus/issues/6484 > > [3]: https://github.com/prometheus/prometheus/issues/7919 > > On Friday, December 20, 2019 at 6:55:08 PM UTC-5 > > [email protected] wrote: > > > > > On Fri, 20 Dec 2019 at 22:57, Yaroslav Skopets <[email protected]> > wrote: > > > > > >> Thank you! > > >> > > >> Do you why the discussion around `http_sd` stop ? > > >> > > > > > > That particular one looks like it started going into the weeds of > > > inventing a brand new SD, and petered out. There's enough of them out > there > > > without us creating more. > > > > > > More generally, why should we have two generic SDs? That kinda defeats > the > > > purpose of having a generic integration, which is to have one thing > that > > > all the other use cases can be handled via. > > > If you want to add a hop that goes over http it's not exactly > difficult to > > > run curl in a cronjob or have a sidecar. > > > > > > Brian > > > > > > > > >> On Fri, Dec 20, 2019 at 11:19 PM Daniel Swarbrick < > > >> [email protected]> wrote: > > >> > > >>> You might want to read this earlier thread first: > > >>> > https://groups.google.com/d/topic/prometheus-developers/3B3jBsErK5M/discussion > > >>> > > >>> I would certainly be in favour of some generic gRPC-based SD method, > > >>> such that people could implement their own gRPC services, exposing > scrape > > >>> targets from whatever funky / proprietary information sources they > use. As > > >>> I wrote in the aforementioned thread, I ended up implementing a bare > > >>> minimum fake Consul service, providing only the API endpoints used > by > > >>> Prometheus, as an SD mechanism for hosts stored in an enterprise > database > > >>> (Postgres, in my case). It has worked flawlessly for over 18 months. > > >>> > > >>> I would of course much rather not have to fake other products' APIs > > >>> though. > > >>> > > >>> On Wednesday, December 18, 2019 at 11:34:36 PM UTC+1, Yaroslav > Skopets > > >>> wrote: > > >>>> > > >>>> Proposal > > >>>> > > >>>> - In the context of a Service Mesh, there is always a notion of > a > > >>>> Control Plane that coordinates telemetry among other things > > >>>> - It would be perfect for overall user experience if a Control > > >>>> Plane could also become a source of scrape targets > (specifically, for > > >>>> collecting metrics out of side-car proxies) > > >>>> - As a way to achieve that, we would like to propose to > introduce a > > >>>> gRPC interface for SD that Control Planes could implement > > >>>> - The choice of gRPC will enable effective server-side streaming > of > > >>>> SD information, and it will remove the requirement on users to > think > > >>>> upfront about refresh intervals they cannot choose reliably > anyway > > >>>> - There is already a well-defined, battle-tested and widely used > > >>>> protocol for delivering discovery information - Envoy xDS API > > >>>> < > https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol> > > >>>> - Based on all the above, we would like to ask if Prometheus > > >>>> community is open for such a contribution > > >>>> > > >>>> -- > > >>> You received this message because you are subscribed to the Google > > >>> Groups "Prometheus Developers" group. > > >>> To unsubscribe from this group and stop receiving emails from it, > send > > >>> an email to [email protected]. > > >>> To view this discussion on the web visit > > >>> > https://groups.google.com/d/msgid/prometheus-developers/cb86c927-8642-45b0-bf33-00d14e09aec7%40googlegroups.com > > >>> < > https://groups.google.com/d/msgid/prometheus-developers/cb86c927-8642-45b0-bf33-00d14e09aec7%40googlegroups.com?utm_medium=email&utm_source=footer > > > > >>> . > > >>> > > >> > > >> > > >> -- > > >> Best regards, > > >> Yaroslav Skopets > > >> > > >> -- > > >> You received this message because you are subscribed to the Google > Groups > > >> "Prometheus Developers" group. > > >> To unsubscribe from this group and stop receiving emails from it, > send an > > >> email to [email protected]. > > >> > > > To view this discussion on the web visit > > >> > https://groups.google.com/d/msgid/prometheus-developers/CAJsMHLKWheCeGT_MiEGjrxbPpR8_t-nFmCgu4Vps5OE66-mekQ%40mail.gmail.com > > >> < > https://groups.google.com/d/msgid/prometheus-developers/CAJsMHLKWheCeGT_MiEGjrxbPpR8_t-nFmCgu4Vps5OE66-mekQ%40mail.gmail.com?utm_medium=email&utm_source=footer > > > > >> . > > >> > > > > > > > > > -- > > > Brian Brazil > > > www.robustperception.io > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/8894963b-f578-4898-a3ff-d0b963a3e3a4n%40googlegroups.com > . > > > -- > Julien Pivotto > @roidelapluie > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/20201009230855.GA1267035%40oxygen > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAOs1UmwNTPKvw4T5p0Omsdfkby3tsfaoBYM%2B%3D4h6DU0yx%3D1zBA%40mail.gmail.com.
