Yeah, that would be nice too but would not solve the original use case of this thread, which was to provide a generic option that did not need to co-locate data next to Prometheus via a filesystem. I think xDS is still a good fit for a generic option, but we would need to define an API.
If we were to just directly use the structure of file_sd, then we're not creating anything new, just joining the xDS discovery protocol with an existing data structure. What are you thinking? Do you want to open a new thread to see if anyone would find an envoy_ds valuable? Best, Austin On Thursday, February 18, 2021 at 4:45:05 PM UTC-5 Julien Pivotto wrote: > Yeah my impression was that eDS would fit here, what envoy uses for > endpoints. > > > On 18 Feb 13:31, Austin Cawley-Edwards wrote: > > Quick follow-up: I think an "envoy discovery" method to scrape > > Envoy-specific metrics might be worthwhile, but I don't think it fits in > > the same conversation as this one. > > > > On Thursday, February 18, 2021 at 4:12:39 PM UTC-5 Austin Cawley-Edwards > > wrote: > > > > > Hey Julien, > > > > > > I think the disconnect here is that xDS is just a protocol used for > many > > > different Envoy services[1], and does not define any actual data that > could > > > be mapped to a target itself. It is purely meant as a base that is to > be > > > extended. Can you clarify what you mean by "current xds api's"? Do you > mean > > > just discovering the current Envoys in a cluster? Or do you mean > trying to > > > discover all the various Listeners/ VirtualHosts[2] (which may or may > not > > > expose a metrics endpoint) from Envoy? Envoy does not implement > anything > > > about metrics of the underlying services it proxies, i.e. there is no > way > > > to say "these services are good to scrape" to my knowledge. > > > > > > For HTTP vs gRPC, I'm not sure why we wouldn't want to support both as > the > > > xDS protocol defines support for both, but gRPC is more common. > > > > > > Best, > > > Austin > > > > > > [1]: > https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/service > > > [2]: > > > > https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol#resource-types > > > > > > On Thursday, February 18, 2021 at 3:20:18 PM UTC-5 Julien Pivotto > wrote: > > > > > >> On 18 Feb 12:10, Austin Cawley-Edwards wrote: > > >> > Hey Julien, > > >> > > > >> > I've finished up a *rough *proof of concept for what xDS support in > > >> > Prometheus could look like [1]. I think it shows the main areas > without > > >> > going too over-the-top. In summary, the two main pieces to the > > >> integration > > >> > seem to be an xDS API definition[2] and an xDS client/ discovery > that > > >> uses > > >> > an implementation of this API [3] (HTTP discovery implementation > > >> here[4]). > > >> > For the xDS API, envoy does not have a native concept of metrics/ > > >> > monitoring, so we need to define this data structure -- I've > currently > > >> > sketched one out (taking from Kuma) that is very close to a > > >> targetgroup[5]. > > >> > > > >> > A few pros: > > >> > - support for both HTTP and gRPC API implementations > > >> > - xDS protocol is well documented > > >> > - versioned API will make modifications possible > > >> > > > >> > A few cons: > > >> > - complexity of the protocol for people who have not used it > > >> > - added dependencies on gRPC and envoy + build processes additions > for > > >> > compiling protobuf > > >> > - xDS adds some overhead for implementors of this API, though this > > >> seems > > >> > like an "advanced" use case where that is ok > > >> > > > >> > Please let me know what you think and if you think it's feasible to > > >> move > > >> > forward with xDS or if we should go back to discussions of a > simpler > > >> HTTP/ > > >> > gRPC approach. > > >> > > >> Thanks, here are my initial thoughts: > > >> > > >> - We should not *add* an API, but discover everything on the xDS API. > > >> The meta labels created would then enable to filter what is needed > and > > >> what is not. It is not the responsibility of the service discovery. > > >> Monitoring is not just about whitebox monitoring, the scope can be > > >> much larger. Also, I want something that could be used out of the box > > >> with current xds api's. > > >> - We need to settle by using grpc or http, we would not support both. > > >> Which one is the most used? > > >> > > >> So overall I don't think there should be any overhead for the > > >> implementors of the API, since my understating is that there are out > of > > >> the box implementations out there, like envoy? What am I missing? > > >> > > >> > > > >> > Best, > > >> > Austin > > >> > > > >> > [1]: > > >> > https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds > > >> > [2]: > > >> > > > >> > https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#xds-api-implementation > > > >> > [3]: > > >> > > > >> > https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#discovery-scraping > > > >> > [4]: > > >> > > > >> > https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/http_discovery.go > > > >> > [5]: > > >> > > > >> > https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/api/v1alpha1/mads.proto > > > >> > On Saturday, February 6, 2021 at 7:04:21 PM UTC-5 Austin > Cawley-Edwards > > >> > wrote: > > >> > > > >> > > Hey Julien, > > >> > > > > >> > > Thanks for reaching back out about this. I’ve been on the midst > of a > > >> job > > >> > > transition and have not made any progress, but that is finally > fully > > >> done > > >> > > and I can try to move this along (though it will still be > slowly). > > >> I’ll get > > >> > > back to research and bring it up at the next Kuma community > meeting > > >> in two > > >> > > weeks, and then report back. > > >> > > > > >> > > Best, > > >> > > Austin > > >> > > > > >> > > On Sat, Feb 6, 2021 at 3:36 PM Julien Pivotto < > > >> [email protected]> > > >> > > wrote: > > >> > > > > >> > >> Hello Austin, > > >> > >> > > >> > >> Is there any progress here? I am really willing to learn more > about > > >> xds > > >> > >> and how it would fit here. > > >> > >> > > >> > >> On 29 Oct 11:09, Austin Cawley-Edwards wrote: > > >> > >> > Perfect, I'll get started collecting that data. Thanks for the > > >> > >> direction. > > >> > >> > > > >> > >> > Best, > > >> > >> > Austin > > >> > >> > > > >> > >> > On Wed, Oct 28, 2020 at 1:07 PM Julien Pivotto < > > >> [email protected] > > >> > >> > > > >> > >> > wrote: > > >> > >> > > > >> > >> > > On 28 Oct 08:28, Austin Cawley-Edwards wrote: > > >> > >> > > > Hey Julien, > > >> > >> > > > > > >> > >> > > > Following up on if you've had time to think about how we > can > > >> move > > >> > >> > > forward > > >> > >> > > > on one of these paths. I haven't been able to find any > formal > > >> > >> proposals > > >> > >> > > for > > >> > >> > > > the other recently merged SDs like Eureka besides this > > >> comment[1]. > > >> > >> Would > > >> > >> > > it > > >> > >> > > > be valuable for us to work on something similar for > envoy's > > >> xDS in > > >> > >> the > > >> > >> > > > meantime? > > >> > >> > > > > >> > >> > > > > >> > >> > > Let's be honest that recent service discoveries are 'small', > and > > >> that > > >> > >> > > big ones are quite old now. > > >> > >> > > > > >> > >> > > I still need to gather intel on xDS; but let's start by > > >> collecting > > >> > >> data: > > >> > >> > > > > >> > >> > > - What would the configuration look like (does it need > multiple > > >> > >> > > `roles`) > > >> > >> > > - Which meta labels could we have exposed > > >> > >> > > - How would we set the address > > >> > >> > > > > >> > >> > > That hopefully should not be too much work and then we can > see > > >> how/if > > >> > >> we > > >> > >> > > move forward with this. > > >> > >> > > > > >> > >> > > > > > >> > >> > > > Thanks, > > >> > >> > > > Austin > > >> > >> > > > > > >> > >> > > > [1]: > > >> > >> > > > > > >> > >> > > > > >> > >> > > >> > https://github.com/prometheus/prometheus/pull/3369#issuecomment-667344846 > > >> > >> > > > > > >> > >> > > > On Wednesday, October 21, 2020 at 6:49:59 PM UTC-4 Austin > > >> > >> Cawley-Edwards > > >> > >> > > > wrote: > > >> > >> > > > > > >> > >> > > > > That makes sense to me, and would be nice to build of an > > >> existing > > >> > >> > > > > protocol, and agree it sounds like a large effort. I’m > > >> pretty new > > >> > >> to > > >> > >> > > xDS as > > >> > >> > > > > well, though we have many people on the Kuma team who > are > > >> quite > > >> > >> > > > > knowledgeable — I’d be happy to set up a call/ bring > them in > > >> to > > >> > >> help > > >> > >> > > size > > >> > >> > > > > the work if you’d be open to that. > > >> > >> > > > > > > >> > >> > > > > And understand + agree on the configuration approach for > the > > >> extra > > >> > >> > > layer. > > >> > >> > > > > The main difficulty for users is configuring the file_sd > + > > >> > >> sidecar, > > >> > >> > > > > relabel_configs should be fairly straightforward and > > >> accessible. > > >> > >> > > > > > > >> > >> > > > > Austin > > >> > >> > > > > > > >> > >> > > > > On Wed, Oct 21, 2020 at 6:37 PM Julien Pivotto < > > >> > >> > > [email protected]> > > >> > >> > > > > wrote: > > >> > >> > > > > > > >> > >> > > > >> On 21 Oct 15:12, Austin Cawley-Edwards wrote: > > >> > >> > > > >> > Hey all, > > >> > >> > > > >> > > > >> > >> > > > >> > I've gathered up some research about our efforts with > the > > >> DNS > > >> > >> SD > > >> > >> > > > >> option, > > >> > >> > > > >> > please let me know if I've overlooked something. > > >> > >> > > > >> > > >> > >> > > > >> I have been looking on my side too - I think that what > the > > >> option > > >> > >> > > that's > > >> > >> > > > >> more likely on the table would not to have a "generic" > > >> > >> implementation, > > >> > >> > > > >> but a xDS protocol service discovery. In that case, it > > >> would not > > >> > >> be an > > >> > >> > > > >> issue to have that in Prometheus (even the gRPC > version). > > >> > >> > > > >> > > >> > >> > > > >> However, the extra layer that kuma is offering in the > SD > > >> should > > >> > >> be > > >> > >> > > done > > >> > >> > > > >> at the Prometheus configuration level - e.g. by the > user, > > >> in the > > >> > >> > > > >> relabel_configs. > > >> > >> > > > >> > > >> > >> > > > >> I think I'd like to better understand xDS but it seems > that > > >> it > > >> > >> could > > >> > >> > > be > > >> > >> > > > >> good option for a service discovery in Prometheus, at > least > > >> at > > >> > >> first > > >> > >> > > > >> sight. > > >> > >> > > > >> > > >> > >> > > > >> We are looking at a huge amount of work here I think. > > >> > >> > > > >> > > >> > >> > > > >> -- > > >> > >> > > > >> Julien Pivotto > > >> > >> > > > >> @roidelapluie > > >> > >> > > > >> > > >> > >> > > > > > > >> > >> > > > > > >> > >> > > > -- > > >> > >> > > > You received this message because you are subscribed to > the > > >> Google > > >> > >> > > Groups "Prometheus Developers" group. > > >> > >> > > > To unsubscribe from this group and stop receiving emails > from > > >> it, > > >> > >> send > > >> > >> > > an email to [email protected]. > > >> > >> > > > To view this discussion on the web visit > > >> > >> > > > > >> > >> > > >> > https://groups.google.com/d/msgid/prometheus-developers/89922987-42e4-4807-9956-5590cb2de558n%40googlegroups.com > > > >> > >> > > . > > >> > >> > > > > >> > >> > > > > >> > >> > > -- > > >> > >> > > Julien Pivotto > > >> > >> > > @roidelapluie > > >> > >> > > > > >> > >> > > > >> > >> > -- > > >> > >> > You received this message because you are subscribed to the > Google > > >> > >> Groups "Prometheus Developers" group. > > >> > >> > To unsubscribe from this group and stop receiving emails from > it, > > >> send > > >> > >> an email to [email protected]. > > >> > >> > To view this discussion on the web visit > > >> > >> > > >> > https://groups.google.com/d/msgid/prometheus-developers/CALGL%2BUBExJmaD9rzXWnTJf5h6my3cbYrrh-uQjCD9U9PGoW-Aw%40mail.gmail.com > > > >> > >> . > > >> > >> > > >> > >> -- > > >> > >> Julien Pivotto > > >> > >> @roidelapluie > > >> > >> > > >> > > > > >> > > > >> > -- > > >> > You received this message because you are subscribed to the Google > > >> Groups "Prometheus Developers" group. > > >> > To unsubscribe from this group and stop receiving emails from it, > send > > >> an email to [email protected]. > > >> > To view this discussion on the web visit > > >> > https://groups.google.com/d/msgid/prometheus-developers/7db91c8f-33e2-4253-8cc6-daf596a1d1ban%40googlegroups.com. > > > > >> > > >> > > >> > > >> -- > > >> Julien Pivotto > > >> @roidelapluie > > >> > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/47fdf607-fac5-41bd-94ea-683fe67e38bfn%40googlegroups.com > . > > > -- > Julien Pivotto > @roidelapluie > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/4deb5ef0-7acd-4746-8627-f10abc7bd846n%40googlegroups.com.
