On 18 Feb 13:52, Austin Cawley-Edwards wrote: > Yeah, that would be nice too but would not solve the original use case of > this thread, which was to provide a generic option that did not need to > co-locate data next to Prometheus via a filesystem. I think xDS is still a > good fit for a generic option, but we would need to define an API. > > If we were to just directly use the structure of file_sd, then we're not > creating anything new, just joining the xDS discovery protocol with an > existing data structure. > > > What are you thinking? Do you want to open a new thread to see if anyone > would find an envoy_ds valuable?
Well I think we should have a xds package which would have envoy_sd, and possibly kuma_sd, but stay scoped and tight to what exists _now_. Having them in a xds package enables us to reuse code. I am really not willing to have file_sd over xds. Note: We use the same pattern in zookeeper, where one discovery package contains 2 sd mechanisms. > > Best, > Austin > > On Thursday, February 18, 2021 at 4:45:05 PM UTC-5 Julien Pivotto wrote: > > > Yeah my impression was that eDS would fit here, what envoy uses for > > endpoints. > > > > > > On 18 Feb 13:31, Austin Cawley-Edwards wrote: > > > Quick follow-up: I think an "envoy discovery" method to scrape > > > Envoy-specific metrics might be worthwhile, but I don't think it fits in > > > the same conversation as this one. > > > > > > On Thursday, February 18, 2021 at 4:12:39 PM UTC-5 Austin Cawley-Edwards > > > wrote: > > > > > > > Hey Julien, > > > > > > > > I think the disconnect here is that xDS is just a protocol used for > > many > > > > different Envoy services[1], and does not define any actual data that > > could > > > > be mapped to a target itself. It is purely meant as a base that is to > > be > > > > extended. Can you clarify what you mean by "current xds api's"? Do you > > mean > > > > just discovering the current Envoys in a cluster? Or do you mean > > trying to > > > > discover all the various Listeners/ VirtualHosts[2] (which may or may > > not > > > > expose a metrics endpoint) from Envoy? Envoy does not implement > > anything > > > > about metrics of the underlying services it proxies, i.e. there is no > > way > > > > to say "these services are good to scrape" to my knowledge. > > > > > > > > For HTTP vs gRPC, I'm not sure why we wouldn't want to support both as > > the > > > > xDS protocol defines support for both, but gRPC is more common. > > > > > > > > Best, > > > > Austin > > > > > > > > [1]: > > https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/service > > > > [2]: > > > > > > https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol#resource-types > > > > > > > > On Thursday, February 18, 2021 at 3:20:18 PM UTC-5 Julien Pivotto > > wrote: > > > > > > > >> On 18 Feb 12:10, Austin Cawley-Edwards wrote: > > > >> > Hey Julien, > > > >> > > > > >> > I've finished up a *rough *proof of concept for what xDS support in > > > >> > Prometheus could look like [1]. I think it shows the main areas > > without > > > >> > going too over-the-top. In summary, the two main pieces to the > > > >> integration > > > >> > seem to be an xDS API definition[2] and an xDS client/ discovery > > that > > > >> uses > > > >> > an implementation of this API [3] (HTTP discovery implementation > > > >> here[4]). > > > >> > For the xDS API, envoy does not have a native concept of metrics/ > > > >> > monitoring, so we need to define this data structure -- I've > > currently > > > >> > sketched one out (taking from Kuma) that is very close to a > > > >> targetgroup[5]. > > > >> > > > > >> > A few pros: > > > >> > - support for both HTTP and gRPC API implementations > > > >> > - xDS protocol is well documented > > > >> > - versioned API will make modifications possible > > > >> > > > > >> > A few cons: > > > >> > - complexity of the protocol for people who have not used it > > > >> > - added dependencies on gRPC and envoy + build processes additions > > for > > > >> > compiling protobuf > > > >> > - xDS adds some overhead for implementors of this API, though this > > > >> seems > > > >> > like an "advanced" use case where that is ok > > > >> > > > > >> > Please let me know what you think and if you think it's feasible to > > > >> move > > > >> > forward with xDS or if we should go back to discussions of a > > simpler > > > >> HTTP/ > > > >> > gRPC approach. > > > >> > > > >> Thanks, here are my initial thoughts: > > > >> > > > >> - We should not *add* an API, but discover everything on the xDS API. > > > >> The meta labels created would then enable to filter what is needed > > and > > > >> what is not. It is not the responsibility of the service discovery. > > > >> Monitoring is not just about whitebox monitoring, the scope can be > > > >> much larger. Also, I want something that could be used out of the box > > > >> with current xds api's. > > > >> - We need to settle by using grpc or http, we would not support both. > > > >> Which one is the most used? > > > >> > > > >> So overall I don't think there should be any overhead for the > > > >> implementors of the API, since my understating is that there are out > > of > > > >> the box implementations out there, like envoy? What am I missing? > > > >> > > > >> > > > > >> > Best, > > > >> > Austin > > > >> > > > > >> > [1]: > > > >> > > https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds > > > >> > [2]: > > > >> > > > > >> > > https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#xds-api-implementation > > > > > >> > [3]: > > > >> > > > > >> > > https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#discovery-scraping > > > > > >> > [4]: > > > >> > > > > >> > > https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/http_discovery.go > > > > > >> > [5]: > > > >> > > > > >> > > https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/api/v1alpha1/mads.proto > > > > > >> > On Saturday, February 6, 2021 at 7:04:21 PM UTC-5 Austin > > Cawley-Edwards > > > >> > wrote: > > > >> > > > > >> > > Hey Julien, > > > >> > > > > > >> > > Thanks for reaching back out about this. I’ve been on the midst > > of a > > > >> job > > > >> > > transition and have not made any progress, but that is finally > > fully > > > >> done > > > >> > > and I can try to move this along (though it will still be > > slowly). > > > >> I’ll get > > > >> > > back to research and bring it up at the next Kuma community > > meeting > > > >> in two > > > >> > > weeks, and then report back. > > > >> > > > > > >> > > Best, > > > >> > > Austin > > > >> > > > > > >> > > On Sat, Feb 6, 2021 at 3:36 PM Julien Pivotto < > > > >> [email protected]> > > > >> > > wrote: > > > >> > > > > > >> > >> Hello Austin, > > > >> > >> > > > >> > >> Is there any progress here? I am really willing to learn more > > about > > > >> xds > > > >> > >> and how it would fit here. > > > >> > >> > > > >> > >> On 29 Oct 11:09, Austin Cawley-Edwards wrote: > > > >> > >> > Perfect, I'll get started collecting that data. Thanks for the > > > >> > >> direction. > > > >> > >> > > > > >> > >> > Best, > > > >> > >> > Austin > > > >> > >> > > > > >> > >> > On Wed, Oct 28, 2020 at 1:07 PM Julien Pivotto < > > > >> [email protected] > > > >> > >> > > > > >> > >> > wrote: > > > >> > >> > > > > >> > >> > > On 28 Oct 08:28, Austin Cawley-Edwards wrote: > > > >> > >> > > > Hey Julien, > > > >> > >> > > > > > > >> > >> > > > Following up on if you've had time to think about how we > > can > > > >> move > > > >> > >> > > forward > > > >> > >> > > > on one of these paths. I haven't been able to find any > > formal > > > >> > >> proposals > > > >> > >> > > for > > > >> > >> > > > the other recently merged SDs like Eureka besides this > > > >> comment[1]. > > > >> > >> Would > > > >> > >> > > it > > > >> > >> > > > be valuable for us to work on something similar for > > envoy's > > > >> xDS in > > > >> > >> the > > > >> > >> > > > meantime? > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > Let's be honest that recent service discoveries are 'small', > > and > > > >> that > > > >> > >> > > big ones are quite old now. > > > >> > >> > > > > > >> > >> > > I still need to gather intel on xDS; but let's start by > > > >> collecting > > > >> > >> data: > > > >> > >> > > > > > >> > >> > > - What would the configuration look like (does it need > > multiple > > > >> > >> > > `roles`) > > > >> > >> > > - Which meta labels could we have exposed > > > >> > >> > > - How would we set the address > > > >> > >> > > > > > >> > >> > > That hopefully should not be too much work and then we can > > see > > > >> how/if > > > >> > >> we > > > >> > >> > > move forward with this. > > > >> > >> > > > > > >> > >> > > > > > > >> > >> > > > Thanks, > > > >> > >> > > > Austin > > > >> > >> > > > > > > >> > >> > > > [1]: > > > >> > >> > > > > > > >> > >> > > > > > >> > >> > > > >> > > https://github.com/prometheus/prometheus/pull/3369#issuecomment-667344846 > > > >> > >> > > > > > > >> > >> > > > On Wednesday, October 21, 2020 at 6:49:59 PM UTC-4 Austin > > > >> > >> Cawley-Edwards > > > >> > >> > > > wrote: > > > >> > >> > > > > > > >> > >> > > > > That makes sense to me, and would be nice to build of an > > > >> existing > > > >> > >> > > > > protocol, and agree it sounds like a large effort. I’m > > > >> pretty new > > > >> > >> to > > > >> > >> > > xDS as > > > >> > >> > > > > well, though we have many people on the Kuma team who > > are > > > >> quite > > > >> > >> > > > > knowledgeable — I’d be happy to set up a call/ bring > > them in > > > >> to > > > >> > >> help > > > >> > >> > > size > > > >> > >> > > > > the work if you’d be open to that. > > > >> > >> > > > > > > > >> > >> > > > > And understand + agree on the configuration approach for > > the > > > >> extra > > > >> > >> > > layer. > > > >> > >> > > > > The main difficulty for users is configuring the file_sd > > + > > > >> > >> sidecar, > > > >> > >> > > > > relabel_configs should be fairly straightforward and > > > >> accessible. > > > >> > >> > > > > > > > >> > >> > > > > Austin > > > >> > >> > > > > > > > >> > >> > > > > On Wed, Oct 21, 2020 at 6:37 PM Julien Pivotto < > > > >> > >> > > [email protected]> > > > >> > >> > > > > wrote: > > > >> > >> > > > > > > > >> > >> > > > >> On 21 Oct 15:12, Austin Cawley-Edwards wrote: > > > >> > >> > > > >> > Hey all, > > > >> > >> > > > >> > > > > >> > >> > > > >> > I've gathered up some research about our efforts with > > the > > > >> DNS > > > >> > >> SD > > > >> > >> > > > >> option, > > > >> > >> > > > >> > please let me know if I've overlooked something. > > > >> > >> > > > >> > > > >> > >> > > > >> I have been looking on my side too - I think that what > > the > > > >> option > > > >> > >> > > that's > > > >> > >> > > > >> more likely on the table would not to have a "generic" > > > >> > >> implementation, > > > >> > >> > > > >> but a xDS protocol service discovery. In that case, it > > > >> would not > > > >> > >> be an > > > >> > >> > > > >> issue to have that in Prometheus (even the gRPC > > version). > > > >> > >> > > > >> > > > >> > >> > > > >> However, the extra layer that kuma is offering in the > > SD > > > >> should > > > >> > >> be > > > >> > >> > > done > > > >> > >> > > > >> at the Prometheus configuration level - e.g. by the > > user, > > > >> in the > > > >> > >> > > > >> relabel_configs. > > > >> > >> > > > >> > > > >> > >> > > > >> I think I'd like to better understand xDS but it seems > > that > > > >> it > > > >> > >> could > > > >> > >> > > be > > > >> > >> > > > >> good option for a service discovery in Prometheus, at > > least > > > >> at > > > >> > >> first > > > >> > >> > > > >> sight. > > > >> > >> > > > >> > > > >> > >> > > > >> We are looking at a huge amount of work here I think. > > > >> > >> > > > >> > > > >> > >> > > > >> -- > > > >> > >> > > > >> Julien Pivotto > > > >> > >> > > > >> @roidelapluie > > > >> > >> > > > >> > > > >> > >> > > > > > > > >> > >> > > > > > > >> > >> > > > -- > > > >> > >> > > > You received this message because you are subscribed to > > the > > > >> Google > > > >> > >> > > Groups "Prometheus Developers" group. > > > >> > >> > > > To unsubscribe from this group and stop receiving emails > > from > > > >> it, > > > >> > >> send > > > >> > >> > > an email to [email protected]. > > > >> > >> > > > To view this discussion on the web visit > > > >> > >> > > > > > >> > >> > > > >> > > https://groups.google.com/d/msgid/prometheus-developers/89922987-42e4-4807-9956-5590cb2de558n%40googlegroups.com > > > > > >> > >> > > . > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > -- > > > >> > >> > > Julien Pivotto > > > >> > >> > > @roidelapluie > > > >> > >> > > > > > >> > >> > > > > >> > >> > -- > > > >> > >> > You received this message because you are subscribed to the > > Google > > > >> > >> Groups "Prometheus Developers" group. > > > >> > >> > To unsubscribe from this group and stop receiving emails from > > it, > > > >> send > > > >> > >> an email to [email protected]. > > > >> > >> > To view this discussion on the web visit > > > >> > >> > > > >> > > https://groups.google.com/d/msgid/prometheus-developers/CALGL%2BUBExJmaD9rzXWnTJf5h6my3cbYrrh-uQjCD9U9PGoW-Aw%40mail.gmail.com > > > > > >> > >> . > > > >> > >> > > > >> > >> -- > > > >> > >> Julien Pivotto > > > >> > >> @roidelapluie > > > >> > >> > > > >> > > > > > >> > > > > >> > -- > > > >> > You received this message because you are subscribed to the Google > > > >> Groups "Prometheus Developers" group. > > > >> > To unsubscribe from this group and stop receiving emails from it, > > send > > > >> an email to [email protected]. > > > >> > To view this discussion on the web visit > > > >> > > https://groups.google.com/d/msgid/prometheus-developers/7db91c8f-33e2-4253-8cc6-daf596a1d1ban%40googlegroups.com. > > > > > > > >> > > > >> > > > >> > > > >> -- > > > >> Julien Pivotto > > > >> @roidelapluie > > > >> > > > > > > > > > > -- > > > You received this message because you are subscribed to the Google > > Groups "Prometheus Developers" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [email protected]. > > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/prometheus-developers/47fdf607-fac5-41bd-94ea-683fe67e38bfn%40googlegroups.com > > . > > > > > > -- > > Julien Pivotto > > @roidelapluie > > > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/4deb5ef0-7acd-4746-8627-f10abc7bd846n%40googlegroups.com. -- Julien Pivotto @roidelapluie -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/20210218215856.GA277849%40oxygen.
