Quick follow-up: I think an "envoy discovery" method to scrape Envoy-specific metrics might be worthwhile, but I don't think it fits in the same conversation as this one.
On Thursday, February 18, 2021 at 4:12:39 PM UTC-5 Austin Cawley-Edwards wrote: > Hey Julien, > > I think the disconnect here is that xDS is just a protocol used for many > different Envoy services[1], and does not define any actual data that could > be mapped to a target itself. It is purely meant as a base that is to be > extended. Can you clarify what you mean by "current xds api's"? Do you mean > just discovering the current Envoys in a cluster? Or do you mean trying to > discover all the various Listeners/ VirtualHosts[2] (which may or may not > expose a metrics endpoint) from Envoy? Envoy does not implement anything > about metrics of the underlying services it proxies, i.e. there is no way > to say "these services are good to scrape" to my knowledge. > > For HTTP vs gRPC, I'm not sure why we wouldn't want to support both as the > xDS protocol defines support for both, but gRPC is more common. > > Best, > Austin > > [1]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/service > [2]: > https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol#resource-types > > On Thursday, February 18, 2021 at 3:20:18 PM UTC-5 Julien Pivotto wrote: > >> On 18 Feb 12:10, Austin Cawley-Edwards wrote: >> > Hey Julien, >> > >> > I've finished up a *rough *proof of concept for what xDS support in >> > Prometheus could look like [1]. I think it shows the main areas without >> > going too over-the-top. In summary, the two main pieces to the >> integration >> > seem to be an xDS API definition[2] and an xDS client/ discovery that >> uses >> > an implementation of this API [3] (HTTP discovery implementation >> here[4]). >> > For the xDS API, envoy does not have a native concept of metrics/ >> > monitoring, so we need to define this data structure -- I've currently >> > sketched one out (taking from Kuma) that is very close to a >> targetgroup[5]. >> > >> > A few pros: >> > - support for both HTTP and gRPC API implementations >> > - xDS protocol is well documented >> > - versioned API will make modifications possible >> > >> > A few cons: >> > - complexity of the protocol for people who have not used it >> > - added dependencies on gRPC and envoy + build processes additions for >> > compiling protobuf >> > - xDS adds some overhead for implementors of this API, though this >> seems >> > like an "advanced" use case where that is ok >> > >> > Please let me know what you think and if you think it's feasible to >> move >> > forward with xDS or if we should go back to discussions of a simpler >> HTTP/ >> > gRPC approach. >> >> Thanks, here are my initial thoughts: >> >> - We should not *add* an API, but discover everything on the xDS API. >> The meta labels created would then enable to filter what is needed and >> what is not. It is not the responsibility of the service discovery. >> Monitoring is not just about whitebox monitoring, the scope can be >> much larger. Also, I want something that could be used out of the box >> with current xds api's. >> - We need to settle by using grpc or http, we would not support both. >> Which one is the most used? >> >> So overall I don't think there should be any overhead for the >> implementors of the API, since my understating is that there are out of >> the box implementations out there, like envoy? What am I missing? >> >> > >> > Best, >> > Austin >> > >> > [1]: >> https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds >> > [2]: >> > >> https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#xds-api-implementation >> >> > [3]: >> > >> https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#discovery-scraping >> >> > [4]: >> > >> https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/http_discovery.go >> >> > [5]: >> > >> https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/api/v1alpha1/mads.proto >> >> > On Saturday, February 6, 2021 at 7:04:21 PM UTC-5 Austin Cawley-Edwards >> > wrote: >> > >> > > Hey Julien, >> > > >> > > Thanks for reaching back out about this. I’ve been on the midst of a >> job >> > > transition and have not made any progress, but that is finally fully >> done >> > > and I can try to move this along (though it will still be slowly). >> I’ll get >> > > back to research and bring it up at the next Kuma community meeting >> in two >> > > weeks, and then report back. >> > > >> > > Best, >> > > Austin >> > > >> > > On Sat, Feb 6, 2021 at 3:36 PM Julien Pivotto < >> [email protected]> >> > > wrote: >> > > >> > >> Hello Austin, >> > >> >> > >> Is there any progress here? I am really willing to learn more about >> xds >> > >> and how it would fit here. >> > >> >> > >> On 29 Oct 11:09, Austin Cawley-Edwards wrote: >> > >> > Perfect, I'll get started collecting that data. Thanks for the >> > >> direction. >> > >> > >> > >> > Best, >> > >> > Austin >> > >> > >> > >> > On Wed, Oct 28, 2020 at 1:07 PM Julien Pivotto < >> [email protected] >> > >> > >> > >> > wrote: >> > >> > >> > >> > > On 28 Oct 08:28, Austin Cawley-Edwards wrote: >> > >> > > > Hey Julien, >> > >> > > > >> > >> > > > Following up on if you've had time to think about how we can >> move >> > >> > > forward >> > >> > > > on one of these paths. I haven't been able to find any formal >> > >> proposals >> > >> > > for >> > >> > > > the other recently merged SDs like Eureka besides this >> comment[1]. >> > >> Would >> > >> > > it >> > >> > > > be valuable for us to work on something similar for envoy's >> xDS in >> > >> the >> > >> > > > meantime? >> > >> > > >> > >> > > >> > >> > > Let's be honest that recent service discoveries are 'small', and >> that >> > >> > > big ones are quite old now. >> > >> > > >> > >> > > I still need to gather intel on xDS; but let's start by >> collecting >> > >> data: >> > >> > > >> > >> > > - What would the configuration look like (does it need multiple >> > >> > > `roles`) >> > >> > > - Which meta labels could we have exposed >> > >> > > - How would we set the address >> > >> > > >> > >> > > That hopefully should not be too much work and then we can see >> how/if >> > >> we >> > >> > > move forward with this. >> > >> > > >> > >> > > > >> > >> > > > Thanks, >> > >> > > > Austin >> > >> > > > >> > >> > > > [1]: >> > >> > > > >> > >> > > >> > >> >> https://github.com/prometheus/prometheus/pull/3369#issuecomment-667344846 >> > >> > > > >> > >> > > > On Wednesday, October 21, 2020 at 6:49:59 PM UTC-4 Austin >> > >> Cawley-Edwards >> > >> > > > wrote: >> > >> > > > >> > >> > > > > That makes sense to me, and would be nice to build of an >> existing >> > >> > > > > protocol, and agree it sounds like a large effort. I’m >> pretty new >> > >> to >> > >> > > xDS as >> > >> > > > > well, though we have many people on the Kuma team who are >> quite >> > >> > > > > knowledgeable — I’d be happy to set up a call/ bring them in >> to >> > >> help >> > >> > > size >> > >> > > > > the work if you’d be open to that. >> > >> > > > > >> > >> > > > > And understand + agree on the configuration approach for the >> extra >> > >> > > layer. >> > >> > > > > The main difficulty for users is configuring the file_sd + >> > >> sidecar, >> > >> > > > > relabel_configs should be fairly straightforward and >> accessible. >> > >> > > > > >> > >> > > > > Austin >> > >> > > > > >> > >> > > > > On Wed, Oct 21, 2020 at 6:37 PM Julien Pivotto < >> > >> > > [email protected]> >> > >> > > > > wrote: >> > >> > > > > >> > >> > > > >> On 21 Oct 15:12, Austin Cawley-Edwards wrote: >> > >> > > > >> > Hey all, >> > >> > > > >> > >> > >> > > > >> > I've gathered up some research about our efforts with the >> DNS >> > >> SD >> > >> > > > >> option, >> > >> > > > >> > please let me know if I've overlooked something. >> > >> > > > >> >> > >> > > > >> I have been looking on my side too - I think that what the >> option >> > >> > > that's >> > >> > > > >> more likely on the table would not to have a "generic" >> > >> implementation, >> > >> > > > >> but a xDS protocol service discovery. In that case, it >> would not >> > >> be an >> > >> > > > >> issue to have that in Prometheus (even the gRPC version). >> > >> > > > >> >> > >> > > > >> However, the extra layer that kuma is offering in the SD >> should >> > >> be >> > >> > > done >> > >> > > > >> at the Prometheus configuration level - e.g. by the user, >> in the >> > >> > > > >> relabel_configs. >> > >> > > > >> >> > >> > > > >> I think I'd like to better understand xDS but it seems that >> it >> > >> could >> > >> > > be >> > >> > > > >> good option for a service discovery in Prometheus, at least >> at >> > >> first >> > >> > > > >> sight. >> > >> > > > >> >> > >> > > > >> We are looking at a huge amount of work here I think. >> > >> > > > >> >> > >> > > > >> -- >> > >> > > > >> Julien Pivotto >> > >> > > > >> @roidelapluie >> > >> > > > >> >> > >> > > > > >> > >> > > > >> > >> > > > -- >> > >> > > > You received this message because you are subscribed to the >> Google >> > >> > > Groups "Prometheus Developers" group. >> > >> > > > To unsubscribe from this group and stop receiving emails from >> it, >> > >> send >> > >> > > an email to [email protected]. >> > >> > > > To view this discussion on the web visit >> > >> > > >> > >> >> https://groups.google.com/d/msgid/prometheus-developers/89922987-42e4-4807-9956-5590cb2de558n%40googlegroups.com >> >> > >> > > . >> > >> > > >> > >> > > >> > >> > > -- >> > >> > > Julien Pivotto >> > >> > > @roidelapluie >> > >> > > >> > >> > >> > >> > -- >> > >> > You received this message because you are subscribed to the Google >> > >> Groups "Prometheus Developers" group. >> > >> > To unsubscribe from this group and stop receiving emails from it, >> send >> > >> an email to [email protected]. >> > >> > To view this discussion on the web visit >> > >> >> https://groups.google.com/d/msgid/prometheus-developers/CALGL%2BUBExJmaD9rzXWnTJf5h6my3cbYrrh-uQjCD9U9PGoW-Aw%40mail.gmail.com >> >> > >> . >> > >> >> > >> -- >> > >> Julien Pivotto >> > >> @roidelapluie >> > >> >> > > >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Prometheus Developers" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/7db91c8f-33e2-4253-8cc6-daf596a1d1ban%40googlegroups.com. >> >> >> >> >> -- >> Julien Pivotto >> @roidelapluie >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/47fdf607-fac5-41bd-94ea-683fe67e38bfn%40googlegroups.com.
