On 18 Feb 14:01, Austin Cawley-Edwards wrote: > We could try out EDS, though it seems like a very envoy-specific feature > without a good way to provide non-envoy labels. The only way that seems > possible with EDS is via the LbEndpoint's metadata[1], though that very > specifically not what it is designed for.
I don't get what you mean here. > > [1]: > https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/endpoint/v3/endpoint_components.proto#config-endpoint-v3-lbendpoint > On Thursday, February 18, 2021 at 4:52:52 PM UTC-5 Austin Cawley-Edwards > wrote: > > > Yeah, that would be nice too but would not solve the original use case of > > this thread, which was to provide a generic option that did not need to > > co-locate data next to Prometheus via a filesystem. I think xDS is still a > > good fit for a generic option, but we would need to define an API. > > > > If we were to just directly use the structure of file_sd, then we're not > > creating anything new, just joining the xDS discovery protocol with an > > existing data structure. > > > > > > What are you thinking? Do you want to open a new thread to see if anyone > > would find an envoy_ds valuable? > > > > Best, > > Austin > > > > On Thursday, February 18, 2021 at 4:45:05 PM UTC-5 Julien Pivotto wrote: > > > >> Yeah my impression was that eDS would fit here, what envoy uses for > >> endpoints. > >> > >> > >> On 18 Feb 13:31, Austin Cawley-Edwards wrote: > >> > Quick follow-up: I think an "envoy discovery" method to scrape > >> > Envoy-specific metrics might be worthwhile, but I don't think it fits > >> in > >> > the same conversation as this one. > >> > > >> > On Thursday, February 18, 2021 at 4:12:39 PM UTC-5 Austin > >> Cawley-Edwards > >> > wrote: > >> > > >> > > Hey Julien, > >> > > > >> > > I think the disconnect here is that xDS is just a protocol used for > >> many > >> > > different Envoy services[1], and does not define any actual data that > >> could > >> > > be mapped to a target itself. It is purely meant as a base that is to > >> be > >> > > extended. Can you clarify what you mean by "current xds api's"? Do > >> you mean > >> > > just discovering the current Envoys in a cluster? Or do you mean > >> trying to > >> > > discover all the various Listeners/ VirtualHosts[2] (which may or may > >> not > >> > > expose a metrics endpoint) from Envoy? Envoy does not implement > >> anything > >> > > about metrics of the underlying services it proxies, i.e. there is no > >> way > >> > > to say "these services are good to scrape" to my knowledge. > >> > > > >> > > For HTTP vs gRPC, I'm not sure why we wouldn't want to support both > >> as the > >> > > xDS protocol defines support for both, but gRPC is more common. > >> > > > >> > > Best, > >> > > Austin > >> > > > >> > > [1]: > >> https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/service > >> > > [2]: > >> > > > >> https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol#resource-types > >> > >> > > > >> > > On Thursday, February 18, 2021 at 3:20:18 PM UTC-5 Julien Pivotto > >> wrote: > >> > > > >> > >> On 18 Feb 12:10, Austin Cawley-Edwards wrote: > >> > >> > Hey Julien, > >> > >> > > >> > >> > I've finished up a *rough *proof of concept for what xDS support > >> in > >> > >> > Prometheus could look like [1]. I think it shows the main areas > >> without > >> > >> > going too over-the-top. In summary, the two main pieces to the > >> > >> integration > >> > >> > seem to be an xDS API definition[2] and an xDS client/ discovery > >> that > >> > >> uses > >> > >> > an implementation of this API [3] (HTTP discovery implementation > >> > >> here[4]). > >> > >> > For the xDS API, envoy does not have a native concept of metrics/ > >> > >> > monitoring, so we need to define this data structure -- I've > >> currently > >> > >> > sketched one out (taking from Kuma) that is very close to a > >> > >> targetgroup[5]. > >> > >> > > >> > >> > A few pros: > >> > >> > - support for both HTTP and gRPC API implementations > >> > >> > - xDS protocol is well documented > >> > >> > - versioned API will make modifications possible > >> > >> > > >> > >> > A few cons: > >> > >> > - complexity of the protocol for people who have not used it > >> > >> > - added dependencies on gRPC and envoy + build processes additions > >> for > >> > >> > compiling protobuf > >> > >> > - xDS adds some overhead for implementors of this API, though this > >> > >> seems > >> > >> > like an "advanced" use case where that is ok > >> > >> > > >> > >> > Please let me know what you think and if you think it's feasible > >> to > >> > >> move > >> > >> > forward with xDS or if we should go back to discussions of a > >> simpler > >> > >> HTTP/ > >> > >> > gRPC approach. > >> > >> > >> > >> Thanks, here are my initial thoughts: > >> > >> > >> > >> - We should not *add* an API, but discover everything on the xDS > >> API. > >> > >> The meta labels created would then enable to filter what is needed > >> and > >> > >> what is not. It is not the responsibility of the service discovery. > >> > >> Monitoring is not just about whitebox monitoring, the scope can be > >> > >> much larger. Also, I want something that could be used out of the > >> box > >> > >> with current xds api's. > >> > >> - We need to settle by using grpc or http, we would not support > >> both. > >> > >> Which one is the most used? > >> > >> > >> > >> So overall I don't think there should be any overhead for the > >> > >> implementors of the API, since my understating is that there are out > >> of > >> > >> the box implementations out there, like envoy? What am I missing? > >> > >> > >> > >> > > >> > >> > Best, > >> > >> > Austin > >> > >> > > >> > >> > [1]: > >> > >> > >> https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds > >> > >> > [2]: > >> > >> > > >> > >> > >> https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#xds-api-implementation > >> > >> > >> > [3]: > >> > >> > > >> > >> > >> https://github.com/austince/prometheus/tree/discovery-xds/discovery/xds#discovery-scraping > >> > >> > >> > [4]: > >> > >> > > >> > >> > >> https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/http_discovery.go > >> > >> > >> > [5]: > >> > >> > > >> > >> > >> https://github.com/austince/prometheus/blob/discovery-xds/discovery/xds/api/v1alpha1/mads.proto > >> > >> > >> > On Saturday, February 6, 2021 at 7:04:21 PM UTC-5 Austin > >> Cawley-Edwards > >> > >> > wrote: > >> > >> > > >> > >> > > Hey Julien, > >> > >> > > > >> > >> > > Thanks for reaching back out about this. I’ve been on the midst > >> of a > >> > >> job > >> > >> > > transition and have not made any progress, but that is finally > >> fully > >> > >> done > >> > >> > > and I can try to move this along (though it will still be > >> slowly). > >> > >> I’ll get > >> > >> > > back to research and bring it up at the next Kuma community > >> meeting > >> > >> in two > >> > >> > > weeks, and then report back. > >> > >> > > > >> > >> > > Best, > >> > >> > > Austin > >> > >> > > > >> > >> > > On Sat, Feb 6, 2021 at 3:36 PM Julien Pivotto < > >> > >> [email protected]> > >> > >> > > wrote: > >> > >> > > > >> > >> > >> Hello Austin, > >> > >> > >> > >> > >> > >> Is there any progress here? I am really willing to learn more > >> about > >> > >> xds > >> > >> > >> and how it would fit here. > >> > >> > >> > >> > >> > >> On 29 Oct 11:09, Austin Cawley-Edwards wrote: > >> > >> > >> > Perfect, I'll get started collecting that data. Thanks for > >> the > >> > >> > >> direction. > >> > >> > >> > > >> > >> > >> > Best, > >> > >> > >> > Austin > >> > >> > >> > > >> > >> > >> > On Wed, Oct 28, 2020 at 1:07 PM Julien Pivotto < > >> > >> [email protected] > >> > >> > >> > > >> > >> > >> > wrote: > >> > >> > >> > > >> > >> > >> > > On 28 Oct 08:28, Austin Cawley-Edwards wrote: > >> > >> > >> > > > Hey Julien, > >> > >> > >> > > > > >> > >> > >> > > > Following up on if you've had time to think about how we > >> can > >> > >> move > >> > >> > >> > > forward > >> > >> > >> > > > on one of these paths. I haven't been able to find any > >> formal > >> > >> > >> proposals > >> > >> > >> > > for > >> > >> > >> > > > the other recently merged SDs like Eureka besides this > >> > >> comment[1]. > >> > >> > >> Would > >> > >> > >> > > it > >> > >> > >> > > > be valuable for us to work on something similar for > >> envoy's > >> > >> xDS in > >> > >> > >> the > >> > >> > >> > > > meantime? > >> > >> > >> > > > >> > >> > >> > > > >> > >> > >> > > Let's be honest that recent service discoveries are > >> 'small', and > >> > >> that > >> > >> > >> > > big ones are quite old now. > >> > >> > >> > > > >> > >> > >> > > I still need to gather intel on xDS; but let's start by > >> > >> collecting > >> > >> > >> data: > >> > >> > >> > > > >> > >> > >> > > - What would the configuration look like (does it need > >> multiple > >> > >> > >> > > `roles`) > >> > >> > >> > > - Which meta labels could we have exposed > >> > >> > >> > > - How would we set the address > >> > >> > >> > > > >> > >> > >> > > That hopefully should not be too much work and then we can > >> see > >> > >> how/if > >> > >> > >> we > >> > >> > >> > > move forward with this. > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > Thanks, > >> > >> > >> > > > Austin > >> > >> > >> > > > > >> > >> > >> > > > [1]: > >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > >> > >> > >> https://github.com/prometheus/prometheus/pull/3369#issuecomment-667344846 > >> > >> > >> > > > > >> > >> > >> > > > On Wednesday, October 21, 2020 at 6:49:59 PM UTC-4 Austin > >> > >> > >> Cawley-Edwards > >> > >> > >> > > > wrote: > >> > >> > >> > > > > >> > >> > >> > > > > That makes sense to me, and would be nice to build of > >> an > >> > >> existing > >> > >> > >> > > > > protocol, and agree it sounds like a large effort. I’m > >> > >> pretty new > >> > >> > >> to > >> > >> > >> > > xDS as > >> > >> > >> > > > > well, though we have many people on the Kuma team who > >> are > >> > >> quite > >> > >> > >> > > > > knowledgeable — I’d be happy to set up a call/ bring > >> them in > >> > >> to > >> > >> > >> help > >> > >> > >> > > size > >> > >> > >> > > > > the work if you’d be open to that. > >> > >> > >> > > > > > >> > >> > >> > > > > And understand + agree on the configuration approach > >> for the > >> > >> extra > >> > >> > >> > > layer. > >> > >> > >> > > > > The main difficulty for users is configuring the > >> file_sd + > >> > >> > >> sidecar, > >> > >> > >> > > > > relabel_configs should be fairly straightforward and > >> > >> accessible. > >> > >> > >> > > > > > >> > >> > >> > > > > Austin > >> > >> > >> > > > > > >> > >> > >> > > > > On Wed, Oct 21, 2020 at 6:37 PM Julien Pivotto < > >> > >> > >> > > [email protected]> > >> > >> > >> > > > > wrote: > >> > >> > >> > > > > > >> > >> > >> > > > >> On 21 Oct 15:12, Austin Cawley-Edwards wrote: > >> > >> > >> > > > >> > Hey all, > >> > >> > >> > > > >> > > >> > >> > >> > > > >> > I've gathered up some research about our efforts > >> with the > >> > >> DNS > >> > >> > >> SD > >> > >> > >> > > > >> option, > >> > >> > >> > > > >> > please let me know if I've overlooked something. > >> > >> > >> > > > >> > >> > >> > >> > > > >> I have been looking on my side too - I think that what > >> the > >> > >> option > >> > >> > >> > > that's > >> > >> > >> > > > >> more likely on the table would not to have a "generic" > >> > >> > >> implementation, > >> > >> > >> > > > >> but a xDS protocol service discovery. In that case, it > >> > >> would not > >> > >> > >> be an > >> > >> > >> > > > >> issue to have that in Prometheus (even the gRPC > >> version). > >> > >> > >> > > > >> > >> > >> > >> > > > >> However, the extra layer that kuma is offering in the > >> SD > >> > >> should > >> > >> > >> be > >> > >> > >> > > done > >> > >> > >> > > > >> at the Prometheus configuration level - e.g. by the > >> user, > >> > >> in the > >> > >> > >> > > > >> relabel_configs. > >> > >> > >> > > > >> > >> > >> > >> > > > >> I think I'd like to better understand xDS but it seems > >> that > >> > >> it > >> > >> > >> could > >> > >> > >> > > be > >> > >> > >> > > > >> good option for a service discovery in Prometheus, at > >> least > >> > >> at > >> > >> > >> first > >> > >> > >> > > > >> sight. > >> > >> > >> > > > >> > >> > >> > >> > > > >> We are looking at a huge amount of work here I think. > >> > >> > >> > > > >> > >> > >> > >> > > > >> -- > >> > >> > >> > > > >> Julien Pivotto > >> > >> > >> > > > >> @roidelapluie > >> > >> > >> > > > >> > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > -- > >> > >> > >> > > > You received this message because you are subscribed to > >> the > >> > >> Google > >> > >> > >> > > Groups "Prometheus Developers" group. > >> > >> > >> > > > To unsubscribe from this group and stop receiving emails > >> from > >> > >> it, > >> > >> > >> send > >> > >> > >> > > an email to [email protected]. > >> > >> > >> > > > To view this discussion on the web visit > >> > >> > >> > > > >> > >> > >> > >> > >> > >> https://groups.google.com/d/msgid/prometheus-developers/89922987-42e4-4807-9956-5590cb2de558n%40googlegroups.com > >> > >> > >> > >> > > . > >> > >> > >> > > > >> > >> > >> > > > >> > >> > >> > > -- > >> > >> > >> > > Julien Pivotto > >> > >> > >> > > @roidelapluie > >> > >> > >> > > > >> > >> > >> > > >> > >> > >> > -- > >> > >> > >> > You received this message because you are subscribed to the > >> Google > >> > >> > >> Groups "Prometheus Developers" group. > >> > >> > >> > To unsubscribe from this group and stop receiving emails from > >> it, > >> > >> send > >> > >> > >> an email to [email protected]. > >> > >> > >> > To view this discussion on the web visit > >> > >> > >> > >> > >> > >> https://groups.google.com/d/msgid/prometheus-developers/CALGL%2BUBExJmaD9rzXWnTJf5h6my3cbYrrh-uQjCD9U9PGoW-Aw%40mail.gmail.com > >> > >> > >> > >> . > >> > >> > >> > >> > >> > >> -- > >> > >> > >> Julien Pivotto > >> > >> > >> @roidelapluie > >> > >> > >> > >> > >> > > > >> > >> > > >> > >> > -- > >> > >> > You received this message because you are subscribed to the Google > >> > >> Groups "Prometheus Developers" group. > >> > >> > To unsubscribe from this group and stop receiving emails from it, > >> send > >> > >> an email to [email protected]. > >> > >> > To view this discussion on the web visit > >> > >> > >> https://groups.google.com/d/msgid/prometheus-developers/7db91c8f-33e2-4253-8cc6-daf596a1d1ban%40googlegroups.com. > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> -- > >> > >> Julien Pivotto > >> > >> @roidelapluie > >> > >> > >> > > > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> Groups "Prometheus Developers" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> an email to [email protected]. > >> > To view this discussion on the web visit > >> https://groups.google.com/d/msgid/prometheus-developers/47fdf607-fac5-41bd-94ea-683fe67e38bfn%40googlegroups.com. > >> > >> > >> > >> > >> -- > >> Julien Pivotto > >> @roidelapluie > >> > > > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/16e4997f-3d3c-493c-b183-f503789cbf66n%40googlegroups.com. -- Julien Pivotto @roidelapluie -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/20210218220552.GA299688%40oxygen.
