Question to the wider audience: would it be sensible to add this to the example/default Kubernetes configuration?
/MR On Tue, Apr 7, 2020, 20:51 Patrick Schless <[email protected]> wrote: > Hi Matthias, > > Thanks for your thoughts. I've experimented some more, and have it > working now. Best as I can tell, it's simply missing from the default > config in the prom/prometheus docker image: > https://gist.github.com/plainlystated/95c79163583d5d107f2d6e85f9d335d5 > > In that config, there are only two "role: service" sections. One is the > pushgateway, and the other forces use of the blackbox app, which seems like > it shouldn't be necessary for what I'm trying to do. > > I added a scrape config, which seems to have done the trick: > >> - honor_labels: true >> job_name: prometheus-service-non-blackbox >> kubernetes_sd_configs: >> - role: service >> relabel_configs: >> - action: keep >> regex: service >> source_labels: >> - __meta_kubernetes_service_annotation_prometheus_io_scrape >> - action: replace >> regex: (.+) >> source_labels: >> - __meta_kubernetes_service_annotation_prometheus_io_path >> target_label: __metrics_path__ >> > > Then I just set the service annotation: > >> prometheus.io/scrape: service >> > > Now my prometheus correctly has 1 target for the service, and no targets > for the endpoints. > > Thanks for your help! > > - Patrick > > On Tue, Apr 7, 2020 at 11:07 AM Matthias Rampke <[email protected]> > wrote: > >> Kubernetes creates the Endpoints object based on the Service object. Does >> it carry over the annotations while doing that? If so, you are getting >> returns in both SD configs because there are two objects in Kubernetes to >> be discovered. >> >> I would suggest using a different value for the scrape annotation, and >> matching on that in the service SD configuration, to distinguish whether >> you want this thing scraped once or for each instance. >> >> /MR >> >> On Tue, Apr 7, 2020 at 3:37 AM Patrick Schless <[email protected]> >> wrote: >> >>> I'm using prometheus on k8s, and trying to add an annotation to my >>> service for the service discovery. The metrics I'm exposing are >>> business-level (eg "# of customers") so I don't need all pods queried. >>> Querying the service once (with a random pod) is sufficient. As I >>> understand it, there is a service discovery role applicable: "service". >>> >>> However, when I add this annotation to my service, it is "discovered" by >>> the endpoints role, which then tries to hit all downstream pods. >>> >>> prometheus.io/scrape: true >>>> >>> >>> >>> The docs clearly make a distinction between service SD and endpoints SD, >>> but it's not clear to me now to choose one or the other. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-users/bbec2f68-afea-4366-8156-b729bc04b373%40googlegroups.com >>> <https://groups.google.com/d/msgid/prometheus-users/bbec2f68-afea-4366-8156-b729bc04b373%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAMV%3D_gZQmR_LUfQfKe7wRC67%3D%2BJci36t%2BD0uOsUuaKTkrcdANQ%40mail.gmail.com.
