Hello Matthias, Many thanks for the suggestion. I will give it a try, although I have the impression that the k8s API will also be off-limits. At the moment I just have a couple of services / pods, but it would be nice to experiment with agent-mode.
Cheers, -- Adriano On Sunday, April 17, 2022 at 7:33:47 AM UTC-3 [email protected] wrote: > If you can, deploy (a) Prometheus into the cluster itself. The easiest way > to manage that is using the Prometheus operator, but if that is not > possible, you can configure it directly using relabeling, as in this > example[0]. > > This Prometheus can scrape the various targets. You have a few options > from there: > > You can use this directly, reaching it through the load balancer, or > through a Grafana deployed to the same cluster. > > Or use remote write to push to another Prometheus or other metric store. > In this case you can run the in-cluster Prometheus in the pared down agent > mode. This also works if you run e.g. one Prometheus per namespace. > > On the more complex but full featured end, you can use Thanos to tie > multiple servers in multiple clusters together with a long term store. > > What is appropriate for you depends on the size of your setup, what you > want monitor, and the restrictions that your admins impose. I hope this > gives you some pointers to discuss with them! > > Best, > Matthias > > > > > [0]: > https://github.com/prometheus/prometheus/blob/main/documentation/examples/prometheus-kubernetes.yml > > On Fri, Apr 15, 2022, 17:19 [email protected] <[email protected]> wrote: > >> Hello, >> >> I'd like to use Prometheus to monitor my (wrapped) k8s services, but >> unfortunately the admin won't let me connect directly to individual pods, I >> can just access them via load balancing. >> >> In this case, I guess I have to use the Pushgateway. It seems to be >> working, however timeseries for old pods persist (the "instance" label is a >> random string). I am thinking of creating a program to periodically delete >> them from the Pushgateway. >> >> This is not an ideal situation but I guess this is the best I can do. Or >> am I missing something? >> >> Thanks, >> >> -- >> Adriano >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/e009b2af-53e3-4c43-8d3c-64b3cc29041bn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/prometheus-users/e009b2af-53e3-4c43-8d3c-64b3cc29041bn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/03e43c7e-81f8-42d0-b7ab-5ba6646203fbn%40googlegroups.com.
