Yeah, I'm using cAdvisor for containers and node_exporter for hosts. Thank you for the link.
On Thursday, April 16, 2020 at 4:46:04 PM UTC+5:30, Ben Kochie wrote: > > > > On Thu, Apr 16, 2020 at 10:28 AM Isabel Noronha <[email protected] > <javascript:>> wrote: > >> >> >> Yeah I just checked relabelling .I need only few metrics and labels like >>> the ones below. >>> ex.container_memory_usage_bytes >>> container_memory_usage_bytes >>> container_cpu_usage_seconds_total >>> >> So I have more metrics to be dropped. I don't want to include(to be >> dropped metrics) in my prometheus.yml file. >> So is there a way to keep only what I need and drop other metrics? >> An example would help in understanding relabelling better. >> >> Yes, I'm monitoring containers per host in grafana using variables. >> And a query like only shows the top 20 containers exceeding the >> threshold value of CPU usage and sends an alert. >> This is what I'm planning to do. >> > > If this is coming from cAdvisor, you can drop metrics by configuring it to > disable some metrics collectors: > > > https://github.com/google/cadvisor/blob/master/docs/runtime_options.md#metrics > > > >> >> >> >> >>> >>> >>> On Thursday, April 16, 2020 at 12:39:24 PM UTC+5:30, Brian Candler wrote: >>>> >>>> > So can Prometheus handle so much load and monitor each container? >>>> >>>> In short yes, although the important figure is the total number of >>>> *metrics* (servers x containers-per-server x metrics-per-container), and >>>> this will affect how much resource you need to throw at your prometheus >>>> server. If it's too much, you may choose to filter the metrics you ingest >>>> to just the ones of interest, using metric relabelling. >>>> >>>> Assuming you are using a modern version of prometheus (2.14 or later) >>>> then the web interface on port 9090 will tell you the stats you need to >>>> know, under Status > Runtime & Build Information. >>>> >>>> As for grafana "hanging": you probably need to configure your >>>> dashboards to select a small enough subset of timeseries up-front, e.g. >>>> using dashboard variables. If you run an initial query which returns >>>> thousands of timeseries, it will indeed take an extremely long time to (a) >>>> return the results from prometheus, and (b) render them. >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/e84f1093-88ed-466c-b3f8-dcef64b3c9ce%40googlegroups.com >> >> <https://groups.google.com/d/msgid/prometheus-users/e84f1093-88ed-466c-b3f8-dcef64b3c9ce%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/660a4b54-600f-4bf0-bd70-2d291263574d%40googlegroups.com.
