Ok, gotcha thanks. Part of the reason I was confused is because Monitoring in gcp has a metric called "container_memory_used_bytes" where you can filter on evictable and non-evictable. If filtering on non-evictable the value seems to correlate with "container_memory_working_set_bytes".
Just adding that here in case someone else stumbles upon this conversation. On Thu, Feb 9, 2023 at 11:03 AM Ben Kochie <sup...@gmail.com> wrote: > Kubernetes will not OOM a pod for page cache use. The > container_memory_used_bytes includes page cache and is not what you want to > compare. What you want to do is look at is container_memory_rss or > container_memory_working_set_bytes. > > On Thu, Feb 9, 2023 at 10:56 AM 'Björn Johansson' via Prometheus Users < > prometheus-users@googlegroups.com> wrote: > >> I'm struggling to understand how prometheus uses memory. I've done some >> reading and I find pieces of information here and there but I don't get the >> full picture. >> >> Some context first. >> We are running prometheus in kubernetes. Not the operator, just vanilla >> prometheus. >> We are monitoring memory usage with the >> "container_memory_limit_utilization" and "container_memory_used_bytes" >> metrics. >> >> I've seen in some previous conversation that these metrics are not ideal >> for monitoring usage since they include the page cache as well. This is >> where I start getting confused. >> Kubernetes doesn't make any distinction if memory is the page cache or >> the actual memory used by prometheus. The pod will OOM and be evicted once >> it hits the limit. >> Will page cache be released when needed and before the limit is hit? >> >> Can you please help me understand how this works. >> How should I monitor prometheus so that I can be sure it has the memory >> it needs? >> >> Thanks >> Björn >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to prometheus-users+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/6428d3f7-2fff-4783-9e4d-13f1a5e442c8n%40googlegroups.com >> <https://groups.google.com/d/msgid/prometheus-users/6428d3f7-2fff-4783-9e4d-13f1a5e442c8n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- Björn Johansson Software Developer Cell: +46 72 052 58 49 www.storytel.com Storytel Sweden AB Box 24167 104 51 Stockholm Visit address: Västra Torggatan 19, 65224 Karlstad -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAFLnuWELTP9qxc-99HguTtsLawkTJL%3DTOrOxL14MO5XwpfqQFw%40mail.gmail.com.
