On 18 Dec 16:03, Brett Larson wrote: > Hello, > I am using a Prometheus 2.21 server and I am seeing that the memory is > growing at an untenable rate. I will start the pod at 4GB and eventually it > will move to 10GB after a few days and go into a crashloop back off state. > > The pod is configured to only keep around 8 hours of data, and this data is > stored to empyDir, not a persistent file system. We are doing remote write > to postgres for only about 4 metrics. > > We do have some no-nos (high cardinality labels & pod names) but > unfortunately these are needed. > > I don't understand why after a retention of 8 hours we would still cause > memory like this to grow and i'm looking for some guidance on how I can > troubleshoot this. > > Please let me know, > Thank you!
Can you share screenshots of the https://grafana.com/grafana/dashboards/12054 dashboard ? -- Julien Pivotto @roidelapluie -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/20201219000904.GA360630%40oxygen.
