Hi , i am running prometheus with this option --storage.tsdb.retention.time=24h but i see that storage space is not cleared for every 24hrs. I see that data is stored from months and disk is getting full and prometheus service is not starting due to this reason.
prometheus[30086]: level=error ts=2021-01-24T04:04:00.908Z caller=main.go:740 err="opening storage failed: open /opt/pm/prometheus/wal/00131056: no space left on device" Could someone please help, why this can happen and how to avoid this. This issue is seen in one instance and we had prometheus in multiple instances. Prometheus version used was 0.18.1 Thanks, Rajesh Dasari. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAPXMrf_CfEED27DS%3DkKGMtGpR4Qf-oNXkkXVL%2BFMatEP2tVwCg%40mail.gmail.com.

