"local storage" means what filesystem - ext4? xfs? something else?
What's the difference between the oldest and newest timestamps? That will show you how long the problem has been going for. ls -ld 00021327 ls -ld 00036028 Are you getting any error output from the prometheus server process? For example, if you're running it under systemd then try journalctl -eu prometheus My guess is that prometheus is failing to update the main database from the WAL, and if so will probably be logging some message saying what it's stuck on. And hence the WAL just keeps growing as new data is scraped. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/abd2fa33-1664-464b-90f7-294909ad189en%40googlegroups.com.
