I did some others tests, on others operating systems (not sure it is relevant because of Docker but without any clues it is better than nothing). For anyone to test it, I also changed and reduced my docker-compose file to:
version: '3.4' services: prometheus: image: prom/prometheus:latest container_name: "prometheus" depends_on: - cadvisor command: - '--config.file=/etc/prometheus/prometheus.yml' - '--storage.tsdb.retention.time=60s' volumes: - /tmp/prometheus:/etc/prometheus ports: - "9090:9090" network_mode: "host" cadvisor: image: gcr.io/google-containers/cadvisor container_name: cadvisor volumes: - /:/rootfs:ro - /var/run:/var/run:ro - /sys:/sys:ro - /var/lib/docker/:/var/lib/docker:ro - /dev/disk/:/dev/disk:ro ports: - "8080:8080" network_mode: "host" And here is my prometheus.yml scrape_configs: - job_name: cadvisor scrape_interval: 5s static_configs: - targets: - localhost:8080 But, the result is still the same with the following curl request: curl 'http://172.21.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-26T00:01:00.000Z&end=2020-03-26T17:00:00.000Z&step=15s' I can still get data that should have been deleted due to out of date of the timestamp and defined in the storage.tsdb.retention.time parameter. According to me, it is a bug. But before opening an issue on Prometehus, I prefer to ask for advices about it. Thanks, Best regards. Le mercredi 25 mars 2020 10:27:22 UTC+1, REMI DRUILHE a écrit : > > Anyone to help me on this subject? > > Thanks. > > Le mardi 24 mars 2020 13:57:05 UTC+1, REMI DRUILHE a écrit : >> >> Hello, >> >> I am using a Docker version of Prometheus (the latest one) in which I am >> setting the value for the retention time (option >> *storage.tsdb.retention.time*) at launch. I trying first with small >> values like *1m* and *60s* (for 1 minute) or *5m* (for 5 minutes) but I >> was not able to see any deletion of the data after this period. Thus, I >> tested again this morning with an higher value: *2h* (for 2 hours). I >> have seen this information here >> <https://medium.com/@valyala/prometheus-storage-technical-terms-for-humans-4ab4de6c3d48> >> >> (in the retention section) that 2h is the minimum value for Prometheus, not >> sure if it is true or not. But even with this value, the data stored 4 >> hours ago can still be retrieved using the HTTP API. Here is my query: *curl >> 'http://172.18.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-24T00:01:00.000Z&end=2020-03-24T17:00:00.000Z&step=15s >> >> <http://172.18.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-24T00:01:00.000Z&end=2020-03-24T17:00:00.000Z&step=15s>'* >> . >> >> On the Prometheus GUI, the flag for this option is correctly setup. >> >> Here is the Docker compose file that is used to launch Prometheus: >> >> prometheus: >> command: '--config.file=/etc/prometheus.yml >> --storage.tsdb.retention.time=2h' >> container_name: remi_prometheus >> depends_on: >> - cadvisor >> expose: >> - '9090' >> image: prom/prometheus:latest >> labels: >> project.run.user: remi >> networks: >> project-bridge: >> aliases: >> - prometheus >> ports: >> - published: 9090 >> target: 9090 >> volumes: >> - remi-prometheus:/prometheus:rw >> - /home/remi/Workspace/project/runtime/configuration/prometheus.yml: >> /etc/prometheus.yml:rw >> >> Note that I am using cAdvisor to populate Prometheus in which I get some >> Golang metrics. >> >> Thus here are my questions: >> >> - What am I doing wrong? >> - Was is the minimum value for storage.tsdb.retention.time? >> - Is there another option that overwrite the value set in >> storage.tsdb.retention.time that I am not aware of? >> - Is there a way to test that the option is working if the minimal >> value is 1 day? I guess it would be to change the system date, but not >> sure >> about it. >> >> >> Thanks for the help, >> >> Best regards and good luck if you are under confinement because of this >> damn virus :) >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/c3550107-62af-4120-b2d1-13926b3fff14%40googlegroups.com.