Yes, this is the general guideline. User identifiable data should not be in Prometheus. Mostly because it tends to be a cardinality problem.
On Fri, Mar 27, 2020, 14:58 Stuart Clark <[email protected]> wrote: > On 27/03/2020 13:55, REMI DRUILHE wrote: > > Hi, > > Thanks for the answer. > > Just to add some context to what I want to do. We are trying to make our > application (Prometheus being a part of it) compliant with GDPR. The > default period for our data retention is 10 days but can be set by the user > to 1h minimum. If the data are not deleted after 1 hour then, it is no > longer compliant with what we declared to the national data protection > authority. > > From your answer, I understand now that the parameter > *storage.tsdb.retention.time* is not hard deadline for the data. Data > might live after the retention time, at least 2 hours. So I launched a new > test with storage.tsdb.retention.time equals to 5m. Then, I waited for 3 > hours to see if the data were deleted but they are always in Prometheus. > > Is it possible to anticipate when the data will be deleted to update ours > compliance with GDPR? Can we change the background schedule to have a > better estimation of when the data will be deleted? Or should I use the > delete method of Prometheus form an external component to be sure that the > data are deleted on time? > > > What data are you expecting Prometheus to hold that would be subject to > GDPR? > > Prometheus stores monitoring metrics so wouldn't have anything related to > personal data, so isn't in scope for GDPR... > > > Thanks, > > Best regards. > > > Le jeudi 26 mars 2020 21:40:35 UTC+1, Christian Hoffmann a écrit : >> >> Hi, >> >> not sure if it's this what you are hitting, but the following may help: >> >> The retention time defines how long Prometheus is expected to keep >> history. This does not imply that data older than this interval will >> vanish immediately. >> In other words: Once the interval has passed, Prometheus is free to >> delete this data, but it isn't forced to. In practice, this will depend >> on maintenance (compaction) running. >> >> This document provides further details, especially: >> >> https://prometheus.io/docs/prometheus/latest/storage/ >> "Expired block cleanup happens on a background schedule. It may take up >> to two hours to remove expired blocks. Expired blocks must be fully >> expired before they are cleaned up." >> >> Maybe this explains things for you? >> >> Kind regards, >> Christian >> >> On 3/26/20 3:33 PM, REMI DRUILHE wrote: >> > I did some others tests, on others operating systems (not sure it is >> > relevant because of Docker but without any clues it is better than >> > nothing). For anyone to test it, I also changed and reduced my >> > docker-compose file to: >> > >> > | >> > version:'3.4' >> > | >> > services: >> > prometheus: >> > image: prom/prometheus:latest >> > container_name: "prometheus" >> > depends_on: >> > - cadvisor >> > command: >> > - '--config.file=/etc/prometheus/prometheus.yml' >> > - '--storage.tsdb.retention.time=60s' >> > volumes: >> > - /tmp/prometheus:/etc/prometheus >> > ports: >> > - "9090:9090" >> > network_mode: "host" >> > cadvisor: >> > image: gcr.io/google-containers/cadvisor >> > container_name: cadvisor >> > volumes: >> > - /:/rootfs:ro >> > - /var/run:/var/run:ro >> > - /sys:/sys:ro >> > - /var/lib/docker/:/var/lib/docker:ro >> > - /dev/disk/:/dev/disk:ro >> > ports: >> > - "8080:8080" >> > network_mode: "host" >> > >> > And here is my prometheus.yml >> > >> > | >> > scrape_configs: >> > - job_name: cadvisor >> > scrape_interval: 5s >> > static_configs: >> > - targets: >> > - localhost:8080 >> > | >> > >> > But, the result is still the same with the following curl request: >> > >> > | >> > curl >> > ' >> http://172.21.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-26T00:01:00.000Z&end=2020-03-26T17:00:00.000Z&step=15s' >> >> > | >> > >> > I can still get data that should have been deleted due to out of date >> of >> > the timestamp and defined in the storage.tsdb.retention.time parameter. >> > >> > According to me, it is a bug. But before opening an issue on >> Prometehus, >> > I prefer to ask for advices about it. >> > >> > Thanks, >> > >> > Best regards. >> > >> > Le mercredi 25 mars 2020 10:27:22 UTC+1, REMI DRUILHE a écrit : >> > >> > Anyone to help me on this subject? >> > >> > Thanks. >> > >> > Le mardi 24 mars 2020 13:57:05 UTC+1, REMI DRUILHE a écrit : >> > >> > Hello, >> > >> > I am using a Docker version of Prometheus (the latest one) in >> > which I am setting the value for the retention time (option >> > *storage.tsdb.retention.time*) at launch. I trying first with >> > small values like *1m* and *60s* (for 1 minute) or *5m* (for 5 >> > minutes) but I was not able to see any deletion of the data >> > after this period. Thus, I tested again this morning with an >> > higher value: *2h* (for 2 hours). I have seen this information >> > here >> > < >> https://medium.com/@valyala/prometheus-storage-technical-terms-for-humans-4ab4de6c3d48> >> >> > (in the retention section) that 2h is the minimum value for >> > Prometheus, not sure if it is true or not. But even with this >> > value, the data stored 4 hours ago can still be retrieved using >> > the HTTP API. Here is my query: *curl >> > ' >> http://172.18.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-24T00:01:00.000Z&end=2020-03-24T17:00:00.000Z&step=15s >> > < >> http://172.18.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes&start=2020-03-24T00:01:00.000Z&end=2020-03-24T17:00:00.000Z&step=15s>'*. >> >> > >> > On the Prometheus GUI, the flag for this option is correctly >> setup. >> > >> > Here is the Docker compose file that is used to launch >> Prometheus: >> > >> > | >> > prometheus: >> > command:'--config.file=/etc/prometheus.yml >> > --storage.tsdb.retention.time=2h' >> > container_name:remi_prometheus >> > depends_on: >> > -cadvisor >> > expose: >> > -'9090' >> > image:prom/prometheus:latest >> > labels: >> > project.run.user:remi >> > networks: >> > project-bridge: >> > aliases: >> > -prometheus >> > ports: >> > -published:9090 >> > target:9090 >> > volumes: >> > -remi-prometheus:/prometheus:rw >> > - >> > >> /home/remi/Workspace/project/runtime/configuration/prometheus.yml:/etc/prometheus.yml:rw >> >> > | >> > >> > Note that I am using cAdvisor to populate Prometheus in which I >> > get some Golang metrics. >> > >> > Thus here are my questions: >> > >> > * What am I doing wrong? >> > * Was is the minimum value for storage.tsdb.retention.time? >> > * Is there another option that overwrite the value set in >> > storage.tsdb.retention.time that I am not aware of? >> > * Is there a way to test that the option is working if the >> > minimal value is 1 day? I guess it would be to change the >> > system date, but not sure about it. >> > >> > >> > Thanks for the help, >> > >> > Best regards and good luck if you are under confinement because >> > of this damn virus :) >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Prometheus Users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to [email protected] >> > <mailto:[email protected]>. >> > To view this discussion on the web visit >> > >> https://groups.google.com/d/msgid/prometheus-users/c3550107-62af-4120-b2d1-13926b3fff14%40googlegroups.com >> > < >> https://groups.google.com/d/msgid/prometheus-users/c3550107-62af-4120-b2d1-13926b3fff14%40googlegroups.com?utm_medium=email&utm_source=footer>. >> >> >> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/27f3c76c-f2b3-4bfe-87d7-02adbf61f8c4%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/27f3c76c-f2b3-4bfe-87d7-02adbf61f8c4%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > > -- > Stuart Clark > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/e056715f-cf27-5e5d-743f-686c835c8207%40Jahingo.com > <https://groups.google.com/d/msgid/prometheus-users/e056715f-cf27-5e5d-743f-686c835c8207%40Jahingo.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmp3j%2BnuxLisQpGP3GtM1orkZvfCfQe1HThdXrh25JnVLA%40mail.gmail.com.
