@jullient Pivotto .. not sure what you mean ... that group does not exist inside the container so don't think so ... maybe building a custom might do something similar ...
``` /prometheus $ cat /etc/group root:x:0: daemon:x:1: bin:x:2: sys:x:3: adm:x:4: tty:x:5: disk:x:6: lp:x:7: mail:x:8: kmem:x:9: wheel:x:10:root cdrom:x:11: dialout:x:18: floppy:x:19: video:x:28: audio:x:29: tape:x:32: www-data:x:33: operator:x:37: utmp:x:43: plugdev:x:46: staff:x:50: lock:x:54: netdev:x:82: users:x:100: nogroup:x:65534: ``` Also ... setting the docker.sock read open doesn't make the fix .. so ... to me all of this sounds like a bug on how prometheus is trying to reach the docker.sock ... how the image is build .. I don't know ... but to me it sounds bug-ish ... but this seems to be being pretty ignored or not well documented ... which one? i don't know ... but seems i am not the only one getting crazy with this .. wish i could help more ... would be good if this gets some traction ... i'd be glad to run any tests to help finding the issue! Ill try to dive in a bit more see how the images are being built and try to find any hints .. but not really sure where to look at. Kind regards, On Sunday, November 15, 2020 at 10:52:07 PM UTC+1 Kimo wrote: > Hello, > I've been facing the exact same issue today and its driving me equally > crazy. I tried running prometheus as root but still: > > level=error ts=2020-11-15T21:45:35.983Z caller=refresh.go:98 > component="discovery manager scrape" discovery=dockerswarm msg="Unable to > refresh target groups" err="error while listing swarm *services*: Cannot > connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker > daemon running?" > level=error ts=2020-11-15T21:45:35.984Z caller=refresh.go:98 > component="discovery manager scrape" discovery=dockerswarm msg="Unable to > refresh target groups" err="error while listing swarm *nodes*: Cannot > connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker > daemon running?" > > I think I've exhausted all the options I could try by myself and would > gladly appreciate any help at this point. > Le dimanche 15 novembre 2020 à 21:54:26 UTC+1, Julien Pivotto a écrit : > >> Can you run prometheus as nobody:docker? >> On 15 Nov 12:23, Carlos Colaço wrote: >> > sorry .. also tried changing the permissions which changed nothing... >> > >> > ``` >> > # chmod +r /var/run/docker.sock >> > # ls -la /var/run/docker.sock >> > srw-rw-r--. 1 root docker 0 Nov 15 20:12 /var/run/docker.sock >> > # docker service update --force monitor_private >> > ``` >> > >> > On Sunday, November 15, 2020 at 9:20:07 PM UTC+1 Carlos Colaço wrote: >> > >> > > Hi all .. Having the same issue... >> > > >> > > https://github.com/prometheus/prometheus/issues/8185 >> > > >> > > >> > > Also don't think changing permissions on docker sock is a good option >> .. >> > > that way you are giving permissions to anyone to access it and that >> is >> > > something not desirable ... >> > > >> > > What i also tried to do instead ... since prometheus runs as Nobody ( >> uid: >> > > 65534 ) ... i added it to the Docker group which changed nothing =/ >> > > >> > > Any hints or solutions for this? driving me crazy trying different >> > > approaches and solutions.. nothing seems to work ... >> > > >> > > On Tuesday, August 11, 2020 at 7:03:12 AM UTC+2 [email protected] >> wrote: >> > > >> > >> Thanks Julien and Tom, >> > >> >> > >> I got the problem which i was facing, actually when we change the >> > >> permissions to read-write for docker.sock, permissions are only >> changed >> > >> till the docker daemon or docker service is restarted. Once the >> > >> docker/daemon is restarted then the permissions for docker sock >> changes >> > >> back to the original one. >> > >> >> > >> Is there any way using which we can make permanent changes for the >> > >> permission of docker.sock or do we need to file a issue for the >> same, as >> > >> docker/daemon might be restarted for various reasons >> > >> >> > >> >> > >> ? >> > >> >> > >> On Monday, 10 August 2020 12:40:17 UTC+5:30, Umang Goel wrote: >> > >>> >> > >>> Hello Julien, >> > >>> >> > >>> group_add is not allowed in docker swarm. Do you have any other >> > >>> workaround for this? >> > >>> >> > >>> -- >> > >>> Umang >> > >>> >> > >>> On Monday, 10 August 2020 12:20:51 UTC+5:30, Julien Pivotto wrote: >> > >>>> >> > >>>> >> > >>>> Can you use: >> > >>>> >> > >>>> --group-add docker? >> > >>>> >> > >>>> or in compose v2 file: >> > >>>> >> > >>>> version: "2.4" >> > >>>> services: >> > >>>> prometheus: >> > >>>> group_add: >> > >>>> - docker >> > >>>> >> > >>>> >> > >>>> On 09 Aug 22:48, Umang Goel wrote: >> > >>>> > ls -l /var/run/docker.sock >> > >>>> > >> > >>>> > - srwxrw-rw- 1 root docker 0 Aug 7 11:31 /var/run/docker.sock >> > >>>> after >> > >>>> > making changes as per Tom, >> > >>>> > >> > >>>> > On Sunday, 9 August 2020 02:16:28 UTC+5:30, Julien Pivotto >> wrote: >> > >>>> > > >> > >>>> > > On 07 Aug 04:36, Umang Goel wrote: >> > >>>> > > > Hello Tom, >> > >>>> > > > >> > >>>> > > > Even this is not working, I am still facing the same issue. >> Can >> > >>>> you help >> > >>>> > > me >> > >>>> > > > how did you implement it. >> > >>>> > > >> > >>>> > > >> > >>>> > > What are you current permissions on the /var/run/docker.sock ? >> > >>>> > > >> > >>>> > > ls -l /var/run/docker.sock >> > >>>> > > >> > >>>> > > > >> > >>>> > > > On Friday, 7 August 2020 16:47:23 UTC+5:30, Tom Kun wrote: >> > >>>> > > > > >> > >>>> > > > > Hello Umang, >> > >>>> > > > > >> > >>>> > > > > What are you current permissions on the >> /var/run/docker.sock ? >> > >>>> > > > > >> > >>>> > > > > I faced the same issue, and to start and no rebuild the >> > >>>> Prometheus >> > >>>> > > image >> > >>>> > > > > with the appropriate user. >> > >>>> > > > > I put the rights to read and write the docker.socket. >> > >>>> > > > > >> > >>>> > > > > sudo chmod 766 /var/run/docker.sock >> > >>>> > > > > >> > >>>> > > > > I hope this gonna help you. >> > >>>> > > > > >> > >>>> > > > > >> > >>>> > > > > On Friday, 7 August 2020 11:59:32 UTC+2, Umang Goel wrote: >> > >>>> > > > >> >> > >>>> > > > >> Hello Community, >> > >>>> > > > >> >> > >>>> > > > >> I tired using Docker Swarm Service Discovery in >> prometheus, >> > >>>> but >> > >>>> > > facing >> > >>>> > > > >> problems using it. I followed the docker swarm support >> > >>>> documentation >> > >>>> > > > >> <https://prometheus.io/docs/guides/dockerswarm/>. >> Created a >> > >>>> > > daemon.json >> > >>>> > > > >> file and mounted /var/run/docker.sock in prometheus >> container. >> > >>>> > > Container is >> > >>>> > > > >> giving permission denied error as prometheus is running >> as >> > >>>> nobody and >> > >>>> > > > >> doesn't have access to mounted /var/run/docker.sock. >> Below is >> > >>>> my >> > >>>> > > > >> prometheus.yml. >> > >>>> > > > >> Prometheus Version : v2.20.1 >> > >>>> > > > >> >> > >>>> > > > >> prometheus: >> > >>>> > > > >> image: prom/prometheus >> > >>>> > > > >> networks: >> > >>>> > > > >> - monitor >> > >>>> > > > >> ports: >> > >>>> > > > >> - "9090:9090" >> > >>>> > > > >> command: >> > >>>> > > > >> - '--config.file=/etc/prometheus/prometheus.yml' >> > >>>> > > > >> - '--storage.tsdb.path=/prometheus' >> > >>>> > > > >> - >> > >>>> '--storage.tsdb.retention=${PROMETHEUS_RETENTION:-24h}' >> > >>>> > > > >> volumes: >> > >>>> > > > >> - prometheus:/prometheus >> > >>>> > > > >> - /home/efs/devops/dsm:/etc/prometheus:ro >> > >>>> > > > >> - /var/run/docker.sock:/var/run/docker.sock:ro >> > >>>> > > > >> deploy: >> > >>>> > > > >> mode: replicated >> > >>>> > > > >> replicas: 1 >> > >>>> > > > >> resources: >> > >>>> > > > >> limits: >> > >>>> > > > >> memory: 1024M >> > >>>> > > > >> reservations: >> > >>>> > > > >> memory: 128M >> > >>>> > > > >> >> > >>>> > > > >> Prometheus.yml >> > >>>> > > > >> >> > >>>> > > > >> scrape_configs: >> > >>>> > > > >> - job_name: 'docker' >> > >>>> > > > >> dockerswarm_sd_configs: >> > >>>> > > > >> - host: unix:///var/run/docker.sock >> > >>>> > > > >> role: nodes >> > >>>> > > > >> >> > >>>> > > > >> Error: >> > >>>> > > > >> [email protected] >> > >>>> <javascript:> | >> > >>>> > > level=error >> > >>>> > > > >> ts=2020-08-06T07:21:19.106Z caller=refresh.go:98 >> > >>>> component="discovery >> > >>>> > > > >> manager scrape" discovery=dockerswarm msg="Unable to >> refresh >> > >>>> target >> > >>>> > > groups" >> > >>>> > > > >> err="error while listing swarm nodes: Got permission >> denied >> > >>>> while >> > >>>> > > trying to >> > >>>> > > > >> connect to the Docker daemon socket at >> > >>>> unix:///var/run/docker.sock: >> > >>>> > > Get >> > >>>> > > > >> \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/nodes\": dial >> unix >> > >>>> > > > >> /var/run/docker.sock: connect: permission denied >> > >>>> > > > >> >> > >>>> > > > > >> > >>>> > > > >> > >>>> > > > -- >> > >>>> > > > You received this message because you are subscribed to the >> > >>>> Google >> > >>>> > > Groups "Prometheus Users" group. >> > >>>> > > > To unsubscribe from this group and stop receiving emails >> from it, >> > >>>> send >> > >>>> > > an email to [email protected] <javascript:>. >> > >>>> > > > To view this discussion on the web visit >> > >>>> > > >> > >>>> >> https://groups.google.com/d/msgid/prometheus-users/e5e55a73-7cc1-4c0c-99e3-0a09270df62bo%40googlegroups.com. >> >> >> > >>>> >> > >>>> > > >> > >>>> > > >> > >>>> > > >> > >>>> > > -- >> > >>>> > > Julien Pivotto >> > >>>> > > @roidelapluie >> > >>>> > > >> > >>>> > >> > >>>> > -- >> > >>>> > You received this message because you are subscribed to the >> Google >> > >>>> Groups "Prometheus Users" group. >> > >>>> > To unsubscribe from this group and stop receiving emails from >> it, >> > >>>> send an email to [email protected]. >> > >>>> > To view this discussion on the web visit >> > >>>> >> https://groups.google.com/d/msgid/prometheus-users/e5614621-f57a-466e-befd-269bf77d69c8o%40googlegroups.com. >> >> >> > >>>> >> > >>>> >> > >>>> >> > >>>> -- >> > >>>> Julien Pivotto >> > >>>> @roidelapluie >> > >>>> >> > >>> >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Prometheus Users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/e058c64f-3db4-45c2-9550-c8db557d2a2cn%40googlegroups.com. >> >> >> >> >> -- >> Julien Pivotto >> @roidelapluie >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6da0bbc1-da13-4b31-9872-5d48008f4832n%40googlegroups.com.
