Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Mantas, I finally found out what was causing this mount issue. We have a global sandboxing configuration for all services, that implies PrivateNetwork=yes. According to manual this will imply PrivateMounts=yes. So, I added a drop-in configuration to user-runtime-dir@.service.d<mailto:user-runtime-dir@.service.d> which defines PrivateMounts=no. Now it works! Thank you very much for your support! Best regards, Christopher Wong From: systemd-devel on behalf of Christopher Wong Date: Wednesday, 13 December 2023 at 11:32 To: Mantas Mikulėnas Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied Hi Mantas, I ran your suggestion and here is the result: root@host:/run/user# ls -la drwxr-xr-x2 root root40 Dec 13 11:24 . drwxr-xr-x 99 root root 2160 Dec 13 11:00 .. root@host:/run/user# /usr/lib/systemd/systemd-user-runtime-dir start 1001 root@host:/run/user# ls -la drwxr-xr-x3 root root60 Dec 13 11:25 . drwxr-xr-x 99 root root 2160 Dec 13 11:00 .. drwx--2 ida ssh-user40 Dec 13 11:25 1001 root@host:/run/user# mount | grep /run tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /run/user/1001 type tmpfs (rw,nosuid,nodev,relatime,size=97096k,nr_inodes=24274,mode=700,uid=1001,gid=118) This seems to be correct, and I could see /run/user/1001 listed using “mount”. Thank you for your explaination! We have probably be running with a uncomplete setup of the mount before. Best regards, Christopher Wong From: Mantas Mikulėnas Date: Wednesday, 13 December 2023 at 10:33 To: Christopher Wong Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Wed, Dec 13, 2023 at 10:36 AM Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, I tried with StopWhenUnneeded=no in user-runtime-dir@.service<mailto:user-runtime-dir@.service>, then when user@1001.service<mailto:user@1001.service> fails the status of user-runtime-dir@.service<mailto:user-runtime-dir@.service> is active. At this state the directory /run/user/1001 is created, it is empty, owned by root. Running the mount command doesn’t show /run/user/1001. Run the "/usr/lib/systemd/systemd-user-runtime-dir start 1001" manually and check whether the mounted filesystem is there afterwards. If it's still not there, then run "mount -t tmpfs -o uid=1001,mode=0700 none /run/user/1001" and then check whether it stays mounted. I have mentioned it before, but I want to point out that if I put “ExecStartPre=+chown %i /run/user/%i” in user@.service<mailto:user@.service> then the user@1001.service<mailto:user@1001.service> can be started manually. The mount command doesn’t show /run/user/1001 either, but since the service is started the path contains bus socket and systemd directory with content, which are the things I am after. The main issue here is that /run/user/1001 is owned by root after user-runtime-dir@.service<mailto:user-runtime-dir@.service> has been exited successfully. No, that's only a symptom of the main issue. The current design that systemd implements is to have a user-specific tmpfs mounted at that location (for quota purposes), and so the underlying mountpoint is deliberately created as owned by root – its ownership is not changed because it's supposed to have a new filesystem mounted on top (which would make the mountpoint hidden and its ownership moot). If you specifically want to *not* have an additional tmpfs there, then you can continue using the manual "ExecStartPre=chown" (or in fact you could replace the entire user-runtime-dir@ with a simpler one that only mkdirs and chowns), but in that case you shouldn't be saying that it's a systemd issue that it doesn't chown something that it was never meant to chown to begin with. Best regards, Christopher Wong From: Mantas Mikulėnas mailto:graw...@gmail.com>> Date: Wednesday, 13 December 2023 at 08:08 To: Christopher Wong mailto:christopher.w...@axis.com>> Cc: Systemd mailto:systemd-devel@lists.freedesktop.org>> Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Tue, Dec 12, 2023 at 6:15 PM Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, After user@1001.service<mailto:user@1001.service> failed, it trigger the stopping process and become inactive. Ah yeah, that makes sense, user-runtime-dir@ has StopWhenUnneeded=yes – so of course after user@1001 crashes you're not going to see anything mounted anymore. Could you try temporarily removing that option / setting it to 'no', just to see what changes? ○ user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> - User Runtime Directory /run/user/1001 Loaded: loaded (/etc/systemd/system/user-runtime-
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Mantas, I ran your suggestion and here is the result: root@host:/run/user# ls -la drwxr-xr-x2 root root40 Dec 13 11:24 . drwxr-xr-x 99 root root 2160 Dec 13 11:00 .. root@host:/run/user# /usr/lib/systemd/systemd-user-runtime-dir start 1001 root@host:/run/user# ls -la drwxr-xr-x3 root root60 Dec 13 11:25 . drwxr-xr-x 99 root root 2160 Dec 13 11:00 .. drwx--2 ida ssh-user40 Dec 13 11:25 1001 root@host:/run/user# mount | grep /run tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /run/user/1001 type tmpfs (rw,nosuid,nodev,relatime,size=97096k,nr_inodes=24274,mode=700,uid=1001,gid=118) This seems to be correct, and I could see /run/user/1001 listed using “mount”. Thank you for your explaination! We have probably be running with a uncomplete setup of the mount before. Best regards, Christopher Wong From: Mantas Mikulėnas Date: Wednesday, 13 December 2023 at 10:33 To: Christopher Wong Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Wed, Dec 13, 2023 at 10:36 AM Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, I tried with StopWhenUnneeded=no in user-runtime-dir@.service<mailto:user-runtime-dir@.service>, then when user@1001.service<mailto:user@1001.service> fails the status of user-runtime-dir@.service<mailto:user-runtime-dir@.service> is active. At this state the directory /run/user/1001 is created, it is empty, owned by root. Running the mount command doesn’t show /run/user/1001. Run the "/usr/lib/systemd/systemd-user-runtime-dir start 1001" manually and check whether the mounted filesystem is there afterwards. If it's still not there, then run "mount -t tmpfs -o uid=1001,mode=0700 none /run/user/1001" and then check whether it stays mounted. I have mentioned it before, but I want to point out that if I put “ExecStartPre=+chown %i /run/user/%i” in user@.service<mailto:user@.service> then the user@1001.service<mailto:user@1001.service> can be started manually. The mount command doesn’t show /run/user/1001 either, but since the service is started the path contains bus socket and systemd directory with content, which are the things I am after. The main issue here is that /run/user/1001 is owned by root after user-runtime-dir@.service<mailto:user-runtime-dir@.service> has been exited successfully. No, that's only a symptom of the main issue. The current design that systemd implements is to have a user-specific tmpfs mounted at that location (for quota purposes), and so the underlying mountpoint is deliberately created as owned by root – its ownership is not changed because it's supposed to have a new filesystem mounted on top (which would make the mountpoint hidden and its ownership moot). If you specifically want to *not* have an additional tmpfs there, then you can continue using the manual "ExecStartPre=chown" (or in fact you could replace the entire user-runtime-dir@ with a simpler one that only mkdirs and chowns), but in that case you shouldn't be saying that it's a systemd issue that it doesn't chown something that it was never meant to chown to begin with. Best regards, Christopher Wong From: Mantas Mikulėnas mailto:graw...@gmail.com>> Date: Wednesday, 13 December 2023 at 08:08 To: Christopher Wong mailto:christopher.w...@axis.com>> Cc: Systemd mailto:systemd-devel@lists.freedesktop.org>> Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Tue, Dec 12, 2023 at 6:15 PM Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, After user@1001.service<mailto:user@1001.service> failed, it trigger the stopping process and become inactive. Ah yeah, that makes sense, user-runtime-dir@ has StopWhenUnneeded=yes – so of course after user@1001 crashes you're not going to see anything mounted anymore. Could you try temporarily removing that option / setting it to 'no', just to see what changes? ○ user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> - User Runtime Directory /run/user/1001 Loaded: loaded (/etc/systemd/system/user-runtime-dir@.service<mailto:/etc/systemd/system/user-runtime-dir@.service>; static) Drop-In: /usr/lib/systemd/system/service.d └─10-axis.conf, 20-axis-sandbox.conf Active: inactive (dead) since Tue 2023-12-12 16:33:35 CET; 36min ago Duration: 315ms Docs: man:user@.service(5) Process: 16325 ExecStartPre=ls -la /run/user (code=exited, status=0/SUCCESS) Process: 16327 ExecStartPre=mount (code=exited, status=0/SUCCESS) Process: 16329 ExecStart=/usr/lib/systemd/systemd-user-runtime-dir start 1001 (code=exited, status=0/SUCCESS) Process: 16334 ExecStartPost=sleep 5 (code=exited, status=0/SUCCESS) Process: 16347
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Mantas, I tried with StopWhenUnneeded=no in user-runtime-dir@.service<mailto:user-runtime-dir@.service>, then when user@1001.service<mailto:user@1001.service> fails the status of user-runtime-dir@.service<mailto:user-runtime-dir@.service> is active. At this state the directory /run/user/1001 is created, it is empty, owned by root. Running the mount command doesn’t show /run/user/1001. ● user-runtime-dir@1001.service - User Runtime Directory /run/user/1001 Loaded: loaded (/etc/systemd/system/user-runtime-dir@.service; static) Drop-In: /usr/lib/systemd/system/service.d └─10-axis.conf, 20-axis-sandbox.conf Active: active (exited) since Wed 2023-12-13 09:16:57 CET; 33s ago Docs: man:user@.service(5) Process: 133303 ExecStartPre=ls -la /run/user (code=exited, status=0/SUCCESS) Process: 133305 ExecStartPre=mount (code=exited, status=0/SUCCESS) Process: 133307 ExecStart=/usr/lib/systemd/systemd-user-runtime-dir start 1001 (code=exited, status=0/SUCCESS) Process: 133309 ExecStartPost=ls -la /run/user/1001 (code=exited, status=0/SUCCESS) Process: 133310 ExecStartPost=mount (code=exited, status=0/SUCCESS) Main PID: 133307 (code=exited, status=0/SUCCESS) CPU: 209ms I have mentioned it before, but I want to point out that if I put “ExecStartPre=+chown %i /run/user/%i” in user@.service<mailto:user@.service> then the user@1001.service<mailto:user@1001.service> can be started manually. The mount command doesn’t show /run/user/1001 either, but since the service is started the path contains bus socket and systemd directory with content, which are the things I am after. The main issue here is that /run/user/1001 is owned by root after user-runtime-dir@.service<mailto:user-runtime-dir@.service> has been exited successfully. Best regards, Christopher Wong From: Mantas Mikulėnas Date: Wednesday, 13 December 2023 at 08:08 To: Christopher Wong Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Tue, Dec 12, 2023 at 6:15 PM Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, After user@1001.service<mailto:user@1001.service> failed, it trigger the stopping process and become inactive. Ah yeah, that makes sense, user-runtime-dir@ has StopWhenUnneeded=yes – so of course after user@1001 crashes you're not going to see anything mounted anymore. Could you try temporarily removing that option / setting it to 'no', just to see what changes? ○ user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> - User Runtime Directory /run/user/1001 Loaded: loaded (/etc/systemd/system/user-runtime-dir@.service<mailto:/etc/systemd/system/user-runtime-dir@.service>; static) Drop-In: /usr/lib/systemd/system/service.d └─10-axis.conf, 20-axis-sandbox.conf Active: inactive (dead) since Tue 2023-12-12 16:33:35 CET; 36min ago Duration: 315ms Docs: man:user@.service(5) Process: 16325 ExecStartPre=ls -la /run/user (code=exited, status=0/SUCCESS) Process: 16327 ExecStartPre=mount (code=exited, status=0/SUCCESS) Process: 16329 ExecStart=/usr/lib/systemd/systemd-user-runtime-dir start 1001 (code=exited, status=0/SUCCESS) Process: 16334 ExecStartPost=sleep 5 (code=exited, status=0/SUCCESS) Process: 16347 ExecStartPost=ls -la /run/user/1001 (code=exited, status=0/SUCCESS) Process: 16351 ExecStartPost=mount (code=exited, status=0/SUCCESS) Process: 16361 ExecStop=/usr/lib/systemd/systemd-user-runtime-dir stop 1001 (code=exited, status=0/SUCCESS) Main PID: 16329 (code=exited, status=0/SUCCESS) CPU: 48ms /etc/fstab don’t include anything on /run/user/1001 and there is no mount unit for run-user-1001.mount either. Best regards, Christopher Wong From: Mantas Mikulėnas mailto:graw...@gmail.com>> Date: Tuesday, 12 December 2023 at 17:05 To: Christopher Wong mailto:christopher.w...@axis.com>> Cc: Systemd mailto:systemd-devel@lists.freedesktop.org>> Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied That sounds like it's getting immediately unmounted (or maybe not being mounted at all despite the program doing so). Does the user-runtime-dir service continue to show as "active" after this, or does it return to "inactive"? Does your /etc/fstab have any mentions of /run/user/1001? Or more generally, are there any run-user-1001.mount units? (If you 'systemctl status' this unit, does the status include a source path?) On Tue, Dec 12, 2023, 17:34 Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, I currently have the following flow: 1. No /run/user/1001 directory 2. systemctl start user@1001.service<mailto:user@1001.service> 3. systemd start user-runtime-dir@1001.service<mailto:user-runtime-dir@1
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Mantas, After user@1001.service<mailto:user@1001.service> failed, it trigger the stopping process and become inactive. ○ user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> - User Runtime Directory /run/user/1001 Loaded: loaded (/etc/systemd/system/user-runtime-dir@.service<mailto:/etc/systemd/system/user-runtime-dir@.service>; static) Drop-In: /usr/lib/systemd/system/service.d └─10-axis.conf, 20-axis-sandbox.conf Active: inactive (dead) since Tue 2023-12-12 16:33:35 CET; 36min ago Duration: 315ms Docs: man:user@.service(5) Process: 16325 ExecStartPre=ls -la /run/user (code=exited, status=0/SUCCESS) Process: 16327 ExecStartPre=mount (code=exited, status=0/SUCCESS) Process: 16329 ExecStart=/usr/lib/systemd/systemd-user-runtime-dir start 1001 (code=exited, status=0/SUCCESS) Process: 16334 ExecStartPost=sleep 5 (code=exited, status=0/SUCCESS) Process: 16347 ExecStartPost=ls -la /run/user/1001 (code=exited, status=0/SUCCESS) Process: 16351 ExecStartPost=mount (code=exited, status=0/SUCCESS) Process: 16361 ExecStop=/usr/lib/systemd/systemd-user-runtime-dir stop 1001 (code=exited, status=0/SUCCESS) Main PID: 16329 (code=exited, status=0/SUCCESS) CPU: 48ms /etc/fstab don’t include anything on /run/user/1001 and there is no mount unit for run-user-1001.mount either. Best regards, Christopher Wong From: Mantas Mikulėnas Date: Tuesday, 12 December 2023 at 17:05 To: Christopher Wong Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied That sounds like it's getting immediately unmounted (or maybe not being mounted at all despite the program doing so). Does the user-runtime-dir service continue to show as "active" after this, or does it return to "inactive"? Does your /etc/fstab have any mentions of /run/user/1001? Or more generally, are there any run-user-1001.mount units? (If you 'systemctl status' this unit, does the status include a source path?) On Tue, Dec 12, 2023, 17:34 Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, I currently have the following flow: 1. No /run/user/1001 directory 2. systemctl start user@1001.service<mailto:user@1001.service> 3. systemd start user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> which ends successfully. 4. The directory /run/user/1001 exists now, but is empty, owned by root with mode 0700 5. I don’t have findmnt on my system, so I used mount, but /run/user/1001 is not listed. 6. systemd start user@1001.service<mailto:user@1001.service> which fails due to permission denied. I can’t explain why the /run/user/1001 is owned by root after user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> successfully exited. I added some personal print in systemd code to ensure that the mount command returned success (r=0). Although, the mount was successful the command “mount” didn’t list it. In the list of mounts starting with /run I could only find these entries: Dec 12 16:19:35 host mount[14500]: tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) Dec 12 16:19:35 host mount[14500]: tmpfs on /run/credentials type tmpfs (ro,nosuid,nodev,noexec,mode=755) Dec 12 16:19:35 host mount[14500]: tmpfs on /run/systemd/incoming type tmpfs (ro,nosuid,nodev,mode=755) If I do a chown of the directory in user@1001.service<mailto:user@1001.service> then it works root@host:/run/user# ls -la 1001 drwx--3 ida root80 Dec 12 16:19 . drwxr-xr-x3 root root60 Dec 12 16:19 .. srw-rw-rw-1 ida ssh-user 0 Dec 12 16:19 bus drwxr-xr-x5 ida ssh-user 140 Dec 12 16:19 systemd The ”mount” command don’t list /run/user/1001 for the successful case either. Best regards, Christopher Wong From: Mantas Mikulėnas mailto:graw...@gmail.com>> Date: Monday, 11 December 2023 at 17:56 To: Christopher Wong mailto:christopher.w...@axis.com>> Cc: Systemd mailto:systemd-devel@lists.freedesktop.org>> Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Mon, Dec 11, 2023, 17:28 Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, I have added ExecStartPre to user@.service<mailto:user@.service> to run “id” and “ls -la”: Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Will mount /run/user/1001 owned by 1001:118 Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Mounting tmpfs (tmpfs) on /run/user/1001 (MS_NOSUID|MS_NODEV "mode=0700,uid=1001,gid=118,size=99426304,nr_inodes=24274")... Dec 11 15:50:34 host systemd[1]: Finished User Runtime Directory /run/user/1001. Dec 11 15:50:34 host systemd[1]: Starting User Manager for UID 1001... Dec 11 15:50:34 host id[40291]: uid=1001(ida) gid=118(ssh-users) groups=1
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Andrei, The systemd logs tells me that /run/user/1001 is mounted as uid=1001, but when I list the path /run/user/1001 it is empty and is owned by root. I can’t find the path when I run the “mount” command. However, even for the successful case the path is not listed with the “mount” command. Best regards, Christopher Wong From: Andrei Borzenkov Date: Monday, 11 December 2023 at 19:34 To: Christopher Wong , Mantas Mikulėnas Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On 11.12.2023 18:28, Christopher Wong wrote: > Hi Mantas, > > I have added ExecStartPre to user@.service<mailto:user@.service> to run “id” > and “ls -la”: > > Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Will mount > /run/user/1001 owned by 1001:118 > Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Mounting tmpfs (tmpfs) > on /run/user/1001 (MS_NOSUID|MS_NODEV > "mode=0700,uid=1001,gid=118,size=99426304,nr_inodes=24274")... > Dec 11 15:50:34 host systemd[1]: Finished User Runtime Directory > /run/user/1001. > Dec 11 15:50:34 host systemd[1]: Starting User Manager for UID 1001... > Dec 11 15:50:34 host id[40291]: uid=1001(ida) gid=118(ssh-users) > groups=118(ssh-users),236(systemd-journal) > Dec 11 15:50:34 host ls[40293]: drwxr-xr-x3 root root60 > Dec 11 15:50 . > Dec 11 15:50:34 host ls[40293]: drwxr-xr-x 98 root root 2120 > Dec 11 15:30 .. > Dec 11 15:50:34 host ls[40293]: drwx--2 root root40 > Dec 11 15:50 1001 > Dec 11 15:50:34 host systemd[40294]: systemd 254.7-2-g9edc143 running in user > mode for user 1001/ida. (-PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK +SECCOMP > +GCRYPT +GNUTLS +OPENSSL -ACL +BLKID +CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC > +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 > +BZIP2 -LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP -SYSVINIT > default-hierarchy=unified) > > The /run/user/1001 belongs to root with mode 0700. Should this belong to root? No. > Is it because I manually start user@1001.service<mailto:user@1001.service> as > root? No. > However, after > user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> has > finished it startup, the user@1001.service<mailto:user@1001.service> is > started as uid=1001 and therefore can’t create any directories under > /run/user/1001. Resulting in user@1001.service<mailto:user@1001.service> > failed to start. > > If I add “ExecStartPre=+chown %i /run/user/%i” to > user@.service<mailto:user@.service> then it works! But I am unsure if this is > really the way fix this. As clearly seen from logs, systemd-user-runtime-dir mounts tmpfs with option uid=1001 over /run/user/1001. Is it still a mounted filesystem when you check it? It sounds like you see mount point which indeed has permissions 700 and owner root, not mounted filesystem. > > Regarding the testing, I have done both restart of everything and manual, but > the result is the same. Now that I have the > “Environment=XDG_RUNTIME_DIR=/run/user/%i” I no longer need to do “systemctl > set-environment …” > > Thank you for taking your time! > > Best regards, > Christopher Wong > > > From: Mantas Mikulėnas > Date: Friday, 8 December 2023 at 21:53 > To: Christopher Wong > Cc: Systemd > Subject: Re: [systemd-devel] Manual start of user@.service failed with > permission denied > On Fri, Dec 8, 2023 at 6:53 PM Christopher Wong > mailto:christopher.w...@axis.com>> wrote: > Hi Mantas, > > I have from your suggestion done the following: > > Putting the below in user@.service > > [Service] > ... > Environment=XDG_RUNTIME_DIR=/run/user/%i > Environment=SYSTEMD_LOG_LEVEL=debug > > Putting the below in user-runtime-dir@.service > > [Service] > ... > Environment=SYSTEMD_LOG_LEVEL=debug > > Then I have disabled the global set-log-level debug (if this is also > required, please let me know). > > Unlike set-environment that's not global, it only affects pid1. > > > What I can see from the logs is that > user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> seems to > be started and mount /run/user/1001, but addition creation of directory under > this mount is getting permission denied. > > Dec 08 17:33:29 host systemd[1]: Created slice User Slice of UID 1001. > Dec 08 17:33:29 host systemd[1]: Starting User Runtime Directory > /run/user/1001... > Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state > UNSET -> OPENING > Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: sd-bus: starting bus by > connecting to /run/dbus
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Belanger, Thank you for your suggestion! I tried putting RuntimeDirectory= into the user@.service<mailto:user@.service>. I suppose I shouldn’t need to do this since the runtime directory should be fixed by the service user-runtime-dir@.service<mailto:user-runtime-dir@.service>. Result is that it didn’t work. Best regards, Christopher Wong From: Belanger, Martin Date: Monday, 11 December 2023 at 17:43 To: Christopher Wong , Mantas Mikulėnas Cc: Systemd Subject: RE: [systemd-devel] Manual start of user@.service failed with permission denied Have you tried using "RuntimeDirectory=" (and optionally "RuntimeDirectoryPreserve=") Ref: https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#RuntimeDirectory= Regards, Martin From: systemd-devel On Behalf Of Christopher Wong Sent: Monday, December 11, 2023 10:28 AM To: Mantas Mikulėnas Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied [EXTERNAL EMAIL] Hi Mantas, I have added ExecStartPre to mailto:user@.service to run “id” and “ls -la”: Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Will mount /run/user/1001 owned by 1001:118 Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Mounting tmpfs (tmpfs) on /run/user/1001 (MS_NOSUID|MS_NODEV "mode=0700,uid=1001,gid=118,size=99426304,nr_inodes=24274")... Dec 11 15:50:34 host systemd[1]: Finished User Runtime Directory /run/user/1001. Dec 11 15:50:34 host systemd[1]: Starting User Manager for UID 1001... Dec 11 15:50:34 host id[40291]: uid=1001(ida) gid=118(ssh-users) groups=118(ssh-users),236(systemd-journal) Dec 11 15:50:34 host ls[40293]: drwxr-xr-x3 root root60 Dec 11 15:50 . Dec 11 15:50:34 host ls[40293]: drwxr-xr-x 98 root root 2120 Dec 11 15:30 .. Dec 11 15:50:34 host ls[40293]: drwx--2 root root40 Dec 11 15:50 1001 Dec 11 15:50:34 host systemd[40294]: systemd 254.7-2-g9edc143 running in user mode for user 1001/ida. (-PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL -ACL +BLKID +CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 +BZIP2 -LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP -SYSVINIT default-hierarchy=unified) The /run/user/1001 belongs to root with mode 0700. Should this belong to root? Is it because I manually start mailto:user@1001.service as root? However, after mailto:user-runtime-dir@1001.service has finished it startup, the mailto:user@1001.service is started as uid=1001 and therefore can’t create any directories under /run/user/1001. Resulting in mailto:user@1001.service failed to start. If I add “ExecStartPre=+chown %i /run/user/%i” to mailto:user@.service then it works! But I am unsure if this is really the way fix this. Regarding the testing, I have done both restart of everything and manual, but the result is the same. Now that I have the “Environment=XDG_RUNTIME_DIR=/run/user/%i” I no longer need to do “systemctl set-environment …” Thank you for taking your time! Best regards, Christopher Wong From: Mantas Mikulėnas <mailto:graw...@gmail.com> Date: Friday, 8 December 2023 at 21:53 To: Christopher Wong <mailto:christopher.w...@axis.com> Cc: Systemd <mailto:systemd-devel@lists.freedesktop.org> Subject: Re: [systemd-devel] Manual start of mailto:user@.service failed with permission denied On Fri, Dec 8, 2023 at 6:53 PM Christopher Wong <mailto:christopher.w...@axis.com> wrote: Hi Mantas, I have from your suggestion done the following: Putting the below in mailto:user@.service [Service] ... Environment=XDG_RUNTIME_DIR=/run/user/%i Environment=SYSTEMD_LOG_LEVEL=debug Putting the below in mailto:user-runtime-dir@.service [Service] ... Environment=SYSTEMD_LOG_LEVEL=debug Then I have disabled the global set-log-level debug (if this is also required, please let me know). Unlike set-environment that's not global, it only affects pid1. What I can see from the logs is that mailto:user-runtime-dir@1001.service seems to be started and mount /run/user/1001, but addition creation of directory under this mount is getting permission denied. Dec 08 17:33:29 host systemd[1]: Created slice User Slice of UID 1001. Dec 08 17:33:29 host systemd[1]: Starting User Runtime Directory /run/user/1001... Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state UNSET -> OPENING Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: sd-bus: starting bus by connecting to /run/dbus/system_bus_socket... Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state OPENING -> AUTHENTICATING Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state AUTHENTICATING -> HELLO Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Sent message type=method_call sender=n/a destination=org.freedesktop.D
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Mantas, I currently have the following flow: 1. No /run/user/1001 directory 2. systemctl start user@1001.service<mailto:user@1001.service> 3. systemd start user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> which ends successfully. 4. The directory /run/user/1001 exists now, but is empty, owned by root with mode 0700 5. I don’t have findmnt on my system, so I used mount, but /run/user/1001 is not listed. 6. systemd start user@1001.service<mailto:user@1001.service> which fails due to permission denied. I can’t explain why the /run/user/1001 is owned by root after user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> successfully exited. I added some personal print in systemd code to ensure that the mount command returned success (r=0). Although, the mount was successful the command “mount” didn’t list it. In the list of mounts starting with /run I could only find these entries: Dec 12 16:19:35 host mount[14500]: tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) Dec 12 16:19:35 host mount[14500]: tmpfs on /run/credentials type tmpfs (ro,nosuid,nodev,noexec,mode=755) Dec 12 16:19:35 host mount[14500]: tmpfs on /run/systemd/incoming type tmpfs (ro,nosuid,nodev,mode=755) If I do a chown of the directory in user@1001.service<mailto:user@1001.service> then it works root@host:/run/user# ls -la 1001 drwx--3 ida root80 Dec 12 16:19 . drwxr-xr-x3 root root60 Dec 12 16:19 .. srw-rw-rw-1 ida ssh-user 0 Dec 12 16:19 bus drwxr-xr-x5 ida ssh-user 140 Dec 12 16:19 systemd The ”mount” command don’t list /run/user/1001 for the successful case either. Best regards, Christopher Wong From: Mantas Mikulėnas Date: Monday, 11 December 2023 at 17:56 To: Christopher Wong Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Mon, Dec 11, 2023, 17:28 Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, I have added ExecStartPre to user@.service<mailto:user@.service> to run “id” and “ls -la”: Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Will mount /run/user/1001 owned by 1001:118 Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Mounting tmpfs (tmpfs) on /run/user/1001 (MS_NOSUID|MS_NODEV "mode=0700,uid=1001,gid=118,size=99426304,nr_inodes=24274")... Dec 11 15:50:34 host systemd[1]: Finished User Runtime Directory /run/user/1001. Dec 11 15:50:34 host systemd[1]: Starting User Manager for UID 1001... Dec 11 15:50:34 host id[40291]: uid=1001(ida) gid=118(ssh-users) groups=118(ssh-users),236(systemd-journal) Dec 11 15:50:34 host ls[40293]: drwxr-xr-x3 root root60 Dec 11 15:50 . Dec 11 15:50:34 host ls[40293]: drwxr-xr-x 98 root root 2120 Dec 11 15:30 .. Dec 11 15:50:34 host ls[40293]: drwx--2 root root40 Dec 11 15:50 1001 Dec 11 15:50:34 host systemd[40294]: systemd 254.7-2-g9edc143 running in user mode for user 1001/ida. (-PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL -ACL +BLKID +CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 +BZIP2 -LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP -SYSVINIT default-hierarchy=unified) The /run/user/1001 belongs to root with mode 0700. Should this belong to root? No, it should be owned by UID 1001 (though still mode 0700). Is it because I manually start user@1001.service<mailto:user@1001.service> as root? Which user started the .service is usually not important, all services get a "fresh" environment that's fully described by the unit file. So even if you did 'systemctl start' as root, the unit has User=%i so the instance parameter tells it which UID to run as, so will be running as UID 1001. Likewise user-runtime-dir@1001 will get the UID for the mount from its instance name (you can see that the "Mounting tmpfs" message has the correct information). However, after user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> has finished it startup, the user@1001.service<mailto:user@1001.service> is started as uid=1001 and therefore can’t create any directories under /run/user/1001. Resulting in user@1001.service<mailto:user@1001.service> failed to start. If I add “ExecStartPre=+chown %i /run/user/%i” to user@.service<mailto:user@.service> then it works! But I am unsure if this is really the way fix this. So far, it sounds like the directory is being created *by something else* before user-runtime-dir@ is even invoked. Try adding the same "-/bin/ls -lad /run/user/%i" as both ExecStartPre and ExecStartPost of user-runtime-dir@ (and maybe even a findmnt). If the directory already exists during ExecStartPre, start looking for other services or cronjob
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Andrei, As indicated in the logs no SELINUX nor APPARMOR is enabled. Best regards, Christopher Wong From: systemd-devel on behalf of Andrei Borzenkov Date: Saturday, 9 December 2023 at 07:13 To: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On 08.12.2023 23:53, Mantas Mikulėnas wrote: ... >> >> Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Will mount >> /run/user/1001 owned by 1001:118 >> >> Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Mounting tmpfs >> (tmpfs) on /run/user/1001 (MS_NOSUID|MS_NODEV >> "mode=0700,uid=1001,gid=118,size=99426304,nr_inodes=24274")... >> >> Dec 08 17:33:29 host systemd[1]: Finished User Runtime Directory >> /run/user/1001. >> >> Dec 08 17:33:29 host systemd[1]: Starting User Manager for UID 1001... >> >> Dec 08 17:33:29 host systemd[36280]: systemd 254.7-2-g9edc143 running in >> user mode for user 1001/ida. (-PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK >> +SECCOMP +GCRYPT +GNUTLS +OPENSSL -ACL +BLKID +CURL -ELFUTILS -FIDO2 -IDN2 >> -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT >> -QRENCODE -TPM2 +BZIP2 -LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP >> -SYSVINIT default-hierarchy=unified) >> >> Dec 08 17:33:29 host systemd[36280]: Failed to create >> '/run/user/1001/systemd/inaccessible', ignoring: Permission denied >> >> Dec 08 17:33:29 host systemd[36280]: Failed to create >> '/run/user/1001/systemd/inaccessible/reg', ignoring: Permission denied >> >> Dec 08 17:33:29 host systemd[36280]: Failed to create >> '/run/user/1001/systemd/inaccessible/dir', ignoring: Permission denied >> >> Dec 08 17:33:29 host systemd[36280]: Failed to create >> '/run/user/1001/systemd/inaccessible/fifo', ignoring: Permission denied >> >> Dec 08 17:33:29 host systemd[36280]: Failed to create >> '/run/user/1001/systemd/inaccessible/sock', ignoring: Permission denied >> >> Dec 08 17:33:29 host systemd[36280]: Failed to create >> '/run/user/1001/systemd/inaccessible/chr', ignoring: Permission denied >> >> Dec 08 17:33:29 host systemd[36280]: Failed to create >> '/run/user/1001/systemd/inaccessible/blk', ignoring: Permission denied >> > > What's the ownership of /run/user/1001 and /run/user/1001/systemd after all > of this? > > Are you rebooting between tests or just manually starting it? > > My current guess is that due to the earlier `systemctl set-environment`, > some *other* thing that's running as root inherited the /run/user/1001 path > and created root-owned directories there? That's the issue with setting > global environment, it needs to be unset afterwards... > "Permission denied" sounds like something LSM related (AppArmor, SELinux, ...)
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Mantas, I have added ExecStartPre to user@.service<mailto:user@.service> to run “id” and “ls -la”: Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Will mount /run/user/1001 owned by 1001:118 Dec 11 15:50:34 host systemd-user-runtime-dir[40287]: Mounting tmpfs (tmpfs) on /run/user/1001 (MS_NOSUID|MS_NODEV "mode=0700,uid=1001,gid=118,size=99426304,nr_inodes=24274")... Dec 11 15:50:34 host systemd[1]: Finished User Runtime Directory /run/user/1001. Dec 11 15:50:34 host systemd[1]: Starting User Manager for UID 1001... Dec 11 15:50:34 host id[40291]: uid=1001(ida) gid=118(ssh-users) groups=118(ssh-users),236(systemd-journal) Dec 11 15:50:34 host ls[40293]: drwxr-xr-x3 root root60 Dec 11 15:50 . Dec 11 15:50:34 host ls[40293]: drwxr-xr-x 98 root root 2120 Dec 11 15:30 .. Dec 11 15:50:34 host ls[40293]: drwx--2 root root40 Dec 11 15:50 1001 Dec 11 15:50:34 host systemd[40294]: systemd 254.7-2-g9edc143 running in user mode for user 1001/ida. (-PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL -ACL +BLKID +CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 +BZIP2 -LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP -SYSVINIT default-hierarchy=unified) The /run/user/1001 belongs to root with mode 0700. Should this belong to root? Is it because I manually start user@1001.service<mailto:user@1001.service> as root? However, after user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> has finished it startup, the user@1001.service<mailto:user@1001.service> is started as uid=1001 and therefore can’t create any directories under /run/user/1001. Resulting in user@1001.service<mailto:user@1001.service> failed to start. If I add “ExecStartPre=+chown %i /run/user/%i” to user@.service<mailto:user@.service> then it works! But I am unsure if this is really the way fix this. Regarding the testing, I have done both restart of everything and manual, but the result is the same. Now that I have the “Environment=XDG_RUNTIME_DIR=/run/user/%i” I no longer need to do “systemctl set-environment …” Thank you for taking your time! Best regards, Christopher Wong From: Mantas Mikulėnas Date: Friday, 8 December 2023 at 21:53 To: Christopher Wong Cc: Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Fri, Dec 8, 2023 at 6:53 PM Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Mantas, I have from your suggestion done the following: Putting the below in user@.service [Service] ... Environment=XDG_RUNTIME_DIR=/run/user/%i Environment=SYSTEMD_LOG_LEVEL=debug Putting the below in user-runtime-dir@.service [Service] ... Environment=SYSTEMD_LOG_LEVEL=debug Then I have disabled the global set-log-level debug (if this is also required, please let me know). Unlike set-environment that's not global, it only affects pid1. What I can see from the logs is that user-runtime-dir@1001.service<mailto:user-runtime-dir@1001.service> seems to be started and mount /run/user/1001, but addition creation of directory under this mount is getting permission denied. Dec 08 17:33:29 host systemd[1]: Created slice User Slice of UID 1001. Dec 08 17:33:29 host systemd[1]: Starting User Runtime Directory /run/user/1001... Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state UNSET -> OPENING Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: sd-bus: starting bus by connecting to /run/dbus/system_bus_socket... Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state OPENING -> AUTHENTICATING Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state AUTHENTICATING -> HELLO Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello cookie=1 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.2536 path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=s error-name=n/a error-message=n/a Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Bus n/a: changing state HELLO -> RUNNING Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Sent message type=method_call sender=n/a destination=org.freedesktop.login1 path=/org/freedesktop/login1 interface=org.freedesktop.DBus.Properties member=Get cookie=2 reply_cookie=0 signature=ss error-name=n/a error-message=n/a Dec 08 17:33:29 host systemd-user-runtime-dir[36278]: Got message type=method_return sender=:1.323 destination=:1.2536 path=n/a interface=n/a member=n/a cookie=15 reply_cookie=2 signature=v error-name=n/a
Re: [systemd-devel] Manual start of user@.service failed with permission denied
6280]: Failed at setting rlimit 1073741816 for resource RLIMIT_NOFILE. Will attempt setting value 524288 instead. Dec 08 17:33:29 host systemd[36280]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 08 17:33:29 host systemd[36280]: Unified cgroup hierarchy is located at /sys/fs/cgroup/user.slice/user-1001.slice/user@1001.service. Dec 08 17:33:29 host systemd[36280]: bpf-firewall: Got EBADF when using BPF_F_ALLOW_MULTI, which indicates it is supported. Yay! Dec 08 17:33:29 host systemd[36280]: Not enough privileges, BPF device control is not supported. Dec 08 17:33:29 host systemd[36280]: Controller 'cpu' supported: yes Dec 08 17:33:29 host systemd[36280]: Controller 'cpuacct' supported: no Dec 08 17:33:29 host systemd[36280]: Controller 'cpuset' supported: no Dec 08 17:33:29 host systemd[36280]: Controller 'io' supported: no Dec 08 17:33:29 host systemd[36280]: Controller 'blkio' supported: no Dec 08 17:33:29 host systemd[36280]: Controller 'memory' supported: yes Dec 08 17:33:29 host systemd[36280]: Controller 'devices' supported: no Dec 08 17:33:29 host systemd[36280]: Controller 'pids' supported: yes Dec 08 17:33:29 host systemd[36280]: Controller 'bpf-firewall' supported: yes Dec 08 17:33:29 host systemd[36280]: Controller 'bpf-devices' supported: no Dec 08 17:33:29 host systemd[36280]: Controller 'bpf-foreign' supported: yes Dec 08 17:33:29 host systemd[36280]: Controller 'bpf-socket-bind' supported: no Dec 08 17:33:29 host systemd[36280]: Controller 'bpf-restrict-network-interfaces' supported: no Dec 08 17:33:29 host systemd[36280]: Set up TFD_TIMER_CANCEL_ON_SET timerfd. Dec 08 17:33:29 host systemd[36280]: Failed to establish memory pressure event source, ignoring: Operation not supported Dec 08 17:33:29 host systemd[36280]: Failed to allocate manager object: Permission denied Dec 08 17:33:29 host systemd[1]: user@1001.service: Main process exited, code=exited, status=1/FAILURE Dec 08 17:33:29 host systemd[1]: user@1001.service: Failed with result 'exit-code'. Dec 08 17:33:29 host systemd[1]: Failed to start User Manager for UID 1001. Dec 08 17:33:29 host systemd[1]: Stopping User Runtime Directory /run/user/1001... Dec 08 17:33:29 host systemd-user-runtime-dir[36283]: Will remove /run/user/1001 Dec 08 17:33:29 host systemd[1]: user-runtime-dir@1001.service: Deactivated successfully. Dec 08 17:33:29 host systemd[1]: Stopped User Runtime Directory /run/user/1001. Dec 08 17:33:29 host systemd[1]: Removed slice User Slice of UID 1001. Best regards, Christopher Wong From: Mantas Mikulėnas Date: Friday, 8 December 2023 at 12:11 To: Christopher Wong Cc: Luca Boccassi , Systemd Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Fri, Dec 8, 2023, 12:22 Christopher Wong mailto:christopher.w...@axis.com>> wrote: Hi Luca, Sorry, for late reply, below is a log with debug. This time I run with a user with higher UID, but the result is the same. root@host:~# systemd-analyze set-log-level debug root@host:~# systemctl set-environment XDG_RUNTIME_DIR="/run/user/1001" I'd avoid doing that globally. If you really want to have a PAM-less system, then edit the unit to set this through its Environment= instead. root@host:~# systemctl start user@1001.service Job for user@1001.service failed because the control process exited with error code. See "systemctl status user@1001.service" and "journalctl -xeu user@1001.service" for details. root@host:~# journalctl -xeu user@1001.service Dec 08 09:35:53 host systemd[1]: /usr/lib/systemd/system/user@.service:19: Support for option PAMName= has been disabled at compile time and it is ignored Dec 08 09:35:53 host systemd[1]: user@1001.service: Trying to enqueue job user@1001.service/start/replace Dec 08 09:35:53 host systemd[1]: user@1001.service: Installed new job user@1001.service/start as 6724 Dec 08 09:35:53 host systemd[1]: user@1001.service: Enqueued job user@1001.service/start as 6724 Dec 08 09:35:53 host systemd[1]: user@1001.service: starting held back, waiting for: user-runtime-dir@1001.service Dec 08 09:35:54 host systemd[1]: user@1001.service: Will spawn child (service_enter_start): /usr/lib/systemd/systemd Dec 08 09:35:54 host systemd[1]: user@1001.service: Failed to set 'memory.zswap.max' attribute on '/user.slice/user-1001.slice/user@1001.service' to 'max': No such file or directory Dec 08 09:35:54 host systemd[1]: user@1001.service: Passing 0 fds to service Dec 08 09:35:54 host systemd[1]: user@1001.service: About to execute: /usr/lib/systemd/systemd --user Dec 08 09:35:54 host systemd[1]: user@1001.service: Forked /usr/lib/systemd/systemd as 6899 Dec 08 09:35:54 host (systemd)[6899]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 08 09:35:54 host systemd[1]: user@1001.service: Changed dead -> start Dec 08 09:35:54 host systemd[1]: Starting User Manager for UID 1001... Dec 08 09:35:54 host (systemd)[6899]: Bind-mounting / on
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Lennart, I know we are not using the pam_systemd. That is the reason we try to run the steps manually. It was possible to start the user@.service in systemd v253, but it fails now with v254 or later. Best regards, Christopher Wong From: Lennart Poettering Date: Thursday, 7 December 2023 at 22:03 To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Do, 07.12.23 18:29, Christopher Wong (christopher.w...@axis.com) wrote: > Hi Lennart, > > We are doing the steps to start up a rootless docker. If I don’t set > XDG_RUNTIME_DIR then I will get the below error: > > systemd[1925]: Trying to run as user instance, but $XDG_RUNTIME_DIR > is not set. pam_systemd is responsible for setting this env var. Most likely you are missing that from the PAM stack that is used by this user@.service instance? > The 503 is a system user. So, just to try it out, I created a user, > which got the UID 1001. Using that UID gave me the same result as > the 503. It's a bad idea to run user stuff as system user. Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Manual start of user@.service failed with permission denied
ystemd[1]: user@1001.service: Main process exited, code=exited, status=1/FAILURE Dec 08 09:35:54 host systemd[1]: user@1001.service: Failed with result 'exit-code'. Dec 08 09:35:54 host systemd[1]: user@1001.service: Service will not restart (restart setting) Dec 08 09:35:54 host systemd[1]: user@1001.service: Changed start -> failed Dec 08 09:35:54 host systemd[1]: user@1001.service: Job 6724 user@1001.service/start finished, result=failed Dec 08 09:35:54 host systemd[1]: Failed to start User Manager for UID 1001. Dec 08 09:35:54 host systemd[1]: user@1001.service: Unit entered failed state. Dec 08 09:35:54 host systemd[1]: user@1001.service: Consumed 63ms CPU time. Dec 08 09:35:54 host systemd[1]: user@1001.service: Releasing resources... Best regards, Christopher Wong From: Luca Boccassi Date: Wednesday, 6 December 2023 at 17:46 To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Wed, 6 Dec 2023 at 16:00, Christopher Wong wrote: > Hi, > > I’m trying to do the following: > > root@host:~# systemctl set-environment XDG_RUNTIME_DIR="/run/user/503" Why are you setting this? Anyway, enable debug level log and attach the output, otherwise it's hard to say
Re: [systemd-devel] Manual start of user@.service failed with permission denied
Hi Lennart, We are doing the steps to start up a rootless docker. If I don’t set XDG_RUNTIME_DIR then I will get the below error: systemd[1925]: Trying to run as user instance, but $XDG_RUNTIME_DIR is not set. The 503 is a system user. So, just to try it out, I created a user, which got the UID 1001. Using that UID gave me the same result as the 503. Best regards, Christopher Wong From: Lennart Poettering Date: Wednesday, 6 December 2023 at 16:50 To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Manual start of user@.service failed with permission denied On Mi, 06.12.23 14:46, Christopher Wong (christopher.w...@axis.com) wrote: > Hi, > > I’m trying to do the following: > > root@host:~# systemctl set-environment > XDG_RUNTIME_DIR="/run/user/503" Why would you do that? user@.service automatically pulls in user-runtime-dir@.service which is responsible for creating that dir with right perms. is 504 a system user? or a regular user? systemd generally assumes the boundary between system and regular users is between 999 and 1000. But user@.service is really just for regular users, not system users, hence my question. Lennart -- Lennart Poettering, Berlin
[systemd-devel] Manual start of user@.service failed with permission denied
Hi, I’m trying to do the following: root@host:~# systemctl set-environment XDG_RUNTIME_DIR="/run/user/503" root@host:~# systemctl start user@503.service<mailto:user@503.service> On systemd 253.4, the user can be started manually, and I can list the directory: root@host:~# ls -la /run/user/503 drwx--3 debugar debugar 80 Dec 4 09:08 . drwxr-xr-x3 root root60 Dec 4 09:08 .. srw-rw-rw-1 debugar debugar 0 Dec 4 09:08 bus drwxr-xr-x4 debugar debugar120 Dec 4 09:08 systemd On last systemd v254-stable and v255-stable, I got the following instead: root@host:/var/log# journalctl -xeu user@503.service<mailto:user@503.service> Dec 06 12:40:41 host systemd[1]: Starting User Manager for UID 503... Dec 06 12:40:41 host systemd[7939]: Failed to copy os-release for propagation, ignoring: Permission denied Dec 06 12:40:41 host systemd[7939]: Failed to allocate manager object: Permission denied Dec 06 12:40:41 host systemd[1]: user@503.service<mailto:user@503.service>: Main process exited, code=exited, status=1/FAILURE Dec 06 12:40:41 host systemd[1]: user@503.service<mailto:user@503.service>: Failed with result 'exit-code'. Dec 06 12:40:41 host systemd[1]: Failed to start User Manager for UID 503. systemd was able to create the dir /run/user/503, but got errno -13 (Permission denied) when it was trying to do mkdirat with path /run/user/503/systemd in function mkdirat_label() in file mkdir-label.c. Do I need to do something else before starting user@.service<mailto:user@%3cuid%3e.service> or is this a bug? Best regards, Christopher Wong
Re: [systemd-devel] systemctl logs to journal/kmsg intended?
Hi Lennart,
Thanks for your response! We will investigate it further why this is happening
in v253 for us.
Sorry, for the confidentiality notice. Used the wrong signature.
Best regards,
Christopher Wong
Från: Lennart Poettering
Skickat: tisdag 20 juni 2023 16:41
Till: Christopher Wong
Kopia: systemd-devel@lists.freedesktop.org
; Felix Nilsson
Ämne: Re: [systemd-devel] systemctl logs to journal/kmsg intended?
On Di, 20.06.23 07:53, Christopher Wong (christopher.w...@axis.com) wrote:
> Hi,
>
> Is it intended that systemctl should send log messages to the
> journal and kmsg with this patch? Because previously it was only
> writing to console.
Nope. log_setup() doesn't do this, unless $SYSTEMD_LOG_TARGET= is
configured to do so, or journald is told to.
> commit 158fdd5326a750fe01fcaa76ac4a56fe197cf1aa
> Author: Yu Watanabe
> watanabe.yu+git...@gmail.com<mailto:watanabe.yu+git...@gmail.com>
> Date: Thu Jul 7 16:38:54 2022 +0900
>
> systemctl: enable colorized logging by default
>
> diff --git a/src/systemctl/systemctl.c b/src/systemctl/systemctl.c
> index 003d2f0c25..fa32c67a08 100644
> --- a/src/systemctl/systemctl.c
> +++ b/src/systemctl/systemctl.c
> @@ -1113,8 +1113,7 @@ static int run(int argc, char *argv[]) {
> int r;
> setlocale(LC_ALL, "");
> -log_parse_environment();
> -log_open();
> +log_setup();
>
>
> *Email Confidentiality Notice* This communication and any
> attached files are confidential and are only intended for the use of
> the individual or entity to which they are directed. They may
> contain information that is privileged and exempt from disclosure
> under applicable law. If an addressing or transmission error has
> misdirected this email, please notify the sender and delete it from
> your system. If you are not the intended recipient, you may not use,
> disclose, distribute, copy, print, or rely on this email.
You are posting this to a public mailing list, you know this? I
presume you don't actually mean this confidentiality notice, do you?
Lennart
--
Lennart Poettering, Berlin
[systemd-devel] systemctl logs to journal/kmsg intended?
Hi,
Is it intended that systemctl should send log messages to the journal and kmsg
with this patch? Because previously it was only writing to console.
commit 158fdd5326a750fe01fcaa76ac4a56fe197cf1aa
Author: Yu Watanabe
watanabe.yu+git...@gmail.com<mailto:watanabe.yu+git...@gmail.com>
Date: Thu Jul 7 16:38:54 2022 +0900
systemctl: enable colorized logging by default
diff --git a/src/systemctl/systemctl.c b/src/systemctl/systemctl.c
index 003d2f0c25..fa32c67a08 100644
--- a/src/systemctl/systemctl.c
+++ b/src/systemctl/systemctl.c
@@ -1113,8 +1113,7 @@ static int run(int argc, char *argv[]) {
int r;
setlocale(LC_ALL, "");
-log_parse_environment();
-log_open();
+log_setup();
Best regards,
Christopher Wong
Senior Engineer
christopher.w...@axis.com<mailto:firstname.lastn...@axis.com>
Tel (direct): +46 46 272 11 30
Mobile: +46 76 144 26 00
Axis Communications AB
Gränden 1, SE-223 69 Lund, Sweden
Tel: +46 46 272 18 00, Fax: +46 46 13 61 30, www.axis.com<http://www.axis.com>
Please consider the environment before printing this e-mail
*Email Confidentiality Notice*
This communication and any attached files are confidential and are only
intended for the use of the individual or entity to which they are directed.
They may contain information that is privileged and exempt from disclosure
under applicable law. If an addressing or transmission error has misdirected
this email, please notify the sender and delete it from your system. If you are
not the intended recipient, you may not use, disclose, distribute, copy, print,
or rely on this email.
Re: [systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent
I had my focus on the child (main daemon process), therefore the term went backward parent and grand-parent. Sorry, if that caused confusion, but let's go with your definitions. Thank you for the explanation and pointing out the steps! Especially 1. makes it much clearer than the list described here https://www.freedesktop.org/software/systemd/man/daemon.html, which only mention that P' shall call exit(). chrony is probably not making sure of 1., were P should wait until P' has exited. Now that you mentioned sysv semantics is old I come across an option in chrony which will not fork. I shall probably try using that instead. Best Regards, Christopher Wong From: Lennart Poettering Sent: Monday, October 31, 2022 11:43:17 AM To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent On Mo, 31.10.22 11:40, Lennart Poettering (lenn...@poettering.net) wrote: > This is almost certainly a bug in chrony. If you use Type=forking, > then the process that systemd forks off (let's call it "P") should > wait until all of the below holds: > > 1. The middle child P' has exited > 2. The grandchild (and main daemon process) P'' is running > 3. The PID file has been successfully written to contain the PID of P''. BTW, let me add an explanation, *why* this is needed: if they leave P'' running for a bit longer, then there's a race: if for some reason the deamon ends up failing shortly after starting up there is a race if P' or P'' die first. If P'' dies first, then the service manager will never see its SIGCHLD and cannot determine there was a failure. If P' dies first then all is good, as the P'' SIGCHLD will be properly collected by the service manager. But anyway, it's 2022, chrony being stuck in sysv semantics is sad. Use sd_notify(). Lennart -- Lennart Poettering, Berlin
[systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent
Hi, We have during boot received the "Supervising process..." warning from systemd related to chronyd.service. This is not always happening, but when it happens systemd receives SIGCHLD from grand-parent (22955) before the parent (22956). See logs below: Oct 25 10:34:55.104980 axis-accc8ed1c728 systemd[22955]: chronyd.service: Executing: /usr/sbin/chronyd -u chronyd -f /run/chrony/chronyd.conf Oct 25 10:34:55.117999 axis-accc8ed1c728 chronyd[22957]: chronyd version 4.2 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP -SCFILTER -SIGND +ASYNCDNS +NTS -SECHASH +IPV6 +DEBUG) Oct 25 10:34:55.120781 axis-accc8ed1c728 chronyd[22957]: Frequency -8.172 +/- 1.366 ppm read from /var/lib/chrony/drift Oct 25 10:34:55.124304 axis-accc8ed1c728 systemd[1]: systemd-journald.service: Received EPOLLHUP on stored fd 82 (stored), closing. Oct 25 10:34:55.126460 axis-accc8ed1c728 systemd[1]: Received SIGCHLD from PID 22955 (chronyd). Oct 25 10:34:55.126708 axis-accc8ed1c728 systemd[1]: Child 22955 (chronyd) died (code=exited, status=0/SUCCESS) Oct 25 10:34:55.126920 axis-accc8ed1c728 systemd[1]: chronyd.service: Child 22955 belongs to chronyd.service. Oct 25 10:34:55.127000 axis-accc8ed1c728 systemd[1]: chronyd.service: Control process exited, code=exited, status=0/SUCCESS (success) Oct 25 10:34:55.127027 axis-accc8ed1c728 systemd[1]: chronyd.service: Got final SIGCHLD for state start. Oct 25 10:34:55.127160 axis-accc8ed1c728 systemd[1]: chronyd.service: Potentially unsafe symlink chain, will now retry with relaxed checks: /run/chrony/chronyd.pid Oct 25 10:34:55.127571 axis-accc8ed1c728 systemd[1]: chronyd.service: New main PID 22957 belongs to service, we are happy. Oct 25 10:34:55.127598 axis-accc8ed1c728 systemd[1]: chronyd.service: Main PID loaded: 22957 Oct 25 10:34:55.127759 axis-accc8ed1c728 systemd[1]: Custom log in process-util.c fnc pid_is_my_child(): pid: 22957, ppid: 22956, cached_pid: 1. Oct 25 10:34:55.127785 axis-accc8ed1c728 systemd[1]: chronyd.service: Supervising process 22957 which is not our child. We'll most likely not notice when it exits. Oct 25 10:34:55.127964 axis-accc8ed1c728 systemd[1]: chronyd.service: Changed start -> running Oct 25 10:34:55.128006 axis-accc8ed1c728 systemd[1]: chronyd.service: Job 117032 chronyd.service/start finished, result=done Oct 25 10:34:55.128053 axis-accc8ed1c728 systemd[1]: Started NTP client/server. ... Oct 25 10:34:55.158173 axis-accc8ed1c728 systemd[1]: Received SIGCHLD from PID 22956 (chronyd). Oct 25 10:34:55.158436 axis-accc8ed1c728 systemd[1]: Child 22956 (chronyd) died (code=exited, status=0/SUCCESS) Oct 25 10:34:55.158679 axis-accc8ed1c728 systemd[1]: chronyd.service: Child 22956 belongs to chronyd.service. The chronyd does two forks. In the normal case the parent will die first and then the grand-parent will die. This behavior is according to the SysV Daemons implementation https://www.freedesktop.org/software/systemd/man/daemon.html However, it seems scheduling for parent and grand-parent can vary and result in a different behavior. When systemd receives the SIGCHLD from the grand-parent (22955) it start to run service_load_pid_file() and set the new main PID, which also later triggers the warning "Supervising process...", but shouldn't systemd wait for the parent (22956) to the main PID to die before considering the service as started? Now chronyd is started before the parent (22956) has died, indicated by the log entry "Started NTP client/server." above. In https://www.freedesktop.org/software/systemd/man/systemd.service.html it is only mentioning that when the parent process exits then the unit is started. It doesn't mention grand-parent when type=forking and the service binary is doing double fork(). Testing with systemd 251.3. Best Regards, Christopher Wong
Re: [systemd-devel] PrivateNetwork=yes is memory costly
Yes, exactly. Best Regards, Christopher Wong From: Lennart Poettering Sent: Thursday, March 10, 2022 2:52:38 PM To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] PrivateNetwork=yes is memory costly On Do, 10.03.22 11:50, Christopher Wong (christopher.w...@axis.com) wrote: > Hi Lennart, > > > It is definitely a functionality we want to use. However, the memory came as > an unexpected side effect. Since we are not only enabling this for one single > service, instead we are applying it globally for all services. > > Now due to this huge memory consumption we are trying to put > everything into the same namespace using > JoinsNamespaceOf=. It seems to consume less memory. This means they will still be isolated from the network, but no longer from each other. Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] PrivateNetwork=yes is memory costly
Hi Lennart, It is definitely a functionality we want to use. However, the memory came as an unexpected side effect. Since we are not only enabling this for one single service, instead we are applying it globally for all services. Now due to this huge memory consumption we are trying to put everything into the same namespace using JoinsNamespaceOf=. It seems to consume less memory. Best Regards, Christopher Wong From: Lennart Poettering Sent: Wednesday, March 9, 2022 4:18:22 PM To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] PrivateNetwork=yes is memory costly On Mo, 07.03.22 15:10, Christopher Wong (christopher.w...@axis.com) wrote: > Hi, > > > It seems that PrivateNetwork=yes is a memory consuming > directive. The kernel seems to allocate quite an amount of memory > for each service (~50 kB) that has this directive enabled. I wonder > if this is expected and if anyone has had similar experience? PrivateNetwork=yes means that a private network namespace is allocated for the service. If you think network namespaces are too expensive in their current implementation, please bring this up with the kernel people, because they are a kernel concept after all, we just allocate them if told so. network namespaces are an effective way to disconnect a service from the network, if the service doesn't need it. It's probably one of the most relevant sandboxing options we offer, since disabling the attack surface called "network" for a service is of such major importance. That said, if you disable the network namespace functionality in the kernel systemd will handle this gracefully, and not use it. If the feature is available in the kernel we will however use it. > Is there any ways to reduce the usage? Besides turning it off? Nothing I was aware of. Lennart -- Lennart Poettering, Berlin
[systemd-devel] PrivateNetwork=yes is memory costly
Hi, It seems that PrivateNetwork=yes is a memory consuming directive. The kernel seems to allocate quite an amount of memory for each service (~50 kB) that has this directive enabled. I wonder if this is expected and if anyone has had similar experience? Is there any ways to reduce the usage? Best Regards Christopher Wong
Re: [systemd-devel] After= and Wants= doesn't seem to have an effect
I think the temperature-controller.service is stopped due to the
Before=temperature-controller.service in coco-wiper-manager.service.
There are no BindsTo=
The Conflicts=shutdown.target only.
I will do some more investigations after New Year.
Thank you for your time and explanations!
Merry X'mas and Happy New Year!
Christopher Wong
From: Anita Zhang
Sent: Thursday, December 23, 2021 9:21:48 AM
To: Christopher Wong
Cc: systemd-devel@lists.freedesktop.org
Subject: Re: [systemd-devel] After= and Wants= doesn't seem to have an effect
I think the telling part is "temperature-controller.service/start finished,
result=canceled". When another job for the same unit comes in (e.g.
"temperature-controller.service: Trying to enqueue job
temperature-controller.service/stop/replace") then the first job gets canceled
and replaced by the new one (this is the default behavior which I think can be
overridden). The bad part is systemd doesn't have logs to tell you why the stop
transaction was issued. Maybe you can use `systemctl show
temperature-controller.service` to see if there are BindsTo= or Conflicts=
dependencies that are not listed in the unit files and causing the stop job to
get issued.
As for your other point about why opticsd.service runs after
temperature-controller.service's stop job, I think this is by design. According
to
https://github.com/systemd/systemd/blob/4d484e14bb9864cef1d124885e625f33bf31e91c/src/core/job.c#L1659-L1698
if "b" starts after "a", but "a" has a stop job queued, then "a" will stop,
then "b" will start. Substitute opticsd for "b" and temperature-controller for
"a". The Wants= property issued a start job so what you're expecting is "a"
starts, then "b" starts. But due to the canceled job mentioned above, you're
instead left with "a" stops, then "b" starts.
Hope that helps,
Anita
On Wed, Dec 22, 2021 at 10:15 AM Christopher Wong
mailto:christopher.w...@axis.com>> wrote:
The code is being run on systemd 247.6. I finally got some debug logs to share
(see below). The starting of opticsd.service is held back, waiting for
temperature-controller.service, which is good. However, the line right after:
Job 721, which is a stop job for temperature-controller finished with
result=done. That seems to trigger the continuation of opticsd.service. I don't
have the logs for when Job 107 was started and why Job 721 is created. My guess
is that temperature-controller is waiting for coco-wiper-manager as indicated
in Job 722 and Job 809. Could this really be a bug and I am that lucky to
stumble into it?
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: Got message type=method_call
sender=n/a destination=org.freedesktop.systemd1 path=/org/freedesktop/systemd1
interface=org.freedesktop.systemd1.Manager member=StopUnit cookie=1
reply_cookie=0 signature=ss error-name=n/a error-message=n/a
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service:
Trying to enqueue job temperature-controller.service/stop/replace
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: Added job
temperature-controller.service/stop to transaction.
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service:
Job 107 temperature-controller.service/start finished, result=canceled
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: Sent message type=signal
sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1
interface=org.freedesktop.systemd1.Manager member=JobRemoved cookie=1
reply_cookie=0 signature=uoss error-name=n/a error-message=n/a
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service:
Installed new job temperature-controller.service/stop as 721
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service:
Enqueued job temperature-controller.service/stop as 721
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: Sent message type=signal
sender=org.freedesktop.systemd1 destination=n/a
path=/org/freedesktop/systemd1/unit/temperature_2dcontroller_2eservice
interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2
reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: Sent message type=signal
sender=org.freedesktop.systemd1 destination=n/a
path=/org/freedesktop/systemd1/unit/temperature_2dcontroller_2eservice
interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=3
reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: Sent message type=signal
sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1
interface=org.freedesktop.systemd1.Manager member=JobNew cookie=4
reply_cookie=0 signature=uos error-name=n/a error-message=n/a
Dec 22 18:03:17 axis-b8
Re: [systemd-devel] After= and Wants= doesn't seem to have an effect
d[1]: temperature-controller.service: Enqueued job temperature-controller.service/start as 722 Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service: starting held back, waiting for: coco-wiper-manager.service Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service: Trying to enqueue job temperature-controller.service/stop/replace Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: Added job temperature-controller.service/stop to transaction. Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service: Job 722 temperature-controller.service/start finished, result=canceled Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service: Installed new job temperature-controller.service/stop as 809 Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service: Enqueued job temperature-controller.service/stop as 809 Dec 22 18:03:17 axis-b8a44f278c56 systemd[1]: temperature-controller.service: Job 809 temperature-controller.service/stop finished, result=done Dec 22 18:03:18 axis-b8a44f278c56 systemd[1]: temperature-controller.service: Trying to enqueue job temperature-controller.service/start/replace Best Regards, Christopher Wong From: Andrei Borzenkov Sent: Tuesday, December 21, 2021 8:24:35 AM To: Christopher Wong; systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] After= and Wants= doesn't seem to have an effect On 20.12.2021 17:05, Christopher Wong wrote: >> # /etc/systemd/system/iris-detection.service >> [Unit] >> Description=Iris detection >> PartOf=opticsd.service > > > How can you tell that it is a loop? iris-detection.service doesn't have any > After= as you stated below. Yes, sorry, you are right. > Is it due to the PartOf=opticsd.service? > I do not think so. Still, loop is possible due to some other dependencies, it is also possible that actual unit definitions are different (e.g. units files have been changed after systemd read them). Running with debig log level may give some more hints. > > Best Regards, > > Christopher Wong > > > From: systemd-devel on behalf > of Andrei Borzenkov > Sent: Monday, December 20, 2021 1:27:42 PM > To: systemd-devel@lists.freedesktop.org > Subject: Re: [systemd-devel] After= and Wants= doesn't seem to have an effect > > On 20.12.2021 15:06, Christopher Wong wrote: >> # /etc/systemd/system/iris-detection.service >> After=temperature-controller.service >> >> # /usr/lib/systemd/system/temperature-controller.service >> After=iris-detection.service > > This is loop and systemd is free to break it by ignoring some dependency. >
Re: [systemd-devel] After= and Wants= doesn't seem to have an effect
> # /etc/systemd/system/iris-detection.service > [Unit] > Description=Iris detection > PartOf=opticsd.service How can you tell that it is a loop? iris-detection.service doesn't have any After= as you stated below. Is it due to the PartOf=opticsd.service? Best Regards, Christopher Wong From: systemd-devel on behalf of Andrei Borzenkov Sent: Monday, December 20, 2021 1:27:42 PM To: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] After= and Wants= doesn't seem to have an effect On 20.12.2021 15:06, Christopher Wong wrote: > # /etc/systemd/system/iris-detection.service > After=temperature-controller.service > > # /usr/lib/systemd/system/temperature-controller.service > After=iris-detection.service This is loop and systemd is free to break it by ignoring some dependency.
[systemd-devel] After= and Wants= doesn't seem to have an effect
Hi, I got a case were After= and Wants= doesn't seem to have an effect for the opticsd.service. ./info.log.1:2021-12-17T21:30:02.684+00:00 [ INFO] systemd[1]: Starting Iris detection... ./info.log.1:2021-12-17T21:30:02.884+00:00 [ INFO] systemd[1]: Starting Wiper power manager... ./info.log.1:2021-12-17T21:30:02.886+00:00 [ INFO] systemd[1]: Starting Optics daemon... ./info.log.1:2021-12-17T21:30:02.888+00:00 [ INFO] systemd[1]: Started Optics daemon. ./info.log.1:2021-12-17T21:30:02.890+00:00 [ INFO] systemd[1]: Started Wiper power manager. ./info.log.1:2021-12-17T21:30:02.890+00:00 [ INFO] systemd[1]: Starting Temperature controller... ./info.log.1:2021-12-17T21:30:02.895+00:00 [ INFO] systemd[1]: Started Temperature controller. # /usr/lib/systemd/system/coco-wiper-manager.service [Unit] Description=Wiper power manager Before=temperature-controller.service wiper-service.service ... [Install] WantedBy=temperature-controller.service wiper-service.service # /etc/systemd/system/opticsd.service [Unit] Description=Optics daemon ... After=iris-detection.service Wants=iris-detection.service After=temperature-controller.service Wants=temperature-controller.service ... [Install] WantedBy=multi-user.target # /etc/systemd/system/iris-detection.service [Unit] Description=Iris detection PartOf=opticsd.service # /usr/lib/systemd/system/wiper-service.service [Unit] Description=Wiper Service Wants=temperature-controller.service After=temperature-controller.service # /usr/lib/systemd/system/temperature-controller.service [Unit] Description=Temperature controller ... After=iris-detection.service Wants=iris-detection.service Is it possible that After= is ignored, even if Wants= is specified? Best Regards, Christopher Wong
Re: [systemd-devel] systemctl service-watchdogs yes does not resume
Some updates: I found the interface for disable/enable at the interface org.freedesktop.systemd1.Manager. I did get another service with WatchdogSec= to print a log every 10s when the service watchdogs was disabled. And after enabling the service watchdogs the SIGABRT was generated when the watchdog fires. So it works as it I expected. However, I don't understand why I only got one watchdog ignored log print when I just added WatchdogSec= in my test service file. I tried to compare with the one that was working, but the only thing I think matter is the Type=notify, while my test service didn't have the Type= defined. See my test service file below:? [Unit] Description=MyTest [Service] ExecStart=/usr/local/bin/mytest Restart=always RestartSec=5s CPUQuota=10% WatchdogSec=5s [Install] WantedBy=multi-user.target ? Best Regards, Christopher Wong From: systemd-devel on behalf of Christopher Wong Sent: Tuesday, May 11, 2021 23:07 To: systemd-devel@lists.freedesktop.org Subject: [systemd-devel] systemctl service-watchdogs yes does not resume Hi, I have put WatchdogSec= in a service file and I can see that the watchdog is firing and getting the SIGABRT. I then run "systemctl service-watchdogs no" to disable the watchdogs in the system. That works, since I am getting the a log print when my service watchdog fires instead of SIGABRT. Then I would like to enable the watchdogs again by using "systemctl service-watchdogs yes". However, my service no longer has a running watchdog unless I restart it. I would expect that all previously fired watchdogs during the disabled period will be resumed, but maybe it works in another way or am I missing something. How is "systemctl service-watchdogs [yes|no]" supposed to work? Is there another interface to access disable/enable all watchdogs, which is more suitable for a c-program? The reason for above is an application that is occupying the CPU for a longer time, which results in many services configured with watchdogs being fired. Would like to disable the watchdogs during that period and resume the watchdogs one the heavy operation is finished. Best Regards, Christopher Wong ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] systemctl service-watchdogs yes does not resume
Hi, I have put WatchdogSec= in a service file and I can see that the watchdog is firing and getting the SIGABRT. I then run "systemctl service-watchdogs no" to disable the watchdogs in the system. That works, since I am getting the a log print when my service watchdog fires instead of SIGABRT. Then I would like to enable the watchdogs again by using "systemctl service-watchdogs yes". However, my service no longer has a running watchdog unless I restart it. I would expect that all previously fired watchdogs during the disabled period will be resumed, but maybe it works in another way or am I missing something. How is "systemctl service-watchdogs [yes|no]" supposed to work? Is there another interface to access disable/enable all watchdogs, which is more suitable for a c-program? The reason for above is an application that is occupying the CPU for a longer time, which results in many services configured with watchdogs being fired. Would like to disable the watchdogs during that period and resume the watchdogs one the heavy operation is finished. Best Regards, Christopher Wong ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Sandboxing options
Hi Lennart, Thanks for your reply! After some struggles I managed to figure out that I was missing the SECCOMP in systemd 244 that I was running. Once I have enabled SECCOMP and managed to build systemd with it then all the below options except for UMask was available for me. I will leave UMask for now, no need to use it at this moment. Best regards, Christopher Wong From: Lennart Poettering Sent: Saturday, December 19, 2020 11:28 To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Sandboxing options On Mo, 28.09.20 17:00, Christopher Wong (christopher.w...@axis.com) wrote: > Hi, > > > There are a bunch of sandboxing options that I am trying to enable > but I got no effects when I am setting them. Below are the options > that I am trying to set, but I can't seem to turn them on. > > LockPersonality=true > MemoryDenyWriteExecute=true > RestrictRealtime=true > RestrictSUIDSGID=true > RestrictNamespaces= > SystemCallArchitectures=native > #SystemCallArchitectures=option > UMask= > #UMask=0033 > > I have enabled the following kernel configurations: > > CONFIG_NAMESPACES=y > CONFIG_NET_NS=y > CONFIG_USER_NS=y > CONFIG_SECCOMP=y > > Is there anything that I am missing? Maybe start with saying which distro you are using, which kernel, which systemd version. Give an example of the unit file you are using. Are you using this in --user or --system mode? (Note that a bunch of sandboxing settings are only available for --system). Have you checked the logs? In particular after enabling debug logging (systemd-analyze log-level debug). Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Sandboxing options
Hi, There are a bunch of sandboxing options that I am trying to enable but I got no effects when I am setting them. Below are the options that I am trying to set, but I can't seem to turn them on. LockPersonality=true MemoryDenyWriteExecute=true RestrictRealtime=true RestrictSUIDSGID=true RestrictNamespaces= SystemCallArchitectures=native #SystemCallArchitectures=option UMask= #UMask=0033 I have enabled the following kernel configurations: CONFIG_NAMESPACES=y CONFIG_NET_NS=y CONFIG_USER_NS=y CONFIG_SECCOMP=y Is there anything that I am missing? Best Regards, Christopher Wong ? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Enable sandboxing options globally for all services
Thanks, seems to be working fine. Best Regards, Christopher Wong From: Michael Biebl Sent: Wednesday, September 9, 2020 22:40 To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Enable sandboxing options globally for all services Am Mi., 9. Sept. 2020 um 21:40 Uhr schrieb Christopher Wong : > > Hi, > > > Is there a way to turn on a sandboxing option for all services? Recent versions of systemd allow to use global drop-in config snippets. See the changelog of v244 * Unit files now support top level dropin directories of the form .d/ (e.g. service.d/) that may be used to add configuration that affects all corresponding unit files. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Enable sandboxing options globally for all services
Hi, Is there a way to turn on a sandboxing option for all services? We would like to on an embedded system enable as much sandboxing options as default. Only whitelist options for services that need access to them. Best Regards, Christopher Wong ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] systemd-timedated: Not possible to set time zone that is a symlink!
Hi, Sorry, for this late reply, vacation and other priorities. > > I am asking because our system mount /usr/share/zoneinfo as > > read-only and because of legacy we need to support the user being > > able to change the TZ string in a tz-file. Installing a symlink that > > point to such a tz-file will allow us to use the systemd-timedated > > interface to set time zone. The changeable tz-file (located at > > /etc/...) can be altered by root and a specific service. Do you see > > any potential risk by doing so? > > consider turning off the sandboxing features, i.e. add a drop-in that > turns off ProtectSystem=, ProtectHome= and suchlike. Just wanted to give feedback that ProtectSystem= and ProtectHome= was turned off already, but we could no longer produce the problem we have seen. So case closed. BR, Christopher Wong ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] systemd-timedated: Not possible to set time zone that is a symlink!
Hi, The systemd-timedated doesn't allow setting a tz-file under /usr/share/zoneinfo to be a symlink. Is it due to security reasons? I am asking because our system mount /usr/share/zoneinfo as read-only and because of legacy we need to support the user being able to change the TZ string in a tz-file. Installing a symlink that point to such a tz-file will allow us to use the systemd-timedated interface to set time zone. The changeable tz-file (located at /etc/...) can be altered by root and a specific service. Do you see any potential risk by doing so? Best Regards, Christopher Wong ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Why don't systemd-timedated list time zones from zone1970.tab?
Hi Lennart, I have created a PR now. https://github.com/systemd/systemd/pull/11856 BR, Christopher Wong From: Lennart Poettering Sent: Monday, February 25, 2019 14:08 To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Why don't systemd-timedated list time zones from zone1970.tab? On Fr, 22.02.19 12:21, Christopher Wong (christopher.w...@axis.com) wrote: > Hi, > > > The systemd-timedated list available time zones from the file > /usr/share/zoneinfo/zone.tab provided by the Time Zone > Database. According to the comments in zone.tab its content is for > backward-compatibility aid for older programs. New programs should > use zone1970.tab.? > > ?Wouldn't it be better to use the zone1970.tab file? Looks like it indeed. Could you file a bug on github about this, so that we keep track of this? Even better: file a bugfix PR! Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Why don't systemd-timedated list time zones from zone1970.tab?
Hi, The systemd-timedated list available time zones from the file /usr/share/zoneinfo/zone.tab provided by the Time Zone Database. According to the comments in zone.tab its content is for backward-compatibility aid for older programs. New programs should use zone1970.tab.? ?Wouldn't it be better to use the zone1970.tab file? Best Regards, Christopher Wong ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] systemd-timedated doesn't return available time zones
?Hi guys, We plan to place an abstraction service over systemd-timedated and we would like to retrieve available time zones. The timedatectl retrieves the list of time zones by itself, but why isn't this functionality available in systemd-timedated? Best Regards, Christopher Wong ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
