Public bug reported: systemd-sysusers.service/systemd.exec fails to start in privileged containers, due to being unable to properly mount /dev for passing credentials, caused by the following config in the .service unit: ``` # Optionally, pick up a root password and shell for the root user from a # credential passed to the service manager. This is useful for importing this # data from nspawn's --set-credential= switch. LoadCredential=passwd.hashed-password.root LoadCredential=passwd.plaintext-password.root LoadCredential=passwd.shell.root ```
Reproducer: $ lxc profile set default security.privileged "true" $ lxc launch ubuntu-daily:jammy test $ lxc exec test bash # add-apt-repository ppa:ci-train-ppa-service/4704 # apt install systemd # install systemd 249.5-2ubuntu1 # systemctl restart systemd-sysusers # systemctl status systemd-sysusers # system --status=failed $ lxc profile set default security.privileged "false" A workaround is to disable it via: $ cat /etc/systemd/system/systemd-sysusers.service.d/override.conf: [Service] LoadCredential= Interesting logs: Nov 12 12:09:44 test systemd[1]: systemd-journald.service: Added fd 42 (n/a) to fd store. Nov 12 12:09:44 test systemd[431]: Mounting /dev (MS_REC|MS_SLAVE "")... Nov 12 12:09:44 test systemd[431]: Failed to mount n/a (type n/a) on /dev (MS_REC|MS_SLAVE ""): Permission denied Nov 12 12:09:44 test systemd[430]: (sd-mkdcreds) failed with exit status 1. Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed to set up credentials: Protocol error Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed at step CREDENTIALS spawning ** Affects: lxd (Ubuntu) Importance: Undecided Status: New ** Affects: systemd (Ubuntu) Importance: Undecided Status: New ** Also affects: systemd (Ubuntu) Importance: Undecided Status: New ** Description changed: systemd-sysusers.service/systemd.exec fails to start in privileged containers, due to being unable to properly mount /dev for passing credentials, caused by the following config in the .service unit: # Optionally, pick up a root password and shell for the root user from a # credential passed to the service manager. This is useful for importing this # data from nspawn's --set-credential= switch. LoadCredential=passwd.hashed-password.root LoadCredential=passwd.plaintext-password.root LoadCredential=passwd.shell.root Reproducer: $ lxc profile set default security.privileged "true" $ lxc launch ubuntu-daily:jammy test $ lxc exec test bash # add-apt-repository ppa:ci-train-ppa-service/4704 # apt install systemd # install systemd 249.5-2ubuntu1 # systemctl restart systemd-sysusers # systemctl status systemd-sysusers # system --status=failed $ lxc profile set default security.privileged "false" A workaround is to disable it via: $ cat /etc/systemd/system/systemd-sysusers.service.d/override.conf: [Service] LoadCredential= Interesting logs: Nov 12 12:09:44 test systemd[1]: systemd-journald.service: Added fd 42 (n/a) to fd store. Nov 12 12:09:44 test systemd[431]: Mounting /dev (MS_REC|MS_SLAVE "")... Nov 12 12:09:44 test systemd[431]: Failed to mount n/a (type n/a) on /dev (MS_REC|MS_SLAVE ""): Permission denied Nov 12 12:09:44 test systemd[430]: (sd-mkdcreds) failed with exit status 1. Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed to set up credentials: Protocol error Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed at step CREDENTIALS spawning - - Debug logs: - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Job 350 systemd-sysusers.service/restart finished, result=done - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Converting job systemd-sysusers.service/restart -> systemd-sysusers.service/start - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: ConditionNeedsUpdate=/etc succeeded. - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Passing 0 fds to service - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: About to execute systemd-sysusers - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Forked systemd-sysusers as 430 - Nov 12 12:09:44 test systemd[430]: Successfully forked off '(sd-mkdcreds)' as PID 431. - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=7 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=8 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2893 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2894 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Changed failed -> start - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=9 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2895 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Starting Create System Users... - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=10 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=11 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2896 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2dsysusers_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2897 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1/job/350 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=12 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/job/350 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2898 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: systemd-journald.service: Got notification message from PID 59 (FDSTORE=1) - Nov 12 12:09:44 test systemd[1]: systemd-journald.service: Added fd 42 (n/a) to fd store. - Nov 12 12:09:44 test systemd[431]: Mounting /dev (MS_REC|MS_SLAVE "")... - Nov 12 12:09:44 test systemd[431]: Failed to mount n/a (type n/a) on /dev (MS_REC|MS_SLAVE ""): Permission denied - Nov 12 12:09:44 test systemd[430]: (sd-mkdcreds) failed with exit status 1. - Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed to set up credentials: Protocol error - Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed at step CREDENTIALS spawning systemd-sysusers: Protocol error - Nov 12 12:09:44 test systemd[1]: Received SIGCHLD from PID 430 ((sysusers)). - Nov 12 12:09:44 test systemd[1]: Child 430 ((sysusers)) died (code=exited, status=243/CREDENTIALS) - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Child 430 belongs to systemd-sysusers.service. - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Main process exited, code=exited, status=243/CREDENTIALS - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Failed with result 'exit-code'. - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Service will not restart (restart setting) - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Changed start -> failed - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=13 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=2899 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a - Nov 12 12:09:44 test systemd[1]: systemd-sysusers.service: Job 350 systemd-sysusers.service/start finished, result=failed - Nov 12 12:09:44 test systemd[1]: Failed to start Create System Users. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1950787 Title: systemd-sysusers cannot mount /dev in privileged containers (to pass credentials) Status in lxd package in Ubuntu: New Status in systemd package in Ubuntu: New Bug description: systemd-sysusers.service/systemd.exec fails to start in privileged containers, due to being unable to properly mount /dev for passing credentials, caused by the following config in the .service unit: ``` # Optionally, pick up a root password and shell for the root user from a # credential passed to the service manager. This is useful for importing this # data from nspawn's --set-credential= switch. LoadCredential=passwd.hashed-password.root LoadCredential=passwd.plaintext-password.root LoadCredential=passwd.shell.root ``` Reproducer: $ lxc profile set default security.privileged "true" $ lxc launch ubuntu-daily:jammy test $ lxc exec test bash # add-apt-repository ppa:ci-train-ppa-service/4704 # apt install systemd # install systemd 249.5-2ubuntu1 # systemctl restart systemd-sysusers # systemctl status systemd-sysusers # system --status=failed $ lxc profile set default security.privileged "false" A workaround is to disable it via: $ cat /etc/systemd/system/systemd-sysusers.service.d/override.conf: [Service] LoadCredential= Interesting logs: Nov 12 12:09:44 test systemd[1]: systemd-journald.service: Added fd 42 (n/a) to fd store. Nov 12 12:09:44 test systemd[431]: Mounting /dev (MS_REC|MS_SLAVE "")... Nov 12 12:09:44 test systemd[431]: Failed to mount n/a (type n/a) on /dev (MS_REC|MS_SLAVE ""): Permission denied Nov 12 12:09:44 test systemd[430]: (sd-mkdcreds) failed with exit status 1. Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed to set up credentials: Protocol error Nov 12 12:09:44 test systemd[430]: systemd-sysusers.service: Failed at step CREDENTIALS spawning To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/lxd/+bug/1950787/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp