** Description changed: [ Impact ] When using cloud-init to add a local user with a specific password/hash to the system at first boot, and sssd-tools also happens to be installed but sssd itself is not configured yet, a sequence of events leads to that user having no password set: - the user is created with the specified password hash just fine - cloud-init calls[1] "passwd -u" on the user - that passwd call ends up also calling "sss_cache -UG"[2], to reset the sssd cache, since the src:shadow package (where bin:passwd comes from) is built with sssd support[3] by default. If sss_cache is not available (bin:sssd-tools not installed), the bug does not manifest itself. - calling "sss_cache -UG" on a system with no valid sssd configuration yet prints an error message to stderr about a missing config.ldb, but exits with 0. This is since commits [4][5]. Those commits also changed sss_cache so that it no longer generates the config.ldb database, hence why it now always complains. - cloud-init checks exit status, and stderr, of the "passwd -u" call. It sees that stderr has content[6], and assumes a failure, proceeding to calling "passwd -d" on that account. This is what deletes the password hash from the user. - For this SRU, I opted to silence the error from the caller's side - (src:shadow) if there is no config.ldb database present. That call is a - best effort attempt to be helpful and clear the sssd cache, and the - result of the call is being ignored anyway. + For this SRU, I had originally opted to silence the error from the + caller's side (src:shadow) if there is no config.ldb database present. + That call is a best effort attempt to be helpful and clear the sssd + cache, and the result of the call is being ignored anyway. - What's not so pretty about this patch is the hardcoded path to the - config.ldb database, but we are contained in ubuntu, the fix is for - ubuntu, and that's the place, so it shouldn't be a problem for the SRU, - but is likely not acceptable for upstream. + But that has a much wider impact: src:shadow is installed everywhere. + Even users not affected by this bug would get the update. Fedora (which is the sssd upstream) opted to entirely disable[7] sssd support in src:shadow. That is an option to consider for devel releases of ubuntu, but not an SRU. - Another option could be to silence the error from within sss_cache, but - it's reasonable to expect a user would like to know about the missing - configuration, even though the exit status is zero. + I then decided to silence the error from within sss_cache instead. It's + the same fix in principle, and only users affected by the bug will + receive the update. - However, it does sound reasonable to add the fix to sss_cache itself. - It's more self contained, blast radius is smaller, only users of - bin:sssd-tools would be affected. The fix would be similar to what was - done to passwd, and that would be to silence the error message if - config.ldb is not present. This also avoids the problem of the hardcoded - path, since sssd knows there that config.ldb will always be in any - particular build. - - By adding the fix to bin:passwd instead (via src:shadow), all ubuntu - users will get this download, even those not using sssd. + The one small change in behavior is that now when sss_cache doesn't find + the config.ldb database, it's silent, so users might not be immediately + aware that no action was taken by the tool. Note, however, that the exit + code remains unchanged. And if you think about the end result of the + tool run, it's the same: the cache was either expunged, or there was + nothing to expunge. 1. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1059 2. https://git.launchpad.net/ubuntu/+source/shadow/tree/src/passwd.c?h=applied/ubuntu/devel#n1175 3. https://git.launchpad.net/ubuntu/+source/shadow/tree/configure.ac?h=applied/ubuntu/devel#n202 4. https://github.com/SSSD/sssd/commit/a008accecd6d0b35e8d57d738ee3d05863aa7d0f 5. https://github.com/SSSD/sssd/commit/50b4579413c1146a2fced5ced19977e13790731c 6. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1063 7. https://src.fedoraproject.org/rpms/shadow-utils/c/1820858d26f633482d3679d1c97324e25e3c1df8?branch=rawhide [ Test Plan ] There is a script attached to this bug which will reproduce the bug. It assumes a working lxd setup, and that "lxc launch ubuntu-daily:<ubuntu- release-name>" produces a working container. The script will exercise the following aspects of this bug: - provide the necessary cloud-init user-data to trigger the bug - check that the user has a password set or not in the end - check that "passwd -u" produces or not the stderr output that triggers the bug - check that "sss_cache -UG" *always* produces the stderr output, i.e., didn't change behavior. This also validates that we will not get false results by seeing no stderr output from "passwd -u" and assume everything is fine just because of that. To run the test, download the script to some directory and make it executable: wget https://bugs.launchpad.net/ubuntu/+source/shadow/+bug/2153951/+attachment/6009255/+files/lp2153951-test.sh chmod +x lp2153951-test.sh Then run it targeting the ubuntu release under test. For example, for resolute: ./lp2153951-test.sh --release resolute That should reproduce the error, i.e., the script must indicate the test as having failed. To test the fixed packages, pass the "--pocket proposed" parameter. Again with the resolute example: ./lp2153951-test.sh --release resolute --pocket proposed This time the script should indicate that all checks have passed, and that completes the verification for this SRU. Here is run that shows the bug being reproduced: $ ./lp2153951-test.sh --release resolute Launching resolute container lp2153951-1955635 (phase 1: repositories + packages) Launching lp2153951-1955635 ...........................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: Installed: 1:4.17.4-2ubuntu3 Candidate: 1:4.17.4-2ubuntu3 Version table: *** 1:4.17.4-2ubuntu3 500 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages 100 /var/lib/dpkg/status === Checks === FAIL: tuser has no hashed password in /etc/shadow (got: '<empty>') FAIL: 'passwd -u tuser' complained about the SSSD config db on stderr: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr 'sss_cache -UG' stderr was: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? RESULT: FAIL Destroying container lp2153951-1955635 And here is a run showing the tests passing (I used a PPA for this example, but the real SRU verification must use "--pocket proposed"): $ ./lp2153951-test.sh --release resolute --pocket ppa:ahasenack/shadow-sssd-cache Launching resolute container lp2153951-1959790 (phase 1: repositories + packages) Launching lp2153951-1959790 ......................................................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: Installed: 1:4.17.4-2ubuntu3.1~ppa2 Candidate: 1:4.17.4-2ubuntu3.1~ppa2 Version table: *** 1:4.17.4-2ubuntu3.1~ppa2 500 500 https://ppa.launchpadcontent.net/ahasenack/shadow-sssd-cache/ubuntu resolute/main amd64 Packages 100 /var/lib/dpkg/status 1:4.17.4-2ubuntu3 500 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages === Checks === PASS: tuser has a hashed password in /etc/shadow (got: '$6$lp2153951$Iq7BfvEAR8qyz9eKAKs5sAGBBtyDoSRW0D2FnBJdG3Xy5.rLbSh29Hrds0VXSNzIx/1PNNmATPReqLjginX7/.') PASS: 'passwd -u tuser' produced no SSSD config-db complaint on stderr PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr 'sss_cache -UG' stderr was: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? RESULT: PASS Destroying container lp2153951-1959790 [ Where problems could occur ] An update to src:shadow affects all ubuntu users. The patch is contained to the code that flushes the sssd cache, and only runs if the sss_cache binary is installed. The cache flush, however, is not confined to the passwd command only. It is used throughout the code, and also called from groupadd and others (again, only if sss_cache is present). This is also the reason why currently you see the same config.ldb errors messages when installing sssd-tools fresh: ... Setting up sssd-common (2.12.0-1ubuntu5.5) ... Creating SSSD system user & group... [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? ... - [ Other Info ] See "Original Description" below for the original scenario where this bug appeared. - [ Original Description ] Installed ubuntu-desktop in 26.04 server, and it didn't generate config.ldb file which kept deleting the password configured by cloud- init user data. ubuntu@ip-10-0-1-145:~$ sudo passwd -u ubuntu 2>&1 [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? passwd: password changed. ubuntu@ip-10-0-1-145:~$ echo "exit: $?" exit: 0 there are output even though exit code 0 which triggers passwd -d in cloud-init script
** Description changed: [ Impact ] When using cloud-init to add a local user with a specific password/hash to the system at first boot, and sssd-tools also happens to be installed but sssd itself is not configured yet, a sequence of events leads to that user having no password set: - the user is created with the specified password hash just fine - cloud-init calls[1] "passwd -u" on the user - that passwd call ends up also calling "sss_cache -UG"[2], to reset the sssd cache, since the src:shadow package (where bin:passwd comes from) is built with sssd support[3] by default. If sss_cache is not available (bin:sssd-tools not installed), the bug does not manifest itself. - calling "sss_cache -UG" on a system with no valid sssd configuration yet prints an error message to stderr about a missing config.ldb, but exits with 0. This is since commits [4][5]. Those commits also changed sss_cache so that it no longer generates the config.ldb database, hence why it now always complains. - cloud-init checks exit status, and stderr, of the "passwd -u" call. It sees that stderr has content[6], and assumes a failure, proceeding to calling "passwd -d" on that account. This is what deletes the password hash from the user. For this SRU, I had originally opted to silence the error from the caller's side (src:shadow) if there is no config.ldb database present. That call is a best effort attempt to be helpful and clear the sssd cache, and the result of the call is being ignored anyway. But that has a much wider impact: src:shadow is installed everywhere. Even users not affected by this bug would get the update. Fedora (which is the sssd upstream) opted to entirely disable[7] sssd support in src:shadow. That is an option to consider for devel releases of ubuntu, but not an SRU. I then decided to silence the error from within sss_cache instead. It's the same fix in principle, and only users affected by the bug will receive the update. The one small change in behavior is that now when sss_cache doesn't find the config.ldb database, it's silent, so users might not be immediately aware that no action was taken by the tool. Note, however, that the exit code remains unchanged. And if you think about the end result of the tool run, it's the same: the cache was either expunged, or there was nothing to expunge. 1. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1059 2. https://git.launchpad.net/ubuntu/+source/shadow/tree/src/passwd.c?h=applied/ubuntu/devel#n1175 3. https://git.launchpad.net/ubuntu/+source/shadow/tree/configure.ac?h=applied/ubuntu/devel#n202 4. https://github.com/SSSD/sssd/commit/a008accecd6d0b35e8d57d738ee3d05863aa7d0f 5. https://github.com/SSSD/sssd/commit/50b4579413c1146a2fced5ced19977e13790731c 6. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1063 7. https://src.fedoraproject.org/rpms/shadow-utils/c/1820858d26f633482d3679d1c97324e25e3c1df8?branch=rawhide [ Test Plan ] - There is a script attached to this bug which will reproduce the bug. It - assumes a working lxd setup, and that "lxc launch ubuntu-daily:<ubuntu- - release-name>" produces a working container. + There is a script attached to this bug which will reproduce the issue. + It assumes a working lxd setup, and that "lxc launch ubuntu- + daily:<ubuntu-release-name>" produces a working container. The script will exercise the following aspects of this bug: - provide the necessary cloud-init user-data to trigger the bug - check that the user has a password set or not in the end - check that "passwd -u" produces or not the stderr output that triggers the bug - - check that "sss_cache -UG" *always* produces the stderr output, i.e., didn't change behavior. This also validates that we will not get false results by seeing no stderr output from "passwd -u" and assume everything is fine just because of that. + - check that "sss_cache -UG" is *always* silent, i.e., no stderr noise about the missing config.ldb To run the test, download the script to some directory and make it executable: wget https://bugs.launchpad.net/ubuntu/+source/shadow/+bug/2153951/+attachment/6009255/+files/lp2153951-test.sh chmod +x lp2153951-test.sh Then run it targeting the ubuntu release under test. For example, for resolute: ./lp2153951-test.sh --release resolute That should reproduce the error, i.e., the script must indicate the test as having failed. To test the fixed packages, pass the "--pocket proposed" parameter. Again with the resolute example: ./lp2153951-test.sh --release resolute --pocket proposed This time the script should indicate that all checks have passed, and that completes the verification for this SRU. Here is run that shows the bug being reproduced: $ ./lp2153951-test.sh --release resolute Launching resolute container lp2153951-1955635 (phase 1: repositories + packages) Launching lp2153951-1955635 ...........................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: Installed: 1:4.17.4-2ubuntu3 Candidate: 1:4.17.4-2ubuntu3 Version table: *** 1:4.17.4-2ubuntu3 500 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages 100 /var/lib/dpkg/status === Checks === FAIL: tuser has no hashed password in /etc/shadow (got: '<empty>') FAIL: 'passwd -u tuser' complained about the SSSD config db on stderr: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr 'sss_cache -UG' stderr was: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? RESULT: FAIL Destroying container lp2153951-1955635 And here is a run showing the tests passing (I used a PPA for this example, but the real SRU verification must use "--pocket proposed"): $ ./lp2153951-test.sh --release resolute --pocket ppa:ahasenack/shadow-sssd-cache Launching resolute container lp2153951-1959790 (phase 1: repositories + packages) Launching lp2153951-1959790 ......................................................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: Installed: 1:4.17.4-2ubuntu3.1~ppa2 Candidate: 1:4.17.4-2ubuntu3.1~ppa2 Version table: *** 1:4.17.4-2ubuntu3.1~ppa2 500 500 https://ppa.launchpadcontent.net/ahasenack/shadow-sssd-cache/ubuntu resolute/main amd64 Packages 100 /var/lib/dpkg/status 1:4.17.4-2ubuntu3 500 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages === Checks === PASS: tuser has a hashed password in /etc/shadow (got: '$6$lp2153951$Iq7BfvEAR8qyz9eKAKs5sAGBBtyDoSRW0D2FnBJdG3Xy5.rLbSh29Hrds0VXSNzIx/1PNNmATPReqLjginX7/.') PASS: 'passwd -u tuser' produced no SSSD config-db complaint on stderr PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr 'sss_cache -UG' stderr was: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? RESULT: PASS Destroying container lp2153951-1959790 [ Where problems could occur ] An update to src:shadow affects all ubuntu users. The patch is contained to the code that flushes the sssd cache, and only runs if the sss_cache binary is installed. The cache flush, however, is not confined to the passwd command only. It is used throughout the code, and also called from groupadd and others (again, only if sss_cache is present). This is also the reason why currently you see the same config.ldb errors messages when installing sssd-tools fresh: ... Setting up sssd-common (2.12.0-1ubuntu5.5) ... Creating SSSD system user & group... [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? ... [ Other Info ] See "Original Description" below for the original scenario where this bug appeared. [ Original Description ] Installed ubuntu-desktop in 26.04 server, and it didn't generate config.ldb file which kept deleting the password configured by cloud- init user data. ubuntu@ip-10-0-1-145:~$ sudo passwd -u ubuntu 2>&1 [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? passwd: password changed. ubuntu@ip-10-0-1-145:~$ echo "exit: $?" exit: 0 there are output even though exit code 0 which triggers passwd -d in cloud-init script ** Attachment removed: "lp2153951-test.sh" https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/2153951/+attachment/6009255/+files/lp2153951-test.sh ** Attachment added: "lp2153951-test.sh" https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/2153951/+attachment/6011062/+files/lp2153951-test.sh ** Description changed: [ Impact ] When using cloud-init to add a local user with a specific password/hash to the system at first boot, and sssd-tools also happens to be installed but sssd itself is not configured yet, a sequence of events leads to that user having no password set: - the user is created with the specified password hash just fine - cloud-init calls[1] "passwd -u" on the user - that passwd call ends up also calling "sss_cache -UG"[2], to reset the sssd cache, since the src:shadow package (where bin:passwd comes from) is built with sssd support[3] by default. If sss_cache is not available (bin:sssd-tools not installed), the bug does not manifest itself. - calling "sss_cache -UG" on a system with no valid sssd configuration yet prints an error message to stderr about a missing config.ldb, but exits with 0. This is since commits [4][5]. Those commits also changed sss_cache so that it no longer generates the config.ldb database, hence why it now always complains. - cloud-init checks exit status, and stderr, of the "passwd -u" call. It sees that stderr has content[6], and assumes a failure, proceeding to calling "passwd -d" on that account. This is what deletes the password hash from the user. For this SRU, I had originally opted to silence the error from the caller's side (src:shadow) if there is no config.ldb database present. That call is a best effort attempt to be helpful and clear the sssd cache, and the result of the call is being ignored anyway. But that has a much wider impact: src:shadow is installed everywhere. Even users not affected by this bug would get the update. Fedora (which is the sssd upstream) opted to entirely disable[7] sssd support in src:shadow. That is an option to consider for devel releases of ubuntu, but not an SRU. I then decided to silence the error from within sss_cache instead. It's the same fix in principle, and only users affected by the bug will receive the update. The one small change in behavior is that now when sss_cache doesn't find the config.ldb database, it's silent, so users might not be immediately aware that no action was taken by the tool. Note, however, that the exit code remains unchanged. And if you think about the end result of the tool run, it's the same: the cache was either expunged, or there was nothing to expunge. 1. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1059 2. https://git.launchpad.net/ubuntu/+source/shadow/tree/src/passwd.c?h=applied/ubuntu/devel#n1175 3. https://git.launchpad.net/ubuntu/+source/shadow/tree/configure.ac?h=applied/ubuntu/devel#n202 4. https://github.com/SSSD/sssd/commit/a008accecd6d0b35e8d57d738ee3d05863aa7d0f 5. https://github.com/SSSD/sssd/commit/50b4579413c1146a2fced5ced19977e13790731c 6. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1063 7. https://src.fedoraproject.org/rpms/shadow-utils/c/1820858d26f633482d3679d1c97324e25e3c1df8?branch=rawhide [ Test Plan ] There is a script attached to this bug which will reproduce the issue. It assumes a working lxd setup, and that "lxc launch ubuntu- daily:<ubuntu-release-name>" produces a working container. The script will exercise the following aspects of this bug: - provide the necessary cloud-init user-data to trigger the bug - check that the user has a password set or not in the end - check that "passwd -u" produces or not the stderr output that triggers the bug - check that "sss_cache -UG" is *always* silent, i.e., no stderr noise about the missing config.ldb To run the test, download the script to some directory and make it executable: - wget https://bugs.launchpad.net/ubuntu/+source/shadow/+bug/2153951/+attachment/6009255/+files/lp2153951-test.sh + wget https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/2153951/+attachment/6011062/+files/lp2153951-test.sh chmod +x lp2153951-test.sh Then run it targeting the ubuntu release under test. For example, for resolute: ./lp2153951-test.sh --release resolute That should reproduce the error, i.e., the script must indicate the test as having failed. To test the fixed packages, pass the "--pocket proposed" parameter. Again with the resolute example: ./lp2153951-test.sh --release resolute --pocket proposed This time the script should indicate that all checks have passed, and that completes the verification for this SRU. Here is run that shows the bug being reproduced: $ ./lp2153951-test.sh --release resolute Launching resolute container lp2153951-1955635 (phase 1: repositories + packages) Launching lp2153951-1955635 ...........................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: Installed: 1:4.17.4-2ubuntu3 Candidate: 1:4.17.4-2ubuntu3 Version table: *** 1:4.17.4-2ubuntu3 500 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages 100 /var/lib/dpkg/status === Checks === FAIL: tuser has no hashed password in /etc/shadow (got: '<empty>') FAIL: 'passwd -u tuser' complained about the SSSD config db on stderr: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr 'sss_cache -UG' stderr was: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? RESULT: FAIL Destroying container lp2153951-1955635 And here is a run showing the tests passing (I used a PPA for this example, but the real SRU verification must use "--pocket proposed"): $ ./lp2153951-test.sh --release resolute --pocket ppa:ahasenack/shadow-sssd-cache Launching resolute container lp2153951-1959790 (phase 1: repositories + packages) Launching lp2153951-1959790 ......................................................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: Installed: 1:4.17.4-2ubuntu3.1~ppa2 Candidate: 1:4.17.4-2ubuntu3.1~ppa2 Version table: *** 1:4.17.4-2ubuntu3.1~ppa2 500 500 https://ppa.launchpadcontent.net/ahasenack/shadow-sssd-cache/ubuntu resolute/main amd64 Packages 100 /var/lib/dpkg/status 1:4.17.4-2ubuntu3 500 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages === Checks === PASS: tuser has a hashed password in /etc/shadow (got: '$6$lp2153951$Iq7BfvEAR8qyz9eKAKs5sAGBBtyDoSRW0D2FnBJdG3Xy5.rLbSh29Hrds0VXSNzIx/1PNNmATPReqLjginX7/.') PASS: 'passwd -u tuser' produced no SSSD config-db complaint on stderr PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr 'sss_cache -UG' stderr was: [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? RESULT: PASS Destroying container lp2153951-1959790 [ Where problems could occur ] An update to src:shadow affects all ubuntu users. The patch is contained to the code that flushes the sssd cache, and only runs if the sss_cache binary is installed. The cache flush, however, is not confined to the passwd command only. It is used throughout the code, and also called from groupadd and others (again, only if sss_cache is present). This is also the reason why currently you see the same config.ldb errors messages when installing sssd-tools fresh: ... Setting up sssd-common (2.12.0-1ubuntu5.5) ... Creating SSSD system user & group... [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? ... [ Other Info ] See "Original Description" below for the original scenario where this bug appeared. [ Original Description ] Installed ubuntu-desktop in 26.04 server, and it didn't generate config.ldb file which kept deleting the password configured by cloud- init user data. ubuntu@ip-10-0-1-145:~$ sudo passwd -u ubuntu 2>&1 [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? passwd: password changed. ubuntu@ip-10-0-1-145:~$ echo "exit: $?" exit: 0 there are output even though exit code 0 which triggers passwd -d in cloud-init script ** Description changed: [ Impact ] When using cloud-init to add a local user with a specific password/hash to the system at first boot, and sssd-tools also happens to be installed but sssd itself is not configured yet, a sequence of events leads to that user having no password set: - the user is created with the specified password hash just fine - cloud-init calls[1] "passwd -u" on the user - that passwd call ends up also calling "sss_cache -UG"[2], to reset the sssd cache, since the src:shadow package (where bin:passwd comes from) is built with sssd support[3] by default. If sss_cache is not available (bin:sssd-tools not installed), the bug does not manifest itself. - calling "sss_cache -UG" on a system with no valid sssd configuration yet prints an error message to stderr about a missing config.ldb, but exits with 0. This is since commits [4][5]. Those commits also changed sss_cache so that it no longer generates the config.ldb database, hence why it now always complains. - cloud-init checks exit status, and stderr, of the "passwd -u" call. It sees that stderr has content[6], and assumes a failure, proceeding to calling "passwd -d" on that account. This is what deletes the password hash from the user. For this SRU, I had originally opted to silence the error from the caller's side (src:shadow) if there is no config.ldb database present. That call is a best effort attempt to be helpful and clear the sssd cache, and the result of the call is being ignored anyway. But that has a much wider impact: src:shadow is installed everywhere. Even users not affected by this bug would get the update. Fedora (which is the sssd upstream) opted to entirely disable[7] sssd support in src:shadow. That is an option to consider for devel releases of ubuntu, but not an SRU. I then decided to silence the error from within sss_cache instead. It's the same fix in principle, and only users affected by the bug will receive the update. The one small change in behavior is that now when sss_cache doesn't find the config.ldb database, it's silent, so users might not be immediately aware that no action was taken by the tool. Note, however, that the exit code remains unchanged. And if you think about the end result of the tool run, it's the same: the cache was either expunged, or there was nothing to expunge. 1. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1059 2. https://git.launchpad.net/ubuntu/+source/shadow/tree/src/passwd.c?h=applied/ubuntu/devel#n1175 3. https://git.launchpad.net/ubuntu/+source/shadow/tree/configure.ac?h=applied/ubuntu/devel#n202 4. https://github.com/SSSD/sssd/commit/a008accecd6d0b35e8d57d738ee3d05863aa7d0f 5. https://github.com/SSSD/sssd/commit/50b4579413c1146a2fced5ced19977e13790731c 6. https://github.com/canonical/cloud-init/blob/6193b2a0d241bed305e28a7aced1bd5c8c09de9a/cloudinit/distros/__init__.py#L1063 7. https://src.fedoraproject.org/rpms/shadow-utils/c/1820858d26f633482d3679d1c97324e25e3c1df8?branch=rawhide [ Test Plan ] There is a script attached to this bug which will reproduce the issue. It assumes a working lxd setup, and that "lxc launch ubuntu- daily:<ubuntu-release-name>" produces a working container. The script will exercise the following aspects of this bug: - provide the necessary cloud-init user-data to trigger the bug - check that the user has a password set or not in the end - check that "passwd -u" produces or not the stderr output that triggers the bug - check that "sss_cache -UG" is *always* silent, i.e., no stderr noise about the missing config.ldb To run the test, download the script to some directory and make it executable: wget https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/2153951/+attachment/6011062/+files/lp2153951-test.sh chmod +x lp2153951-test.sh Then run it targeting the ubuntu release under test. For example, for resolute: ./lp2153951-test.sh --release resolute That should reproduce the error, i.e., the script must indicate the test as having failed. To test the fixed packages, pass the "--pocket proposed" parameter. Again with the resolute example: ./lp2153951-test.sh --release resolute --pocket proposed This time the script should indicate that all checks have passed, and that completes the verification for this SRU. Here is run that shows the bug being reproduced: - $ ./lp2153951-test.sh --release resolute - Launching resolute container lp2153951-1955635 (phase 1: repositories + packages) - Launching lp2153951-1955635 - ...........................................................status: done + $ ./lp2153951-test.sh --release stonking + Launching stonking container lp2153951-192830 (phase 1: repositories + packages) + Launching lp2153951-192830 + .....................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: - Installed: 1:4.17.4-2ubuntu3 - Candidate: 1:4.17.4-2ubuntu3 - Version table: - *** 1:4.17.4-2ubuntu3 500 - 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages - 100 /var/lib/dpkg/status + Installed: 1:4.19.3-2ubuntu2 + Candidate: 1:4.19.3-2ubuntu2 + Version table: + *** 1:4.19.3-2ubuntu2 500 + 500 http://archive.ubuntu.com/ubuntu stonking/main amd64 Packages + 100 /var/lib/dpkg/status === Checks === FAIL: tuser has no hashed password in /etc/shadow (got: '<empty>') FAIL: 'passwd -u tuser' complained about the SSSD config db on stderr: - [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured - Can't find configuration db, was SSSD configured and run? - [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured - Can't find configuration db, was SSSD configured and run? - PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr - 'sss_cache -UG' stderr was: - [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured - Can't find configuration db, was SSSD configured and run? + [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured + Can't find configuration db, was SSSD configured and run? + [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured + Can't find configuration db, was SSSD configured and run? + FAIL: 'sss_cache -UG' reported unexpected content on stderr + 'sss_cache -UG' stderr was: + [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured + Can't find configuration db, was SSSD configured and run? RESULT: FAIL - Destroying container lp2153951-1955635 + Destroying container lp2153951-192830 And here is a run showing the tests passing (I used a PPA for this example, but the real SRU verification must use "--pocket proposed"): - $ ./lp2153951-test.sh --release resolute --pocket ppa:ahasenack/shadow-sssd-cache - Launching resolute container lp2153951-1959790 (phase 1: repositories + packages) - Launching lp2153951-1959790 - ......................................................................................status: done + $ ./lp2153951-test.sh --release stonking --pocket ppa:ahasenack/silence-sss-cache + Launching stonking container lp2153951-196703 (phase 1: repositories + packages) + Launching lp2153951-196703 + ..........................................................................................status: done Resetting cloud-init state and rebooting (phase 2: user configuration) status: done === apt-cache policy passwd (package source used in this test) === passwd: - Installed: 1:4.17.4-2ubuntu3.1~ppa2 - Candidate: 1:4.17.4-2ubuntu3.1~ppa2 - Version table: - *** 1:4.17.4-2ubuntu3.1~ppa2 500 - 500 https://ppa.launchpadcontent.net/ahasenack/shadow-sssd-cache/ubuntu resolute/main amd64 Packages - 100 /var/lib/dpkg/status - 1:4.17.4-2ubuntu3 500 - 500 http://archive.ubuntu.com/ubuntu resolute/main amd64 Packages + Installed: 1:4.19.3-2ubuntu2 + Candidate: 1:4.19.3-2ubuntu2 + Version table: + *** 1:4.19.3-2ubuntu2 500 + 500 http://archive.ubuntu.com/ubuntu stonking/main amd64 Packages + 100 /var/lib/dpkg/status === Checks === PASS: tuser has a hashed password in /etc/shadow (got: '$6$lp2153951$Iq7BfvEAR8qyz9eKAKs5sAGBBtyDoSRW0D2FnBJdG3Xy5.rLbSh29Hrds0VXSNzIx/1PNNmATPReqLjginX7/.') PASS: 'passwd -u tuser' produced no SSSD config-db complaint on stderr - PASS: 'sss_cache -UG' reported the expected SSSD config-db error on stderr - 'sss_cache -UG' stderr was: - [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured - Can't find configuration db, was SSSD configured and run? + PASS: 'sss_cache -UG' was silent + 'sss_cache -UG' stderr was: + RESULT: PASS - Destroying container lp2153951-1959790 + Destroying container lp2153951-196703 [ Where problems could occur ] An update to src:shadow affects all ubuntu users. The patch is contained to the code that flushes the sssd cache, and only runs if the sss_cache binary is installed. The cache flush, however, is not confined to the passwd command only. It is used throughout the code, and also called from groupadd and others (again, only if sss_cache is present). This is also the reason why currently you see the same config.ldb errors messages when installing sssd-tools fresh: ... Setting up sssd-common (2.12.0-1ubuntu5.5) ... Creating SSSD system user & group... [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? ... [ Other Info ] See "Original Description" below for the original scenario where this bug appeared. [ Original Description ] Installed ubuntu-desktop in 26.04 server, and it didn't generate config.ldb file which kept deleting the password configured by cloud- init user data. ubuntu@ip-10-0-1-145:~$ sudo passwd -u ubuntu 2>&1 [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? [sss_cache] [sss_tool_confdb_init] (0x0010): Can't access '/var/lib/sss/db/config.ldb', probably SSSD isn't configured Can't find configuration db, was SSSD configured and run? passwd: password changed. ubuntu@ip-10-0-1-145:~$ echo "exit: $?" exit: 0 there are output even though exit code 0 which triggers passwd -d in cloud-init script ** Changed in: sssd (Ubuntu) Status: Invalid => In Progress ** Changed in: sssd (Ubuntu Resolute) Status: Invalid => In Progress ** Changed in: sssd (Ubuntu Stonking) Status: Invalid => In Progress ** Changed in: shadow (Ubuntu) Status: In Progress => Won't Fix ** Changed in: sssd (Ubuntu Resolute) Importance: Undecided => Medium ** Changed in: shadow (Ubuntu Resolute) Status: In Progress => Won't Fix ** Changed in: shadow (Ubuntu Stonking) Status: In Progress => Won't Fix ** Changed in: shadow (Ubuntu Resolute) Importance: Undecided => Medium ** Changed in: shadow (Ubuntu) Importance: High => Medium ** Changed in: shadow (Ubuntu Stonking) Importance: High => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2153951 Title: no config.ldb file under /var/lib/sss/db in 26.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/shadow/+bug/2153951/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
