Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
On Sat, 2024-04-27 at 03:15 +0200, Guilhem Moulin wrote: > Yup that'd make sense to me (and I see you did that already), thanks! :-) Unfortunately I doubt it will be possibly to do some fully generic solution. So best we'll get is probably either an unconditional inclusion or some simpler copy_* function. Thanks for your help with the hint on the 2.34 change... I had already been at my wits end, why pthread didn't show up at ldd ^^ Best wishes, Chris.
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
On Sat, 27 Apr 2024 at 02:07:21 +0200, Christoph Anton Mitterer wrote: > So you say it's a glibc thingy, that this doesn't show up anymore? Yup, that's what I wrote https://bugs.debian.org/1032235#97 | It was intentional, see the article | https://developers.redhat.com/articles/2021/12/17/why-glibc-234-removed-libpthread . | Unfortunately that change broke initramfs-tools' fix for https://bugs.debian.org/950254 | which we (src:cryptsetup maintainers) relied on for cryptsetup-initramfs. | Until last week src:argon2 had never been rebuilt with the newer glibc, | so it's just unfortunate that we missed that at the time. On the one hand it was unfortunate to find such a severe RC bug (unbootable system with the default config) so late in the freeze, on the other hand it would have been even worse if src:argon2 would have been uploaded to bookworm-security or -pu after the stable release :-) > $ ldd /usr/bin/argon2 > linux-vdso.so.1 (0x7ffcc67d4000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f1a1ee48000) > /lib64/ld-linux-x86-64.so.2 (0x7f1a1f058000) > > I should use libc for determination? Don't know if that's the best or most robust solution but that's what I'd do as workaround at least. > Would you recommend me to reassign #1069912 to initramfs-tools, asking > for a more user-friendly solution? Yup that'd make sense to me (and I see you did that already), thanks! -- Guilhem. signature.asc Description: PGP signature
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
On Sat, 2024-04-27 at 01:48 +0200, Guilhem Moulin wrote: > built using glibc ≥2.34. AFAICT the “if the ldd output includes > libpthread then run copy_libgcc()” logic from initramfs-tools is > mostly moot > now Ah, I just realised glibc "merged" libpthread ^^ Therefore... > but despite what I promised elbrus in > https://bugs.debian.org/1032235#87 it > looks like I didn't file a bug. ... I helped you fulfilling your promise and re-assigned my #1069912, making it the bug that asks for some more generic solution :-) Thanks, Chris.
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Hey Guilhem On Sat, 2024-04-27 at 01:48 +0200, Guilhem Moulin wrote: > Even it weren't, libpthread wouldn't show up since src:argon2 from > bookworm > and later is built using glibc ≥2.34. When argon2 builds, it uses -pthread ... not really sure what that does exactly, the manpage merely says it links against pthread, but statically? Dynamically? Anyway, when building it manually and even with -lpthread, it doesn't show up - just as you say. Tried now for an hour or so to make it show up (eventually gave up and filed #1069912). So you say it's a glibc thingy, that this doesn't show up anymore? > AFAICT the “if the ldd output includes > libpthread then run copy_libgcc()” logic from initramfs-tools is > mostly moot > now, see https://bugs.debian.org/1032235#97 . Shouldn't initramfstools then not simply generally include libgcc? > We used the following workaround to manually call copy_libgcc() > > > https://salsa.debian.org/cryptsetup-team/cryptsetup/-/commit/4cade1eae57abd93e0d6491eebce5f5f163ef186#a630d04e2df57150e6a092fc23f955c6ea0ce412 > > https://salsa.debian.org/cryptsetup-team/cryptsetup/-/commit/369cb651abad11a3844fa38ea86ece40f4f40078 As a workaround I've used now: copy_libgcc /lib/x86_64-linux-gnu with the libpath hardcoded... but I have no idea how I should get that path properly. As far as I can see from your commit, you simply used the path that libargon2 was using? But I don't have that since argon2 doesn't link against it ;-) Any idea what would be the best solution in my case? > for Bookworm, but removed it with 2:2.7.2-1 since src:cryptsetup no > longer uses libargon. There is no stability guaranty for transitive > dependencies so I'd indeed argue the regression is not > src:cryptsetup's > fault :-) Adapting the above workarounds to your custom hookfile > should > solve the issue, though. Hmm yes, but I'm not sure if that would be a more proper solution than mine (with the hardcoded path), cause: In principle, my argon2 binary tool, might be from another arch as the installed libargon2 (if it's installed at all, which isn't necessarily the case). So not sure, but maybe: $ ldd /usr/bin/argon2 linux-vdso.so.1 (0x7ffcc67d4000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f1a1ee48000) /lib64/ld-linux-x86-64.so.2 (0x7f1a1f058000) I should use libc for determination? > > > Did anyone of your report this issue anywhere else? > > Some years ago I asked the initramfs-tools maintainers to > inconditionally copy > libgcc_s to the initramfs image, but IIRC Ben argued it was too > seldom used to > justify the cost in size and impemented the libpthread detection > logic + > copy_libgcc() instead. > > I don't know if the detection logic can be improved/fixed in > initramfs-tools, > but despite what I promised elbrus in > https://bugs.debian.org/1032235#87 it > looks like I didn't file a bug. So... I assume the change in glibc is proper then... and a fix (if at all) would rather need to go to initramfs-tools? Would you recommend me to reassign #1069912 to initramfs-tools, asking for a more user-friendly solution? Thanks, Chris.
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Hi, On Sat, 27 Apr 2024 at 00:33:51 +0200, Christoph Anton Mitterer wrote: > Now the problem is that argon2 is statically linked, so there's no > libpthread showing up in its ldd, and thus copy_exec doesn't realise it > needs to invoke copy_libgcc. Even it weren't, libpthread wouldn't show up since src:argon2 from bookworm and later is built using glibc ≥2.34. AFAICT the “if the ldd output includes libpthread then run copy_libgcc()” logic from initramfs-tools is mostly moot now, see https://bugs.debian.org/1032235#97 . We used the following workaround to manually call copy_libgcc() https://salsa.debian.org/cryptsetup-team/cryptsetup/-/commit/4cade1eae57abd93e0d6491eebce5f5f163ef186#a630d04e2df57150e6a092fc23f955c6ea0ce412 https://salsa.debian.org/cryptsetup-team/cryptsetup/-/commit/369cb651abad11a3844fa38ea86ece40f4f40078 for Bookworm, but removed it with 2:2.7.2-1 since src:cryptsetup no longer uses libargon. There is no stability guaranty for transitive dependencies so I'd indeed argue the regression is not src:cryptsetup's fault :-) Adapting the above workarounds to your custom hookfile should solve the issue, though. > Did anyone of your report this issue anywhere else? Some years ago I asked the initramfs-tools maintainers to inconditionally copy libgcc_s to the initramfs image, but IIRC Ben argued it was too seldom used to justify the cost in size and impemented the libpthread detection logic + copy_libgcc() instead. I don't know if the detection logic can be improved/fixed in initramfs-tools, but despite what I promised elbrus in https://bugs.debian.org/1032235#87 it looks like I didn't file a bug. -- Guilhem. signature.asc Description: PGP signature
Bug#1068849: [pkg-cryptsetup-devel] Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Hey guys.
I kinda ran into a similar issue.
I use my own OpenPGP keyscript which is highly improved upon that
("decrypt_gnupg") shipped by the package.
One thing that I do is offer optionally feeding the entered passphrase
trough argon2 (the standalone tool from the package of the same name)
which I copy via copy_exec.
Now the problem is that argon2 is statically linked, so there's no
libpthread showing up in its ldd, and thus copy_exec doesn't realise it
needs to invoke copy_libgcc.
Did anyone of your report this issue anywhere else? I mean it's
obviously not crypsetup.
But I cannot really blame argon2 either, nor can I really blame update-
initramfs.
Cheers,
Chris.
Processed: Re: Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Processing control commands:
> reopen -1
Bug #1068849 {Done: Guilhem Moulin } [cryptsetup-initramfs]
cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Bug #1068848 {Done: Guilhem Moulin } [cryptsetup-initramfs]
cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Bug reopened
Ignoring request to alter fixed versions of bug #1068849 to the same values
previously set
Ignoring request to alter fixed versions of bug #1068848 to the same values
previously set
> tag -1 - unreproducible moreinfo
Bug #1068849 [cryptsetup-initramfs] cryptsetup: Fails to unlock the filesystem
with missing libgcc_s.so.1
Bug #1068848 [cryptsetup-initramfs] cryptsetup: Fails to unlock the filesystem
with missing libgcc_s.so.1
Removed tag(s) unreproducible and moreinfo.
Removed tag(s) unreproducible and moreinfo.
--
1068848: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068848
1068849: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068849
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Control: reopen -1 Control: tag -1 - unreproducible moreinfo On Sun, 14 Apr 2024 at 21:26:25 +0200, Guilhem Moulin wrote: > At this point something triggered rebuilding a new initramfs image, but > that's not src:cryptsetup as none of its binary packages have been > upgraded yet. On second thought that was likely incorrect. The log doesn't show anything from src:cryptsetup but likely cryptsetup-initramfs was already upgraded at that point while libcryptsetup12 wasn't. The package dependency constraints are indeed not strict enough. cryptsetup-initramfs now assumes neither cryptsetup(8) nor libcryptsetup.so.12 are linked with libargon2, but since no new symbols were added in 2.7.2 cryptsetup-bin only has Depends: libcryptsetup12 (>= 2:2.7), and it's therefore possible to upgrade cryptsetup-initramfs while keeping the old libcryptsetup12. Upgrading from testing (2:2.6.1-6) works thanks to the Depends: libcryptsetup12 (>= 2:2.7), but upgrading from ≥2:2.7~, <2:2.7.2-1 may yield a broken initramfs image if libcryptsetup12 is not upgraded before cryptsetup-initramfs. -- Guilhem. signature.asc Description: PGP signature
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
I can not anymore reproduce the issue anymore: while trying to capture the full error message, I created an initrd without libgcc_s in it and it booted up. Below I added some--hopefully relevant-- parts of the dpkg log and original requested phread output, maybe that still sheds some light on this. If not I guess this can be closed. On Fri, 12 Apr 2024 at 16:23, Guilhem Moulin wrote: > FWIW, I later noticed you used a splash screen (plymouth) and thought it yes, sorry: GUI = Splash screen > Otherwise, my guess is a race where the initramfs image was somehow > rebuilt before libcryptsetup12 was upgraded, but AFAICT the dependencies > are properly set to avoid that. Does the broken initramfs image contain > libargon2.so, if so does its libcryptsetup12 use it? According to `lsinitramfs initrd.img-6.7.9-amd64 |grep libargon2.so` it's not used. I've saved away the dpkg.log and this is an excerpt (via `rg "trigproc initramfs-tools|libcryptsetup12|libargon2|libssl3t64|libc6|libglib2|libgcc" ~/tmp/dpkg.log`) with added output from `rg "Start-Date|Commandline" /var/log/apt/history.log`. The last two initramfs-tools trigger calls should correspond to me trying to fix the problem by reinstalling some packages and maybe not even realizing it's already fixed by the dist-upgrade? 2:Start-Date: 2024-04-10 11:30:00 3:Commandline: apt upgrade -y 7:End-Date: 2024-04-10 11:33:19 87:2024-04-10 11:30:04 upgrade libgcc-13-dev:amd64 13.2.0-18 13.2.0-23 88:2024-04-10 11:30:04 status half-configured libgcc-13-dev:amd64 13.2.0-18 89:2024-04-10 11:30:04 status unpacked libgcc-13-dev:amd64 13.2.0-18 90:2024-04-10 11:30:04 status half-installed libgcc-13-dev:amd64 13.2.0-18 91:2024-04-10 11:30:04 status unpacked libgcc-13-dev:amd64 13.2.0-23 107:2024-04-10 11:30:06 upgrade libgcc-12-dev:amd64 12.3.0-15 12.3.0-17 108:2024-04-10 11:30:06 status half-configured libgcc-12-dev:amd64 12.3.0-15 109:2024-04-10 11:30:06 status unpacked libgcc-12-dev:amd64 12.3.0-15 110:2024-04-10 11:30:06 status half-installed libgcc-12-dev:amd64 12.3.0-15 111:2024-04-10 11:30:06 status unpacked libgcc-12-dev:amd64 12.3.0-17 158:2024-04-10 11:30:07 upgrade libgcc-s1:amd64 14-20240303-1 14-20240330-1 159:2024-04-10 11:30:07 status half-configured libgcc-s1:amd64 14-20240303-1 160:2024-04-10 11:30:07 status unpacked libgcc-s1:amd64 14-20240303-1 161:2024-04-10 11:30:07 status half-installed libgcc-s1:amd64 14-20240303-1 162:2024-04-10 11:30:07 status unpacked libgcc-s1:amd64 14-20240330-1 164:2024-04-10 11:30:07 configure libgcc-s1:amd64 14-20240330-1 165:2024-04-10 11:30:07 status unpacked libgcc-s1:amd64 14-20240330-1 166:2024-04-10 11:30:07 status half-configured libgcc-s1:amd64 14-20240330-1 167:2024-04-10 11:30:07 status installed libgcc-s1:amd64 14-20240330-1 210:2024-04-10 11:30:08 upgrade libc6-dev:amd64 2.37-15.1 2.37-16 211:2024-04-10 11:30:08 status half-configured libc6-dev:amd64 2.37-15.1 212:2024-04-10 11:30:08 status unpacked libc6-dev:amd64 2.37-15.1 213:2024-04-10 11:30:08 status half-installed libc6-dev:amd64 2.37-15.1 214:2024-04-10 11:30:08 status unpacked libc6-dev:amd64 2.37-16 235:2024-04-10 11:30:09 upgrade libc6:amd64 2.37-15.1 2.37-16 236:2024-04-10 11:30:09 status half-configured libc6:amd64 2.37-15.1 237:2024-04-10 11:30:09 status unpacked libc6:amd64 2.37-15.1 238:2024-04-10 11:30:09 status half-installed libc6:amd64 2.37-15.1 239:2024-04-10 11:30:09 status unpacked libc6:amd64 2.37-16 241:2024-04-10 11:30:09 configure libc6:amd64 2.37-16 242:2024-04-10 11:30:09 status unpacked libc6:amd64 2.37-16 243:2024-04-10 11:30:09 status half-configured libc6:amd64 2.37-16 244:2024-04-10 11:30:10 status installed libc6:amd64 2.37-16 755:2024-04-10 11:30:33 upgrade libgcc-11-dev:amd64 11.4.0-7 11.4.0-8 756:2024-04-10 11:30:33 status half-configured libgcc-11-dev:amd64 11.4.0-7 757:2024-04-10 11:30:33 status unpacked libgcc-11-dev:amd64 11.4.0-7 758:2024-04-10 11:30:33 status half-installed libgcc-11-dev:amd64 11.4.0-7 759:2024-04-10 11:30:34 status unpacked libgcc-11-dev:amd64 11.4.0-8 951:2024-04-10 11:30:43 status triggers-pending libglib2.0-0:amd64 2.78.4-1 1491:2024-04-10 11:31:04 upgrade libglib2.0-data:all 2.78.4-4 2.78.4-6 1492:2024-04-10 11:31:04 status half-configured libglib2.0-data:all 2.78.4-4 1493:2024-04-10 11:31:04 status unpacked libglib2.0-data:all 2.78.4-4 1494:2024-04-10 11:31:04 status half-installed libglib2.0-data:all 2.78.4-4 1495:2024-04-10 11:31:04 status unpacked libglib2.0-data:all 2.78.4-6 2711:2024-04-10 11:32:13 configure libglib2.0-data:all 2.78.4-6 2712:2024-04-10 11:32:13 status unpacked libglib2.0-data:all 2.78.4-6 2713:2024-04-10 11:32:13 status half-configured libglib2.0-data:all 2.78.4-6 2714:2024-04-10 11:32:13 status installed libglib2.0-data:all 2.78.4-6 3527:2024-04-10 11:32:42 configure libgcc-12-dev:amd64 12.3.0-17 3528:2024-04-10 11:32:42 status unpacked libgcc-12-dev:amd64 12.3.0-17 3529:2024-04-10 11:32:42 status half-configured libgcc-12-dev:amd64 12.3.0-17 3530:2024-04-10 11:32:42
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
On Fri, 12 Apr 2024 at 14:37:16 +0200, Guilhem Moulin wrote:
> What is that “GUI” view? src:cryptsetup doesn't provide that, I wonder
> if it might be what needs libphtread.
FWIW, I later noticed you used a splash screen (plymouth) and thought it
might be because of that, but I still cannot reproduce the issue in a
bookworm VM upgraded to sid (using d-i's “encrypted LVM” layout + a
splash screen).
>> After a ctrl-alt-del, I got to the console and there it showed an error about
>> libgcc_s.so.1 not available and aborting.
What was that error message exactly?
> If you still have the broken initramfs image, please extract it with
> `unmkinitramfs /path/to/broken.initrd.img /path/to/destdir` and try to
> find which executable / shared library needs libphtread, for instance
> with
>
>cd /path/to/destdir
>for p in $(find -P {usr/,}lib/x86_64-linux-gnu {usr/,}{,s}bin/ -type f);
> do objdump -p "$p" 2>/dev/null | grep -q pthread && echo "$p"; done
Erm no, that likely won't work since pthread functions have moved from
libpthread.so.1 to libc.so.6 with glibc 2.34. Otherwise copy_exec()
would have pulled in libgcc_s. New attempt:
rm -rf /tmp/initramfs-destdir && \
mkdir -m0700 /tmp/initramfs-destdir && \
unmkinitramfs /path/to/broken.initrd.img /tmp/initramfs-destdir && \
for p in $(find /tmp/initramfs-destdir/main -type f | sort); do \
objdump -T "$p" 2>&1 | grep pthread_exit && echo "^^ $p"; \
done
--
Guilhem.
signature.asc
Description: PGP signature
Processed: Re: Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Processing control commands: > tag -1 + unreproducible moreinfo Bug #1068849 [cryptsetup-initramfs] cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1 Bug #1068848 [cryptsetup-initramfs] cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1 Added tag(s) unreproducible and moreinfo. Added tag(s) unreproducible and moreinfo. -- 1068848: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068848 1068849: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068849 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
Control: tag -1 + unreproducible moreinfo
On Fri, 12 Apr 2024 at 12:45:09 +0200, Milan Broz wrote:
> Just FYI (for upstream code): if cryptsetup/libcryptsetup is linked with
> OpenSSL >= 3.2,
> it does not need libphtread (as threads are implemented in OpenSSL for Argon2
> internally).
Thanks for confirming! We're now linking with OpenSSL 3.2 indeed, and
our CI confirms that unlocking works in that environment.
2:2.7.2-1 builds with OpenSSL 3.2 *and* removes the libgcc_s/libargon2
workaround from the initramfs hook:
https://tracker.debian.org/news/1517996/accepted-cryptsetup-2272-1-source-into-unstable/
>> It would also be nice if the "gui" view could show the error or at
>> least tell the user to pres ctrl-alt-del to get to a more informative
>> view, took me ages to figure out that one :-(
What is that “GUI” view? src:cryptsetup doesn't provide that, I wonder
if it might be what needs libphtread.
Otherwise, my guess is a race where the the initramfs image was somehow
rebuilt before libcryptsetup12 was upgraded, but AFAICT the dependencies
are properly set to avoid that. Does the broken initramfs image contain
libargon2.so, if so does its libcryptsetup12 use it?
If you still have the broken initramfs image, please extract it with
`unmkinitramfs /path/to/broken.initrd.img /path/to/destdir` and try to
find which executable / shared library needs libphtread, for instance
with
cd /path/to/destdir
for p in $(find -P {usr/,}lib/x86_64-linux-gnu {usr/,}{,s}bin/ -type f);
do objdump -p "$p" 2>/dev/null | grep -q pthread && echo "$p"; done
This will hopefully shed some light on this.
--
Guilhem.
signature.asc
Description: PGP signature
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
On Fri, 12 Apr 2024 at 14:00, Chris Hofstaedtler wrote: > Can you tell us which part was wanting libgcc_s.so.1? > cryptsetup in the initramfs doesn't seem to be the (original) > problem of that. How would I find out? (I've never debugged such a problem, and have basically no clue what is happening behind the scenes during encrypted hd unlock and how to figure out "what wants it" :-( ) I basically inputted a password (both on the "UI" and the console after ctrl-alt-del) and on the console it showed me that error after pressing enter. It then showed the passwort entry box/line again. Given that it was the password validation, I figured I should file a bug here. Jan
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
On Fri, Apr 12, 2024 at 09:11:51AM +0200, Jan Katins wrote: > Package: cryptsetup > Version: 2:2.7.2-1 > Severity: normal > > Dear Maintainer, > > After a recent apt upgrade on a debian sid system (installed about 3 > years ago with an installer, choosing to encrypt the filesystem, no > idea what actually ended up on my system as a crypt setup. Since then, > the laptop runs debian unstable), my system failed to unlock. After a > ctrl-alt-del, I got to the console and there it showed an error about > libgcc_s.so.1 not available and aborting. Can you tell us which part was wanting libgcc_s.so.1? cryptsetup in the initramfs doesn't seem to be the (original) problem of that. Chris
Bug#1068849: cryptsetup: Fails to unlock the filesystem with missing libgcc_s.so.1
On 4/12/24 9:11 AM, Jan Katins wrote: I snooped around in the source code a bit and found that libgcc_s seems to be dlopened and is special cased: https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/master/hook-functions?ref_type=heads#L248-249 (original bugreport: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=950254). So my guess is that nothing depends on libpthread anymore, Just FYI (for upstream code): if cryptsetup/libcryptsetup is linked with OpenSSL >= 3.2, it does not need libphtread (as threads are implemented in OpenSSL for Argon2 internally). Ditto for libargon2 and possible dependences of above (libgcc_s). So if the reason is cryptsetup, all these dependences should be removed, we do not need it anymore. Milan
