Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Wed, Apr 14, 2021 at 08:50:46PM +0200, Paul Gevers wrote: > Hi Ivo, Marco, > > On 06-04-2021 22:10, Ivo De Decker wrote: > > I ran a number of (partial and full) upgrade tests, and they all seem to > > work > > fine. In all cases, libcrypt1 is installed before libc6, and there is no > > intermediate situations where libcrypt.so.1 is missing. > > The patch looks sensible after reading the discussion in these bugs. Can > we have an upload soon to have exposure? Hi, thanks for your work on this and sorry I'm chiming in so late. I'm concerned that AFAICS no change in src:libxcrypt can make sure that the new libc6 is never unpacked before libcrypt1. (buster-amd64)# dpkg --unpack libc6_2.31-11_amd64.deb (Reading database ... 12224 files and directories currently installed.) Preparing to unpack libc6_2.31-11_amd64.deb ... debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.) debconf: falling back to frontend: Readline Unpacking libc6:amd64 (2.31-11) over (2.28-10) ... (buster-amd64)# perl perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory It may well be enough to make this improbable enough in practice. Ivo's testing certainly seems to indicate so. It still makes me a bit uneasy. I'm happy to be proved wrong of course. Do the Important or Protected fields in the patch affect apt's behaviour in this, for instance? The solution Aurelien mentioned of copying the old libcrypt in libc6.preinst and cleaning up in libc6.postinst would eliminate this failure mode totally. But I can see it's not very desirable and may be hard to make robust. Just wanted to bring this up explicitly. Not objecting if we deem the risk of remaining corner case issues small enough that it's worth taking. -- Niko
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Hi Marco, On Thu, Apr 15, 2021 at 02:27:10PM +0200, Marco d'Itri wrote: > On Apr 14, Paul Gevers wrote: > > > The patch looks sensible after reading the discussion in these bugs. Can > > we have an upload soon to have exposure? > Unless there are any objections I will do a libxcrypt upload in a couple > of days. OK, thanks! > I think that I can leave the udeb library in /usr/lib/. Yes. There is no upgrade issue there, and making sure the udeb doesn't change avoids potential issues with the installer. Ivo
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Apr 14, Paul Gevers wrote: > The patch looks sensible after reading the discussion in these bugs. Can > we have an upload soon to have exposure? Unless there are any objections I will do a libxcrypt upload in a couple of days. I think that I can leave the udeb library in /usr/lib/. -- ciao, Marco signature.asc Description: PGP signature
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Hi Ivo, Marco, On 06-04-2021 22:10, Ivo De Decker wrote: > I ran a number of (partial and full) upgrade tests, and they all seem to work > fine. In all cases, libcrypt1 is installed before libc6, and there is no > intermediate situations where libcrypt.so.1 is missing. The patch looks sensible after reading the discussion in these bugs. Can we have an upload soon to have exposure? Paul OpenPGP_signature Description: OpenPGP digital signature
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Hi,
On Fri, Mar 19, 2021 at 04:52:57PM +0100, Ivo De Decker wrote:
> > > I wonder if all this might be caused by the breaks from libcrypt1 (against
> > > libc6 (<< 2.29-4)). Is there a way to remove the breaks without causing
> > > issues? Maybe this is somewhat similar to the situation in
>
> I tried this with a modified libcrypt1 binary (removing the breaks), and it
> fails (/lib/x86_64-linux-gnu/libcrypt.so.1 is missing after the libc6 unpack).
> I guess that's because of #953562: libc6 shipped
> /lib/x86_64-linux-gnu/libcrypt.so.1 while libcrypt1 ships
> /usr/lib/x86_64-linux-gnu/libcrypt.so.1 Obviously, for dpkg these are 2
> different files, but on a system with merged /usr they are not.
I created a patch to move libcrypt back to /lib, and that seems to solve the
issue. The patch is attached. With this patch, libcrypt1 is installed before
libc6 is upgraded, and the takeover of the library works correctly.
This also fixes #953562, which reported that this move is necessary.
Note that the original move from /lib to /usr/lib was more complicated than my
patch:
https://salsa.debian.org/md/libxcrypt/-/commit/1fb86fde5410088580f8032834037facb26d2d49
I didn't look into the details of the -dev package, because they are not
relevant to this bug. The patch might need to be adapted.
I ran a number of (partial and full) upgrade tests, and they all seem to work
fine. In all cases, libcrypt1 is installed before libc6, and there is no
intermediate situations where libcrypt.so.1 is missing.
> > The problem of removing the break, is that we loose the possibility of
> > downgrades. Is it something acceptable?
>
> Well, I guess if that is the cost of not breaking people's systems on upgrade,
> that sounds like an acceptable trade-off. But I might be overlooking certain
> scenarios.
>
>
> > > https://bugs.debian.org/972936#24: libgcc-s1 takes over binaries from
> > > libgcc1,
> > > but only 'replaces' it, without breaks (note that extra steps had to be
> > > taken
> > > to avoid further issues (adding Important: yes and Protected: yes)).
> >
> > Are this extra-steps basically required to prevent downgrades?
>
> They are required to prevent you from removing libcrypt1 again: on a buster
> system, install such a hypothetical libcrypt1 from bullseye (which takes over
> libcrypt.so.1). Then remove that again. Now libcrypt.so.1 is missing. If the
> breaks is present, libcrypt1 can only be installed together with the new
> libc6, which prevents you from removing libcrypt1 afterwards.
As in #972936, my patch adds Important and Protected, which prevents removal of
libcrypt1 once it's installed.
Cheers,
Ivo
diff --git a/debian/changelog b/debian/changelog
index b5088dc..16d25ff 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,16 @@
+libxcrypt (2:4.4.17-1.1) UNRELEASED; urgency=medium
+
+ * Make sure takeover of libcrypt.so.1 from libc6 works correctly on upgrades
+from buster to bullseye (Closes: #974552):
+- Move the library back from /usr/lib/ to /lib/, because that's where it
+ was in the old libc6 (Closes: #953562)
+- Remove breaks from libcrypt1, to allow installing libcrypt1 before libc6
+ is upgraded
+- Mark libcrypt1 as Important and Protected, to prevent removal after a
+ partial upgrade
+
+ -- Ivo De Decker Tue, 06 Apr 2021 19:46:44 +
+
libxcrypt (1:4.4.17-1) unstable; urgency=medium
* New upstream release.
diff --git a/debian/control b/debian/control
index 35939dd..a2ae60a 100644
--- a/debian/control
+++ b/debian/control
@@ -16,11 +16,8 @@ Architecture: any
Multi-Arch: same
Pre-Depends: ${misc:Pre-Depends}
Depends: ${shlibs:Depends}, ${misc:Depends}
-Breaks:
- libc6 (<< 2.29-4),
- libc6.1 (<< 2.29-4) [alpha ia64],
- libc0.1 (<< 2.29-4) [kfreebsd-amd64 kfreebsd-i386],
- libc0.3 (<< 2.29-4) [hurd-i386],
+XB-Important: yes
+Protected: yes
Replaces:
libc6 (<< 2.29-4),
libc6.1 (<< 2.29-4) [alpha ia64],
diff --git a/debian/libcrypt-dev.files b/debian/libcrypt-dev.files
index 94387de..fc172a1 100644
--- a/debian/libcrypt-dev.files
+++ b/debian/libcrypt-dev.files
@@ -1,5 +1,5 @@
/usr/include/
/usr/share/man/
-/usr/lib/*/pkgconfig/
-/usr/lib/*/*.a
-/usr/lib/*/*.so
+/lib/*/pkgconfig/
+/lib/*/*.a
+/lib/*/*.so
diff --git a/debian/rules b/debian/rules
index 2969306..26f2bcd 100755
--- a/debian/rules
+++ b/debian/rules
@@ -34,11 +34,11 @@ CONFFLAGS = --disable-werror --prefix=/usr \
CONFFLAGS_deb = $(CONFFLAGS) \
$(shell DEB_BUILD_MAINT_OPTIONS="hardening=+bindnow" \
dpkg-buildflags --export=configure || true) \
- --libdir=/usr/lib/$(DEB_HOST_MULTIARCH)
+ --libdir=/lib/$(DEB_HOST_MULTIARCH)
CONFFLAGS_udeb = $(CONFFLAGS) \
$(subst -O2,-Os -fomit-frame-pointer,$(shell DEB_BUILD_MAINT_OPTIONS="hardening=-all" \
dpkg-buildflags --export=configure || true)) \
- --libdir=/usr/lib
+ --libdir=/lib
##
DEB_HOST_MULTIARCH ?= $(shell dpkg-architecture
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Hi, On Fri, Mar 19, 2021 at 03:51:34PM +0100, Aurelien Jarno wrote: > > However, in all of my tests, between the unpack of the new libc6 and > > libcrypt1 > > only other unpacks where done, and no dpkg hooks where run. If you have a > > way > > to reproduce the upgrade where dpkg ran the hook in between, that might be > > interesting. Do you still have a list of packages that was installed before > > you started the upgrade? > > Have you tried to install a foreign libc6? It usually makes the upgrade > a bit more tricky and could be what triggers the issue. I installed whois:i386, which pulled in libc6:i386 as well. Feel free to suggest a larger set of i386 for me to try. > > Note that I manually changed the binaries and didn't rebuild glibc, so I > > might > > have made a mistake, but this scenario should certainly be tested before > > something like this is uploaded to unstable. Maybe an upload to experimental > > might allow people to test this more easily? > > > > I Cced the apt maintainers to see if they have other suggestions to get > > apt/dpkg to unpack libcrypt1 before libc6. > > Another (ugly) suggestions we discussed at some point was: > 1) in the preinst copy the existing libcrypt1 library to a private and > add it to ldconfig with lower priority than /usr/lib/$(MULTIARCH) > 2) in the postinst remove these copy and ldconfig configuration. Do you think this is something that you can get to work in a reliable way? > > I wonder if all this might be caused by the breaks from libcrypt1 (against > > libc6 (<< 2.29-4)). Is there a way to remove the breaks without causing > > issues? Maybe this is somewhat similar to the situation in I tried this with a modified libcrypt1 binary (removing the breaks), and it fails (/lib/x86_64-linux-gnu/libcrypt.so.1 is missing after the libc6 unpack). I guess that's because of #953562: libc6 shipped /lib/x86_64-linux-gnu/libcrypt.so.1 while libcrypt1 ships /usr/lib/x86_64-linux-gnu/libcrypt.so.1 Obviously, for dpkg these are 2 different files, but on a system with merged /usr they are not. > The problem of removing the break, is that we loose the possibility of > downgrades. Is it something acceptable? Well, I guess if that is the cost of not breaking people's systems on upgrade, that sounds like an acceptable trade-off. But I might be overlooking certain scenarios. > > https://bugs.debian.org/972936#24: libgcc-s1 takes over binaries from > > libgcc1, > > but only 'replaces' it, without breaks (note that extra steps had to be > > taken > > to avoid further issues (adding Important: yes and Protected: yes)). > > Are this extra-steps basically required to prevent downgrades? They are required to prevent you from removing libcrypt1 again: on a buster system, install such a hypothetical libcrypt1 from bullseye (which takes over libcrypt.so.1). Then remove that again. Now libcrypt.so.1 is missing. If the breaks is present, libcrypt1 can only be installed together with the new libc6, which prevents you from removing libcrypt1 afterwards. Cheers, Ivo
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On 2021-03-19 15:24, Ivo De Decker wrote: > Hi, > > On Fri, Nov 20, 2020 at 01:44:50PM +0100, Alois Wohlschlager wrote: > > Am Freitag, den 20.11.2020, 09:13 + schrieb Niko Tyni: > > > I don't think this is related to the recent perl 5.30 -> 5.32 > > > transition. > > > The report is about a buster -> bullseye upgrade, and perl in > > > bullseye > > > was still at 5.30 at the time. > > > > > > In any case, the report says perl (presumably perl-base as well) was > > > still at the buster version when the breakage happened, so it didn't > > > have the libcrypt1 dependency. > > > > This is indeed the case. perl-base was only upgraded to the bullseye > > version after I manually fixed the breakage. (Indeed, when I wrote > > "Perl" I actually meant "perl-base", but perl itself was also at the > > buster version FWIW.) > > > > > FWIW the breakage can be reproduced just by manually unpacking the > > > new > > > libc6 on a buster system. > > > > > > I don't know why it hasn't been encountered earlier. > > > > I found out by now that the problem actually has been encountered > > earlier(#946599, where it broke libc's maintainer scripts). In my case, > > it broke the check-support-status hook providd by debian-security- > > support. (I'm not sure whether it's such a great idea for dpkg to run > > hooks when dependencies are broken, but that's what was happening on my > > system.) > > I filed #984539 against debian-security-support, to make sure the hook never > fails. That doesn't fix this bug, but at least it might reduce the potential > impact of issues like these. > > > > Yeah, I missed the libcrypt1 -> libc6 dependency. That prevents this > > > option. (I wonder if the circular dependency is a factor in apt > > > choosing > > > the upgrade order that results in this breakage.) > > > > I'm not sure what's going on here, as libcrypt1's libc6 Depends should > > be satisfied by the buster version. So it seems to me like installing > > libcrypt1 before upgrading libc6 should be strictly better than doing > > it the other way round, even purely in terms of satisfaction of > > dependencies. > > > > Maybe it's worth investigating why apt decides to proceed the "wrong" > > way anyway, should I try setting up a VM to reproduce the problem? > > I did some upgrade tests starting from the > debian-10-generic-amd64-20210208-542.qcow2 cloud image, and in all my tests, > apt chooses to unpack libc6 before libcrypt1. > > However, in all of my tests, between the unpack of the new libc6 and libcrypt1 > only other unpacks where done, and no dpkg hooks where run. If you have a way > to reproduce the upgrade where dpkg ran the hook in between, that might be > interesting. Do you still have a list of packages that was installed before > you started the upgrade? Have you tried to install a foreign libc6? It usually makes the upgrade a bit more tricky and could be what triggers the issue. > Note that even if the hook isn't run or doesn't fail, there is still a period > between the unpack of libc6 and libcrypt1 where perl is broken. If other > operations are done in between that cause maintainer scripts to run, these > could potentially call perl, which will be broken in this case. > > > > > > Another option might be to have the new libc6 Conflict with old > > > > > versions > > > > > of Essential:yes packages that need libcrypt1, forcing those > > > > > Essential:yes > > > > > packages to get upgraded first. A quick check based on libcrypt1 > > > > > reverse > > > > > dependencies in sid shows perl-base, login and util-linux. I'm > > > > > not sure > > > > > if this list is exhaustive, though. > > > > Having libc6 Conflict with one of those packages should be enough, > > right? > > I tried creating libc6 packages with either a conflits or a breaks agains the > old perl-base or the old login, and in each case I get this error: > > E: This installation run will require temporarily removing the essential > package libc6:amd64 due to a Conflicts/Pre-Depends loop. This is often bad, > but if you really want to do it, activate the APT::Force-LoopBreak option. > E: Internal Error, Could not early remove libc6:amd64 (2) > > So it doesn't look like this will work. I haven't tried, but it's what I was expecting. > Note that I manually changed the binaries and didn't rebuild glibc, so I might > have made a mistake, but this scenario should certainly be tested before > something like this is uploaded to unstable. Maybe an upload to experimental > might allow people to test this more easily? > > I Cced the apt maintainers to see if they have other suggestions to get > apt/dpkg to unpack libcrypt1 before libc6. Another (ugly) suggestions we discussed at some point was: 1) in the preinst copy the existing libcrypt1 library to a private and add it to ldconfig with lower priority than /usr/lib/$(MULTIARCH) 2) in the postinst remove these copy and ldconfig configuration. > I wonder if all this might be caused by the breaks
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Hi, On Fri, Nov 20, 2020 at 01:44:50PM +0100, Alois Wohlschlager wrote: > Am Freitag, den 20.11.2020, 09:13 + schrieb Niko Tyni: > > I don't think this is related to the recent perl 5.30 -> 5.32 > > transition. > > The report is about a buster -> bullseye upgrade, and perl in > > bullseye > > was still at 5.30 at the time. > > > > In any case, the report says perl (presumably perl-base as well) was > > still at the buster version when the breakage happened, so it didn't > > have the libcrypt1 dependency. > > This is indeed the case. perl-base was only upgraded to the bullseye > version after I manually fixed the breakage. (Indeed, when I wrote > "Perl" I actually meant "perl-base", but perl itself was also at the > buster version FWIW.) > > > FWIW the breakage can be reproduced just by manually unpacking the > > new > > libc6 on a buster system. > > > > I don't know why it hasn't been encountered earlier. > > I found out by now that the problem actually has been encountered > earlier(#946599, where it broke libc's maintainer scripts). In my case, > it broke the check-support-status hook providd by debian-security- > support. (I'm not sure whether it's such a great idea for dpkg to run > hooks when dependencies are broken, but that's what was happening on my > system.) I filed #984539 against debian-security-support, to make sure the hook never fails. That doesn't fix this bug, but at least it might reduce the potential impact of issues like these. > > Yeah, I missed the libcrypt1 -> libc6 dependency. That prevents this > > option. (I wonder if the circular dependency is a factor in apt > > choosing > > the upgrade order that results in this breakage.) > > I'm not sure what's going on here, as libcrypt1's libc6 Depends should > be satisfied by the buster version. So it seems to me like installing > libcrypt1 before upgrading libc6 should be strictly better than doing > it the other way round, even purely in terms of satisfaction of > dependencies. > > Maybe it's worth investigating why apt decides to proceed the "wrong" > way anyway, should I try setting up a VM to reproduce the problem? I did some upgrade tests starting from the debian-10-generic-amd64-20210208-542.qcow2 cloud image, and in all my tests, apt chooses to unpack libc6 before libcrypt1. However, in all of my tests, between the unpack of the new libc6 and libcrypt1 only other unpacks where done, and no dpkg hooks where run. If you have a way to reproduce the upgrade where dpkg ran the hook in between, that might be interesting. Do you still have a list of packages that was installed before you started the upgrade? Note that even if the hook isn't run or doesn't fail, there is still a period between the unpack of libc6 and libcrypt1 where perl is broken. If other operations are done in between that cause maintainer scripts to run, these could potentially call perl, which will be broken in this case. > > > > Another option might be to have the new libc6 Conflict with old > > > > versions > > > > of Essential:yes packages that need libcrypt1, forcing those > > > > Essential:yes > > > > packages to get upgraded first. A quick check based on libcrypt1 > > > > reverse > > > > dependencies in sid shows perl-base, login and util-linux. I'm > > > > not sure > > > > if this list is exhaustive, though. > > Having libc6 Conflict with one of those packages should be enough, > right? I tried creating libc6 packages with either a conflits or a breaks agains the old perl-base or the old login, and in each case I get this error: E: This installation run will require temporarily removing the essential package libc6:amd64 due to a Conflicts/Pre-Depends loop. This is often bad, but if you really want to do it, activate the APT::Force-LoopBreak option. E: Internal Error, Could not early remove libc6:amd64 (2) So it doesn't look like this will work. Note that I manually changed the binaries and didn't rebuild glibc, so I might have made a mistake, but this scenario should certainly be tested before something like this is uploaded to unstable. Maybe an upload to experimental might allow people to test this more easily? I Cced the apt maintainers to see if they have other suggestions to get apt/dpkg to unpack libcrypt1 before libc6. I wonder if all this might be caused by the breaks from libcrypt1 (against libc6 (<< 2.29-4)). Is there a way to remove the breaks without causing issues? Maybe this is somewhat similar to the situation in https://bugs.debian.org/972936#24: libgcc-s1 takes over binaries from libgcc1, but only 'replaces' it, without breaks (note that extra steps had to be taken to avoid further issues (adding Important: yes and Protected: yes)). Cheers, Ivo
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Hi, On Fri, 20 Nov 2020 13:44:50 +0100 Alois Wohlschlager wrote: > > > > Another option might be to have the new libc6 Conflict with old > > > > versions > > > > of Essential:yes packages that need libcrypt1, forcing those > > > > Essential:yes > > > > packages to get upgraded first. A quick check based on libcrypt1 > > > > reverse > > > > dependencies in sid shows perl-base, login and util-linux. I'm > > > > not sure > > > > if this list is exhaustive, though. > > Having libc6 Conflict with one of those packages should be enough, > right? Shouldn't this bug be reassigned to libc6? Paul OpenPGP_signature Description: OpenPGP digital signature
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Am Freitag, den 20.11.2020, 09:13 + schrieb Niko Tyni: > I don't think this is related to the recent perl 5.30 -> 5.32 > transition. > The report is about a buster -> bullseye upgrade, and perl in > bullseye > was still at 5.30 at the time. > > In any case, the report says perl (presumably perl-base as well) was > still at the buster version when the breakage happened, so it didn't > have the libcrypt1 dependency. This is indeed the case. perl-base was only upgraded to the bullseye version after I manually fixed the breakage. (Indeed, when I wrote "Perl" I actually meant "perl-base", but perl itself was also at the buster version FWIW.) > FWIW the breakage can be reproduced just by manually unpacking the > new > libc6 on a buster system. > > I don't know why it hasn't been encountered earlier. I found out by now that the problem actually has been encountered earlier(#946599, where it broke libc's maintainer scripts). In my case, it broke the check-support-status hook providd by debian-security- support. (I'm not sure whether it's such a great idea for dpkg to run hooks when dependencies are broken, but that's what was happening on my system.) > > > Yeah, I missed the libcrypt1 -> libc6 dependency. That prevents this > option. (I wonder if the circular dependency is a factor in apt > choosing > the upgrade order that results in this breakage.) I'm not sure what's going on here, as libcrypt1's libc6 Depends should be satisfied by the buster version. So it seems to me like installing libcrypt1 before upgrading libc6 should be strictly better than doing it the other way round, even purely in terms of satisfaction of dependencies. Maybe it's worth investigating why apt decides to proceed the "wrong" way anyway, should I try setting up a VM to reproduce the problem? > > > Another option might be to have the new libc6 Conflict with old > > > versions > > > of Essential:yes packages that need libcrypt1, forcing those > > > Essential:yes > > > packages to get upgraded first. A quick check based on libcrypt1 > > > reverse > > > dependencies in sid shows perl-base, login and util-linux. I'm > > > not sure > > > if this list is exhaustive, though. Having libc6 Conflict with one of those packages should be enough, right? -- Alois signature.asc Description: This is a digitally signed message part
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Fri, Nov 20, 2020 at 09:30:23AM +0100, Aurelien Jarno wrote: > On 2020-11-16 16:39, Niko Tyni wrote: > > On Fri, Nov 13, 2020 at 08:48:19PM +0100, Sven Joachim wrote: > > > On 2020-11-13 18:23 +0100, Niels Thykier wrote: > > > > > > > Control: reassign -1 perl-base > > > > Control: affects -1 upgrade-reports > > > > Control: severity -1 grave > > > > > > > > Hi Perl team, > > > > > > > > I have reassigned this bug to perl because perl-base being essential > > > > must remain functional during an upgrade and AFAICT perl-base fails in > > > > this case here. > > > > > > > > If it is a direct linkage, then you might be needing a Pre-Depends. If > > > > it is an indirect linkage then I am not sure how to fix it. :-/ > > > > > > I don't think perl-base is doing anything wrong here, it already uses > > > Pre-Depends. AFAICS the problem is that libcrypt.so.1 can temporarily > > > go missing if libc6 2.31-4 is unpacked before libcrypt1, breaking the > > > assumption that binaries from essential packages are always usable. > > I don't understand what happened there. Sure there has been the perl > transition, but the fact that perl depends on libcrypt1, it is the case > for more than 6 months. I don't think this is related to the recent perl 5.30 -> 5.32 transition. The report is about a buster -> bullseye upgrade, and perl in bullseye was still at 5.30 at the time. In any case, the report says perl (presumably perl-base as well) was still at the buster version when the breakage happened, so it didn't have the libcrypt1 dependency. FWIW the breakage can be reproduced just by manually unpacking the new libc6 on a buster system. I don't know why it hasn't been encountered earlier. > > As to the fix, I suspect we need a pre-dependency from libc6 to libcrypt1 > > for one release cycle, so that libc6 cannot be unpacked before libcrypt1 > > is fully installed. > > This has been tried, that works for upgrade, but then apt refuses to > allow new installation of libc6+libxcrypt1, which makes it impossible to > install foreign-arch versions on an existing system. Yeah, I missed the libcrypt1 -> libc6 dependency. That prevents this option. (I wonder if the circular dependency is a factor in apt choosing the upgrade order that results in this breakage.) > > Another option might be to have the new libc6 Conflict with old versions > > of Essential:yes packages that need libcrypt1, forcing those Essential:yes > > packages to get upgraded first. A quick check based on libcrypt1 reverse > > dependencies in sid shows perl-base, login and util-linux. I'm not sure > > if this list is exhaustive, though. > > This is something we can try indeed. It looks like perl-base 5.30.0-10 was the first version to acquire the pre-dependency on libcrypt1. -- Niko
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Hi, On 2020-11-16 16:39, Niko Tyni wrote: > On Fri, Nov 13, 2020 at 08:48:19PM +0100, Sven Joachim wrote: > > On 2020-11-13 18:23 +0100, Niels Thykier wrote: > > > > > Control: reassign -1 perl-base > > > Control: affects -1 upgrade-reports > > > Control: severity -1 grave > > > > > > Hi Perl team, > > > > > > I have reassigned this bug to perl because perl-base being essential > > > must remain functional during an upgrade and AFAICT perl-base fails in > > > this case here. > > > > > > If it is a direct linkage, then you might be needing a Pre-Depends. If > > > it is an indirect linkage then I am not sure how to fix it. :-/ > > > > I don't think perl-base is doing anything wrong here, it already uses > > Pre-Depends. AFAICS the problem is that libcrypt.so.1 can temporarily > > go missing if libc6 2.31-4 is unpacked before libcrypt1, breaking the > > assumption that binaries from essential packages are always usable. I don't understand what happened there. Sure there has been the perl transition, but the fact that perl depends on libcrypt1, it is the case for more than 6 months. > Indeed. As perl-base isn't upgraded yet at that point, there's nothing > we can do on that side. > > Apparently the new libc6 is still considered to satisfy the old perl-base > pre-dependency even when it (libc6) is only unpacked and its dependencies > are not yet satisfied. This seems similar to this paragraph from Debian > policy, section 7.2: > > When a package declaring a pre-dependency is about to be unpacked > the pre-dependency can be satisfied if the depended-on package > is either fully configured, or even if the depended-on package(s) > are only in the “Unpacked” or the “Half-Configured” state, > provided that they have been configured correctly at some point > in the past (and not removed or partially removed since). In this > case, both the previously-configured and currently “Unpacked” > or “Half-Configured” versions must satisfy any version clause > in the Pre-Depends field. > > The libc6 package has been configured correctly in the past, when > it still contained libcrypt1. > > As to the fix, I suspect we need a pre-dependency from libc6 to libcrypt1 > for one release cycle, so that libc6 cannot be unpacked before libcrypt1 > is fully installed. This has been tried, that works for upgrade, but then apt refuses to allow new installation of libc6+libxcrypt1, which makes it impossible to install foreign-arch versions on an existing system. > Another option might be to have the new libc6 Conflict with old versions > of Essential:yes packages that need libcrypt1, forcing those Essential:yes > packages to get upgraded first. A quick check based on libcrypt1 reverse > dependencies in sid shows perl-base, login and util-linux. I'm not sure > if this list is exhaustive, though. This is something we can try indeed. Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Nov 19, Sven Joachim wrote: > I am not one of them, but AFAICS that would introduce a fatal circular > dependency between libc6 and libcrypt1: libc6 needs libcrypt1 to be > configured before it can be unpacked, but libcrypt1 depends on libc6 so > it cannot be configured before libc6 is at least unpacked. Good point, you are right. :-( Then we are up to plan B, which is a bit more complex but should still be approachable: Another option might be to have the new libc6 Conflict with old versions of Essential:yes packages that need libcrypt1, forcing those Essential:yes packages to get upgraded first. A quick check based on libcrypt1 reverse dependencies in sid shows perl-base, login and util-linux. I'm not sure if this list is exhaustive, though. -- ciao, Marco signature.asc Description: PGP signature
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On 2020-11-19 19:47 +0100, Marco d'Itri wrote: > On Nov 16, Niko Tyni wrote: > >> As to the fix, I suspect we need a pre-dependency from libc6 to libcrypt1 >> for one release cycle, so that libc6 cannot be unpacked before libcrypt1 >> is fully installed. > I think that Niko is right, so I would like to know the opinion of the > glibc maintainers. I am not one of them, but AFAICS that would introduce a fatal circular dependency between libc6 and libcrypt1: libc6 needs libcrypt1 to be configured before it can be unpacked, but libcrypt1 depends on libc6 so it cannot be configured before libc6 is at least unpacked. Cheers, Sven
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Nov 16, Niko Tyni wrote: > As to the fix, I suspect we need a pre-dependency from libc6 to libcrypt1 > for one release cycle, so that libc6 cannot be unpacked before libcrypt1 > is fully installed. I think that Niko is right, so I would like to know the opinion of the glibc maintainers. -- ciao, Marco signature.asc Description: PGP signature
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Fri, Nov 13, 2020 at 08:48:19PM +0100, Sven Joachim wrote: > On 2020-11-13 18:23 +0100, Niels Thykier wrote: > > > Control: reassign -1 perl-base > > Control: affects -1 upgrade-reports > > Control: severity -1 grave > > > > Hi Perl team, > > > > I have reassigned this bug to perl because perl-base being essential > > must remain functional during an upgrade and AFAICT perl-base fails in > > this case here. > > > > If it is a direct linkage, then you might be needing a Pre-Depends. If > > it is an indirect linkage then I am not sure how to fix it. :-/ > > I don't think perl-base is doing anything wrong here, it already uses > Pre-Depends. AFAICS the problem is that libcrypt.so.1 can temporarily > go missing if libc6 2.31-4 is unpacked before libcrypt1, breaking the > assumption that binaries from essential packages are always usable. Indeed. As perl-base isn't upgraded yet at that point, there's nothing we can do on that side. Apparently the new libc6 is still considered to satisfy the old perl-base pre-dependency even when it (libc6) is only unpacked and its dependencies are not yet satisfied. This seems similar to this paragraph from Debian policy, section 7.2: When a package declaring a pre-dependency is about to be unpacked the pre-dependency can be satisfied if the depended-on package is either fully configured, or even if the depended-on package(s) are only in the “Unpacked” or the “Half-Configured” state, provided that they have been configured correctly at some point in the past (and not removed or partially removed since). In this case, both the previously-configured and currently “Unpacked” or “Half-Configured” versions must satisfy any version clause in the Pre-Depends field. The libc6 package has been configured correctly in the past, when it still contained libcrypt1. As to the fix, I suspect we need a pre-dependency from libc6 to libcrypt1 for one release cycle, so that libc6 cannot be unpacked before libcrypt1 is fully installed. Another option might be to have the new libc6 Conflict with old versions of Essential:yes packages that need libcrypt1, forcing those Essential:yes packages to get upgraded first. A quick check based on libcrypt1 reverse dependencies in sid shows perl-base, login and util-linux. I'm not sure if this list is exhaustive, though. The first option looks less intrusive to me. Disclaimer: I haven't tested any of this :) -- Niko Tyni nt...@debian.org
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Sat, Nov 14, 2020 at 03:01:10PM +0100, Marco d'Itri wrote: > On Nov 14, Dominic Hargreaves wrote: > > > This seems to be same as #953562 which was reported in March. > Why do you think that this is the same? The symptoms seem identical, at least. Maybe there is more than one root cause that I'm not aware of - feel free to unmerge if you don't think the new problem is caused by Replaces not working. Thanks Dominic
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On Nov 14, Dominic Hargreaves wrote: > This seems to be same as #953562 which was reported in March. Why do you think that this is the same? -- ciao, Marco signature.asc Description: PGP signature
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
Control: reassign -1 libcrypt1 Control: forcemerge -1 953562 On Fri, Nov 13, 2020 at 08:48:19PM +0100, Sven Joachim wrote: > On 2020-11-13 18:23 +0100, Niels Thykier wrote: > > > Control: reassign -1 perl-base > > Control: affects -1 upgrade-reports > > Control: severity -1 grave > > > > Hi Perl team, > > > > I have reassigned this bug to perl because perl-base being essential > > must remain functional during an upgrade and AFAICT perl-base fails in > > this case here. > > > > If it is a direct linkage, then you might be needing a Pre-Depends. If > > it is an indirect linkage then I am not sure how to fix it. :-/ > > I don't think perl-base is doing anything wrong here, it already uses > Pre-Depends. AFAICS the problem is that libcrypt.so.1 can temporarily > go missing if libc6 2.31-4 is unpacked before libcrypt1, breaking the > assumption that binaries from essential packages are always usable. > > I don't have a good idea how to fix that, though. :-( Right - perl-base 5.32.0-5 has the following: Pre-Depends: libc6 (>= 2.29), libcrypt1 (>= 1:4.1.0) I don't think we can do anything about this on the perl side, so reassigning to libcrypt. This seems to be same as #953562 which was reported in March.
Re: Bug#974552: upgrade-reports: libc6/libcrypt split breaks perl during buster->bullseye upgrade
On 2020-11-13 18:23 +0100, Niels Thykier wrote: > Control: reassign -1 perl-base > Control: affects -1 upgrade-reports > Control: severity -1 grave > > Hi Perl team, > > I have reassigned this bug to perl because perl-base being essential > must remain functional during an upgrade and AFAICT perl-base fails in > this case here. > > If it is a direct linkage, then you might be needing a Pre-Depends. If > it is an indirect linkage then I am not sure how to fix it. :-/ I don't think perl-base is doing anything wrong here, it already uses Pre-Depends. AFAICS the problem is that libcrypt.so.1 can temporarily go missing if libc6 2.31-4 is unpacked before libcrypt1, breaking the assumption that binaries from essential packages are always usable. I don't have a good idea how to fix that, though. :-( Cheers, Sven > Alois Wohlschlager: >> Package: upgrade-reports >> Severity: critical >> Justification: breaks the whole system >> X-Debbugs-Cc: alo...@gmx-topmail.de >> >> Dear Maintainer, >> >> *** Reporter, please consider answering these questions, where appropriate >> *** >> >>* What led up to the situation? >> >>Do an upgrade from buster to bullseye. >> >> >>* What exactly did you do (or not do) that was effective (or >> ineffective)? >> >>1. adjust sources.list >>2. apt upgrade >>3. apt dist-upgrade >> >>* What was the outcome of this action? >> >>apt dist-upgrade goes horribly wrong. Excerpt from the log: >> >> --- >> Entpacken von libc6:amd64 (2.31-4) über (2.28-10) ... >> Vormals nicht ausgewähltes Paket libc6:i386 wird gewählt. >> Vorbereitung zum Entpacken von .../4-libc6_2.31-4_i386.deb ... >> Entpacken von libc6:i386 (2.31-4) ... >> Vormals nicht ausgewähltes Paket libgcc-s1:i386 wird gewählt. >> Vorbereitung zum Entpacken von .../5-libgcc-s1_10.2.0-16_i386.deb ... >> Entpacken von libgcc-s1:i386 (10.2.0-16) ... >> Vormals nicht ausgewähltes Paket gcc-10-base:i386 wird gewählt. >> Vorbereitung zum Entpacken von .../6-gcc-10-base_10.2.0-16_i386.deb ... >> Entpacken von gcc-10-base:i386 (10.2.0-16) ... >> /usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot >> open >> shared object file: No such file or directory >> dpkg: Fehler: Fehler beim Ausführen des Hooks »if [ -x /usr/share/debian- >> security-support/check-support-status.hook ] ; then /usr/sh >> are/debian-security-support/check-support-status.hook ; fi«, Exitkode 32512 >> --- >> >>At this point, perl is still the version from buster, and libcrypt1 is not >> yet installed. The missing libcrypt.so.1 also completely breaks PAM, so login >> and sudo don't work any more. >>To recover from this outcome, I had to boot with "init=/bin/sh", install >> the >> libcrypt1 package with dpkg and run "apt -f install" twice. This rendered the >> system operational again and a further "apt dist-upgrade" ran through >> smoothly. >> >>* What outcome did you expect instead? >> >>libcrypt1 is installed before libcrypt.so.1 is required again, so the >> dist- >> upgrade can proceed normally. >> >> [...]
