Bug#1061248: glibc: DEP17: move most files but rtld to /usr
On Sun, Feb 04, 2024 at 09:20:09PM +0100, Helmut Grohne wrote: > Hi Aurelien and Sven, > > On Wed, Jan 24, 2024 at 09:19:12PM +0100, Aurelien Jarno wrote: > > On 2024-01-23 17:40, Helmut Grohne wrote: > > > Conflicting runtime dynamic linkers between multiarch packages > > > == > > > > > > Some architecture combinations such as s390, powerpc, hppa, m68k, > > > mipsn32, and hurd-i386 or alpha, i386, sh4, and sparc have conflicting > > > runtime dynamic loaders. In theory this violates Multi-Arch: same, but > > > there is basically nothing we can do about this and dropping Multi-Arch: > > > same from the packages would completely break any kind of multiarch > > > setup. There is little we can do here and this is also unrelated to > > > DEP17. > > > > We tried to add conflicts for those, like it's done for the multilib > > packages, but at the time the infrastructure exploded (see #745552). I > > don't remember the details, but I guess it was either dak or > > dose-builddebcheck. > > Yeah, I also remember something like that. It's not the first time I > trip into this. "There is little we can do here" still counts. > > > I guess the symlink in the maintainer script could indeed work. I am not > > sure why it has been done like that, it was part of the multiarch > > patchset from Steve Langasek more than 10 years ago. > > Practically speaking, it appears to work rather well. On the flip side, > I also thought that way of my first patch. ;) > > > Note however that the those packages are used by cross-toolchain-base > > (which builds them from glibc-source) and mangled to install them in the > > cross path. See for instance libc6-amd64-x32-cross. For such cases, we > > probably do want to keep the dynamic linker symlink, as those packages > > do not have maintainer scripts. > > I don't think those -$arch-cross packages need the runtime dynamic > linker at all. Unlike the libc6-$multilib packages, we don't use > -$arch-cross packages to actaully run any binary. Simply not installing > it might just work in practice. > > So here is an updated patch with a few notes: > * This patch moves all the files including the runtime dynamic linker >in the main multiarch library. Therefore, this patch needs to be >synced with the corresponding base-files change to add the aliasing >symlinks or debootstrap breaks. In other words: Don't upload this >yet. > * As mentioned earlier, only the multiarch packages install the runtime >dynamic linker via data.tar. All the multilib packages install it via >maintainer scripts now. > * When installing libc6-x32, /libx32 will be created because partial >upgrades might otherwise be broken. Removing libc6-x32 will not >remove /libx32 though. I suggest fixing this in base-files by >installing a trigger interest in /usr/libx32 and having >base-files.postinst create/remove /libx32 as needed. This way, we >centralize the management of aliasing links into base-files. libx32 >only serves as an example here and it works the same way for any >other non-essential multilib directory. Do you agree with the >approach? > * The multilib packages install a trigger interest on the runtime >dynamic linker such that they notice when a multiarch package deletes >it and can then recreate it as needed. Thus the multiarch packages do >not have to pay attention to a possibly installed multilib package >when they are removed. > * I've tried quite a few multiarch upgrades involving amd64 and i386 >using dpkg --auto-deconfigure --unpack with various packages and even >cross grading libc6-x32 from :i386 to :amd64 during the upgrade. >While dpkg --verify was occasionally unhappy during a partial upgrade >(where half-unpacked packages were around), once no package was >half-installed, dpkg --verify was also happy again in all attempts. I >did not come up with a systematic enumeration of possible upgrade >scenarios though. > * The patch works will deboostrap/cdebootstrap/mmdebstrap (in >combination with more patches including base-files, bash, dash and >util-linux). > * The change to install ldconfig to /usr/sbin can be uploaded right >away. It's limited to debian/debhelper.in/libc-bin.install and >debian/debhelper.in/libc-bin.lintian-overrides and doesn't contribute >much diff, so doing it later is fine as well. > > I hope you don't manage to punch holes into my theory this time around. Ubuntu testing revealed that due to /lib64 going away in the package we also are going to need Depends or PreDepends on base-files, such that base-files trigger can then create the symlink. Or we temporarily handle the trigger ourselves or so. I am not sure if Depends are enough, I guess if you do a release upgrade we could end up with unpack base-files unpack libc6 preinst for something we want to unpack -> poof, no /lib64 here, can't
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Hi Helmut, I finally got time to look at your patch and do very basic testing of it. Overall it looks good, I have a few points or questions. On 2024-02-04 21:20, Helmut Grohne wrote: > So here is an updated patch with a few notes: > * This patch moves all the files including the runtime dynamic linker >in the main multiarch library. Therefore, this patch needs to be >synced with the corresponding base-files change to add the aliasing >symlinks or debootstrap breaks. In other words: Don't upload this >yet. Ok. > * As mentioned earlier, only the multiarch packages install the runtime >dynamic linker via data.tar. All the multilib packages install it via >maintainer scripts now. > * When installing libc6-x32, /libx32 will be created because partial >upgrades might otherwise be broken. Removing libc6-x32 will not >remove /libx32 though. I suggest fixing this in base-files by >installing a trigger interest in /usr/libx32 and having >base-files.postinst create/remove /libx32 as needed. This way, we >centralize the management of aliasing links into base-files. libx32 >only serves as an example here and it works the same way for any >other non-essential multilib directory. Do you agree with the >approach? It sounds good yes. I never liked the fact the fact that the top level symlinks like libx32 have to be handled by libc6. I explained that clearly in #926699, and even suggested to do that in base-files, however I didn't get the clever idea to use triggers. I got pressure from a member of the TC to be constructive and given I didn't have a patch to provide, I had to accept getting it managed by glibc... > * The multilib packages install a trigger interest on the runtime >dynamic linker such that they notice when a multiarch package deletes >it and can then recreate it as needed. Thus the multiarch packages do >not have to pay attention to a possibly installed multilib package >when they are removed. Does it means we can just remove the Replaces: in the multiarch and multilib libc? It should not be necessary anymore, even if without file conflicts, they should not be an issue. However not sure what could happen during the upgrade between the old and new packages. > * I've tried quite a few multiarch upgrades involving amd64 and i386 >using dpkg --auto-deconfigure --unpack with various packages and even >cross grading libc6-x32 from :i386 to :amd64 during the upgrade. >While dpkg --verify was occasionally unhappy during a partial upgrade >(where half-unpacked packages were around), once no package was >half-installed, dpkg --verify was also happy again in all attempts. I >did not come up with a systematic enumeration of possible upgrade >scenarios though. Great. > * The patch works will deboostrap/cdebootstrap/mmdebstrap (in >combination with more patches including base-files, bash, dash and >util-linux). Ok > * The change to install ldconfig to /usr/sbin can be uploaded right >away. It's limited to debian/debhelper.in/libc-bin.install and >debian/debhelper.in/libc-bin.lintian-overrides and doesn't contribute >much diff, so doing it later is fine as well. Ok noted. The idea was to get the 2.38 into testing, but the glibc transition is currently blocked by the time_t transition, so the development is currently stalled, and I am not sure when we'll do an upload. I also noticed that you clearly marked the code that can be removed only after "released:forky", thanks for doing so. Do you really mean after forky is released, so in the development cycle of forky+1? Or do you mean for the release of forky, so in the development cycle of forky? Cheers Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://aurel32.net
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Hi Helmut, On 2024-02-05 07:44, Helmut Grohne wrote: > Hi Aurelien, > > So this confirms your initial suspicion on the actually affected case. > Thank you. > > c-t-b has repacking code with arch-specific mangling (of slibdir) > already. > https://sources.debian.org/src/cross-toolchain-base/68/debian/rules/#L563 > Adding to that wouldn't be the worst. A relatively easy measure would be > running the libc-alt.postinst manually with DPKG_ROOT set to have it > create the symlink that way. Do you think this is too much of a hack? That's indeed an option, with the hope that we do not add more incompatible things to that file at a later point. However it's probably the best to ask Matthias as the maintainer of c-t-b. If doing that way the uploads need to be synchronized to not break c-t-b. Cheers Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://aurel32.net
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Hi Aurelien, On Sun, Feb 04, 2024 at 09:43:53PM +0100, Aurelien Jarno wrote: > No, it is actually needed. For instance using the arm64 cross-compiler > on amd64: > > $ rm /usr/aarch64-linux-gnu/lib/ld-linux-aarch64.so.1 > $ arch64-linux-gnu-gcc -o test test.c > /usr/lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/bin/ld: > cannot find /usr/aarch64-linux-gnu/lib/ld-linux-aarch64.so.1: No such file or > directory > collect2: error: ld returned 1 exit status Right. For one thing this is not exactly the case where we're after. This is the plain architecture case where the rtld link will just be there by virtue of being there in the multiarch libc package. The case where we're after is running a cross compiler with multilib. Say I install i686-linux-gnu-gcc and pass -mx32: $ i686-linux-gnu-gcc -mx32 test.c -o test $ rm /usr/i686-linux-gnu/libx32/ld-linux-x32.so.2 $ i686-linux-gnu-gcc -mx32 test.c -o test /usr/lib/gcc-cross/i686-linux-gnu/13/../../../../i686-linux-gnu/bin/ld: cannot find /usr/i686-linux-gnu/libx32/ld-linux-x32.so.2: No such file or directory collect2: error: ld returned 1 exit status $ So this confirms your initial suspicion on the actually affected case. Thank you. c-t-b has repacking code with arch-specific mangling (of slibdir) already. https://sources.debian.org/src/cross-toolchain-base/68/debian/rules/#L563 Adding to that wouldn't be the worst. A relatively easy measure would be running the libc-alt.postinst manually with DPKG_ROOT set to have it create the symlink that way. Do you think this is too much of a hack? Helmut
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Hi Helmut, Thanks I haven't checked the whole patch yet, but here is a first comment. On 2024-02-04 21:20, Helmut Grohne wrote: > I don't think those -$arch-cross packages need the runtime dynamic > linker at all. Unlike the libc6-$multilib packages, we don't use > -$arch-cross packages to actaully run any binary. Simply not installing > it might just work in practice. No, it is actually needed. For instance using the arm64 cross-compiler on amd64: $ rm /usr/aarch64-linux-gnu/lib/ld-linux-aarch64.so.1 $ arch64-linux-gnu-gcc -o test test.c /usr/lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/bin/ld: cannot find /usr/aarch64-linux-gnu/lib/ld-linux-aarch64.so.1: No such file or directory collect2: error: ld returned 1 exit status Cheers Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://aurel32.net
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Hi Aurelien and Sven, On Wed, Jan 24, 2024 at 09:19:12PM +0100, Aurelien Jarno wrote: > On 2024-01-23 17:40, Helmut Grohne wrote: > > Conflicting runtime dynamic linkers between multiarch packages > > == > > > > Some architecture combinations such as s390, powerpc, hppa, m68k, > > mipsn32, and hurd-i386 or alpha, i386, sh4, and sparc have conflicting > > runtime dynamic loaders. In theory this violates Multi-Arch: same, but > > there is basically nothing we can do about this and dropping Multi-Arch: > > same from the packages would completely break any kind of multiarch > > setup. There is little we can do here and this is also unrelated to > > DEP17. > > We tried to add conflicts for those, like it's done for the multilib > packages, but at the time the infrastructure exploded (see #745552). I > don't remember the details, but I guess it was either dak or > dose-builddebcheck. Yeah, I also remember something like that. It's not the first time I trip into this. "There is little we can do here" still counts. > I guess the symlink in the maintainer script could indeed work. I am not > sure why it has been done like that, it was part of the multiarch > patchset from Steve Langasek more than 10 years ago. Practically speaking, it appears to work rather well. On the flip side, I also thought that way of my first patch. ;) > Note however that the those packages are used by cross-toolchain-base > (which builds them from glibc-source) and mangled to install them in the > cross path. See for instance libc6-amd64-x32-cross. For such cases, we > probably do want to keep the dynamic linker symlink, as those packages > do not have maintainer scripts. I don't think those -$arch-cross packages need the runtime dynamic linker at all. Unlike the libc6-$multilib packages, we don't use -$arch-cross packages to actaully run any binary. Simply not installing it might just work in practice. So here is an updated patch with a few notes: * This patch moves all the files including the runtime dynamic linker in the main multiarch library. Therefore, this patch needs to be synced with the corresponding base-files change to add the aliasing symlinks or debootstrap breaks. In other words: Don't upload this yet. * As mentioned earlier, only the multiarch packages install the runtime dynamic linker via data.tar. All the multilib packages install it via maintainer scripts now. * When installing libc6-x32, /libx32 will be created because partial upgrades might otherwise be broken. Removing libc6-x32 will not remove /libx32 though. I suggest fixing this in base-files by installing a trigger interest in /usr/libx32 and having base-files.postinst create/remove /libx32 as needed. This way, we centralize the management of aliasing links into base-files. libx32 only serves as an example here and it works the same way for any other non-essential multilib directory. Do you agree with the approach? * The multilib packages install a trigger interest on the runtime dynamic linker such that they notice when a multiarch package deletes it and can then recreate it as needed. Thus the multiarch packages do not have to pay attention to a possibly installed multilib package when they are removed. * I've tried quite a few multiarch upgrades involving amd64 and i386 using dpkg --auto-deconfigure --unpack with various packages and even cross grading libc6-x32 from :i386 to :amd64 during the upgrade. While dpkg --verify was occasionally unhappy during a partial upgrade (where half-unpacked packages were around), once no package was half-installed, dpkg --verify was also happy again in all attempts. I did not come up with a systematic enumeration of possible upgrade scenarios though. * The patch works will deboostrap/cdebootstrap/mmdebstrap (in combination with more patches including base-files, bash, dash and util-linux). * The change to install ldconfig to /usr/sbin can be uploaded right away. It's limited to debian/debhelper.in/libc-bin.install and debian/debhelper.in/libc-bin.lintian-overrides and doesn't contribute much diff, so doing it later is fine as well. I hope you don't manage to punch holes into my theory this time around. Helmut diff --minimal -Nru glibc-2.37/debian/changelog glibc-2.37/debian/changelog --- glibc-2.37/debian/changelog 2024-01-23 07:13:18.0 +0100 +++ glibc-2.37/debian/changelog 2024-01-19 15:56:06.0 +0100 @@ -1,3 +1,10 @@ +glibc (2.37-14.1) UNRELEASED; urgency=medium + + * Non-maintainer upload. + * DEP17: Move files to /usr. (Closes: #-1) + + -- Helmut Grohne Fri, 19 Jan 2024 15:56:06 +0100 + glibc (2.37-14) unstable; urgency=medium [ Aurelien Jarno ] diff --minimal -Nru glibc-2.37/debian/debhelper.in/libc-alt.install glibc-2.37/debian/debhelper.in/libc-alt.install --- glibc-2.37/debian/debhelper.in/libc-alt.install 2022-09-22
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Hi, On 2024-01-23 17:40, Helmut Grohne wrote: > Hi, > > TL;DR: I brainstorm solutions and appreciated feedback, but no action is > required at this time. I am not sure I understood everything from that mail, but let me provide a few answers. > On Sun, Jan 21, 2024 at 10:32:29PM +0100, Helmut Grohne wrote: > > This seems pretty much unfxiable to me now. > > Unfixable was a bit too strong. With much help from Aurelien and ideas > from Enrico I looked at this more systematically. > > I first tried to understand what kind of file sharing we between glibc > packages. To that end I wrote a collect.sh (attached) that downlods all > the glibc packages and a diagnose.py (attached) that tries to scrape > relevant detail from them. This results in an output.txt (attached). > This is all quite hacky, but it gets the job done. For the purpose of > this analysis, I am assuming that files that differ in content also > differ in size (which of course is not generally correct, but simplifies > matters). That output.txt lists files (normalized to lack any /usr > prefix) and the packages shipping them (as a 3-tuple package name, > architecture, size or symlink target). Assuming that all the packages > were Multi-Arch: same, files with identical content (i.e. size) are > discarded. We're left with three kinds of file sharing: > > debhelper version on sh4 > > > Apparently sh4 has a different debhelper that emits different files in > /usr/share/doc. This breaks M-A:same, but is otherwise boring in the > DEP17 context. > > Conflicting multilibs > = > > Around 300 files in /lib64 conflict between libc6-ppc64:powerpc, > libc6-amd64:i386 and libc6-amd64:x32. Likewise, around 300 files in > /lib32 conflict between libc6-s390:s390x, libc6-sparc:sparc64, > libc6-i386:amd64 libc6-powerpc:ppc64, libc6-i386:x32 and > libc6-mipsn32:mips64el. These are declared file conflicts. > Unfortunately, we learned that Conflicts do not reliably prevent > concurrent unpacks in the presence of aliasing, so when moving these > libraries, we can construct a file loss, for example: > > * apt install libc6:mips64el libc6-i386:amd64 > * Add hypothetical apt sources with a moved glibc. > * echo libc6-i386:amd64 deinstall | dpkg --set-selections > * dpkg --auto-deconfigure --install libc6-mipsn32_usrmoved_mips64el.deb > > In essence, this will be upgrading from bookworm to trixe while > simultaneously replacing libc6-i386 with libc6-mipsn32 for some reason. > On top of that, I guess that apt would prefer removing libc6-i386 before > unpacking libc6-mipsn32 such that the loss scenario likely requires > working with dpkg directly. > > There is a relatively simple mitigation. For every file in SLIBDIR in > the trixie, libc-alt.preinst can set up a diversion for the > corresponding aliased location diverting to some non-existent filename. > libc-alt.postinst then removes these diversions. The Conflicts require > the conflicting libc-alt to actually be removed before libc-alt.postinst > is run. (Breaks is not sufficient as I learned the hard way.) These are > very temporary diversions, but they're also quite many (300). > > At the time of this writing, I have not prototyped this approach. Let me > already pose the question of how you assess the involved risks here. > Actually triggering this failure is a rather strange corner case and it > is not clear to me whether this corner case is worth risking possible > implementation bugs in the mitigation. > > Conflicting runtime dynamic linkers between multiarch packages > == > > Some architecture combinations such as s390, powerpc, hppa, m68k, > mipsn32, and hurd-i386 or alpha, i386, sh4, and sparc have conflicting > runtime dynamic loaders. In theory this violates Multi-Arch: same, but > there is basically nothing we can do about this and dropping Multi-Arch: > same from the packages would completely break any kind of multiarch > setup. There is little we can do here and this is also unrelated to > DEP17. We tried to add conflicts for those, like it's done for the multilib packages, but at the time the infrastructure exploded (see #745552). I don't remember the details, but I guess it was either dak or dose-builddebcheck. > Conflicting runtime dynamic linkers between multiarch and multilib > == > > Runtime dynamic linkers need to be installed both into libc:ARCH and > libc-ARCH:SIBLINGARCH. An example is libc6:i386 and libc6-i386:amd64 > both containing /lib/ld-linux.so.2. Both packages really need to ensure > presence of the runtime dynamic linker on installation in the exact > location so there is little we can do about this. The current situation > is that the multiarch package Replaces the multilib one such that > ownership is automatically transferred to the multiarch one when both > are installed. The multiarch postrm
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Hi, TL;DR: I brainstorm solutions and appreciated feedback, but no action is required at this time. On Sun, Jan 21, 2024 at 10:32:29PM +0100, Helmut Grohne wrote: > This seems pretty much unfxiable to me now. Unfixable was a bit too strong. With much help from Aurelien and ideas from Enrico I looked at this more systematically. I first tried to understand what kind of file sharing we between glibc packages. To that end I wrote a collect.sh (attached) that downlods all the glibc packages and a diagnose.py (attached) that tries to scrape relevant detail from them. This results in an output.txt (attached). This is all quite hacky, but it gets the job done. For the purpose of this analysis, I am assuming that files that differ in content also differ in size (which of course is not generally correct, but simplifies matters). That output.txt lists files (normalized to lack any /usr prefix) and the packages shipping them (as a 3-tuple package name, architecture, size or symlink target). Assuming that all the packages were Multi-Arch: same, files with identical content (i.e. size) are discarded. We're left with three kinds of file sharing: debhelper version on sh4 Apparently sh4 has a different debhelper that emits different files in /usr/share/doc. This breaks M-A:same, but is otherwise boring in the DEP17 context. Conflicting multilibs = Around 300 files in /lib64 conflict between libc6-ppc64:powerpc, libc6-amd64:i386 and libc6-amd64:x32. Likewise, around 300 files in /lib32 conflict between libc6-s390:s390x, libc6-sparc:sparc64, libc6-i386:amd64 libc6-powerpc:ppc64, libc6-i386:x32 and libc6-mipsn32:mips64el. These are declared file conflicts. Unfortunately, we learned that Conflicts do not reliably prevent concurrent unpacks in the presence of aliasing, so when moving these libraries, we can construct a file loss, for example: * apt install libc6:mips64el libc6-i386:amd64 * Add hypothetical apt sources with a moved glibc. * echo libc6-i386:amd64 deinstall | dpkg --set-selections * dpkg --auto-deconfigure --install libc6-mipsn32_usrmoved_mips64el.deb In essence, this will be upgrading from bookworm to trixe while simultaneously replacing libc6-i386 with libc6-mipsn32 for some reason. On top of that, I guess that apt would prefer removing libc6-i386 before unpacking libc6-mipsn32 such that the loss scenario likely requires working with dpkg directly. There is a relatively simple mitigation. For every file in SLIBDIR in the trixie, libc-alt.preinst can set up a diversion for the corresponding aliased location diverting to some non-existent filename. libc-alt.postinst then removes these diversions. The Conflicts require the conflicting libc-alt to actually be removed before libc-alt.postinst is run. (Breaks is not sufficient as I learned the hard way.) These are very temporary diversions, but they're also quite many (300). At the time of this writing, I have not prototyped this approach. Let me already pose the question of how you assess the involved risks here. Actually triggering this failure is a rather strange corner case and it is not clear to me whether this corner case is worth risking possible implementation bugs in the mitigation. Conflicting runtime dynamic linkers between multiarch packages == Some architecture combinations such as s390, powerpc, hppa, m68k, mipsn32, and hurd-i386 or alpha, i386, sh4, and sparc have conflicting runtime dynamic loaders. In theory this violates Multi-Arch: same, but there is basically nothing we can do about this and dropping Multi-Arch: same from the packages would completely break any kind of multiarch setup. There is little we can do here and this is also unrelated to DEP17. Conflicting runtime dynamic linkers between multiarch and multilib == Runtime dynamic linkers need to be installed both into libc:ARCH and libc-ARCH:SIBLINGARCH. An example is libc6:i386 and libc6-i386:amd64 both containing /lib/ld-linux.so.2. Both packages really need to ensure presence of the runtime dynamic linker on installation in the exact location so there is little we can do about this. The current situation is that the multiarch package Replaces the multilib one such that ownership is automatically transferred to the multiarch one when both are installed. The multiarch postrm also checks whether a multilib package is remaining and restores the symlink if it was stolen. This way of doing things works badly with the /usr-move. In the DEP17 document, this amounts to a P1 problem where a file is both moved from one package to another and from / to /usr. Since we actually want to permit concurrent unpack, we cannot use Conflicts (M7). We will have to employ protective diversions one way or another. Unfortunately, the /usr-move makes it difficult to safely transition the ownership of the runtime dynamic
Processed: Re: Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Processing control commands: > tags -1 - patch + moreinfo Bug #1061248 [src:glibc] glibc: DEP17: move most files but rtld to /usr Removed tag(s) patch. Bug #1061248 [src:glibc] glibc: DEP17: move most files but rtld to /usr Added tag(s) moreinfo. -- 1061248: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1061248 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
On Sun, Jan 21, 2024 at 07:39:04PM +0100, Helmut Grohne wrote: > I shall rework the patch and also exempt multilib rtlds from moving. I'm > sorry for not having realized this. I suspect that dumat would have > reported this problem. This seems pretty much unfxiable to me now. Essentially, keeping all (including multilib) rtlds aliased works badly with the matching in debian/debhelper.in/libc-alt.install. Trying to refine those patterns in a way that retains the rtlds aliased while moving everything else is quite difficult, because sometimes SLIBDIR and RTLDDIR overlap. Even if managing this, it breaks my simplistic approach to fixing symlinks. When base-files gains the aliasing links, we can move both files from SLIBDIR and RTLDDIR restoring this simplicity. The value we gain from moving files in SLIBDIR now already seems to diminish given the complexity it would incur. If we disregard SLIBDIR moves, not much is left in my patch. Essentially all we can move is ldconfig and moving that now or later doesn't pose a significant difference. So it seems, that the proposed change is not very useful in the end. I've learned a few things about how to do the coordinated NMU move and that's about it, it seems. Sorry for the noise. Is it ok, to repurpose and leave open this bug open for the later upload moving all of the files or do you prefer a new bug then? Helmut
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Control: tags -1 - patch + moreinfo On Sun, Jan 21, 2024 at 06:37:35PM +0100, Sven Joachim wrote: > I have not studied the details, but this looks rather dangerous to me. > If you install the runtime dynamic linker in multilib packages below > /usr, but keep the native one at its current place, you risk losing it > when the multilib packages are removed. Thank you for reviewing my patch. > For instance, I have both libc6:i386 and libc6-i386:amd64 installed. If > the latter starts shipping /usr/lib/ld-linux.so.2 rather than > /lib/ld-linux.so.2, the "Replaces" in libc6:i386 becomes ineffective, > and we have basically a case of Dep17 P1. I shall rework the patch and also exempt multilib rtlds from moving. I'm sorry for not having realized this. I suspect that dumat would have reported this problem. Helmut
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Am 21.01.2024 um 15:25 schrieb Helmut Grohne: > Source: glibc > Version: 2.37-13 > Tags: patch > User: helm...@debian.org > Usertags: dep17m2 > > Hi Aurelien, > > thanks for your answers on IRC to my design question. As promised here > comes a patch that moves most files in binary packages built from glibc > from aliased locations to /usr. This excludes the runtime dynamic linker > for native libc packages (i.e. not multilib), because moving it would > break filesystem bootstrap unless base-files installs the aliasing > symlinks at the same time. I have not studied the details, but this looks rather dangerous to me. If you install the runtime dynamic linker in multilib packages below /usr, but keep the native one at its current place, you risk losing it when the multilib packages are removed. For instance, I have both libc6:i386 and libc6-i386:amd64 installed. If the latter starts shipping /usr/lib/ld-linux.so.2 rather than /lib/ld-linux.so.2, the "Replaces" in libc6:i386 becomes ineffective, and we have basically a case of Dep17 P1. True, there is already a file loss problem today. If I were to remove libc6:i386 now, I would be left without /lib/ld-linux.so.2 as well. But in such a situation it is always possible to remedy the situation by reinstalling libc6-i386. This is not ensured if only libc6-i386 is removed, as essential programs might depend on libc6:i386, leaving no easy way of recovery. Cheers, Sven
Bug#1061248: glibc: DEP17: move most files but rtld to /usr
Source: glibc Version: 2.37-13 Tags: patch User: helm...@debian.org Usertags: dep17m2 Hi Aurelien, thanks for your answers on IRC to my design question. As promised here comes a patch that moves most files in binary packages built from glibc from aliased locations to /usr. This excludes the runtime dynamic linker for native libc packages (i.e. not multilib), because moving it would break filesystem bootstrap unless base-files installs the aliasing symlinks at the same time. That move is a later step and is what I asked for in https://lists.debian.org/20230912181509.ga2588...@subdivi.de. What I'm asking for here is staging the changes to glibc in two phases where the majority of the move happens before that coordinated upload. This patch is that majority move. Regarding the implementation, I asked whether you'd prefer to change slibdir or not and you answered that you'd rather not change it as e.g. Fedora is not changing it either. My first implementation changed slibdir and this second iteration that does not change slibdir is quite a bit simpler. I also asked about how to deal with symbolic links that point at aliased locations. Your answer felt a little inconclusive to me, but fixing them practically is a requirement: During filesystem bootstrap, libc6 needs to briefly operate in an unmerged state (temporarily until that coordinated move) and therefore those symlinks in libc6 cannot rely on the aliasing symlinks having been set up. Hence, I added code for fixing those links before letting dh_link perform their canonicalization according to Debian policy. In practice, this turns most of the links (but runtime dynamic linker links) into relative ones. We may be able to drop this after the coordinated move if you disagree about the approach taken here, but I think it is best to accept this temporarily at least. Are you also comfortable with keeping this link fixing permanently? The change at hand requires significant testing, because there is a significant risk of breaking stuff and doing so is very annoying to many developers. I've performed the following steps: * Reviewed the file lists of created .debs to see that all files but runtime dynamic linkers have moved out of aliased locations. * Reviewed all symbolic links in created .debs. * Ran piuparts. It complained about /lib32 and /libx32 not being cleaned up after removal of multilib packages. I think this is vaguely fine. Do you agree? If not, I can add postrm code that checks whether /usr/lib32 and /usr/libx32 vanished and removes the aliasing links in those cases. * I set up a custom reprepro repository with these packages and ran a number of filesystem bootstraps: * debootstrap * debootstrap --variant=minbase * cdebootstrap --flavour=standard * cdebootstrap --flavour=minimal * mmdebstrap --variant=debootstrap * mmdebstrap --variant=minbase * mmdebstrap --variant=apt All of these bootstraps do not contain any glibc-owned files in aliased locations with the exception of the runtime dynamic linker. * I compiled a minimal C program in chroot both with -m32 and without and verified that the embedded location of the runtime dynamic linker still is aliased and that the resulting program still runs. * In addition to testing on amd64, I performed a i386 build. I note that my builds are nocheck builds, due to failing tests unrelated to my changes. I did not bother figuring out what local configuration causes those the test failures. Do you miss any testing here? Helmut diff --minimal -Nru glibc-2.37/debian/changelog glibc-2.37/debian/changelog --- glibc-2.37/debian/changelog 2023-12-03 14:23:52.0 +0100 +++ glibc-2.37/debian/changelog 2024-01-19 15:56:06.0 +0100 @@ -1,3 +1,10 @@ +glibc (2.37-13.1) UNRELEASED; urgency=medium + + * Non-maintainer upload. + * DEP17: Move most files but rtld to /usr. (Closes: #-1) + + -- Helmut Grohne Fri, 19 Jan 2024 15:56:06 +0100 + glibc (2.37-13) unstable; urgency=medium [ Aurelien Jarno ] diff --minimal -Nru glibc-2.37/debian/debhelper.in/libc-alt.install glibc-2.37/debian/debhelper.in/libc-alt.install --- glibc-2.37/debian/debhelper.in/libc-alt.install 2022-09-22 22:06:02.0 +0200 +++ glibc-2.37/debian/debhelper.in/libc-alt.install 2024-01-19 15:56:06.0 +0100 @@ -1,6 +1,6 @@ # This file is used for biarch libraries. -TMPDIR/RTLDDIR/*.so* RTLDDIR -TMPDIR/SLIBDIR/*.so* SLIBDIR +TMPDIR/RTLDDIR/*.so* usr/RTLDDIR +TMPDIR/SLIBDIR/*.so* usr/SLIBDIR TMPDIR/LIBDIR/gconv/* LIBDIR/gconv/ TMPDIR/etc/ld.so.conf.d /etc diff --minimal -Nru glibc-2.37/debian/debhelper.in/libc-alt.postrm glibc-2.37/debian/debhelper.in/libc-alt.postrm --- glibc-2.37/debian/debhelper.in/libc-alt.postrm 2023-10-03 21:07:14.0 +0200 +++ glibc-2.37/debian/debhelper.in/libc-alt.postrm 2024-01-19 15:56:06.0 +0100 @@ -7,8 +7,8 @@ # multiarch package are installed, then the multiarch package is
