Processed: Re: Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Processing control commands:
> severity -1 normal
Bug #926699 [debootstrap,usrmerge] libc6 foreign/biarch: installing, removing,
reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Severity set to 'normal' from 'serious'
--
926699: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926699
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Control: severity -1 normal On Aug 22, Marco d'Itri wrote: > Another option is agreeing that this cannot be fixed in a sane and > practical way until non-merged systems have to be supported, and > document somewhere that if anybody does this install-remove-reinstall > dance on a libc-* package then they will have to run again the > conversion program. I am lowering the severity of this bug since it does not appear that it will be fixed soon and that just running the conversion again will restore the correct directories layout. -- ciao, Marco signature.asc Description: PGP signature
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
On Aug 19, Aurelien Jarno wrote:
Thank you for expressing your position in more detail.
> usrmerge works by moving all data from /lib into /usr/lib and then
> creating a symlink /lib -> /usr/lib. The same is done for the biarch
> or triarch directories, namely /lib32, /lib64, /libx32 and /libo32
> depending on the architecture. Note that this part is done by usrmerge,
> but doesn't appear in its description, nor in the debconf question
> asked to the user. It is important to note that it is done directly on
> the filesystem and that this change is not known to dpkg.
I do not think that describing this implementation detail would be
generally useful to users, but you are right. This complexity is an
effect of having to keep supporting non-merged systems, but I am not
willing to fight that battle right now.
> There is nothing special done with that regard in the glibc maintainer
> scripts.
Agreed.
> The usrmerge people considers that it has to be fixed in the glibc
> maintainer scripts.
I am not dead set on this, but so far it seems to me that, as long as we
need to keep supporting non-merged systems, such a solution would both
solve all issues and be the simplest one to implement, as long as it is
correctly defined in scope.
> As explained above, it would mean that the preinst
> script is responsible, if usrmerge is activated, to create the
> /lib{32,64,o32,x32} -> /usr/lib{32,64,o32,x32} symlink if it doesn't
> exist. The directory might not be empty as a result of a partial
> conversion to usrmerge (yes those system will exists). In turn this
An half-converted system is (more or less) broken and it is not libc's
responsability to fix it. Pseudocode:
# is this a merged-/usr system?
if [ -L /lib ]; then
# has the link already been created?
if [ -L /lib32/ ]; then
:
# if it has not, is a directory already there?
elif [ ! -d /lib32/ ]; then
# if not, then create it
ln -s /usr/lib32/ /lib32/
# is this needed or not?
[ -d /usr/lib32/ ] || mkdir /usr/lib32/
fi
fi
Implementing this would cover all non-pathological cases and improve on
the current situation.
> directory might be the one of the dynamic linker used for executing
> the maintainer scripts (dash/bash, coreutils). So this means the
> maintainer scripts have to handle the case of moving the dynamic
> linker without breaking the system.
No, it would never attempt to move any file: either e.g. /lib32/ does
not exist, and then libc6-* would create the link, or it does, and that
system is half-converted but it's not libc's fault, and continuing to
install some of the files in / would not break anything.
> In short it means *the most tricky part of usrmerge has to be
> implemented and maintained in the glibc preinst scripts*. This is even
I disagree: only the most basic function would need to be implemented by
the libc packages: creating a symlink if one is needed and it does not
already exist.
I would definitely never propose to move to other packages the migration
support currently implemented in usrmerge.
> more difficult as the preinst script will be executed all the time and
> not when the user ask for that. It won't ask the user for its permission
> before. It doesn't have the option to abort if something looks wrong as
> this will lead to a system difficultly recoverable for many users if
> they are dist-upgrading their system to a new release (usually apt-get
> install -f is not enough).
As explained, I think that this code can always be successful.
> Therefore I really believe the best solution for this issue is to
> make dpkg aware of the symlink, which means creating a package
> containing this symlinks. This could be for example:
> - in base-files, for example by depending on base-files-usrmerge |
> base-files-nousrmerge
I wonder what the base-files and deboostrap maintainers think about
this, since it would require some changes in their packages (non-trivial
ones in deboostrap, I think).
This also has the downside that this way it would be impossible to
delete the /libx32/ etc links when they are not needed, since they would
be created every time the package is updated.
> - in usrmerge after reconsidering that the package can be uninstalled
> after the conversion.
I am not sure that this could actually work: if the converter package is
designed to be installed on a non-merged system to convert it, what
would happen when it tried to install a /lib/ -> /usr/lib/ link in such
a non-merged system, where /lib/ already is a directory?
(It would also have all the downsides of the precedent point and
moreover require installing three extra Perl modules on every Debian
system.)
But let's assume that there would be also a base-files-usrmerge package
just to provide the links, to be installed later: how could this be
automated by the converter package, since dpkg does not support being
invoked by itself?
> - directly in the libc6 and libc6-xxx packages if the TC reconsiders the
>
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Aurelien Jarno writes:
> control: retitle 926699 libc6 foreign/biarch: installing, removing,
> reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
> control: affects 926699 libc6
>
> Hi,
>
> I have been asked privately to work constructively on a solution,
> otherwise my bad behaviour will likely lead to a decision that I will
> regret if this discussion leads towards an appeal to the TC.
I'm very sorry you took what I wrote in that way, so please accept my
apologies for my failure to express myself sufficiently clearly.
Regardless of that, thank you very much for setting out what your view
of the situation is.
I hope that this will enable a constructive discussion to continue,
hopefully leading to a solution that all find acceptable.
Cheers, Phil.
--
|)| Philip Hands [+44 (0)20 8530 9560] HANDS.COM Ltd.
|-| http://www.hands.com/http://ftp.uk.debian.org/
|(| Hugo-Klemm-Strasse 34, 21075 Hamburg,GERMANY
signature.asc
Description: PGP signature
Processed: Re: Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Processing control commands:
> retitle 926699 libc6 foreign/biarch: installing, removing, reinstalling in a
> --merged-usr system results in unmerged /lib{32,x32}
Bug #926699 [debootstrap,usrmerge] libc6-{i386,x32}: installing, removing,
reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Changed Bug title to 'libc6 foreign/biarch: installing, removing, reinstalling
in a --merged-usr system results in unmerged /lib{32,x32}' from
'libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system
results in unmerged /lib{32,x32}'.
> affects 926699 libc6
Bug #926699 [debootstrap,usrmerge] libc6 foreign/biarch: installing, removing,
reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Added indication that 926699 affects libc6
--
926699: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926699
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
control: retitle 926699 libc6 foreign/biarch: installing, removing,
reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
control: affects 926699 libc6
Hi,
I have been asked privately to work constructively on a solution,
otherwise my bad behaviour will likely lead to a decision that I will
regret if this discussion leads towards an appeal to the TC.
So let me explain in details my position on the subject.
The issue
-
First of all let's describe the problem.
usrmerge works by moving all data from /lib into /usr/lib and then
creating a symlink /lib -> /usr/lib. The same is done for the biarch
or triarch directories, namely /lib32, /lib64, /libx32 and /libo32
depending on the architecture. Note that this part is done by usrmerge,
but doesn't appear in its description, nor in the debconf question
asked to the user. It is important to note that it is done directly on
the filesystem and that this change is not known to dpkg.
When a user install for example libc6-i386 on an amd64 system, dpkg will
follow the symlink and will install everything that should go on /lib32
into /usr/lib32 as expected. This works as expected. However when the
package is removed, /lib32 will be removed. In turn when it is installed
again, the symlink /lib32 -> /usr/lib32 has disappeared.
This is the consequence of:
1) the dpkg policy to remove the last used directory or symlink on
package removal.
2) libc6-i386 being the last package using /lib32.
There is nothing special done with that regard in the glibc maintainer
scripts.
Similarly, that issue also affects the libc6 package when used as a
foreign architecture. For example installing libc6:amd64 on an i386
system and then deinstalling and reinstalling it will cause the /lib64
-> /usr/lib64 to disappear. I guess there might also be some more
issues when installing biarch packages as foreign packages.
The solution envisaged by the usrmerge team
---
The usrmerge people considers that it has to be fixed in the glibc
maintainer scripts. As explained above, it would mean that the preinst
script is responsible, if usrmerge is activated, to create the
/lib{32,64,o32,x32} -> /usr/lib{32,64,o32,x32} symlink if it doesn't
exist. The directory might not be empty as a result of a partial
conversion to usrmerge (yes those system will exists). In turn this
directory might be the one of the dynamic linker used for executing
the maintainer scripts (dash/bash, coreutils). So this means the
maintainer scripts have to handle the case of moving the dynamic
linker without breaking the system.
In short it means *the most tricky part of usrmerge has to be
implemented and maintained in the glibc preinst scripts*. This is even
more difficult as the preinst script will be executed all the time and
not when the user ask for that. It won't ask the user for its permission
before. It doesn't have the option to abort if something looks wrong as
this will lead to a system difficultly recoverable for many users if
they are dist-upgrading their system to a new release (usually apt-get
install -f is not enough).
On the glibc side experience we have very bad experience of moving the
dynamic linker from one directory to another in the maintainer scripts,
and this has resulted in many broken systems. There are still even a few
bugs opened about that for which we haven't really understood the
problem. Looking at the usrmerge's archived bugs, it seems it wasn't an
easy road on the usrmerge side either.
Alternative solutions
-
I do not understand why I should be the one responsible of finding a
solution to this issue. Anyway let me provide a few alternatives
solutions.
In my opinion, the root of the problem is that the symlink is created
without dpkg being aware of that. Therefore it removes it once there are
no users for it. This is exactly why for example base-files ships
/usr/src and we do not require all packages that ship a file in /usr/src
to have postrm script to make sure the directory still exists after the
package removal.
Therefore I really believe the best solution for this issue is to
make dpkg aware of the symlink, which means creating a package
containing this symlinks. This could be for example:
- in base-files, for example by depending on base-files-usrmerge |
base-files-nousrmerge
- in usrmerge after reconsidering that the package can be uninstalled
after the conversion.
- directly in the libc6 and libc6-xxx packages if the TC reconsiders the
"weak" situation for at least the /lib{32,64,o32,x32} directories.
Another alternative solution is simply to only consider /lib and
/usr/lib for usrmerge (like it is alreayd done in the user visible
interface) and ignore the /lib{32,64,o32,x32} directories, at least
until the state stays as "weak".
I would be happy if you could also work on your side to provide
alternative solutions.
Regards,
Aurelien
--
Aurelien Jarno GPG: 40
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
On 2019-08-17 23:00, Marco d'Itri wrote: > On Aug 17, Aurelien Jarno wrote: > > > One package should be responsible for providing those links so that > > glibc is not the last package using them. The same way that base-files > > ensure that some directories are present. > usrmerge is only needed to be installed during the conversion of a > non-merged system, so it cannot do this. > > Putting the links inside some package would not solve the problem of > e.g. /lib32/ and /libx32/ being always created even on systems which do > not need them. > And worse, deleting them would become impossible because they would be > created again every time the package is upgraded. > > Doing this in base-files would require having both a "base-files" and > a "base-files-not-merged" package as long as we will keep supporting > non-merged systems, and I think that the complexity of managing this > (in deboostrap and possibly somewhere else) makes this a non-starter. > > If this were implemented in a different package then it would need to be > Essential (and still require special-casing in debootstrap as long as we > will want to support non-merged systems), so I am not sure if this plan > would be accepted by all the stakeholders. Fine it's your opinion. Then I leave you imagine other ideas. I am still believing cluttering the maintainer scripts of the glibc is not the right way to go, even if it's the easy way for you. -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net signature.asc Description: PGP signature
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
On Aug 17, Aurelien Jarno wrote: > One package should be responsible for providing those links so that > glibc is not the last package using them. The same way that base-files > ensure that some directories are present. usrmerge is only needed to be installed during the conversion of a non-merged system, so it cannot do this. Putting the links inside some package would not solve the problem of e.g. /lib32/ and /libx32/ being always created even on systems which do not need them. And worse, deleting them would become impossible because they would be created again every time the package is upgraded. Doing this in base-files would require having both a "base-files" and a "base-files-not-merged" package as long as we will keep supporting non-merged systems, and I think that the complexity of managing this (in deboostrap and possibly somewhere else) makes this a non-starter. If this were implemented in a different package then it would need to be Essential (and still require special-casing in debootstrap as long as we will want to support non-merged systems), so I am not sure if this plan would be accepted by all the stakeholders. -- ciao, Marco signature.asc Description: PGP signature
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
On 2019-08-17 22:20, Marco d'Itri wrote:
> On Aug 17, Aurelien Jarno wrote:
>
> > > > The preinst scripts could check whether the package is being installed
> > > > in a --merged-usr environment and create (dangling) symlinks if
> > > > /usr/lib{32,x32} is missing. And postrm remove could recreate them if
> > > > they went missing.
> Yes: this is how I hoped that this could be implemented, to stop having
> useles e.g. /libx32/ on all amd64 systems which will never see x32
> packages.
>
> > As explained it's not a bug of the glibc package, but a design flaw of
> > usrmerge. I am therefore reassigning the bug to debootstrap + usrmerge.
> Do you have any suggestions about how you would like this to be fixed?
One package should be responsible for providing those links so that
glibc is not the last package using them. The same way that base-files
ensure that some directories are present.
To take an analogy, base-files provides /usr/src to ensure it is always
there. It is not created by deboostrap and we do not ask all packages
writing to /usr/src to have a postrm file recreating the directory.
--
Aurelien Jarno GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net
signature.asc
Description: PGP signature
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
On Aug 17, Aurelien Jarno wrote:
> > > The preinst scripts could check whether the package is being installed
> > > in a --merged-usr environment and create (dangling) symlinks if
> > > /usr/lib{32,x32} is missing. And postrm remove could recreate them if
> > > they went missing.
Yes: this is how I hoped that this could be implemented, to stop having
useles e.g. /libx32/ on all amd64 systems which will never see x32
packages.
> As explained it's not a bug of the glibc package, but a design flaw of
> usrmerge. I am therefore reassigning the bug to debootstrap + usrmerge.
Do you have any suggestions about how you would like this to be fixed?
--
ciao,
Marco
signature.asc
Description: PGP signature
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
reassign -1 debootstrap,usrmerge
thanks
On 2019-04-09 16:53, Aurelien Jarno wrote:
> On 2019-04-09 12:27, Andreas Beckmann wrote:
> > Package: libc6-x32,libc6-i386
> > Version: 2.28-8
> > Severity: serious
> > User: debian...@lists.debian.org
> > Usertags: piuparts
> >
> > Hi,
> >
> > during a test with piuparts in a --merged-usr environment I noticed that
> > installing, removing, and installing again a package shipping /lib32,
> > /libx32 will actually unmerge that directory.
> > The package will take ownership of the preexisting symlinks
> > /lib{32,x32} -> /usr/lib{32,x32} that were created by debootstrap,
> > remove them and create plain /usr/lib{32,x32} directories in the next
> > installation.
> > (/lib64 should be mostly safe due to /lib64/ld-linux-x86-64.so.2, but
> > perhaps on !x86 architectures)
/lib64 is an issue for at least libc6:amd64 on an i386 system. And there
are many more cases if you also consider other official architectures
and ports architectures.
> Hmm the only fault of the libc6-i386 and libc6-x32 packages (plus I
> guess all the other bi/tri-arch ones on other architectures) is to be
> the last user of those directory when being removed. They do not do
> anything tricky in their directories.
>
> > The preinst scripts could check whether the package is being installed
> > in a --merged-usr environment and create (dangling) symlinks if
> > /usr/lib{32,x32} is missing. And postrm remove could recreate them if
> > they went missing.
The glibc maintainer scripts are already cluttered with many ugly and
fragile workarounds to handle the co-installability of foreign and biarch
glibc. I do not want to adds more workarounds that might will make the
whole things a nightmare.
> This looks like an ugly workaround to me, and might not work if a
> package start adding files there without depending on libc6. This looks
> to me like a flaw in the usrmerge design. The base-files package is
> designed to prevent directories or symlinks to be removed, so I wonder
> if we need a usrmerge version of it.
As explained it's not a bug of the glibc package, but a design flaw of
usrmerge. I am therefore reassigning the bug to debootstrap + usrmerge.
I am not opposed to a workaround in the glibc package, kept for one
release cycle only, *once a real solution* is implemented for this bug.
--
Aurelien Jarno GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net
signature.asc
Description: PGP signature
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
On Tue, Apr 09, 2019 at 12:27:45PM +0200, Andreas Beckmann wrote: > Package: libc6-x32,libc6-i386 > Version: 2.28-8 > Severity: serious > User: debian...@lists.debian.org > Usertags: piuparts > > Hi, > > during a test with piuparts in a --merged-usr environment I noticed that > installing, removing, and installing again a package shipping /lib32, > /libx32 will actually unmerge that directory. So I understand that may be undesirable, but what is the justification for this being "serious"? Does anything actually break? Cheers, Julien
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
On 2019-04-09 12:27, Andreas Beckmann wrote:
> Package: libc6-x32,libc6-i386
> Version: 2.28-8
> Severity: serious
> User: debian...@lists.debian.org
> Usertags: piuparts
>
> Hi,
>
> during a test with piuparts in a --merged-usr environment I noticed that
> installing, removing, and installing again a package shipping /lib32,
> /libx32 will actually unmerge that directory.
> The package will take ownership of the preexisting symlinks
> /lib{32,x32} -> /usr/lib{32,x32} that were created by debootstrap,
> remove them and create plain /usr/lib{32,x32} directories in the next
> installation.
> (/lib64 should be mostly safe due to /lib64/ld-linux-x86-64.so.2, but
> perhaps on !x86 architectures)
Hmm the only fault of the libc6-i386 and libc6-x32 packages (plus I
guess all the other bi/tri-arch ones on other architectures) is to be
the last user of those directory when being removed. They do not do
anything tricky in their directories.
> The preinst scripts could check whether the package is being installed
> in a --merged-usr environment and create (dangling) symlinks if
> /usr/lib{32,x32} is missing. And postrm remove could recreate them if
> they went missing.
This looks like an ugly workaround to me, and might not work if a
package start adding files there without depending on libc6. This looks
to me like a flaw in the usrmerge design. The base-files package is
designed to prevent directories or symlinks to be removed, so I wonder
if we need a usrmerge version of it.
--
Aurelien Jarno GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net
signature.asc
Description: PGP signature
Bug#926699: libc6-{i386,x32}: installing, removing, reinstalling in a --merged-usr system results in unmerged /lib{32,x32}
Package: libc6-x32,libc6-i386
Version: 2.28-8
Severity: serious
User: debian...@lists.debian.org
Usertags: piuparts
Hi,
during a test with piuparts in a --merged-usr environment I noticed that
installing, removing, and installing again a package shipping /lib32,
/libx32 will actually unmerge that directory.
The package will take ownership of the preexisting symlinks
/lib{32,x32} -> /usr/lib{32,x32} that were created by debootstrap,
remove them and create plain /usr/lib{32,x32} directories in the next
installation.
(/lib64 should be mostly safe due to /lib64/ld-linux-x86-64.so.2, but
perhaps on !x86 architectures)
The preinst scripts could check whether the package is being installed
in a --merged-usr environment and create (dangling) symlinks if
/usr/lib{32,x32} is missing. And postrm remove could recreate them if
they went missing.
Andreas
