Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Mon, 03 Dec 2018 at 17:45:11 +0100, Svante Signell wrote: > On Sun, 2018-12-02 at 21:04 +0100, Marc Haber wrote: > > moving hundreds of megabytes from /usr to / over time. > > This solution was proposed by GNU/Hurd several years ago, and was scrapped due > to not being big enough player in the *NIX world: > https://www.gnu.org/software/hurd/faq/slash_usr_symlink.html This did make more sense than the arbitrary split, but with hindsight I'm glad it did't take off, because the /usr merge does have an important advantage over the old Hurd proposal: it takes all the static OS files (which are more similar than they are different) and groups them together. Unifying those files into / as Hurd used to do would have made it harder to disentangle them from the sysadmin-modifiable /etc directory, which needs to be on the root filesystem anyway because it's where we find /etc/fstab. If I remember correctly, the people who implemented the /usr merge in Fedora actually started by proposing that the static files were unified into / (as in older Hurd versions), and later switched their design around when they realised that grouping those directories together would be better. The name /usr is indeed an unfortunate historical accident. > We think that we have found a more > flexible solution called union filesystems, which allow to create virtual > filesystems which are the union of several other filesystems. However, > support > for union filesystems is still in early development. I get the impression that union mounts in Linux aren't completely reliable either (and have some awkward corner-cases), so solutions that don't require them seem likely to be more robust. smcv
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On 2018-12-03 00:37 +0100, Wouter Verhelst wrote: > This is incorrect. The current plan has some systems be merged-/usr and > others not while we are in the transition. It therefore introduces > Debian-wide complexity in that maintainers are expected to support both > merged and unmerged /usr, which causes problems (as we see here). It > also effects a change without the maintainers of the software in > question being involved, which could have unintended side effects with > some packages that we don't know yet (and that we probably won't know > about until the release of buster). > > Changing this through a policy change, as we've always done, would not > have those problems: > > - Maintainers are expected to move their own package to merged-/usr, so > they would be aware of issues that might ensue and would be able to > deal with them. > - We are not expected to be supporting merged-/usr and unmerged-/usr at > the same time; instead, we'll be gradually moving towards a > merged-/usr situation. > - We don't end up with packages' files being moved from under them. I don't want to add pile of verbiage here, but I'd just like to say that everything Wouter said makes a whole lot of sense to me. We know how to do this sort of transition, and yes it takes some time, but that's OK. Using usrmerge to try and shortcut this, producing the awkward 'dual-state' issues does not seem to me to be a good way to go. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sun, 2018-12-02 at 21:04 +0100, Marc Haber wrote: > > The next debhelper change might choose to give / instead of /usr as a > target directory by default, moving hundreds of megabytes from /usr to / > over time. This solution was proposed by GNU/Hurd several years ago, and was scrapped due to not being big enough player in the *NIX world: https://www.gnu.org/software/hurd/faq/slash_usr_symlink.html The distinction between / and /usr has historical reasons. Back when Unix systems were booted from two tapes, a small root tape and a big user tape. Today, we like to use different partitions for these two spaces. The Hurd throws this historical garbage away. We think that we have found a more flexible solution called union filesystems, which allow to create virtual filesystems which are the union of several other filesystems. However, support for union filesystems is still in early development. About unionfs: Unionfs allows you to simply union one directory or translator into another one, so you see the files of both of them side by side. More here: https://www.gnu.org/software/hurd/hurd/translator/unionfs.html
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sun, Dec 02, 2018 at 11:31:13PM +0100, Marco d'Itri wrote: > On Dec 02, Wouter Verhelst wrote: > > > One thing that has not been answered yet in this discussion (and if the > > TC is to make a decision about it, I think it should be) is "why are we > > doing this". That is, what is the problem that usrmerge is meant to > > solve, and how does it attempt to solve it? As far as I know, the reason > > usrmerge exists is so that no files will be installed in /bin anymore; > > but that seems like an XY problem. > https://lists.debian.org/20181121140542.ga31...@espresso.pseudorandom.co.uk Thanks; somehow I missed that, sorry. > > Also, I would like to ask why the traditional solution in Debian -- make > > a policy change that says no package can ship anything in /bin except > > for a compatibility symlink, and make that rule RC at some point in the > > future -- is not being considered. That seems like a solution that would > > cause far less pain than what we're going through right now. > This is not a solution. For a start it would take many years. The fact that doing something in one particular way takes longer does not in and of itself make it a bad solution. It also need not take that long. We can declare *right now* that shipping something in /bin (as opposed to /usr/bin) that is not a compatibility symlink will be RC in bullseye. This would be plenty of time for maintainers to be aware of it and to start looking at updating their packages. With your usrmerge package, it's not at all clear to me when the migration would be finished. > Even ignoring that, it would not bring any improvement over the current > plan: This is incorrect. The current plan has some systems be merged-/usr and others not while we are in the transition. It therefore introduces Debian-wide complexity in that maintainers are expected to support both merged and unmerged /usr, which causes problems (as we see here). It also effects a change without the maintainers of the software in question being involved, which could have unintended side effects with some packages that we don't know yet (and that we probably won't know about until the release of buster). Changing this through a policy change, as we've always done, would not have those problems: - Maintainers are expected to move their own package to merged-/usr, so they would be aware of issues that might ensue and would be able to deal with them. - We are not expected to be supporting merged-/usr and unmerged-/usr at the same time; instead, we'll be gradually moving towards a merged-/usr situation. - We don't end up with packages' files being moved from under them. > even if your idea were executed perfectly and quickly then the > conversion program would still be in the same exact situation as it is > now: Yes, obviously, that's the whole point. [...] > So this would make the transition unacceptably slow while providing no > benefit at all, I don't agree with both these statements. > but it would also cost a huge amount of work to change many packages. Yes, and at the same time it would reduce a huge amount of *different* work, since packages now won't need to be changed so that "being built on merged-/usr" won't result in packages that don't build on unmerged-/usr. -- To the thief who stole my anti-depressants: I hope you're happy -- seen somewhere on the Internet on a photo of a billboard
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sun, Dec 02, 2018 at 11:31:13PM +0100, Marco d'Itri wrote: > On Dec 02, Wouter Verhelst wrote:> > > One thing that has not been answered yet in this discussion (and if the > > TC is to make a decision about it, I think it should be) is "why are we > > doing this". That is, what is the problem that usrmerge is meant to > > solve, and how does it attempt to solve it? As far as I know, the reason > > usrmerge exists is so that no files will be installed in /bin anymore; > > but that seems like an XY problem. > https://lists.debian.org/20181121140542.ga31...@espresso.pseudorandom.co.uk That post is too long-winded and touches so many unrelated issues that it's difficult to respond to it. Could you please give us a terse list of claimed benefits? Without a clear claim, it's too easy to both do or be accused of doing a straw-man. By my reading, most of that post either falls into the XY problem or explains how merged-usr fixes problems caused by merged-usr itself. That leaves only one concrete claim: * that merged-usr makes it easier to have a part of the system immutable (be it for recovery purposes or for sharing) My response to that claim is: there's a long list of techniques that can be used for such an effect without sweeping distribution-wide changes. Heck, not even a couple of hours ago I restored from a snapshot multiple times in order to troubleshoot a broken udev problem. Here's your "This means you always have a known-working filesystem to fall back on (if the most recently updated filesystem doesn't boot, use the other one)". Or, container sharing: works fine for me without merged-usr either. Thus, in order to make this claim stick, you need to not only list its benefits but also explain how it's better than other approaches: filesystem-based (btrfs, ZFS, XFS), vfs-based (various overlays), lvm-based/-like, block-device based, multiple mounts, etc. And, many of those can do CoW which is so much better than immutability. (Please explain if I understood you wrong.) Meow! -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Ivan was a wordly man: born in St. Petersburg, raised in ⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned ⠈⠳⣄ to the city of his birth to die.
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sun, 02 Dec 2018 at 21:04:55 +0100, Marc Haber wrote: > The next debhelper change might choose to give / instead of /usr as a > target directory by default, moving hundreds of megabytes from /usr to / > over time. I don't think anyone is proposing that. There's no reason why it would be preferred over the merged /usr arrangement implemented in both usrmerge and debhelper --merged-usr, which is the same as is implemented in some other Linux distributions (e.g. Fedora): /usr is a real directory, and /bin, /sbin, /lib* are symlinks to the corresponding directories in /usr. Unifying /usr with / by making /usr a symlink to / is the opposite of the merged /usr arrangement that is currently implemented. The Debian hurd-i386 port did try having a /usr -> / symlink a few years ago as a simplification, but it has all the same transitional issues as merged /usr, with fewer advantages, which makes it unappealing. The purpose of merged /usr is to group together all the static files (/bin, /sbin, /lib* and the current /usr), which are more similar than they are diferent, into one place that can be a (maybe read-only) mount point (/usr). If you do the opposite, your root filesystem is still a mixture of mutable files in /etc and static files in /bin, /sbin, /lib*, so you haven't gained as much simplification as you would have had with merged /usr. (You talk about "default target directories", but there is nothing so elaborate: debootstrap --merged-usr simply unpacks packages into a chroot that already contains the symbolic links like /bin -> /usr/bin, instead of into an empty chroot.) smcv
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Dec 02, Wouter Verhelst wrote: > One thing that has not been answered yet in this discussion (and if the > TC is to make a decision about it, I think it should be) is "why are we > doing this". That is, what is the problem that usrmerge is meant to > solve, and how does it attempt to solve it? As far as I know, the reason > usrmerge exists is so that no files will be installed in /bin anymore; > but that seems like an XY problem. https://lists.debian.org/20181121140542.ga31...@espresso.pseudorandom.co.uk > Also, I would like to ask why the traditional solution in Debian -- make > a policy change that says no package can ship anything in /bin except > for a compatibility symlink, and make that rule RC at some point in the > future -- is not being considered. That seems like a solution that would > cause far less pain than what we're going through right now. This is not a solution. For a start it would take many years. Even ignoring that, it would not bring any improvement over the current plan: even if your idea were executed perfectly and quickly then the conversion program would still be in the same exact situation as it is now: either everything in /bin/, /sbin and /lib (and its own subdirectories) was created by the packaging system, and then we already know that it can be converted automatically, or it was not, and then we know that there are a few cases when the local administrator has to decide what to do about things that were installed by himself in the past in the wrong place. So this would make the transition unacceptably slow while providing no benefit at all, but it would also cost a huge amount of work to change many packages. -- ciao, Marco signature.asc Description: PGP signature
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Wed, Nov 28, 2018 at 01:49:45PM +, Ian Jackson wrote: > Control: reassign -1 tech-ctte > > Dear Technical Committee. I don't know if you are all aware of the > discussion surrounding this, so I will recap: > > Recently debootstrap was changed to do merged-/usr by default, so that > /bin -> /usr/bin etc. > > It was discovered that when this change took effect on the Debian > buildds, the buildds started to build packages which do not work on > non-merged-/usr systems. > > This is a special case of a general problem: buster systems with > merged-/usr sometimes build packages which are broken on > non-merged-/usr systems. > > Some people have suggested that this should be fixed by making > merged-/usr mandatory for buster. However, there is no transition > plan for this as yet and I don't think Debian is ready to commit to do > that. > > So I believe that this change should be immediately reverted in sid > and buster, so that we do not prejudge the situation by increasing the > number of buster installs in the field which generate packages which > are broken on non-merged-/usr systemss. One thing that has not been answered yet in this discussion (and if the TC is to make a decision about it, I think it should be) is "why are we doing this". That is, what is the problem that usrmerge is meant to solve, and how does it attempt to solve it? As far as I know, the reason usrmerge exists is so that no files will be installed in /bin anymore; but that seems like an XY problem. Also, I would like to ask why the traditional solution in Debian -- make a policy change that says no package can ship anything in /bin except for a compatibility symlink, and make that rule RC at some point in the future -- is not being considered. That seems like a solution that would cause far less pain than what we're going through right now. -- To the thief who stole my anti-depressants: I hope you're happy -- seen somewhere on the Internet on a photo of a billboard
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Marc Haber writes: > The next debhelper change might choose to give / instead of /usr as a > target directory by default, moving hundreds of megabytes from /usr to / > over time. My question was about the distant future, and not the current > snapshot of things. If anything then /usr would be the prefix for all packages; nothing would be installed to /bin, /sbin or /lib. (And there is no /share or /local.) But this is obviously only possible if non-merged-/usr is not supported at all as that would mean shipping only /usr/bin/sh and no /bin/sh; the /bin -> /usr/bin symlink would then be required. Ansgar
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sun, 02 Dec 2018 at 19:54:39 +, Simon McVittie wrote: > When installed on a merged /usr system: Oops, of course this should have said: .deb contains -> /bin/grep/usr/bin/perl /bin/fooexists thanks to /bin symlink exists thanks to /bin symlink /usr/bin/foo physical locationphysical location
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Marc Haber writes: > On Sat, Dec 01, 2018 at 11:36:45PM +0100, Didier 'OdyX' Raboud wrote: >> Le samedi, 1 décembre 2018, 19.29:59 h CET Marc Haber a écrit : >> > Will binaries move from /usr/bin to /bin? Or will binaries move from >> > /bin to /usr/bin? >> >> A merged-/usr has a /bin → /usr/bin symlink; so a .deb package unpacking >> /bin/grep will make that binary end up in /usr/bin/grep; but the >> /bin → /usr/bin symlink ensures that you can either access /usr/bin/grep >> directly or /bin/grep through the symlink. > > And where will the binaries and up on an un-usrmerged system with a > dedicated /usr? in /usr, I hope? They won't move on a system that doesn't have merged-/usr. /bin/bash will stay in /bin/bash. If you switch to a merged-/usr (for example by installing usrmerge) then they will be moved to /usr. Ansgar
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sun, Dec 02, 2018 at 07:54:39PM +, Simon McVittie wrote: > I would encourage anyone with questions about the behaviour of merged and > unmerged /usr systems to try them, either by building one chroot, container > or VM with merged /usr and one without, or by building a chroot, container > or VM with unmerged /usr, copying it, and installing usrmerge in the copy. That would give me an impression about how things are today. > The answer to your question is that non-/usr-merged systems behave the > same way they always have: binaries end up wherever the source package > chooses to put them when it builds the .deb (typically determined by > where they appear under debian/tmp). The next debhelper change might choose to give / instead of /usr as a target directory by default, moving hundreds of megabytes from /usr to / over time. My question was about the distant future, and not the current snapshot of things. And, it has been satisfactorily answered by now. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sun, 02 Dec 2018 at 20:16:10 +0100, Marc Haber wrote: > On Sat, Dec 01, 2018 at 11:36:45PM +0100, Didier 'OdyX' Raboud wrote: > > A merged-/usr has a /bin → /usr/bin symlink; so a .deb package unpacking > > /bin/grep will make that binary end up in /usr/bin/grep; but the > > /bin → /usr/bin symlink ensures that you can either access /usr/bin/grep > > directly or /bin/grep through the symlink. > > And where will the binaries and up on an un-usrmerged system with a > dedicated /usr? in /usr, I hope? I would encourage anyone with questions about the behaviour of merged and unmerged /usr systems to try them, either by building one chroot, container or VM with merged /usr and one without, or by building a chroot, container or VM with unmerged /usr, copying it, and installing usrmerge in the copy. The answer to your question is that non-/usr-merged systems behave the same way they always have: binaries end up wherever the source package chooses to put them when it builds the .deb (typically determined by where they appear under debian/tmp). To avoid breaking hard-coded paths on existing non-/usr-merged systems, that has to be where the package has traditionally put them, unless the package takes special steps to create a compatibility symlink on non-merged-/usr systems (which I would not recommend[1]). When installed on an unmerged-/usr system: .deb contains -> /bin/grep/usr/bin/perl /bin/foo physical location does not exist /usr/bin/foo does not exist physical location When installed on a merged /usr system: .deb contains -> /bin/grep/usr/bin/perl /bin/fooexists thanks to /bin symlink physical location /usr/bin/fooexists thanks to /bin symlink physical location None of the above depends on whether /usr is a mount point. If /usr is a mount point, then /bin/grep is physically located on the /usr filesystem if the system is merged-/usr, or on the root filesystem otherwise. Replace /bin/grep in the above with any executable that is canonically in /bin or /sbin (its source package would typically install something like debian/tmp/bin/grep), and replace /usr/bin/perl with any executable that is canonically in /usr/bin or /usr/sbin (its source package would typically install something like debian/tmp/usr/bin/perl). smcv [1] https://lists.debian.org/msgid-search/20181122205532.ga2...@espresso.pseudorandom.co.uk
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sat, Dec 01, 2018 at 11:36:45PM +0100, Didier 'OdyX' Raboud wrote: > Le samedi, 1 décembre 2018, 19.29:59 h CET Marc Haber a écrit : > > Will binaries move from /usr/bin to /bin? Or will binaries move from > > /bin to /usr/bin? > > A merged-/usr has a /bin → /usr/bin symlink; so a .deb package unpacking > /bin/grep will make that binary end up in /usr/bin/grep; but the > /bin → /usr/bin symlink ensures that you can either access /usr/bin/grep > directly or /bin/grep through the symlink. And where will the binaries and up on an un-usrmerged system with a dedicated /usr? in /usr, I hope? Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Hello Didier, Thanks for providing your reasoning. On Sun, Dec 02, 2018 at 01:28:20AM +0100, Didier 'OdyX' Raboud wrote: [...] > In other words, if the debootstrap maintainers now agree with Ian about > toggling the "merged-/usr" default back to false, there's really no point for > the TC to overrule unnecessarily, or even continue discussing this. FWIW If I where to receive the previous mail you adressed to debootstrap maintainers I would very likely have interpretted what you where saying more like: "I'm telling you with my TC hat on to change the default, or we will proceed to voting on overruling your decision anyway." ... which if I'm reading you correctly was not at all your intention. > And if they don't agree, their _current_ reasoning will provide value > to the discussion, and ideally provide for a better TC decision. Exactly what I was thinking. Might have been better with just asking for their current view on things and if they are planning on changing anything soon based on the recent discussions (and maybe also tell Ian to talk directly to the maintainer instead of using TC as a proxy). Regards, Andreas Henriksson
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Hello, On Sat, Dec 01, 2018 at 11:36:45PM +0100, Didier 'OdyX' Raboud wrote: > Le samedi, 1 décembre 2018, 19.29:59 h CET Marc Haber a écrit : > > This might sound like a stupid question, what will happen when a package > > built on a usrmerged System will be installed on a non-usrmerged system? > > FTR, I try to always assume that no question is stupid. Only answers can be. > > My understanding is that there's no universal answer to this; for a lot of > packages, the answer is "nothing"; it will just work. For some packages > though, if they embed executable paths such as /usr/bin/grep where only > /bin/grep exists on the host system, this will break. Or it could end up as /usr/local/bin/grep or as /home/randomuser/bin/grep just as well. This is really a generic problem of packages not being reproducible when embedding full path found in PATH or the build system. Even if debootstrap and usrmerge both where nuked into orbit the problem wouldn't go away! (This is not a problem for packages built on official buildds already. It's a problem when built on users machines in a "dirty" environment and then shared.) > > > Will binaries move from /usr/bin to /bin? Or will binaries move from > > /bin to /usr/bin? > > A merged-/usr has a /bin → /usr/bin symlink; so a .deb package unpacking > /bin/grep will make that binary end up in /usr/bin/grep; but the > /bin → /usr/bin symlink ensures that you can either access /usr/bin/grep > directly or /bin/grep through the symlink. The opposite can also happen as some packages fiddle with PATH (eg. by using AC_PATH_PROGs fourth argument) and you can thus end up embedding /bin/awk (as on usrmerged systems /bin -> /usr/bin), when a non-merged system only has /usr/bin/awk. > > An open question I think is whether dpkg should/could at one point in the > future always unpack binaries to /usr/bin and never in /bin no matter what > the > package contains (+ setup a convenience symlink in /bin if /bin is a real dir > ?). I don't see how this is any better than just using usrmerge. The long term solution, once noone any longer carries the opinion that non-merged systems needs to be supported, would be to rebuild packages so that the binary archives doesn't ship things in /bin et.al anymore. Then dpkg doesn't need to do anything (and dpkg-query will be all happy again). If you think modifying packages individually for this is to be avoided, then someone has already mentioned the idea of doing a dh_usrmerge that would relocate a packages files before the binary package is compressed. Regards, Andreas Henriksson
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sat, Dec 01, 2018 at 07:20:57PM +0100, Didier 'OdyX' Raboud wrote:
> As a way to improve my understanding of the issue at hand, here's my current
> collection of data points regarding the "merged-/usr" question:
>
> * "merged-/usr" was enabled by default in debootstrap on June 13. 2018, some
> 5+ months ago. Any buster rootfs setup since has the /bin → /usr/bin
> symlink (and other caracteristics of merged-/usr).
>
>
> https://tracker.debian.org/news/965045/accepted-debootstrap-10102-source-all-into-unstable/
For completeness sake you might want to start the story at the
point in pre-stretch-freeze era where the default was changed:
https://tracker.debian.org/news/807889/accepted-debootstrap-1085-source-into-unstable/
And then disabled and post-postponed to post-stretch release:
https://tracker.debian.org/news/815610/accepted-debootstrap-1087-source-all-into-unstable/
>
> * With my TC hat on, I have formally asked the debootstrap maintainers
> (and specifically Hideki Yamane) if they would consider a toggle of the
> default.
>
> https://bugs.debian.org/914897#73
May I ask you to share why you did that? I'm interested to know the
reasoning.
(It's not my understanding that this is how TC normally operates.)
>
> * a discussion "usrmerge -- plan B?" was started on Nov. 20 on debian-
> devel@l.d.o
>
> https://lists.debian.org/20181120211617.gxnuwxpx2hy44...@angband.pl
>
> It apparently follows #913229, #914204 and others.
>
> * Currently, according to my `apt-file`, 259 binaries are shipped in /bin
> directly, accross 85 packages. (for /sbin, 597 binaries for 190 packages).
>
> * There exists a 'usrmerge' package since 2015, which transforms a rootfs
> with
> separate /{bin,sbin,lib} into a "merged-usr/" rootfs. It has had 28 bugs
> over its lifetime; of which 4 are currently open. After successful
> "installation" (when the postinst successfully ran the
> /usr/lib/convert-usrmerge program), /{bin,sbin,lib} are made symlinks and
> the package can be removed. The package doesn't include a way to revert the
> rootfs to its previous state.
>
> * Building source packages in a merged-/usr rootfs can make the resulting
> binary packages broken in non-merged-/usr rootfs'es, because they'd be
> embedding references to /usr/bin/grep (e.g.), which don't exist in non-
> merged-/usr rootfs'es.
--->
> To ensure this doesn't happen for packages built on
> Debian infrastructure, debootstrap has been updated to disable merged-/usr
> for the `buildd` variant (and buildd chroots have been rebuilt).
... while that change happened, I don't think that's what technically
disabled merged-usr on the debian buildd chroots. AIUI the buildd
chroots are created using debootstrap from stable-backports repository.
There was a new upload to stable-backports that enabled the merged-usr
by default which triggered a few problems. The admins thus explicitly
disabled merged-usr via the debootstrap commandline flag in the
continuous deployment system for buildd chroots and triggered chroot
recreation jobs.
>
> * According to investigation done thanks to reproducible builds (which now
> also varies the merged-/usr state of the build rootfs), and by others,
> packages affected by this problem have begun receiving either
> reproducibility issue tags or Debian bugs:
>
>
> https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html
> (currently: 59)
>
>
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?users=m...@linux.it;tag=usrmerge
> (current total: 81; outstanding: 17)
FWIW I've analyzed all packages that are tagged on the reproducible
build page. Messing with the debian bts is just too time consuming as
if looking at this wasn't boring enough to begin with. If anyone thinks
there are some interesting data points to be found please ask.
>
> I'll post my thoughts separately; please enhance or correct the above data
> as needed!
>
> Cheers,
> OdyX
Regards,
Andreas Henriksson
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Le samedi, 1 décembre 2018, 19.29:59 h CET Marc Haber a écrit : > This might sound like a stupid question, what will happen when a package > built on a usrmerged System will be installed on a non-usrmerged system? FTR, I try to always assume that no question is stupid. Only answers can be. My understanding is that there's no universal answer to this; for a lot of packages, the answer is "nothing"; it will just work. For some packages though, if they embed executable paths such as /usr/bin/grep where only /bin/grep exists on the host system, this will break. > Will binaries move from /usr/bin to /bin? Or will binaries move from > /bin to /usr/bin? A merged-/usr has a /bin → /usr/bin symlink; so a .deb package unpacking /bin/grep will make that binary end up in /usr/bin/grep; but the /bin → /usr/bin symlink ensures that you can either access /usr/bin/grep directly or /bin/grep through the symlink. An open question I think is whether dpkg should/could at one point in the future always unpack binaries to /usr/bin and never in /bin no matter what the package contains (+ setup a convenience symlink in /bin if /bin is a real dir ?). Cheers, OdyX signature.asc Description: This is a digitally signed message part.
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Marc Haber writes:
> On Sat, Dec 01, 2018 at 07:20:57PM +0100, Didier 'OdyX' Raboud wrote:
>> * Currently, according to my `apt-file`, 259 binaries are shipped in /bin
>> directly, accross 85 packages. (for /sbin, 597 binaries for 190 packages).
>
> This might sound like a stupid question, what will happen when a package
> built on a usrmerged System will be installed on a non-usrmerged system?
>
> Will binaries move from /usr/bin to /bin? Or will binaries move from
> /bin to /usr/bin?
They will be installed in the place the package installs them. Note
that there are no /bin -> /usr/bin symlink in the staging area where the
package is built (i.e. debian/${package} or debian/tmp).
Packages have to continue shipping binaries in /bin unless we decide to
make merged-/usr mandatory and convert existing systems.
Ansgar
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On Sat, Dec 01, 2018 at 07:20:57PM +0100, Didier 'OdyX' Raboud wrote:
> * Currently, according to my `apt-file`, 259 binaries are shipped in /bin
> directly, accross 85 packages. (for /sbin, 597 binaries for 190 packages).
This might sound like a stupid question, what will happen when a package
built on a usrmerged System will be installed on a non-usrmerged system?
Will binaries move from /usr/bin to /bin? Or will binaries move from
/bin to /usr/bin?
The former is likely to cause fun on upgrades, since root file systems
on systems that were installed with /usr on its own partition tend to be
rather smallish. Dumping the entire former content of /usr/{bin,sbin} to
/{bin,sbin} is likely to wreck those systems, most of them probably
being either embedded or in big datacenters where people usually don't
waste disk space for root file systems.
Greetings
Marc
--
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
As a way to improve my understanding of the issue at hand, here's my current
collection of data points regarding the "merged-/usr" question:
* "merged-/usr" was enabled by default in debootstrap on June 13. 2018, some
5+ months ago. Any buster rootfs setup since has the /bin → /usr/bin
symlink (and other caracteristics of merged-/usr).
https://tracker.debian.org/news/965045/accepted-debootstrap-10102-source-all-into-unstable/
* With my TC hat on, I have formally asked the debootstrap maintainers
(and specifically Hideki Yamane) if they would consider a toggle of the
default.
https://bugs.debian.org/914897#73
* a discussion "usrmerge -- plan B?" was started on Nov. 20 on debian-
devel@l.d.o
https://lists.debian.org/20181120211617.gxnuwxpx2hy44...@angband.pl
It apparently follows #913229, #914204 and others.
* Currently, according to my `apt-file`, 259 binaries are shipped in /bin
directly, accross 85 packages. (for /sbin, 597 binaries for 190 packages).
* There exists a 'usrmerge' package since 2015, which transforms a rootfs with
separate /{bin,sbin,lib} into a "merged-usr/" rootfs. It has had 28 bugs
over its lifetime; of which 4 are currently open. After successful
"installation" (when the postinst successfully ran the
/usr/lib/convert-usrmerge program), /{bin,sbin,lib} are made symlinks and
the package can be removed. The package doesn't include a way to revert the
rootfs to its previous state.
* Building source packages in a merged-/usr rootfs can make the resulting
binary packages broken in non-merged-/usr rootfs'es, because they'd be
embedding references to /usr/bin/grep (e.g.), which don't exist in non-
merged-/usr rootfs'es. To ensure this doesn't happen for packages built on
Debian infrastructure, debootstrap has been updated to disable merged-/usr
for the `buildd` variant (and buildd chroots have been rebuilt).
* According to investigation done thanks to reproducible builds (which now
also varies the merged-/usr state of the build rootfs), and by others,
packages affected by this problem have begun receiving either
reproducibility issue tags or Debian bugs:
https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html
(currently: 59)
https://bugs.debian.org/cgi-bin/pkgreport.cgi?users=m...@linux.it;tag=usrmerge
(current total: 81; outstanding: 17)
I'll post my thoughts separately; please enhance or correct the above data
as needed!
Cheers,
OdyX
signature.asc
Description: This is a digitally signed message part.
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Ian Jackson wrote: Obviously I disagree. I think the question is urgent. Please would you rapidly overrule the debootstrap maintainers. After we have ceased introducing new lossage we can have a proper conversation about what the plan ought to be for buster and bullseye. Well, please take into account that a version of debootstrap which defaulted to merged /usr has been in Debian Testing (and thus, if I am not mistaken, available through daily builds of the netinst CDs) since June 25 or so. Five months. No matter what we decide finally, it is necessary to support systems that were installed during that five-month period. In other words, it's already too late to "minimize" the damage by undoing the change, and there is no point in acting quickly. At least in my opinion. -- Alexander E. Patrakov
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On 11/28/18 4:14 PM, Ian Jackson wrote:
> Julien Cristau writes ("Re: Bug#914897: debootstrap, buster: Please disabled
> merged /usr by default"):
>> On 11/28/18 2:49 PM, Ian Jackson wrote:
>>> This is a special case of a general problem: buster systems with
>>> merged-/usr sometimes build packages which are broken on
> ...
>> I'd suggest that this should be fixed by not shipping any packages that
>> aren't built on buildds.
>
> It would be quite a radical departure for Debian to no longer support
> "I built this package for my mate to install on their computer".
>
It'll be just as "supported" as it is today (which is to say, basically
not at all). Likely to work in many cases, but also likely to explode
in a million pieces depending on the build environment. I'd just like
to stop inflicting this risk on people downloading debs from *.debian.org.
Cheers,
Julien
Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Hello,
On Wed, Nov 28, 2018 at 03:14:20PM +, Ian Jackson wrote:
> Julien Cristau writes ("Re: Bug#914897: debootstrap, buster: Please disabled
> merged /usr by default"):
[...]
> > I'd suggest that this should be fixed by not shipping any packages that
> > aren't built on buildds.
>
> It would be quite a radical departure for Debian to no longer support
> "I built this package for my mate to install on their computer".
For the case of locally built binaries, bringing any problem
that usrmerge would hit to the light would be preferable.
eg. Consider the case of:
cat << EOF > /usr/local/bin/grep
#!/bin/sh
grep --color $@
EOF
chmod +x /usr/local/bin/grep
(Or export PATH=$HOME/bin:$PATH and something similar as above.)
I've certainly in my maintainer history had people demand fixes
for problems that where caused by their local build environment
setup containing alot crazier things than the above.
Regards,
Andreas Henriksson
PS. As previously mentioned reproducible builds has already been set up
to help catch issues so I don't understand the urgency, see:
https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html
Better ask release-team for permission to treat these as RC if you feel
strongly about these issues.
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Julien Cristau writes ("Re: Bug#914897: debootstrap, buster: Please disabled
merged /usr by default"):
> On 11/28/18 2:49 PM, Ian Jackson wrote:
> > This is a special case of a general problem: buster systems with
> > merged-/usr sometimes build packages which are broken on
...
> I'd suggest that this should be fixed by not shipping any packages that
> aren't built on buildds.
It would be quite a radical departure for Debian to no longer support
"I built this package for my mate to install on their computer".
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On 11/28/18 2:49 PM, Ian Jackson wrote: > Recently debootstrap was changed to do merged-/usr by default, so that > /bin -> /usr/bin etc. > > It was discovered that when this change took effect on the Debian > buildds, the buildds started to build packages which do not work on > non-merged-/usr systems. > > This is a special case of a general problem: buster systems with > merged-/usr sometimes build packages which are broken on > non-merged-/usr systems. > > Some people have suggested that this should be fixed by making > merged-/usr mandatory for buster. However, there is no transition > plan for this as yet and I don't think Debian is ready to commit to do > that. > I'd suggest that this should be fixed by not shipping any packages that aren't built on buildds. Cheers, Julien
Processed: Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Processing control commands: > reassign -1 tech-ctte Bug #914897 [debootstrap] debootstrap, buster: Please disabled merged /usr by default Bug reassigned from package 'debootstrap' to 'tech-ctte'. Ignoring request to alter found versions of bug #914897 to the same values previously set Ignoring request to alter fixed versions of bug #914897 to the same values previously set -- 914897: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914897 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
Control: reassign -1 tech-ctte
Dear Technical Committee. I don't know if you are all aware of the
discussion surrounding this, so I will recap:
Recently debootstrap was changed to do merged-/usr by default, so that
/bin -> /usr/bin etc.
It was discovered that when this change took effect on the Debian
buildds, the buildds started to build packages which do not work on
non-merged-/usr systems.
This is a special case of a general problem: buster systems with
merged-/usr sometimes build packages which are broken on
non-merged-/usr systems.
Some people have suggested that this should be fixed by making
merged-/usr mandatory for buster. However, there is no transition
plan for this as yet and I don't think Debian is ready to commit to do
that.
So I believe that this change should be immediately reverted in sid
and buster, so that we do not prejudge the situation by increasing the
number of buster installs in the field which generate packages which
are broken on non-merged-/usr systemss.
I filed this bug against debootstrap but its maintainers do not agree:
Julien Cristau writes ("Re: Bug#914897: debootstrap, buster: Please disabled
merged /usr by default"):
> We already have a change queued to revert it for build chroots. I don't
> believe anything more is warranted at this stage.
Obviously I disagree. I think the question is urgent. Please would
you rapidly overrule the debootstrap maintainers.
After we have ceased introducing new lossage we can have a proper
conversation about what the plan ought to be for buster and bullseye.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
