Bug#1050001: Unwinding the directory aliasing mistake
Thanks for your considered reply.
Helmut Grohne writes ("Bug#1050001: Unwinding the directory aliasing mistake"):
> It feels rather strange to submit this as a bug report when you do not
> want feedback. A mail to the list is easier to not respond to, but a bug
> needs a closure one way or another eventually.
I need to clarify this, evidently. I didn't mean I'm not going to be
reading replies, and I didn't mean I don't want feedback here. I will
participate here.
It's just that I will deal with it in batches, with a significant lag,
and given the way these discussions tend to go that means that others
will have to carry much of the more interactive load.
I'll write more about the more technical / practical aspects of your
mail at a later point.
Thanks,
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Bug#1050001: Unwinding the directory aliasing mistake
Hi Ian, While I have a CTTE hat, this response should be considered a Freexian/personal response rather than an official CTTE response. On Fri, Aug 18, 2023 at 07:57:14AM +0100, Ian Jackson wrote: > Thanks to work funded by Freexian we now have a list of many of these > malfunctions[2] (although new ones keep popping up (eg [3]). Let me point out that the DEP17 document referenced as [3] now has a proposal section. Let me also point out that as a result of this work, we've started systematically addressing limitations e.g.: https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=debian-qa%40lists.debian.org=fileconflict https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg%40debian.org=dep17p2 https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg%40debian.org=dep17p6 https://salsa.debian.org/systemd-team/systemd/-/merge_requests/204 https://salsa.debian.org/systemd-team/systemd/-/merge_requests/203 https://salsa.debian.org/systemd-team/systemd/-/merge_requests/205 https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/93 https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/96 https://salsa.debian.org/installer-team/debian-installer/-/merge_requests/39 > The most serious malfunction is the disappearing files bug, which has > prompted a seriously inconvenient file move moratoriam which has now > been in place for many years. After 4 years, we still don't have a > clear path forward to resolving that and other problems [4]. I disagree that this is the most serious malfunction. It was the first observed one, but the resulting moratorium has prevented us from observing others. As we start moving files, we'll be observing those others. > It should now be clear to most observers that the decision to go down > this path was a mistake.[5] I have not been part of that decision back then and I kinda feel similar, but keep in mind that our knowledge now is different from that back then. I thought that I'd know a lot about the /usr-merge back then and in working on it, I learn again and again that I still don't see the whole picture. It appears to me that you call it a mistake now as we progress with seeing that picture and finding a way out. > Fixing the mistake > > I think we can probably fix it by backing out this change, and then > doing usrmerge the traditional Debian way by making changes to > debhelper, so that we move the files package by package, in the .debs. There is two sides to this. One is that backing out the change is something that sounds easy but in reality is not. This keeps popping up as a suggestion and I've even suggested it myself about a year ago, but to this date, we don't have a reliable way to unmerge a system. So if we consider backing it out, we must consider the amount of work our users have to go through and that's a huge cost. The other side is that there may have been better ways of doing the migration. It's an academic question that I've given thoughts to as well. While there is something to learn from here, this doesn't really give us more options as the cost of reversion is huge. > But given the atmosphere, such a proposal would need clear political > backing from the TC. It wouldn't be worth anyone's time or emotional > energy to attempt to make a concrete and detailed plan if the TC is > not minded to consider it. I disagree. While the atmosphere has been hot at times and we had to invoke DAM more than once, I also think that progress is happening. The d-devel@l.d.o discussion has resulted in quite a few consensus items already and that's recorded in the latest DEP17 draft. So what you think not to be worth anyone's time is actually happening right now. > So I would like to pose the following questions for the TC: > > * Given the information that the Committee has now, that it didn't >have in 2019, does the Commitee agree that the decision in 2019 >was questionable ? All decisions are made with limited information. I don't think this adds any value. > * Is the Committee open to the idea of a plan being developed which >reverses the mistake, rather than merely "mitigating" the problems ? >Would such a plan be considered on its merits ? Anyone is entitled to propose a way forward. Such work is considered design work and therefore not carried out by the CTTE itself. I caution though that the amount of analysis work to be done here is non-trivial. For one thing, we'd need a convincing argument that there is a non-painful method to reverse the layout. For another, we need an understanding of how the resulting breakage to packages is being dealt with. For instance, cron no longer has /bin in its search path (#1042894, thanks to pabs for pointing this out). And then we need to compare it with DEP17 to see which of the plans is favourable. > I appreciate that I'm asking the TC to revisit a previous and > controversial decision. That this isn't a step we should take > lightly, but I think in this case it's
Bug#1050001: Unwinding the directory aliasing mistake
On 18/08/2023 09:05, Christoph Berg wrote: Re: Ian Jackson Protecting my mental health I will try to avoid regularly reading this thread. I hope that now that I have made the suggestion, others will be able to carry the conversation. I will be configuring my mail client to disregard my personal copies of messages sent to this bug; when I want to read the thread I will look at the mailing list. Also, if you disagree with my decision to raise this now, please don't email me. If you feel I have overstepped a boundary, please contact the Community Team or DAM. I think this is just gross. Submitting a proposal and then telling CT/DAM to deal with the fallout is rude. I disagree; the TC has handled a number of issues now where at least one of the main participants has declined to participate in the discussion thread. It's obviously not ideal, but I don't think it's entirely out of order (and the TC has not considered it as such in the past). Regards, Matthew Disclaimer: I know Ian personally, but I don't think that's affecting my opinion here
Bug#1050001: Unwinding the directory aliasing mistake
Re: Ian Jackson > Protecting my mental health > > I will try to avoid regularly reading this thread. I hope that now > that I have made the suggestion, others will be able to carry the > conversation. I will be configuring my mail client to disregard my > personal copies of messages sent to this bug; when I want to read > the thread I will look at the mailing list. > > Also, if you disagree with my decision to raise this now, please don't > email me. If you feel I have overstepped a boundary, please contact > the Community Team or DAM. I think this is just gross. Submitting a proposal and then telling CT/DAM to deal with the fallout is rude. Christoph
Bug#1050001: Unwinding the directory aliasing mistake
Package: tech-ctte Background and current status In 2019 the TC decided[1] that Debian would achieve the largely-agreed goal of having only one place to put most files, /usr, by using symlinks to alias from /bin etc. to e.g. /usr/bin. At the time, concerns were raised that package management systems and other software would malfunction but the set of malfunctions was not enumerated; proponents of the aliasing approach characterised them as FUD. In the absence of the results of the research work which has now been done, that characterisation seems to have been implicitly accepted as true by the then TC. Thanks to work funded by Freexian we now have a list of many of these malfunctions[2] (although new ones keep popping up (eg [3]). The most serious malfunction is the disappearing files bug, which has prompted a seriously inconvenient file move moratoriam which has now been in place for many years. After 4 years, we still don't have a clear path forward to resolving that and other problems [4]. It should now be clear to most observers that the decision to go down this path was a mistake.[5] Fixing the mistake I think we can probably fix it by backing out this change, and then doing usrmerge the traditional Debian way by making changes to debhelper, so that we move the files package by package, in the .debs. But given the atmosphere, such a proposal would need clear political backing from the TC. It wouldn't be worth anyone's time or emotional energy to attempt to make a concrete and detailed plan if the TC is not minded to consider it. So I would like to pose the following questions for the TC: * Given the information that the Committee has now, that it didn't have in 2019, does the Commitee agree that the decision in 2019 was questionable ? * Is the Committee open to the idea of a plan being developed which reverses the mistake, rather than merely "mitigating" the problems ? Would such a plan be considered on its merits ? I appreciate that I'm asking the TC to revisit a previous and controversial decision. That this isn't a step we should take lightly, but I think in this case it's clearly warranted. Timeline for a fix Any plan to solve this would probably take a few releases (ie, many more years) to sort out, sadly. We would probably need to wait with shipping packages that move files in a conventional-for-Debian way, until we have our userbase's systems restored to a non-aliased state. So I think we would need trixie to undo the aliasing, and in trixie+1 we could move the files. This delay is unfortunate, but - unlike pressing forward - it has relatively low levels of risk, most of which is front-loaded. I think we can develop tools which will reliably de-alias a system; and, once users' systems have been de-aliased, the actual file movement is routine work that Debian knows how to do. I can see that there could possibly be ways of going straight to the desired state (un-aliased systems but with nothing much in /), but I haven't given them much thought them through because they seem risky to me and involve some grievous hacks. Protecting my mental health I will try to avoid regularly reading this thread. I hope that now that I have made the suggestion, others will be able to carry the conversation. I will be configuring my mail client to disregard my personal copies of messages sent to this bug; when I want to read the thread I will look at the mailing list. Also, if you disagree with my decision to raise this now, please don't email me. If you feel I have overstepped a boundary, please contact the Community Team or DAM. If the Comittee gives a positive indication, I will be happy to re-engage at the level of technical work to try to make it happen. Thanks, Ian. [1] https://lists.debian.org/debian-devel-announce/2019/03/msg1.html [2] https://subdivi.de/~helmut/dep17.html [3] https://lists.debian.org/debian-devel/2023/05/msg00311.html [4] https://lists.debian.org/debian-devel/2023/06/msg00353.html [5] Perhaps merged-usr via directory aliasing worked well in some other distros with less sophisticated package management systems. But we obtained almost all the practical benefits of abolishing the distinction between / and /usr very early, by deciding that the initramfs would mount /usr too. We have inflicted all this pain and confusion on ourselves simply to do some tidying up. The result has been the opposite. If we had just moved the files rather than trying to rush things with the directory alias symlinks, we would have been finished by now.
