Bug#1020792: tech-ctte: Halt merged-/usr transition until dpkg filesystem damage bugs are fixed
On Mon, Sep 26, 2022, at 4:30 PM, Sean Whitton wrote: > I believe that this request is invalid, for two reasons: > > - the specific things you ask for are all or mostly things that we think > are currently up to the Release Team, and the TC cannot override > delegates I'm surprised to hear you say that, given that, in the past, the TC has required bugs of various severities to be filed, and has required maintainers not to alter bug severities. Almost all of what I'm asking for would follow "by operation of Policy", as it were, from the requested s:critical bugs on usrmerge and usr-is-merged; I only spelled them out for explicitness's sake. And I didn't file the bugs myself because they would certainly be rejected by the maintainers, and then I'd have to escalate _that_ to you, so I'm trying to save time by skipping that step. > - you might be lacking the full context of TC-involving discussions over > the past few months, but so far as I can see, you are asking for us to > undo a decision that we only just made, which doesn't make sense. As far as _I_ am aware, I'm acting on this bit of the "more specific advice re #978636", issued last year: # If a suitable transition mechanism is not available by the time the # Debian 12 freeze is reached, the Technical Committee will rescind our # advice in #978636 and modify our advice in this resolution to reflect # the situation that has been reached. We hope that this will not be # necessary. In my opinion, a "suitable transition mechanism" _must_ include a fix for the dpkg bugs, and it is almost certain at this point that the dpkg bugs will _not_ be fixed in time for the freeze, hence it is appropriate at this point for the TC to "rescind our advice in #978636 and modify our advice in this resolution", along the lines I requested. If there has been substantive discussion or progress on the dpkg bugs "over the past few months" I have not seen it either on -ctte or on -devel. I would appreciate a concrete summary of what has happened so far and what is planned to happen. zw
Bug#1020792: tech-ctte: Halt merged-/usr transition until dpkg filesystem damage bugs are fixed
> "Sean" == Sean Whitton writes: Sean> - you might be lacking the full context of TC-involving Sean> discussions over the past few months, but so far as I can see, Sean> you are asking for us to undo a decision that we only just Sean> made, which doesn't make sense. Sean, as someone who has tried to follow this for a while, I'm confused by a third thing. Doesn't the TC recommendation given additional weight by the RT that we not migrate file paths until the dpkg bug has been fixed make it impossible for the pre-conditions for disappearing files to happen? I'd like to make sure that the bug submitter has not identified something new here. >Also, to the bug submitter: >needs to be fixed urgently. Several other equally dire scenarios >are listed in https://wiki.debian.org/Teams/Dpkg/MergedUsr under the >“merged-/usr-via-aliased-dirs” subheading, albeit only tersely.) I think that if you are going to ask for drastic action like you are doing, a turse mention on a wiki page is not sufficient documentation. Simon's mail got significant attention because he actually worked through and documented the situation. I and I believe several people who have looked at this situation believe that the RT guidance will prevent the issue Simon raised from happening. I suspect that if someone were to clearly document a different situation that had not previously been considered, people would look at it. I'd hope the TC would be open to considering newly documented technical problems as an example. But because the decision has been made, the bar for blocking things is a lot higher at this point than turse descriptions on a wiki page.
Bug#1020792: tech-ctte: Halt merged-/usr transition until dpkg filesystem damage bugs are fixed
Hello, On Mon 26 Sep 2022 at 03:28PM -04, Zack Weinberg wrote: > Package: tech-ctte > Severity: normal > X-Debbugs-Cc: z...@owlfolio.org > > I formally request that the Technical Committee call a halt to the > merged-/usr transition until such time as all of the bugs in dpkg that > can, on a merged-/usr system, cause damage to the contents of the > filesystem (e.g. packaged files being silently deleted on upgrade) > have been fixed to the satisfaction of the dpkg maintainers. I believe that this request is invalid, for two reasons: - the specific things you ask for are all or mostly things that we think are currently up to the Release Team, and the TC cannot override delegates - you might be lacking the full context of TC-involving discussions over the past few months, but so far as I can see, you are asking for us to undo a decision that we only just made, which doesn't make sense. Therefore, we will likely just close this bug, I'm afraid. -- Sean Whitton signature.asc Description: PGP signature
Bug#1020792: tech-ctte: Halt merged-/usr transition until dpkg filesystem damage bugs are fixed
Package: tech-ctte
Severity: normal
X-Debbugs-Cc: z...@owlfolio.org
I formally request that the Technical Committee call a halt to the
merged-/usr transition until such time as all of the bugs in dpkg that
can, on a merged-/usr system, cause damage to the contents of the
filesystem (e.g. packaged files being silently deleted on upgrade)
have been fixed to the satisfaction of the dpkg maintainers.
## Background
It has been known for some time that dpkg has bugs which prevent it
from correctly handling merged-/usr systems. #858331/#848622 is the
only such bug (that I can find) that has actually been recorded in the
BTS, and it *appears* to be a relatively harmless problem, affecting
only dpkg-query output.
However, Simon Richter showed in
https://lists.debian.org/debian-devel/2021/08/msg00326.html that the
bad dpkg-query output is only the most obvious symptom of a much more
serious problem, and that, under conditions that will plausibly occur
in the archive after the release of bookworm (assuming all continues
as presently planned) upgrading packages on a merged-/usr system can
cause packaged files to disappear from the filesystem. The files most
likely to be affected are the files that are currently packaged in
/bin, /sbin, and /lib, including, just to mention a few, /bin/bash,
/bin/systemd, and /lib/$ARCH/libc.so.6. Thus, the dpkg bugs can
render systems unbootable on upgrade, and should be considered
critical severity.
(N.B. Simon’s message is the only thing I can find that gives
step-by-step instructions to reproduce a problem that could plausibly
cause catastrophic filesystem damage during package upgrades, but the
scenario it describes should _not_ be considered the only problem that
needs to be fixed urgently. Several other equally dire scenarios
are listed in https://wiki.debian.org/Teams/Dpkg/MergedUsr under the
“merged-/usr-via-aliased-dirs” subheading, albeit only tersely.)
Despite the severity of the dpkg bugs having been known since
_at least_ August of 2021, the people working on the merged-/usr
transition have made almost no effort to fix them. A patch exists
(https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=858331;filename=file_move_moratorium.patch;msg=46)
that at least partially addresses the problem, but only one desultory
attempt was made to get it merged.
The people working on the merged-/usr transition have repeatedly
claimed that the several years of experience Debian now has with
systems that were _originally installed_ with merged /usr, and the
somewhat longer baseline for Ubuntu doing the same thing, indicates
the dpkg bugs are not a problem in practice. That claim is incorrect.
The dpkg bugs are *latent* right now because, until the actual release
of bookworm, packages have to continue supporting non-merged systems.
Therefore they cannot move files from /{bin,sbin,lib} to
/usr/{bin,sbin,lib}, which is one of the conditions required to
trigger the most serious known issue (disappearance of packaged files).
I anticipate that if the merged-/usr transition proceeds as planned,
*all systems* that track either testing or unstable will be rendered
unbootable at least once in the bookworm+1 development cycle. This
level of fallout is not acceptable, even in potentiality. Since it
seems clear that the people working on the merged-/usr transition have
no intention of getting the bugs fixed, project-level action is
required.
## Specific actions requested
I ask the Committee to require all of the following, partially
reversing the decision taken in #978636, and overriding maintainers
as necessary:
- The usrmerge and usr-is-merged packages are to be removed from
testing immediately, and severity:critical (justification: “makes
unrelated software break” and/or “breaks the entire system”) bugs
are to be filed to prevent them from re-entering testing.
- Those bugs may not be closed, nor may their severity be lowered,
until *the dpkg maintainers agree* that dpkg can now fully support
merged-/usr systems.
- No package may declare a dependency on either usrmerge or
usr-is-merged until the respective bug is closed. Any packages in
the archive that currently declare such a dependency must drop it
immediately (or else be removed from testing as well).
- No *other* mechanism which converts existing non-merged-/usr
installations to merged-/usr, as a side-effect of package upgrade
or installation, may enter testing, until at least the usr-is-merged
bug is closed. [I can imagine at least one possible resolution
which would make it safe to use dpkg on a system where /usr has
been merged ever since installation, but *not* on a system that was
converted the way the usrmerge package does it.]
- If merged-/usr conversion fails to reach testing before the release
freeze for bookworm, then bookworm shall continue to support
non-merged /usr for its lifetime, and similarly for future
(post-bookworm) releases.
- Optionally:
