Re: Bug#1051237: transition: move files from / to /usr to finalize /usr-merge

2024-03-09 Thread Sebastian Ramacher
Hi

On 2024-03-09 21:39:50 +0100, Helmut Grohne wrote:
> Hi release team and essential maintainers,
> 
> On Mon, Sep 04, 2023 at 10:33:54PM +0200, Helmut Grohne wrote:
> > Once these issues have been resolved, we can move most files except for
> > a small set of essential packages. For those, a coordinated upload
> > moving their files will be needed as will be an upload of base-files
> > adding the aliasing symlinks there.
> 
> We're well into this now. Most of the essential set is moved and I've
> most of the remaining pieces. I hope that within one week, we're left
> with only:
>  - base-files
>  - bash
>  - dash
>  - glibc
>  - util-linux
> 
> Patches for these have been prepared. The current patches are available
> from https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. These
> changes have been uploaded to Ubuntu noble already and feedback has been
> incorporated. In particular, base-files will now divert to dot files to
> avoid cluttering the / view during the transition and base-files will
> remove unnecessary diversions (those where it ships symlinks).
> 
> I'd now like to coordinate a time of upload. Given that chroots are
> rebuilt in Wednesday and Sunday, I suggest we pick a Thursday morning
> for the actual upload. If I unexpectedly break stuff, I still have a few
> days to fix. How about March 21st?

If and only if time64_t is done by then. Adding more changes where
transition has to be coordinated to the ongoing mega transition does not
help.

Cheers
-- 
Sebastian Ramacher



Re: transition: move files from / to /usr to finalize /usr-merge

2024-03-09 Thread Helmut Grohne
Hi release team and essential maintainers,

On Mon, Sep 04, 2023 at 10:33:54PM +0200, Helmut Grohne wrote:
> Once these issues have been resolved, we can move most files except for
> a small set of essential packages. For those, a coordinated upload
> moving their files will be needed as will be an upload of base-files
> adding the aliasing symlinks there.

We're well into this now. Most of the essential set is moved and I've
most of the remaining pieces. I hope that within one week, we're left
with only:
 - base-files
 - bash
 - dash
 - glibc
 - util-linux

Patches for these have been prepared. The current patches are available
from https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. These
changes have been uploaded to Ubuntu noble already and feedback has been
incorporated. In particular, base-files will now divert to dot files to
avoid cluttering the / view during the transition and base-files will
remove unnecessary diversions (those where it ships symlinks).

I'd now like to coordinate a time of upload. Given that chroots are
rebuilt in Wednesday and Sunday, I suggest we pick a Thursday morning
for the actual upload. If I unexpectedly break stuff, I still have a few
days to fix. How about March 21st?

Once this has uploaded, we need to ensure that these packages migrate
together. Also note that dash's autopkgtest will fail unless it uses the
updated base-files, but it cannot depend on base-files. If you prefer, I
can mark the relevant test case as flaky and unmark it in a second
upload. Having a temporary migration block on these packages would also
be a good idea.

Once agreed, I shall announce this change to d-d-a as I cannot fully
rule out it being disruptive despite the extensive testing performed.

> We probably have to use NMUs to convert remaining packages at this
> point. Once everything is moved, we may think we're done, but we're not.

Speaking of the rest of packages. At the time of this writing, the
numbers are:
 * 224 source packages can be moved via dh-sequence-movetousr.
 * 191 source packages have a dep17 usertagged bug report (most with
   patches).
 * 141 source packages can be moved with a no-change sourceful upload.
   This is about Arch:all packages as we already binNMUed the others.
 * 35 source packages cannot be analyzed, because they FTBFS (reported).
 * A 1-digit number of packages (e.g. the bootstrap ones above) needs
   special handling, but this is communicated and monitored.

I hope that these numbers go down moving forward (especially the patches
one). At some point, I want to mass-NMU the remaining packages.

> As packages are restructured throughout the release cycle, they may
> introduce file loss scenarios. Continued monitoring for problems until
> trixie is released is crucial.

The biggest chunk of findings was due to time64. I think the reports are
timely and actionable. Generally, I hope that we'll run into less
fallout moving forward as the "big" stuff is being handled. One
exception here is that time64 has introduced a pile of "risky replaces".
These are not accounted as buggy in the above numbers but need to be
addressed before we can mass-NMU. That'll happen once the dust settles
on time64.

Any objections so far?

Helmut