Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Luca Boccassi]

On Thu, 2021-07-22 at 14:04 +0200, Marco d'Itri wrote:
> On Jul 22, "Barak A. Pearlmutter"  wrote:
> 
> > People seem pretty worked up over this, but honestly I'm not
> > understanding why.
> Because they are scared by new things, hence they oppose merged-/usr 
> (and systemd before that, and udev even before...).
> But since they cannot admit that, then they need to rationalize by 
> inflating the effects of minor bugs or usability issues.
> 
> Basically all new systems are installed with merged-/usr since Debian 
> 10, and since obviously this has not caused noticeable issues (or at 
> least, any which could not be fixed) then we must treat with skepticism 
> any such claims.

Not only all Debian systems, but all Ubuntu systems since 2019 (iirc)
and other derivatives too. Still waiting for the sky to fall.

-- 
Kind regards,
Luca Boccassi


signature.asc
Description: This is a digitally signed message part

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Wouter Verhelst]

On Mon, Jul 19, 2021 at 03:10:42PM +0200, Michael Biebl wrote:
> Hi Guillem
> 
> Am 19.07.21 um 03:36 schrieb Guillem Jover:
> > What I've also said multiple times, is that
> > merged-usr-via-moves-and-symlink-farms could have been implemented in
> > a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> > all with full cooperation and managed by dpkg, with .debs shipping
> > actual tracked pathnames
> I'm convinced this view is way too naive and not implementable in practice
> (and yes, openSUSE is a data point that confirms that)
> 
> What you propose is, that each and every package does its /usr-merge
> transition on its own. This only works, if packages are independent (enough)
> so this actually works.
> Unfortunately this is not the case. Take PAM for example. You can't just
> recompile src:pam and have debhelper automatically move all files to /usr.
> This would break all packages that install a PAM plugin. You have a
> transition here, involving many packages.

Why?

Nobody is saying the old path should cease to function. The whole point
of a symlink farm is that *YOU ADD A SYMLINK* to replace the old path.

So then you have /lib/*/security/pam_foo.so ->
/usr/lib/*/security/pam_foo.so and your old-pam plugin will still work
with new-pam (and vice versa) and there is no need for a transition.

I've suggested previously that we can easily make it RC for bookworm to
have a file outside a limited set of directories (/etc and /usr would be
OK, but notably /bin /lib and /sbin wouldn't be) that is not a symlink.
This is easy to detect with a lintian check and reasonably easy to
implement, and would not confuse dpkg *at all*.

But whenever I bring this up, I hear people say "oh but suse tried it
and failed" (well suse aren't using dpkg and there's no reason to assume
we'll have the same problem, why don't we try?) or "oh but the /usr/doc
transition that worked that way 20 years ago took forever" (that was 20
years ago, our tooling is way more advanced these days) or "oh but that
will break bash and you can't upgrade safely without bash" (true, but
bash is just the one package and we already have /bin/sh be a symlink
and that never made upgrades fail permanently so I don't see how
usrmerge is somehow special).

I've grown tired of the whole discussion, and the "we must go forward
and only our way will work and your ideas are stupid just shut up
already" mentality the proponents of usrmerge seem to have.

I can understand the use case for usrmerge, and I won't cry over my
/bin/sh being essentially the same file as /usr/bin/sh -- but I long for
the good old days of Debian where we did things the right way, not
whichever is the fastest, because that way, things would *work* in *all*
cases, not just the cases that the proponents of some new feature care
about.

It took us forever to implement the /usr/doc transition, but it was
finished and nobody's machine broke.

It took us a fairly large time to implement multiarch, but we did it and
it works *way* better than in the RPM world.

I fail to see why usrmerge is so special that it can't wait until we do
things the right way.

Sure, there are technical issues with doing things the right way, and we
should deal with them. But just throwing them under the carpet and
deciding they're only a problem for other people isn't going to help
anyone.

-- 
 w@uter.{be,co.za}
wouter@{grep.be,fosdem.org,debian.org}


signature.asc
Description: PGP signature

Re: Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Guillem Jover]

[ Barak, I appreciate your mail, but *sigh*, it's still frustrating,
  as pretty much many of the mails related to this, as they keep
  ignoring what has been said, and I feel like talking to a wall. :/ ]

On Thu, 2021-07-22 at 11:48:34 +0100, Barak A. Pearlmutter wrote:
> Seriously? Being able to type
> 
> dpkg -S $(which cat)
> 
> instead of
> 
> dpkg -S $(which cat|sed 's:^/usr::')
> 
> is the big user-level pain point?

No. As I've mentioned before, f.ex.:

 * dpkg, dpkg-divert, or update-alternatives are unable to detect file
   conflicts and thus might allow silent overwrites of random stuff on
   disk (3rd-party/local packages where we have no control over, or
   even packages from old Debian releases come to mind),
 * when moving files across packages and across aliased directories,
   these pathnames might end up disappearing depending on the unpack
   order (new/old Debian or 3rd-party/local packages too),
 * dpkg-deb -x on the root directory (yes, people use this to recover
   systems) with any .deb that contains files on real directories under
   «/» (3rd-party/local packages), will replace the symlinked directories
   with real ones,
 * dpkg-statoverride used with aliased pathnames that exist on disk
   but unknown to dpkg, will fail to apply the overrides (itself and
   dpkg on unpack), this could have security implications f.ex.,
 * dpkg-triggers used with aliased pathnames that exist on disk but
   unknown to dpkg, will fail to activate triggers,

Also, yes, anything using dpkg-query is now broken, take for example the
cruft package, the problem is that there are way more things depending
on this interface.

But the aliasing problem is a general one. Take the /etc/shells
mentioned recently on d-d, the admin needs to remember to always add
two entries (/ and /usr) to make sure things will match by any random
program accessing the file for its checks.


Regarding your dpkg-query example above, using which(1) and assuming
a PATH set to prefer /usr directories (which AFAIR is the current
default) would mean that once all objects have moved in the .debs into
/usr, the former would always work, but at that point the latter would
not. Also that assumes people would use which(1) or similar, or even
fully canonicalize the pathname, which might or might not have anything
to do with what dpkg knows about on its database.

> People seem pretty worked up over this, but honestly I'm not
> understanding why. We already have $PATH which (let's be honest) is an
> ancient crappy workaround for the original Unix sin of not keeping all
> the executables in one place. (And analogous wheel-reinventing goo for
> /lib vs /usr/lib vs /usr/lib/x86_64-linux-gnu, etc.) Given that,
> moving them around isn't supposed to be a big fuss. Oh, but there are
> also shebang #!/bin/foo issues and other hard coding. The shebang is
> already such a mess that people use #!/usr/bin/env foo, just to get
> foo searched in $PATH. (85 of the 890 scripts in /usr/bin/* on the
> machine I'm typing this on use a /usr/bin/env trampoline.) Nobody
> would ever consider having /bin/foo and /usr/bin/foo be different
> programs, that would be madness. The TC basically concluded that the
> distinction between /bin and /usr/bin, etc, had totally broken down
> and there's no advantage to keeping them distinct, plus initrd is the
> new /bin. (Pretty much the same argument as in the design of plan9.)
> I'm not seeing much argument against that, except for nostalgia.

In case this is not clear, and as I've mentioned also countless times
already, I have no problem with merging contents of directories from /
into /usr, I've done that for all packages I maintain (where no compat
symlinks were required). I do have a huge problem with the approach
that has been forced into the distribution disregarding the packaging
system and going on its back, though.

You might want to take a peek at
.


Can any of the above be avoided? Well certainly, in the same way you
can probably cross a mine field safely, you'll just need to always be
aware of any of these things, and verify everything you install, and
all pathnames you use, etc. I'm not sure how this is in any reasonable
universe an acceptable way to expect end users to use the system TBH.


Thanks,
Guillem

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marco d'Itri]

On Jul 22, "Barak A. Pearlmutter"  wrote:

> People seem pretty worked up over this, but honestly I'm not
> understanding why.
Because they are scared by new things, hence they oppose merged-/usr 
(and systemd before that, and udev even before...).
But since they cannot admit that, then they need to rationalize by 
inflating the effects of minor bugs or usability issues.

Basically all new systems are installed with merged-/usr since Debian 
10, and since obviously this has not caused noticeable issues (or at 
least, any which could not be fixed) then we must treat with skepticism 
any such claims.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Barak A. Pearlmutter]

Seriously? Being able to type

dpkg -S $(which cat)

instead of

dpkg -S $(which cat|sed 's:^/usr::')

is the big user-level pain point?

People seem pretty worked up over this, but honestly I'm not
understanding why. We already have $PATH which (let's be honest) is an
ancient crappy workaround for the original Unix sin of not keeping all
the executables in one place. (And analogous wheel-reinventing goo for
/lib vs /usr/lib vs /usr/lib/x86_64-linux-gnu, etc.) Given that,
moving them around isn't supposed to be a big fuss. Oh, but there are
also shebang #!/bin/foo issues and other hard coding. The shebang is
already such a mess that people use #!/usr/bin/env foo, just to get
foo searched in $PATH. (85 of the 890 scripts in /usr/bin/* on the
machine I'm typing this on use a /usr/bin/env trampoline.) Nobody
would ever consider having /bin/foo and /usr/bin/foo be different
programs, that would be madness. The TC basically concluded that the
distinction between /bin and /usr/bin, etc, had totally broken down
and there's no advantage to keeping them distinct, plus initrd is the
new /bin. (Pretty much the same argument as in the design of plan9.)
I'm not seeing much argument against that, except for nostalgia.

It seems that dpkg is ground zero for problems that occur during the
transition itself, and for the problem of handling the transition
gracefully. I for one am enormously impressed with the quality of
dpkg: its amazing robustness, the way it's evolved to support hooks
and such and really been the cornerstone of the distribution. So I
think we should take the dpkg maintainer's concerns seriously. But
frankly I don't quite understand them.

--Barak Pearlmutter.

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marc Haber]

On Tue, 20 Jul 2021 23:15:33 +0200, Svante Signell
 wrote:
>It is really stunning that the Debian project, including the TC
>overrides the dpkg developer and maintainer Guillem, and still using
>dpkg for package management. Maybe Debian should switch to some other
>software, like rpm-based used by Fedora or even guix used by GNU??

This suggestion doesnt get any smart by repeating it.

-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-20 10:07 p.m., Brian Thompson wrote:
> On Tue, 2021-07-20 at 21:13 -0400, Polyna-Maude Racicot-Summerside
> wrote:
>> Ended up with a 3 month useless discussion regarding if this would
>> give
>> a bad impression, that we need to use node for doing development.
>> Later on I was working on a plugin that treated huge amount of data.
>> So
>> I introduced Vue.JS, again a three month discussion with people
>> saying
>> it's a overhead, even after explaining that you can't always do
>> server
>> side processing and serve all the data at a time. Even if people
>> didn't
>> understand a word about the global concept and the system as a whole,
>> they halted the project.
>> So I just let go my participation.
>> Now 4 years later, they are integrating Vue.JS and Node doing code
>> validation while development.
>>
>> One of the main reason some people don't want to invest time in FOSS
>> project is exactly because of that type of toxic situation that make
>> everyone look like crazy nut head.
> 
> I find it pretty crazy that you think FOSS is toxic, when in reality
> the only toxic thing about this situation was when you labeled a
> discussion as "useless".  They didn't want to use your technology stack
> four years ago.  Now they do.  Throwing a temper tantrum when you don't
> get your way isn't doing anyone else any favors. 
> 
You either cherrypicked or only took what you wanted in what I said.

So I'll do it again...

I was asked by one of the project manager to implement a new plugin that
would allow some machine learning and data processing (using chart with
plotly.js using database on a CMS).
To do so efficiently, I had to use some type of client side processing
and decided to use Vue.JS.

This didn't change nothing for anyone except me.

But all the developers started saying it wasn't good because it would
project the idea that we need to use Node to do development (false) or
that Vue.JS is needed for the CMS (false).

Most of the person talking about this were people who didn't have a clue
what they were talking about. Mostly people doing translation for text
string, HTML/CSS front end themes and such. Most of the time, when we
talked about PHP they would say "that they ain't developer".

For you information, I didn't throw a tamper tantrum, I simply felt that
my time would be better invested somewhere I could use my time in a
positive manner.

Having to explain that "It's not because there's a package.json file
that you'll need Node ecosystem. As there's already a .gitla-ci.yml and
you don't need to have a Gitlab runner on your PC to do development".
Having to explain that sending all the data at one time, when processing
thousands of record is not a possibility if you want to have a system
that can be scaled up. And much more...
This is time consuming and doesn't help much going a project forward.

It wasn't a technology stack that I've chosen but the one that was
requested by the project manager who paid me. But I had to deal with
those questions on the mailing list.

And I did get tired so went somewhere else...

So we we're in front of people that (like many) had a opinion but didn't
have much to support it.

I don't know many business we're we pass more time discussing about
solution than implementing them. Unless, I got it wrong, and FOSS
project are some type of IT consultant box ?

I come from health science and we do stuff, don't smoke butterfly powder
thinking about what would be the best in a ideal world. I don't think
about what's the best solution for a ideal world.

And if I was running Windows for the computer used for medical purpose
in my office, there was one reason : I didn't have nothing that made if
efficient for me to go Linux. All the advantage were overwhelmed by the
time consuming of process changing, staff training or simply because no
such solution existed.

If all the time wasted in fork was used to making project grow better
then it'd be long ago that we wouldn't see as many Windows.

> P.S. You may be the biggest hippocrit in these mailing lists.
> 
You don't have a clue who I am...
So let me put you in the box where it says "I talk about what I don't
know and think I'm s great".

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Brian Thompson]

On Tue, 2021-07-20 at 21:13 -0400, Polyna-Maude Racicot-Summerside
wrote:
> Ended up with a 3 month useless discussion regarding if this would
> give
> a bad impression, that we need to use node for doing development.
> Later on I was working on a plugin that treated huge amount of data.
> So
> I introduced Vue.JS, again a three month discussion with people
> saying
> it's a overhead, even after explaining that you can't always do
> server
> side processing and serve all the data at a time. Even if people
> didn't
> understand a word about the global concept and the system as a whole,
> they halted the project.
> So I just let go my participation.
> Now 4 years later, they are integrating Vue.JS and Node doing code
> validation while development.
> 
> One of the main reason some people don't want to invest time in FOSS
> project is exactly because of that type of toxic situation that make
> everyone look like crazy nut head.

I find it pretty crazy that you think FOSS is toxic, when in reality
the only toxic thing about this situation was when you labeled a
discussion as "useless".  They didn't want to use your technology stack
four years ago.  Now they do.  Throwing a temper tantrum when you don't
get your way isn't doing anyone else any favors. 

P.S. You may be the biggest hippocrit in these mailing lists.

-- 
BT


signature.asc
Description: This is a digitally signed message part

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

> Hi Holger, I would have expected a reply like this from you. I do still
> use Debian, some of my boxes are still Debian-based. Soon they will
> probably be converted to Devuan though. I do still contribute to
> Debian, mainly to Debian GNU/Hurd and Debian GNU/kFreeBSD. As long as
> these ports are still alive within Debian I won't go away.
> 

I'd love to know more about what made you switch to Devuan and what
could I also benefit from using Debian/kFreeBSD.
I used to have boxes on FreeBSD long time ago (6.0 I think).

Thanks

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-20 5:51 p.m., Holger Levsen wrote:
> On Tue, Jul 20, 2021 at 11:15:33PM +0200, Svante Signell wrote:
> [...]
>> Debian, the Universal Operating System was used some years ago!
> 
> Svante, fine. You are unhappy with Debian since years, you're not using it
> anymore, you are not contributing, this is debian-devel@ not debian-rant@,
> so please STFU.
> 
You are so nice with people who have a different opinion than yours.
This is really a example of someone who's mature.
> 

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

> It is really stunning that the Debian project, including the TC
> overrides the dpkg developer and maintainer Guillem, and still using
> dpkg for package management. Maybe Debian should switch to some other
> software, like rpm-based used by Fedora or even guix used by GNU?? Or
> perhaps the Debian project should come to senses with the force of
> things upon users and developers/maintainers without their approval
> (consensus-free)? 
> 
> Debian, the Universal Operating System was used some years ago!
> 

It's really the kind of problem that make Debian look like a crazy non
sense operating system. People using program options as a way to shout
their opinion to the world...
It's a problem with many FOSS project that benefits mainly from
volunteer work. Because they need to job done, they can't have any
coercion against developer (as a difference, your boss can impose you
things). This may be a good thing as it promote creativity and sometime
allow emergence of better practice.
But in other case, most of the time, it just give a bad look.

Reminds me of when I worked with the Tiki CMS project. I was using node
with some tools to do pre-processing of JS code and HTML validation. So
I added my package.json to the distribution, in case other developer
want it.
Ended up with a 3 month useless discussion regarding if this would give
a bad impression, that we need to use node for doing development.
Later on I was working on a plugin that treated huge amount of data. So
I introduced Vue.JS, again a three month discussion with people saying
it's a overhead, even after explaining that you can't always do server
side processing and serve all the data at a time. Even if people didn't
understand a word about the global concept and the system as a whole,
they halted the project.
So I just let go my participation.
Now 4 years later, they are integrating Vue.JS and Node doing code
validation while development.

One of the main reason some people don't want to invest time in FOSS
project is exactly because of that type of toxic situation that make
everyone look like crazy nut head.

Sometime we also have to accept choices we dislike (hey, we teach this
to kids).

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Holger Levsen]

On Wed, Jul 21, 2021 at 12:09:52AM +0200, Svante Signell wrote:
> Hi Holger, I would have expected a reply like this from you. I do still
> use Debian, some of my boxes are still Debian-based. Soon they will
> probably be converted to Devuan though. I do still contribute to
> Debian, mainly to Debian GNU/Hurd and Debian GNU/kFreeBSD. 

ok, cool! & sorry for assuming you didn't contribute!

> Holger (and others), please consider the arguments and facts given in
> the latest postings to this thread, these issues are serious and not
> something you just can easily close with an STFU argument to me!
 
well, sigh, your last mail (to which I replied) was just ranting,
which I still think is wrong here. (Once ok, twice too/maybe, ...)

however, this ship has sailed, there has been a proper discussion by
a proper process and if you don't like it, please start a GR, as this
is the way to overrule the TC, if there is a majority for it.

writing mails to debian-devel will not change anything. and repeating
a discussion every other month without new arguments will just annoy
people as one can see here.


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Words may inspire but only action creates change.


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Svante Signell]

On Tue, 2021-07-20 at 21:51 +, Holger Levsen wrote:
> On Tue, Jul 20, 2021 at 11:15:33PM +0200, Svante Signell wrote:
> [...]
> > Debian, the Universal Operating System was used some years ago!
> 
> Svante, fine. You are unhappy with Debian since years, you're not
> using it anymore, you are not contributing, this is debian-devel@ not
> debian-rant@, so please STFU.

Hi Holger, I would have expected a reply like this from you. I do still
use Debian, some of my boxes are still Debian-based. Soon they will
probably be converted to Devuan though. I do still contribute to
Debian, mainly to Debian GNU/Hurd and Debian GNU/kFreeBSD. As long as
these ports are still alive within Debian I won't go away.

Holger (and others), please consider the arguments and facts given in
the latest postings to this thread, these issues are serious and not
something you just can easily close with an STFU argument to me!

Thanks!

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Holger Levsen]

On Tue, Jul 20, 2021 at 11:15:33PM +0200, Svante Signell wrote:
[...]
> Debian, the Universal Operating System was used some years ago!

Svante, fine. You are unhappy with Debian since years, you're not using it
anymore, you are not contributing, this is debian-devel@ not debian-rant@,
so please STFU.


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

The upcoming clima apocalypse is the big elephant in every room now.


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Svante Signell]

On Tue, 2021-07-20 at 15:34 -0400, Polyna-Maude Racicot-Summerside
wrote:
> Hi,
> 
> On 2021-07-20 3:30 p.m., Marc Haber wrote:
> > On Tue, 20 Jul 2021 14:47:04 +0200, Svante Signell
> >  wrote:
> > > According to the dpkg developer and maintainer Guillem users can
> > > still rescue their systems from merged-/usr-via-aliased-dirs with
> > > the aid of dpkg-fsys-usrunmess(8), see 
> > > https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
> > 
> > The naming of the utility alone gives me the impression that we
> > have a dpkg maintainer who has gone to war with the rest of the
> > distribution. I am not sure whether Debian should accept that.
> > 
> I think that the option "usrunmess" says pretty much the state of
> mind regarding the person who named it this way.
> 
> Something like "usrunmerge" would be more realistic.

It is really stunning that the Debian project, including the TC
overrides the dpkg developer and maintainer Guillem, and still using
dpkg for package management. Maybe Debian should switch to some other
software, like rpm-based used by Fedora or even guix used by GNU?? Or
perhaps the Debian project should come to senses with the force of
things upon users and developers/maintainers without their approval
(consensus-free)? 

Debian, the Universal Operating System was used some years ago!

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-20 3:30 p.m., Marc Haber wrote:
> On Tue, 20 Jul 2021 14:47:04 +0200, Svante Signell
>  wrote:
>> According to the dpkg developer and maintainer Guillem users can still
>> rescue their systems from merged-/usr-via-aliased-dirs with the aid of 
>> dpkg-fsys-usrunmess(8), see 
>> https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
> 
> The naming of the utility alone gives me the impression that we have a
> dpkg maintainer who has gone to war with the rest of the distribution.
> I am not sure whether Debian should accept that.
> 
I think that the option "usrunmess" says pretty much the state of mind
regarding the person who named it this way.

Something like "usrunmerge" would be more realistic.
> Greetings
> Marc
> 

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marc Haber]

On Tue, 20 Jul 2021 14:47:04 +0200, Svante Signell
 wrote:
>According to the dpkg developer and maintainer Guillem users can still
>rescue their systems from merged-/usr-via-aliased-dirs with the aid of 
>dpkg-fsys-usrunmess(8), see 
>https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F

The naming of the utility alone gives me the impression that we have a
dpkg maintainer who has gone to war with the rest of the distribution.
I am not sure whether Debian should accept that.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Svante Signell]

On Tue, 2021-07-20 at 13:41 +0200, Andreas Metzler wrote:
> 
> Hello,
> Isn't this kind of crying over spilt milk? I also wish we never had
> ended up with the buster/bullseye state where both unmerged and
> merged-/usr-via-aliased-dirs are fully supported. However there is
> now a huge number of merged-/usr-via-aliased-dirs installations out
> there and we cannot make them magically disappear. Undoing
> merged-/usr-via-aliased-dirs would be very error-prone while afaiui
> we have a relatively simple plan to get a clean merged /usr in
> bookworm or bookworm +1:

According to the dpkg developer and maintainer Guillem users can still
rescue their systems from merged-/usr-via-aliased-dirs with the aid of 
dpkg-fsys-usrunmess(8), see 
https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F

I for one will always use that whenever I (accidentally?) install
Debian in the future. All my old installations does not carry this bug.
Or is there some easier way to avoid merged-/usr except using --no-
merged-usr in debootstrap with new installations as written in 
https://wiki.debian.org/Teams/Dpkg/MergedUsr?

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Andreas Metzler]

On 2021-07-20 Guillem Jover  wrote:
> On Mon, 2021-07-19 at 16:41:42 +0200, Johannes Schauer Marin Rodrigues wrote:
>> So what what is actually the roadmap after the bullseye release?
>> What is the way forward? Should I rather file bugs with patches
>> against individual packages to move their files from
>> /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we already have a
>> debhelper patch to do that move for us?

> Unfortunately, when the supporters of the merged-/usr-via-aliased-dirs
> pushed their approach into the distribution, that meant that package
> stopped being able to ship compatibility symlinks under «/», and those
> needed to be "handled" in maintscripts (by reimplementing poorly and
> unsafely what dpkg is supposed to do). This means dpkg is not in the
> loop and cannot perform a safe upgrade moving these pathnames safely,
> as long as merged-/usr-via-aliased-dirs is supported.
[...]

Hello,
Isn't this kind of crying over spilt milk? I also wish we never had ended
up with the buster/bullseye state where both unmerged and
merged-/usr-via-aliased-dirs are fully supported. However there is now a
huge number of merged-/usr-via-aliased-dirs installations out there and
we cannot make them magically disappear. Undoing
merged-/usr-via-aliased-dirs would be very error-prone while afaiui we
have a relatively simple plan to get a clean merged /usr in bookworm or
bookworm +1:

1. Make merged-/usr-via-aliased-dirs the only supported layout and make
this information available to apt. (Like we did for multi-arch-support.)
2. After that individual packages can safely move files from / to /usr,
pre-depending on merged-usr-support.

cu Andreas
-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Guillem Jover]

On Tue, 2021-07-20 at 11:31:37 +0200, Guillem Jover wrote:
> Unfortunately, when the supporters of the merged-/usr-via-aliased-dirs
> pushed their approach into the distribution, that meant that package
> stopped being able to ship compatibility symlinks under «/», and those
> needed to be "handled" in maintscripts (by reimplementing poorly and
> unsafely what dpkg is supposed to do). This means dpkg is not in the
> loop and cannot perform a safe upgrade moving these pathnames safely,
> as long as merged-/usr-via-aliased-dirs is supported.

Sorry, how sloppy of me, let me qualify that last word, to avoid any
confusion, "supported". As in inflicting that into unsuspecting users,
and in having to bear and pay the price for its current existence, not
as in an approach that is supported by the packaging system.

Regards,
Guillem

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Guillem Jover]

On Mon, 2021-07-19 at 16:41:42 +0200, Johannes Schauer Marin Rodrigues wrote:
> So what what is actually the roadmap after the bullseye release? What is the
> way forward? Should I rather file bugs with patches against individual 
> packages
> to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
> already have a debhelper patch to do that move for us?

Unfortunately, when the supporters of the merged-/usr-via-aliased-dirs
pushed their approach into the distribution, that meant that package
stopped being able to ship compatibility symlinks under «/», and those
needed to be "handled" in maintscripts (by reimplementing poorly and
unsafely what dpkg is supposed to do). This means dpkg is not in the
loop and cannot perform a safe upgrade moving these pathnames safely,
as long as merged-/usr-via-aliased-dirs is supported.

This is all kinds of nasty, because a proposal that has a complete
disregard for the packaging system, and is founded on assumptions
about features that do not even exist, is also forcing that design
breakage into a solution that would otherwise require no such hacks.
:(

> This would mean that we only have to bear with the problems mentioned by
> guillem until that move is complete, correct?

No. Even if today we magically could get all .debs shipping object
directly under «/usr» instead of «/», and had the aliased symlinks in
place, the problems I've mentioned would pretty much be in place.

Regards,
Guillem

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Guillem Jover]

On Mon, 2021-07-19 at 15:10:42 +0200, Michael Biebl wrote:
> Am 19.07.21 um 03:36 schrieb Guillem Jover:
> > What I've also said multiple times, is that
> > merged-usr-via-moves-and-symlink-farms could have been implemented in
> > a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> > all with full cooperation and managed by dpkg, with .debs shipping
> > actual tracked pathnames

> What you propose is, that each and every package does its /usr-merge
> transition on its own. This only works, if packages are independent (enough)
> so this actually works.
> Unfortunately this is not the case. Take PAM for example. You can't just
> recompile src:pam and have debhelper automatically move all files to /usr.
> This would break all packages that install a PAM plugin. You have a
> transition here, involving many packages.
> Same is true for udev rules, systemd service files, basically every package
> that provides interfaces/hooks to other packages is affected.
> So it's not that simple unfortunately. You can't fully automate that.

Not at all. pam or whatever we transition via cooperation from dpkg,
would be kept compiling using the directories it currently uses, and
debhelper would simply move the objects on the .deb from «/» to «/usr»,
and create the compat symlinks. That means pam, and any plugins migrated
or not, would still be available in the current pathnames causing no
breakage. This part would be done automatically.

And as I've said elsewhere, removal of the compat symlinks would
require manual intervention (at the maintainer discretion), at some
later time, to modify the configured installation paths (which is
something that cannot be automated in debhelper, due to packaging
overrides and similar), at which point it can be decided whether to
declare say a local pam flag day, or add the needed Breaks and similar,
or add support to pam to look in both pathnames to avoid the two
previous items.

> According to
> apt-file search -x '^/(lib|bin|sbin)'
> on my Debian sid/amd64 system, we have 1747 packages shipping 24583 files in
> those directories. There are *many* such entangled transitions hidden in
> there, so I fear this is not manageable.

See my comment above, and josch reply.

> As Luca pointed out, even distros with a much stricter governance model were
> not able to do that.

Well if they did it poorly, no wonder, I guess.

> The /usr-merge transition as described and decided on in the TC bug, seems
> to me is the only viable way forward.

I obviously disagree.

> Yes, it does break dpkg -S, but your idea of using a list of mapped paths as
> in [1] seems like an entirely reasonable approach to solve this.

Did you miss the section in the mail you are replying where I mention
that f.ex. dpkg tools (dpkg, dpkg-divert, u-a) missing to then detect
file conflicts, or files disappearing on moves, or dpkg-deb -x (or tar)
destroying merged-/usr-via-aliased-dirs systems?

All these, including dpkg -S (which BTW also breaks after paths have
been moved, as when say bash ships as /usr/bin/bash, then dpkg -S will
also fail to find the expected /bin/bash), are currently affecting any
system being installed by the default installer, or ones having been
switched by the usrmerge hack.

> Once we have this global switch to merged-usr, packages can bit by bit,
> completely independent, update their debian/rules to use --prefix=/usr and
> after a few years, we don't have any packages anymore installing any files
> in /. We could aid this process with a lintian check that flags packages
> that install files in /(sbin|bin|lib)/.

The same can be said about my proposal, except that then dpkg is kept
in the loop, the transition is done safely by dpkg as part of individual
package upgrades, instead of having to use something off-band and as
disturbing as usrmerge during upgrade, on dpkg's back w/o any
cooperation from it…

Regards,
Guillem

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Niels Thykier]

Johannes Schauer Marin Rodrigues:
> Quoting Michael Biebl (2021-07-19 15:10:42)
>> [...]
>>
>> According to
>> apt-file search -x '^/(lib|bin|sbin)'
>> on my Debian sid/amd64 system, we have 1747 packages shipping 24583 
>> files in those directories.
> 
> more precisely, on amd64 unstable:
> 
> /bin 247 files
> /lib{32,64,x32} 83 files
> /lib/firmware 2379 files
> /lib/live 115 files
> /lib/modules 17500 files
> /lib/systemd 2221 files
> /lib/udev 614 files
> /lib/x86_64-linux-gnu 343 files
> /lib/* 441 files
> /sbin 547 files
> 
> So most files come from /lib/modules, where only 14 packages are involved,

and debhelper's dh_installmodules that will need to be tweaked to look
into /usr/lib/modules as well.
  But I have no idea if/when we would be ready for that and I will not
be changing debhelper until we are ready to move these bits to /usr.

Likewise for udev, here dh_installudev still uses /lib/udev and will
continue to do so until I know that udev also checks /usr/lib/udev.


If you are involved in udev or/and /lib/modules and know that they are
ready to move to the relevant directory under /usr, then please feel
free to file a bug against debhelper requesting that the /usr directory
will be used in the future.
  Please do also note in that bug report whether you also want debhelper
to automatically migrate existing files from /lib to /usr/lib.  This
should only be the case where we expect (almost) zero breakage from an
automatic "unordered" transition.

(NB: Please use a bug report as I will first be implementing this after
the bullseye release and rely to this thread is likely to be lost in my
inbox)

> /lib/systemd which will be fixed by an update to dh_installsystemd, and

Indeed, I have heard this request and the systemd maintainers confirmed
to me that systemd in bullseye checks both /lib/systemd and
/usr/lib/systemd.

I have a branch lying around trying to support this.  The main key
feature missing is the migration of /lib/systemd -> /usr/lib/systemd
(which needs to handle that both exist and merge them into one somehow).

> /lib/firmware where only 36 packages are involved. The remainder then doesn't
> sound so scary anymore as it only involves 656 unique packages and not 1747.
> And again many of those remaining packages will be fixed by an update to
> debhelper, correct? Given that 90% of source packages use dh, that would 
> reduce
> the number to a very manageable size.
> 

Indeed, about 90% of all packages uses dh according to trends.

Though for good measure I thought I would explicitly answer the proposal
(from another mail in this thread) that debhelper could move everything
from /lib to /usr/lib:

  No, I will not support an unconditional move from /lib to /usr/lib
  via debhelper during bookworm.

There are already two distinct examples in this thread for how this
could break things. I will be sticking to "targeted" migrations where
the major consumers have informed me that they are ready to support the
migration.


>> [...]
>>
>> Once we have this global switch to merged-usr, packages can bit by bit, 
>> completely independent, update their debian/rules to use --prefix=/usr 
>> and after a few years, we don't have any packages anymore installing any 
>> files in /. We could aid this process with a lintian check that flags
>> packages that install files in /(sbin|bin|lib)/.
> 
> So what what is actually the roadmap after the bullseye release? What is the
> way forward? Should I rather file bugs with patches against individual 
> packages
> to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
> already have a debhelper patch to do that move for us?
> 

For some, debhelper will your problem but there will packages that will
need manual migration. As I see it, there will *not* be "the debhelper
patch to fix them all" - even if there will /some/ debhelper patchers
that will fix "most of them".

Related, I have no intention of supporting / maintaining a rewrite of
"--prefix/PREFIX" parameters to configure/make/cmake or whatever.  (Not
saying anyone proposed it - but mentioning it as another "there will
definitely be manual clean up" data point).

~Niels

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marc Haber]

On Mon, 19 Jul 2021 15:19:32 +0200, Michael Biebl 
wrote:
>Am 19.07.21 um 07:23 schrieb Marc Haber:
>> I am NOT looking forward having to manually convert legacy systems to
>> merged /usr and I do sincerely hope that Debian will choose a way to
>> get away without throwing away systems that have just a small /, still
>> supporting a dedicated /usr as long as it's mounted by initramfs. I am
>> not sure whether we ever issued a clear statement about that.
>
>I think this is a misunderstanding. Files from / would be moved to /usr. 
>So the only way this could fail is, if your /usr partition was too 
>small.That's still a possibility for existing systems, but a much 
>smaller one then moving files from /usr to /. Typically a separate /usr 
>partition is larger then /.

Right, that sounds much easier. It's still an Open Heart Operation,
especially for systems that I don't have out of band access for, which
is rather common for smaller installations.

>I assume you are referring to the sulogin issue here [1], i.e. whether 
>we require a root password on an emergency failure or not.

Yes. From my point of view, this is taking away a freedom from the
local admin.

In an ideal world, boot failure behavior would be locally
configurable.

A mis-booted sysv system, if I remember correctly (I have been a
mostly happy systemd user for already quite some time), could be told
to "just try to continue and show me how far you get", which in the
vast majority of cases led to a regular login prompt from which the
user-login-plus-sudo routine just worked. This didn't hand out any
more root shells than the current way of stopping dead and refusing to
do anything without the "real" root password, as far as I understand.
I probably don't have enough experience to have the final call on
that. But it's just a pet peeve of mine that I can easily live with.

>Fwiw, this is mostly me being paranoid and not handing out root shells.

It's good to be paranoid by default. It's bad to force that paranoia
on the local admin who might have a choice to move to a different
distribution. But alas, the others do it the same way. So it's just
freedom lost.

>This has nothing to do with merged-/usr.

I never said it has.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Johannes Schauer Marin Rodrigues]

Quoting Michael Biebl (2021-07-19 15:10:42)
> Am 19.07.21 um 03:36 schrieb Guillem Jover:
> > What I've also said multiple times, is that
> > merged-usr-via-moves-and-symlink-farms could have been implemented in
> > a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> > all with full cooperation and managed by dpkg, with .debs shipping
> > actual tracked pathnames
> I'm convinced this view is way too naive and not implementable in 
> practice (and yes, openSUSE is a data point that confirms that)
> 
> What you propose is, that each and every package does its /usr-merge 
> transition on its own. This only works, if packages are independent 
> (enough) so this actually works.
> Unfortunately this is not the case. Take PAM for example. You can't just 
> recompile src:pam and have debhelper automatically move all files to 
> /usr. This would break all packages that install a PAM plugin. You have 
> a transition here, involving many packages.
> Same is true for udev rules, systemd service files, basically every 
> package that provides interfaces/hooks to other packages is affected.
> So it's not that simple unfortunately. You can't fully automate that.
> 
> According to
> apt-file search -x '^/(lib|bin|sbin)'
> on my Debian sid/amd64 system, we have 1747 packages shipping 24583 
> files in those directories.

more precisely, on amd64 unstable:

/bin 247 files
/lib{32,64,x32} 83 files
/lib/firmware 2379 files
/lib/live 115 files
/lib/modules 17500 files
/lib/systemd 2221 files
/lib/udev 614 files
/lib/x86_64-linux-gnu 343 files
/lib/* 441 files
/sbin 547 files

So most files come from /lib/modules, where only 14 packages are involved,
/lib/systemd which will be fixed by an update to dh_installsystemd, and
/lib/firmware where only 36 packages are involved. The remainder then doesn't
sound so scary anymore as it only involves 656 unique packages and not 1747.
And again many of those remaining packages will be fixed by an update to
debhelper, correct? Given that 90% of source packages use dh, that would reduce
the number to a very manageable size.

> There are *many* such entangled transitions 
> hidden in there, so I fear this is not manageable.
> As Luca pointed out, even distros with a much stricter governance model 
> were not able to do that.
> 
> The /usr-merge transition as described and decided on in the TC bug, 
> seems to me is the only viable way forward.
> 
> Yes, it does break dpkg -S, but your idea of using a list of mapped 
> paths as in [1] seems like an entirely reasonable approach to solve this.
> 
> Once we have this global switch to merged-usr, packages can bit by bit, 
> completely independent, update their debian/rules to use --prefix=/usr 
> and after a few years, we don't have any packages anymore installing any 
> files in /. We could aid this process with a lintian check that flags
> packages that install files in /(sbin|bin|lib)/.

So what what is actually the roadmap after the bullseye release? What is the
way forward? Should I rather file bugs with patches against individual packages
to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
already have a debhelper patch to do that move for us?

This would mean that we only have to bear with the problems mentioned by
guillem until that move is complete, correct?

And another question: once there are no more files shipped by any package in
/(sbin|bin|lib)/ we can let base-file create the symlinks?

Thanks!

cheers, josch

signature.asc
Description: signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Michael Biebl]

Am 19.07.21 um 07:23 schrieb Marc Haber:


I am NOT looking forward having to manually convert legacy systems to
merged /usr and I do sincerely hope that Debian will choose a way to
get away without throwing away systems that have just a small /, still
supporting a dedicated /usr as long as it's mounted by initramfs. I am
not sure whether we ever issued a clear statement about that.


I think this is a misunderstanding. Files from / would be moved to /usr. 
So the only way this could fail is, if your /usr partition was too 
small.That's still a possibility for existing systems, but a much 
smaller one then moving files from /usr to /. Typically a separate /usr 
partition is larger then /.



There are some technical reasons to separate /boot if you are using a file
system for other partitions that isn't suitable for early boot (or if
you're using cryptsetup or other file system layers).  /boot/efi is always
a separate partition because of how it works.  Apart from those two
special cases, the only reason to put something on a separate file system
is if you have a clear and compelling reason why you expect a given file
system to run out of space and you want to ensure that it cannot take
space from other parts of the system.


I also believe that smaller file systems are unlikely to break and
that a system that can boot up to a ssh-able state even with a broken
file system is way easier to fix. We have taken a huge step back in
that regard with systemd since the systemd rescue mode requiring the
"real" root password even for minor startup failures is way more
unfriendly than what we had before.


I assume you are referring to the sulogin issue here [1], i.e. whether 
we require a root password on an emergency failure or not.

Fwiw, this is mostly me being paranoid and not handing out root shells.
This has nothing to do with merged-/usr.


Michael


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802211



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Michael Biebl]

Hi Guillem

Am 19.07.21 um 03:36 schrieb Guillem Jover:

What I've also said multiple times, is that
merged-usr-via-moves-and-symlink-farms could have been implemented in
a fully automated way, by debhelper, w/o requiring any maintainer scripts,
all with full cooperation and managed by dpkg, with .debs shipping
actual tracked pathnames
I'm convinced this view is way too naive and not implementable in 
practice (and yes, openSUSE is a data point that confirms that)


What you propose is, that each and every package does its /usr-merge 
transition on its own. This only works, if packages are independent 
(enough) so this actually works.
Unfortunately this is not the case. Take PAM for example. You can't just 
recompile src:pam and have debhelper automatically move all files to 
/usr. This would break all packages that install a PAM plugin. You have 
a transition here, involving many packages.
Same is true for udev rules, systemd service files, basically every 
package that provides interfaces/hooks to other packages is affected.

So it's not that simple unfortunately. You can't fully automate that.

According to
apt-file search -x '^/(lib|bin|sbin)'
on my Debian sid/amd64 system, we have 1747 packages shipping 24583 
files in those directories. There are *many* such entangled transitions 
hidden in there, so I fear this is not manageable.
As Luca pointed out, even distros with a much stricter governance model 
were not able to do that.


The /usr-merge transition as described and decided on in the TC bug, 
seems to me is the only viable way forward.


Yes, it does break dpkg -S, but your idea of using a list of mapped 
paths as in [1] seems like an entirely reasonable approach to solve this.


Once we have this global switch to merged-usr, packages can bit by bit, 
completely independent, update their debian/rules to use --prefix=/usr 
and after a few years, we don't have any packages anymore installing any 
files in /. We could aid this process with a lintian check that flags 
packages that install files in /(sbin|bin|lib)/.





Regards,
Michael


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858331#33




OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Stephan Lachnit]

On Mon, Jul 19, 2021 at 3:37 AM Guillem Jover  wrote:
> What I've also said multiple times, is that
> merged-usr-via-moves-and-symlink-farms could have been implemented in
> a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> all with full cooperation and managed by dpkg, with .debs shipping
> actual tracked pathnames, if it had not been for the mess required
> by merged-usr-via-aliased-dirs. :/

Maybe I get this wrong, but I don't think this conflicts with the
decision from the TC.

Until Debian 12 gets released, we have a lot of time for a transition.
Maybe we should start discussing the transition and less whether or
not we do it, as this has been decided now anyway.

We could start with collecting the packages that install to /bin*
instead of /usr/bin, and adjust the packaging so that they don't do
that. Of course, we would need to add a maintainer script that detects
un-merged usr and creates a symlink. Actually, I think it would be
enough to just let debhelper detect if a package installs to /bin, and
adjust the package automatically. For packages not using debhelper,
lintian can add a warning if the package installs to /bin. After all
packages that installed to /bin have been rebuilt, nothing would
install to /bin except for symlinks. At this point, it shouldn't
matter if you run a merged usr system or not, or am I forgetting
something? IMHO it would make way more sense to discuss how to merge
usr once the packages are fixed.

Anyway, I think the discussion made clear that we shouldn't
immediately start with merging usr once bullseye is released, and I
wouldn't interpret the TC decision as such.

Regards,
Stephan

* using /bin as an example, same goes for /lib etc

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marco d'Itri]

On Jul 19, Marc Haber  wrote:

> I am NOT looking forward having to manually convert legacy systems to
> merged /usr and I do sincerely hope that Debian will choose a way to
> get away without throwing away systems that have just a small /, still
> supporting a dedicated /usr as long as it's mounted by initramfs. I am
They cannot be supported without keeping a lot of unneeded complexity 
around, but there is no reason to reinstall them: you can move / inside 
/usr instead. You may use either sash or a live CD image.

> I also believe that smaller file systems are unlikely to break and
> that a system that can boot up to a ssh-able state even with a broken
> file system is way easier to fix. We have taken a huge step back in
And "apt install grml-rescueboot" is even better.
Also, with merged-/usr you may keep your *whole* OS in a read only file 
system.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Gunnar Wolf]

As I said, on a separate mail...

Marc Haber dijo [Mon, Jul 19, 2021 at 07:12:24AM +0200]:
> In an ideal world, would the package manager not be a service utility
> to SUPPORT policy and adapt to changing environment contitions instead
> being a showstopper for innovation?
> 
> Who is the dpkg maintainer to challenge the decisions of the entire
> project? I fully understand that there is only ONE dpkg maintainer,
> but a utility THIS central to the entire ecosystem not being team
> maintained is a HUGE part of the problem.
> 
> And no, I cannot help and no, you wouldn't want me to write a single
> line of code in a package THIS central.
> (...)
> Would it not be dpkg's job to work around these flaws? It's not that
> every other component of a Debian system are perfect.

FWIW, I mostly agree with what you say here -- If the project decides
to a new standard (and, in this case, it has via the TC's decision --
which can of course be repealed by GR, if things come to that),
packages that behave differently... are buggy and should be fixed.

Of course, dpkg is a very special case for obvious reasons; I did try
to reach out to Guillem when we started discussing the bug at the TC,
and was disheartened by his harsh reply basically negating all
possibility of discussion because his non-belief in the TC itself.

There are technical issues that this migration will bring, and yes,
there is a nonzero chance some users will be bitten by the dissonance
between dpkg and reality. But after two TC bug resolutions (#914897
and #978636), and after lots of bytes have been spilled by various
people, I can only see work has to be put into making possibly
problematic cases less likely -- rebuilding and checking packages
don't ship files in the root directories will cover a great deal of
the distance. If aliasing the directories via symlinks is so messy,
well... we should focus on the root cause, and fix the rasons for it
to be broken as much as we can. The symlinks could probably be an
unconsequential footnote if this is done right.


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Gunnar Wolf]

Sorry to single you out here, Marc -- This goes to many people. This
goes, in fact, to the discussion itself.

Marc Haber dijo [Mon, Jul 19, 2021 at 07:12:24AM +0200]:
> In an ideal world, would the package manager not be a service utility
> to SUPPORT policy and adapt to changing environment contitions instead
> being a showstopper for innovation?
> 
> Who is the dpkg maintainer to challenge the decisions of the entire
> project? I fully understand that there is only ONE dpkg maintainer,
> but a utility THIS central to the entire ecosystem not being team
> maintained is a HUGE part of the problem.
> 
> And no, I cannot help and no, you wouldn't want me to write a single
> line of code in a package THIS central.
> (...)

While I agree with what you write here (will answer on a separate
mail), I'll ask you -and everybody- to please moderate the tone. It is
frustrating to speak in different wavelengths and not be able to hear
one another, but we are not going to get anywhere if we just SHOUT
LOUDER using our same wavelength. We must find some alternate
frequencies to get to a constructive situation.

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Stephan Verbücheln]

Nowadays you can also have BTRFS subvolumes, which does not require you
to define sizes in advance. In that case it is nice for snapshotting to
have separate subvolumes for things like home directories.

Regards

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marc Haber]

On Sun, 18 Jul 2021 16:21:24 -0700, Russ Allbery 
wrote:
>I agree with the other feedback that you are overpartitioning your disk.

It is especially evident in the df output where there are two-digit
amounts of gigabytes free on most of those HUGE partitions.

>I used to do this back when I was first learning UNIX in the 1990s because
>it seems like a good idea and it does isolate one part of the system from
>another if it uses an excessive amount of space.  But what I found in
>practice, and what almost everyone who does this eventually finds in
>practice, is that this much partitioning drastically reduces the long-term
>flexibility of the system.  It requires you predict in advance what parts
>of the system will grow, and when you guess wrong, you end up with
>symlinks trying to move directories from a partition with no free space to
>another partition with free space, with all the complexity and breakage
>that can cause.

Nowadays, with LVM, file systems can easily grow, even online. I have
stopped putting /usr on a dedicated file system as the usrmerge began
to show up on the horizon, but I still use dedicated file systems for
/home and /var.

I am NOT looking forward having to manually convert legacy systems to
merged /usr and I do sincerely hope that Debian will choose a way to
get away without throwing away systems that have just a small /, still
supporting a dedicated /usr as long as it's mounted by initramfs. I am
not sure whether we ever issued a clear statement about that.

>There are some technical reasons to separate /boot if you are using a file
>system for other partitions that isn't suitable for early boot (or if
>you're using cryptsetup or other file system layers).  /boot/efi is always
>a separate partition because of how it works.  Apart from those two
>special cases, the only reason to put something on a separate file system
>is if you have a clear and compelling reason why you expect a given file
>system to run out of space and you want to ensure that it cannot take
>space from other parts of the system.

I also believe that smaller file systems are unlikely to break and
that a system that can boot up to a ssh-able state even with a broken
file system is way easier to fix. We have taken a huge step back in
that regard with systemd since the systemd rescue mode requiring the
"real" root password even for minor startup failures is way more
unfriendly than what we had before. Many installations closely guard
the root password for real emergencies (I have been working on big
installations for years and have NEVER seena case where the "real"
root password was actually used - it is usually easier to rebuild
affected systems from scratch).

>This can be a good justification for putting /home on a separate partition
>*if* you are running a multi-user system.  But otherwise, separating out
>things like /var or /usr/local or /opt or /srv is more likely to cause you
>long-term headaches than it is to do anything useful.

I disagree for /var (maybe just for /var/log or parts of /var/lib).

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marc Haber]

On Mon, 19 Jul 2021 03:36:59 +0200, Guillem Jover 
wrote:
>If by "very minor bugs" you mean that f.ex.:
>
> * dpkg, dpkg-divert, or update-alternatives are unable to detect file
>   conflicts and thus might allow silent overwrites of random stuff on
>   disk,
> * when moving files across packages and across aliased directories,
>   these pathnames might end up disappearing depending on the unpack
>   order,
> * dpkg-deb -x on the root directory (yes, people use this to recover
>   systems) with any .deb that contains files on real directories under
>   «/», will replace the symlinked directories with real ones,
>
>then, yes, I guess "very minor" indeed. But then I completely object
>to this being classified as bugs in dpkg, as this has been shoved in
>disregarding that dpkg does *not* support this, and it would require
>new *features* to be implemented, so this "transition" is founded on
>assuming features that do not exist, or completely going behind
>dpkg's back, which sounds great I guess…

In an ideal world, would the package manager not be a service utility
to SUPPORT policy and adapt to changing environment contitions instead
being a showstopper for innovation?

Who is the dpkg maintainer to challenge the decisions of the entire
project? I fully understand that there is only ONE dpkg maintainer,
but a utility THIS central to the entire ecosystem not being team
maintained is a HUGE part of the problem.

And no, I cannot help and no, you wouldn't want me to write a single
line of code in a package THIS central.

>The problem in general is that this layout introduces unreliability
>and silently induced breakage stemming from this flawed filesystem
>layout, which is going to affect the people that are going to benefit
>the less from its properties, and are the less experience to deal with
>it.

Would it not be dpkg's job to work around these flaws? It's not that
every other component of a Debian system are perfect.

>And here we are getting the project installing by default systems that
>are fighting the packaging system and going on its back, to enable a
>filesystem design layout mostly experienced admins will benefit from
>in very special deployment conditions, where final users can very
>easily suffer from its introduced unreliability (from 3rd-party repos
>or locally built packages, etc, f.ex.).

I must be missing some thing, but isn't it the experienced admins
facing reinstalls of vital systems when we finally move to a
completely merged /usr, because these usually currently have /usr on a
dedicated mountpoint?

>Because the above has been brought up before and the proponents are well
>aware of these, I'm afraid at this point the only thing that comes to
>mind is negligence, TBH. :/

on both sides of the conflict, yes.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Russ Allbery]

Polyna-Maude Racicot-Summerside  writes:

> I had the belief that some software used /tmp for temporary file that
> may grow many GB (example DVD creation).

> I have 32 GB

It should not, or at least it should let you specify a different path,
because using tmpfs for /tmp is very common these days.  (/var/tmp is
available if one needs a file system more likely to be on a larger disk.)

-- 
Russ Allbery (r...@debian.org)  

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Guillem Jover]

On Thu, 2021-07-15 at 10:13:47 +0100, Luca Boccassi wrote:
> As it has been said and written many times already, in reality this is
> not broken by design at all and in fact it is the only successful
> strategy that has been deployed by other distros - it's what is being
> called merged-usr-via-moves-and-symlink-farms that is broken. We can
> say this with certainty because both approaches have been tried, so
> there's actual real-world data to look at. Just go look at the absolute
> mess that Suse got themselves into by following that solution - a 10-
> years-long massive half-done-never-finished headache that took an
> inordinate amount of work to back out of, and move to the actual
> working solution that everybody else are using - merged-usr-via-
> aliased-dirs. On the other hand Fedora/RHEL had a smooth and simple
> transition using merged-usr-via-aliased-dirs, and that was the end of
> it.

Yes, a single (a couple!?) data point(s) of a strategy used in another
unrelated distribution, with completely different packaging stacks and
ecosystems, which was done very poorly, has been presented repeatedly.
I'm not sure why that has much value TBH.

The above seems to also be confusing how and if a design has been
deployed, with its inherent (short and long term) properties. A badly
performed *deployment* for a better design does not make its properties
bad, in the same way that *deploying* a flawed design faster does not
make its properties better…

What I've also said multiple times, is that
merged-usr-via-moves-and-symlink-farms could have been implemented in
a fully automated way, by debhelper, w/o requiring any maintainer scripts,
all with full cooperation and managed by dpkg, with .debs shipping
actual tracked pathnames, if it had not been for the mess required
by merged-usr-via-aliased-dirs. :/

> Dpkg has some very minor bugs that rpm/dnf/yum/zypper/whatever do not
> suffer from. So what? It is perfectly normal as it's software and all
> softwares have bugs. They could be fixed, worked around, or ignored,
> like all other bugs.

If by "very minor bugs" you mean that f.ex.:

 * dpkg, dpkg-divert, or update-alternatives are unable to detect file
   conflicts and thus might allow silent overwrites of random stuff on
   disk,
 * when moving files across packages and across aliased directories,
   these pathnames might end up disappearing depending on the unpack
   order,
 * dpkg-deb -x on the root directory (yes, people use this to recover
   systems) with any .deb that contains files on real directories under
   «/», will replace the symlinked directories with real ones,

then, yes, I guess "very minor" indeed. But then I completely object
to this being classified as bugs in dpkg, as this has been shoved in
disregarding that dpkg does *not* support this, and it would require
new *features* to be implemented, so this "transition" is founded on
assuming features that do not exist, or completely going behind
dpkg's back, which sounds great I guess…

The problem in general is that this layout introduces unreliability
and silently induced breakage stemming from this flawed filesystem
layout, which is going to affect the people that are going to benefit
the less from its properties, and are the less experience to deal with
it.

I've tried to update the 
with more explicit information, in case some people might have missed
that from previous instances of this discussion, and added an initial
table of properties for both proposals, to avoid cluttering and repeating
it on the list, and to ease updating it.




We do actually have experience with "an aliased" layout from symlinked
/usr/share/doc/ directories, where those are for optional/removable parts
of the filesystem, and the symlinks are even known and managed by dpkg.
And those have been a major and constant source of packaging bugs.

And here we are getting the project installing by default systems that
are fighting the packaging system and going on its back, to enable a
filesystem design layout mostly experienced admins will benefit from
in very special deployment conditions, where final users can very
easily suffer from its introduced unreliability (from 3rd-party repos
or locally built packages, etc, f.ex.).

Because the above has been brought up before and the proponents are well
aware of these, I'm afraid at this point the only thing that comes to
mind is negligence, TBH. :/

But *shrug*, have at it, I'm tired of the continued and complete
disregard of the unreliability issues and subversion of the packaging
system, which are supposed to be pillars of our project, every single
time. I just replied so that people that might not want to be forced
into this stuff know that there's going to be a way out.

Regards,
Guillem

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-18 7:21 p.m., Russ Allbery wrote:
> Polyna-Maude Racicot-Summerside  writes:
> 
>> Here's my actual config (with 2TB) and yes I have a separate /home
> 
>> What is tmpfs and why is it set to 3.2 GB ?
> 
> tmpfs is a RAM-backed temporary file system that is automatically used for
> paths like /run and /dev/shm that are supposed to be cleared on each
> reboot and hold only small files (or memory references, in the case of
> /dev/shm).
> 
> I see that you have your system configured to store /tmp on your disk.
> This is generally not recommended these days.  Storing /tmp in tmpfs is
> much faster for some applications and automatically achieves the desired
> and standard /tmp behavior of clearing it on reboot.  About the only
> reason not to use tmpfs is if you have a very memory-constrained system
> and don't want to use any member at all for memory-backed file systems.
> 
I had the belief that some software used /tmp for temporary file that
may grow many GB (example DVD creation).

I have 32 GB

>> And /dev have 16G free ? Where does this come from...
> 
> The size of the udev file system is essentially meaningless.
> 
>> I'm wasting some space with /tmp !
> 
> I agree with the other feedback that you are overpartitioning your disk.
> I used to do this back when I was first learning UNIX in the 1990s because
> it seems like a good idea and it does isolate one part of the system from
> another if it uses an excessive amount of space.  But what I found in
> practice, and what almost everyone who does this eventually finds in
> practice, is that this much partitioning drastically reduces the long-term
> flexibility of the system.  It requires you predict in advance what parts
> of the system will grow, and when you guess wrong, you end up with
> symlinks trying to move directories from a partition with no free space to
> another partition with free space, with all the complexity and breakage
> that can cause.
> 
> There are some technical reasons to separate /boot if you are using a file
> system for other partitions that isn't suitable for early boot (or if
> you're using cryptsetup or other file system layers).  /boot/efi is always
> a separate partition because of how it works.  Apart from those two
> special cases, the only reason to put something on a separate file system
> is if you have a clear and compelling reason why you expect a given file
> system to run out of space and you want to ensure that it cannot take
> space from other parts of the system.
> 
> This can be a good justification for putting /home on a separate partition
> *if* you are running a multi-user system.  But otherwise, separating out
> things like /var or /usr/local or /opt or /srv is more likely to cause you
> long-term headaches than it is to do anything useful.
> 

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Andy Smith]

Hi,

On Sun, Jul 18, 2021 at 05:54:33PM -0400, Polyna-Maude Racicot-Summerside wrote:
> On 2021-07-18 5:07 p.m., Andy Smith wrote:
> > I recommend understanding the issue before putting forth an opinion.
> > 
> Maybe I shall correct what I said as it may be misunderstood.

It's unclear to me why you've taken my reply to your mail on
debian-user, and instead sent it to me privately and also to
debian-devel.

As neither of us are Debian Developers and you don't seem to have
done much research regarding merged-/usr, I don't feel it is
on-topic for me to continue that conversation with you on
debian-devel.

I recommend doing some more reading of these threads and the
relevant wiki pages and if things are still unclear then asking
user-level questions only on debian-user.

Thanks,
Andy

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Russ Allbery]

Russ Allbery  writes:

> I see that you have your system configured to store /tmp on your disk.
> This is generally not recommended these days.  Storing /tmp in tmpfs is
> much faster for some applications and automatically achieves the desired
> and standard /tmp behavior of clearing it on reboot.  About the only
> reason not to use tmpfs is if you have a very memory-constrained system
> and don't want to use any member at all for memory-backed file systems.

Argh, that should have been "and don't want to use any *memory* at all for
memory-backed file systems."

Apologies for the additional message, but that word substitution was
sufficiently confusing that I'm not sure the intended meaning was clear.

-- 
Russ Allbery (r...@debian.org)  

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Russ Allbery]

Svante Signell  writes:

> Again, everybody is just hiding, I wonder from who, the big wolf??  Who
> is hen? Anybody having the courage to reply to this list about this
> issue, not only workarounds/diversions?

I'm not discussing the issue on the list because I think the current
direction in which Debian is heading seems reasonable and forward progress
is being made, so there doesn't seem to be anything to argue about or any
point in doing so.

The folks who have been upset about this direction for years are still
upset and I'm sorry that they're still upset and that we haven't found
some approach that makes everyone happy.  But so far as I can tell, there
have been no new substantive arguments and no changes in direction, so
there doesn't seem to be anything new to reply about, particularly given
that I'm not involved in the implementation and therefore am not involved
in analyzing any of the technical concerns they raise.

"Courage" seems to me like an odd label to put on any of this.

-- 
Russ Allbery (r...@debian.org)  

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Russ Allbery]

Polyna-Maude Racicot-Summerside  writes:

> Here's my actual config (with 2TB) and yes I have a separate /home

> What is tmpfs and why is it set to 3.2 GB ?

tmpfs is a RAM-backed temporary file system that is automatically used for
paths like /run and /dev/shm that are supposed to be cleared on each
reboot and hold only small files (or memory references, in the case of
/dev/shm).

I see that you have your system configured to store /tmp on your disk.
This is generally not recommended these days.  Storing /tmp in tmpfs is
much faster for some applications and automatically achieves the desired
and standard /tmp behavior of clearing it on reboot.  About the only
reason not to use tmpfs is if you have a very memory-constrained system
and don't want to use any member at all for memory-backed file systems.

> And /dev have 16G free ? Where does this come from...

The size of the udev file system is essentially meaningless.

> I'm wasting some space with /tmp !

I agree with the other feedback that you are overpartitioning your disk.
I used to do this back when I was first learning UNIX in the 1990s because
it seems like a good idea and it does isolate one part of the system from
another if it uses an excessive amount of space.  But what I found in
practice, and what almost everyone who does this eventually finds in
practice, is that this much partitioning drastically reduces the long-term
flexibility of the system.  It requires you predict in advance what parts
of the system will grow, and when you guess wrong, you end up with
symlinks trying to move directories from a partition with no free space to
another partition with free space, with all the complexity and breakage
that can cause.

There are some technical reasons to separate /boot if you are using a file
system for other partitions that isn't suitable for early boot (or if
you're using cryptsetup or other file system layers).  /boot/efi is always
a separate partition because of how it works.  Apart from those two
special cases, the only reason to put something on a separate file system
is if you have a clear and compelling reason why you expect a given file
system to run out of space and you want to ensure that it cannot take
space from other parts of the system.

This can be a good justification for putting /home on a separate partition
*if* you are running a multi-user system.  But otherwise, separating out
things like /var or /usr/local or /opt or /srv is more likely to cause you
long-term headaches than it is to do anything useful.

-- 
Russ Allbery (r...@debian.org)  

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-18 6:17 p.m., Marco d'Itri wrote:
> On Jul 19, Polyna-Maude Racicot-Summerside  wrote:
> 
>> So if I get it right...
> Except for /boot/, which may be required for technical reasons, there 
> is no need to further partition your file system unless you actually 
> have reasons to do it.
> 
>> One partiton for /boot
>> One partition for /usr
>> One partition for /usr/local (if you feel like it)
>> One / partiton that will contain not much stuff other than config files ?
>> One partition for /var (if you feel)
>> One partition for /opt, /srv ...
>> One partition for /tmp
> If you are aiming for overcomplexity then I think that you forgot /home.
> 
>> The root partition can be small as 16 Gb as it won't contain much ?
> If you create a partition for everything else as described here then 
> / will only contain /etc and /root, so even 1 GB will be enough.
> But again, I do not really recommend this.
> 
> I do not recommend to partition general purpose systems with less than 
> a few hundreds of GBs of allocated disk space.
> 

Here's my actual config (with 2TB) and yes I have a separate /home

What is tmpfs and why is it set to 3.2 GB ?
And /dev have 16G free ? Where does this come from...
I'm wasting some space with /tmp !

Filesystem  Size  Used Avail Use% Mounted on
udev 16G 0   16G   0% /dev
tmpfs   3.2G  2.1M  3.2G   1% /run
/dev/sda3   184G   70G  105G  40% /
tmpfs16G  163M   16G   2% /dev/shm
tmpfs   5.0M  4.0K  5.0M   1% /run/lock
/dev/sda5   262G   50G  199G  21% /var
/dev/sda4   175G   86M  166G   1% /tmp
/dev/sda888G   60M   83G   1% /usr/local
/dev/sda792G  3.6G   84G   5% /opt
/dev/sda10   92G   60M   87G   1% /srv
/dev/sda1   487M  3.3M  483M   1% /boot/efi
/dev/sda11  733G  649G   47G  94% /home
tmpfs   3.2G   84K  3.2G   1% /run/user/118
tmpfs   3.2G  124K  3.2G   1% /run/user/1000


-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Svante Signell]

On Sun, 2021-07-18 at 20:58 +0200, Svante Signell wrote:
> On Wed, 2021-07-14 at 23:40 +0200, Guillem Jover wrote:
> > On Wed, 2021-07-14 at 19:54:56 +, Thorsten Glaser wrote:
> > > Sean Whitton dixit:
> > > > * #978636 move to merged-usr-only?
> > > > 
> > > >  We were asked to decide whether or not Debian 'bookworm'
> > > > should
> > > >  continue to support systems which are not using the merged-usr
> > > >  filesystem layout.  We decided that support should not
> > > > continue
> > > > beyond Debian 'bullseye'.
> > > 
> > > What? WHAT? ＷＨＡＴ？
> > > 
> > > >  The decision is captured here:
> > > >  
> > > 
> > > No reason provided either. This stinks. I’m v̲e̲r̲y̲
> > > disappointed.
> > > Debian is becoming untenable. Years ago, I had hoped it won’t.
> > 
> > I've been meaning to send a note about this for some time now, but
> > as I feel it keeps getting ignored, it always seems a bit
> > pointless.
> > 
> > But in any case, given that merged-usr-via-aliased-dirs is not
> > really
> > supported by dpkg anyway, it is broken by design [B], I have no
> > intention whatsoever to break any of my systems with such layout
> > going forward, I'm thus planning to spend any necessary volunteer
> > time implementing any fix, workaround or solution required to avoid
> > having to use it, in detriment of other Debian volunteer time. I
> > alreadystarted some time ago with dpkg-fsys-usrunmess(8), present
> > already inthe upcoming bullseye release.
> > 
> [B] 
> https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
> 
> Since the dpkg developer and maintainer Guillem considers merged /usr
> broken by design, maybe Debian should consider to use some other
> package management software for the peace of mind for people involved
> in the project? Maybe guix could be usable?

Again, everybody is just hiding, I wonder from who, the big wolf??
Who is hen? Anybody having the courage to reply to this list about this
issue, not only workarounds/diversions?

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marco d'Itri]

On Jul 19, Polyna-Maude Racicot-Summerside  wrote:

> So if I get it right...
Except for /boot/, which may be required for technical reasons, there 
is no need to further partition your file system unless you actually 
have reasons to do it.

> One partiton for /boot
> One partition for /usr
> One partition for /usr/local (if you feel like it)
> One / partiton that will contain not much stuff other than config files ?
> One partition for /var (if you feel)
> One partition for /opt, /srv ...
> One partition for /tmp
If you are aiming for overcomplexity then I think that you forgot /home.

> The root partition can be small as 16 Gb as it won't contain much ?
If you create a partition for everything else as described here then 
/ will only contain /etc and /root, so even 1 GB will be enough.
But again, I do not really recommend this.

I do not recommend to partition general purpose systems with less than 
a few hundreds of GBs of allocated disk space.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hello (Hi) !

On 2021-07-18 5:07 p.m., Andy Smith wrote:
> Hello,
> 
> I think all of this is quite clearly explained in:
> 
> https://salsa.debian.org/md/usrmerge/raw/master/debian/README.Debian
> 
> which is linked from:
> 
> https://wiki.debian.org/UsrMerge
> 
> If you think it's not then you should probably raise a bug against
> the usrmerge package with your suggested edits/clarifications.
>
I've read the link above plus the link to
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/

So if I get it right...

One partiton for /boot
One partition for /usr
One partition for /usr/local (if you feel like it)
One / partiton that will contain not much stuff other than config files ?
One partition for /var (if you feel)
One partition for /opt, /srv ...
One partition for /tmp

The root partition can be small as 16 Gb as it won't contain much ?

Am I getting this right ?

Here's my actual partition table (2TB HD)

udev 16G 0   16G   0% /dev
tmpfs   3.2G  2.1M  3.2G   1% /run
/dev/sda3   184G   70G  105G  40% /
tmpfs16G  163M   16G   2% /dev/shm
tmpfs   5.0M  4.0K  5.0M   1% /run/lock
/dev/sda5   262G   50G  199G  21% /var
/dev/sda4   175G   86M  166G   1% /tmp
/dev/sda888G   60M   83G   1% /usr/local
/dev/sda792G  3.6G   84G   5% /opt
/dev/sda10   92G   60M   87G   1% /srv
/dev/sda1   487M  3.3M  483M   1% /boot/efi
/dev/sda11  733G  649G   47G  94% /home

I currently have a 70G usage of the root partition.
Running Buster.

> Andy
> 

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-18 5:07 p.m., Andy Smith wrote:
> Hello,
> 
> On Sun, Jul 18, 2021 at 04:31:11PM -0400, Polyna-Maude Racicot-Summerside 
> wrote:
>> My personal opinion is that Debian is going into a mostly "we got the
>> best idea in the world but forgot that not everyone implement things the
>> same way".
> 
> I recommend understanding the issue before putting forth an opinion.
> 
Maybe I shall correct what I said as it may be misunderstood.

Without regard to this specific issue, I seem to have noticed that the
Debian project has made some change in good faith, based on some
technological conception of what is the best. But in the end, most user
seem not to need those change and be a source of problem for most of them.

I've had hell of a mess to understand the change to SystemD.

Not so long ago I even learned that it caused a fork for a Debian
derivative called Devuan.

For me it was a huge departure from the KISS philosophy and I feel that
this "usrmerge" is going the same way.

I will take time to review what you have gave me as link and see what
will be my opinion after this.

>> I currently have a different partition for my /usr and this has been the
>> case since the end of 1990s when I started on Linux. Maybe I'm wrong but
>> I like it this way.
>>
>> Will the merge-usr cause myself problem ?
> 
> No. Not as long as you use an initramfs created by any of the
> supported Debian tools like initramfs-tools or dracut, which you
> will do unless you have gone out of your way to do something
> different.
> 
I've did many Debian install, using the standard installation software,
creating those partition from the "expert" choice of install mode. And
never had problem, I believe that the initramfs tools must be doing
their job.

> And regardless of merged-/usr, /usr on separate partition has not
> been supported in Debian without an initramfs since the stretch
> release in 2017.
> 
I've used Stretch with a separate /usr without much problem, as said above.

> I think all of this is quite clearly explained in:
> 
> https://salsa.debian.org/md/usrmerge/raw/master/debian/README.Debian
> 
> which is linked from:
> 
> https://wiki.debian.org/UsrMerge
> 
> If you think it's not then you should probably raise a bug against
> the usrmerge package with your suggested edits/clarifications.
> 
> Andy
> 

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-18 5:31 p.m., Svante Signell wrote:
> Hi, is it OK to forward your mail to debian-devel. I don't think
> mailing to debian-user will have any effect on this issue?
>  
Sure ! Honestly it's my mistake to have sent it to debian-user.
I get everything in one mailbox. I need to have this sorted out and use
one mailbox for each mailing list.

Thanks

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Helmut Grohne]

On Fri, Jul 16, 2021 at 01:15:37PM +0200, Magissia wrote:
> In this case, this page should be updated to reflect the fact it is not
> broken.
> 
> https://wiki.debian.org/Teams/Dpkg/MergedUsr

Logic does not quite work that way. Just because we selected that way of
doing things doesn't imply it would not be broken. Arguably, if it
wasn't broken, there would not have been a need to defer the decision to
the CTTE as there would have been consensus on choosing the non-broken
way already. The CTTE was tasked to choose the less broken one of two
broken options and it did.

>From a dpkg pov, the aliasing technique continues to be broken and the
solution continues to manifest as unintuitive behaviour and related
issues. Calling that broken sounds fair to me.

The thing that makes me a little grumpy about this is that the
proponents of the /usr merge continue stuffing ever more fingers into
their ears when faced with problems. I for one wouldn't care as much if
the /usr merge wasn't that much of a time sink to me. For instance, I
happened to be one of the first affected users of dpkg-shlibdeps ceasing
to work. I'd really like to ignore the whole mess and leave it to
others (and that includes not opposing it).

I also note that there is a subtle difference between those who disagree
with the /usr merge and those that disagree with its implementation
strategy. The former ones seems like a very small minority to me. As a
result, reiterating the "why" is kinda pointless as there already seems
to be consensus on that front. As for the how, there seems to be rough
consensus that all implementation strategies are broken one way or
another. We just get to pick the least painful one and deferred the
choice to the CTTE.

Maybe someone could fix #858331 et al and we can move on?

Helmut

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Svante Signell]

On Wed, 2021-07-14 at 23:40 +0200, Guillem Jover wrote:
> On Wed, 2021-07-14 at 19:54:56 +, Thorsten Glaser wrote:
> > Sean Whitton dixit:
> > > * #978636 move to merged-usr-only?
> > > 
> > >  We were asked to decide whether or not Debian 'bookworm' should
> > >  continue to support systems which are not using the merged-usr
> > >  filesystem layout.  We decided that support should not continue
> > > beyond Debian 'bullseye'.
> > 
> > What? WHAT? ＷＨＡＴ？
> > 
> > >  The decision is captured here:
> > >  
> > 
> > No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
> > Debian is becoming untenable. Years ago, I had hoped it won’t.
> 
> I've been meaning to send a note about this for some time now, but
> as I feel it keeps getting ignored, it always seems a bit pointless.
> 
> But in any case, given that merged-usr-via-aliased-dirs is not really
> supported by dpkg anyway, it is broken by design [B], I have no
> intention whatsoever to break any of my systems with such layout
> going forward, I'm thus planning to spend any necessary volunteer
> time implementing any fix, workaround or solution required to avoid
> having to use it, in detriment of other Debian volunteer time. I
> alreadystarted some time ago with dpkg-fsys-usrunmess(8), present
> already inthe upcoming bullseye release.
> 
[B] 
https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
>

Since the dpkg developer and maintainer Guillem considers merged /usr
broken by design, maybe Debian should consider to use some other
package management software for the peace of mind for people involved
in the project? Maybe guix could be usable?

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Andrey Rahmatullin]

On Sun, Jul 18, 2021 at 10:11:22PM +0500, Andrey Rahmatullin wrote:
> While, AFAIK, it's indeed only Debian Policy stopping you from not
> shipping /usr/share/doc/*/copyright, and that and common sense stopping
> you from not shipping /usr/share/doc/*/changelog, that's just yet another
> case of something broken that we probably are not required to support.
... even though these files really belong with other dpkg metadata in
/var/lib/dpkg...

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Andrey Rahmatullin]

On Sun, Jul 18, 2021 at 04:50:49PM +, Stephan Verbücheln wrote:
> Thank you for the explanation. I think it covers most use cases.
> However, it does not cover packages that do not actually install
> programs but only perform changes to /etc or install something to /opt,
> is that correct?
While, AFAIK, it's indeed only Debian Policy stopping you from not
shipping /usr/share/doc/*/copyright, and that and common sense stopping
you from not shipping /usr/share/doc/*/changelog, that's just yet another
case of something broken that we probably are not required to support.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Stephan Verbücheln]

Thank you for the explanation. I think it covers most use cases.
However, it does not cover packages that do not actually install
programs but only perform changes to /etc or install something to /opt,
is that correct?

Regards

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marco d'Itri]

On Jul 18, Stephan Verbücheln  wrote:

> On Sun, 2021-07-18 at 11:13 +0200, Marco d'Itri wrote:
> > I link /var/lib/dpkg/ to somewhere in /usr/, and I think that this is
> > 
> What? No matter whether we merge “/bin” or not, “/usr” should stay
> read-only.
The dpkg database IS read-only as long as you are not installing or 
removing packages, which you cannot do anyway if /usr is read-only.
In other words, the read-only state of the dpkg database is tightly 
coupled with the read-only state of /usr.

The only corner case which would require some more thinking is how to 
handle changing diversions of files in /etc in the systems with the 
read-only /usr, and for the time being "don't do that" could be 
a totally valid solution.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Andrey Rahmatullin]

On Sun, Jul 18, 2021 at 11:09:49AM +, Stephan Verbücheln wrote:
> On Sun, 2021-07-18 at 11:13 +0200, Marco d'Itri wrote:
> > I link /var/lib/dpkg/ to somewhere in /usr/, and I think that this is
> > 
> 
> What? No matter whether we merge “/bin” or not, “/usr” should stay
> read-only.
On Debian /usr is changed only when /var/lib/dpkg is changed (ideally).

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Stephan Verbücheln]

On Sun, 2021-07-18 at 11:13 +0200, Marco d'Itri wrote:
> I link /var/lib/dpkg/ to somewhere in /usr/, and I think that this is
> 

What? No matter whether we merge “/bin” or not, “/usr” should stay
read-only.

Regards

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Bastian Blank]

On Sun, Jul 18, 2021 at 11:13:37AM +0200, Marco d'Itri wrote:
> On Jul 18, Simon McVittie  wrote:
> > If a machine's /usr is not in sync with its /etc and /var, then it is likely
> > to work incorrectly: at a minimum, asking dpkg which packages and versions
> But in my experience (with shared-/usr containers) this works great as 
> long as everything is aligned to the same major Debian release.

So we can just merge it and make it break even less likely?

Bastian

-- 
A princess should not be afraid -- not with a brave knight to protect her.
-- McCoy, "Shore Leave", stardate 3025.3

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marco d'Itri]

On Jul 18, Simon McVittie  wrote:

> If a machine's /usr is not in sync with its /etc and /var, then it is likely
> to work incorrectly: at a minimum, asking dpkg which packages and versions
But in my experience (with shared-/usr containers) this works great as 
long as everything is aligned to the same major Debian release.

> are installed will give you an answer that does not match what is actually
> in /usr.
I link /var/lib/dpkg/ to somewhere in /usr/, and I think that this is 
something that we should do anyway.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Simon McVittie]

On Sat, 17 Jul 2021 at 20:34:41 +, Johannes Drexl wrote:
> /usr is allowed to be not only on a separate partition, but even on a
> network device

This has been discussed at considerable length before, but I'll try to
recap:

A separate or network-mounted /usr is possible in Debian, whether
merged-/usr is used or not, but only if /usr is already mounted
by the time the system hands over from the initramfs (usually generated by
initramfs-tools, or sometimes dracut or another alternative) to the main
system (which starts systemd, sysvinit or some other init system as the
main pid 1 and proceeds from there). Merged /usr is actually better for
this configuration than unmerged /usr in some ways: it lets you share
*all* the non-modifiable files between machines, not just the ones that
are in /usr (as opposed to /lib, /bin, /sbin).

This is not a new requirement: Debian >= 9 (2017) officially did not support
mounting /usr halfway through the main system boot process. Older Debian
releases aimed to support /usr being mounted during boot (for example
during the rcS sequence in sysv-rc) but it frequently didn't actually
work in practice, particularly if combined with network mounts.

The reason why mounting /usr during main-system boot is no longer
supported is that it often didn't work. A major reason why it often didn't
work is that depending on how the system is configured, mounting /usr can
require networking; but bringing up networking can require arbitrarily
many programs and libraries (networking infrastructure like ifupdown or
NetworkManager often has either a plugin architecture, or an arbitrary
hooks mechanism that executes programs of the sysadmin's choice that
will often have dependencies in /usr, or both).

Similarly, udevd needs to run early in the main system boot to bring up
/dev, but udev rules can run arbitrary programs, some of them in /usr;
and for the main system, those programs often need data from /usr/share.

More generally, if we took every library and every program that could
conceivably be a dependency of /usr and moved them from /usr into the
root filesystem, then the root filesystem would become increasingly large
over time, negating any benefit that you hoped to gain by mounting /usr
separately. Over time, this approach would tend towards the layout that
the Debian Hurd port briefly tried to use, which was the opposite of
merged-/usr (you could call it "merged rootfs"), with /usr as either a
symlink to /, or containing only symlinks to /lib, /bin and so on.

Instead, we require everything that is needed in *your* configuration
(not necessarily in *anyone's* configuration, just yours!) to be part
of the initramfs, which is generated per-machine.

Effectively, the requirement to mount /usr before pivoting from initramfs
to main system means that instead of a small rootfs (which can be used for
recovery) and a larger /usr, we have a small initramfs (which can be used
for recovery) and a large main system.

One key advantage of this is that decisions about what to include in
the rootfs (of non-merged-/usr systems) have to be made globally for all
of Debian, but decisions about what to include in the initramfs can be
made per-machine, allowing some otherwise impossible situations to be
resolved. If your /usr is mounted via NFS, but bringing up my network
requires /usr (perhaps for a VPN), in the old model with a small rootfs
and a larger /usr it was impossible for us both to have what we needed:
we could not bring up networking both before /usr (as you would have
needed) and after /usr (as I would have needed).

> /usr is allowed to be [...] on a network device shared by other machines

The FHS may allow this, but it has significant practical problems,
unrelated to merged-/usr, on machines that receive security/stable updates
(which I would hope by now should mean all machines). Updates to Debian
packages can touch both mutable per-machine files (/etc, /var) and
immutable/shareable per-(package,version) files (/usr, /lib*, /bin, /sbin).
If a machine's /usr is not in sync with its /etc and /var, then it is likely
to work incorrectly: at a minimum, asking dpkg which packages and versions
are installed will give you an answer that does not match what is actually
in /usr.

On non-merged-/usr systems, the machine's /usr and {/lib*,/bin,/sbin}
must also be kept in sync: for example, there is no guarantee that
/usr/bin/dbus-daemon (in /usr) will work correctly with a mismatched
version of /lib/x86_64-linux-gnu/libdbus-1.so.3 (on the rootfs of a
non-merged-/usr system). On merged-/usr systems, it is impossible for
those directories to become out-of-sync, so this particular requirement
becomes trivial to achieve.

A more robust approach to sharing /usr between machines (or containers)
might be to give each machine (or container) its own private /usr or even
its own private root filesystem, and then carry out file- or block-level
deduplication on the storage backend (for example, if /usr is 

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Johannes Drexl]

Am Freitag, dem 16.07.2021 um 10:09 +0200 schrieb Thomas Goirand:
> Merging binaries in /usr and getting rid of /bin and /sbin, at the
> end,
> WILL be an improvement. Debian cannot be the last distro not doing
> the
> move, I hope you understand that.
> 
> Also, I'm having a hard time understanding why moving binaries around
> should just break any system

Well, I'm only an interested SysOp here, but I share his reluctance to
embrace getting rid of /sbin and /bin. FHS 3.0 explicitely states that
/usr is allowed to be not only on a separate partition, but even on a
network device shared by other machines:

Quoting FHS 3.0, chapter 4.1, Purpose of /usr: 
'/usr is the second major section of the filesystem. /usr is shareable,
read-only data. That means that /usr should be shareable between
various FHS-compliant hosts and must not be written to.'

On the other hand /bin and /sbin need to be on the root partition in
case mounting of other partitions doesn't work (as in "no network
available").

Quoting FHS 3.0, chapter 3.16.1, Purpose of /sbin:
'/sbin contains binaries essential for booting, restoring, recovering,
and/or repairing the system in addition to the binaries in /bin.
Programs executed after /usr is known to be mounted (when there are no
problems) are generally placed into /usr/sbin.'

So yes, with getting rid of /bin and /sbin you break the systems
emergency resilience and deviate from a known and not really that old
standard. I'm not quite sure how this is an improvement - rather I'm
curious how one could perceive this as such. For all I've seen in my
time as a SysOp, I'd rather have my emergency tools in /sbin and /bin
than booting a live system every time a server has broken mounts.

That all said, I'm not fully against the change, I just don't see the
benefit but instead quite a lot of problems down the road.

BR
Jo


signature.asc
Description: This is a digitally signed message part

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Magissia]

In this case, this page should be updated to reflect the fact it is not
broken.

https://wiki.debian.org/Teams/Dpkg/MergedUsr

Nol'

Le jeudi 15 juillet 2021 à 10:13 +0100, Luca Boccassi a écrit :
> On Wed, 2021-07-14 at 23:40 +0200, Guillem Jover wrote:
> > On Wed, 2021-07-14 at 19:54:56 +, Thorsten Glaser wrote:
> > > Sean Whitton dixit:
> > > > * #978636 move to merged-usr-only?
> > > > 
> > > >  We were asked to decide whether or not Debian 'bookworm'
> > > > should
> > > >  continue to support systems which are not using the merged-usr
> > > >  filesystem layout.  We decided that support should not
> > > > continue beyond
> > > >  Debian 'bullseye'.
> > > 
> > > What? WHAT? ＷＨＡＴ？
> > > 
> > > >  The decision is captured here:
> > > >  
> > > 
> > > No reason provided either. This stinks. I’m v̲e̲r̲y̲
> > > disappointed.
> > > Debian is becoming untenable. Years ago, I had hoped it won’t.
> > 
> > I've been meaning to send a note about this for some time now, but
> > as I feel it keeps getting ignored, it always seems a bit
> > pointless.
> > 
> > But in any case, given that merged-usr-via-aliased-dirs is not
> > really
> > supported by dpkg anyway, it is broken by design [B], I have no
> > intention whatsoever to break any of my systems with such layout
> > going
> > forward, I'm thus planning to spend any necessary volunteer time
> > implementing any fix, workaround or solution required to avoid
> > having
> > to use it, in detriment of other Debian volunteer time. I already
> > started some time ago with dpkg-fsys-usrunmess(8), present already
> > in
> > the upcoming bullseye release.
> > 
> > [B] <
> > https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
> > >
> > 
> > Thanks,
> > Guillem
> 
> Hi,
> 
> As it has been said and written many times already, in reality this
> is
> not broken by design at all and in fact it is the only successful
> strategy that has been deployed by other distros - it's what is being
> called merged-usr-via-moves-and-symlink-farms that is broken. We can
> say this with certainty because both approaches have been tried, so
> there's actual real-world data to look at. Just go look at the
> absolute
> mess that Suse got themselves into by following that solution - a 10-
> years-long massive half-done-never-finished headache that took an
> inordinate amount of work to back out of, and move to the actual
> working solution that everybody else are using - merged-usr-via-
> aliased-dirs. On the other hand Fedora/RHEL had a smooth and simple
> transition using merged-usr-via-aliased-dirs, and that was the end of
> it.
> 
> Dpkg has some very minor bugs that rpm/dnf/yum/zypper/whatever do not
> suffer from. So what? It is perfectly normal as it's software and all
> softwares have bugs. They could be fixed, worked around, or ignored,
> like all other bugs.
> 

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Thomas Goirand]

On 7/16/21 10:09 AM, Thomas Goirand wrote:
> On 7/14/21 9:54 PM, Thorsten Glaser wrote:
>> Sean Whitton dixit:
>>
>>> * #978636 move to merged-usr-only?
>>>
>>>  We were asked to decide whether or not Debian 'bookworm' should
>>>  continue to support systems which are not using the merged-usr
>>>  filesystem layout.  We decided that support should not continue beyond
>>>  Debian 'bullseye'.
>>
>> What? WHAT? ＷＨＡＴ？
>>
>>>  The decision is captured here:
>>>  
>>
>> No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
>> Debian is becoming untenable. Years ago, I had hoped it won’t.
>>
>> bye,
>> //mirabilos
> 
> Hi Thorsten,
> 
> Sent privately, I hope you don't mind.

It was sent to the list, never mind, there's nothing private here, just
a call to be more constructive...

Cheers,

Thomas Goirand (zigo)

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Thomas Goirand]

On 7/14/21 9:54 PM, Thorsten Glaser wrote:
> Sean Whitton dixit:
> 
>> * #978636 move to merged-usr-only?
>>
>>  We were asked to decide whether or not Debian 'bookworm' should
>>  continue to support systems which are not using the merged-usr
>>  filesystem layout.  We decided that support should not continue beyond
>>  Debian 'bullseye'.
> 
> What? WHAT? ＷＨＡＴ？
> 
>>  The decision is captured here:
>>  
> 
> No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
> Debian is becoming untenable. Years ago, I had hoped it won’t.
> 
> bye,
> //mirabilos

Hi Thorsten,

Sent privately, I hope you don't mind.

I very much like you, and appreciate the work you're doing in Debian.
However, I would not like at all if this is the start of yet-another
drama in Debian. We had enough of this.

Instead of complaining about change, it'd be nice if instead, you were
embracing it, and tried to contribute fixes for the parts that you're
maintaining in Debian. I'm not asking you to be enthusiastic about
changes you don't like, but maybe you could at least tried to understand
others are trying to improve things, rather than destroying your work.

Merging binaries in /usr and getting rid of /bin and /sbin, at the end,
WILL be an improvement. Debian cannot be the last distro not doing the
move, I hope you understand that.

Also, I'm having a hard time understanding why moving binaries around
should just break any system, especially if we have workarounds, like
symlink and/or bind mounts or the like. If non-systemd setups are having
issues, please just fix them... The end result *will* be a better Debian.

Last thing, the discussion happened a long time ago, you had plenty of
time to react, especially before the TC voted. Reacting so late, with
such emotion is really uncalled for.

Cheers,

Thomas Goirand (zigo)

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Sean Whitton]

Hello,

On Thu 15 Jul 2021 at 09:56AM +02, Jonathan Carter wrote:

> Hi Sean
>
> On 2021/07/15 09:04, Sean Whitton wrote:
>> Just to confirm, when you say "merged-usr-via-aliased-dirs", you mean
>> what I would get if I typed 'debootstrap bullseye /foo', right?
>>
>> I would like to note that the TC decision did not specify any particular
>> implementation of merged-/usr.  It was just about whether to continue to
>> try to support both.
>
> I think a more detailed explanation/expansion/clarification on what
> exactly this means (and ideally also the rationale behind) that in the
> bug report would be appreciated.

You're right, it would have been good if our Bits mail had linked to
some other messages in that thread rather than just the statement of the
result of the vote.  I'm sorry we didn't spot that before sending it.

Additionally, as someone else was kind enough to point out to me
off-list, my statement that "the TC decision did not specify any
particular implementation of merged-/usr" was rather misleading.  What I
should have written was that we did not specify any particular
implementation of the *migration* to merged-/usr for existing systems.

ISTM that "merged-usr-via-aliased-dirs" could refer to the migration
path implemented by the usrmerge package, and/or simply the replacement
of the directories /lib, /bin etc. with symlinks.  To the extent that it
refers to the latter, the TC decision does indeed specify that we will
implement merged-/usr using merged-usr-via-aliased-dirs.

Here are some useful messages we should have linked to:




and of course


-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Jonathan Carter]

On 2021/07/15 10:47, Marc Haber wrote:
> Can we please delay this discussion until after the release?

To be clear, I was requesting further details from the TC, not a
re-evaluation or further discussion.

-Jonathan

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Luca Boccassi]

On Wed, 2021-07-14 at 23:40 +0200, Guillem Jover wrote:
> On Wed, 2021-07-14 at 19:54:56 +, Thorsten Glaser wrote:
> > Sean Whitton dixit:
> > > * #978636 move to merged-usr-only?
> > > 
> > >  We were asked to decide whether or not Debian 'bookworm' should
> > >  continue to support systems which are not using the merged-usr
> > >  filesystem layout.  We decided that support should not continue beyond
> > >  Debian 'bullseye'.
> > 
> > What? WHAT? ＷＨＡＴ？
> > 
> > >  The decision is captured here:
> > >  
> > 
> > No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
> > Debian is becoming untenable. Years ago, I had hoped it won’t.
> 
> I've been meaning to send a note about this for some time now, but
> as I feel it keeps getting ignored, it always seems a bit pointless.
> 
> But in any case, given that merged-usr-via-aliased-dirs is not really
> supported by dpkg anyway, it is broken by design [B], I have no
> intention whatsoever to break any of my systems with such layout going
> forward, I'm thus planning to spend any necessary volunteer time
> implementing any fix, workaround or solution required to avoid having
> to use it, in detriment of other Debian volunteer time. I already
> started some time ago with dpkg-fsys-usrunmess(8), present already in
> the upcoming bullseye release.
> 
> [B] 
> 
> 
> Thanks,
> Guillem

Hi,

As it has been said and written many times already, in reality this is
not broken by design at all and in fact it is the only successful
strategy that has been deployed by other distros - it's what is being
called merged-usr-via-moves-and-symlink-farms that is broken. We can
say this with certainty because both approaches have been tried, so
there's actual real-world data to look at. Just go look at the absolute
mess that Suse got themselves into by following that solution - a 10-
years-long massive half-done-never-finished headache that took an
inordinate amount of work to back out of, and move to the actual
working solution that everybody else are using - merged-usr-via-
aliased-dirs. On the other hand Fedora/RHEL had a smooth and simple
transition using merged-usr-via-aliased-dirs, and that was the end of
it.

Dpkg has some very minor bugs that rpm/dnf/yum/zypper/whatever do not
suffer from. So what? It is perfectly normal as it's software and all
softwares have bugs. They could be fixed, worked around, or ignored,
like all other bugs.

-- 
Kind regards,
Luca Boccassi


signature.asc
Description: This is a digitally signed message part

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Marc Haber]

On Thu, 15 Jul 2021 09:56:18 +0200, Jonathan Carter 
wrote:
>On 2021/07/15 09:04, Sean Whitton wrote:
>> Just to confirm, when you say "merged-usr-via-aliased-dirs", you mean
>> what I would get if I typed 'debootstrap bullseye /foo', right?
>> 
>> I would like to note that the TC decision did not specify any particular
>> implementation of merged-/usr.  It was just about whether to continue to
>> try to support both.
>
>I think a more detailed explanation/expansion/clarification on what
>exactly this means (and ideally also the rationale behind) that in the
>bug report would be appreciated.

Can we please delay this discussion until after the release? I don't
think we can afford an additional time sink at the moment. Please, get
bullseye out of the door first, then decide how many existing users
we're going to drive away from Debian in the next round.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Jonathan Carter]

Hi Sean

On 2021/07/15 09:04, Sean Whitton wrote:
> Just to confirm, when you say "merged-usr-via-aliased-dirs", you mean
> what I would get if I typed 'debootstrap bullseye /foo', right?
> 
> I would like to note that the TC decision did not specify any particular
> implementation of merged-/usr.  It was just about whether to continue to
> try to support both.

I think a more detailed explanation/expansion/clarification on what
exactly this means (and ideally also the rationale behind) that in the
bug report would be appreciated.

thanks,

-Jonathan

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Sean Whitton]

Hello Guillem,

On Wed 14 Jul 2021 at 11:40PM +02, Guillem Jover wrote:

> On Wed, 2021-07-14 at 19:54:56 +, Thorsten Glaser wrote:
>> Sean Whitton dixit:
>> >* #978636 move to merged-usr-only?
>> >
>> >  We were asked to decide whether or not Debian 'bookworm' should
>> >  continue to support systems which are not using the merged-usr
>> >  filesystem layout.  We decided that support should not continue beyond
>> >  Debian 'bullseye'.
>>
>> What? WHAT? ＷＨＡＴ？
>>
>> >  The decision is captured here:
>> >  
>>
>> No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
>> Debian is becoming untenable. Years ago, I had hoped it won’t.
>
> I've been meaning to send a note about this for some time now, but
> as I feel it keeps getting ignored, it always seems a bit pointless.
>
> But in any case, given that merged-usr-via-aliased-dirs is not really
> supported by dpkg anyway, it is broken by design [B], I have no
> intention whatsoever to break any of my systems with such layout going
> forward, I'm thus planning to spend any necessary volunteer time
> implementing any fix, workaround or solution required to avoid having
> to use it, in detriment of other Debian volunteer time. I already
> started some time ago with dpkg-fsys-usrunmess(8), present already in
> the upcoming bullseye release.
>
> [B] 
> 

Just to confirm, when you say "merged-usr-via-aliased-dirs", you mean
what I would get if I typed 'debootstrap bullseye /foo', right?

I would like to note that the TC decision did not specify any particular
implementation of merged-/usr.  It was just about whether to continue to
try to support both.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Guillem Jover]

On Wed, 2021-07-14 at 19:54:56 +, Thorsten Glaser wrote:
> Sean Whitton dixit:
> >* #978636 move to merged-usr-only?
> >
> >  We were asked to decide whether or not Debian 'bookworm' should
> >  continue to support systems which are not using the merged-usr
> >  filesystem layout.  We decided that support should not continue beyond
> >  Debian 'bullseye'.
> 
> What? WHAT? ＷＨＡＴ？
> 
> >  The decision is captured here:
> >  
> 
> No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
> Debian is becoming untenable. Years ago, I had hoped it won’t.

I've been meaning to send a note about this for some time now, but
as I feel it keeps getting ignored, it always seems a bit pointless.

But in any case, given that merged-usr-via-aliased-dirs is not really
supported by dpkg anyway, it is broken by design [B], I have no
intention whatsoever to break any of my systems with such layout going
forward, I'm thus planning to spend any necessary volunteer time
implementing any fix, workaround or solution required to avoid having
to use it, in detriment of other Debian volunteer time. I already
started some time ago with dpkg-fsys-usrunmess(8), present already in
the upcoming bullseye release.

[B] 


Thanks,
Guillem

merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Thorsten Glaser]

Sean Whitton dixit:

>* #978636 move to merged-usr-only?
>
>  We were asked to decide whether or not Debian 'bookworm' should
>  continue to support systems which are not using the merged-usr
>  filesystem layout.  We decided that support should not continue beyond
>  Debian 'bullseye'.

What? WHAT? ＷＨＡＴ？

>  The decision is captured here:
>  

No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
Debian is becoming untenable. Years ago, I had hoped it won’t.

bye,
//mirabilos
-- 
Yes, I hate users and I want them to suffer.
-- Marco d'Itri on gmane.linux.debian.devel.general