Bug#1050001: Unwinding directory aliasing [and 3 more messages] [and 1 more messages]
On Fri, 1 Sept 2023 at 09:23, Helmut Grohne wrote: > > Hi Luca, > > At least three DDs have asked you to stop. Why do you continue? > > In all of your mails to this bug, I've seen little attempt at trying to > understand other participants. In a project as large as Debian, it is to > be expected that disagreement arises. That's not the end of the world, > but it is important to disagree respectfully. I'm not seeing that > respect in your interactions. Extraordinary claims require extraordinary evidence, as the saying goes. Asking for such evidence when it's not provided, for example for the claim that Ansible, Puppet, Chef, Docker, Rsync and whatnot no longer work on Debian/Ubuntu/Fedora/Arch/SUSE/CentOS/RHEL/etc, is not disrespectful.
Bug#1050001: Unwinding directory aliasing [and 3 more messages] [and 1 more messages]
Hi Ian, On Wed, 2023-08-30 at 12:52 +0100, Ian Jackson wrote: > Debian has historically been simply much more reliable. Could you quantify this? This is not my experience. As far as I understand you often use non-standard configurations (split-/usr, non-standard init system, ...) which might explain your impression. But non-standard ocnfigurations have been less reliable since forever. > usrmerge is a facet of Debian's culture wars. I would appreciate keeping the discussion to technical parts; could I ask you to keep your concerns about cancel culture elsewhere? > > > This argument is basically drawing together the common Affected > > > tooling includes not just our .debs, but also remote > > > configuration management systems like Ansible, image construction > > > (Dockerfiles), and 3rd-party software installation progrmas > > > (including > > > both 3rd-party .debs and 3rd-party script-based installation > > > systems). > > > > I don't follow the reasoning. [...] > > Sam has posted a helpful set of examples of things that one might do, > whose consequences with aliased directories are unclear. > > These are of course only examples. Tools like Ansible, Dockerfiles, ... have probably already been changed to handle merged-/usr. If they are still horribly broken, please document major issues (there are probably always cases that need a change). I somehow doubt many tools will be broken on many distributions for many years and nobody caring. So please substantiate this claim; it seems wild speculation that tools no longer work on Debian, Ubuntu, Red Hat, SuSE, Arch, Gentoo, ... and that they haven't been changed to work (in so far as this has been neccessary). I see this more as a reason against moving to the proposed Jackson layout with symlink farms: this new layout seems more likely to introduce problems with tooling like Ansible, Docker, ... than an existing layout shared between most(?) larger distributions and already the default for many years. Is there any risk evaluation what happen when moving to symlink farms? > > * Become more precise about what your layout looks like precisely. > > Our > ... > > Let me give some examples to get you started: > > libreoffice -> /bin/python > > ghostscript, imagemagick, mesa -> /bin/env > > bind9, manpages, net-snmp, qtbase-opensource-src -> /bin/perl > > Of these I would hope (by, say, 2033) to only include env. I think "hope" is not a good decision mechanism to produce a high quality operating system. How do you plan to do this? Drop links by throwing a coin, releasing stable, then waiting for user systems to be broken? This has been asked several times, but you refuse to give any answer. > perl upstream doctrine used to be that /usr/bin/perl was the correct > canonical path. I haven't checked if that recommendation still > stands. > > I don't know what Python upstream doctrine is on /bin vs /usr/bin; if > it were that Python is supposed to be in /bin, then we might need to > follow that. (But, are they dropping support for the BSDs?) > > With another hat on in an upstream project I've have been receiving > patches to change #!/bin/bash in CI scripts to #!/usr/bin/env bash. How do you ensure all users follow this scheme required by your proposed symlink farm layout, including for local scripts or scripts taken from systems where they just work? > I think it's worth pointing out that any software which is trying to > be portable to Unix systems other than just Linux (which includes the > BSDs and MacOS) will need to avoid assuming directory aliasing. Which seems irrelevant for what we do in Debian. On portable system you can't assume `/bin/sh` to be there either... Ansgar
Bug#1050001: Unwinding directory aliasing [and 3 more messages] [and 1 more messages]
Hi Luca, At least three DDs have asked you to stop. Why do you continue? In all of your mails to this bug, I've seen little attempt at trying to understand other participants. In a project as large as Debian, it is to be expected that disagreement arises. That's not the end of the world, but it is important to disagree respectfully. I'm not seeing that respect in your interactions. Even if working from the assumption that Ian's request was totally impractical, listening to him can help figure out what issues we cause in moving forward and what use cases we break. That sort of critical feedback is valuable as it points to weaknesses in our currently favoured plan. Past discussions of the matter have clearly alienated other participants. Therefore it is now difficult to receive critical feedback. If you want to continue discussing this matter on this issue, please only do so if you are able to respect the views of others. Helmut
Bug#1050001: Unwinding directory aliasing [and 3 more messages] [and 1 more messages]
On Wed, 30 Aug 2023 at 12:57, Ian Jackson
wrote:
>
> On the burden of proof and the correctness of software:
>
> I'm afraid I see a pattern, where blanket statements are made which
> are only "mostly" or "roughly" or "generally" true. But the
> discrepancies and details matter. When we make computer systems, it's
> not good enough to if they're only "mostly" right.
>
> Every program is an unproven conjecture whose truth we end up relying
> on. We need simple axioms, and rigorously correct reasoning.
Yes, and fact-free and evidence-free threads such as #1050001 go in
the exact opposite direction.
> Aliased usrmerge, when deployed, was a conjecture that was disputed:
> "usrmerge via directory aliasing functions correctly".
And it demonstrably does: it works beautifully and flawlessly on an
uncountable number of Debian, Ubuntu, Mint, Fedora, RHEL, CentOS,
Arch, SUSE, Yocto, Gentoo, etc. installations, for a decade in some of
those cases.
> Helmut Grohne writes ("Bug#1050001: Unwinding directory aliasing [and 3 more
> messages]"):
> > In order to prefer Debian over something else, we want it to be similar
> > enough to make switching feasible while making it differ from others
> > enough to make switching worthwhile. Not having aliasing symlinks hardly
> > seems like an aspect that makes Debian more suitable to people. I guess
> > that you disagree with this and that is fine.
>
> Debian has historically been simply much more reliable. That
> reliability doesn't come from one particular decision. It comes from
> an aggregate of, on the whole, making better decisions. That
> reliability is very valuable to our users and downstreams. It's
> one of our our most compelling advantages.
It comes in the largest part from having reliable upstreams that can
use common, stable, reliable and proven interfaces, such as the
usr-merged filesystem layout which has been the de-facto standard
across the vast majority of Linux distros for the past decade.
> > My impression has always been that the disagreement on the end
> > state was involving a minority.
>
> Well, if you disagree with aliased usrmerge and say you don't want it,
> you get abuse. Even here, many abusive messages have been posted with
> impunity by persons with considerable status.
Sure, insofar as asking for evidence for outrageous and unproven
claims can be considered 'abuse'
> > > This argument is basically drawing together the common factor in the
> > > DEP-17 problem list.
> >
> > And that's precisely why I don't understand your argument. All of the
> > DEP-17 problems are supposed to go away. So it appears as if you cite a
> > list of problems that I expect to not be problems for much longer as a
> > reason for changing the end goal.
>
> DEP-17's list of *problems* forms a category: directory-aliasing-
> induced malfunctions. But the DEP-17 *solutions* are ad-hoc, and many
> are only applicable to to malfunctions that occur during migration.
No, it's a list of potential bugs caused by dpkg being hopelessly
broken, and effective and demonstrably correct temporary workarounds
for them.
> > > Affected tooling includes not just our .debs, but also remote
> > > configuration management systems like Ansible, image construction
> > > (Dockerfiles), and 3rd-party software installation progrmas (including
> > > both 3rd-party .debs and 3rd-party script-based installation systems).
> >
> > I don't follow the reasoning. [...]
>
> Sam has posted a helpful set of examples of things that one might do,
> whose consequences with aliased directories are unclear.
>
> These are of course only examples.
Yes, they are only examples, more specifically evidence-free examples.
It would be trivial to demonstrate that Ansible/Puppet/whatever are
broken and don't work on Debian/Ubuntu/Fedora/RHEL/CentOS/Arch/etc,
but somehow evidence remains scarce. I wonder why.
> > > I would be much more convinced that all of this isn't a problem, if
> > > there existed, and had been formally adopted (eg by existing in some
> > > manual somewhere) a short and clear set of rules about what is and
> > > isn't allowed - not just rules for us within Debian, but rules for
> > > everyone, everywhere, referring to and modifying files.
> >
> > It appears to me that the empty set of rules (outside packaging) is too
> > simple for you to consider.
>
> "Packaging" is doing a lot of work there. I find your comment rather
> tendentious, I'm afraid.
>
> I think we need a short and clear set of rules for what you can do
> with ansible, rsync, docker, etc. etc. etc., as well as just apt and
> dpkg.
Again, the rule is simple and short: vendor files go under /usr,
ignore legacy locations. Doesn't get any easier than that.
> > > > We already have the Debian Usr Merge Analysis Tool available at
> > > > https://salsa.debian.org/helmutg/dumat and its output at
> > > > https://subdivi.de/~helmut/dumat.yaml. As explained on d-d@l.d.o, I want
> > > > to turn those findings into automatic RC
Bug#1050001: Unwinding directory aliasing [and 3 more messages] [and 1 more messages]
On the burden of proof and the correctness of software:
I'm afraid I see a pattern, where blanket statements are made which
are only "mostly" or "roughly" or "generally" true. But the
discrepancies and details matter. When we make computer systems, it's
not good enough to if they're only "mostly" right.
Every program is an unproven conjecture whose truth we end up relying
on. We need simple axioms, and rigorously correct reasoning.
Aliased usrmerge, when deployed, was a conjecture that was disputed:
"usrmerge via directory aliasing functions correctly".
In the time since the TC dismissed the objections and ruled in favour
of believing in the truth of this conjecture, has been disproven.
It is false in such important wsys that even in Debian, where we have
every possible advantage, we have been significantly inconvenienced.
Now we have a long list of counterexamples demonstrating the
invalidity of the original thesis.
Nevertheless, in mathematical terms, we are trying to patch up
the conjecture with many qualifications - the mitigations.
Also, we are apparently trying to impose on 3rd parties qualifications
about when it is OK to refer to files by the "wrong" name, which
cannot even be stated clearly, let along succinctly.
Despite all this, apparently when we argue for continued reliance on
this disproven assertion we go back to imprecise statements.
Helmut Grohne writes ("Bug#1050001: Unwinding directory aliasing [and 3 more
messages]"):
> In order to prefer Debian over something else, we want it to be similar
> enough to make switching feasible while making it differ from others
> enough to make switching worthwhile. Not having aliasing symlinks hardly
> seems like an aspect that makes Debian more suitable to people. I guess
> that you disagree with this and that is fine.
Debian has historically been simply much more reliable. That
reliability doesn't come from one particular decision. It comes from
an aggregate of, on the whole, making better decisions. That
reliability is very valuable to our users and downstreams. It's
one of our our most compelling advantages.
> My impression has always been that the disagreement on the end
> state was involving a minority.
Well, if you disagree with aliased usrmerge and say you don't want it,
you get abuse. Even here, many abusive messages have been posted with
impunity by persons with considerable status.
usrmerge is a facet of Debian's culture wars. (Debian's culture wars
are mostly orthogonal to the wider world's, but it would be foolish to
ignore the huge social factors at play.) If one is on the
currently-losing side one is not optimistic about the value of making
one's views widely known. Most of my friends agree with me on
substance but think it's a lost cause.
Of course, I could try to "solve" this invisibility by making a blog
post drawing attention and asking random people across the internet to
"contribute" to this discussion (or even just ask them to "me too").
That might even be effective, since currently I'm rather alone here.
I'm not doing that because Debian is quite toxic enough. (And also
because as an apparent minority, seen as a reactionary outgroup, we
would get blamed for the behaviour of extremists.)
> > This argument is basically drawing together the common factor in the
> > DEP-17 problem list.
>
> And that's precisely why I don't understand your argument. All of the
> DEP-17 problems are supposed to go away. So it appears as if you cite a
> list of problems that I expect to not be problems for much longer as a
> reason for changing the end goal.
DEP-17's list of *problems* forms a category: directory-aliasing-
induced malfunctions. But the DEP-17 *solutions* are ad-hoc, and many
are only applicable to to malfunctions that occur during migration.
There is nothing inherently migration-specific about aliasing-induced
malfunction; migration is in this context mostly just a lot of Things
happening, each of which has a chance to go wrong.
The result of the current plan is that all directory-aliasing-induced
malfunctions must be averted, individually, forever. By us, but also
by everyone who has to modify any Debian-derived system.
> > Affected tooling includes not just our .debs, but also remote
> > configuration management systems like Ansible, image construction
> > (Dockerfiles), and 3rd-party software installation progrmas (including
> > both 3rd-party .debs and 3rd-party script-based installation systems).
>
> I don't follow the reasoning. [...]
Sam has posted a helpful set of examples of things that one might do,
whose consequences with aliased directories are unclear.
These are of course only examples.
> > I would be much more convinced that all of this isn't a problem, if
> > there existed, and had been formally adopted (eg by existing in some
> > manual somewhere) a short and clear set of rules about what is and
> > isn't allowed - not just rules for us within Debian, but rules for
> > everyone,
