Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Ansgar]

Matthew Garrett writes:
> With my non-CTTE hat on, I think this is a demonstration that Debian 
> *does* care about derivatives. It's important to ensure that derivatives 
> are aware of the subtle interactions between dpkg and usrmerge, 
> otherwise they may suffer from hard to diagnose issues on upgrades. The 
> message from dpkg says nothing about reverting to split-/usr, and 
> instead points at a wiki page. If our documentation on how to avoid 
> these issues is incomplete (ie, if we're only describing how to avoid it 
> by using split-/usr rather than describing the mitigations we've imposed 
> within Debian to avoid triggering the issues), perhaps a better approach 
> would be to improve that documentation? We don't benefit our users by 
> pretending there isn't a problem here.

Okay, I followed your recommendation and added a small warning to the
FAQ: https://wiki.debian.org/Teams/Dpkg/FAQ?action=diff=78=79

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Matthew Garrett]

On Tue, Jun 13, 2023 at 09:14:53PM +0200, Ansgar wrote:
> On Tue, 2023-06-13 at 20:01 +0100, Matthew Garrett wrote:
> > After discussing this at our monthly meeting, we concluded that the 
> > technical committee isn't going to take action on this at the moment.
> > There's a legitimate technical consideration behind the warning, and 
> > it's necessary for derivatives to ensure that they handle the
> > associated scenarios properly.
> 
> Okay, I take that as "Debian doesn't care about derivatives". 
> Suggesting users to revert to split-/usr *will* break stuff for users
> once more code Debian assumes merged-/usr.

With my non-CTTE hat on, I think this is a demonstration that Debian 
*does* care about derivatives. It's important to ensure that derivatives 
are aware of the subtle interactions between dpkg and usrmerge, 
otherwise they may suffer from hard to diagnose issues on upgrades. The 
message from dpkg says nothing about reverting to split-/usr, and 
instead points at a wiki page. If our documentation on how to avoid 
these issues is incomplete (ie, if we're only describing how to avoid it 
by using split-/usr rather than describing the mitigations we've imposed 
within Debian to avoid triggering the issues), perhaps a better approach 
would be to improve that documentation? We don't benefit our users by 
pretending there isn't a problem here.

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Ansgar]

On Tue, 2023-06-13 at 20:01 +0100, Matthew Garrett wrote:
> After discussing this at our monthly meeting, we concluded that the 
> technical committee isn't going to take action on this at the moment.
> There's a legitimate technical consideration behind the warning, and 
> it's necessary for derivatives to ensure that they handle the
> associated scenarios properly.

Okay, I take that as "Debian doesn't care about derivatives". 
Suggesting users to revert to split-/usr *will* break stuff for users
once more code Debian assumes merged-/usr.

> While those underlying technical issues exist, we
> think it's premature for the committee to intervene on this specific 
> issue.

Okay, I guess the very long drama about this last year was not
sufficiently long and we did not discuss it in sufficient detail.

I'm a bit disappointed how the ctte appears to do as much as they can
to avoid deciding on this :-(

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Ansgar]

On Tue, 2023-06-06 at 11:04 +0100, Sean Whitton wrote:
> I don't think the TC has or should have any authority over dpkg
> upstream, but with dpkg being a native package, any implementation of
> our decision for the Debian archive is also implemented for dpkg
> upstream.  And it might be that the dpkg developers would be against
> the TC override solely or mostly because of this fact.  So possibly
> changing that would resolve things peacefully.

Given the Debian project owns dpkg.org, the upstream mailing list is
@lists.debian.org, official releases are published on deb.debian.org
and so on, I think the Debian project *is* the upstream for dpkg.

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Sean Whitton]

Hello Emilio,

On Thu 11 May 2023 at 12:30PM +02, Emilio Pozuelo Monfort wrote:

> I think you're conflating two independent things.
>
> If you override the dpkg maintainer to remove that warning that occurs on
> derivatives, then anyone could NMU dpkg and the maintainer wouldn't introduce
> it back, effectively removing the warning from "dpkg upstream".
>
> OTOH if the dpkg maintainer switches to non-native packages, anyone could NMU
> it adding the change as a patch, however the maintainer will just NACK the NMU
> before or after it happens.
>
> So I don't see a problem with dpkg being native, just like e.g. apt is, and
> that won't magically solve the issue at hand.

I don't think the TC has or should have any authority over dpkg
upstream, but with dpkg being a native package, any implementation of
our decision for the Debian archive is also implemented for dpkg
upstream.  And it might be that the dpkg developers would be against the
TC override solely or mostly because of this fact.  So possibly changing
that would resolve things peacefully.

I don't see how this conflates things, but would be grateful for more
explanation if you still think I'm doing so.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Matthew Vernon]

Hi,

This thread has rather veered off the initial bug report.

On 11/05/2023 13:16, Simon Richter wrote:

Hi,

On 5/11/23 10:59, Sean Whitton wrote:


Dear ctte, please consider overruling the dpkg maintainer to include
the patch from #994388[1].



Currently dpkg contains code to emit the merged-/usr warning, that's
dead code on Debian, but which becomes active when packages from the
Debian archive are copied unmodified into derivatives.


The way I see it (but I'm not a dpkg maintainer), the current 
implementation is correct, as dpkg does not support aliased directories, 
but Debian has decided to use it in such an environment nonetheless. The 
tech-ctte decision not to roll back usrmerge accepts responsibility for 
this decision, so silencing the warning on Debian is correct, but no one 
has accepted that responsibility for derived distributions.


Any derived distribution can easily go on record and request inclusion 
in the list of distributions where this warning is suppressed, by typing 
the phrase "Yes, I understand that this is a bad idea." into an email 
client.


I have considerable sympathy for this point of view. Further, given 
ongoing (and quite fruitful) discussion on how to resolve the 
outstanding issues around /usr-merge and dpkg, I don't think the 
question of dpkg's warning (and its unfortunate wording) is one that is 
useful for the technical committee (and the dpkg maintainers) to be 
spending time on right now.


I think I would feel differently if there were derivatives who had asked 
the dpkg maintainers to likewise exclude their distro from the warning 
had been rebuffed (though I suspect such folk will just be patching it 
out in their own builds).


Likewise I would expect that once we have finished sorting out the 
outstanding /usr-merge & dpkg issues that the warning would be removed.


But those scenarios aren't where we're at now, so I think the project 
should continue to focus on moving ourselves to the point where dpkg 
does support /usr-merge as implemented in Debian.


Regards,

Matthew

Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg)

[Luca Boccassi]

On Sun, 21 May 2023 at 15:51, Ansgar  wrote:
>
> On Sun, 2023-05-21 at 16:25 +0200, Helmut Grohne wrote:
> > On a process level, I think I miss attempts to resolve this with the
> > dpkg maintainer in a constructive way.
>
> The patch was already suggested to the dpkg maintainer and rejected.
>
> > Does anyone mind just closing the bug?
>
> Yes, I do.
>
> Please pass a resolution that you don't want to override the dpkg
> maintainer and that telling derivative users to configure their system
> in a way that will cause breakage is okay to do for packages in Debian.

I do as well. Recently a very strong consensus emerged that even
accidentally causing damage and/or incompatibility to
downstreams/external use cases is strongly frowned upon, to say the
least. Talking the talk is easy, now we have to walk the walk. Both
the warning and the 'unmess' tool cause significant damage and break
cross-compatibility, so they both need to be removed.

A "mind the moratorium" message would be of course very sensible to have.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg)

[Ansgar]

On Sun, 2023-05-21 at 16:25 +0200, Helmut Grohne wrote:
> On a process level, I think I miss attempts to resolve this with the
> dpkg maintainer in a constructive way.

The patch was already suggested to the dpkg maintainer and rejected.

> Does anyone mind just closing the bug?

Yes, I do.

Please pass a resolution that you don't want to override the dpkg
maintainer and that telling derivative users to configure their system
in a way that will cause breakage is okay to do for packages in Debian.

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg)

[Helmut Grohne]

Hi Ansgar,

I'm speaking with a CTTE hat here, but not representing CTTE consensus.

On Wed, May 10, 2023 at 11:47:42PM +0200, Ansgar wrote:
> Dear ctte, please consider overruling the dpkg maintainer to include
> the patch from #994388[1].

I think we need to reject this request on multiple levels.

On a social level, I see that you are frustrated with how dpkg is being
maintained and how communication does not work out in practice. While
part of that can be attributed to the dpkg maintainer, this goes both
ways and the way you refer this matter to the CTTE without even
attempting to resolve it by other means just serves to deepen that
dispute. I see that the dpkg maintainer has recently contributed
constructively to the discussion about how dpkg can be part of a
solution for the problems resulting from the /usr merge. I have a hard
time saying the same about your interaction here. Therefore, I think
your request should be rejected as not being serious.

On a process level, I think I miss attempts to resolve this with the
dpkg maintainer in a constructive way. The present discussion clearly
shows that dpkg's support for how Debian deals with merged /usr is
lacking. We are dealing with multiple file-loss scenarios (something we
otherwise consider grave) and issuing a warning about such behaviour
seems fine to me. On the other hand, much of the project seems to agree
that the way this warning is worded is far from optimal and evidently
leads to confused users. And while it may seem that any attempt at
resolving may lead nowhere, the same can be said about our dpkg
maintainer's concerns about our /usr-merge strategy and him pointing out
real problems affecting Debian installations in practice. Given the
recent constructive communication, I think it is far from clear that
this option is exhausted. In particular, acknowledging the problems
presented and proposing strategies for dealing with them could go a long
way towards a cooperative solution.  At present, I see the options to
keep and to delete the warning on the table, but there clearly is
unexplored middle ground.  As such, I think your request should be
rejected as not having exhausted the solution space.

On a technical level, Debian's support for merged /usr is currently
founded on the moratorium preventing breakage. Please observe that this
moratorium applies to Debian and Debian only. We have implemented
processes to validate this. If a derivative wants to use merged /usr
(which probably every derivative should), it certainly needs to prevent
breakage in some way - presumably using a similar process. I think that
the cost of patching dpkg is minor compared to the cost of a process
that prevents breakage arising from our merged /usr strategy.  Given
this, a warning-by-default (worded in a better way) for derivatives
actually can be a good thing, because it makes derivatives aware that
they cannot just ignore merged /usr, but have to act. As such, I think
your request should be rejected since the measure is technically
reasonable in principle.

Does anyone mind just closing the bug?

At the same time, I recommend changing the warning. The amount of
feedback we get regarding the warning should make it clear that the
current wording is still seen as offensive and causes confusion. Keeping
the warning in its current form also serves little but deepening the
dispute.

For instance, the wording "going behind dpkg's back" may be considered
technically correct, but it also can be objectively described as passive
aggressive. Just deleting this aspect and instead saying "This system
uses merged-usr-via-aliased-dirs which violates core assumptions of
dpkg." would probably keep the intended message in a less
confrontational way.  Elaborating that file loss is being mitigated by a
process (moratorium) on Debian would also help. Looking into the wiki, a
recommendation of dpkg-fsys-usrunmess should caution that using it now
breaks other tool's assumptions such as systemd and is generally not
being tested with QA anymore.

Even if one were thinking that the warning were appropriate as is,
adapting it again due to community feedback would demonstrate empathy
and be a step towards a cooperative solution.

I think our priority should be finding a way to terminate the
moratorium.

Helmut

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Ansgar]

Hi,

On Fri, 2023-05-19 at 07:09 -0700, Sean Whitton wrote:
> On Thu 18 May 2023 at 07:55PM +02, Ansgar wrote:
> 
> > Why not?
> 
> We will not move that fast.

So there is no real reason?

As there doesn't seem to be anything you think need to be talked about
that is missing to get to a decision (given you ignored all other
questions multiple times).

Do you expect this to take longer than 2-3 months? I would suggest to
just use a GR in that case: it seems quicker and less painful than a
multi-month long process for a simple(!) question.

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Sean Whitton]

Hello,

On Thu 18 May 2023 at 07:55PM +02, Ansgar wrote:

> Why not?

We will not move that fast.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Arnaud Rebillout]

On 19/05/2023 01:33, Luca Boccassi wrote:

We heard so much in the past couple of weeks about how important it is
for the project not to cause issues for derivatives and
cross-compatibility use cases, even speculatively. This is not even
speculative, it is certain to cause damage (as we experienced first
hard last year), I don't see how we can ignore it after all of these
discussions.

Speaking as Kali maintainer, we patched it out already a while ago:
https://gitlab.com/kalilinux/packages/dpkg/-/commit/bff5fa3c

Best,

Arnaud

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Gunnar Wolf]

Bastian Blank dijo [Thu, May 18, 2023 at 09:05:44PM +0200]:
> But why does the state of the package (native vs non-native) can have
> any effect on a CTTE decision?  Or do you want to say I can block CTTE
> from reaching any kind of decision just by uploading a package as
> native?  Sorry, but this does not compute.
> (...)
> Sure, but this is a direct violation of a CTTE decision.  How often do
> you think someone could do that?

During my time as a Technical Committe member, /usr-merge was the
point we most came back to. And yes, the way the TC decisions were
dodged or omitted by the dpkg maintainers was... quite depressing.

However, my reply should only be read regarding what I believe should
be done in the following ~month before the release.

Of course, I don't see the situation as ideal, nor as something that
should persist. I hope a _fixed_ dpkg is uploaded and becomes part of
Bookworm's first point release.

But, even if it were on the table (which is not AFAICT), I would (in a
strictly personal capacity) oppose the TC requiring such a patch at
this point.

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Matthias Klumpp]

Am Do., 18. Mai 2023 um 20:39 Uhr schrieb Luca Boccassi :
> [...]
> We heard so much in the past couple of weeks about how important it is
> for the project not to cause issues for derivatives and
> cross-compatibility use cases, even speculatively. This is not even
> speculative, it is certain to cause damage (as we experienced first
> hard last year), I don't see how we can ignore it after all of these
> discussions.

Speaking as maintainer of two Debian derivatives (PureOS and an
internal one), keeping this warning means we will need to patch dpkg
which of course is possible, but also a bit annoying. It is also odd
that Debian's configuration suddenly becomes "invalid" just by
changing the name of the OS.
(FWIW, PureOS has been usrmerged before Debian did that officially,
and so was Ubuntu - so far we haven't experienced any issues and our
users are happy - syncing dpkg without patching it will for sure cause
a lot of confusion though).

Cheers,
Matthias

-- 
I welcome VSRE emails. See http://vsre.info/

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Bastian Blank]

Hi Gunar

On Thu, May 18, 2023 at 12:14:42PM -0600, Gunnar Wolf wrote:
> dpkg has many bits that make it special. It has been discussed whethe
> dpkg should be a native package or it should become non-native; if it
> were non-native, having a patch that contradicts the upstream author's
> wishes would be easier (and I'm not saying that I'd be up for patching
> this warning out as it is).

But why does the state of the package (native vs non-native) can have
any effect on a CTTE decision?  Or do you want to say I can block CTTE
from reaching any kind of decision just by uploading a package as
native?  Sorry, but this does not compute.

> If we were to force a patch silencing out this warning right now and
> for all of the Bookworm cycle, and the dpkg authors disagree with it,
> they could remove (even omit to include it) in any updates.

Sure, but this is a direct violation of a CTTE decision.  How often do
you think someone could do that?

Bastian

-- 
"Life and death are seldom logical."
"But attaining a desired goal always is."
-- McCoy and Spock, "The Galileo Seven", stardate 2821.7

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Thu, 18 May 2023 at 19:27, Ansgar  wrote:
>
> On Thu, 2023-05-18 at 12:14 -0600, Gunnar Wolf wrote:
> > Ansgar dijo [Thu, May 18, 2023 at 07:55:03PM +0200]:
> > > Why not?
> > >
> > > Do you think the implications of removing the warning are unclear?
> > >
> > > Do you think we need to explore alternative solutions?
> >
> > (I am no longer part of the Committee, answering just as another
> > developer)
> >
> > dpkg has many bits that make it special. It has been discussed whethe
> > dpkg should be a native package or it should become non-native; if it
> > were non-native, having a patch that contradicts the upstream
> > author's wishes would be easier (and I'm not saying that I'd be up
> > for patching this warning out as it is).
>
> Do you think this implementation detail is relevant for what we do in
> Debian? I don't care how a patch is applied and don't think that detail
> has to be part of the decision.
>
> I also don't see any further active discussion on this aspect (unless I
> missed something).
>
>
> > If we were to force a patch silencing out this warning right now and
> > for all of the Bookworm cycle, and the dpkg authors disagree with it,
> > they could remove (even omit to include it) in any updates.
>
> So? That is the case with any ruling the ctte makes, including the non-
> binding one the ctte just did under Constitution 6.1.5.
>
> >  Upstream
> > has repeatedly expressed their opposition to the way usrmerge has
> > been brought forward, and the warning silenced specifically for
> > Debian is already the best compromise situation we have been able to
> > reach -- even though we are aware the situation is far from ideal.
>
> If the best solution we have been able to reach is telling users of
> derivative distributions to configure their system in a way that is
> expected to cause breakage, then it would be worth documenting that
> this is the case and we cannot do more for derivative users.
>
> If the ctte believes this to be fine, then the ctte can decide to not
> overrule the maintainer.
>
> I don't think this is a good reason to delay the decision indefinitely
> unless there is some reason to believe something will change within a
> reasonable period of time (which I don't see happening).

We heard so much in the past couple of weeks about how important it is
for the project not to cause issues for derivatives and
cross-compatibility use cases, even speculatively. This is not even
speculative, it is certain to cause damage (as we experienced first
hard last year), I don't see how we can ignore it after all of these
discussions.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Ansgar]

On Thu, 2023-05-18 at 12:14 -0600, Gunnar Wolf wrote:
> Ansgar dijo [Thu, May 18, 2023 at 07:55:03PM +0200]:
> > Why not?
> > 
> > Do you think the implications of removing the warning are unclear?
> > 
> > Do you think we need to explore alternative solutions?
> 
> (I am no longer part of the Committee, answering just as another
> developer)
> 
> dpkg has many bits that make it special. It has been discussed whethe
> dpkg should be a native package or it should become non-native; if it
> were non-native, having a patch that contradicts the upstream
> author's wishes would be easier (and I'm not saying that I'd be up
> for patching this warning out as it is).

Do you think this implementation detail is relevant for what we do in
Debian? I don't care how a patch is applied and don't think that detail
has to be part of the decision.

I also don't see any further active discussion on this aspect (unless I
missed something).


> If we were to force a patch silencing out this warning right now and
> for all of the Bookworm cycle, and the dpkg authors disagree with it,
> they could remove (even omit to include it) in any updates.

So? That is the case with any ruling the ctte makes, including the non-
binding one the ctte just did under Constitution 6.1.5.

>  Upstream
> has repeatedly expressed their opposition to the way usrmerge has
> been brought forward, and the warning silenced specifically for
> Debian is already the best compromise situation we have been able to
> reach -- even though we are aware the situation is far from ideal.

If the best solution we have been able to reach is telling users of
derivative distributions to configure their system in a way that is
expected to cause breakage, then it would be worth documenting that
this is the case and we cannot do more for derivative users.

If the ctte believes this to be fine, then the ctte can decide to not
overrule the maintainer.

I don't think this is a good reason to delay the decision indefinitely
unless there is some reason to believe something will change within a
reasonable period of time (which I don't see happening).

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Gunnar Wolf]

Ansgar dijo [Thu, May 18, 2023 at 07:55:03PM +0200]:
> Why not?
> 
> Do you think the implications of removing the warning are unclear?
> 
> Do you think we need to explore alternative solutions?

(I am no longer part of the Committee, answering just as another
developer)

dpkg has many bits that make it special. It has been discussed whethe
dpkg should be a native package or it should become non-native; if it
were non-native, having a patch that contradicts the upstream author's
wishes would be easier (and I'm not saying that I'd be up for patching
this warning out as it is).

If we were to force a patch silencing out this warning right now and
for all of the Bookworm cycle, and the dpkg authors disagree with it,
they could remove (even omit to include it) in any updates. Upstream
has repeatedly expressed their opposition to the way usrmerge has been
brought forward, and the warning silenced specifically for Debian is
already the best compromise situation we have been able to reach --
even though we are aware the situation is far from ideal.

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Ansgar]

Hi,

On Thu, 2023-05-18 at 10:48 -0700, Sean Whitton wrote:
> On Thu 18 May 2023 at 07:21PM +02, Ansgar wrote:
> 
> > The full freeze is approaching and there has been no progress on
> > this
> > issue. Does the ctte think a decision before the release is still
> > possible?
> 
> Not speaking for the whole ctte, but I don't think that is possible.

Why not?

Do you think the implications of removing the warning are unclear?

Do you think we need to explore alternative solutions?

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Sean Whitton]

Hello,

On Thu 18 May 2023 at 07:21PM +02, Ansgar wrote:

> The full freeze is approaching and there has been no progress on this
> issue. Does the ctte think a decision before the release is still
> possible?

Not speaking for the whole ctte, but I don't think that is possible.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg)

[Ansgar]

Hi,

On Thu, 2023-05-11 at 00:32 +0200, Ansgar wrote:
> On Wed, 2023-05-10 at 23:47 +0200, Ansgar wrote:
> > Cool, then let's ask tech-ctte.
> > 
> > Dear ctte, please consider overruling the dpkg maintainer to
> > include
> > the patch from #994388[1].
> > 
> > Thanks,
> > Ansgar
> > 
> >   [1]: https://bugs.debian.org/994388#397
> 
> For derivatives based on Debian stable it might be worth having this
> included in the next stable release; this would need a fairly quick
> decision on this issue.

The full freeze is approaching and there has been no progress on this
issue. Does the ctte think a decision before the release is still
possible?

As asked earlier I'm also interested in whether the ctte thinks there
is enough consensus about how this issue should be solved or do we
need a longer discussion to explore the solution space?

I admit not having read all mails in the thread as it went fairly off
topic IMHO.

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Roger Lynn]

On 15/05/2023 19:00, Simon McVittie wrote:
> On Sun, 14 May 2023 at 23:37:34 +0200, Josh Triplett wrote:
>> People build things on Debian that are not Debian packages. People
>> compile binaries on Debian, and expect them to work on any system that
>> has sufficiently new libraries.
> 
> *raises hand*
> 
> Hello, I represent an example of those people. With my $work hat on,
> I'm involved in maintaining a family of Debian derivatives (the Steam
> Runtime) that is used to run Steam games on arbitrary distributions,
> including but not limited to Debian. Some of these binaries are built
> on a Debian derivative, but run on an arbitrary other distribution,
> using a RPATH[1] to find their non-glibc dependencies.

As another much smaller example, which I hope is still relevant, I have
taken binaries from Debian stable or oldstable armel and run them for
diagnostic purposes on embedded boards which only have Busybox installed and
for which I don't have a convenient build environment.

Regards,
Roger

PS Apologies if I've followed up incorrectly - I read debian-devel through
an NNTP gateway.

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Tue, 16 May 2023 at 09:27, Simon McVittie  wrote:
>
> On Tue, 16 May 2023 at 02:50:48 +0100, Luca Boccassi wrote:
> > This sounds like a very interesting use case, and the first real one
> > mentioned, which is great to see - but I do not fully follow yet, from
> > what you are saying it seems to me that what you need is for your
> > binaries to use the usual pt_interp, that bit is clear. But why does
> > it matter if /usr/bin/ls on the host uses a different one?
>
> We don't need to run the ls from the host, but we do need to run
> glibc-related executables like ldconfig and localedef from either the
> host or the container runtime, whichever is newer. Because glibc is
> a single source package, executables and libraries within the glibc
> bubble sometimes make use of private symbols in libraries that are also
> within the glibc bubble (and IMO they have a right to do so), even though
> executables from outside glibc would be discouraged or disallowed from
> doing so. This means that when we have chosen a particular version of
> glibc (which, again, must be whichever one is newer), we try to use its
> matching version for *everything* originating in the glibc source package.
>
> In principle we could get exactly the same situation if we've imported a
> library from the host system (as a dependency of the graphics stack) that
> calls an executable as a subprocess and expects it to be >= the version
> it was compiled for - hopefully not (/usr)/bin/ls, but potentially others.

Thanks for the clarification, so if I understood correctly, your use
case is that sometimes (eg: when they are newer) you pull binaries
(eg: ldconfig) from the host, and run them from the container? So, in
case let's say ldconfig on the host points to /usr/lib/ld, but because
your container is not usr-merged, it wouldn't find the interpreter and
fail?

> The wider point than my specific use-case, though, is that when there's a
> standard, you can't predict what other software authors have looked at the
> statement "you can rely on this" and ... relied on it. See also Russ's
> (as ever, excellent) mails to the same thread.
>
> I appreciate that you are trying to explore the edges of the
> problem/constraint space and say "what if we did this, could that work?",
> and it's good that you are doing that; but part of that process is
> working with the other people on this list when they say "no, we can't
> do that because...", and respecting their input.

I respect and appreciate the input, but I want to understand it too,
hence the "because..." part is what I was looking for - so thanks for
providing it, it is really useful.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Simon McVittie]

On Tue, 16 May 2023 at 02:50:48 +0100, Luca Boccassi wrote:
> This sounds like a very interesting use case, and the first real one
> mentioned, which is great to see - but I do not fully follow yet, from
> what you are saying it seems to me that what you need is for your
> binaries to use the usual pt_interp, that bit is clear. But why does
> it matter if /usr/bin/ls on the host uses a different one?

We don't need to run the ls from the host, but we do need to run
glibc-related executables like ldconfig and localedef from either the
host or the container runtime, whichever is newer. Because glibc is
a single source package, executables and libraries within the glibc
bubble sometimes make use of private symbols in libraries that are also
within the glibc bubble (and IMO they have a right to do so), even though
executables from outside glibc would be discouraged or disallowed from
doing so. This means that when we have chosen a particular version of
glibc (which, again, must be whichever one is newer), we try to use its
matching version for *everything* originating in the glibc source package.

In principle we could get exactly the same situation if we've imported a
library from the host system (as a dependency of the graphics stack) that
calls an executable as a subprocess and expects it to be >= the version
it was compiled for - hopefully not (/usr)/bin/ls, but potentially others.

The wider point than my specific use-case, though, is that when there's a
standard, you can't predict what other software authors have looked at the
statement "you can rely on this" and ... relied on it. See also Russ's
(as ever, excellent) mails to the same thread.

I appreciate that you are trying to explore the edges of the
problem/constraint space and say "what if we did this, could that work?",
and it's good that you are doing that; but part of that process is
working with the other people on this list when they say "no, we can't
do that because...", and respecting their input.

Thanks,
smcv

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Mon, 15 May 2023 at 18:54, Simon McVittie  wrote:
>
> On Sun, 14 May 2023 at 23:37:34 +0200, Josh Triplett wrote:
> > People build things on Debian that are not Debian packages. People
> > compile binaries on Debian, and expect them to work on any system that
> > has sufficiently new libraries.
>
> *raises hand*
>
> Hello, I represent an example of those people. With my $work hat on,
> I'm involved in maintaining a family of Debian derivatives (the Steam
> Runtime) that is used to run Steam games on arbitrary distributions,
> including but not limited to Debian. Some of these binaries are built
> on a Debian derivative, but run on an arbitrary other distribution,
> using a RPATH[1] to find their non-glibc dependencies.
>
> At the moment those binaries are built in ancient environments (based
> on Ubuntu 12.04 and Debian 8), but as newer versions of glibc become
> ubiquitous, we'll want to start relying on newer versions of Debian
> (which will still be very old versions *at the time*, but I'm sure that
> by 2030 or 2040 we'll want to be using versions of Debian that, in 2023,
> are not released yet). In this use-case, yes we do need to be using the
> interoperable ELF interpreter path.
>
> We were able to use (Ubuntu and) Debian for this *because* Debian is
> a relatively "ordinary" distribution that tends to follow cross-distro
> standards. The major counterexample is multiarch library paths, but
> multiarch library paths have some compelling technical advantages, and
> because there's no ambiguity about which architecture uses which directory,
> they're actually better for interop in some ways than /usr/lib (which
> is ambiguous, because the Red Hat family of distros expects to find 32-bit
> libraries there, but the Arch family expects 64-bit libraries, and it's
> not possible to construct a tree where both get what they want).
>
> I've spent a lot of the last 5 years working on putting Steam games in
> containers so that they'll work more reliably on all distros, including
> Debian, with less reliance on weird library search paths (although we
> still have to use binaries built on an ancient Debian derivative with a
> non-trivial RPATH to bootstrap the container environment). Because of
> constraints around recent GPUs needing current graphics drivers even
> if running on an otherwise ancient library stack, Steam's container
> runtime has to construct a hybrid environment where glibc is either the
> one from the host or the one from the container runtime environment,
> whichever is newer; and while doing that, we have experienced some
> broken situations that were caused by distributions diverging from the
> interoperable ELF interpreter. One concrete example is that Arch Linux
> uses the interoperable ELF interpreter for *almost* all executables,
> but uses a different ELF interpreter for executables from the glibc
> source package, for whatever reason; we were able to work around this,
> but for a while it caused Arch to fail to launch these containers where
> Debian/Fedora/etc. could.

This sounds like a very interesting use case, and the first real one
mentioned, which is great to see - but I do not fully follow yet, from
what you are saying it seems to me that what you need is for your
binaries to use the usual pt_interp, that bit is clear. But why does
it matter if /usr/bin/ls on the host uses a different one? It's your
executables that you ship as part of that runtime that are the entry
points that need the usual loader path for your chroot-on-steroids,
no? The loader would still be reachable as it always was in this
theoretical exercise. I am probably missing something in how this
works in details.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Mon, 15 May 2023 at 16:18, Russ Allbery  wrote:
>
> Luca Boccassi  writes:
> > On Mon, 15 May 2023 at 02:26, Russ Allbery  wrote:
>
> >> (Also, no slight on the GUIX folks, but GUIX is not exactly an, uh,
> >> major player in Linux distributions, and I'm not sure how much they
> >> care about compatibility with anyone else.)
>
> > This is a counter-example to confute the assertion that *everybody* does
> > the same thing, which has been made multiple times. I'm not sure whether
> > it's an experiment or not, I mean if you ask their users/developers I
> > think they'd tell you it's very much production ready, but it is largely
> > irrelevant: it exists, and that's the only reason why it was mentioned,
> > as it shows that it is _possible_ to do that and be a working
> > distribution. Doesn't imply it's automatically desirable, but that was
> > not the intention.
>
> Ah, okay, I'm happy to agree with that point: you can violate the ABI and
> continue to be a working distribution.  (There are a lot of parts of the
> ABI that if you violated them you would not have a working distribution,
> but this is not one of them so far as I can tell.)
>
> > Not quite: my argument is that binaries from these packages are not
> > intended and not required to be ran on non-Debian systems, so there's no
> > incompatibility introduced in the first place - everything still works
> > where it is supposed to, exactly as it was before.
>
> I think we're saying the same thing but quibbling over phrasing.  I'd put
> that as saying that it's fine for the binaries of certain core Debian
> packages to be incompatible with the ABI because they're not intended to
> be used outside of Debian.  (In other words, I'm talking about
> incompatibility as a concrete, testable property of a binary, and I think
> you're talking about incompatibility as a more abstract concept of a
> distribution.)
>
> > No aggression intended whatsoever, sorry if it appeared that way. I am
> > trying to understand what the rules are.
>
> Well, the rule that I'd ideally set is don't break the ABI, even if it's
> not obvious why breaking the ABI is a bad idea or you can't see any bad
> consequences that could come from it, unless the reason for breaking the
> ABI is absolutely central to the mission and purpose of Debian.
>
> That said, it's not like we've never shipped a binary in Debian with a
> different PT_INTERP.  (I vaguely remember that some programming language
> uses PT_INTERP tricks for some sort of private binary scheme?  Objective
> CAML or something?  I ran across it once years ago and can't remember the
> details.  Also, IIRC klibc does some sort of PT_INTERP trick in some
> situations that I don't remember the details of, although I don't think it
> does that with general binaries.)  So I do see your point that you would
> prefer the rule to be more pragmatic than that.
>
> My counterargument is that this proposal seems to mostly be about avoiding
> having to create a symlink at a critical point in the bootstrap process,
> and while it's tricky to get the timing right (and therefore kind of
> annoying), the resulting usable system has to have that symlink anyway (I
> think there's no disagreement about that).  Not following the ABI for core
> binaries seems like a scary change with unknown consquences to a bunch of
> core packages to solve what looks like a relatively minor (if admittedly
> annoying!) problem.
>
> Note that the target of PT_INTERP on Debian is *already* a symlink, to
> /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 which was the multiarch path
> that we actually want to use.  We're already ensuring compatibility with a
> symlink and I think we should just keep doing that and not venture into
> these waters.

It's not even a proposal, it's a discussion to see "what would
actually break if X happened". That's why I'm asking for concrete use
cases, rather than theoretical points. Also, I am interested in what
the rules are. For example, glibc compatibility is just as important
if not more, why does that follow a different standard? If anything,
adding a missing symlink is trivial and a single command. Adding a
missing symbol to a shared object... not quite. There is an endless
list of downstream-only debianisms that break cross-compatibility in
just the same way (if not worse), and yet nobody cares about those.
Why?

> >> The world does not divide neatly into supported and unsupported use
> >> cases.  There are a lot of things I do to computers that I expect to
> >> work in some situations but not in others.  That includes, say, having
> >> a Debian chroot on some other OS and running binaries from that chroot
> >> without going into the chroot.  Often that will absolutely not work.
> >> Sometimes it will work, and it's convenient that it will work for some
> >> obscure recovery situations or other weird one-off use cases.  I've
> >> also copied files from working systems to broken systems running a
> >> different distribution before, 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Simon McVittie]

On Mon, 15 May 2023 at 06:48:04 +0200, Johannes Schauer Marin Rodrigues wrote:
> Obviously, with Luca's proposal, binaries from packages built with a different
> dynamic linker path in them would not work on distributions without 
> merged-/usr
> symlinks. But if the property of stuff from Debian being able to run on
> non-Debian non-merged-/usr systems is an important one, then why was it okay 
> to
> have merged-/usr as the default? Because with merged-/usr we already changed
> the interface contract for a lot of things because now binaries and libraries
> can also be found at other locations than on non-merged-/usr systems. A script
> with a /usr/bin/bash shebang built on and for Debian will not work on a system
> without the symlinks.

pre-bookworm gcc writes /lib64/ld-linux-x86-64.so.2 into the ELF header
of amd64 binaries and I think post-bookworm gcc should continue to do so
even though that has never been the physical path to the ELF interpreter
on Debian (it's really /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2, or
historically a version-numbered path). People who are only testing on
Debian systems, even in pre-merged-/usr releases like Debian 9, could
already have been relying on /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
existing; and now that we're saying merged-/usr is mandatory,
people who are only testing on Debian >= 12 could also start
relying on /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 or
/usr/lib64/ld-linux-x86-64.so.2; but we arrange for both/all paths
to exist, and we advise developers that they should only rely on the
interoperable /lib64/ld-linux-x86-64.so.2, treating all other paths that
lead to the same binary as an implementation detail.

I don't think it's any different to say that both /usr/bin/sh and /bin/sh
exist and will work, but ask that everyone should continue to use /bin/sh
and treat /usr/bin/sh as an implementation detail.

The way I think about merged /usr is a variant of Postel's principle:
be conservative in what you require, but be liberal about what will work.
A merged-/usr system can run software that assumes merged-/usr, and can
also run software that doesn't (because of the compatibility symlinks);
a non-merged-/usr system can run software that doesn't assume merged-/usr,
but software that makes that assumption will sometimes fail to work on it.

Consistent with that, despite being in favour of merged-/usr myself,
I didn't switch my development laptop or my autopkgtest VM images
to the merged-/usr setup until it became effectively mandatory
during the bookworm cycle - because if a package was doing something
not-maximally-portable, I wanted my development laptop or my test VM to
be one of the places it would fail to work, so that I would notice and
report that bug.

Conversely, I *did* switch non-development computers (servers and family
laptops) to merged-/usr, because on those systems the important thing is
for software to work, even if in theory it "doesn't deserve" to work.

On post-bookworm, merged-/usr Debian systems, if we keep using #!/bin/sh
and #!/usr/bin/perl scripts, and avoiding #!/usr/bin/sh and #!/bin/perl
scripts, then I think that's a positive thing for cross-distro interop -
and using the interoperable path to the ELF interpreter (dynamic linker)
is completely consistent with that.

As far as I can see, post-bookworm Lintian will continue to warn about
the non-interoperable spellings like #!/usr/bin/sh and #!/bin/perl,
unless changed to special-case them.

Note that the paths that are canonical from the point of view
of cross-distro interoperability (like #!/bin/sh) are not always
the same as the paths that are canonical from the point of view of
realpath(). *At the moment*, they are usually canonical from dpkg's
point of view, but that won't be the case forever, and I think that's
fine. /lib64/ld-linux-x86-64.so.2 isn't considered to be canonical by
either realpath() or dpkg either (dpkg knows it's a symlink, even on
non-merged-/usr systems where /lib64 is a real directory), but it's
canonical for the x86_64-linux-gnu ABI and that's what we say is most
important.

> With merged-/usr we already *did* "change fundamental things around" for
> reasons that are really not clear to me (but which i do not want to discuss
> here) and as a result did not "care about interoperability" (with those who do
> not also adopt it).

Didn't we? With merged /usr, the "theoretically correct" #!/bin/sh
scripts that have always worked will continue to work, and additionally,
"incorrect" #!/usr/bin/sh scripts will *also* work. That sounds like
caring about interoperability to me!

The one piece of interop we lose with merged-/usr becoming mandatory is
that if a developer has only tested on Debian (and other merged-/usr distros
like Fedora), if they have used a less-interoperable spelling of the path
like #!/usr/bin/sh or #!/bin/perl, their testing will not highlight that
source of potential non-interop with others.

However, I don't think it's credible to claim 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Simon McVittie]

On Sun, 14 May 2023 at 23:37:34 +0200, Josh Triplett wrote:
> People build things on Debian that are not Debian packages. People
> compile binaries on Debian, and expect them to work on any system that
> has sufficiently new libraries.

*raises hand*

Hello, I represent an example of those people. With my $work hat on,
I'm involved in maintaining a family of Debian derivatives (the Steam
Runtime) that is used to run Steam games on arbitrary distributions,
including but not limited to Debian. Some of these binaries are built
on a Debian derivative, but run on an arbitrary other distribution,
using a RPATH[1] to find their non-glibc dependencies.

At the moment those binaries are built in ancient environments (based
on Ubuntu 12.04 and Debian 8), but as newer versions of glibc become
ubiquitous, we'll want to start relying on newer versions of Debian
(which will still be very old versions *at the time*, but I'm sure that
by 2030 or 2040 we'll want to be using versions of Debian that, in 2023,
are not released yet). In this use-case, yes we do need to be using the
interoperable ELF interpreter path.

We were able to use (Ubuntu and) Debian for this *because* Debian is
a relatively "ordinary" distribution that tends to follow cross-distro
standards. The major counterexample is multiarch library paths, but
multiarch library paths have some compelling technical advantages, and
because there's no ambiguity about which architecture uses which directory,
they're actually better for interop in some ways than /usr/lib (which
is ambiguous, because the Red Hat family of distros expects to find 32-bit
libraries there, but the Arch family expects 64-bit libraries, and it's
not possible to construct a tree where both get what they want).

I've spent a lot of the last 5 years working on putting Steam games in
containers so that they'll work more reliably on all distros, including
Debian, with less reliance on weird library search paths (although we
still have to use binaries built on an ancient Debian derivative with a
non-trivial RPATH to bootstrap the container environment). Because of
constraints around recent GPUs needing current graphics drivers even
if running on an otherwise ancient library stack, Steam's container
runtime has to construct a hybrid environment where glibc is either the
one from the host or the one from the container runtime environment,
whichever is newer; and while doing that, we have experienced some
broken situations that were caused by distributions diverging from the
interoperable ELF interpreter. One concrete example is that Arch Linux
uses the interoperable ELF interpreter for *almost* all executables,
but uses a different ELF interpreter for executables from the glibc
source package, for whatever reason; we were able to work around this,
but for a while it caused Arch to fail to launch these containers where
Debian/Fedora/etc. could.

This is not something that any of the distributions involved is
going to formally support, so the overall system consists of various
unsupported-but-works-in-practice things happening; but there is
absolutely no chance of Valve/Steam shipping separate binaries for
each distro (there are too many distros for that, even if you don't
count source-based distros like Gentoo that have an almost unlimited
number of concrete instantiations as binaries). However loudly we
might insist that our small subset-of-a-subset is the one they should
be targeting, what we're going to get is binaries that work on "mostly
interoperable" distros. As a result, if we want games on Linux (and
anything else requiring binary interop) to continue to be a thing,
pragmatically we should aim to be one of those "mostly interoperable"
distros, and encourage our friends and/or rivals in other distros to
do the same. Otherwise, the most likely outcome is for developers to go
back to an attitude of "I'm not going to support Linux, too many random
differences" and releasing for Windows only, and we all lose out.

As a result of all this, I would strongly prefer our compiler to
default to hard-coding the same interoperable ELF interpreter that
glibc upstream and the various major distros have agreed on (for example
/lib64/ld-linux-x86-64.so.2 on amd64), and I would also prefer it if we
can use that interoperable path in all the binaries we ship, including
src:glibc and the rest of the transitively Essential set.

One way that I like to think about this sort of thing is that we have a
finite number of "weird Debianism" tokens available, and we should aim
to spend them on the things that give us the best cost/benefit ratio.
We've never considered changing the ELF interpreter to be one of those,
to the extent of having a /lib64 solely for its benefit (on amd64)
even though by policy we don't generally use lib64.

(Incidentally, Steam's container runtime is always a merged-/usr
environment, to provide an environment that maximizes the probability
of any given script or binary working correctly; but it 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Sam Hartman]

> "Matthew" == Matthew Vernon  writes:

Matthew> On 15/05/2023 16:54, Bdale Garbee wrote:
>> I could.
>> 
>> Can you provide an example of actual value delivered to Debian
>> from merged-/usr?

Matthew> With respect, I don't think this line of argument is going
Matthew> to get us very far - this bug isn't about whether we should
Matthew> undo usr-merge, so I don't think a debate on the merits or
Matthew> otherwise of usr-merge is germane.

I actually think the whole ABI issue is basically irrelevant for this
bug.
I think that in this bug at least we are hoping to discuss the
appropriateness of the dpkg warning for sources downstreams grab from
the Debian archivje.
That issue seems quite sufficient for one bug:-)

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Bdale Garbee]

You are of course correct.  

I remain unconvinced that anything related to the work on merged-/usr to date 
should be considered as a positive justification for actions discussed in this 
thread, but we can just let the rest drop.

Bdale 

On May 15, 2023 10:08:00 AM MDT, Matthew Vernon  wrote:
>On 15/05/2023 16:54, Bdale Garbee wrote:
>> I could.
>> 
>> Can you provide an example of actual value delivered to Debian from 
>> merged-/usr?
>
>With respect, I don't think this line of argument is going to get us very far 
>- this bug isn't about whether we should undo usr-merge, so I don't think a 
>debate on the merits or otherwise of usr-merge is germane.
>
>[I think this applies to the contrary side of the argument also]
>
>Regards,
>
>Matthew
>

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Matthew Vernon]

On 15/05/2023 16:54, Bdale Garbee wrote:

I could.

Can you provide an example of actual value delivered to Debian from 
merged-/usr?


With respect, I don't think this line of argument is going to get us 
very far - this bug isn't about whether we should undo usr-merge, so I 
don't think a debate on the merits or otherwise of usr-merge is germane.


[I think this applies to the contrary side of the argument also]

Regards,

Matthew

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Bdale Garbee]

I could.  

Can you provide an example of actual value delivered to Debian from merged-/usr?

Bdale 

On May 15, 2023 7:15:53 AM MDT, Luca Boccassi  wrote:
>On Mon, 15 May 2023 at 13:51, Bdale Garbee  wrote:
>>
>> Merged-/usr seems to me to have brought great pain with no discernable 
>> benefit to Debian so far, and I at least have completely lost the thread on 
>> what the point of doing it was supposed to be.  So, using it as a 
>> justification for further harm to user and system expectations isn't 
>> compelling.
>
>Are you able to provide an example of such "harm"?
>
>Kind regards,
>Luca Boccassi
>

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Russ Allbery]

Luca Boccassi  writes:
> On Mon, 15 May 2023 at 02:26, Russ Allbery  wrote:

>> (Also, no slight on the GUIX folks, but GUIX is not exactly an, uh,
>> major player in Linux distributions, and I'm not sure how much they
>> care about compatibility with anyone else.)

> This is a counter-example to confute the assertion that *everybody* does
> the same thing, which has been made multiple times. I'm not sure whether
> it's an experiment or not, I mean if you ask their users/developers I
> think they'd tell you it's very much production ready, but it is largely
> irrelevant: it exists, and that's the only reason why it was mentioned,
> as it shows that it is _possible_ to do that and be a working
> distribution. Doesn't imply it's automatically desirable, but that was
> not the intention.

Ah, okay, I'm happy to agree with that point: you can violate the ABI and
continue to be a working distribution.  (There are a lot of parts of the
ABI that if you violated them you would not have a working distribution,
but this is not one of them so far as I can tell.)

> Not quite: my argument is that binaries from these packages are not
> intended and not required to be ran on non-Debian systems, so there's no
> incompatibility introduced in the first place - everything still works
> where it is supposed to, exactly as it was before.

I think we're saying the same thing but quibbling over phrasing.  I'd put
that as saying that it's fine for the binaries of certain core Debian
packages to be incompatible with the ABI because they're not intended to
be used outside of Debian.  (In other words, I'm talking about
incompatibility as a concrete, testable property of a binary, and I think
you're talking about incompatibility as a more abstract concept of a
distribution.)

> No aggression intended whatsoever, sorry if it appeared that way. I am
> trying to understand what the rules are.

Well, the rule that I'd ideally set is don't break the ABI, even if it's
not obvious why breaking the ABI is a bad idea or you can't see any bad
consequences that could come from it, unless the reason for breaking the
ABI is absolutely central to the mission and purpose of Debian.

That said, it's not like we've never shipped a binary in Debian with a
different PT_INTERP.  (I vaguely remember that some programming language
uses PT_INTERP tricks for some sort of private binary scheme?  Objective
CAML or something?  I ran across it once years ago and can't remember the
details.  Also, IIRC klibc does some sort of PT_INTERP trick in some
situations that I don't remember the details of, although I don't think it
does that with general binaries.)  So I do see your point that you would
prefer the rule to be more pragmatic than that.

My counterargument is that this proposal seems to mostly be about avoiding
having to create a symlink at a critical point in the bootstrap process,
and while it's tricky to get the timing right (and therefore kind of
annoying), the resulting usable system has to have that symlink anyway (I
think there's no disagreement about that).  Not following the ABI for core
binaries seems like a scary change with unknown consquences to a bunch of
core packages to solve what looks like a relatively minor (if admittedly
annoying!) problem.

Note that the target of PT_INTERP on Debian is *already* a symlink, to
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 which was the multiarch path
that we actually want to use.  We're already ensuring compatibility with a
symlink and I think we should just keep doing that and not venture into
these waters.

>> The world does not divide neatly into supported and unsupported use
>> cases.  There are a lot of things I do to computers that I expect to
>> work in some situations but not in others.  That includes, say, having
>> a Debian chroot on some other OS and running binaries from that chroot
>> without going into the chroot.  Often that will absolutely not work.
>> Sometimes it will work, and it's convenient that it will work for some
>> obscure recovery situations or other weird one-off use cases.  I've
>> also copied files from working systems to broken systems running a
>> different distribution before, and there's a list of caveats as long as
>> my arm, but sometimes it's a quick fix for something.

> Vast, vast majority of binaries from existing packages will already
> not work out of the box in that use case though.

I'm not sure why you think this is true and it makes me wonder if maybe my
intuition about cross-distribution compatibility is wrong.  I would expect
to be able to copy, say, most (all?) binaries from coreutils from a Debian
system to some other distribution and run it (and that's exactly the sort
of binary that is useful in this kind of cross-distribution rescue case).
Is this not true today?  What breaks?

Note that we're not talking about complicated packages with lots of
runtime like, say, Emacs.  As I understand it your proposal wouldn't
change PT_INTERP for that binary anyway. 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Mon, 15 May 2023 at 14:36, Steve McIntyre  wrote:
>
> Hey Johannes,
>
> On Mon, May 15, 2023 at 06:48:04AM +0200, Johannes Schauer Marin Rodrigues 
> wrote:
> >So did we not years ago decide, that the result of the "cross- and
> >inter-project discussion" is, that everybody is going merged-/usr and that's
> >why we need it too and that's why it is okay to build a system where binaries
> >and scripts built for it just may not run on those other systems that do not 
> >do
> >it?  With merged-/usr we already *did* "change fundamental things around" for
> >reasons that are really not clear to me (but which i do not want to discuss
> >here) and as a result did not "care about interoperability" (with those who 
> >do
> >not also adopt it). In my own Debian work I so far only got extra work 
> >because
> >of merged-/usr and I do not see the benefits (yet) and I was hoping that
> >"changing fundamental things around Linux and (basically) not caring about
> >interoperability" was *not* Debian's attitude but alas here we are.
> >
> >So have we not already burned the bridges to the non-merged-/usr world? Why 
> >was
> >it okay back then to say "we can make this change because all other important
> >players are doing merged-/usr so we can/have to as well". And now in the
> >PT_INTERP discussion somehow we care again about those systems? I thought we
> >already had the "cross- and inter-project discussion" about merged-/usr and
> >because the result was "yes, go for it" we did it too. But if that is the 
> >case,
> >why do we now care for a subset of the interoperability problems caused by
> >merged-/usr for systems that don't have it?
>
> This change is absolutely *not* needed to make merged-/usr work; if
> anybody is claiming that it is, then they are not being 100% honest
> with us. All the other distros doing merged-/usr have done it without
> making this change, and it's also been working OK for us so far
> without this change.

That is absolutely true, it is not mandatory. It is one possible
solution (of many) to a particular use case being sounded out, that's
all. I don't think it was mentioned by anybody as needed, if it was,
happy to clarify.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Steve McIntyre]

Hey Johannes,

On Mon, May 15, 2023 at 06:48:04AM +0200, Johannes Schauer Marin Rodrigues 
wrote:
>Quoting Steve McIntyre (2023-05-15 02:54:02)
>> 
>> Pointing at gentoo or nixos as examples of projects that have decided
>> to break compatibility doesn't cut it, I'm afraid. They're well known
>> for changing fundamental things around Linux and (basically) not caring about
>> interoperability. That attitude is *not* Debian's.
>
>me and Luca have different ideas about how bootstrapping a Debian chroot should
>look like and I don't want to make an argument *for* changing PT_INTERP here as
>I think that keeping compatibility with others by following ABI is a good thing
>and because I think (and hope -- but Helmut is doing that analysis right now)
>that the debootstrap problem can be solved in a way I envision without changing
>PT_INTERP. But what I do not understand about the argument against Luca's
>proposal is:
>
>Obviously, with Luca's proposal, binaries from packages built with a different
>dynamic linker path in them would not work on distributions without merged-/usr
>symlinks. But if the property of stuff from Debian being able to run on
>non-Debian non-merged-/usr systems is an important one, then why was it okay to
>have merged-/usr as the default? Because with merged-/usr we already changed
>the interface contract for a lot of things because now binaries and libraries
>can also be found at other locations than on non-merged-/usr systems. A script
>with a /usr/bin/bash shebang built on and for Debian will not work on a system
>without the symlinks.

Despite the massive upheavals of merged-/usr in *other* ways, it's
actually a *minor* change as far as compatibility is concerned
here. The runtime linker (aka interpreter) is responsible for
resolving symbols and finding needed libraries. So long as *that* bit
works OK, then ~everything else should follow OK. This is how it's
possible to have things work across distros: binaries don't actually
care where the libraries live, or whether they came from rpm or deb
packaging, etc.

The issue at hand here is that the interpreter path is the most basic
thing that matters for this compatibility. Break this and *nothing*
can work.

>So did we not years ago decide, that the result of the "cross- and
>inter-project discussion" is, that everybody is going merged-/usr and that's
>why we need it too and that's why it is okay to build a system where binaries
>and scripts built for it just may not run on those other systems that do not do
>it?  With merged-/usr we already *did* "change fundamental things around" for
>reasons that are really not clear to me (but which i do not want to discuss
>here) and as a result did not "care about interoperability" (with those who do
>not also adopt it). In my own Debian work I so far only got extra work because
>of merged-/usr and I do not see the benefits (yet) and I was hoping that
>"changing fundamental things around Linux and (basically) not caring about
>interoperability" was *not* Debian's attitude but alas here we are.
>
>So have we not already burned the bridges to the non-merged-/usr world? Why was
>it okay back then to say "we can make this change because all other important
>players are doing merged-/usr so we can/have to as well". And now in the
>PT_INTERP discussion somehow we care again about those systems? I thought we
>already had the "cross- and inter-project discussion" about merged-/usr and
>because the result was "yes, go for it" we did it too. But if that is the case,
>why do we now care for a subset of the interoperability problems caused by
>merged-/usr for systems that don't have it?

This change is absolutely *not* needed to make merged-/usr work; if
anybody is claiming that it is, then they are not being 100% honest
with us. All the other distros doing merged-/usr have done it without
making this change, and it's also been working OK for us so far
without this change.

Breaking an agreed interface contract like this is axiomatically
*wrong* and will hurt us and the rest of the Linux ecosystem.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"You can't barbecue lettuce!" -- Ellie Crane

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Mon, 15 May 2023 at 13:51, Bdale Garbee  wrote:
>
> Merged-/usr seems to me to have brought great pain with no discernable 
> benefit to Debian so far, and I at least have completely lost the thread on 
> what the point of doing it was supposed to be.  So, using it as a 
> justification for further harm to user and system expectations isn't 
> compelling.

Are you able to provide an example of such "harm"?

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Bdale Garbee]

Merged-/usr seems to me to have brought great pain with no discernable benefit 
to Debian so far, and I at least have completely lost the thread on what the 
point of doing it was supposed to be.  So, using it as a justification for 
further harm to user and system expectations isn't compelling.

Bdale 

On May 14, 2023 10:48:04 PM MDT, Johannes Schauer Marin Rodrigues 
 wrote:
>Hi,
>
>Quoting Steve McIntyre (2023-05-15 02:54:02)
>> On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote:
>> >On Sun, 14 May 2023 at 22:37, Josh Triplett  wrote:
>> >
>> >> The x86-64 ABI is set. Feel free to make the case to the next
>> >> architecture designer that their new ABI should have the dynamic linker
>> >> in `/usr/lib`. That would *not* have the same downsides, as long as
>> >> everyone agrees on a path.
>> >
>> >In practice it is not, though. There are other distributions that
>> >change PT_INTERP for their own purposes, they've already been listed
>> >in this thread. And I am still not hearing any concrete, factual use
>> >case that would be impaired by such a change. I'm beginning to
>> >seriously think there aren't any? Is that really the case?
>> 
>> The ABI has been agreed and set down in documentation that *just
>> about* everybody has been following since its inception. This includes
>> the most basic set of definitions of what an x86-64 program must look
>> like, including the interpreter path. If this path is changed, then
>> *at the most basic level* we'd be making programs that are not valid
>> by the ABI we've agreed to. This is an *external interface contract*,
>> not something we should ever consider changing without significant
>> cross- and inter-project discussion.
>> 
>> Pointing at gentoo or nixos as examples of projects that have decided
>> to break compatibility doesn't cut it, I'm afraid. They're well known
>> for changing fundamental things around Linux and (basically) not caring about
>> interoperability. That attitude is *not* Debian's.
>
>me and Luca have different ideas about how bootstrapping a Debian chroot should
>look like and I don't want to make an argument *for* changing PT_INTERP here as
>I think that keeping compatibility with others by following ABI is a good thing
>and because I think (and hope -- but Helmut is doing that analysis right now)
>that the debootstrap problem can be solved in a way I envision without changing
>PT_INTERP. But what I do not understand about the argument against Luca's
>proposal is:
>
>Obviously, with Luca's proposal, binaries from packages built with a different
>dynamic linker path in them would not work on distributions without merged-/usr
>symlinks. But if the property of stuff from Debian being able to run on
>non-Debian non-merged-/usr systems is an important one, then why was it okay to
>have merged-/usr as the default? Because with merged-/usr we already changed
>the interface contract for a lot of things because now binaries and libraries
>can also be found at other locations than on non-merged-/usr systems. A script
>with a /usr/bin/bash shebang built on and for Debian will not work on a system
>without the symlinks.
>
>So did we not years ago decide, that the result of the "cross- and
>inter-project discussion" is, that everybody is going merged-/usr and that's
>why we need it too and that's why it is okay to build a system where binaries
>and scripts built for it just may not run on those other systems that do not do
>it?  With merged-/usr we already *did* "change fundamental things around" for
>reasons that are really not clear to me (but which i do not want to discuss
>here) and as a result did not "care about interoperability" (with those who do
>not also adopt it). In my own Debian work I so far only got extra work because
>of merged-/usr and I do not see the benefits (yet) and I was hoping that
>"changing fundamental things around Linux and (basically) not caring about
>interoperability" was *not* Debian's attitude but alas here we are.
>
>So have we not already burned the bridges to the non-merged-/usr world? Why was
>it okay back then to say "we can make this change because all other important
>players are doing merged-/usr so we can/have to as well". And now in the
>PT_INTERP discussion somehow we care again about those systems? I thought we
>already had the "cross- and inter-project discussion" about merged-/usr and
>because the result was "yes, go for it" we did it too. But if that is the case,
>why do we now care for a subset of the interoperability problems caused by
>merged-/usr for systems that don't have it?
>
>As I said, I don't care much about the PT_INTERP value but I don't understand
>yet, why this argument about interoperability with non-merged-/usr systems is
>working now but it didn't wasn't enough to stop another very fundamental change
>in how we build a Linux distro.
>
>Thanks!
>
>cheers, josch

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Steve McIntyre (2023-05-15 02:54:02)
> On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote:
> >On Sun, 14 May 2023 at 22:37, Josh Triplett  wrote:
> >
> >> The x86-64 ABI is set. Feel free to make the case to the next
> >> architecture designer that their new ABI should have the dynamic linker
> >> in `/usr/lib`. That would *not* have the same downsides, as long as
> >> everyone agrees on a path.
> >
> >In practice it is not, though. There are other distributions that
> >change PT_INTERP for their own purposes, they've already been listed
> >in this thread. And I am still not hearing any concrete, factual use
> >case that would be impaired by such a change. I'm beginning to
> >seriously think there aren't any? Is that really the case?
> 
> The ABI has been agreed and set down in documentation that *just
> about* everybody has been following since its inception. This includes
> the most basic set of definitions of what an x86-64 program must look
> like, including the interpreter path. If this path is changed, then
> *at the most basic level* we'd be making programs that are not valid
> by the ABI we've agreed to. This is an *external interface contract*,
> not something we should ever consider changing without significant
> cross- and inter-project discussion.
> 
> Pointing at gentoo or nixos as examples of projects that have decided
> to break compatibility doesn't cut it, I'm afraid. They're well known
> for changing fundamental things around Linux and (basically) not caring about
> interoperability. That attitude is *not* Debian's.

me and Luca have different ideas about how bootstrapping a Debian chroot should
look like and I don't want to make an argument *for* changing PT_INTERP here as
I think that keeping compatibility with others by following ABI is a good thing
and because I think (and hope -- but Helmut is doing that analysis right now)
that the debootstrap problem can be solved in a way I envision without changing
PT_INTERP. But what I do not understand about the argument against Luca's
proposal is:

Obviously, with Luca's proposal, binaries from packages built with a different
dynamic linker path in them would not work on distributions without merged-/usr
symlinks. But if the property of stuff from Debian being able to run on
non-Debian non-merged-/usr systems is an important one, then why was it okay to
have merged-/usr as the default? Because with merged-/usr we already changed
the interface contract for a lot of things because now binaries and libraries
can also be found at other locations than on non-merged-/usr systems. A script
with a /usr/bin/bash shebang built on and for Debian will not work on a system
without the symlinks.

So did we not years ago decide, that the result of the "cross- and
inter-project discussion" is, that everybody is going merged-/usr and that's
why we need it too and that's why it is okay to build a system where binaries
and scripts built for it just may not run on those other systems that do not do
it?  With merged-/usr we already *did* "change fundamental things around" for
reasons that are really not clear to me (but which i do not want to discuss
here) and as a result did not "care about interoperability" (with those who do
not also adopt it). In my own Debian work I so far only got extra work because
of merged-/usr and I do not see the benefits (yet) and I was hoping that
"changing fundamental things around Linux and (basically) not caring about
interoperability" was *not* Debian's attitude but alas here we are.

So have we not already burned the bridges to the non-merged-/usr world? Why was
it okay back then to say "we can make this change because all other important
players are doing merged-/usr so we can/have to as well". And now in the
PT_INTERP discussion somehow we care again about those systems? I thought we
already had the "cross- and inter-project discussion" about merged-/usr and
because the result was "yes, go for it" we did it too. But if that is the case,
why do we now care for a subset of the interoperability problems caused by
merged-/usr for systems that don't have it?

As I said, I don't care much about the PT_INTERP value but I don't understand
yet, why this argument about interoperability with non-merged-/usr systems is
working now but it didn't wasn't enough to stop another very fundamental change
in how we build a Linux distro.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Mon, 15 May 2023 at 02:26, Russ Allbery  wrote:
>
> Luca Boccassi  writes:
>
> > That's self-evidently not true, as there are other distributions where
> > that already happens, it's been already mentioned.
>
> You've mentioned this a couple of times but I don't think I've seen the
> message where the details were explained.  Maybe this was only in your
> message posted to debian-gcc, which wasn't part of this thread?  (It's
> also possible that I just missed it somewhere.)
>
> That message only mentions GUIX, which I don't know very much about, but
> my recollection (maybe wrong?) is that it's a NIX variant that is doing
> special tricks to support immutable package trees and
> roll-forward/roll-back upgrades.  I can see why that might be motivation
> to build incompatible binaries in order to preserve some other invariant
> they're trying for as the point of their distribution (in particular, I
> suspect they're pinning binaries to a specific version of the dynamic
> loader as part of the whole immutable tree strategy).  That's a perfectly
> fine decision in a distribution that's trying to do something very
> different and is a bit of a science experiment, but I don't think that
> describes Debian.
>
> (Also, no slight on the GUIX folks, but GUIX is not exactly an, uh, major
> player in Linux distributions, and I'm not sure how much they care about
> compatibility with anyone else.)

This is a counter-example to confute the assertion that *everybody*
does the same thing, which has been made multiple times. I'm not sure
whether it's an experiment or not, I mean if you ask their
users/developers I think they'd tell you it's very much production
ready, but it is largely irrelevant: it exists, and that's the only
reason why it was mentioned, as it shows that it is _possible_ to do
that and be a working distribution. Doesn't imply it's automatically
desirable, but that was not the intention.

> > Besides, we are not talking about sacred religious texts - the point is
> > making things work. If they do, is it _really_
> > non-compliant/incompatible?
>
> I understand your point in making this argument, but please understand
> that this sort of willingness to change things if one didn't think they
> would cause problems didn't work very well, and was part of what led to
> the development of standardized ABIs in the first place.  Those of us who
> have been around longer than Linux have ABIs have a bit of a strong
> reaction here (I think this is also what you're seeing from Steve),
> because we remember the bad old days.  I still have compatibility code
> around to handle the fact that gcc on IRIX miscompiled calls to inet_ntoa
> because gcc didn't correctly implement the IRIX ABI.
>
> People are very bad at judging whether their new idea would be *really*
> incompatible.  This is why these days everyone tries to follow the ABI
> pretty closely.
>
> And in any case, changing PT_INTERP is trivially and obviously
> incompatible; the binary will simply not run on a system that doesn't have
> that path.  So it's not like we have to carefully judge nuance here.  Your
> argument, so far as I can tell, is basically "but no one will ever want to
> run those binaries on a non-/usr-merged system anyway," which is basically
> conceding the incompatibility point since the ABI doesn't require merged
> /usr.

Not quite: my argument is that binaries from these packages are not
intended and not required to be ran on non-Debian systems, so there's
no incompatibility introduced in the first place - everything still
works where it is supposed to, exactly as it was before.

> There's also some other history here: Debian is not super-happy with the
> PT_INTERP because ideally we'd prefer it use a path compatible with our
> multiarch approach.  I believe we raised that and no one had any interest
> in trying to change anything, so we lived with the limitations that
> creates.  (And I think that was the right decision.)
>
> > Thanks, that's the first actual real example mentioned so far. And it's
> > an interesting one: taking a $random Debian package and using it on a
> > completely different, non-Debian system. Is that a supported use case?
> > If so, does that mean that I can go ahead and raise a Severity: serious
> > bug on any package that doesn't work in such a way?
>
> I feel like you're distorting my argument here to try to make some sort of
> slippery slope argument, and it's coming across as possibly more
> aggressive than you had intended.

No aggression intended whatsoever, sorry if it appeared that way. I am
trying to understand what the rules are.

> The world does not divide neatly into supported and unsupported use cases.
> There are a lot of things I do to computers that I expect to work in some
> situations but not in others.  That includes, say, having a Debian chroot
> on some other OS and running binaries from that chroot without going into
> the chroot.  Often that will absolutely not work.  Sometimes it will 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Russ Allbery]

Luca Boccassi  writes:

> That's self-evidently not true, as there are other distributions where
> that already happens, it's been already mentioned.

You've mentioned this a couple of times but I don't think I've seen the
message where the details were explained.  Maybe this was only in your
message posted to debian-gcc, which wasn't part of this thread?  (It's
also possible that I just missed it somewhere.)

That message only mentions GUIX, which I don't know very much about, but
my recollection (maybe wrong?) is that it's a NIX variant that is doing
special tricks to support immutable package trees and
roll-forward/roll-back upgrades.  I can see why that might be motivation
to build incompatible binaries in order to preserve some other invariant
they're trying for as the point of their distribution (in particular, I
suspect they're pinning binaries to a specific version of the dynamic
loader as part of the whole immutable tree strategy).  That's a perfectly
fine decision in a distribution that's trying to do something very
different and is a bit of a science experiment, but I don't think that
describes Debian.

(Also, no slight on the GUIX folks, but GUIX is not exactly an, uh, major
player in Linux distributions, and I'm not sure how much they care about
compatibility with anyone else.)

> Besides, we are not talking about sacred religious texts - the point is
> making things work. If they do, is it _really_
> non-compliant/incompatible?

I understand your point in making this argument, but please understand
that this sort of willingness to change things if one didn't think they
would cause problems didn't work very well, and was part of what led to
the development of standardized ABIs in the first place.  Those of us who
have been around longer than Linux have ABIs have a bit of a strong
reaction here (I think this is also what you're seeing from Steve),
because we remember the bad old days.  I still have compatibility code
around to handle the fact that gcc on IRIX miscompiled calls to inet_ntoa
because gcc didn't correctly implement the IRIX ABI.

People are very bad at judging whether their new idea would be *really*
incompatible.  This is why these days everyone tries to follow the ABI
pretty closely.

And in any case, changing PT_INTERP is trivially and obviously
incompatible; the binary will simply not run on a system that doesn't have
that path.  So it's not like we have to carefully judge nuance here.  Your
argument, so far as I can tell, is basically "but no one will ever want to
run those binaries on a non-/usr-merged system anyway," which is basically
conceding the incompatibility point since the ABI doesn't require merged
/usr.

There's also some other history here: Debian is not super-happy with the
PT_INTERP because ideally we'd prefer it use a path compatible with our
multiarch approach.  I believe we raised that and no one had any interest
in trying to change anything, so we lived with the limitations that
creates.  (And I think that was the right decision.)

> Thanks, that's the first actual real example mentioned so far. And it's
> an interesting one: taking a $random Debian package and using it on a
> completely different, non-Debian system. Is that a supported use case?
> If so, does that mean that I can go ahead and raise a Severity: serious
> bug on any package that doesn't work in such a way?

I feel like you're distorting my argument here to try to make some sort of
slippery slope argument, and it's coming across as possibly more
aggressive than you had intended.

The world does not divide neatly into supported and unsupported use cases.
There are a lot of things I do to computers that I expect to work in some
situations but not in others.  That includes, say, having a Debian chroot
on some other OS and running binaries from that chroot without going into
the chroot.  Often that will absolutely not work.  Sometimes it will work,
and it's convenient that it will work for some obscure recovery situations
or other weird one-off use cases.  I've also copied files from working
systems to broken systems running a different distribution before, and
there's a list of caveats as long as my arm, but sometimes it's a quick
fix for something.

But mostly my reaction is because breaking the ABI is a Really Big Deal.
Constructing the Linux ABI and getting the details actually published was
a hard-fought, arduous endeavor.  I doubt anyone enjoyed it; it's the sort
of annoying compatibility work that provides tons of small, subtle
benefits and takes a great deal of truly thankless work, and people often
don't realize all the tiny ways that it has made the world a better place,
or the range of weird compatibility problems that can arise from messing
with it.  Diverging from it is not something to do lightly, precisely
*because* it's often extremely difficult to understand what the effects
could be or what might break.

While I appreciate how it would make bootstrapping Debian somewhat more

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Steve McIntyre]

I'm *trying* to assume good faith here, but I'm running out of energy
to do so.

On Mon, May 15, 2023 at 01:42:27AM +0100, Luca Boccassi wrote:
>On Mon, 15 May 2023 at 01:14, Russ Allbery  wrote:
>
>> Incidentally, that remains true even if we only do that in distribution
>> packages.  I certainly have copied binaries from a Debian package to other
>> Linux systems before for various reasons and expected them to run.  Sure,
>> this might not work for other reasons outside of our control, but that's
>> no reason to be gratuitously incompatible by breaking the ABI,
>> particularly for what seem to be annoyances of our own creation with known
>> workarounds.
>
>Thanks, that's the first actual real example mentioned so far. And
>it's an interesting one: taking a $random Debian package and using it
>on a completely different, non-Debian system. Is that a supported use
>case? If so, does that mean that I can go ahead and raise a Severity:
>serious bug on any package that doesn't work in such a way?

Russ has described copying *binaries* out of packages and running them
elsewhere. I've done that too, from time to time. This is one of the
things made possible by the ABI contract being followed.

You are the one proposing to break that contract, thereby
*guaranteeing* this will fail on systems where otherwise it could
work. I think the onus is on *you* to justify why this is a valid and
useful thing to do. Your apparent lack of care for agreed standards
here is horrifying.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"We're the technical experts.  We were hired so that management could
 ignore our recommendations and tell us how to do our jobs."  -- Mike Andrews

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Steve McIntyre]

On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote:
>On Sun, 14 May 2023 at 22:37, Josh Triplett  wrote:
>
>> The x86-64 ABI is set. Feel free to make the case to the next
>> architecture designer that their new ABI should have the dynamic linker
>> in `/usr/lib`. That would *not* have the same downsides, as long as
>> everyone agrees on a path.
>
>In practice it is not, though. There are other distributions that
>change PT_INTERP for their own purposes, they've already been listed
>in this thread. And I am still not hearing any concrete, factual use
>case that would be impaired by such a change. I'm beginning to
>seriously think there aren't any? Is that really the case?

The ABI has been agreed and set down in documentation that *just
about* everybody has been following since its inception. This includes
the most basic set of definitions of what an x86-64 program must look
like, including the interpreter path. If this path is changed, then
*at the most basic level* we'd be making programs that are not valid
by the ABI we've agreed to. This is an *external interface contract*,
not something we should ever consider changing without significant
cross- and inter-project discussion.

Pointing at gentoo or nixos as examples of projects that have decided
to break compatibility doesn't cut it, I'm afraid. They're well known
for changing fundamental things around Linux and (basically) not
caring about interoperability. That attitude is *not* Debian's.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"The problem with defending the purity of the English language is that
 English is about as pure as a cribhouse whore. We don't just borrow words; on
 occasion, English has pursued other languages down alleyways to beat them
 unconscious and rifle their pockets for new vocabulary."  -- James D. Nicoll

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Mon, 15 May 2023 at 01:14, Russ Allbery  wrote:
>
> Luca Boccassi  writes:
>
> > Why would "software compiled on Debian" fail to work in other
> > environments? Well, there are many reasons actually, people invented
> > containers/flatpaks/snaps exactly for that reason. But nothing do with
> > anything discussed here though, as far as I can tell?
>
> My understanding is that this specific thread is about a mention that we
> may want to change PT_INTERP to /usr/lib64/ld-linux-x86-64.so.2 or some
> similar path.
>
> If PT_INTERP points to a file that doesn't exist, the program is obviously
> not going to run.  The Linux x86_64 ABI says it must point to
> /lib64/ld-linux-x86-64.so.2.  If we build binaries that use some other
> value, then we are not building ABI-compliant binaries and they may not
> run on other systems.  This is the whole point of an ABI.

This is not about locally compiled software or such, only packages
(and maybe even just a subset of them).

> An obvious specific example of such a system would be one that didn't
> merge /usr and thus only had /lib64/ld-linux-x86-64.so.2 and not any other
> path, but that's just one obvious example.  There may be others; the whole
> point of an ABI is that you do not change things like this, not even if
> you can't personally imagine why your change wouldn't be harmful.  There's
> a whole process for changing an ABI that involves everyone else agreeing
> as well, and unless one goes through that process, the ABI is what it is.
> Debian not building ABI-compliant binaries would be highly surprising.

That's self-evidently not true, as there are other distributions where
that already happens, it's been already mentioned. Besides, we are not
talking about sacred religious texts - the point is making things
work. If they do, is it _really_ non-compliant/incompatible?

> Incidentally, that remains true even if we only do that in distribution
> packages.  I certainly have copied binaries from a Debian package to other
> Linux systems before for various reasons and expected them to run.  Sure,
> this might not work for other reasons outside of our control, but that's
> no reason to be gratuitously incompatible by breaking the ABI,
> particularly for what seem to be annoyances of our own creation with known
> workarounds.

Thanks, that's the first actual real example mentioned so far. And
it's an interesting one: taking a $random Debian package and using it
on a completely different, non-Debian system. Is that a supported use
case? If so, does that mean that I can go ahead and raise a Severity:
serious bug on any package that doesn't work in such a way?

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Mon, 15 May 2023 at 01:07, Peter Pentchev  wrote:
>
> On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote:
> > On Sun, 14 May 2023 at 22:37, Josh Triplett  wrote:
> > >
> > > On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote:
> > > > The loader is still available via the old path, so external/third
> > > > party/local/other software works unchanged. This should negatively
> > > > only affect our 1st party packages, when running on a non-merged
> > > > distro.
> > > > And are _all_ our packages really 100% compatible with other distros
> > > > at all? Are they even supposed to be?
> > >
> > > People build things on Debian that are not Debian packages. People
> > > compile binaries on Debian, and expect them to work on any system that
> > > has sufficiently new libraries.
> > >
> > > This is *not* about Debian packages failing to work on other
> > > distributions; this is about *software compiled on Debian* faliing to
> > > work in other environments.
> >
> > Why would "software compiled on Debian" fail to work in other
> > environments? Well, there are many reasons actually, people invented
> > containers/flatpaks/snaps exactly for that reason. But nothing do with
> > anything discussed here though, as far as I can tell?
>
> If an ELF executable, compiled on Debian, records its interpreter as
> /usr/lib/ld-linux.so.2, what happens when one tries to run it on
> a non-usr-merged system? Even one with a recent enough glibc version?

This is not about locally built ELF executables, no difference in those.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Russ Allbery]

Luca Boccassi  writes:

> Why would "software compiled on Debian" fail to work in other
> environments? Well, there are many reasons actually, people invented
> containers/flatpaks/snaps exactly for that reason. But nothing do with
> anything discussed here though, as far as I can tell?

My understanding is that this specific thread is about a mention that we
may want to change PT_INTERP to /usr/lib64/ld-linux-x86-64.so.2 or some
similar path.

If PT_INTERP points to a file that doesn't exist, the program is obviously
not going to run.  The Linux x86_64 ABI says it must point to
/lib64/ld-linux-x86-64.so.2.  If we build binaries that use some other
value, then we are not building ABI-compliant binaries and they may not
run on other systems.  This is the whole point of an ABI.

An obvious specific example of such a system would be one that didn't
merge /usr and thus only had /lib64/ld-linux-x86-64.so.2 and not any other
path, but that's just one obvious example.  There may be others; the whole
point of an ABI is that you do not change things like this, not even if
you can't personally imagine why your change wouldn't be harmful.  There's
a whole process for changing an ABI that involves everyone else agreeing
as well, and unless one goes through that process, the ABI is what it is.
Debian not building ABI-compliant binaries would be highly surprising.

Incidentally, that remains true even if we only do that in distribution
packages.  I certainly have copied binaries from a Debian package to other
Linux systems before for various reasons and expected them to run.  Sure,
this might not work for other reasons outside of our control, but that's
no reason to be gratuitously incompatible by breaking the ABI,
particularly for what seem to be annoyances of our own creation with known
workarounds.

-- 
Russ Allbery (r...@debian.org)  

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Peter Pentchev]

On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote:
> On Sun, 14 May 2023 at 22:37, Josh Triplett  wrote:
> >
> > On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote:
> > > The loader is still available via the old path, so external/third
> > > party/local/other software works unchanged. This should negatively
> > > only affect our 1st party packages, when running on a non-merged
> > > distro.
> > > And are _all_ our packages really 100% compatible with other distros
> > > at all? Are they even supposed to be?
> >
> > People build things on Debian that are not Debian packages. People
> > compile binaries on Debian, and expect them to work on any system that
> > has sufficiently new libraries.
> >
> > This is *not* about Debian packages failing to work on other
> > distributions; this is about *software compiled on Debian* faliing to
> > work in other environments.
> 
> Why would "software compiled on Debian" fail to work in other
> environments? Well, there are many reasons actually, people invented
> containers/flatpaks/snaps exactly for that reason. But nothing do with
> anything discussed here though, as far as I can tell?

If an ELF executable, compiled on Debian, records its interpreter as
/usr/lib/ld-linux.so.2, what happens when one tries to run it on
a non-usr-merged system? Even one with a recent enough glibc version?

G'luck,
Peter
 
-- 
Peter Pentchev  r...@ringlet.net r...@debian.org p...@storpool.com
PGP key:http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13


signature.asc
Description: PGP signature

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Sun, 14 May 2023 at 22:37, Josh Triplett  wrote:
>
> On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote:
> > The loader is still available via the old path, so external/third
> > party/local/other software works unchanged. This should negatively
> > only affect our 1st party packages, when running on a non-merged
> > distro.
> > And are _all_ our packages really 100% compatible with other distros
> > at all? Are they even supposed to be?
>
> People build things on Debian that are not Debian packages. People
> compile binaries on Debian, and expect them to work on any system that
> has sufficiently new libraries.
>
> This is *not* about Debian packages failing to work on other
> distributions; this is about *software compiled on Debian* faliing to
> work in other environments.

Why would "software compiled on Debian" fail to work in other
environments? Well, there are many reasons actually, people invented
containers/flatpaks/snaps exactly for that reason. But nothing do with
anything discussed here though, as far as I can tell?

> If you build a dynamically linked binary that only depends on glibc, you
> can expect it to be reasonably portable, to any system that uses glibc
> and has a sufficiently new version.
>
> Debian stable is, in fact, one of the common environments people use to
> compile binaries for distribution.

"sufficiently new version" is doing a lot of work there. We have
shlibs dependencies for a reason. In fact, the most common environment
used to distribute binaries is the EOL Ubuntu 16.04, slowly switching
to the soon-to-be-EOL 18.04. glibc is not forward-compatible, new
symbols are added all the time and are used all the time, and you
don't jump on a brand new distribution to do that kind of work, that
would be self-defeating.

> > So, what I am asking is, what actual, real difference does it make if,
> > by default (and with an override available for example), packages
> > built on Debian for Debian record the ld path to point to its (actual)
> > location on Debian, via say a compiler spec file that is injected in a
> > deb build?
>
> Making binaries built *on* Debian different than binaries built *for*
> Debian would introduce a needless additional source of complexity,
> compared to just compiling code the same way in both cases.

That's not how it works today already. There are several significant
differences between just running "gcc sources.c" and building a
package via debhelper on a buildd, they are not the same thing at all,
and they haven't been since forever, there are dozens of
compiler/linker options that the Debian package build environment
sets. Or will you now also ask the distribution to rollback multiarch,
hardening, SOURCE_DATE_EPOCH, -ffile-prefix-map and all the other
reproducibility options, and so on? These and many more are all
"needless additional sources of complexity, compared to just compiling
code the same way" too. Because guess what, there are people who
couldn't possibly care less about
multiarch/security/reproducibility/etc, and there will also be a
subset of users who considers a subset of those compiler options
"needless". So are you going to push to have all of that reverted? And
also are you going to propose a Policy change that forbids adding any
new compiler/linker option to the package build process?

> To frame this in different terms: consider that one of the major goals
> of systemd has been to harmonize across distributions and eliminate
> needless variations that don't serve much actual purpose (e.g.
> variations in config file paths for the same config file). Consider how
> much effort systemd went to work with distributions, understand and deal
> with the *important* variations, and try to convince them to abandon the
> *unimportant* variations. Now imagine if someone came along and said
> "let's patch systemd to put unit files in /purple/; it'll work with
> everything in our distribution".

Pretty sure the Nix folks are already doing pretty much that. And if
it works for their case, all the power to them.

> Or, imagine if someone said "let's inject an argument to gzip, only for
> building the .gz files sihpped in our packages of course, to modify the
> gzip header and remove a few of the extraneous additional fields; it'll
> be fine, because we've patched our gzip to parse it"

Not really related, archives are _intended_ to be opened anywhere for
any reason. Do you have any actual related use case that would no
longer work? Because that would be the easiest and most convincing
counter-factual that could be provided.

> The x86-64 ABI is set. Feel free to make the case to the next
> architecture designer that their new ABI should have the dynamic linker
> in `/usr/lib`. That would *not* have the same downsides, as long as
> everyone agrees on a path.

In practice it is not, though. There are other distributions that
change PT_INTERP for their own purposes, they've already been listed
in this thread. And I am still not 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Josh Triplett]

On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote:
> The loader is still available via the old path, so external/third
> party/local/other software works unchanged. This should negatively
> only affect our 1st party packages, when running on a non-merged
> distro.
> And are _all_ our packages really 100% compatible with other distros
> at all? Are they even supposed to be?

People build things on Debian that are not Debian packages. People
compile binaries on Debian, and expect them to work on any system that
has sufficiently new libraries.

This is *not* about Debian packages failing to work on other
distributions; this is about *software compiled on Debian* faliing to
work in other environments.

If you build a dynamically linked binary that only depends on glibc, you
can expect it to be reasonably portable, to any system that uses glibc
and has a sufficiently new version.

Debian stable is, in fact, one of the common environments people use to
compile binaries for distribution.

> So, what I am asking is, what actual, real difference does it make if,
> by default (and with an override available for example), packages
> built on Debian for Debian record the ld path to point to its (actual)
> location on Debian, via say a compiler spec file that is injected in a
> deb build?

Making binaries built *on* Debian different than binaries built *for*
Debian would introduce a needless additional source of complexity,
compared to just compiling code the same way in both cases.

To frame this in different terms: consider that one of the major goals
of systemd has been to harmonize across distributions and eliminate
needless variations that don't serve much actual purpose (e.g.
variations in config file paths for the same config file). Consider how
much effort systemd went to work with distributions, understand and deal
with the *important* variations, and try to convince them to abandon the
*unimportant* variations. Now imagine if someone came along and said
"let's patch systemd to put unit files in /purple/; it'll work with
everything in our distribution".

Or, imagine if someone said "let's inject an argument to gzip, only for
building the .gz files sihpped in our packages of course, to modify the
gzip header and remove a few of the extraneous additional fields; it'll
be fine, because we've patched our gzip to parse it"

The x86-64 ABI is set. Feel free to make the case to the next
architecture designer that their new ABI should have the dynamic linker
in `/usr/lib`. That would *not* have the same downsides, as long as
everyone agrees on a path.

- Josh Triplett

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Fri, 12 May 2023 at 15:30, Steve McIntyre  wrote:
>
> On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote:
> >On Fri, 12 May 2023 at 12:08, Steve McIntyre  wrote:
> >>
> >> On Fri, May 12, 2023 at 11:40:05AM +0100, Steve McIntyre wrote:
> >> >On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote:
> >> >>On Fri, 12 May 2023 at 09:40, Steve McIntyre  wrote:
> >> >>>
> >> >>> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
> >> >>> >
> >> >>> >The core issue as I see it is as follows:
> >> >>> >
> >> >>> >- Debian has decided to support only merged-/usr, including possibly
> >> >>> >  moving /bin/sh to /usr/bin/sh or using 
> >> >>> > /usr/lib*/ld-linux-x86-64.so.2
> >> >>> >  as the interpreter in binaries.
> >> >>>
> >> >>> WTF? *Nobody* has been talking about breaking ABI like this, that I've
> >> >>> seen. The interpreter must *not* be changed willy-nilly.
> >> >>
> >> >>Nothing's happening 'willy-nilly'. We are discussing a bunch of
> >> >>seemingly crazy options, as in, "what would _actually_ explode if we
> >> >>do this or do that?", on this very d-devel thread. I posted a longer
> >> >>version here some days ago:
> >> >>
> >> >>https://lists.debian.org/debian-gcc/2023/05/msg00030.html
> >> >
> >> >Oh holy fuck.
> >> >
> >> >You're talking about changing ABI by doing this. That *is* utterly
> >> >crazy. No.
> >>
> >> People have asked me to expand on this further...
> >>
> >> I've been involved in defining ABI before, specifically for
> >> armhf. It's not a quick and easy process. It needs buy-in from all
> >> sides to make things work, and people *really* value the
> >> interoperability that it enables.
> >>
> >> The interpreter path is one of the most important parts of the ABI
> >> spec, the bit that makes binaries compatible between all the various
> >> stakeholders: compiler/tools people, distros, software vendors,
> >> etc. Lots of the rest of the details downstream of this can be
> >> changed, and people do this all the time - compare multilib to
> >> multi-arch for example. That all works fine *so long as* the runtime
> >> linker can be located and started OK.
> >
> >The loader is still available via the old path, so external/third
> >party/local/other software works unchanged. This should negatively
> >only affect our 1st party packages, when running on a non-merged
> >distro.
>
> So why the hell do you want to break this in the first place? Does a
> symlink in the "wrong" place offend you for some reason? For that you
> want to change a core assumption in *every single binary* in Debian?
> Believe me, I've been here in the past when we made changes in armhf
> to accommodate earlier mistakes. That was just for one
> architecture. What possible benefit do you see in this change?

As it was mentioned on the list, because it makes bootstrapping
self-contained, that's a real and concrete benefit that some
developers like Helmut care greatly about, and that's why we are
talking about it. To me, it sounds very attractive to have a
self-contained and canonicalized distro-wide configuration. If the
canonical location where certain files are stored in /usr/bin or
/usr/lib, it seems sensible to me to configure Debian software to look
for it where we actually put it, while maintaining compatibility for
external/local software so that it keeps working. And it is also
unclear so far what would outright break - the externally defined ABI
in terms of where the loader can be accessed at, would still be
respected. Hence why questions are being asked. Nobody's being forced
to do anything, this is just a discussion.

> >And are _all_ our packages really 100% compatible with other distros
> >at all? Are they even supposed to be?
> >
> >For example, if I download efibootmgr from Bookworm on an Ubuntu Focal
> >machine, when I try to run it, it fails:
> >
> >root@focal:/tmp# wget
> >http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb
> >--2023-05-12 12:46:17--
> >http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb
> >Resolving ftp.de.debian.org (ftp.de.debian.org)... 141.76.2.4
> >Connecting to ftp.de.debian.org (ftp.de.debian.org)|141.76.2.4|:80... 
> >connected.
> >HTTP request sent, awaiting response... 200 OK
> >Length: 27572 (27K) [application/vnd.debian.binary-package]
> >Saving to: 'efibootmgr_17-2_amd64.deb'
> >
> >efibootmgr_17-2_amd64.deb
> >100%[===>]  26.93K
> >--.-KB/sin 0.04s
> >
> >2023-05-12 12:46:17 (740 KB/s) - 'efibootmgr_17-2_amd64.deb' saved 
> >[27572/27572]
> >
> >root@focal:/tmp# dpkg -x efibootmgr_17-2_amd64.deb ebm
> >root@focal:/tmp# ./ebm/bin/efibootmgr
> >./ebm/bin/efibootmgr: /lib/x86_64-linux-gnu/libc.so.6: version
> >`GLIBC_2.34' not found (required by ./ebm/bin/efibootmgr)
> >
> >Should I file a severity: serious bug against efibootmgr because it is
> >not interoperable?
>
> You're wilfully missing the point, and you know it.

I'm trying to determine 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Steve McIntyre]

On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote:
>On Fri, 12 May 2023 at 12:08, Steve McIntyre  wrote:
>>
>> On Fri, May 12, 2023 at 11:40:05AM +0100, Steve McIntyre wrote:
>> >On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote:
>> >>On Fri, 12 May 2023 at 09:40, Steve McIntyre  wrote:
>> >>>
>> >>> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
>> >>> >
>> >>> >The core issue as I see it is as follows:
>> >>> >
>> >>> >- Debian has decided to support only merged-/usr, including possibly
>> >>> >  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
>> >>> >  as the interpreter in binaries.
>> >>>
>> >>> WTF? *Nobody* has been talking about breaking ABI like this, that I've
>> >>> seen. The interpreter must *not* be changed willy-nilly.
>> >>
>> >>Nothing's happening 'willy-nilly'. We are discussing a bunch of
>> >>seemingly crazy options, as in, "what would _actually_ explode if we
>> >>do this or do that?", on this very d-devel thread. I posted a longer
>> >>version here some days ago:
>> >>
>> >>https://lists.debian.org/debian-gcc/2023/05/msg00030.html
>> >
>> >Oh holy fuck.
>> >
>> >You're talking about changing ABI by doing this. That *is* utterly
>> >crazy. No.
>>
>> People have asked me to expand on this further...
>>
>> I've been involved in defining ABI before, specifically for
>> armhf. It's not a quick and easy process. It needs buy-in from all
>> sides to make things work, and people *really* value the
>> interoperability that it enables.
>>
>> The interpreter path is one of the most important parts of the ABI
>> spec, the bit that makes binaries compatible between all the various
>> stakeholders: compiler/tools people, distros, software vendors,
>> etc. Lots of the rest of the details downstream of this can be
>> changed, and people do this all the time - compare multilib to
>> multi-arch for example. That all works fine *so long as* the runtime
>> linker can be located and started OK.
>
>The loader is still available via the old path, so external/third
>party/local/other software works unchanged. This should negatively
>only affect our 1st party packages, when running on a non-merged
>distro.

So why the hell do you want to break this in the first place? Does a
symlink in the "wrong" place offend you for some reason? For that you
want to change a core assumption in *every single binary* in Debian?
Believe me, I've been here in the past when we made changes in armhf
to accommodate earlier mistakes. That was just for one
architecture. What possible benefit do you see in this change?

>And are _all_ our packages really 100% compatible with other distros
>at all? Are they even supposed to be?
>
>For example, if I download efibootmgr from Bookworm on an Ubuntu Focal
>machine, when I try to run it, it fails:
>
>root@focal:/tmp# wget
>http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb
>--2023-05-12 12:46:17--
>http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb
>Resolving ftp.de.debian.org (ftp.de.debian.org)... 141.76.2.4
>Connecting to ftp.de.debian.org (ftp.de.debian.org)|141.76.2.4|:80... 
>connected.
>HTTP request sent, awaiting response... 200 OK
>Length: 27572 (27K) [application/vnd.debian.binary-package]
>Saving to: 'efibootmgr_17-2_amd64.deb'
>
>efibootmgr_17-2_amd64.deb
>100%[===>]  26.93K
>--.-KB/sin 0.04s
>
>2023-05-12 12:46:17 (740 KB/s) - 'efibootmgr_17-2_amd64.deb' saved 
>[27572/27572]
>
>root@focal:/tmp# dpkg -x efibootmgr_17-2_amd64.deb ebm
>root@focal:/tmp# ./ebm/bin/efibootmgr
>./ebm/bin/efibootmgr: /lib/x86_64-linux-gnu/libc.so.6: version
>`GLIBC_2.34' not found (required by ./ebm/bin/efibootmgr)
>
>Should I file a severity: serious bug against efibootmgr because it is
>not interoperable?

You're wilfully missing the point, and you know it.

>The answer is obviously not, because it would be absurd to expect a
>binary compiled against libraries from one distro to "just work" on an
>entirely different distro. Glibc itself is not forward compatible, and
>is allowed to add new symbols, that are not present in older versions,
>and packages are allowed to depend on them. Aren't those also ABI
>breakages? What about all the libraries that bump soname? What about
>binaries that rely on newer kernel interfaces, or IPC interfaces?
>
>So, what I am asking is, what actual, real difference does it make if,
>by default (and with an override available for example), packages
>built on Debian for Debian record the ld path to point to its (actual)
>location on Debian, via say a compiler spec file that is injected in a
>deb build?
>There very likely is some real difference and impact, and I am
>genuinely and honestly asking what it could be. If nothing else, it's
>an interesting topic, even if likely nothing comes out of it.

I have better things to do than argue about this. I refuse to engage
with this right now. You're talking about 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Fri, 12 May 2023 at 13:21, Simon Richter  wrote:
>
> Hi,
>
> On 5/12/23 02:51, Luca Boccassi wrote:
>
> > Or alternatively, we can establish that a documentation/post-facto
> > approach is enough for derivatives, and then that's valid for all
> > changes and transitions.
>
> For the context of this bug, the notice *is* the after-the-fact
> documentation of an interface change, i.e. the bare minimum.
>
> It would have been better to get input from derived distributions about
> this interface change before it happened, but given that the interface
> change was not caused by a change in dpkg code, but by the introduction
> of the usrmerge package, there was not much of a remedy available then.

Not really? This notice is new, and suggests something that will make
the downstream irreparably incompatible with Debian, which I thought
was a big no-no.
If negatively affecting downstreams is a problem, this needs to be
reverted. If it's enough to say somewhere else "actually, that is bad
advice, if you follow it you are on your own", then that's fine too by
me, but then it's fine for other changes as well.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Simon Richter]

Hi,

On 5/12/23 02:51, Luca Boccassi wrote:


Or alternatively, we can establish that a documentation/post-facto
approach is enough for derivatives, and then that's valid for all
changes and transitions.


For the context of this bug, the notice *is* the after-the-fact 
documentation of an interface change, i.e. the bare minimum.


It would have been better to get input from derived distributions about 
this interface change before it happened, but given that the interface 
change was not caused by a change in dpkg code, but by the introduction 
of the usrmerge package, there was not much of a remedy available then.


   Simon

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Fri, 12 May 2023 at 12:08, Steve McIntyre  wrote:
>
> On Fri, May 12, 2023 at 11:40:05AM +0100, Steve McIntyre wrote:
> >On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote:
> >>On Fri, 12 May 2023 at 09:40, Steve McIntyre  wrote:
> >>>
> >>> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
> >>> >
> >>> >The core issue as I see it is as follows:
> >>> >
> >>> >- Debian has decided to support only merged-/usr, including possibly
> >>> >  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
> >>> >  as the interpreter in binaries.
> >>>
> >>> WTF? *Nobody* has been talking about breaking ABI like this, that I've
> >>> seen. The interpreter must *not* be changed willy-nilly.
> >>
> >>Nothing's happening 'willy-nilly'. We are discussing a bunch of
> >>seemingly crazy options, as in, "what would _actually_ explode if we
> >>do this or do that?", on this very d-devel thread. I posted a longer
> >>version here some days ago:
> >>
> >>https://lists.debian.org/debian-gcc/2023/05/msg00030.html
> >
> >Oh holy fuck.
> >
> >You're talking about changing ABI by doing this. That *is* utterly
> >crazy. No.
>
> People have asked me to expand on this further...
>
> I've been involved in defining ABI before, specifically for
> armhf. It's not a quick and easy process. It needs buy-in from all
> sides to make things work, and people *really* value the
> interoperability that it enables.
>
> The interpreter path is one of the most important parts of the ABI
> spec, the bit that makes binaries compatible between all the various
> stakeholders: compiler/tools people, distros, software vendors,
> etc. Lots of the rest of the details downstream of this can be
> changed, and people do this all the time - compare multilib to
> multi-arch for example. That all works fine *so long as* the runtime
> linker can be located and started OK.

The loader is still available via the old path, so external/third
party/local/other software works unchanged. This should negatively
only affect our 1st party packages, when running on a non-merged
distro.
And are _all_ our packages really 100% compatible with other distros
at all? Are they even supposed to be?

For example, if I download efibootmgr from Bookworm on an Ubuntu Focal
machine, when I try to run it, it fails:

root@focal:/tmp# wget
http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb
--2023-05-12 12:46:17--
http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb
Resolving ftp.de.debian.org (ftp.de.debian.org)... 141.76.2.4
Connecting to ftp.de.debian.org (ftp.de.debian.org)|141.76.2.4|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 27572 (27K) [application/vnd.debian.binary-package]
Saving to: 'efibootmgr_17-2_amd64.deb'

efibootmgr_17-2_amd64.deb
100%[===>]  26.93K
--.-KB/sin 0.04s

2023-05-12 12:46:17 (740 KB/s) - 'efibootmgr_17-2_amd64.deb' saved [27572/27572]

root@focal:/tmp# dpkg -x efibootmgr_17-2_amd64.deb ebm
root@focal:/tmp# ./ebm/bin/efibootmgr
./ebm/bin/efibootmgr: /lib/x86_64-linux-gnu/libc.so.6: version
`GLIBC_2.34' not found (required by ./ebm/bin/efibootmgr)

Should I file a severity: serious bug against efibootmgr because it is
not interoperable?

The answer is obviously not, because it would be absurd to expect a
binary compiled against libraries from one distro to "just work" on an
entirely different distro. Glibc itself is not forward compatible, and
is allowed to add new symbols, that are not present in older versions,
and packages are allowed to depend on them. Aren't those also ABI
breakages? What about all the libraries that bump soname? What about
binaries that rely on newer kernel interfaces, or IPC interfaces?

So, what I am asking is, what actual, real difference does it make if,
by default (and with an override available for example), packages
built on Debian for Debian record the ld path to point to its (actual)
location on Debian, via say a compiler spec file that is injected in a
deb build?
There very likely is some real difference and impact, and I am
genuinely and honestly asking what it could be. If nothing else, it's
an interesting topic, even if likely nothing comes out of it.

> Changing the interpreter path would mean moving to a Debian-specific
> ABI, breaking that compatibility. Hand-waving that away with (and I
> quote):
>
>   "The vast majority of distros today ship the loader in /usr/lib as
>   /lib is just a symlink, so it would be interoperable."
>
> is appalling arrogance. No. You do *not* get to break ABI with that
> argument. The point of the ABI spec is that *everybody* follows
> it. You don't change it just because you think it'll make your life a
> little easier when bootstrapping a system.

AFAIK there are at least 3 distros where the default interpreter path
is changed to follow distro-specific customizations: Gentoo, Nix,
Guix. So evidently, some people *do* get to "break 

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Steve McIntyre]

On Fri, May 12, 2023 at 11:40:05AM +0100, Steve McIntyre wrote:
>On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote:
>>On Fri, 12 May 2023 at 09:40, Steve McIntyre  wrote:
>>>
>>> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
>>> >
>>> >The core issue as I see it is as follows:
>>> >
>>> >- Debian has decided to support only merged-/usr, including possibly
>>> >  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
>>> >  as the interpreter in binaries.
>>>
>>> WTF? *Nobody* has been talking about breaking ABI like this, that I've
>>> seen. The interpreter must *not* be changed willy-nilly.
>>
>>Nothing's happening 'willy-nilly'. We are discussing a bunch of
>>seemingly crazy options, as in, "what would _actually_ explode if we
>>do this or do that?", on this very d-devel thread. I posted a longer
>>version here some days ago:
>>
>>https://lists.debian.org/debian-gcc/2023/05/msg00030.html
>
>Oh holy fuck.
>
>You're talking about changing ABI by doing this. That *is* utterly
>crazy. No.

People have asked me to expand on this further...

I've been involved in defining ABI before, specifically for
armhf. It's not a quick and easy process. It needs buy-in from all
sides to make things work, and people *really* value the
interoperability that it enables.

The interpreter path is one of the most important parts of the ABI
spec, the bit that makes binaries compatible between all the various
stakeholders: compiler/tools people, distros, software vendors,
etc. Lots of the rest of the details downstream of this can be
changed, and people do this all the time - compare multilib to
multi-arch for example. That all works fine *so long as* the runtime
linker can be located and started OK.

Changing the interpreter path would mean moving to a Debian-specific
ABI, breaking that compatibility. Hand-waving that away with (and I
quote):

  "The vast majority of distros today ship the loader in /usr/lib as
  /lib is just a symlink, so it would be interoperable."

is appalling arrogance. No. You do *not* get to break ABI with that
argument. The point of the ABI spec is that *everybody* follows
it. You don't change it just because you think it'll make your life a
little easier when bootstrapping a system.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
Welcome my son, welcome to the machine.

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Fri, 12 May 2023 at 11:40, Steve McIntyre  wrote:
>
> On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote:
> >On Fri, 12 May 2023 at 09:40, Steve McIntyre  wrote:
> >>
> >> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
> >> >
> >> >The core issue as I see it is as follows:
> >> >
> >> >- Debian has decided to support only merged-/usr, including possibly
> >> >  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
> >> >  as the interpreter in binaries.
> >>
> >> WTF? *Nobody* has been talking about breaking ABI like this, that I've
> >> seen. The interpreter must *not* be changed willy-nilly.
> >
> >Nothing's happening 'willy-nilly'. We are discussing a bunch of
> >seemingly crazy options, as in, "what would _actually_ explode if we
> >do this or do that?", on this very d-devel thread. I posted a longer
> >version here some days ago:
> >
> >https://lists.debian.org/debian-gcc/2023/05/msg00030.html
>
> Oh holy fuck.
>
> You're talking about changing ABI by doing this. That *is* utterly
> crazy. No.

It's a thought experiment on a mailing list. If we can't even have
those anymore, something went very wrong somewhere.

You seem to be aware of things that wouldn't work anymore (I think?).
If you have a couple of minutes to spare, may I please ask you to
reply to that thread with such examples? I am genuinely interested in
understanding and talking about it. Thank you.

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Steve McIntyre]

On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote:
>On Fri, 12 May 2023 at 09:40, Steve McIntyre  wrote:
>>
>> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
>> >
>> >The core issue as I see it is as follows:
>> >
>> >- Debian has decided to support only merged-/usr, including possibly
>> >  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
>> >  as the interpreter in binaries.
>>
>> WTF? *Nobody* has been talking about breaking ABI like this, that I've
>> seen. The interpreter must *not* be changed willy-nilly.
>
>Nothing's happening 'willy-nilly'. We are discussing a bunch of
>seemingly crazy options, as in, "what would _actually_ explode if we
>do this or do that?", on this very d-devel thread. I posted a longer
>version here some days ago:
>
>https://lists.debian.org/debian-gcc/2023/05/msg00030.html

Oh holy fuck.

You're talking about changing ABI by doing this. That *is* utterly
crazy. No.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"...In the UNIX world, people tend to interpret `non-technical user'
 as meaning someone who's only ever written one device driver." -- Daniel Pead

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Fri, 12 May 2023 at 09:40, Steve McIntyre  wrote:
>
> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
> >
> >The core issue as I see it is as follows:
> >
> >- Debian has decided to support only merged-/usr, including possibly
> >  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
> >  as the interpreter in binaries.
>
> WTF? *Nobody* has been talking about breaking ABI like this, that I've
> seen. The interpreter must *not* be changed willy-nilly.

Nothing's happening 'willy-nilly'. We are discussing a bunch of
seemingly crazy options, as in, "what would _actually_ explode if we
do this or do that?", on this very d-devel thread. I posted a longer
version here some days ago:

https://lists.debian.org/debian-gcc/2023/05/msg00030.html

Kind regards,
Luca Boccassi

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Steve McIntyre]

On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote:
>
>The core issue as I see it is as follows:
>
>- Debian has decided to support only merged-/usr, including possibly
>  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
>  as the interpreter in binaries.

WTF? *Nobody* has been talking about breaking ABI like this, that I've
seen. The interpreter must *not* be changed willy-nilly.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"I've only once written 'SQL is my bitch' in a comment. But that code 
 is in use on a military site..." -- Simon Booth

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Ansgar]

On Wed, 2023-05-10 at 19:01 -0700, Sean Whitton wrote:
> On Wed 10 May 2023 at 11:47PM +02, Ansgar wrote:
> > Cool, then let's ask tech-ctte.
> > 
> > Dear ctte, please consider overruling the dpkg maintainer to
> > include
> > the patch from #994388[1].
> > 
> > Thanks,
> > Ansgar
> > 
> >   [1]: https://bugs.debian.org/994388#397
> 
> This would require a new, maintainer-overruling vote.
> Our existing decisions do not apply, so far as I can tell.

Yes, I agree.

> I have written a separate message to the bug and to debian-dpkg with
> a proposal to avoid having to have such a vote.

That seems to be about an implementation detail on how to apply the
patch. I don't think that is the core of the issue?

The core issue as I see it is as follows:

- Debian has decided to support only merged-/usr, including possibly
  moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2
  as the interpreter in binaries.

- This change breaks on non-merged-/usr systems, including derivatives
  that do not revert *all* relevant changes. (Do you know one that
  does this or plans to do so?)

- dpkg recommends derivative users to move to non-merged-/usr.

I think this contradiction is not good and the core conflict. For me a
distribution should have some coherence. It is not just a distribution
of unrelated parts (like linux, libc, dpkg, dash, ...), but also
integrates them to work together.

And this also means not one package telling users to do X which breaks
other packages. Or (if other packages would do similar things as dpkg)
one package asking users to do X and the other asking users to do the
opposite of X. Just imagine dpkg asking users to move to split-/usr and
then another package starting to warn users to move back to merged-
/usr. Would that be a good state? I think not which is why this bug
exists.

Do you think this summary of the issue is right?

Is there some consensus about how this issue should be solved or do we
need a longer discussion to explore the solution space?

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Luca Boccassi]

On Thu, 11 May 2023 21:16:34 +0900 Simon Richter 
wrote:
> Hi,
> 
> On 5/11/23 10:59, Sean Whitton wrote:
> 
> >> Dear ctte, please consider overruling the dpkg maintainer to
include
> >> the patch from #994388[1].
> 
> > Currently dpkg contains code to emit the merged-/usr warning,
that's
> > dead code on Debian, but which becomes active when packages from
the
> > Debian archive are copied unmodified into derivatives.
> 
> The way I see it (but I'm not a dpkg maintainer), the current 
> implementation is correct, as dpkg does not support aliased
directories, 
> but Debian has decided to use it in such an environment nonetheless.
The 
> tech-ctte decision not to roll back usrmerge accepts responsibility
for 
> this decision, so silencing the warning on Debian is correct, but no
one 
> has accepted that responsibility for derived distributions.
> 
> Any derived distribution can easily go on record and request
inclusion 
> in the list of distributions where this warning is suppressed, by
typing 
> the phrase "Yes, I understand that this is a bad idea." into an email
> client.

The crux of the issue is that we are hearing how negatively affecting
derivatives in any way, even purely theoretically, is a big no-no, in
this very same thread and topic. Reaching out and asking for
directions/help/whatever is not enough in that context. So it follows
that it cannot be enough in this context either, and it must be fixed
instead.

Or alternatively, we can establish that a documentation/post-facto
approach is enough for derivatives, and then that's valid for all
changes and transitions.

Either of these are valid approaches.

What I cannot find acceptable is that some changes get a free pass, and
some get roadblocks after roadblocks thrown at them.

-- 
Kind regards,
Luca Boccassi


signature.asc
Description: This is a digitally signed message part

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Simon Richter]

Hi,

On 5/11/23 10:59, Sean Whitton wrote:


Dear ctte, please consider overruling the dpkg maintainer to include
the patch from #994388[1].



Currently dpkg contains code to emit the merged-/usr warning, that's
dead code on Debian, but which becomes active when packages from the
Debian archive are copied unmodified into derivatives.


The way I see it (but I'm not a dpkg maintainer), the current 
implementation is correct, as dpkg does not support aliased directories, 
but Debian has decided to use it in such an environment nonetheless. The 
tech-ctte decision not to roll back usrmerge accepts responsibility for 
this decision, so silencing the warning on Debian is correct, but no one 
has accepted that responsibility for derived distributions.


Any derived distribution can easily go on record and request inclusion 
in the list of distributions where this warning is suppressed, by typing 
the phrase "Yes, I understand that this is a bad idea." into an email 
client.



dpkg has an upstream existence that's independent of Debian, and it's
perfectly legitimate for that version of dpkg to emit the warning.  The
problem here might be caused by how the Debian archive is implicitly
being used to distribute upstream dpkg.


I'm skeptical about splitting development and packaging, though, 
especially in this context, because we'd need to clarify how far we'd 
want upstream dpkg and Debian dpkg to deviate.


Basically, non-native dpkg has two scenarios: either it is maintained by 
the same people as now, which causes them extra work for no clear 
benefit, or we find maintainers willing to deal with complex bug reports 
that upstream is fully entitled to reject because Debian brought this 
onto themselves by deciding that one upstream project's "unsupported 
configuration" needs to be avoided by moving to another upstream 
project's "unsupported configuration."


Those same people who would volunteer to maintain such a package could 
spend their energy finding a solution that can be merged into "upstream" 
dpkg, that would be more productive.


   Simon

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Emilio Pozuelo Monfort]

Hi Sean,

On 11/05/2023 03:59, Sean Whitton wrote:

Hello,

On Wed 10 May 2023 at 11:47PM +02, Ansgar wrote:


Dear ctte, please consider overruling the dpkg maintainer to include
the patch from #994388[1].


Currently dpkg contains code to emit the merged-/usr warning, that's
dead code on Debian, but which becomes active when packages from the
Debian archive are copied unmodified into derivatives.

The heart of the issue is how dpkg is a native package.  What we're
talking about is not the Debian system, but the Debian archive as it
exists independently of the Debian system.

dpkg has an upstream existence that's independent of Debian, and it's
perfectly legitimate for that version of dpkg to emit the warning.  The
problem here might be caused by how the Debian archive is implicitly
being used to distribute upstream dpkg.

This is not in itself a problem -- we distribute a lot of stuff in
source packages that does not form part of the Debian system.  But in
this case, this distribution that's occurring might conflict with how
Debian is seeking to provide a product not just to end users, but also
to those building derivatives.

One simple solution is for dpkg to become a non-native package, carrying
Debian-specific patches to do things like remove the warning code.


I think you're conflating two independent things.

If you override the dpkg maintainer to remove that warning that occurs on 
derivatives, then anyone could NMU dpkg and the maintainer wouldn't introduce it 
back, effectively removing the warning from "dpkg upstream".


OTOH if the dpkg maintainer switches to non-native packages, anyone could NMU it 
adding the change as a patch, however the maintainer will just NACK the NMU 
before or after it happens.


So I don't see a problem with dpkg being native, just like e.g. apt is, and that 
won't magically solve the issue at hand.


Cheers,
Emilio

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Sean Whitton]

Hello,

On Wed 10 May 2023 at 11:47PM +02, Ansgar wrote:

> Dear ctte, please consider overruling the dpkg maintainer to include
> the patch from #994388[1].

Currently dpkg contains code to emit the merged-/usr warning, that's
dead code on Debian, but which becomes active when packages from the
Debian archive are copied unmodified into derivatives.

The heart of the issue is how dpkg is a native package.  What we're
talking about is not the Debian system, but the Debian archive as it
exists independently of the Debian system.

dpkg has an upstream existence that's independent of Debian, and it's
perfectly legitimate for that version of dpkg to emit the warning.  The
problem here might be caused by how the Debian archive is implicitly
being used to distribute upstream dpkg.

This is not in itself a problem -- we distribute a lot of stuff in
source packages that does not form part of the Debian system.  But in
this case, this distribution that's occurring might conflict with how
Debian is seeking to provide a product not just to end users, but also
to those building derivatives.

One simple solution is for dpkg to become a non-native package, carrying
Debian-specific patches to do things like remove the warning code.

Guillem & the other dpkg maintainers: would that change be something you
are willing to consider?  It might forestall this and similar issues
from becoming contentious.  Having dpkg as a non-native package would
reflect the reality, that we can celebrate, that dpkg has an upstream
existence independent of Debian.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1035904: dpkg currently warning about merged-usr systems (revisited)

[Sean Whitton]

Hello,

On Wed 10 May 2023 at 11:47PM +02, Ansgar wrote:

> Cool, then let's ask tech-ctte.
>
> Dear ctte, please consider overruling the dpkg maintainer to include
> the patch from #994388[1].
>
> Thanks,
> Ansgar
>
>   [1]: https://bugs.debian.org/994388#397

This would require a new, maintainer-overruling vote.
Our existing decisions do not apply, so far as I can tell.

I have written a separate message to the bug and to debian-dpkg with a
proposal to avoid having to have such a vote.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg)

[Ansgar]

On Wed, 2023-05-10 at 23:47 +0200, Ansgar wrote:
> Cool, then let's ask tech-ctte.
> 
> Dear ctte, please consider overruling the dpkg maintainer to include
> the patch from #994388[1].
> 
> Thanks,
> Ansgar
> 
>   [1]: https://bugs.debian.org/994388#397

For derivatives based on Debian stable it might be worth having this
included in the next stable release; this would need a fairly quick
decision on this issue.

Ansgar

Bug#1035904: dpkg currently warning about merged-usr systems (revisited) (was: Re: DEP 17: Improve support for directory aliasing in dpkg)

[Ansgar]

Package: tech-ctte
X-Debbugs-Cc: Russ Allbery , Sean Whitton 
, Helmut Grohne , Luca Boccassi 
, debian-d...@lists.debian.org, debian-de...@lists.debian.org

On Wed, 2023-05-10 at 14:36 -0700, Russ Allbery wrote:
> Ansgar  writes:
> > Debian going out of its way to tell derivative users to switch back from
> > merged-/usr to split-/usr is the *opposite* of trying to make things as
> > smooth for them as possible.
> 
> Yes, I agree with that part and I think I objected to that at the time.
> Nonetheless, one bad decision doesn't mean that it is Debian policy that
> we don't care about derivatives or their users.  I think we made a mistake
> there which is not in alignment with our ideals or our goals.  We should
> try to reverse that mistake, not double down on it.

Cool, then let's ask tech-ctte.

Dear ctte, please consider overruling the dpkg maintainer to include
the patch from #994388[1].

Thanks,
Ansgar

  [1]: https://bugs.debian.org/994388#397