Re: Bug#1041207: debootstrap: bad NMU produces buildds not supported by dpkg _and_ CTTE

[Bill Allombert]

On Sun, Jul 16, 2023 at 12:42:11PM +0100, Luca Boccassi wrote:
> If there is somebody who's ignoring things, that would be yourself,
> given this change has been not only been explicitly requested, but even
> provided _BY_ the CTTE, as you would have easily found out if you
> actually went and checked:
> 
> https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/93
> http://meetbot.debian.net/debian-ctte/2023/debian-ctte.2023-07-11-17.58.log.html
> 
> Debian Community Team, Adam is once again sabotaging the CTTE's work
> with hostile NMUs, could you please intervene? Thank you.

This is premature.

> In the meanwhile, I'll immediately revert the sabotage.

If this package is so important, why is it maintained by NMUs ?
Why cannot the maintainers do a proper upload ?

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Re: Bug#1041207: debootstrap: bad NMU produces buildds not supported by dpkg _and_ CTTE

[Vagrant Cascadian]

On 2023-07-16, Simon McVittie wrote:
> On Sat, 15 Jul 2023 at 18:27:24 +0200, Adam Borowski wrote:
>> But, what matters here is the CTTE ruling in #1035831 -- for the time being,
>> packages must not move files between locations affected by the aliasing.
>
> If that happens in reality, then yes, that's bad, and reverting the change
> is a mitigation. What packages have this behaviour?
>
> We are going to need to bring back this change relatively early in the
> trixie cycle in any case, for the reasons given in the commit message.
> I have not yet analyzed whether we need this change before we can lift
> the moratorium on file moves, but I suspect we might.
>
>> Packages built in an usrmerged chroot place such files under /usr while
>> built without usrmerge into whatever place they were installed to -- which
>> is a direct breach of the ruling.
>
> Do you have examples of packages that differ in this way when built in
> a merged- or unmerged-/usr environment? I think we should treat this
> as a RC-for-trixie bug in those packages (and in fact I would have been
> tempted to call it RC for bookworm as well, again for the reasons given
> by the TC, even though during the trixie cycle it was mitigated by using
> unmerged-/usr fro buildds).
>
> During most of the bookworm cycle, https://reproducible-builds.org/ has
> been doing "build1" in unmerged-/usr and "build2" in merged-/usr, with
> differences tracked in
> 
> (that list is not necessarily complete, there can also be unidentified
> differences in
> ).

For what it is worth, there were various points during the bookworm
cycle where this was not being tested on reproducible builds
infrastructure, as the mechanisms to disable it changed several
times...

We used to just be able to build a non-usrmerge tarball, and then
install usrmerge in the second build, but I think usr-is-merged or some
similar package is installed out of the box now, and the inverse
operation is non-trivial.

... which lead to some of the identified issues being systematically
removed for packages that were otherwise reproducible (you could still
look through git history to find more, but some many may be actually
fixed).

There are differing opinions on weather reproducible builds test
infrastrure should test usrmerge variations at all, given the direction
of Debian, though any alternate test infrastructure would essentially
have to implement a reproducible builds style test to check for
differences...

After upgrading the infrastructure to bookworm, testing usrmerge
variations broke again, and so is currently disabled... though I have
configured the paths_vary_due_to_usrmerge issue so that old known issues
are not automatically removed anymore.


live well,
  vagrant


signature.asc
Description: PGP signature

Re: Bug#1041207: debootstrap: bad NMU produces buildds not supported by dpkg _and_ CTTE

[Simon McVittie]

On Sat, 15 Jul 2023 at 18:27:24 +0200, Adam Borowski wrote:
> bluca's NMU on 2023-07-15 makes debootstrap produce chroots using the
> aliased-dirs scheme.

My intention in the MR that was included in the NMU[1] was to default to
merged-/usr chroots in all cases for trixie and up, but continue to
produce unmerged-/usr chroots for bookworm. When I tested the MR, that's
the result I got[2]. Have you observed something different?

There was consensus among the TC[3] that this change was appropriate for
trixie/sid at this time.

To the best of my knowledge, the official Debian buildds run on stable
(possibly oldstable in some cases) and use debootstrap from there, so any
changes here would not affect official buildds until/unless they are
backported into (old)stable point releases via -proposed-updates.

smcv

[1] https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/93
[2] 
https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/93#note_410656
[3] 
http://meetbot.debian.net/debian-ctte/2023/debian-ctte.2023-07-11-17.58.log.html
18:39 to 18:50 in the timestamps' time zone

Re: Bug#1041207: debootstrap: bad NMU produces buildds not supported by dpkg _and_ CTTE

[Simon McVittie]

On Sat, 15 Jul 2023 at 18:27:24 +0200, Adam Borowski wrote:
> But, what matters here is the CTTE ruling in #1035831 -- for the time being,
> packages must not move files between locations affected by the aliasing.

If that happens in reality, then yes, that's bad, and reverting the change
is a mitigation. What packages have this behaviour?

We are going to need to bring back this change relatively early in the
trixie cycle in any case, for the reasons given in the commit message.
I have not yet analyzed whether we need this change before we can lift
the moratorium on file moves, but I suspect we might.

> Packages built in an usrmerged chroot place such files under /usr while
> built without usrmerge into whatever place they were installed to -- which
> is a direct breach of the ruling.

Do you have examples of packages that differ in this way when built in
a merged- or unmerged-/usr environment? I think we should treat this
as a RC-for-trixie bug in those packages (and in fact I would have been
tempted to call it RC for bookworm as well, again for the reasons given
by the TC, even though during the trixie cycle it was mitigated by using
unmerged-/usr fro buildds).

During most of the bookworm cycle, https://reproducible-builds.org/ has
been doing "build1" in unmerged-/usr and "build2" in merged-/usr, with
differences tracked in

(that list is not necessarily complete, there can also be unidentified
differences in
).

I did some bug-reporting for this during the bookworm cycle and I don't
remember reporting any bugs where a package changed whether it installed
/bin/foo or /usr/bin/foo (or sbin or lib* equivalents) according to
whether it was built in a merged-/usr chroot or not: the bugs I was
reporting were generally about file contents, not the file list.

Most of the remaining merged- vs. unmerged-/usr differences
have been packages that install an Autotools-generated
Makefile into /usr/share/doc/PACKAGE/examples, like for example
 which says "EGREP = /bin/grep -E"
when built in an unmerged-/usr chroot, but /usr/bin/grep in a merged-/usr
chroot. That's annoying but not really a serious issue, because it's only
an example file, and in my opinion should not be RC (but I would consider
it potentially RC if the path was in a functionally-necessary file).

If a package installs in the typical multiple-binary-package way:

- make
- make install DESTDIR=$(pwd)/debian/tmp
- dh_install splits up debian/tmp using debian/*.install
- dh_builddeb

then a mismatch between the expected /bin/foo and a new /usr/bin/foo
will generally cause FTBFS rather than a misbuilt package, because
dh_install will fail to find /bin/foo. So I think the highest risks
for this failure mode are going to be packages that use the simplified
single-binary-package code path:

- make
- make install DESTDIR=$(pwd)/debian/my-package
- dh_builddeb

and packages that do not use debhelper in the conventional way.

smcv