Bug#1072218: apt dist-upgrade fails to install essential packages but exits successfully

[Johannes Schauer Marin Rodrigues]

Package: apt
Version: 2.6.1
Severity: wishlist

Control: affects -1 + mmdebstrap

Hi David,

do you remember our conversation in #debian-dpkg from 2018?

 } elsif ($options->{variant} eq 'apt') {
 # if we just want to install Essential:yes packages, apt and their
 # dependencies then we can make use of libapt treating apt as
 # implicitly essential. An upgrade with the (currently) empty status
 # file will trigger an installation of the essential packages plus 
apt. 
 #
 # 2018-09-02, #debian-dpkg on OFTC, times in UTC+2
 # 23:39 < josch> I'll just put it in my script and if it starts
 #breaking some time I just say it's apt's fault. :P
 # 23:42 < DonKult> that is how it usually works, so yes, do that :P 
(<- 
 #  and please add that line next to it so you can
 #  remind me in 5+ years that I said that after I 
wrote 
 #  in the bugreport: "Are you crazy?!? Nobody in his
 #  right mind would even suggest depending on it!")
 @dl_debs = run_apt_download_progress({
 APT_ARGV => ['dist-upgrade'],
 dryrun   => $options->{dryrun},
 },
 );

Well, it is now even six years later. It is time!

The problem first showed up yesterday in the sbuild autopkgtest for riscv:

https://ci.debian.net/data/autopkgtest/testing/riscv64/s/sbuild/47087233/log.gz

Log is also attached for when it gets deleted from ci.d.n but the important
bits are these:

  MarkInstall util-linux:riscv64 < none -> 2.40.1-2 @un uN Ib > FU=0
  Installing libblkid1:riscv64 as PreDepends of util-linux:riscv64
MarkInstall libblkid1:riscv64 < none -> 2.40.1-2 @un uN > FU=0
  Installing libcap-ng0:riscv64 as PreDepends of util-linux:riscv64
MarkInstall libcap-ng0:riscv64 < none -> 0.8.5-1 @un uN > FU=0
  Installing libcrypt1:riscv64 as PreDepends of util-linux:riscv64
MarkInstall libcrypt1:riscv64 < none -> 1:4.4.36-4 @un uN > FU=0
  Installing libmount1:riscv64 as PreDepends of util-linux:riscv64
MarkInstall libmount1:riscv64 < none -> 2.40.1-2 @un uN > FU=0
  Installing libpam0g:riscv64 as PreDepends of util-linux:riscv64
MarkInstall libpam0g:riscv64 < none -> 1.5.3-7 @un uN Ib > FU=0
Installing libaudit1:riscv64 as Depends of libpam0g:riscv64
  libaudit1:riscv64 Depends on libaudit-common:riscv64 < none | 
1:3.1.2-2 @un uH > (>= 1:3.1.2-2.1) can't be satisfied!
  libpam0g:riscv64 Depends on libaudit1:riscv64 < none @un H > (>= 
1:2.2.1) can't be satisfied! (dep)
util-linux:riscv64 PreDepends on libpam0g:riscv64 < none @un H > (>= 
0.99.7.1) can't be satisfied! (dep)
[...]
Setting up usrmerge (39) ...
Can't exec "mountpoint": No such file or directory at 
/usr/lib/usrmerge/convert-usrmerge line 431.
Failed to execute mountpoint -q /lib/modules/: No such file or directory
E: usrmerge failed.
dpkg: error processing package usrmerge (--install):
 installed usrmerge package post-installation script subprocess returned 
error exit status 1

Because apt decides not to install util-linux, the usrmerge postinst fails.
This all boils down to mmdebstrap using the funny "apt dist-upgrade" trick to
let apt choose all the Essential:yes packages plus itself for the mmdebstrap
"apt" chroot variant.

How hard would it be to let apt dist-upgrade fail if it is unable to install an
Essential:yes package?

This bug is certainly wishlist and might as well be marked as wontfix without
much value being lost. But I thought that given your comment back in 2018
including your spot-on prediction (only off by one year!) I should absolutely
share this with you. :)

Thanks!

cheers, josch


47087233.gz
Description: application/gzip

Bug#966621: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Peter Pentchev (2024-05-30 10:33:08)
> > thank you for having discussed this change on d-devel and for adding
> > documentation to NEWS and release notes to announce this change. I also
> > think it is sensible to roll this only out on new installations and to keep
> > the behaviour on existing setups. Thank you for implementing that as well.
> > 
> > That being said, maybe some Perl wizard knows how to do a flock on a 
> > directory
> > in Perl?
> 
> I wouldn't call myself a Perl wizard by a long stretch, but I can give it a 
> try :)
> 
> >  I tried this:
> > 
> > use Fcntl qw(LOCK_EX);
> > opendir(my $handle, ".") or die "opendir: $!";
> [snip]
> 
> Here lies your problem. The flock(2) syscall works on file descriptors,
> the things returned by open(2), not on the dirent structures returned by
> opendir(3). So you need something like this:
> 
> use v5.10;  # I really should switch to at least 5.16 if not 5.24
> use strict;
> use warnings;
> 
> use Fcntl qw(O_RDONLY O_DIRECTORY LOCK_EX);
> 
> my $dirname = "/tmp/to-be-locked";
> sysopen(my $handle, "/tmp/to-be-locked", O_RDONLY | O_DIRECTORY) or
> die "Could not open $dirname: $!\n";
> flock($handle, LOCK_EX) or
> die "Could not lock $dirname: $!\n";
> 
> say "locked, it seems";
> sleep(3600);'
> 
> I only put the sleep() part so I could check using lsof that
> the directory was indeed locked. And yeah, the v5.10 part is a leftover
> from the days (...until a month or two ago...) when I still had to
> support stock CentOS 7 systems; I really should train my fingers to
> put 5.24 there.
> 
> Hope that helps!

it absolutely does! Thank you! I was misled by `perldoc -f flock` which states
that it works on filehandles. I'll add your name to the git commit message
unless you object.  :)

I also found another issue with this change in systemd. After the upload to
unstable, 76 out of 264 mmdebstrap tests on jenkins.debian.net started to fail:

https://jenkins.debian.net/job/mmdebstrap-jenkins-worker/692/consoleText

The problem is, that debootstrap wants to mknod which will not work on a tmpfs
mounted with nodev:

+ debootstrap --no-merged-usr --variant=buildd oldstable /tmp/tmp.nWmx8YeAh3 
http://127.0.0.1/debian
/usr/sbin/debootstrap: 1840: cannot create /tmp/tmp.nWmx8YeAh3/test-dev-null: 
Permission denied
E: Cannot install into target '/tmp/tmp.nWmx8YeAh3' mounted with noexec or nodev

Maybe this affects more CI scripts and test setups which attempt to create a
temporary chroot with debootstrap in /tmp.

The fix which is documented in systemd NEWS makes everything work again:

--customize-hook='touch "$1/etc/systemd/system/tmp.mount"'

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1072205: prevent re-using package versions for NMUs

[Johannes Schauer Marin Rodrigues]

Package: ftp.debian.org
Severity: normal
X-Debbugs-Cc: reproducible-b...@lists.alioth.debian.org
User: reproducible-bui...@lists.alioth.debian.org
Usertags: infrastructure
User: ftp.debian@packages.debian.org
Usertags: dak

Hi,

the binary package bash 5.2.15-2+b3 was uploaded to the archive twice. Once to
bookworm and once to sid but with differing content. Here is a diff of their
d/changelog:

@@ -1,6 +1,6 @@
-bash (5.2.15-2+b3) bookworm; urgency=low, binary-only=yes
+bash (5.2.15-2+b3) sid; urgency=low, binary-only=yes
 
   * Binary-only non-maintainer upload for arm64; no source changes.
   * Rebuild for outdated Built-Using (glibc/2.36-9)
 
- -- arm Build Daemon (arm-ubc-03)   
Fri, 29 Mar 2024 13:22:36 +
+ -- arm Build Daemon (arm-ubc-02)   
Thu, 13 Jul 2023 09:12:53 +

This is not only confusing for apt (it will not be able to figure out which of
the two is installed because dpkg does not keep track of package hashes) but it
is also problematic for reproducible builds because the buildinfo file records
packages by their name/architecture/version tuple and relies on those to be
unique throughout the history of Debian.

This example with bash is especially problematic since bash is Essential:yes so
there will now be a large portion of buildinfo files where it is not possible
to figure out with which of the two differing bash packages the sources were
compiled.

snapshot.d.o also shows the issue and could probably be used to get an idea how
many packages are affected:

http://snapshot.debian.org/package/bash/5.2.15-2/#bash_5.2.15-2:2b:b3

Maybe this issue is blocked by #620356?

Thanks!

cheers, josch

Bug#966621: Make /tmp/ a tmpfs and cleanup /var/tmp/ on a timer by default [was: Re: systemd: tmpfiles.d not cleaning /var/tmp by default]

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Luca Boccassi (2024-05-28 01:54:08)
> Thanks for the useful input, the following has been done:
> 
> - existing installations pre-trixie will get an orphaned tmpfiles.d in
> /etc/ that keeps the existing behaviour unchanged (no cleanup of
> /var/tmp)
> - openssh and tmux have been fixed to provide a tmpfiles.d exception
> to retain their respective files
> - the /tmp/ description in debian-installer has been updated to note
> it is a tmpfs by default (via a commit in partman-basicfilesystems,
> will upload if nobody gets around to it before Trixie's freeze)
> - two paragraphs have been provided for the release notes ticket
> - the changes are also noted in NEWS, with instructions on how to
> override locally
> - as mentioned, the latest upload to unstable makes /tmp/ a tmpfs by
> default and for new installations 10+ days old files in /tmp/ and 30+ days
> old files in /var/tmp/ are cleaned up daily

thank you for having discussed this change on d-devel and for adding
documentation to NEWS and release notes to announce this change. I also think
it is sensible to roll this only out on new installations and to keep the
behaviour on existing setups. Thank you for implementing that as well.

That being said, maybe some Perl wizard knows how to do a flock on a directory
in Perl? I tried this:

use Fcntl qw(LOCK_EX);
opendir(my $handle, ".") or die "opendir: $!";
flock($handle, LOCK_EX) or die "flock: $!";

And got this:

flock() on unopened filehandle $handle at test.pl line 8.
(Are you trying to call flock() on dirhandle $handle?)
flock: Bad file descriptor at test.pl line 8.

Wrapping $handle in fileno() doesn't make a difference either. Perl sources
seem to indicate that directory handles are "evil_fh"?

https://sources.debian.org/src/perl/5.38.2-4/util.c/?hl=3783#L3783

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071031: ezurio-qcacld-2.0-dkms: several module build failures

[Johannes Schauer Marin Rodrigues]

Quoting Andreas Beckmann (2024-05-28 10:57:37)
> On 28/05/2024 10.16, Johannes Schauer Marin Rodrigues wrote:
> > But I wonder, the autopkgtest results now say (for example for arm64):
> > 
> > 657s I: Summary:
> > 657s I: PASS ezurio-qcacld-2.0/0.0~git20230623.2cd31b6 6.7.12-arm64
> > 657s I: SKIP ezurio-qcacld-2.0/0.0~git20230623.2cd31b6 6.7.12-cloud-arm64
> > 657s I: PASS ezurio-qcacld-2.0/0.0~git20230623.2cd31b6 6.7.12-rt-arm64
> > 
> > As a result, the whole test is marked as "skipped" and not "successful", so 
> > I
> 
> No. The neutral state comes from autopkgtest-pkg-dkms being 
> "superficial". As this only tests building the module but not its 
> functionality, this is only some kind of smoketest and thus has to be 
> marked "superficial". It is now possible to have "isolation-machine" 
> tests that could test module functionality (see e.g. dm-writeboost-dkms) 
> but not if this requires special hardware.

Okay, thank you!

> So there is probably no migration boost possible for most -dkms packages. But
> at least we should be able to catch build failures on new kernels earlier.
> (But that will need some work on britney as a new kernel upload does not yet
> trigger all -dkms packages.)

I don't know how the field "Testsuite: autopkgtest-pkg-dkms" gets implemented
in practice, but when just using debian/tests/control, one can add this to let
another package foo trigger the tests:

Features: test-name=hint-testsuite-triggers
Test-Command: false
Depends: foo
Restrictions: hint-testsuite-triggers

This is for example done for the package debvm:

https://sources.debian.org/src/debvm/0.3/debian/tests/control/#L15

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071031: ezurio-qcacld-2.0-dkms: several module build failures

[Johannes Schauer Marin Rodrigues]

Hi Andreas,

I noticed that you are one of the dkms uploaders, so maybe you can answer a
follow-up question? :)

Quoting Johannes Schauer Marin Rodrigues (2024-05-27 23:45:08)
> Quoting Andreas Beckmann (2024-05-27 18:52:58)
> > On 27/05/2024 18.32, Johannes Schauer Marin Rodrigues wrote:
> > > I'm inclined to just restrict the kernel to amd64, arm64 and riscv64 as 
> > > the
> > > relevant hardware is unlikely (or rather impossible) to be found on other
> > > architectures. Or do you think that this would be a bed "solution"?
> > 
> > No objections.
> > 
> > Either make the package Architecture:  instead of all or try
> > BUILD_EXCLUSIVE_ARCH="" in dkms.conf
> > (directive exists, but I've never used it and AFAIK no other Debian 
> > package does use it; maybe needs kernel architectures instead of Debian 
> > architectures)
> > or if you can anchor it on some CONFIG_* option
> > BUILD_EXCLUSIVE_CONFIG="CONFIG_FOO !CONFIG_BAR"
> > (it's an AND-ed list, no OR possible)
> > 
> > You may need to add debian/tests/autopkgtest-pkg-dkms.conf with
> > architecture = 
> > if the package is not arch:all as autodep8 may not (be able to) take the
> > architecture list of the binary packages into account.
> 
> the package is already using:
> 
> BUILD_EXCLUSIVE_CONFIG="CONFIG_WIRELESS"
> 
> which prevents building it for the cloud kernel, for example. I read a bit 
> into
> the dkms.in sources and found that BUILD_EXCLUSIVE_ARCH is a bash regex, so I
> set it up like tihs now:
> 
> BUILD_EXCLUSIVE_ARCH="^(aarch64|x86_64|riscv64)$"
> 
> The architectures seem to be what "uname -m" is printing.

this seems to have done the trick:

https://tracker.debian.org/pkg/ezurio-qcacld-2.0-dkms

But I wonder, the autopkgtest results now say (for example for arm64):

657s I: Summary:
657s I: PASS ezurio-qcacld-2.0/0.0~git20230623.2cd31b6 6.7.12-arm64
657s I: SKIP ezurio-qcacld-2.0/0.0~git20230623.2cd31b6 6.7.12-cloud-arm64
657s I: PASS ezurio-qcacld-2.0/0.0~git20230623.2cd31b6 6.7.12-rt-arm64

As a result, the whole test is marked as "skipped" and not "successful", so I
do not get the few days migration bonus. The module is not meant to be built
for the cloud kernel, so can I somehow tell the dkms autopkgtest that so that
it only tries to build it on the kernels where it makes sense and thus the
whole test becomes "green" and not "yellow"?

It's of course no big deal if there is no way. :)

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071031: ezurio-qcacld-2.0-dkms: several module build failures

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Andreas Beckmann (2024-05-27 18:52:58)
> On 27/05/2024 18.32, Johannes Schauer Marin Rodrigues wrote:
> > I'm inclined to just restrict the kernel to amd64, arm64 and riscv64 as the
> > relevant hardware is unlikely (or rather impossible) to be found on other
> > architectures. Or do you think that this would be a bed "solution"?
> 
> No objections.
> 
> Either make the package Architecture:  instead of all or try
> BUILD_EXCLUSIVE_ARCH="" in dkms.conf
> (directive exists, but I've never used it and AFAIK no other Debian 
> package does use it; maybe needs kernel architectures instead of Debian 
> architectures)
> or if you can anchor it on some CONFIG_* option
> BUILD_EXCLUSIVE_CONFIG="CONFIG_FOO !CONFIG_BAR"
> (it's an AND-ed list, no OR possible)
> 
> You may need to add debian/tests/autopkgtest-pkg-dkms.conf with
> architecture = 
> if the package is not arch:all as autodep8 may not (be able to) take the
> architecture list of the binary packages into account.

the package is already using:

BUILD_EXCLUSIVE_CONFIG="CONFIG_WIRELESS"

which prevents building it for the cloud kernel, for example. I read a bit into
the dkms.in sources and found that BUILD_EXCLUSIVE_ARCH is a bash regex, so I
set it up like tihs now:

BUILD_EXCLUSIVE_ARCH="^(aarch64|x86_64|riscv64)$"

The architectures seem to be what "uname -m" is printing.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1072049: please provide a link to the artifacts on the page displaying the log

[Johannes Schauer Marin Rodrigues]

Source: debci
Severity: wishlist

Hi,

when I stored artifacts for my autopkgtest job, I somehow expected the
link to download the artifacts to be shown next to the log:

https://ci.debian.net/packages/r/reform-setup-wizard/testing/s390x/47038806/

Because this is the page one is shown when clicking on failed results
via tracker.d.o

Thanks!

cheers, josch

Bug#1071031: ezurio-qcacld-2.0-dkms: several module build failures

[Johannes Schauer Marin Rodrigues]

Hi Andreas,

On Mon, 13 May 2024 10:32:23 +0200 Andreas Beckmann  wrote:
> The module fails to build for several architectures and kernel versions:
> 
> Linux 6.6.*:
> https://ci.debian.net/packages/e/ezurio-qcacld-2.0-dkms/testing/amd64/45828215/#L3674
> 243s 
> /var/lib/dkms/ezurio-qcacld-2.0/0.0~git20230623.2cd31b6/build/CORE/HDD/src/wlan_hdd_cfg80211.c:
>  In function ‘__wlan_hdd_cfg80211_change_beacon’:
> 243s 
> /var/lib/dkms/ezurio-qcacld-2.0/0.0~git20230623.2cd31b6/build/CORE/HDD/src/wlan_hdd_cfg80211.c:19956:48:
>  error: invalid use of undefined type ‘struct cfg80211_ap_update’
> 
> struct cfg80211_ap_update was introduced in Linux 6.7, so I'd recommend
> to add
> 
>   # struct cfg80211_ap_update was added in "wifi: cfg80211: split struct
>   # cfg80211_ap_settings" bb55441c57ccc5cc2eab44e1a97698b9d708871d 
> (v6.7-rc1)
>   BUILD_EXCLUSIVE_KERNEL_MIN="6.7"
> 
> to dkms.conf.

thank you, this seems sensible!

> 
> i386, Linux 6.7.*:
> ERROR: modpost: "__udivdi3" 
> [/var/lib/dkms/ezurio-qcacld-2.0/0.0~git20230623.2cd31b6/build/wlan.ko] 
> undefined!
> 
> This probably affects more 32-bit platforms.
> 
> IIRC __udivdi3 is a compiler generated function call for 
>   (long long) a / (long long) b
> for architectures that don't have native support for long long
> arithmetic. This symbol is part of libgcc_s, but that cannot be linked
> into the kernel for obvious reasons.
> I do not know the solution for this case, but this is something affecting
> more out-of-tree kernel modules.

I'm inclined to just restrict the kernel to amd64, arm64 and riscv64 as the
relevant hardware is unlikely (or rather impossible) to be found on other
architectures. Or do you think that this would be a bed "solution"?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071736: sbuild --chroot-mode=unshare fails when /etc/resolv.conf is a symlink /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Helmut Grohne (2024-05-24 14:12:38)
> while working with debusine.debian.net, I ran into a rather crazier kind
> of issue with the unshare backend. It seems like debusine.debian.net
> creates a chroot.tar where resolv.conf is a symbolic link:
> 
> | lrwxrwxrwx 0/0   0 2024-05-20 17:15 ./etc/resolv.conf -> 
> ../run/systemd/resolve/stub-resolv.conf
> 
> Notably, /run/systemd/resolve does not exist inside the tar nor does
> sbuild run systemd-resolved nor systemd-tmpfiles for creating this
> location. When building, the unshare backend tries to bind mount
> /etc/resolv.conf:
> 
> | --: 13: cannot create /tmp/tmp.sbuild.OQ0pOU6LQg/etc/resolv.conf: Directory 
> nonexistent
> https://debusine.debian.net/artifact/427489/hostname_3.23+nmu2_amd64-2024-05-24T10:06:30Z.build
> 
> This fails, because mount attempts to dereference the symbolic link and
> finds that an intermediate directory does not exist. As a result, this
> fails and network generally does not work resulting in all sorts of
> badness.

I'm not sure where you see bind-mounting /etc/resolv.conf being done in the
$network_setup code. If network is enabled, it reads:

[ -f /etc/resolv.conf ] && cat /etc/resolv.conf > 
"$rootdir/etc/resolv.conf" || echo "nameserver 127.0.0.53" > 
"$rootdir/etc/resolv.conf";

and when it's disabled:

ip link set lo up;> "$rootdir/etc/resolv.conf";

> Technically speaking, you can bind mount onto a symbolic link. You just
> cannot do so using the mount(2) API nor the mount(1) command. Unless you
> pass MOVE_MOUNT_T_SYMLINKS to move_mount(2), it will not dereference a
> symlink being pointed at. I'm not sure we want to go this extra mile
> though.
> 
> On the debusine side, I think we want to work around this issue in some
> way to avoid imposing a high version constraint in sbuild. I am
> reporting it here as it kinda is a bug (up to your judgement) and it
> helps having the diagnosis written down.

in unshare mode, we are always working with an ephemeral chroot. Would there be
any downside to sbuild just first running "rm -f $rootdir/etc/resolv.conf" and
then re-creating it as a real file in the $network_setup snippet of
_get_exec_argv() in lib/Sbuild/ChrootUnshare.pm?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071861: plakativ: prepare for next python3-fitz version

[Johannes Schauer Marin Rodrigues]

Hi,

On Sat, 25 May 2024 16:48:08 +0200 Bastian Germann  wrote:
> python3-fitz's exported module is going to be renamed from fitz to fitz_old
> with the next unstable revision (already in experimental). Please prepare for
> the rename.

I found this for context: https://github.com/pymupdf/PyMuPDF/discussions/2680

According to that document:

> The rebased implementation will behave identically to the classic
> implementation, and will not require any changes to user code.

or:

> You can try the rebased implementation with import fitz_new as fitz (no other
> changes to your code are needed).

or:

> Users of PyMuPDF will be able to carry on using PyMuPDF throughout the
> migration without making any changes to their code.

So I don't understand why I have to rename the import.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071462: installing/upgrading libc6 does not work in sbuild when systemd is installed as ischroot declines

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Aurelien Jarno (2024-05-20 11:49:32)
> > > > That's all legacy stuff and I really don't want to touch it anymore.
> > > > Going from the other side, maybe libc6.postinst could use something
> > > > more reliable than ischroot()? Is systemd-detect-virt able to figure
> > > > out the situation a bit better?
> > > Nope.
> > What's the output? With SYSTEMD_LOG_LEVEL=debug exported
> In sbuild using unshare chroot running in a VM:
> 
> | SYSTEMD_LOG_LEVEL=debug /usr/bin/systemd-detect-virt
> | Failed to read $container of PID 1, ignoring: Permission denied
> | Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
> | Found container virtualization none.
> | Virtualization QEMU found in DMI (/sys/class/dmi/id/sys_vendor)
> | UML virtualization not found in /proc/cpuinfo. 
> | Virtualization XEN not found, /proc/xen does not exist
> | Virtualization found, CPUID=KVMKVMKVM
> | Found VM virtualization kvm
> | kvm

would it help (and be correct) if PID 1 in sbuild unshare mode would have the
"container" environment variable set to something? And/or if sbuild unshare
mode would create the file /run/systemd/container with some content?

I'd need input from somebody who knows how containers (if this is one) are
supposed to work. :)

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071462: installing/upgrading libc6 does not work in sbuild when systemd is installed as ischroot declines

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Chris Hofstaedtler (2024-05-20 10:38:04)
> * Johannes Schauer Marin Rodrigues  [240520 07:35]:
> > [..] But maybe it [glibc's postinst] should be doing some
> > more involved checks about what PID 1 is? It could then make sure to only 
> > call
> > systemd telinit if systemd is pid 1. [..]
> 
> Well, that would probably suck. Putting the knowledge when to call
> telinit, and a specific telinit, into a ton of different things
> makes all those things hard to get right, hard to update, the usual.
> 
> I've checked the sysvinit and the systemd implementations now, and
> they are not that different. Both try to talk to their respective
> pid1 daemons first using their respective communication socket.
> 
> But then, if that doesn't work, they diverge:
> - sysvinit's telinit just gives up
> - systemd's telinit, *as an explicit fallback*, sends signals.
> 
> systemd's telinit (aka systemctl) helpfully exits if it detects
> being in a chroot, before doing any of that.
> 
> IWSTM systemd's telinit could, if called as telinit, not do the
> fallback to stick with sysvinit's behaviour?
> 
> As a bonus, sysvinit's telinit could also gain the chroot check, and glibc's
> postinst (and other places) can become simpler again.

via irc, jochen also pointed out: telinit could be the component which checks
what PID 1 actually is and only do its thing after it confirmed that it is
indeed an init system like systemd that is providing PID 1?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071462: installing/upgrading libc6 does not work in sbuild when systemd is installed as ischroot declines

[Johannes Schauer Marin Rodrigues]

Quoting Helmut Grohne (2024-05-20 07:17:54)
> Hi Chris,
> 
> On Mon, May 20, 2024 at 01:02:32AM +0200, Chris Hofstaedtler wrote:
> > "..., when using telinit from systemd-sysv"
> > 
> > It would seem like a reasonable assumption that systemd-sysv's
> > telinit uses systemd-specific stuff, like SIGTERM.
> 
> That also is an interesting angle to it. sbuild didn't ask for systemd-sysv
> to be installed.

this might shift the bug back to glibc postinst, no? Currently, it just checks
the exit of ischroot before calling telinit. But maybe it should be doing some
more involved checks about what PID 1 is? It could then make sure to only call
systemd telinit if systemd is pid 1. And sysvinit telinit if sysvinit is pid 1
and not call telinit at all in unknown cases. Is it too much to ask for glibc
postinst to know about the PID 1 providers?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071238: please consider cherry-picking set -e pipefail support from upstream

[Johannes Schauer Marin Rodrigues]

Package: dash
Version: 0.5.12-6
Severity: wishlist
Tags: patch

Hi,

please consider cherry-picking this patch from upstream git:

https://git.kernel.org/pub/scm/utils/dash/dash.git/commit/?id=6347b9fc52d742f36a0276cdea06cd9ad1f02c77

It even works fine when backported to bookworm dash :)

Before:

josch@reform:/tmp/dash-0.5.12$ dash
$ set -o pipefail
dash: 1: set: Illegal option -o pipefail

After:

josch@reform:/tmp/dash-0.5.12$ dash
$ set -o pipefail
$ echo $?
0

Thanks!

cheers, josch


-- System Information:
Debian Release: 12.5
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: arm64 (aarch64)
Foreign Architectures: armhf, i386, amd64

Kernel: Linux 6.5.0-0.deb12.4-reform2-arm64 (SMP w/6 CPU threads)
Kernel taint flags: TAINT_CRAP, TAINT_OOT_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages dash depends on:
ii  debianutils  5.7-0.5~deb12u1
ii  dpkg 1.21.22
ii  libc62.36-9+deb12u7

dash recommends no packages.

dash suggests no packages.

-- no debconf information

Bug#1071000: sbuild-qemu: runs lintian on .changes file with Distribution != changelog target and lintian complains

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-05-14 00:13:31)
> > Quoting Francesco Poli (wintermute) (2024-05-12 21:06:24)
> [...]
> > > Please note that my current setup with pbuilder does not have this issue:
> > > pdebuild generates a .changes file with 'Distribution: UNRELEASED'
> > > and with the latest changelog entry correctly quoted (with 'UNRELEASED'
> > > after the version number parenthesis), and hence lintian is happy...
> > 
> > Wait... how is Lintian happy with UNRELEASED as the changelog entry?
> 
> After digging Lintian code for a while (sorry, my Perl knowledge is just a
> smattering, and it is also a bit rusty at the time of writing!), I found the
> following [code]
> 
> # issue only when not mentioned in the Distribution field
> if ((any { $_ eq 'UNRELEASED' } @changesdists)
> && none { $_ eq 'UNRELEASED' } @distributions) {
> $self->hint('unreleased-changes');
> return;
> }
> 
> [code]: 
> <https://salsa.debian.org/lintian/lintian/-/blob/69b9209b02ab1a9e2d40d83931541ab6629f9226/lib/Lintian/Check/Fields/Distribution.pm#L135>
> 
> It seems to me that the 'unreleased-changes' tag is issued, if the
> distribution found in the Changes field is 'UNRELEASED' and the
> distribution found in the Distribution field is not 'UNRELEASED'.
> Or am I completely off-track?

yes, this is how it works. For details see bug #542747 which is also referenced
in the description of the unreleased-changes tag. More details also here:

https://salsa.debian.org/lintian/lintian/-/commit/25be3ba45884334e7e07c7643b5e4fd4ad5a21ee#note_182777

> On Sun, 12 May 2024 23:10:14 +0200 Johannes Schauer Marin Rodrigues wrote:
> [...]

> 
> [...]
> > The only difference between sbuild and pbuilder should be this:
> > 
> > >   W: $PKG_NAME: changelog-distribution-does-not-match-changes-file 
> > > unreleased != unstable [usr/share/doc/$PKG_NAME/changelog.gz:1]
> > 
> > The reason for that difference is, that sbuild-qemu calls sbuild with the -d
> > option and that overwrites the distribution field. Christian, why does
> > sbuild-qemu use the -d option?
> > 
> > But this lintian error should also be present with pbuilder:
> > 
> > >   E: $PKG_NAME changes: unreleased-changes
> > 
> > Can you clarify?
> 
> I checked by running lintian again on the .changes file generated by
> pdebuild:
> 
>   $ lintian ${PKG_NAME}_${VERSION}_${ARCH}.changes
>   
>   $ lintian -EviIL +pedantic ${PKG_NAME}_${VERSION}_${ARCH}.changes
> 
> These commands produce no output.
> 
> Assuming the above analysis of what the Perl code of Lintian actually
> does is correct, it seems the reason is that both Distribution and
> Changes have 'UNRELEASED' as distribution:
> 
>   $ grep UNRELEASED ${PKG_NAME}_${VERSION}_${ARCH}.changes
>   Distribution: UNRELEASED
>$PKG_NAME ($VERSION) UNRELEASED; urgency=medium
> 

What you observe is not specific to sbuild-qemu. You run into this when you use
the -d or --dist option with sbuild. As the man page describes, the option will
set the Distribution field of the produced .changes file to the specified
distribution name. This is not necessarily the same value as the top of
debian/changelog lists.

In general, you should avoid using -d or --dist with sbuild. But I see that
sbuild-qemu always adds this option to the sbuild call. I am unable to say why
it does that. Maybe Christian can clarify?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071078: Skip creation of start-stop-daemon compat symlink if /sbin does not exist

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Guillem Jover (2024-05-14 10:24:09)
> When I proposed suppressing the creation of the symlink I had in mind and had
> locally the first approach, because it's less noisy, and I assume users on
> those kind of systems will have logic somewhere to handle that case anyway.

sure, works for me! Thank you! :)

> But I've been thinking that this still seems wrong from a conceptual PoV, in
> that I think base-files should be unpacked as one of, if not the first
> package as part of the bootstrapping protocol, as it encodes the structure
> and metadata for the base filesystem. I think next would be base-passwd as
> that also defines the metadata for the filesystem.
> 
> For example the symlink to dir and dir to symlink avoidance behavior
> in dpkg would be one reason for this, another is that directory
> metadata does not get updated if the directory exists, so the first
> unpack wins.
> 
> I realize that we have continuously talked about wanting to avoid
> encoding this kind of ordering in external tools, and this information
> should be part of the packages themselves (which I still think is the
> right approach). But I think we still could extend the packaging
> and/or possibly the bootstrap protocol to cover this case.
> 
> One option would be to add dependencies within the bootstrap set on
> both of these packages, but that goes counter to the no-depends on
> essential packages, and would not be safe against other external
> packages that might conflict with metadata definitions in base-files,
> but it would work for the specific dpkg issue, and perhaps that would
> be the better immediate fix. Another could be to mark these two base-*
> packages somehow so that the bootstrap tools know in a generic way
> that they need to unpack them first. I guess a field could be used,
> something like the following comes to mind:
> 
>   base-files
> Bootstrap: fsys-tree
> 
>   base-passwd
> Bootstrap: fsys-meta
> 
> (or Bootstrap-Phase or Bootstrap-Order, or …).
> 
> Where I think the order should be:
> 
>   fsys-tree → fsys-meta → rest of essential + dependencies
> 
> This does not solve the other bootstrap problem we've had in the past
> with base-files requiring passwd information from base-passwd before
> that has been unpacked, but I guess that might partially go away once
> we can declare optional paths and template-based initialization
> through fsys metadata so that the base-files maintscript can disappear
> completely. Although an external dpkg will still have a similar problem
> as base-files currently does, and will need to either assume some name
> to id mappings or use the ones from outside the chroot, but meh.
> 
> (But perhaps the solution to that last bit is for base-files to declare
> the user/groups it requires as dpkg sysusers out of previously agreed
> definitions from base-passwd, then that would make it all self-contained.)

Please note that the issue I encountered was *without* base-files installed so
this issue only exists in completely unsupported situations. Thus I think that
the fact that you now added support for it is not an argument for needing
dependencies between essential:yes packages.

Personally I prefer a new field over hard-coding package names but I prefer
even more not needing either of those. You gave the example of base-files
needing information from base-passwd and while this is correct, this dependency
can be avoided by hardcoding the needed information from base-passwd into
base-files at build time. I think helmut has proposed that in the past.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1071078: Skip creation of start-stop-daemon compat symlink if /sbin does not exist

[Johannes Schauer Marin Rodrigues]

Package: dpkg
Version: 1.21.22
Severity: wishlist
Tags: patch

Hi,

dpkg postinst fails to create the start-stop-daemon compatibility
symlink if /usr does not exist:

ln: failed to create symbolic link '/sbin/start-stop-daemon': No such file 
or directory

Getting into this situation requires some creativity. Here is an example:

$ mmdebstrap --variant=custom 
--include=dpkg,dash,diffutils,coreutils,libc-bin,sed unstable /dev/null

Here are two patches that fix this problem by just not creating the
compatibility symlink in this situation:

--- a/debian/dpkg.postinst
+++ b/debian/dpkg.postinst
@@ -59,7 +59,8 @@ setup_aliases()
 
   # Add a backward compatibility symlink alias for s-s-d, which is now
   # installed in its canonical location.
-  if [ ! -f "$DPKG_ROOT/sbin/$prog" ]; then
+  # Skip creation of the compat symlink if /sbin is not an existing directory
+  if [ ! -f "$DPKG_ROOT/sbin/$prog" ] && [ -d "$DPKG_ROOT/sbin" ]; then
 ln -s "/usr/sbin/$prog" "$DPKG_ROOT/sbin/$prog"
   fi
 }

--- a/debian/dpkg.postinst
+++ b/debian/dpkg.postinst
@@ -59,8 +59,10 @@ setup_aliases()
 
   # Add a backward compatibility symlink alias for s-s-d, which is now
   # installed in its canonical location.
+  # Failure to create the compat symlink (for example if /usr does not
+  # exist) is non-fatal.
   if [ ! -f "$DPKG_ROOT/sbin/$prog" ]; then
-ln -s "/usr/sbin/$prog" "$DPKG_ROOT/sbin/$prog"
+ln -s "/usr/sbin/$prog" "$DPKG_ROOT/sbin/$prog" || true
   fi
 }
 

The second patch has the advantage, that the user will receive the error
message from ln about its inability to create the symlink together with the
reason for the failure.

I'm working around this issue in the mmdebstrap test suite by creating /sbin
manually in a hook. I'm filing this bug to not loose track of the situation
and to be able to add a bugnumber to the hook.

Thanks!

cheers, josch

Bug#1068775: sbuild-qemu: fails to work, if piuparts is requested, but not installed on the host

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-05-12 00:32:42)
> On Sat, 13 Apr 2024 14:50:05 +0200 Francesco Poli wrote:
> > On Fri, 12 Apr 2024 19:57:23 +0200 Johannes Schauer Marin Rodrigues wrote:
> > > Quoting Francesco Poli (wintermute) (2024-04-11 00:13:51)
> > > > Why does sbuild-qemu insist that piuparts be installed on the *host*
> > > > system?
> > > Because it needs to be installed on the host. In the same way as
> > > autopkgtest needs to be on the host. What can sbuild improve?
> > My point is that sbuild{,-qemu} should install piuparts inside the build
> > environment (chroot or VM guest system) and run it from within the build
> > environment, if this is possible.
> If this is possible for piuparts, could you please explain why sbuild-qemu
> does not do so?
> 
> Otherwise, if this is not possible, could please explain why?

it is possible if somebody writes the code for it.

If you want me to write the code for it, you'd first need to convince me that
it is a good idea to run piuparts inside the build environment as it is done
with lintian.

> > Does sbuild{,-qemu} do so for lintian?
> I checked: sbuild-qemu (temporarily) installs lintian inside the build
> environment and runs it from within the build environment.

Correct.

> I think this is the best strategy, especially for lintian, as I have
> previously explained.

Please refresh my memory: where did you explain why installing piuparts inside
the build environment would be the best strategy?

> That reinforces my opinion that the same strategy should probably be followed
> for piuparts, as well...

What is it that reinforces your opinion? If it is the fact that it is done for
Lintian, I do not see the connection. Why would it be a good idea to do this
for piuparts if it is done for lintian?

> Anyway, I am having issues in running piuparts, even after installing it on
> the host.
> 
> I tried to run
> 
>   $ sbuild-qemu --boot=efi --overlay-dir=/dev/shm
> 
> with the following configuration file:
> 
>   $ cat ~/.sbuildrc 
>   $source_only_changes = 1;
>   $autopkgtest_require_success = 1;
>   $lintian_require_success = 0;
>   $piuparts_require_success = 1;
>   $run_autopkgtest = 1;
>   $run_lintian = 1;
>   $run_piuparts = 1;
>   $build_dir = "$HOME/.cache/sbuild/build";
>   
>   # don't remove this, Perl needs it:
>   1;
> 
> It builds the package successfully, but, once it reaches the piuparts
> step, sudo asks for my (host system) regular user's password.
> My understanding is that it should not need to use sudo, since it should
> use the QEMU virtual machine image (where it has root access).
> Is that right? Or am I misunderstanding something?

This is not a functionality that piuparts has at the moment.

> The same thing happens at the autopkgtest step, where, again, sudo asks for
> my regular user's password. Once again, this should not be needed:
> autopkgtest should use the QEMU virtual machine as a testbed.  At least, when
> I manually run autopkgtest, I can use the QEMU virtual machine and no
> password is asked for.
> 
> I tried to modify the configuration file:
> 
>   $ cat ~/.sbuildrc
>   $source_only_changes = 1;
>   $run_lintian = 1;
>   $lintian_require_success = 0;
>   $run_piuparts = 1;
>   $piuparts_root_args = '';
>   $piuparts_require_success = 1;
>   $run_autopkgtest = 1;
>   $autopkgtest_root_args = '';
>   $autopkgtest_require_success = 1;
>   $build_dir = "$HOME/.cache/sbuild/build";
>   
>   # don't remove this, Perl needs it:
>   1;
> 
> It no longer asks for passwords, but fails to run piuparts:
> 
>   piuparts
>   
> 
>   You need to be root to use piuparts.
> 
>   E: Piuparts run failed.
> 
> And it also fails to run autopkgtest:
> 
>   autopkgtest
>   ---
>   
>   autopkgtest [19:56:04]: starting date and time: 2024-05-11 19:56:04+0200
>   [...]
>   autopkgtest [19:56:05]: test unit_test: preparing testbed
>   autopkgtest [19:56:05]: ERROR: "sh -ec
> type apt-ftparchive >/dev/null 2>&1 || DEBIAN_FRONTEND=noninteractive 
> apt-get install -y apt-utils 2>&1
> (cd /tmp/autopkgtest.jrHchc/binaries; apt-ftparchive packages . > 
> Packages; apt-ftparchive release . > Release)
> printf 'Package: *\nPin: origin ""\nPin-Priority: 1002\n' > 
> /etc/apt/preferences.d/90autopkgtest
> echo "deb [ trusted=yes ] file:///tmp/autopkgtest.jrHchc/binaries /" 
> >/etc/apt/sources.list.d/autopkgtest.list
> if [ "x`ls /var/lib/dpkg/updates`" != x ]; then
>   echo >&2 "/var/lib/dpkg/updates contains some files, aargh"; exit 1
> fi
> apt-get --q

Bug#1071000: sbuild-qemu: runs lintian on .changes file with Distribution != changelog target and lintian complains

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (wintermute) (2024-05-12 21:06:24)
> I am still trying to set up sbuild-qemu to build (and check/test) Debian
> packages.
> 
> After creating the virtual machine image:
> 
>   $ mkdir -p ~/.cache/sbuild/build
>   $ cd /dev/shm
>   $ TMPDIR=/dev/shm mmdebstrap-autopkgtest-build-qemu \
> --size=25G --boot=efi sid unstable-autopkgtest-amd64.img
>   $ mv -i unstable-autopkgtest-amd64.img ~/.cache/sbuild/
> 
> I prepared the following configuration file:
> 
>   $ cat ~/.sbuildrc 
>   $source_only_changes = 1;
>   $run_lintian = 1;
>   $lintian_require_success = 0;
>   $run_piuparts = 1;
>   $piuparts_root_args = '';
>   $piuparts_require_success = 1;
>   $run_autopkgtest = 1;
>   $autopkgtest_root_args = '';
>   $autopkgtest_require_success = 1;
>   $build_dir = "$HOME/.cache/sbuild/build";
>   
>   # don't remove this, Perl needs it:
>   1;
> 
> I can update the virtual machine:
> 
>   $ sbuild-qemu-update --boot=efi unstable-autopkgtest-amd64.img
> 
> and I can build a package from within the unpacked source tree:
> 
>   $ sbuild-qemu --boot=efi --overlay-dir=/dev/shm

cool!

> However, when I do so, I am not necessarily planning to upload the
> Debian package to the Debian archive: it could be still in development
> (not yet ready for an upload) and I just want to check that it can be
> built and perform the usual checks on it, namely lintian (first of all!),
> followed by piuparts and autopkgtest.
> If this is the case, my latest changelog entry will read 'UNRELEASED' in the
> distribution field, but, at the same time, I want to build the package
> with the 'unstable-autopkgtest-amd64.img' VM image (to build it for
> unstable).

Yes, doing that should be possible.

> What happens here? It turns out that sbuild-qemu modifies the Distribution
> field in the .changes file and sets it to 'unstable', despiting reading
> 'UNRELEASED' in the latest changelog entry.
> Hence lintian complains with an error:
> 
>   [...]
>   Running lintian...
>   E: $PKG_NAME changes: unreleased-changes
>   W: $PKG_NAME: changelog-distribution-does-not-match-changes-file unreleased 
> != unstable [usr/share/doc/$PKG_NAME/changelog.gz:1]
>   
>   E: Lintian run failed (runtime error)
>   [...]
> 
> How can I fix this issue?
> Shouldn't it work out of the box?

But it does. Your d/changelog has UNRELEASED as the distribution and Lintian
classifies that as an error.

> I found bug report [#934721], which, however, seems to advocate that
> this is the right behavior ?!? Or am I misinterpreting it?
> 
> [#934721]: 
> 
> How can I run lintian on a still-in-developed UNRELEASED package (with
> sbuild-qemu)?

You just did. And it reports a failure because in its current state, your
package is not ready to be uploaded.

> Please note that my current setup with pbuilder does not have this issue:
> pdebuild generates a .changes file with 'Distribution: UNRELEASED'
> and with the latest changelog entry correctly quoted (with 'UNRELEASED'
> after the version number parenthesis), and hence lintian is happy...

Wait... how is Lintian happy with UNRELEASED as the changelog entry?

> It has to work out of the box with sbuild-qemu, as well!
> 
> Please fix the issue and/or explain what's wrong.
> 
> Thanks for your time and patience.

The only difference between sbuild and pbuilder should be this:

>   W: $PKG_NAME: changelog-distribution-does-not-match-changes-file unreleased 
> != unstable [usr/share/doc/$PKG_NAME/changelog.gz:1]

The reason for that difference is, that sbuild-qemu calls sbuild with the -d
option and that overwrites the distribution field. Christian, why does
sbuild-qemu use the -d option?

But this lintian error should also be present with pbuilder:

>   E: $PKG_NAME changes: unreleased-changes

Can you clarify?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1070020: autopkgtest: (unrelated) packages not found

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Johannes Schauer Marin Rodrigues (2024-04-28 19:34:07)
> unfortunately, this problem is not transient but reproducible. The problem
> is, that the involved libraries which are 404 have alternative 64bit time_t
> versions in unstable:
> 
> libssl3 -> libssl3t64
> libdb5.3 -> libdb5.3t64
> libgdbm6 -> libgdbm6t64
> libgdbm-compat4 -> libgdbm-compat4t64
> 
> And apt chooses to satisfy the (virtual) dependencies on those differently,
> depending on the surrounding dependency satisfaction problem. Related,
> debootstrap is also broken by the virtual providers of libssl3: #1069787
> 
> Feel free to tell the release team to ignore these failures if necessary.
> They will go away once the 64bit time_t transition is finished.

do you agree that there is nothing for mmdebstrap to do here? If yes, could you
close this bug?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1070109: fakechroot: apt crashes inside fakechroot

[Johannes Schauer Marin Rodrigues]

Control: tag -1 + unreproducible

Hi,

Quoting Richard Ulrich (2024-04-30 10:44:46)
> We use fakechroot for building a live OS that starts out with debootstrap.
> This worked fine for a while, but started to fail last week. Now apt crashes
> when it ties to download anything:
> [...]
> I am reporting this from my trixie system, but the same happens on bookworm
> systems, and bookworm is used inside the relevant Docker container.

read the section LIMITATIONS of the fakechroot man page. You have to use the
exact system on the outside of the chroot as you are using inside of the
chroot. I have CI systems testing fakechroot daily on a fresh Debian unstable
setup and it works fine. So if it doesn't work for you, then it is very likely
due to the limitations of fakechroot itself. Thus, tagging this bug as
unreproducible.

Now... I had a look at your Dockerfile and at your project livedeb. I think you
can do a number of improvements which in the end will allow you to completely
get rid of fakechroot.

Firstly, you are are using docker for isolation. Why do you even bother with
fakechroot? Can you not just use normal chroot inside of docker?

Secondly, you can completely get rid of docker if you replace debootstrap and
fakechroot with mmdebstrap in unshare mode. Using mmdebstrap has several
advantages for your use-case:

 1. it does not require superuser privileges, so you do not need to run
everything inside a docker container anymore

 2. its native output format is a tarball so you can just run:

mmdebstrap [...] | mksquashfs - filesystem.squashfs

 3. even better, squashfs is one of the native output formats of mmdebstrap so
if you don't need any special squashfs options you could even just call

mmdebstrap [..] filesystem.squashfs

 4. you want the result to be bit-by-bit reproducible. The output of mmdebstrap
is reproducible by default with the same SOURCE_DATE_EPOCH

The only reason I'm still maintaining fakechroot is because it's a fun hacking
project but given that unshared user namespaces provide far superior
functionality I heavily recommend against using fakechroot in production.

Write me if you need any help converting your dockerfile into mmdebstrap calls.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1070042: libarchive: archive_entry_stat returns zero size on mips64el with _FILE_OFFSET_BITS=64

[Johannes Schauer Marin Rodrigues]

Source: libarchive
Version: 3.7.2-2
Severity: normal
X-Debbugs-Cc: debian-m...@lists.debian.org, ty...@mit.edu
Control: affects -1 src:e2fsprogs

Hi,

the upload of e2fsprogs 1.47.1~rc1-1 to unstable yesterday uses a new
feature where libarchive is used to create filesystem images from
tarballs. This works on all architectures (main as well as ports) except
on mips64el:

https://buildd.debian.org/status/package.php?p=e2fsprogs

I attached a minimal reproducer. It includes an embedded gzipped tarball with a
single file called "myfilename" of size 10. It works fine if _FILE_OFFSET_BITS
is not set but with it, it reports a wrong archive member size of zero on
mips64el:

$ gcc -Wall -g -O0 libarchivetest.c -larchive -D_FILE_OFFSET_BITS=64
$ ./a.out
myfilename: 0

Since the reproducer only makes use of libarchive, I'm filing this as a
libarchive bug. Since this problem is mips64el specific, I'm putting the
porter-list in X-Debbugs-Cc.

The problem can also be reproduced on bookworm.

gdb can show what is going on (thanks to Theodore Ts'o for the initial
debugging):

(gdb) run
[...]
Breakpoint 1, main () at test.c:49
49  st = archive_entry_stat(entry);
(gdb) next
50  fprintf(stderr, "%s: %ld\n", 
archive_entry_pathname(entry), st->st_size);
(gdb) p *st
$2 = {st_dev = 1, st_pad1 = {0, 0, 0}, st_ino = 1, st_mode = 33188, st_nlink = 
0, 
  st_uid = 1000, st_gid = 1000, st_rdev = 0, st_pad2 = {0, 0, 10}, st_size = 0, 
  st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 1714375615, tv_nsec 
= 0}, 
  st_ctim = {tv_sec = 0, tv_nsec = 0}, st_blksize = 0, st_pad4 = 0, st_blocks = 
0, 
  st_pad5 = {0 }}

As one can see, the size "10" is not in st_size but in the padding before the
st_size member. This only happens when _FILE_OFFSET_BITS=64 and only on
mips64el.

Any idea what is going on with mips64el?

Thanks!

cheers, josch
#include 
#include 
#include 

// gzipped tarball created with xxd --include
unsigned char archive_tar_gz[] = {
  0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0xed, 0xce,
  0x41, 0x0a, 0xc2, 0x30, 0x10, 0x85, 0xe1, 0x59, 0x7b, 0x8a, 0x1c, 0x21,
  0xa3, 0xd3, 0xe4, 0x3c, 0xa5, 0x44, 0xb4, 0x98, 0x14, 0xda, 0xb8, 0xe8,
  0xed, 0xad, 0xb8, 0xd0, 0x95, 0xd2, 0x45, 0x29, 0xc2, 0xff, 0x6d, 0x1e,
  0xc3, 0x9b, 0xc5, 0xcb, 0xf3, 0xf9, 0x7a, 0x4b, 0xa5, 0xcd, 0x49, 0x36,
  0xe3, 0x17, 0xc1, 0xec, 0x99, 0x1a, 0x1b, 0xff, 0x99, 0x2f, 0x7a, 0x14,
  0xb5, 0xa0, 0xa7, 0x60, 0x4d, 0x88, 0x51, 0x96, 0x5b, 0x4d, 0xc5, 0xf9,
  0xed, 0x26, 0xbd, 0xdd, 0xa7, 0xda, 0x8e, 0xce, 0x49, 0x3f, 0x4c, 0xdd,
  0xe5, 0xcb, 0xdf, 0xaf, 0xfe, 0x4f, 0xe5, 0xb9, 0x1b, 0x4a, 0x4d, 0xa5,
  0x1e, 0xf6, 0x5e, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x58, 0xe3, 0x01, 0x16, 0x64, 0x3c, 0x30,
  0x00, 0x28, 0x00, 0x00
};
unsigned int archive_tar_gz_len = 136;

int main() {
int ret;
struct archive_entry *entry;
const struct stat *st;

struct archive *a = archive_read_new();
if (a == NULL) {
fprintf(stderr, "archive_read_new\n");
return 1;
}
archive_read_support_filter_all(a);
archive_read_support_format_all(a);
ret = archive_read_open_memory(a, archive_tar_gz, archive_tar_gz_len);
if (ret != ARCHIVE_OK) {
fprintf(stderr, "archive_read_open_memory\n");
return 1;
}
for (;;) {
ret = archive_read_next_header(a, );
if (ret == ARCHIVE_EOF) {
break;
}
if (ret != ARCHIVE_OK) {
fprintf(stderr, "archive_read_next_header\n");
return 1;
}
st = archive_entry_stat(entry);
fprintf(stderr, "%s: %ld\n", archive_entry_pathname(entry), 
st->st_size);
}
archive_read_close(a);
archive_read_free(a);
return 0;
}

Bug#1070007: sbuild/unshare: writing to /dev/stdout denied

[Johannes Schauer Marin Rodrigues]

Quoting Aurelien Jarno (2024-04-28 15:57:29)
> When running sbuild in unshare chroot mode, it is not possible to write to
> /dev/stdout:
> 
> | echo test > /dev/stdout
> | sh: 1: cannot create /dev/stdout: Permission denied
> 
> This is the reason of the FTBFS of at least clisp and supervisor when using
> the unshare chroot mode of sbuild.

This works in podman. So somehow it's possible to connect /dev/stdout in a way
which preserves its intended functionality. Probably it would be useful to find
out how podman does this. For what its worth, mmdebstrap itself suffers from
the same problem, so whatever fix is used in sbuild should probably also be
added to mmdebstrap.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1070020: autopkgtest: (unrelated) packages not found

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Chris Hofstaedtler (2024-04-28 18:30:56)
> the autopkgtests for mmdebstrap as part of migration tests for testing/amd64
> fail with apt reporting 'Not found' errors.
> 
> As an example, for this scenario:
> mmdebstrap 1.4.3-6
> util-linux/2.40-8 gdm3/46.0-2 sssd/2
> src:util-linux from unstable
> src:gdm3 from unstable
> src:sssd from unstable
> https://ci.debian.net/packages/m/mmdebstrap/testing/amd64/46033215/
> 
> 1085s Err:22 http://127.0.0.1/debian unstable/main amd64 libssl3 amd64 3.1.5-1
> 1085s   404  File not found [IP: 127.0.0.1 80]
> ...
> 1085s Err:41 http://127.0.0.1/debian unstable/main amd64 libdb5.3 amd64 
> 5.3.28+dfsg2-4+b1
> 1085s   404  File not found [IP: 127.0.0.1 80]
> ...
> 1085s Err:73 http://127.0.0.1/debian unstable/main amd64 libgdbm6 amd64 
> 1.23-5+b1
> 1085s   404  File not found [IP: 127.0.0.1 80]
> 1085s Err:74 http://127.0.0.1/debian unstable/main amd64 libgdbm-compat4 
> amd64 1.23-5+b1
> 1085s   404  File not found [IP: 127.0.0.1 80]
> 
> This looks like a pinning problem or some other issue. Given libssl3 is
> a problem here, which is NOT a package considered for migration in this
> test, I don't see how the test failure is actually related to the
> involved packages.
> If this is a transient situation, please detect it and exit with status 77.

unfortunately, this problem is not transient but reproducible. The problem is,
that the involved libraries which are 404 have alternative 64bit time_t
versions in unstable:

libssl3 -> libssl3t64
libdb5.3 -> libdb5.3t64
libgdbm6 -> libgdbm6t64
libgdbm-compat4 -> libgdbm-compat4t64

And apt chooses to satisfy the (virtual) dependencies on those differently,
depending on the surrounding dependency satisfaction problem. Related,
debootstrap is also broken by the virtual providers of libssl3: #1069787

Feel free to tell the release team to ignore these failures if necessary. They
will go away once the 64bit time_t transition is finished.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1069793: wrong package name mnt-reform-setup-wizard -- should be reform-setup-wizard

[Johannes Schauer Marin Rodrigues]

Package: reform-setup-wizard
Version: 0.1.0-1
Severity: serious

As it says in subject.

Bug#953844: This appears to be due to "provide-stdint-for-embedded.patch"

[Johannes Schauer Marin Rodrigues]

Hi Keith,

On Thu, 14 Jul 2022 15:19:22 +0100 Steve Capper  wrote:
> With some stracing, it became apparent that the newlib headers were 
> pulling in an extra stdint.h (that wasn't part of newlib). I rebuilt the 
> gcc-arm-none-eabi package with the "provide-stdint-for-embedded.patch" 
> removed. That then allowed me to build the SCP firmware without issue.

I made the same discovery when building my package pico-sdk (currently in NEW).
The package builds fine if I remove provide-stdint-for-embedded.patch from
gcc-arm-none-eabi. But with gcc-arm-none-eabi in unstable I have to apply the
following patch to my package pico-sdk:

--- a/test/pico_stdlib_test/pico_stdlib_test.c
+++ b/test/pico_stdlib_test/pico_stdlib_test.c
@@ -9,6 +9,7 @@
 #include "pico/stdlib.h"
 #include "pico/bit_ops.h"
 #include 
+#define PRIu64 "lu"
 
 void test_builtin_bitops() {
 int32_t x = 0;
--- a/test/pico_time_test/pico_time_test.c
+++ b/test/pico_time_test/pico_time_test.c
@@ -12,6 +12,7 @@
 #include "pico/test.h"
 #include 
 PICOTEST_MODULE_NAME("pico_time_test", "pico_time test harness");
+#define PRIi64 "li"
 
 #define NUM_TIMEOUTS 500
 #define MAX_TIMERS_PER_POOL 250

I landed here as Raspberry PI developer Andrew Scheller found this bug with the
Debian packaging of of gcc-arm-none-eabi.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1063501: building curl from source fails the island test

[Johannes Schauer Marin Rodrigues]

Quoting Milan Kupcevic (2024-04-21 01:03:12)
> On 4/20/24 15:59, Johannes Schauer Marin Rodrigues wrote:
> > Quoting Milan Kupcevic (2024-04-20 21:46:14)
> >> On 4/20/24 15:05, Johannes Schauer Marin Rodrigues wrote: [...]
> >>> Quoting Milan Kupcevic (2024-04-20 20:50:27)
> >>>> This package builds just fine either on or off an island. The "pre-built
> >>>> artifacts" is actually the build support provided by the upstream for 
> >>>> their
> >>>> official release package. It is nice to rebuild the build support, but 
> >>>> is not
> >>>> required nor always desired.
> >>>
> >>> what is your reasoning to not rebuild them and to instead use the 
> >>> pre-built
> >>> artifacts from the release package?
> >>>
> >>> Would anything break?
> >>>
> >>
> >> Stunt lines injected in the building scripts would be very undesirable.
> > 
> > How about using the upstream git instead of the release tarball as the base 
> > for
> > the packaging?
> I would rather stick with the official release tarballs as they get signed
> with the upstream developer's key.

I think we just recently had a long discussion in Debian about using the
upstream git as source for the packaging instead of the release tarball in the
light of how the recent xz-utils attack was performed. Maybe you can convince
upstream to sign their git commits and/or tags.

If you think there is nothing actionable about this bug, feel free to close it.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1063501: building curl from source fails the island test

[Johannes Schauer Marin Rodrigues]

Quoting Milan Kupcevic (2024-04-20 21:46:14)
> On 4/20/24 15:05, Johannes Schauer Marin Rodrigues wrote: [...]
> > Quoting Milan Kupcevic (2024-04-20 20:50:27)
> >> This package builds just fine either on or off an island. The "pre-built
> >> artifacts" is actually the build support provided by the upstream for their
> >> official release package. It is nice to rebuild the build support, but is 
> >> not
> >> required nor always desired.
> > 
> > what is your reasoning to not rebuild them and to instead use the pre-built
> > artifacts from the release package?
> > 
> > Would anything break?
> > 
> 
> Stunt lines injected in the building scripts would be very undesirable.

How about using the upstream git instead of the release tarball as the base for
the packaging?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1063501: building curl from source fails the island test

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Milan Kupcevic (2024-04-20 20:50:27)
> This package builds just fine either on or off an island. The "pre-built
> artifacts" is actually the build support provided by the upstream for their
> official release package. It is nice to rebuild the build support, but is not
> required nor always desired.

what is your reasoning to not rebuild them and to instead use the pre-built
artifacts from the release package?

Would anything break?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1069300: ITP: autotiling -- automatically switch the window split orientation in sway and i3

[Johannes Schauer Marin Rodrigues]

Package: wnpp
Severity: wishlist
Owner: Johannes Schauer Marin Rodrigues 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: autotiling
  Version : 1.9.1
  Upstream Contact: Piotr Miller 
* URL : https://github.com/nwg-piotr/autotiling/
* License : GPL-3+
  Programming Lang: Python
  Description : automatically switch the window split orientation in sway 
and i3

This script uses the i3ipc-python library to switch the layout splith/splitv
depending on the currently focused window dimensions. It works on both sway and
i3 window managers.

Bug#1069293: ITP: pico-sdk -- headers and libraries to write programs for RP2040-based devices

[Johannes Schauer Marin Rodrigues]

Package: wnpp
Severity: wishlist
Owner: Johannes Schauer Marin Rodrigues 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: pico-sdk
  Version : 1.5.1
  Upstream Contact: https://github.com/raspberrypi/pico-sdk/issues
* URL : https://github.com/raspberrypi/pico-sdk/
* License : BSD-3-clause
  Programming Lang: C/C++
  Description : headers and libraries to write programs for RP2040-based 
devices

The Raspberry Pi Pico SDK provides the headers, libraries and build system
necessary to write programs for the RP2040-based devices such as the Raspberry
Pi Pico in C, C++ or assembly language.

The SDK is designed to provide an API and programming environment that is
familiar both to non-embedded C developers and embedded C developers alike. A
single program runs on the device at a time and starts with a conventional
main() method. Standard C/C++ libraries are supported along with C level
libraries/APIs for accessing all of the RP2040's hardware include PIO.

Additionally the SDK provides higher level libraries for dealing with timers,
synchronization, USB (TinyUSB) and multi-core programming along with various
utilities.

The SDK can be used to build anything from simple applications, to fully
fledged runtime environments such as MicroPython, to low level software such
as RP2040's on-chip bootrom itself.

Bug#1069290: ITP: picotool -- interact with RP2040 devices or binaries

[Johannes Schauer Marin Rodrigues]

Package: wnpp
Severity: wishlist
Owner: Johannes Schauer Marin Rodrigues 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: picotool
  Version : 1.1.2
  Upstream Contact: Graham Sanderson 
* URL : https://github.com/raspberrypi/picotool/
* License : BSD-3-clause
  Programming Lang: C++
  Description : interact with RP2040 devices or binaries

Tool for interacting with a RP2040 device in BOOTSEL mode, or with a RP2040
binary. It allows one to load programs onto the device or save programs from
flash to a file. It allows one to verify device contents or reboot the device.

Bug#1069245: ITP: tinyusb -- USB stack for embedded systems

[Johannes Schauer Marin Rodrigues]

Package: wnpp
Severity: wishlist
Owner: Johannes Schauer Marin Rodrigues 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: tinyusb
  Version : 0.16.0
  Upstream Contact: Ha Thach 
* URL : https://www.tinyusb.org/
* License : Expat
  Programming Lang: C
  Description : USB stack for embedded systems

TinyUSB is an open-source cross-platform USB Host/Device stack for embedded
system, designed to be memory-safe with no dynamic allocation and thread-safe
with all interrupt events are deferred then handled in the non-ISR task
function.

Bug#1068775: sbuild-qemu: fails to work, if piuparts is requested, but not installed on the host

[Johannes Schauer Marin Rodrigues]

Hi Francesco,

Quoting Francesco Poli (wintermute) (2024-04-11 00:13:51)
> I am trying to set up sbuild-qemu to build (and test) Debian packages.
> 
> After creating the virtual machine image:
> 
>   $ mkdir -p ~/.cache/sbuild/build
>   $ cd /dev/shm
>   $ TMPDIR=/dev/shm mmdebstrap-autopkgtest-build-qemu \
> --size=25G --boot=efi sid unstable-autopkgtest-amd64.img
>   $ mv -i unstable-autopkgtest-amd64.img ~/.cache/sbuild/
> 
> I prepared the following configuration file:
> 
>   $ cat ~/.sbuildrc 
>   $source_only_changes = 1;
>   $autopkgtest_require_success = 1;
>   $lintian_require_success = 0;
>   $piuparts_require_success = 1;
>   $run_autopkgtest = 1;
>   $run_lintian = 1;
>   $run_piuparts = 1;
>   $build_dir = "$HOME/.cache/sbuild/build";
>   
>   # don't remove this, Perl needs it:
>   1;
> 
> I can update the virtual machine (if I create a symlink, see bug [#1061816]):
>
>   $ sbuild-qemu-update --boot=efi unstable-autopkgtest-amd64.img
> 
> [#1061816]: 

I think 1061816 was fixed with 0.85.7 and the changelog was just missing the
"closes" entry:

https://tracker.debian.org/news/1518576/accepted-sbuild-0857-source-into-unstable/

> But, when I try to build a package from withing the unpacked source
> tree:
> 
>   $ sbuild-qemu --boot=efi --overlay-dir=/dev/shm
>   sbuild --dist unstable --purge-build=never --purge-deps=never 
> --chroot-mode=autopkgtest --autopkgtest-virt-server=qemu 
> --autopkgtest-virt-server-opt --overlay-dir=/dev/shm 
> --autopkgtest-virt-server-opt --qemu-architecture=x86_64 
> --autopkgtest-virt-server-opt --ram-size=2048 --autopkgtest-virt-server-opt 
> --cpus=2 --autopkgtest-virt-server-opt --boot=efi 
> --autopkgtest-virt-server-opt 
> /home/$USER/.cache/sbuild/unstable-autopkgtest-amd64.img 
> --bd-uninstallable-explainer apt
> Error reading configuration: PIUPARTS binary 'piuparts' does not exist or is 
> not executable at /usr/share/perl5/Sbuild/Conf.pm line 76.
> 
> Now, piuparts is indeed not installed on the host:
> 
>   $ apt policy piuparts
>   piuparts:
> Installed: (none)
> Candidate: 1.4.1
> Version table:
>1.4.1 800
>   800 http://deb.debian.org/debian testing/main amd64 Packages
>   500 http://deb.debian.org/debian unstable/main amd64 Packages
> 
> However, I thought that setting $run_piuparts = 1 in ~/.sbuildrc caused
> piuparts to be run inside the virtual machine guest system, not on the
> host system!

What made you think that? Is the error message above not clear enough? Do you
think it should be improved to make things clearer? Do you have suggestions for
a message that would've better explained that piuparts needs to be installed on
the host system?

> Hence, I thought that piuparts was going to be automatically installed
> and executed on the guest system (actually on the throw-away overlay).
> And I thought that the same was going to happen for $run_autopkgtest = 1
> and for $run_lintian = 1 ...
> 
> Am I completely off-track?
> What did I fail to understand?
> 
> Why does sbuild-qemu insist that piuparts be installed on the *host*
> system?

Because it needs to be installed on the host. In the same way as autopkgtest
needs to be on the host. What can sbuild improve?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1068382: sbuild: Support tarballs not including ./ when using the unshare chroot mode

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Santiago Vila (2024-04-04 20:03:08)
> El 4/4/24 a las 19:29, Johannes Schauer Marin Rodrigues escribió:
> > Also I'm curious: what is your motivation for using unshare mode if you are
> > creating your chroots using superuser privileges?
> >
> > And are you really storing your chroots in /srv instead of letting them get
> > picked up automatically in ~/.cache/sbuild/unstable-arm64.tar?
> 
> I am testing the unshare backend at the request of Jochen.

I now also talked with Jochen on IRC. :)

> As we speak, I am building all packages in bookworm using unshare.

Thank you for doing that! I'd assume that quite a few testsuites break in that
environment.

> I already had a working setup for building packages using the other backends.
> In my setup, I run debootstrap in a master server, create the tarballs there,
> and the build nodes retrieve those tarballs using wget before they start to
> build packages.
> 
> So in my setup the logical thing to do to test the unshare backend
> was to make symlinks to the already existing tarballs at /srv, since the
> nodes are not creating any tarballs.

That makes sense, yes. I think it makes sense for sbuild to accept tarballs
without the leading ./. I think the best way to achieve this, is to filter out
the device nodes from the input tarball. The utility that I know that can do
this is mmtarfilter like this:

mmtarfilter --type-exclude CHRTYPE --type-exclude BLKTYPE

Unfortunately, that utility is written in Python and sbuild currently does not
depend on Python. I wonder if the same can be done in Perl? Maybe I should
write a very crude tarball parser that is able to do just enough parsing...

signature.asc
Description: signature

Bug#834736: [buildd-tools-devel] Bug#834736: sbuild: Use basic format for ISO 8601 timestamps (for build logs filenames)

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Santiago Vila (2024-04-04 20:35:47)
> El 4/4/24 a las 19:44, Johannes Schauer Marin Rodrigues escribió:
> > instead of doing that, you could've worked around this by just placing the
> > build log into a dedicated temporary directory and then copying it to where 
> > you
> > want it after the build is finished.
> 
> That would be an option, yes, but maybe it's too convoluted for my taste.

it's a trade-of between doing that and patching sbuild over and over and over
again. Or working on a patch for this bug I guess. :D

> > I think I'd find an option useful which allows to pass a completely custom
> > build log filename. There is already the LOG_DIR configuration variable.
> > Maybe add the LOG_FILENAME configuration variable containing the filename
> > that the log will have when placed in LOG_DIR. The command line option can
> > be named --log-filename accordingly.
> 
> But I only need the date part. I still want sbuild to take care of the 
> filename,
> just not of the timestamp part of the filename.
> 
> I'll take a look at your hints above and see what I can do.

You could also add a LOG_TIMEFORMAT variable which by default would be set to
"%FT%TZ" but which you could customize to your liking.

> > If you run sbuild with a fake time, you also run apt with a fake time which
> > means you potentially get into trouble with https mirrors and their
> > certificate expiration times, for example.
> 
> I know, but that would violate the principle of desert island for QA,
> which I formulate as follows:
> 
> "It must be possible to build all packages in stable several years after
> the release without having to update any of them".
> 
> or in other words:
> 
> "Packages in stable must not contain time bombs preventing their succesful 
> build
> in the few years which follow the release of stable"
> 
> In practice I already found the expiration problem for the release files
> that you mention, so I already know how far in the future I can go.
> 
> My plan is to ask RMs to consider RC any FTBFS bug which happens not just
> the day before we release trixie as stable (which is what we did for bookworm)
> but also some years ahead (for example, maybe two for stable, two for
> oldstable, and one more for general LTS).

Ah okay, I understand. Yes, in that case it makes sense to run sbuild itself in
the future as well. But then you should not run sbuild with LOG_TIMEFORMAT
because a person in the future will not do that. To faithfully re-create what
will happen in the future, you might want to use sbuild in the same way as
well.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#834736: [buildd-tools-devel] Bug#834736: sbuild: Use basic format for ISO 8601 timestamps (for build logs filenames)

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Santiago Vila (2024-04-04 15:02:05)
> El 4/4/24 a las 14:07, Johannes Schauer Marin Rodrigues escribió:
> > well this is an old bug. How have you worked around it being open for the 
> > past
> > six years?
> 
> This is important for me, so I'm still patching my own sbuild version.  Yes,
> every single time.

instead of doing that, you could've worked around this by just placing the
build log into a dedicated temporary directory and then copying it to where you
want it after the build is finished.

> Can you give me a small hint about how I should proceed?  (For example, which
> files in the source would require to be modified?) (Preferred name for the
> command line?)

An example commit that adds a new option would be
a44b29e76450ce8c80467e0957ebc6909167e425 by Jochen. You need to touch these
files:

 - lib/Sbuild/Build.pm to implement the thing
 - lib/Sbuild/Conf.pm to add the configuration option
 - lib/Sbuild/Options.pm to add the command line argument changing the config
   option
 - man/sbuild.1.in to document the command line argument

> > I also think that there is a more elegant solution to this. If this is just
> > about the timestamp, why not add a new configuration option which allows
> > customizing the default strftime format %FT%TZ to be anything else? You
> > could use such a mechanism to store your own timestamp. This is what you
> > suggested doing earlier in this bug report.
> 
> Unfortunately, it's not just about the date format (that's only how it 
> started),
> it's mainly about sbuild allowing me to specify the build log filename 
> beforehand,
> without it second-guessing the string I really want to use.

I think I'd find an option useful which allows to pass a completely custom
build log filename. There is already the LOG_DIR configuration variable. Maybe
add the LOG_FILENAME configuration variable containing the filename that the
log will have when placed in LOG_DIR. The command line option can be named
--log-filename accordingly.

> I have a recent case where the usefulness of this may be seen easily:
> 
> Last time I tried to rebuild bookworm from source, I found several packages
> which FTBFS because of expired SSL certificates being used in the tests.
> 
> So I decided to build bookworm "in the future" to know which packages will 
> fail
> for similar reasons in the next years.
> 
> I first tried using libfaketime but did not managed for it to work easily.
> 
> At the end, the most reliable way I found was to actually change the
> system clock, like this:
> 
> timedatectl set-time '2031-01-14 12:00:00'
> 
> Now, here is the thing: When I do that, sbuild would think (rightfully)
> that I'm on such date, and would use it for both the contents of
> the build log and also the build log filename.
> 
> However, in this case I still want the build log filename to be
> the present day, because it's the real date where I tried such build in the
> future.

Don't run sbuild in the future. Only run dpkg inside sbuild in the future. This
is what the BUILD_ENV_CMND option lets you do. From the man page:

> This command is run with the dpkg-buildpackage command line passed to it (in
> the chroot, if doing a chrooted build).  It is used by the sparc buildd
> (which is sparc64) to call the wrapper script that sets the environment to
> sparc (32-bit).  It could be used for other build environment setup scripts.

If you run sbuild with a fake time, you also run apt with a fake time which
means you potentially get into trouble with https mirrors and their certificate
expiration times, for example.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1068382: sbuild: Support tarballs not including ./ when using the unshare chroot mode

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Santiago Vila (2024-04-04 15:24:13)
> > how did you create that tarball?
> 
> debootstrap to a directory
> cd /chroot/directory
> tar czvf /srv/whatever.tar.gz *
> 
> Yes, I know what using "." instead of "*" would solve the problem, but as I 
> said,
> sbuild already supports perfectly tarballs without ./ in the "file" backend,
> so the consistent thing would be to support them for unshare as well.

you can do this in less commands by using

tar -C /chroot/directory -czvf /srv/whatever.tar.gz .

In that case, the "*" wildcard would of course not work which is why usually,
people use '.' instead of "*". Another reason against using the glob operator
with tar is, that it will exclude hidden files and then you will end up with a
tarball that does not contain everything in the current directory. Using '.' is
really the safer option independent on whether you use -C or not.

Also I'm curious: what is your motivation for using unshare mode if you are
creating your chroots using superuser privileges?

And are you really storing your chroots in /srv instead of letting them get
picked up automatically in ~/.cache/sbuild/unstable-arm64.tar?

> > Your addition of --anchored drops support for tarballs with members that
> > start with ././ or with ./././ and so on.
> 
> Yes, but those tarballs are a lot more uncommon, so if we had to choose 
> between
> supporting "" and "./" or supporting "./" and "././" and "./././" etc, I guess
> supporting "" and "./" would be preferred.
> 
> So, well spotted, but I don't think that dropping support for ././
> would be a big deal.

I think those tarballs are even more uncommon than yours, yes. But then you are
also the first in six years that the unshare backend existed to have come
across that problem.  :)

> > Your second patch is described as "Do not extract anything in /dev" but
> > what it actually excludes is the directory itself and not just everything
> > in it.
> That's why I said "untested" :-) The point was to convey the idea, not the
> implementation.

Unfortunately the idea cannot work. Because the point of the exclude patterns
is to exclude everything that is a character special file. The point is not to
exclude everything in /dev.

> > Maybe a better solution would be to pipe the tarballs through mmtarfilter
> > and just remove all the device nodes from them. This avoids requiring any
> > --exclude options for tar.
> 
> Hmm, but if we get to such point, maybe we should really advocate for
> debootstrap and friends to stop including any /dev/* files at all.

I'd be very much in favour of doing that. With mmdebstrap you can create a
chroot without device nodes by running this:

mmdebstrap --variant=buildd --skip=output/mknod unstable 
~/.cache/sbuild/unstable-arm64.tar

The reason they are created in debootstrap (I think) is so that you can easily
just chroot into them without having to think of bind-mounting /dev beforehand.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1068382: sbuild: Support tarballs not including ./ when using the unshare chroot mode

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Santiago Vila (2024-04-04 12:17:14)
> While trying to use the unshare backend I found this error:
> 
> tar: dev/full: Cannot mknod: Operation not permitted
> tar: dev/urandom: Cannot mknod: Operation not permitted
> tar: dev/console: Cannot mknod: Operation not permitted
> tar: dev/ptmx: Cannot mknod: Operation not permitted
> tar: dev/random: Cannot mknod: Operation not permitted
> [...]
> 
> The reason (as Jochen identified quickly) is that my tarball did not
> include ./ entries, so the exclude patterns in lib/Sbuild/ChrootUnshare.pm
> had no effect.
> 
> Since I don't think a tarball without ./ is really "wrong" to the point
> that it needs to be recreated (this is in fact the very first in my life
> that a tarball without ./ causes any kind of trouble), I think it would be
> desirable to support those tarballs as well.

how did you create that tarball?

> So, we (Jochen and myself) wonder if any of the following patches would be
> acceptable to you.
> 
> The first patch adds --anchored option to tar invocation so that the exclude
> patterns are matched from the beginning only (not anywhere in the filename),
> then adds the remaining eight exclude patterns for tarballs without "./".
> 
> I could agree that the end result is not very nice, but it's simple,
> effective, and imo it's not really so much ugly.
> 
> However, while we are at it, I wonder why it's necessary to uncompress
> anything in /dev at all these days. Would it work if everything in /dev is
> excluded?
> 
> The second patch (untested) supports tarballs with or without ./ and at the
> same time simplifies the exclude patterns to just two.

Your addition of --anchored drops support for tarballs with members that start
with ././ or with ./././ and so on.

Your second patch is described as "Do not extract anything in /dev" but what it
actually excludes is the directory itself and not just everything in it.

Maybe a better solution would be to pipe the tarballs through mmtarfilter and
just remove all the device nodes from them. This avoids requiring any --exclude
options for tar.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#834736: [buildd-tools-devel] Bug#834736: sbuild: Use basic format for ISO 8601 timestamps (for build logs filenames)

[Johannes Schauer Marin Rodrigues]

Hi,

well this is an old bug. How have you worked around it being open for the past
six years?

Quoting Santiago Vila (2024-04-04 12:57:13)
> I've simplified the proposed patch to the one in the attach, and now it is
> probably as small as it can be without losing readability (two lines).
> 
> It works for me, I have been using it for eight years now, and it's
> still my preferred way to tell sbuild the filename that I want to use
> for the build log.
> 
> [ If the environment variable name is not good enough, just use one which
> makes sense. I will gladly adapt my scripts ]

I'm not happy with using an environment variable for this. Sbuild only has a
single environment variable that is specific to it: SBUILD_CONFIG. The usual
way to customize an sbuild run is via the config or command line arguments, not
via environment variables. Also, your patch is missing documentation.

I also think that there is a more elegant solution to this. If this is just
about the timestamp, why not add a new configuration option which allows
customizing the default strftime format %FT%TZ to be anything else? You could
use such a mechanism to store your own timestamp. This is what you suggested
doing earlier in this bug report.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061816: mmdebstrap-autopkgtest-build-qemu VM image cannot be updated with sbuild-qemu-update

[Johannes Schauer Marin Rodrigues]

Hi Christian,

Quoting Johannes Schauer Marin Rodrigues (2024-03-28 10:05:53)
> Christian, can you give this a try as well?

I think sbuild-qemu-boot and sbuild-qemu-update should do the same as
autopkgtest did here:

https://salsa.debian.org/ci-team/autopkgtest/-/commit/7a4954ded0f24221ac34ca0aaf10f3f9b083afa2

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061816: mmdebstrap-autopkgtest-build-qemu VM image cannot be updated with sbuild-qemu-update

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-03-28 09:40:27)
> > But your system does not have /usr/share/OVMF/OVMF_CODE.fd?
> Where should it be?

it's from the package ovmf.

> Debian package [search says] it should be in package 'ovmf'.  However, the
> [file list] seems to disagree...
> 
> [search says]: 
> 
> [file list]: 

It's in bookworm but not in trixie:

https://packages.debian.org/bookworm/all/ovmf/filelist

This was changed in edk2 (= 2023.11-2), see the corresponding NEWS.Debian:

$ cat /usr/share/doc/ovmf/NEWS.Debian 
edk2 (2023.11-2) unstable; urgency=medium

  The 2MB ovmf pflash images, OVMF_CODE.*.fd and OVMF_VARS.*.fd, have now
  been removed. Users of the 2MB pflash images should migrate to their 4MB
  image counterparts:

OVMF_CODE.fd -> OVMF_CODE_4M.fd
OVMF_CODE.ms.fd  -> OVMF_CODE_4M.ms.fd
OVMF_CODE.secboot.fd -> OVMF_CODE_4M.secboot.fd
OVMF_VARS.fd -> OVMF_VARS_4M.fd
OVMF_VARS.ms.fd  -> OVMF_VARS_4M.ms.fd

  2MB VAR images are not compatible with 4MB CODE images. Users must
  migrate both CODE and VARS images simultaneously.

  A migration guide is provided at:
/usr/share/doc/ovmf/howto-2M-to-4M-migration.md.gz

 -- dann frazier   Wed, 27 Dec 2023 10:15:33 -0700

Could you temporarily add a symlink from OVMF_CODE.fd to OVMF_CODE_4M.fd and
check if it still works?

Christian, can you give this a try as well?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061816: mmdebstrap-autopkgtest-build-qemu VM image cannot be updated with sbuild-qemu-update

[Johannes Schauer Marin Rodrigues]

On 2024-03-28 08:26, Francesco Poli wrote:

  $ sbuild-qemu-update --boot=efi \
  ~/var/cache/sbuild/sid-amd64.img 1> update.out 2> update.err


You added --boot=efi, good!


  before (last 100 chars): "/share/OVMF/OVMF_CODE.fd: Could not open
'/usr/share/OVMF/OVMF_CODE.fd': No such file or directory\r\n"


But your system does not have /usr/share/OVMF/OVMF_CODE.fd?

Bug#1067216: ITP: tuigreet -- Graphical console greeter for greetd

[Johannes Schauer Marin Rodrigues]

Package: wnpp
Severity: wishlist
Owner: Johannes Schauer Marin Rodrigues 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: tuigreet
  Version : 0.8.0
  Upstream Contact: Antoine POPINEAU 
* URL : https://github.com/apognu/tuigreet/
* License : GPL-3.0
  Programming Lang: Rust
  Description : Graphical console greeter for greetd

A terminal greeter for the greetd login manager daemon using Rust tui to render
login prompt widgets to the terminal. It supports:

 - displaying the current date and time
 - fetching sessions from /usr/share/xsessions and /usr/share/wayland-sessions
 - remembering the last user and their chosen session
 - shutdown and reboot
 - displaying all usernames returned by NSS in a drop-down menu

Bug#1067153: ITP: reform-setup-wizard -- GTK-based first-boot setup wizard

[Johannes Schauer Marin Rodrigues]

Package: wnpp
Severity: wishlist
Owner: Johannes Schauer Marin Rodrigues 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: reform-setup-wizard
  Version : 0.1.0
  Upstream Contact: Lukas F. Hartmann 
* URL : https://source.mnt.re/reform/mnt-reform-setup-wizard
* License : GPL-3+
  Programming Lang: Rust
  Description : GTK-based first-boot setup wizard

This package provides a step-by-step graphical setup wizard for systems that
are not setup using the Debian installer. As such it is supposed to be executed
only once upon the first boot of a vanilla system image. Using a GTK-based user
interface, it provides the following functionalities:

 - choosing the keyboard layout
 - choosing the local timezone
 - setting up a new user and password

This utility will be used by the MNT Reform system images as provided on
mntre.com. I intend to package this software for Debian so that I can also add
this functionality to the system images I'm offering on
https://reform.debian.net without having to resort to pre-built binary blobs.
The reform-setup-wizard is not specific to the MNT Reform laptop but can be
re-used on other platforms which are installed using a system image as well,
like Raspberry Pi based systems.

One of its dependencies, rust-pwhash, is not in Debian yet. I provided
packaging for it to the Rust team here:
https://salsa.debian.org/rust-team/debcargo-conf/-/merge_requests/631

Thanks!

cheers, josch

Bug#895570: devscripts: wrap-and-sort should default to -ast (--wrap-always, --short-indent, --trailing-comma)

[Johannes Schauer Marin Rodrigues]

Hi,

On Wed, 6 Mar 2024 11:04:01 +0100 Paride Legovini  wrote:
> On Sun, 03 Mar 2024 16:26:41 +0100 Benjamin Drung  wrote:
> > Time to join this discussion. The current default was the preference of
> > the author 14 years ago. My taste has change a bit since then. I am open
> > to change the default. --trailing-comma for wrapped lines is a good
> > idea, --short-indent is okay. I don't like --wrap-always in case there
> > are only two or three entries.
> > 
> > The new default should not only based on the preference, but also on
> > what is used the most. Can someone collect the information: which teams
> > use which options, how many packages use what?
> 
> I did some unscientific research on codesearch looking for d/changelog entries
> mentioning `wrap-and-sort -` (i.e. wrap-and-sort with some options). This is
> the query:
> 
> https://codesearch.debian.net/search?q=path%3Adebian%2Fchangelog+wrap-and-sort+-=1
> 
> Apparently -a is the single most used option, very often used together with
> -s and -t. I found similar results by searching GitHub:
> 
> https://github.com/search?q=path%3Adebian%2Fchangelog+%22wrap-and-sort+-%22=code
> 
> Looks like salsa (GitLab Free) doesn't allow to do instance-wide code 
> searches.

I disagree that the new default should be what is used the most. For example
debhelper versions do not become the default only after they are used the most.
The thing that makes switching defaults easier for debhelper is the explicit
opt-in for a new debhelper compat version which we don't have for wrap-and-sort
and I think it would very much be over-engineering to add such a feature for
something that is, in my opinion, of very little consequence. Furthermore, I
would not be surprised if many people using wrap-and-sort use the default
expecting that this is what is most well-liked by the project (because why else
would it be the default?). The question was asked in this email sub-thread:

https://lists.debian.org/161289428547.4135738.4002254931040787...@auryn.jones.dk

In that thread, same as in this bug, -ast was proposed as the default and
unless I missed something, there were no objections on debian-devel. Some even
argued, to make -b the default as well. So instead of asking "what is used
most" I'd like to ask, are there users of wrap-and-sort without -ast who would
be strongly against having to pass -AST to overwrite a potential new default?

That being said, I downloaded all debian/control files for all 36832 source
packages in Debian and ran the following shell script on them to figure out how
many source packages comply with which wrap-and-sort set of options:

for p in control/*; do
rm -f debian/control;
ln -s "../$p" debian/control;
for opt in "" -a -as -ast -astb -at -atb -ab -s -st -stb -sb -t -tb -b; 
do
wrap-and-sort --dry-run --file=debian/control $opt \
| grep --quiet debian/control \
|| echo $p >> w-a-s$opt;
done
done

It's of course still not possible to say whether a control file adheres to a
certain wrap-and-sort formatting style by accident or intentionally. Also,
packages can easily fall in multiple categories at the same time, for example
packages with only a single binary package which comply with -ast will
automatically also comply with -astb. There are also packages which comply with
-ast as well as with no options at all simply because they have no
Build-Depends nor Depends fields in debian/control. Here is the result sorted
by popularity in ascending order:

96 wrap-and-sort -st
96 wrap-and-sort -stb
   434 wrap-and-sort -as
   465 wrap-and-sort -tb
   489 wrap-and-sort -t
   579 wrap-and-sort -atb
   641 wrap-and-sort -at
  1341 wrap-and-sort -sb
  1405 wrap-and-sort -s
  2381 wrap-and-sort -astb
  2705 wrap-and-sort -ast
  2732 wrap-and-sort -b
  3020 wrap-and-sort
  3950 wrap-and-sort -ab
  4089 wrap-and-sort -a

So, wrap-and-sort -ast is very popular but it is not as popular as the current
default.

Do with that data what you wish. :)

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061388: sbuild doesn't support autotpkgest-virt-incus: Error: mkdir /sbuild-nonexistent: permission denied

[Johannes Schauer Marin Rodrigues]

Hi Stefano & Christian,

On Tue, 23 Jan 2024 14:30:07 + stefa...@debian.org wrote:
> Hi Debian (2024.01.23_13:41:47_+)
> > This seems to be because "incus exec" is trying to write ~/.config/incus,
> > but HOME has been set to /sbuild-nonexistent outside it.
> 
> Filed an incus upstream bug about handling this situation more gracefully:
> https://github.com/lxc/incus/issues/422

in that issue you asked exactly the question I was about to ask you. :)

Though it seems incus should now be able to deal gracefully with the situation
and there is nothing else that sbuild needs to do to handle this, correct?

Note, that the autopkgtest/podman backend has a similar issue, see #1033352 for
details. To fix this, Christian submitted this MR against sbuild:

https://salsa.debian.org/debian/sbuild/-/merge_requests/55

Reading how incus worked around this problem, maybe podman can do the same?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1064568: Stop trying to reject the project.

[Johannes Schauer Marin Rodrigues]

Hi PrelevatedInsider17763 and User0,

On Sat, 24 Feb 2024 11:51:29 + Member Madden Insider Dev 25120 
 wrote:
> I don't think that this project will not be worth it, I think this is a good
> masterpiece and will be worth it. By the way, if there was a beginner in this
> Linux distro, and if they need to apply any disk image file to a drive,
> include this package. In fact, this would help the user to get the disk image
> file and apply to a drive if they want to. Commonly, the user wouldn't use
> any other file than a disk image file.
> 
> And, it is used for disk image files only, not for photos, videos, text
> files, nor any other stuff.

neither Tino nor myself can veto your project being included into Debian.

That being said, you need to find a sponsor who has permission to upload this
into Debian unstable and who thinks that doing so is a good idea. Once you have
that, the only people who really can veto your package are ftp-master and maybe
the release team. I cannot speak for either of those two teams.

But that being said, I do agree with Tino that including this package into
Debian would be a bad idea.

In my opinion, it is bad software engineering to add a new tool for something
that is literally just a shell pipeline involving two processes and nothing
else.

What I want to bring across with my message is, that you might want to
reconsider investing energy into getting this into Debian as I think it is
quite likely that you will face more resistance than just from Tino and myself
further down that road.

Good luck.

cheers, josch

signature.asc
Description: signature

Bug#1060838: meson: should use the host gobject-introspection-1.0.pc for looking up g-ir-scanner

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Helmut Grohne (2024-01-15 13:04:27)
> Simon worked a lot on making gobject-introspection workable with cross
> compilation and this work has hit unstable now. While this seems to work
> fine with autotools, it tends to fail with meson.

I've just learned from Simon McVittie in #debian-devel that there is a way to
cross-build packages using meson and gobject-introspection without this patch
by configuring them with --cross-file. Here is an example from libportal:

https://salsa.debian.org/debian/libportal/-/commit/4efec56ef735eac25836acca867fe59812c3dc6d

I successfully used the same technique for gst-plugins-bad1.0 and
gst-plugins-good1.0. I'm recording this here so this technique doesn't get lost
and in case others want to have their packages cross-built before this or a
similar patch can get accepted.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1033352: sbuild: autokpgtest-virt-server needs host $HOME

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Christian Kastner (2023-03-23 09:53:05)
> Attempting to build a package with the autopkgtest-virt-podman backend fails
> because of what I suspect is an issue with $HOME directory handling. podman
> needs $HOME on the host to find containers, but it defaults to
> /sbuild-nonexistent, which I guess is meant for the target enviromnent.

is this a duplicate of #1061388?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1064266: python-lsp-black: FTBFS with black 24.2.0-1

[Johannes Schauer Marin Rodrigues]

Source: python-lsp-black
Version: 2.0.0-1
Severity: serious
Tags: ftbfs
Justification: fails to build from source
X-Debbugs-Cc: sylves...@debian.org

Hi,

python-lsp-black currently FTBFS in unstable. To track down the reason,
I ran debbisect:

DEBIAN_BISECT_SRCPKG=python-lsp-black debbisect --cache="$(pwd)/cache" 
2024-01-01 today /usr/share/doc/devscripts/examples/debbisect_buildsrc.sh
[...]
#6: trying 20240214T085951Z...
test script output: good
snapshot timestamp difference: 3.758796296296296 days
computation time left: 0:37:59.675485
approximately 4 steps left to test
#7: trying 20240215T151223Z...
test script output: bad
bisection finished successfully
  last good timestamp: 20240214T085951Z
  first bad timestamp: 20240215T151223Z
only one package differs: black 24.1.1-1 -> 24.2.0-1

I haven't investigated the failure yet.

I have put Sylvestre, who uploaded black 24.2.0-1 a few days ago in
X-Debbugs-Cc.

Sylvestre, do you have an idea what the cause for this error could be? From the
failing build log:

= test session starts ==
platform linux -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /<>
collected 21 items / 2 deselected / 19 selected

tests/test_plugin.py ..FF... [100%]

=== FAILURES ===
__ test_load_config_defaults ___

config = {'fast': False, 'line_length': 88, 'preview': False, 'pyi': False, ...}

def test_load_config_defaults(config):
config = load_config(str(fixtures_dir / "example.py"), config)

>   assert config == {
"line_length": 88,
"target_version": set(),
"pyi": False,
"fast": False,
"skip_magic_trailing_comma": False,
"skip_string_normalization": False,
"preview": False,
}
E   AssertionError: assert {'fast': Fals...': False, ...} == {'fast': 
Fals...': False, ...}
E Omitting 6 identical items, use -vv to show
E Differing items:
E {'target_version': {, , , }} != {'target_version': 
set()}
E Use -v to get more diff

It seems that the target_version member is now somehow empty?

Thanks!

cheers, josch

Bug#1063939: different timezones produce different pdf /ID with equal SOURCE_DATE_EPOCH

[Johannes Schauer Marin Rodrigues]

Source: fop
Version: 1:2.8-2
Severity: normal
Tags: patch
X-Debbugs-Cc: reproducible-b...@lists.alioth.debian.org
User: reproducible-bui...@lists.alioth.debian.org
Usertags: timezones

Hi,

even when keeping SOURCE_DATE_EPOCH equal, varying the timezone via the TZ
environment variable produces different PDFs when using fop. Specifically,
their /ID differs. Here is some reprotest output that shows the issue:

https://salsa.debian.org/debian/xmlstarlet/-/jobs/5296216

To reproduce, you can run this in the xmlstarlet source:

TZ=GMT-1 SOURCE_DATE_EPOCH=0 /usr/bin/fop -q doc/xmlstarlet-ug.fo 
doc/xmlstarlet-ug.pdf
TZ=GMT+1 SOURCE_DATE_EPOCH=0 /usr/bin/fop -q doc/xmlstarlet-ug.fo 
doc/xmlstarlet-ug.pdf

The reason is, that generateFileID() uses SimpleDateFormat().format() which is
not timezone aware and will use the current timezone for the string it
produces. My patch replaces that for DateFormatUtil.formatPDFDate() with UTC:

https://salsa.debian.org/java-team/fop/-/merge_requests/2

Thanks!

cheers, josch

Bug#1063895: Please mark fop as Multi-Arch: foreign

[Johannes Schauer Marin Rodrigues]

Package: fop
Version: 1:2.8-2
Severity: normal
Tags: patch

Hi,

if I'm not mistaken, the fop utility is not doing anything architecture
dependent. If that assertion is correct, please consider marking it as
Multi-Arch: foreign. I submitted a patch here:

https://salsa.debian.org/java-team/fop/-/merge_requests/1

Without this marking, source packages that build-depend on fop are not
able to satisfy their cross-build dependencies because they will try to
pull in the host-architecture fop which cannot work because fop is an
Architecture:all package. This affects at least 19 source packages in
Debian (I only checked the direct dependencies).

Thanks!

cheers, josch

Bug#1061816: Bug#1061636: mmdebstrap-autopkgtest-build-qemu VM image cannot be updated with sbuild-qemu-update

[Johannes Schauer Marin Rodrigues]

Control: tag -1 + pending

Hi Francesco


Quoting Francesco Poli (2024-02-12 22:46:29)
> On Mon, 29 Jan 2024 21:07:52 +0100 Francesco Poli wrote:
> > Yes, I think it makes sense to have a separate bug report, so that I
> > can be easily notified, once it becomes pending and once it gets closed.
> > 
> > Thanks!
> I see that [MR 54] has been merged.
> 
> [MR 54]: 

yes, thanks to the work of Christian (in CC).

> Is there anything else that needs to be fixed/enhanced in sbuild-qemu,
> in order for sbuild-qemu-update to be able to update
> mmdebstrap-autopkgtest-build-qemu VM images?

Theoretically not. If you want to make sure, feel free to clone the git and try
it out.

> If not, I think this bug report (#1061816) should be marked as pending.

Done.

> And probably mentioned in some debian/changelog entry. Or in some
> commit message, if you generate debian/changelog from commit messages...

Christian is probably going to do this on the next upload.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1063648: krb5: FTBFS on arm64, armel and ppc64el with "Can't resolve hostname" in dh_auto_test

[Johannes Schauer Marin Rodrigues]

Source: krb5
Version: 1.20.1-5+b1
Severity: serious
Tags: ftbfs

Hi,

there as a binNMU "Rebuild to sync binNMU versions" for krb5 and that
failed for arm64, armel and ppc64el:

https://buildd.debian.org/status/package.php?p=krb5

The error logs look very similar:

making check in lib/rpc...
make[4]: Entering directory '/<>/build/lib/rpc'
making check in lib/rpc/unit-test...
make[5]: Entering directory '/<>/build/lib/rpc/unit-test'
PYTHONPATH=../../../../src/util VALGRIND="" python3 
../../../../src/lib/rpc/unit-test/t_rpc.py 
*** Failure: ./client failed with code 1.
*** Last command (#10): ./client -t arm-conova-03 58621 host@arm-conova-03 1026
*** Output of last command:
Can't resolve hostname arm-conova-03
For details, see: /<>/build/lib/rpc/unit-test/testlog
Or re-run this test script with the -v flag:
cd /<>/build/lib/rpc/unit-test
PYTHONPATH=/<>/src/util /usr/bin/python3 
../../../../src/lib/rpc/unit-test/t_rpc.py -v

Use --debug=NUM to run a command under a debugger.  Use
--stop-after=NUM to stop after a daemon is started in order to
attach to it with a debugger.  Use --help to see other
options.
make[5]: *** [Makefile:644: check-pytests] Error 1
make[5]: Leaving directory '/<>/build/lib/rpc/unit-test'
make[4]: *** [Makefile:1595: check-recurse] Error 1
make[4]: Leaving directory '/<>/build/lib/rpc'
make[3]: *** [Makefile:973: check-recurse] Error 1
make[3]: Leaving directory '/<>/build/lib'
make[2]: *** [Makefile:1521: check-recurse] Error 1
make[2]: Leaving directory '/<>/build'
dh_auto_test: error: cd build && make -j1 check "TESTSUITEFLAGS=-j1 --verbose" 
VERBOSE=1 returned exit code 2
make[1]: *** [debian/rules:150: override_dh_auto_test] Error 25
make[1]: Leaving directory '/<>'
make: *** [debian/rules:153: build-arch] Error 2
dpkg-buildpackage: error: debian/rules build-arch subprocess returned exit 
status 2


Thanks!

cheers, josch

Bug#1063639: initscripts.postinst: avoid calling uname for non-empty DPKG_ROOT

[Johannes Schauer Marin Rodrigues]

Source: sysvinit
Version: 3.08-6
Severity: normal
Tags: patch
X-Debbugs-Cc: debian-h...@lists.debian.org
User: debian-h...@lists.debian.org
Usertags: hurd-i386
User: debian-d...@lists.debian.org
Usertags: dpkg-root-support

Hi,

this bug is similar to #1063629 in that it removes another use of "uname -s"
from maintainer scripts so that it will become possible to create chroots of a
foreign kernel using dpkg --force-script chroootless. In contrast to the other
bug, the solution I propose in this bug uses a non-empty DPKG_ROOT as an
indicator not to call "uname -s". The patch can be found here:

https://salsa.debian.org/debian/sysvinit/-/merge_requests/13

Thanks!

cheers, josch

Bug#1063638: util-linux.postinst: please avoid calling uname in postinst for foreign kernel DPKG_ROOT support

[Johannes Schauer Marin Rodrigues]

Package: util-linux
Version: 2.39.3-6
Severity: normal
Tags: patch
X-Debbugs-Cc: debian-h...@lists.debian.org
User: debian-h...@lists.debian.org
Usertags: hurd-i386
User: debian-d...@lists.debian.org
Usertags: dpkg-root-support

Hi,

when using dpkg --force-script-chrootless to create foreign architecture
chroots, the use of `uname -s` should be avoided as the uname utility will
print information about the kernel of the system creating the chroot. This goes
wrong if, for example, one attempts creating a hurd chroot on linux. In case of
util-linux, avoiding the call to uname can easily be achieved by checking for
the value of DEB_HOST_ARCH_OS during the build and only emitting the offending
postinst snippet for linux. Patch is here:

https://salsa.debian.org/debian/util-linux/-/merge_requests/29

Thanks!

cheers, josch

Bug#1063629: sysvinit-core.postinst: remove obsolete code path for /etc/inittab fixup on hurd in 2014

[Johannes Schauer Marin Rodrigues]

Source: sysvinit
Version: 3.08-6
Severity: normal
Tags: patch
User: debian-d...@lists.debian.org
Usertags: dpkg-root-support
X-Debbugs-Cc: debian-h...@lists.debian.org

Hi,

In 2014, code was added to to the sysvinit-core.postinst to fix up
/etc/inittab files on hurd which had /libexec/getty instead of
/sbin/getty and which were missing the following line:

c:23:respawn:/sbin/getty 38400 console

Also since 2014 (commit 6af0110c) ./debian/share/inittab.gnu is
installed on enduser systems which contains aforementioned fixes
already. The fixup code in sysvinit-core.postinst is thus long obsolete
as any systems that were affected should've long been upgraded and new
systems were installed using inittab.gnu which is not broken.

I prepared a merge request which removes the obsolete code path here:

https://salsa.debian.org/debian/sysvinit/-/merge_requests/12

Removing this old codepath helps with DPKG_ROOT because when creating
chrootless systems, uname -s output is potentially incorrect, for example when
preparing a hurd chroot on linux. See bug #1063624 for a similar instance of
maintainer scripts calling uname causing problems for creating chroots using
the DPKG_ROOT mechanism.

Thanks!

cheers, josch

Bug#1063624: libc.preinst: please skip kernel check via uname when DPKG_ROOT is not empty for gnu hurd support

[Johannes Schauer Marin Rodrigues]

Source: glibc
Version: 2.37-15
Severity: normal
Tags: patch
User: debian-h...@lists.debian.org
Usertags: hurd
X-Debbugs-Cc: debian-h...@lists.debian.org, debian-cr...@lists.debian.org

Hi,

one of the reasons for DPKG_ROOT support in packages close to the essential and
build-essential set is to build chroots for architectures that do not have
qemu-user support. Building chroots for hurd from linux is one such scenario
where qemu-user emulation will never allow running hurd binaries from the
chroot on the linux system creating that chroot and thus the only way to create
it (short of running the whole thing inside a full virtual machine) is to use
chrootless mode.

In our last round of DPKG_ROOT related patches we built chroots for other linux
architectures on linux. Now we try building chroots for foreign kernels. In
this case: hurd on linux. In the process we discovered another problem class:
maintainer scripts running uname. This is a problem because "uname -s" will
print "Linux" and thus the linux-specific parts of the maintainer script get
executed but we want to build a hurd chroot and need the hurd-specific bits
instead.

In case of libc.preinst, its use of "uname -s" and "uname -r" can be avoided by
adding another condition on $DPKG_ROOT being empty, i.e to only check the
kernel version for normal installations but not when glibc is installed with
dpkg --force-script-chrootless as in that case, there exists no way for the
script to know what kernel version will be running on the final system where
the chroot will be deployed. So if "$DPKG_ROOT" is not empty, the kernel
version checks just get skipped.

I submitted a patch as a merge request here:

https://salsa.debian.org/glibc-team/glibc/-/merge_requests/20

Please consider applying it to close this bug.

What do you think?

Thanks!

cheers, josch

Bug#1063501: building less from source fails the island test

[Johannes Schauer Marin Rodrigues]

Control: retitle -1 building less from source fails the island test

On Fri, 09 Feb 2024 00:24:00 +0100 Johannes Schauer Marin Rodrigues 
 wrote:
> the current curl packaging

The first sentence in my mail should've been

> the current less packaging

Apologies for the confusion!

cheers, josch

signature.asc
Description: signature

Bug#1063501: building curl from source fails the island test

[Johannes Schauer Marin Rodrigues]

Source: less
Version: 590-2
Severity: serious
Tags: patch

Hi,

the current curl packaging uses pre-built artifacts from the upstream
tarball without regenerating them. Attempting to regenerate them by
running "make -f Makefile.aut" proceeds to call curl to download stuff
from ftp://ftp.unicode.org. To fix this, I added a build dependency on
unicode-data and symlinked the relevant files in the source tree to the
files shipped by the unicode-data package. While I was at it, my patch
also regenerates all the other files which were so far just copypasted
from the upstream tarball without verifying whether they can really be
built using Debian main. This is the patch:

diff -Nru less-590/debian/control less-590/debian/control
--- less-590/debian/control 2023-03-12 15:49:03.0 +0100
+++ less-590/debian/control 2024-02-08 23:12:54.0 +0100
@@ -4,7 +4,8 @@
 Maintainer: Milan Kupcevic 
 Build-Depends:
  debhelper (>= 12),
- libncurses-dev
+ libncurses-dev,
+ unicode-data
 Standards-Version: 4.6.2
 Vcs-Git: https://salsa.debian.org/debian/less.git
 Vcs-Browser: https://salsa.debian.org/debian/less
diff -Nru less-590/debian/rules less-590/debian/rules
--- less-590/debian/rules   2023-02-12 11:17:35.0 +0100
+++ less-590/debian/rules   2024-02-08 23:16:58.0 +0100
@@ -12,3 +12,20 @@
dh_auto_configure -- \
  --with-regex=gnu \
  --with-editor=/usr/bin/editor
+
+execute_before_dh_auto_build:
+   mkdir -p unicode
+   ln -s /usr/share/unicode/UnicodeData.txt unicode/UnicodeData.txt
+   ln -s /usr/share/unicode/EastAsianWidth.txt unicode/EastAsianWidth.txt
+   make -f Makefile.aut
+
+execute_before_dh_auto_clean:
+   set -e; for t in "" echo key; do mv "less$$t.nro" "less$$t.bak"; done
+   make -f Makefile.aut clean
+   rm -f *.nro *.man help.c funcs.h defines.h.in configure
+   rm -f unicode/UnicodeData.txt unicode/EastAsianWidth.txt
+   [ ! -d unicode ] || rmdir unicode
+   set -e; for t in "" echo key; do mv "less$$t.bak" "less$$t.nro"; touch 
"less$$t.nro.VER" "less$$t.nro"; done
+
+execute_before_dh_auto_install:
+   make -f Makefile.aut distfiles


The stunt with preserving the *.nro files is necessary because the upstream
tarball does not ship the *.nro.VER files which are then made into *.nro files
by replacing @@VERSION@@ and @@DATE@@ with their respective values. Technically
this is a case where the original source is missing from the Debian tarball but
this replacement is probably trivial enough to not be a DFSG violation.

The patch could be made much simpler if you were using a tarball from the
upstream git instead of the distribution tarball which is missing sources but
you probably have your reasons for doing it this way.

Thanks!

cheers, josch

Bug#1063337: navit FTCBFS: uses CMAKE_TOOLCHAIN_FILE=/etc/dpkg-cross/cmake/CMakeCross.txt

[Johannes Schauer Marin Rodrigues]

Source: navit
Version: 0.5.6+dfsg.1-2
Severity: normal
Tags: patch
User: debian-cr...@lists.debian.org
Usertags: ftcbfs
X-Debbugs-Cc: debian-cr...@lists.debian.org

Hi,

TLDR; navit fails to cross build from source because it sets
CMAKE_TOOLCHAIN_FILE=/etc/dpkg-cross/cmake/CMakeCross.txt in
debian/rules. Navit cross builds just fine after deleting it. Trivial
patch:

--- a/debian/rules
+++ b/debian/rules
@@ -14,7 +14,6 @@ CMAKEFLAGS = -DCMAKE_BUILD_TYPE=debian
 ifneq ($(DEB_HOST_MULTIARCH),$(DEB_BUILD_MULTIARCH))
 export CC=$(DEB_HOST_MULTIARCH)-gcc
 export CXX=$(DEB_HOST_MULTIARCH)-g++
-export CMAKE_TOOLCHAIN_FILE=/etc/dpkg-cross/cmake/CMakeCross.txt
 CMAKEFLAGS += -DCMAKE_LIBRARY_PATH="/usr/$(DEB_HOST_MULTIARCH)/lib" 
-DCMAKE_REQUIRED_INCLUDES="/usr/$(DEB_HOST_MULTIARCH)/include;/usr/$(DEB_HOST_MULTIARCH)/include/$(DEB_HOST_MULTIARCH)"
 \
-DCMAKE_C_FLAGS_DEBIAN="$(shell dpkg-buildflags --get CFLAGS) $(shell 
dpkg-buildflags --get CPPFLAGS) -I/usr/$(DEB_HOST_MULTIARCH)/include 
-I/usr/$(DEB_HOST_MULTIARCH)/include/$(DEB_HOST_MULTIARCH)" \
-DCMAKE_CXX_FLAGS_DEBIAN="$(shell dpkg-buildflags --get CXXFLAGS) 
$(shell dpkg-buildflags --get CPPFLAGS) -I/usr/$(DEB_HOST_MULTIARCH)/include 
-I/usr/$(DEB_HOST_MULTIARCH)/include/$(DEB_HOST_MULTIARCH)"


Long story:

Helmut Grohne discovered that navit FTCBFS:

http://crossqa.subdivi.de/build/navit_0.5.5%2Bdfsg.1-2_i386_20211022094850.log
http://crossqa.subdivi.de/build/navit_0.5.6%2Bdfsg.1-2_arm64_20231230114317.log

Symptoms:

-- Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR) 
-- Could NOT find Freetype (missing: FREETYPE_LIBRARY FREETYPE_INCLUDE_DIRS) 

But the respective packages are installed, so CMake should be able to find it.
Why does it not?

Here is a minimal reproducer of the problem. I'm on an arm64 machine (build
architecture) and am attempting to cross compile for amd64 (host architecture):

mkdir -p src/build
cat << END > "src/CMakeLists.txt"
cmake_minimum_required(VERSION 3.2)
find_package(ZLIB REQUIRED)
END
env --chdir=src/build \
CMAKE_TOOLCHAIN_FILE='/etc/dpkg-cross/cmake/CMakeCross.txt' \
CONFIG_SITE='/etc/dpkg-cross/cross-config.amd64' \
CXX='x86_64-linux-gnu-g++' \
DEB_HOST_GNU_TYPE='x86_64-linux-gnu' \
CC='x86_64-linux-gnu-gcc' \
cmake "-GUnix Makefiles" \
-DCMAKE_C_COMPILER=x86_64-linux-gnu-gcc \
-DCMAKE_CXX_COMPILER=x86_64-linux-gnu-g\+\+ \
-DCMAKE_SYSTEM_PROCESSOR=x86_64 \
-DPKG_CONFIG_EXECUTABLE=/usr/bin/x86_64-linux-gnu-pkg-config \
-DPKGCONFIG_EXECUTABLE=/usr/bin/x86_64-linux-gnu-pkg-config \
-DCMAKE_LIBRARY_PATH=/usr/x86_64-linux-gnu/lib \

-DCMAKE_REQUIRED_INCLUDES=/usr/x86_64-linux-gnu/include\;/usr/x86_64-linux-gnu/include/x86_64-linux-gnu
 \
..

I put this into a script and threw it at debbisect:

debbisect --arch=arm64,amd64 
--depends=cmake,build-essential,crossbuild-essential-amd64,zlib1g-dev:amd64 
--cache="$(pwd)/cache" 2020-01-01 2024-01-01 ./test.sh
[...]
bisection finished successfully
  last good timestamp: 20210915T092301Z
  first bad timestamp: 20210915T150047Z
the following packages differ between the last good and first bad timestamp:
  cmake 3.18.4-2 -> 3.21.2-1
  cmake-data 3.18.4-2 -> 3.21.2-1
  libjs-underscore 1.9.1~dfsg-3 -> 1.9.1~dfsg-4
test upgrading cmake 3.18.4-2 -> 3.21.2-1...
using cached result (was bad) from debbisect.20210915T092301Z.cmake.log.bad
  upgrading cmake triggered the problem
  additional packages that got upgraded/installed at the same time:
cmake-data 3.18.4-2 -> 3.21.2-1
test upgrading cmake-data 3.18.4-2 -> 3.21.2-1...
using cached result (was bad) from debbisect.20210915T092301Z.cmake-data.log.bad
  upgrading cmake-data triggered the problem
  additional packages that got upgraded/installed at the same time:
cmake 3.18.4-2 -> 3.21.2-1
test upgrading libjs-underscore 1.9.1~dfsg-3 -> 1.9.1~dfsg-4...
  upgrading libjs-underscore does not cause the problem

The upstream git diff between v3.18.4 and v3.21.2 is 364800 lines long, so lets
bisect cmake git. Luckily, the Debian packaging of these two upstream versions
is near identical, so we can just bisect upstream git, create a new source
tarball for every commit we test, build a Debian package from that and then
test that package inside the test environment from above. Both building the
package as well as running the test is done inside a Debian chroot from
snapshot timestamp 20210915T092301Z:

mmdebstrap --variant=buildd --aptopt='Acquire::Check-Valid-Until "false"' 
unstable \
~/.cache/sbuild/unstable-bisect-arm64.tar 
"http://127.0.0.1:8000/archive/debian/20210915T092301Z;

The following script can be passed to "git bisect run" to find the offending
version:

git archive --format=tar --prefix=cmake/ HEAD | gzip > 
../cmake_3.21.2.orig.tar.gz
cp -a /tmp/cmake_3.21.2-1+bisect.debian debian
DEB_BUILD_OPTIONS=nocheck sbuild --verbose -d unstable-bisect --no-run-lintian
mmdebstrap 

Bug#1061820: (no subject)

[Johannes Schauer Marin Rodrigues]

Hi,

On 2024-01-31 09:15, Ken Sharp wrote:
Do I need to submit the patch somewhere else or is this the correct 
place?


the bts is the correct place but debootstrap is maintained by unpaid 
volunteers like many other open source projects. So getting your patch 
merged depends on somebody finding some time to do so.


I have created a merge request to make application of your patch a bit 
easier for the debootstrap maintainers:


https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/111

Thanks!

cheers, josch

Bug#1061960: systemd-binfmt.service should get started when something gets added into a previously empty /usr/lib/binfmt.d

[Johannes Schauer Marin Rodrigues]

Package: systemd
Version: 252.19-1~deb12u1
Severity: normal

Hi,

steps to reproduce:

 $ debvm-create
 [...]
 $ debvm-run
 [...]
 root@testvm:~# systemctl status systemd-binfmt.service
 ○ systemd-binfmt.service - Set Up Additional Binary Formats
  Loaded: loaded (/usr/lib/systemd/system/systemd-binfmt.service; static)
  Active: inactive (dead)
   Condition: start condition unmet at Tue 2024-01-30 13:44:24 UTC; 25s ago
  ├─ ConditionDirectoryNotEmpty=|/lib/binfmt.d was not met
  ├─ ConditionDirectoryNotEmpty=|/usr/lib/binfmt.d was not met
  ├─ ConditionDirectoryNotEmpty=|/usr/local/lib/binfmt.d was not met
  ├─ ConditionDirectoryNotEmpty=|/etc/binfmt.d was not met
  └─ ConditionDirectoryNotEmpty=|/run/binfmt.d was not met
 root@testvm:~# apt install qemu-user-static
 [...]
 root@testvm:~# systemctl status systemd-binfmt.service
 ○ systemd-binfmt.service - Set Up Additional Binary Formats
  Loaded: loaded (/usr/lib/systemd/system/systemd-binfmt.service; static)
  Active: inactive (dead)
   Condition: start condition unmet at Tue 2024-01-30 13:44:24 UTC; 2min 47s ago
  ├─ ConditionDirectoryNotEmpty=|/lib/binfmt.d was not met
  ├─ ConditionDirectoryNotEmpty=|/usr/lib/binfmt.d was not met
  ├─ ConditionDirectoryNotEmpty=|/usr/local/lib/binfmt.d was not met
  ├─ ConditionDirectoryNotEmpty=|/etc/binfmt.d was not met
  └─ ConditionDirectoryNotEmpty=|/run/binfmt.d was not met


As a result, just installing a binfmt provider will not immediately enable it.

Starting or restarting makes it work as tested by Jochen Sprickerhof

Maybe the binfmt.d trigger should restart systemd-binfmt.service so that
enabling a new binfmt works without rebooting the machine.

Thanks!

cheers, josch

Bug#1061634: mmdebstrap: possible mmdebstrap-autopkgtest-build-qemu improvements

[Johannes Schauer Marin Rodrigues]

Quoting Francesco Poli (2024-01-29 20:54:25)
> On Mon, 29 Jan 2024 10:21:29 +0100 Johannes Schauer Marin Rodrigues wrote:
> 
> [...]
> > With that said, I think mmdebstrap-autopkgtest-build-qemu should keep 
> > using raw images.
> 
> OK, after reading these opinions, I can agree that it makes sense to go
> on with raw images.  Thank you so much for taking the time to consult the
> experts!

There is actually one good argument to go with qcow2: That is the default
format that autopkgtest-build-qemu uses. And since
mmdebstrap-autopkgtest-build-qemu tries to be like autopkgtest-build-qemu it
might make sense to choose the same output format...

But then I also thought: if the output format is raw, the user can always
easily run qemu-img afterwards if they want qcow2. The user can then even
decide for themselves if and what kind of compression or other features they
want. I think it is suboptimal that autopkgtest-build-qemu doesn not make this
optional and just enforces the step that converts to qcow2, eating up runtime
and cpu cycles even if the user doesn't require qcow2.

Since this question will come up again in the future, I added this list to the
code:

# The image is raw and not in qcow2 format because:
#  - faster run-time as the "qemu-image convert" step is not needed
#  - image can be used independent of qemu tooling
#  - modifying the image just with "mount" instead of requiring qemu-nbd
#  - sparse images make the file just as small as with qcow2
#  - trim support is more difficult on qcow2
#  - snapshots and overlays work just as well with raw images
#  - users who prefer qcow2 get to choose to run it themselves with their own
#custom options like compression

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061634: mmdebstrap: possible mmdebstrap-autopkgtest-build-qemu improvements

[Johannes Schauer Marin Rodrigues]

Hi,

On 2024-01-29 00:07, Francesco Poli wrote:

I was under the impression that the qcow2 format was the recommended
one for QEMU/KVM virtual machine images. At least, qemu-img(1)
describes it as "the most versatile format"...

Anyway, if you think that the raw format is a better choice for
autopkgtest and sbuild VM images, I take your word for it.


as I said in my other mail, I'm not the expert on the topic, so I asked 
some experts. :)


In the other mail I already paraphrased what f_g said. I now also got 
feedback from mjt (Michael Tokarev) our QEMU maintainer in Debian:


09:10 < mjt> for me, snapshots in qcow2 isn't of much use (I usually do 
it the other way, by stacking another qcow2 on

 top of the main image)
09:12 < jochensp> mjt: afair stacking only works if the base image is 
already qcow2, or?
09:12 < mjt> speaking of qcow2 vs raw, both has their own good and bad 
sides, mostly minor, and the preference is more
 about taste than actual technical differences.  People tend 
to forget about sparseness of a raw image,
 becoming surprizing the drivee usage increases dramatically 
after a copy

09:13 < mjt> jochensp: no, stacking works on top of any image format
09:14 < mjt> ..on the other hand, trim support is more difficult on 
qcow2
09:14 < mjt> myself, I choose raw for regular work and qcow2 when I have 
to transfer the image somewhere
09:15 < mjt> it's quite easy to mount a qcow2 image on the host too, but 
this needs extra layer (I usually use

 qemu-nbd for this)
09:17 < mjt> for direct manipulation, when you have libext2fs and the 
tools, raw is the only way to go


With that said, I think mmdebstrap-autopkgtest-build-qemu should keep 
using raw images.


Thanks!

cheers, josch

Bug#1061636: mmdebstrap-autopkgtest-build-qemu VM image cannot be updated with sbuild-qemu-update

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-01-29 00:12:27)
> Maybe this bug report should cloned and reassigned to sbuild-qemu?

if you like (for example because you want to subscribe to it and notified when
it gets closed) then feel free to clone it. Personally, as long as the issue is
tracked some place obvious (and that includes a salsa MR) I'm fine with it as I
just want to make sure that the issue is not getting lost. Whether it gets
tracked in the debian bts or as a salsa MR dosen't matter to me. But as i said,
if for you, a bts issue makes more sense, feel free to clone.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061634: mmdebstrap: possible mmdebstrap-autopkgtest-build-qemu improvements

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-01-29 00:07:08)
> What's *your* umask? Is it Debian default (022), by chance?

yes.

> This seems to work with Debian default umask (022):
> 
>   $ printf '%o\n' "$(( 0666 - 00022 ))"
>   644
> 
> but fails whenever a umask includes an octal digit equal to 7, due to
> the carryover:
> 
>   $ printf '%o\n' "$(( 0666 - 7 ))"
>   657
> 
> Shouldn't this use the bitwise AND combined with the bitwise NOT?
> 
>   $ umask
>   0007
>   $ printf '%o\n' "$(( 0666 & ~0$(umask) ))"
>   660
>   $ umask 022
>   $ umask
>   0022
>   $ printf '%o\n' "$(( 0666 & ~0$(umask) ))"
>   644
> 
> I would think that mmdebstrap-autopkgtest-build-qemu should run:
> 
>   chmod "$(printf %o "$(( 0666 & ~0$(umask) ))")" "$IMAGE"
> 
> Does it make sense to you?

I now have this locally. Is this attribution (name/email) correct?

>From 8410dc6636817c4e02cf9eba090a70051c494c48 Mon Sep 17 00:00:00 2001
From: Francesco Poli 
Date: Mon, 29 Jan 2024 08:28:53 +0100
Subject: [PATCH] mmdebstrap-autopkgtest-build-qemu: fix octal mode computation

---
 mmdebstrap-autopkgtest-build-qemu | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mmdebstrap-autopkgtest-build-qemu 
b/mmdebstrap-autopkgtest-build-qemu
index 5684cbe..1ed14db 100755
--- a/mmdebstrap-autopkgtest-build-qemu
+++ b/mmdebstrap-autopkgtest-build-qemu
@@ -357,7 +357,7 @@ echo "mmdebstrap $*"
 mmdebstrap "$@" || die "mmdebstrap failed"
 
 unshare -U -r --map-groups=auto chown 0:0 "$IMAGE"
-chmod "$(printf %o "$(( 0666 - 0$(umask) ))")" "$IMAGE"
+chmod "$(printf %o "$(( 0666 & ~0$(umask) ))")" "$IMAGE"
 
 echo "root=LABEL=autopkgtestvm rw console=ttyS0" > "$WORKDIR/cmdline"
 
-- 
2.39.2

> > Can you list some reasons why qcow2 should be preferred over
> > raw images?
> 
> I was under the impression that the qcow2 format was the recommended
> one for QEMU/KVM virtual machine images. At least, qemu-img(1)
> describes it as "the most versatile format"...
> 
> Anyway, if you think that the raw format is a better choice for
> autopkgtest and sbuild VM images, I take your word for it.

I'm asking because I'm not the expert. :)

I asked in #debian-devel and f_g (Fabian Grünbichler) commented that the main
reasoning for choosing qcow over raw images is snapshot, link clone and backing
image support.

I don't think these features are useful for users of autopkgtest or sbuild with
these images. Or do you see a use for them in this area?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061687: u-boot-menu: Prevents kernel installation/removal when removed without purging

[Johannes Schauer Marin Rodrigues]

Hi,

On Sun, 28 Jan 2024 19:00:25 +0100 Tomasz Rybak  wrote:
> I'm running MNT Reform, so using non-Debian kernel. The rest of packages are
> from Debian.

hello, fellow Reform user. :)

> I'm not sure which solution would be better:
> a) Move required file from /usr/share to /etc (e.g. /etc/default?)
> b) leave it in /usr/share but mark as conffile.

The correct solution is

  c) check whether u-boot-menu was uninstalled before running this script and 
exit 0 otherwise

I implemented this here: 
https://salsa.debian.org/debian/u-boot-menu/-/merge_requests/15

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061706: ITP: distrobuilder -- System container image builder for LXC and Incus

[Johannes Schauer Marin Rodrigues]

Hi,

On Sun, 28 Jan 2024 20:29:30 + Mathias Gibbens  wrote:
>  distrobuilder is an image building tool for LXC and Incus.

independent on whether this is packaged or not, could you complete the entry
for distrobuilder in this table?

https://wiki.debian.org/SystemBuildTools#General_tools

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061636: mmdebstrap-autopkgtest-build-qemu VM image cannot be updated with sbuild-qemu-update

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Johannes Schauer Marin Rodrigues (2024-01-28 00:46:10)
> Quoting Francesco Poli (wintermute) (2024-01-27 19:44:15)
> > Is there any special tweak or configuration needed to use
> > sbuild-qemu-update with mmdebstrap-autopkgtest-build-qemu VM images?  Where
> > is this documented?
> > 
> > Otherwise, if this is an actual bug, please fix
> > mmdebstrap-autopkgtest-build-qemu
> 
> The images you create with mmdebstrap-autopkgtest-build-qemu require efi. But
> sbuild-qemu does pass the --boot parameter to autopkgtest-virt-qemu. If you 
> are
> on amd64 and if your image is for amd64, then autopkgtest-virt-qemu will
> default to bios boot which is incompatible with the images created by
> mmdebstrap-autopkgtest-build-qemu. You could add a dirty local hack to
> sbuild-qemu and let it add --boot=efi via --autopkgtest-virt-server-opt to
> see if this is it. If that is the problem, please re-assign this bug to
> sbuild.

you found bugs in both sbuild as well as in mmdebstrap. The fixes for sbuild
are in this MR:

https://salsa.debian.org/debian/sbuild/-/merge_requests/54

Feel free to comment and improve on the code as well to get this forward.

The mmdebstrap bug is fixed here:


https://gitlab.mister-muffin.de/josch/mmdebstrap/commit/3e233e10dfe414e43b31d328eecb0c776afc2ec3

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061636: mmdebstrap-autopkgtest-build-qemu VM image cannot be updated with sbuild-qemu-update

[Johannes Schauer Marin Rodrigues]

Quoting Francesco Poli (wintermute) (2024-01-27 19:44:15)
> As I said in bug report [#1061634], I've been able to create a QEMU/KVM
> virtual machine image for autopkgtests:
> 
>   $ mkdir -p ~/var/cache/sbuild/
>   $ cd /dev/shm
>   $ TMPDIR=/dev/shm mmdebstrap-autopkgtest-build-qemu \
> --size=25G --boot=efi sid sid-amd64.img
>   $ chmod 660 sid-amd64.img
>   $ mv -i sid-amd64.img ~/var/cache/sbuild/
> 
> [#1061634]: 
> 
> It works for autopkgtests, but I wanted to use the same virtual machine
> image to build Debian packages with sbuild-qemu.
> The first thing I tested is the update of the VM image with
> sbuild-qemu-update:
> 
>   $ sbuild-qemu-update --arch=amd64 sid-amd64.img
>   qemu-system-x86_64 -enable-kvm -object 
> rng-random,filename=/dev/urandom,id=rng0 -device 
> virtio-rng-pci,rng=rng0,id=rng-device0 -device virtio-serial -nic 
> user,model=virtio -m 1024 -smp 1 -nographic sid-amd64.img
> 
> This command does not seem to work: it uses 100 % of one CPU core and
> seemingly does nothing, until I interrupt it by pressing [Ctrl+C].
> 
> Is there any special tweak or configuration needed to use
> sbuild-qemu-update with mmdebstrap-autopkgtest-build-qemu VM images?
> Where is this documented?
> 
> Otherwise, if this is an actual bug, please fix
> mmdebstrap-autopkgtest-build-qemu

The images you create with mmdebstrap-autopkgtest-build-qemu require efi. But
sbuild-qemu does pass the --boot parameter to autopkgtest-virt-qemu. If you are
on amd64 and if your image is for amd64, then autopkgtest-virt-qemu will
default to bios boot which is incompatible with the images created by
mmdebstrap-autopkgtest-build-qemu. You could add a dirty local hack to
sbuild-qemu and let it add --boot=efi via --autopkgtest-virt-server-opt to see
if this is it. If that is the problem, please re-assign this bug to sbuild.

> or help sbuild-qemu developers to fix sbuild-qemu-update.

Did you look up who the sbuild developers are? ;)

> Thanks for your time and dedication!

Thank you for your bugs!! <3

cheers, josch

signature.asc
Description: signature

Bug#1061634: mmdebstrap: possible mmdebstrap-autopkgtest-build-qemu improvements

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (wintermute) (2024-01-27 19:20:08)
> I've been able to create a QEMU/KVM virtual machine image for autopkgtests:
> 
>   $ mkdir -p ~/var/cache/sbuild/
>   $ cd /dev/shm
>   $ TMPDIR=/dev/shm mmdebstrap-autopkgtest-build-qemu \
> --size=25G --boot=efi sid sid-amd64.img
>   $ chmod 660 sid-amd64.img
>   $ mv -i sid-amd64.img ~/var/cache/sbuild/
> 
> It is able to run autopkgtests:
> 
>   $ autopkgtest --output-dir ./${SRCPKG}_autopkgtest.out \
> --summary./${SRCPKG}_autopkgtest.summary \
> --apt-upgrade -B ./${SRCPKG}_amd64.changes -- \
> qemu --boot=efi --overlay-dir /dev/shm \
> ~/var/cache/sbuild/sid-amd64.img
> 
> My comments / suggestions for improvements:
> 
>   * I had to manually fix the permissions, since the image file was
> originally created with the following weird ones:
> 
>   $ ls -l --si sid-amd64.img 
>   -rw-r-xrwx 1 $USER $USER 27G Jan 27 17:48 sid-amd64.img

those are some funny permission bits. When I run
mmdebstrap-autopkgtest-build-qemu I get:

$ ls -lha disk.img
-rw-r--r-- 1 josch josch 26G Jan 28 00:03 disk.img

Looks fine to me.

> I think the file permissions should be set (possibly at the end of the
> mmdebstrap-autopkgtest-build-qemu run) as they would "naturally" result
> from the user's umask:
> 
>   $ cd /dev/shm
>   $ touch foo
>   $ ls -l --si foo 
>   -rw-rw 1 $USER $USER 0 Jan 27 19:00 foo

They are set, taking the user's umask into account. What is your umask?

> That's why I manually changed the permissions at the end:
> 
>   $ chmod 660 sid-amd64.img
>   $ ls -l --si sid_amd64.img
>   -rw-rw 1 $USER $USER 27G Jan 27 17:48 sid-amd64.img
> 
> I suggest that mmdebstrap-autopkgtest-build-qemu applies this fix
> automatically.

Lets first understand the problem before adding a workaround.
mmdebstrap-autopkgtest-build-qemu runs this command to set the permissions:

chmod "$(printf %o "$(( 0666 - 0$(umask) ))")" "$IMAGE"

Could you add some debugging output to the script to figure out what went wrong
and where?

>   * I was surprised to see a 25 GiB image fit into a 7.7 GiB filesystem:
> 
>   $ cd /dev/shm
>   $ ls -l -h sid-amd64.img
>   -rw-rw 1 $USER $USER 26G Jan 27 17:48 sid-amd64.img
>   $ df -h .
>   Filesystem  Size  Used Avail Use% Mounted on
>   tmpfs   7.7G  765M  7.0G  10% /dev/shm
> 
> Well, the mystery is solved by looking at the allocated size:
> 
>   $ ls -l -h -s sid-amd64.img 
>   662M -rw-rw 1 $USER $USER 26G Jan 27 17:48 sid-amd64.img
> 
> Would it be less confusing, if mmdebstrap-autopkgtest-build-qemu created
> .qcow2 images?

I don't understand why sparse files are confusing to you. Why would qcow images
be less confusing? Can you list some reasons why qcow2 should be preferred over
raw images? Compression comes to mind but that is at the cost of doing that
compression in the first place and it is not obvious to me whether cpu cycles
or disk space are more scarce. For example, my platform is an ARM Cortex A53.
To give you some idea how slow it is: it takes 30 seconds before a youtube
video starts playing. At the same time, the system has a 2 TB hard disk. I much
prefer a bigger disk image which needs less CPU when I want to create and use
it. What is your use-case?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060961: clapper: FTBFS: make: *** [debian/rules:6: binary] Error 25

[Johannes Schauer Marin Rodrigues]

Hi,

On Tue, 16 Jan 2024 20:43:11 +0100 Lucas Nussbaum  wrote:
> During a rebuild of all packages in sid, your package failed to build on
> amd64.

thank you for your bug!

> Couldn't find include 'GObject-2.0.gir' (search path: '['/usr/share/gir-1.0', 
> '/usr/lib/x86_64-linux-gnu/gir-1.0', 
> '/<>/debian/.debhelper/generated/_source/home/.local/share/gir-1.0',
>  'gir-1.0', '/usr/local/share/gir-1.0', '/usr/share/gir-1.0', 
> '/usr/lib/x86_64-linux-gnu/gir-1.0', '/usr/share/gir-1.0', 
> '/usr/share/gir-1.0']')

I'm unable to reproduce your bug in unstable. The day before the date from your
build log, a disruptive upload of gobject-introspection happened to unstable
and there have been more uploads of gobject-introspection since then. Maybe the
issue is fixed now?

Would you mind re-trying the build? I am building the package on arm64 and
maybe that makes a difference? In any case, I'm not able to reproduce it.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi Francesco,

Quoting Johannes Schauer Marin Rodrigues (2024-01-23 08:56:24)
> I'm planning to close this bug with this patch:
> 
> https://paste.debian.net/hidden/844859ff/
> 
> Amongst other improvements, if "$IMAGE" cannot be accessed by the unshared
> user, it will error out early with a (hopefully) helpful error message.

this has now been uploaded as mmdebstrap 1.4.1. Are you satisfied with the
resolution? Do you have other remarks? I'm going to make another bugfix release
very soon, so if you like some more changes to be included, now is the time to
speak up. :)

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Control: tag -1 + pending

Hi,

I'm planning to close this bug with this patch:

https://paste.debian.net/hidden/844859ff/

Amongst other improvements, if "$IMAGE" cannot be accessed by the unshared
user, it will error out early with a (hopefully) helpful error message.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1061327: libimath-dev: please drop dependency on python3-imath

[Johannes Schauer Marin Rodrigues]

Package: libimath-dev
Version: 3.1.9-3
Severity: normal
Tags: patch
User: debian-cr...@lists.debian.org
Usertags: cross-satisfiability

Hi,

five months ago, bug #1050349 was reported due to a missing dependency of
libimath-dev on python3-imath. The bug was resolved by adding that dependency.
Choosing this solution resulted in the following 134 source packages gaining
one additional cross build satisfiability problem because they will attempt to
install the host architecture version of python:

actiona amberol auto-multiple-choice beads blender budgie-control-center
calligra chafa cheese cimg converseen darknet darktable digikam dmtx-utils
drawtiming dvdauthor dx embree enblend-enfuse eviacam exactimage ffcv field3d
freecad freeimage frei0r gegl gem gimp gmic gnome-authenticator
gnome-dvb-daemon gnome-metronome gnome-sound-recorder gnome-subtitles gnunet
gnustep-gui gst-plugins-bad1.0 gstreamer-editing-services1.0 gstreamer-vaapi
gst-rtsp-server1.0 gtk4 hugin imagemagick imview inkscape jmagick jpeg-xl
kimageformats kio-extras krita kxstitch kylin-scanner lebiniou libopenshot
libvigraimpex luminance-hdr megapixels mgba mia mldemos mlt monado mrgingham
mrpt mupen64plus-core nheko nip2 nitrokey-authenticator node-opencv nomacs
nvidia-texture-tools nvidia-vaapi-driver obs-advanced-scene-switcher odr-padenc
olive-editor opencfu opencolorio opencv openexr openexr-viewers openimageio
openvdb os-autoinst osm2pgsql otb otpclient performous pfstools photoqt
php-facedetect php-imagick pink-pony pitivi povray pqiv pragha pstoedit
pulseeffects pythonmagick pytorch pyzbar qimgv rails r-cran-magick
ros-image-pipeline ros-image-transport-plugins ros-opencv-apps
ros-vision-opencv rss-glx ruby-image-processing ruby-rmagick ruby-vips
rust-gstreamer-play rust-gstreamer-play-sys rust-zbar-rust sayonara sight
simple-whip-client siril slic3r-prusa swayimg synfig synfigstudio
ukui-biometric-auth ukui-biometric-manager ukui-control-center ukui-greeter
ukui-screensaver uprightdiff vdr-plugin-skinenigmang vips virtuoso-opensource

To resolve the situation I approached imath upstream in this issue:

https://github.com/AcademySoftwareFoundation/Imath/issues/360

The resulting pull request was since merged into main:

https://github.com/AcademySoftwareFoundation/Imath/pull/361

In this bug I'm asking you to cherry-pick that commit from the upstream main
branch and apply it to the package even before the next upstream release with
that commit in it happens. I attached a debdiff that is doing exactly that for
your convenience.

Doing so will greatly help with maintaining a downstream Debian Pure Blends
that I am maintaining for the MNT Reform open source laptop.  For gstreamer to
do the right thing, we are backporting some patches from gstreamer upstream but
we need to cross-compile it because the MNT Reform is an arm64 laptop and our
build machine is a normal Intel box. With this patch applied, build times get
reduced from 1.5 hours to just under 4 minutes. This is a tremendous help when
developing for that platform as you can probably imagine (and helps reducing
our carbon emissions by a tiny bit).

Note also, that this bug is distinct from #1060843. That bug is about imath
itself failing to cross-build. This bug is about packages that need imath
failing to cross-build.

Thank you for considering!

cheers, josch
diff -Nru imath-3.1.9/debian/changelog imath-3.1.9/debian/changelog
--- imath-3.1.9/debian/changelog2023-09-04 14:11:55.0 +0200
+++ imath-3.1.9/debian/changelog2024-01-22 17:54:01.0 +0100
@@ -1,3 +1,12 @@
+imath (3.1.9-3.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * cherry-pick upstream commit to prevent reverse B-D of libimath-dev
+from requiring python3-imath being installed
+  * libimath-dev: drop dependency on python3-imath (Closes: #-1)
+
+ -- Johannes Schauer Marin Rodrigues   Mon, 22 Jan 2024 
17:54:01 +0100
+
 imath (3.1.9-3) unstable; urgency=medium
 
   * debian/control: add python3-imath as dep to -dev
diff -Nru imath-3.1.9/debian/control imath-3.1.9/debian/control
--- imath-3.1.9/debian/control  2023-09-04 14:09:31.0 +0200
+++ imath-3.1.9/debian/control  2024-01-22 17:54:01.0 +0100
@@ -47,7 +47,6 @@
 Multi-Arch: same
 Depends:
  libimath-3-1-29 (= ${binary:Version}),
- python3-imath,
  ${misc:Depends},
  ${shlibs:Depends}
 Pre-Depends:
diff -Nru 
imath-3.1.9/debian/patches/0001-src-python-config-ModuleDefine.cmake-do-not-install-.patch
 
imath-3.1.9/debian/patches/0001-src-python-config-ModuleDefine.cmake-do-not-install-.patch
--- 
imath-3.1.9/debian/patches/0001-src-python-config-ModuleDefine.cmake-do-not-install-.patch
  1970-01-01 01:00:00.0 +0100
+++ 
imath-3.1.9/debian/patches/0001-src-python-config-ModuleDefine.cmake-do-not-install-.patch
  2024-01-22 17:48:31.0 +0100
@@ -0,0 +1,29 @@
+From 7d58bb722faccd21d336cd0de52f611bac00a9c5 Mon Sep 17 00:00:00 2001
+From: josch 
+Date: Mon, 22 Jan 2024 00:55:39 +
+Subject

Bug#1061287: libllvmspirvlib-15-dev is needed even with rusticl disabled

[Johannes Schauer Marin Rodrigues]

Source: mesa
Version: 24.0.0~rc2-1.1
Severity: normal

Hi,

I'm trying to build mesa 24 from experimental on bookworm. Since rusticl
requires rust-bindgen 0.66 and since building that and the surrounding
rust packages on bookworm is infeasible, I decided to disable rusticl by
excluding my host architecture from RUSTICL_ARCHS in debian/rules.
Unfortunately it seems, that libllvmspirvlib-15-dev which is pulled in
only when rusticl is enabled, is also necessary without rusticl:

Determining dependency 'LLVMSPIRVLib' with pkg-config executable 
'/usr/bin/pkg-config'
env[PKG_CONFIG_PATH]:█
env[PKG_CONFIG]: /usr/bin/pkg-config
---
Called: `/usr/bin/pkg-config --modversion LLVMSPIRVLib` -> 1
stderr:
Package LLVMSPIRVLib was not found in the pkg-config search path.
Perhaps you should add the directory containing `LLVMSPIRVLib.pc'
to the PKG_CONFIG_PATH environment variable
Package 'LLVMSPIRVLib', required by 'virtual:world', not found
---
CMake binary for host machine is cached as not found
Dependency lookup for LLVMSPIRVLib with method 'cmake' failed: CMake binary 
for machine host machine not found.Giving up.
Run-time dependency llvmspirvlib found: NO (tried pkgconfig)

../meson.build:1844:21: ERROR: Dependency "LLVMSPIRVLib" not found, tried 
pkgconfig


Reading meson.build, with_opencl_spirv seems to be set when opencl is not
disabled and thus does not only depend on rusticl. Could you change
debian/control.in such that the dependency on libllvmspirvlib-15-dev is not
only emitted when rusticl is enabled but also otherwise?

Let me also quickly send you two missing minimum version restrictions I found:

../meson.build:831:4: ERROR: Problem encountered: rusticl requires meson 
1.3.1 or newer

and:

Message: libdrm 2.4.119 needed because amdgpu has the highest requirement
Dependency libdrm_amdgpu found: NO found 2.4.114 but need: '>=2.4.119'
Run-time dependency libdrm_amdgpu found: NO█

../meson.build:1674:6: ERROR: Dependency lookup for libdrm_amdgpu with 
method 'pkgconfig' failed: Invalid version, need 'libdrm_amdgpu' 
['>=2.4.119'] found '2.4.114'.

Thanks!

cheers, josch

Bug#1061111: RFS: dpkg-buildenv/1.0.0 [ITP] -- Builds debian packages in a docker container.

[Johannes Schauer Marin Rodrigues]

Hi Aidan,

On Thu, 18 Jan 2024 16:52:26 +0100 Andrey Rakhmatullin  wrote:
> I see you added this tool to the list of similar tools on the wiki so you
> at least know about that list. So how is your tool better than other tools
> on that list, or at least than the ones packaged in Debian? 
> Please also note that if you followed the procedure outlined at
> https://mentors.debian.net/intro-maintainers/ and filed an ITP bug before
> doing the packaging this discussion would happen there, not on the RFS bug.

I'm the current maintainer of sbuild. You found the wiki page and you saw that
there already exist 10 different implementations of utilities that build Debian
packages inside a docker container. You still decided to add an eleventh
implementation so I guess my plea here will not amount to much but anyways here
goes my sales pitch:

If you want your code to be in Debian, please consider reviewing the patch
attached to this bug report:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867176

I'm not a docker user, so I cannot verify whether this is indeed doing the
right thing but you are, so it should be easy for you to take it, test it and
finalize it.

I'm afraid contributing to an existing tool is not as sexy as supplying one
that is 100% written by you but I can guarantee you that if you can clean up
that patch, I will apply it to sbuild and your code will be in Debian proper.

Maybe, hopefully we can make sbuilder and/or pbuilder work with docker instead
of having eleven competing implementations doing the same thing?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi,

thank you Helmut for your (again) very thorough reply. Just as with your 
last large mail to this bug, I can only second the things you said and 
have very little to add.


On 2024-01-16 07:29, Helmut Grohne wrote:

On Tue, Jan 16, 2024 at 12:21:46AM +0100, Francesco Poli wrote:

Well, performance was not an issue here.
The copy of

  $ ls -lFs --si sid_amd64.img
  690M -rw-r-xrwx 1 $USER $USER 2.3G Jan 15 23:26 sid_amd64.img*

was really quick.


Probably. Copying huge chunks of data probably just feels wrong and
technically isn't. We could also generate the file in $TMPDIR in 
general

(hoping it is backed by tmpfs and that the system performing the image
build has at least 4GB ram) and then copy it to the final destination 
in

general. That's usability beating performance.


the patch I posted in an earlier mail to this bug also "solves" the 
problem of missing permission via copying things. Instead of letting 
/sbin/mkfs.ext4 (which runs inside the unshared user namespace) work on 
a file outside of it, the patch runs genext2fs on the outside (thus it 
will happily write to files inside your $HOME) and receives a tarball of 
the rootfs via a pipe from inside the unshared namespace. Then another 
copy is done to shift the image that genext2fs does by the correct 
offset to make room for the first partition.


  d.  configure sbuild-qemu and learn to use it to build Debian 
packages


Does this provide significant advantages over sbuild in unshare mode?


  e.  report to you and/or Christian Kastner (and suggest possible
  improvements for mmdebstrap-autopkgtest-build-qemu and/or
  sbuild-qemu)


I suspect mmdebstrap-autopkgtest-build-qemu very much is unfinished. We
got it into a barely-works.


Yes, but I think there are some immediate things that can be done to 
make the experience a bit better already. For starters, the error 
message that Francesco got initially is really bad and this thing should 
be caught much earlier and give the user an adequate explanation about 
what went wrong and what workaround exists.


I want to do another mmdebstrap release soon and will integrate such a 
check in it. I'll send a patch soon if Francesco wants to try it out 
first.



I think making this all more useful eventually requires finding a
solution for crossing uid boundaries in a sane way. At this time, I see
two fairly generic ways to do this:

a) Have a very simple proxy fuse driver in the initial namespace and
   mount it inside the target namespace. That way an individual
   directory from a different user can be exposed to the namespace
   without granting full access. (Good containment, non-trivial setup,
   slight performance degradation.)

b) Map the calling user in the user namespace. We don't have to map the
   user to root. We can map it to some high uid > 65535. Any process
   that has CAP_DAC_OVERRIDE (and that capability is initially 
available

   and typically available to any process running as uid 0 inside) can
   also access files by your user. (Bad containment, easy setup, no
   performance impact.)


I also think that getting this merged will improve the situation:

https://github.com/tytso/e2fsprogs/pull/118

With that feature, /sbin/mkfs.ext4 can be run on the outside with the 
desired offset for the partition location on the disk image and receive 
a tarball from mmdebstrap via a pipe. Yes, this again means to copy some 
stuff but while the disk itself is 25G, the actual rootfs is not and 
even on my ARM Cortex A53 this is not a noticeable performance impact.


Theodore Ts'o indicated that this pull request will be "shortly 
integrated" here: 
https://lists.debian.org/debian-devel/2023/11/msg00090.html So maybe 
this is another option.


Thanks!

cheers, josch

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-01-14 18:33:14)
>   The partition table has been altered.
>   Syncing disks.
>   $ echo $?
>   0
>   $ ls -lF --si sid_amd64.img 
>   -rw-r-xrwx 1 $USER $USER 670M Jan 14 17:21 sid_amd64.img*
> 
> However, if I attempt to use the resulting image, autopkgtest
> fails:
> 
>   $ autopkgtest --output-dir ./${SRCPKG}_autopkgtest.out  --summary 
> ./${SRCPKG}_autopkgtest.summary --apt-upgrade -B ./${SRCPKG}_amd64.changes -- 
> qemu --overlay-dir /dev/shm ~/Downloads/sid_amd64.img
>   autopkgtest [17:27:29]: starting date and time: 2024-01-14 17:27:29+0100
>   autopkgtest [17:27:29]: version 5.32
>   autopkgtest [17:27:29]: host ${HOST}; command line: /usr/bin/autopkgtest 
> --output-dir './${SRCPKG}_autopkgtest.out' --summary 
> './${SRCPKG}_autopkgtest.summary' --apt-upgrade -B 
> './${SRCPKG}_amd64.changes' -- qemu --overlay-dir /dev/shm 
> ${HOME}/Downloads/sid_amd64.img
>   qemu-system-x86_64: terminating on signal 15 from pid 115770 
> (/usr/bin/python3)
>   : failure: timed out waiting for 'login prompt on serial 
> console'
>   autopkgtest [17:28:29]: ERROR: testbed failure: unexpected eof from the 
> testbed
> 
> 
> Was it just a test to investigate further?
> Or did it have a chance to produce a usable image?

The qemu image you built uses efi booting. Try to add --boot=efi. The default
is --boot=auto which chooses bios boot on amd64.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060843: imath FTCBFS: python3 build dependencies not satisfiable

[Johannes Schauer Marin Rodrigues]

Source: imath
Version: 3.1.9-3
Severity: normal
Tags: patch
User: debian-cr...@lists.debian.org
Usertags: cross-satisfiability

Hi,

imath currently fails to cross build from source because it doesn't declare its
dependencies correctly. The patch at the bottom fixes this problem.

 - it marks those build dependencies that are just used to create
   Architecture:all documentation packages with  (no need to
   cross-build these)
 - it picks the native architecture python3-numpy package (host architecture is
   the default)
 - it turns python3-dev:any into python3-dev:native, libpython3-dev

Thanks!

cheers, josch

diff -Nru imath-3.1.9/debian/control imath-3.1.9/debian/control
--- imath-3.1.9/debian/control  2023-09-04 14:09:31.0 +0200
+++ imath-3.1.9/debian/control  2024-01-15 13:08:40.0 +0100
@@ -10,11 +10,11 @@
  doxygen,
  libboost-python-dev,
  pkg-config,
- python3-breathe,
- python3-dev:any,
- python3-numpy,
- python3-sphinx,
- python3-sphinx-press-theme
+ python3-breathe ,
+ python3-dev:native, libpython3-dev,
+ python3-numpy:native,
+ python3-sphinx ,
+ python3-sphinx-press-theme 
 Standards-Version: 4.6.2
 Rules-Requires-Root: no
 Homepage: https://www.openexr.com
@@ -74,6 +73,7 @@
 Section: doc
 Architecture: all
 Multi-Arch: foreign
+Build-Profiles: 
 Depends:
  libjs-jquery,
  sphinx-common,

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Johannes Schauer Marin Rodrigues (2024-01-13 20:18:34)
> Quoting Helmut Grohne (2024-01-12 21:58:00)
> > > What can cause mkfs.ext4 to fail with a "Permission denied" error?
> > I think this is our typical problem when dealing with user namespaces. I
> > guess that the thing that fails here is mkfs.ext4 opening the target image
> > file (to be formatted). That file has earlier been chowned to the root uid
> > inside the namespace, so permission should be there, but you need more. You
> > also need execute permission (to the first uid of your namespace) for the
> > containing directory up until the root. I guess that none of those are
> > world-executable and not by chanced owned by your first subuid nor owned by
> > the first group in your subgid range.
> 
> I'm not yet convinced that this is it. The problem occurs for Francesco when
> using either /tmp or /dev/shm as a temporary directly. By default, those two
> locations should have the desired permission bits set. Lets check whether
> world-execute permissions are set for all directories up until root. I have
> this:
> 
> $ stat -c "%a %n" / /dev /dev/shm /tmp
> 755 /
> 755 /dev
> 1777 /dev/shm
> 1777 /tmp
> 
> Francesco, what are the world execute permissions on all path components up
> to /tmp and /devv/shm in your case?

scratch what I said in your last mail. Your problem is with the creation of the
image and not with the temporary chroot directory. What Helmut said is likely
correct and exactly the problem you are facing. To verify, you could run the
command you posted initially inside your $HOME directory and show us the
permission bits of your $HOME. Mine are these:

stat -c "%a %n" $HOME
711 /home/josch

I assume in your case, you have a 0 at the end of the octal permissions?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Helmut Grohne (2024-01-12 21:58:00)
> > What can cause mkfs.ext4 to fail with a "Permission denied" error?
> I think this is our typical problem when dealing with user namespaces. I
> guess that the thing that fails here is mkfs.ext4 opening the target image
> file (to be formatted). That file has earlier been chowned to the root uid
> inside the namespace, so permission should be there, but you need more. You
> also need execute permission (to the first uid of your namespace) for the
> containing directory up until the root. I guess that none of those are
> world-executable and not by chanced owned by your first subuid nor owned by
> the first group in your subgid range.

I'm not yet convinced that this is it. The problem occurs for Francesco when
using either /tmp or /dev/shm as a temporary directly. By default, those two
locations should have the desired permission bits set. Lets check whether
world-execute permissions are set for all directories up until root. I have
this:

$ stat -c "%a %n" / /dev /dev/shm /tmp
755 /
755 /dev
1777 /dev/shm
1777 /tmp

Francesco, what are the world execute permissions on all path components up to
/tmp and /devv/shm in your case?

Arguably, mmdebstrap maybe should do this check at start-up. To have a
real-world test-case for such a feature, I'm eager to see how Francesco's
system looks like.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-01-12 20:05:02)
> On Thu, 11 Jan 2024 21:04:16 +0100 Johannes Schauer Marin Rodrigues wrote:
> > Do you see yourself debugging this further by yourself? You probably
> > understand that it's hard for me to investigate something that I cannot
> > test myself.
> 
> Well, you wrote the script, together with Helmut Grohne (who reads us
> in Cc).
> I rather see myself helping the two of you in debugging it...   :-)

could you try applying this patch and then try again:

--%<-
--- a/mmdebstrap-autopkgtest-build-qemu
+++ b/mmdebstrap-autopkgtest-build-qemu
@@ -302,17 +302,12 @@ if test -n "$SCRIPT"; then
'--customize-hook=rm -f "$1/userscript"'
 fi
 
-EXT4_OFFSET_BYTES=$(( (FAT_OFFSET_SECTORS + FAT_SIZE_SECTORS) * 512))
-EXT4_OPTIONS="offset=$EXT4_OFFSET_BYTES,assume_storage_prezeroed=1"
 set -- "$@" \
"--customize-hook=download vmlinuz '$WORKDIR/kernel'" \
"--customize-hook=download initrd.img '$WORKDIR/initrd'" \
-   '--customize-hook=mount --bind "$1" "$1/mnt"' \
-   '--customize-hook=mount --bind "$1/mnt/mnt" "$1/mnt/dev"' \
-   '--customize-hook=/sbin/mkfs.ext4 -d "$1/mnt" -L autopkgtestvm -E 
'"'$EXT4_OPTIONS' '$IMAGE' '$SIZE'" \
-   '--customize-hook=umount --lazy "$1/mnt"' \
+   --format=ext2 \
"$RELEASE" \
-   /dev/null
+   "$IMAGE"
 
 test -n "$MIRROR" && set -- "$@" "$MIRROR"
 test -n "$KEYRING" && set -- "$@" "--keyring=$KEYRING"
@@ -320,6 +315,9 @@ test -n "$KEYRING" && set -- "$@" "--keyring=$KEYRING"
 echo "mmdebstrap $*"
 mmdebstrap "$@" || die "mmdebstrap failed"
 
+EXT4_OFFSET_BYTES=$(( (FAT_OFFSET_SECTORS + FAT_SIZE_SECTORS) * 512))
+fallocate --insert-range --offset=0 --length=$EXT4_OFFSET_BYTES "$IMAGE"
+
 unshare -U -r --map-groups=auto chown 0:0 "$IMAGE"
 chmod "$(printf %o "$(( 0666 - 0$(umask) ))")" "$IMAGE"
 
-->%-

> Indeed, the error does not seem to have anything to do with a "No space left
> on device" failure:
> 
> [...]
>   mke2fs 1.47.0 (5-Feb-2023)
>   mkfs.ext4: Permission denied while trying to determine filesystem size
>   E: setup failed: E: command failed: /sbin/mkfs.ext4 -d "$1/mnt" -L 
> autopkgtestvm -E 'offset=134217728,assume_storage_prezeroed=1' 
> 'sid_amd64.img' '25G'
> [...]
> 
> What can cause mkfs.ext4 to fail with a "Permission denied" error?

This is unshare mode. Funny things happen. :)

The error message you see comes from misc/mke2fs.c in e2fsprogs as a result of
a failed call to ext2fs_open_file(). What puzzles me is, that it's permission
denied for you and working fine for me.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1059869: autopkgtest fails

[Johannes Schauer Marin Rodrigues]

On 2024-01-11 23:40, Bastian Germann wrote:

Ping. It is now 1 week until this will auto-remove several packages.
Please consider releasing and packaging the new release.


Whoops, sorry!! And thank you for the ping.

This has now been taken care of. The version I just uploaded has the 
autopkgtest running fine on salsa so I guess this issue is indeed fixed.


Thanks!

cheers, josch

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2024-01-11 08:58:49)
> On Thu, 11 Jan 2024 00:45:50 +0100 Johannes Schauer Marin Rodrigues wrote:
> > Interesting! I'm unable to reproduce either of these issues and I'm also
> > puzzled why you get this "permission denied" error. On my system, I'm able 
> > run
> > your command with the default (in /tmp) as well as in /dev/shm. Here is the
> > free space each:
> > 
> > $ df --si /tmp /dev/shm
> > Filesystem  Size  Used Avail Use% Mounted on
> > tmpfs   8.6G  4.1k  8.6G   1% /tmp
> > tmpfs   2.0G  418k  2.0G   1% /dev/shm
> So 2.0 GB should be enough.
> And yet, it fails for me with 8.2 GB available on /dev/shm ...  :-(

it's hard for me to debug this as I don't see the error myself. I wonder how I
can be of further assistance to you to figure out what is going on?

> > Maybe something is mounted in a funny way? Both my /tmp and my /dev/shm are
> > tmpfs. The former is mounted with
> > rw,nosuid,nodev,relatime,size=8388608k,inode64 and the latter with
> > rw,nosuid,nodev,inode64.
> 
>   $ mount | grep '/tmp\|shm'
>   tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
>   /dev/md3 on /tmp type ext4 (rw,relatime,discard)

Yes, that looks harmless.

> > I doubt that it fails for lack of system memory unless you have less than 
> > 3.7
> > GB of RAM.
>   $ free --giga
>  totalusedfree  shared  buff/cache   
> available
>   Mem:  16   2  12   0   2
>   14
>   Swap: 17   0  17

$ free --giga
   totalusedfree  shared  buff/cache   
available
Mem:   3   3   0   0   1
   0
Swap:  8   0   8

Your values are all twice as big or more than mine.

> > Maybe permissions are somehow whacky? Both my /tmp and my /dev/shm are set 
> > to
> > drwxrwxrwt (so there is a sticky bit set) and owned by root:root.
> 
>   $ ls -ld /tmp /dev/shm
>   drwxrwxrwt  3 root root  200 Jan 11 08:25 /dev/shm
>   drwxrwxrwt 13 root root 4096 Jan 11 08:36 /tmp

Looks good!

> > What happens with other locations for TMPDIR?
> 
> Which other locations?
> I cannot use anything within my home directory, as you may recall from
> our past [discussions]
> 
> [discussions]: <https://bugs.debian.org/944485#216>

Yes, this also reminds me of #1052471 which found libpam-tmpdir directories
like /tmp/user/1000 to be unusable in unshare mode.

> > Maybe. It's currently .img because it's easier to debug stuff and use kpartx
> > with it without having to extract it first. :)
> Do I understand correctly that you intend to switch to .qcow2 in the future?
> Or otherwise, what's your plan?

I don't have one yet. It's a work in progress. Your input helps me to decide in
which direction to move. :)

> > > Please clarify and/or improve mmdebstrap-autopkgtest-build-qemu .
> > > Thanks for your time!
> > Thanks for your bug! Lets hope we get to the bottom of this.
> You're welcome.
> 
> Looking forward to hearing back from you!

Do you see yourself debugging this further by yourself? You probably understand
that it's hard for me to investigate something that I cannot test myself.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1060416: mmdebstrap-autopkgtest-build-qemu fails on mkfs.ext4; does it require a 25 GiB TMPDIR?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (wintermute) (2024-01-10 23:54:30)
> I am giving mmdebstrap-autopkgtest-build-qemu a try.

thank you!

> The following command fails:
> 
>   $ mmdebstrap-autopkgtest-build-qemu --boot=efi sid sid_amd64.img
> 
> during some package installation with "no space left on device" error,
> since I have /tmp on a somewhat small physical partition:
> 
>   $ df --si /tmp/
>   Filesystem  Size  Used Avail Use% Mounted on
>   /dev/md3868M   99k  806M   1% /tmp

yes, that looks too small.

> I tried with a TMPDIR in system memory:
> 
>   $ TMPDIR=/dev/shm mmdebstrap-autopkgtest-build-qemu \
> --boot=efi sid sid_amd64.img
> 
> but it again fails with the following (final chunk of) output:
> 
>   [...]
>   cp: warning: behavior of -n is non-portable and may change in future; use 
> --update=none instead
>   I: running special hook: download vmlinuz '/dev/shm/tmp.uNRUbVgiOu/kernel'
>   I: running special hook: download initrd.img 
> '/dev/shm/tmp.uNRUbVgiOu/initrd'
>   I: running --customize-hook in shell: sh -c 'mount --bind "$1" "$1/mnt"' 
> exec /dev/shm/mmdebstrap.IXehDNUWIf
>   I: running --customize-hook in shell: sh -c 'mount --bind "$1/mnt/mnt" 
> "$1/mnt/dev"' exec /dev/shm/mmdebstrap.IXehDNUWIf
>   I: running --customize-hook in shell: sh -c '/sbin/mkfs.ext4 -d "$1/mnt" -L 
> autopkgtestvm -E 'offset=134217728,assume_storage_prezeroed=1' 
> 'sid_amd64.img' '25G'' exec /dev/shm/mmdebstrap.IXehDNUWIf
>   mke2fs 1.47.0 (5-Feb-2023)
>   mkfs.ext4: Permission denied while trying to determine filesystem size
>   E: setup failed: E: command failed: /sbin/mkfs.ext4 -d "$1/mnt" -L 
> autopkgtestvm -E 'offset=134217728,assume_storage_prezeroed=1' 
> 'sid_amd64.img' '25G'
>   W: hooklistener errored out: E: received eof on socket
>   
>   I: main() received signal PIPE: waiting for setup...
>   I: removing tempdir /dev/shm/mmdebstrap.IXehDNUWIf...
>   E: mmdebstrap failed to run
>   mmdebstrap failed
> 
> Does it fail because I do not have enough system memory?
> 
>   $ df --si /dev/shm/
>   Filesystem  Size  Used Avail Use% Mounted on
>   tmpfs   8.3G  108M  8.2G   2% /dev/shm
> 
> Is this the explanation?
> Otherwise, what went wrong?

Interesting! I'm unable to reproduce either of these issues and I'm also
puzzled why you get this "permission denied" error. On my system, I'm able run
your command with the default (in /tmp) as well as in /dev/shm. Here is the
free space each:

$ df --si /tmp /dev/shm
Filesystem  Size  Used Avail Use% Mounted on
tmpfs   8.6G  4.1k  8.6G   1% /tmp
tmpfs   2.0G  418k  2.0G   1% /dev/shm

Maybe something is mounted in a funny way? Both my /tmp and my /dev/shm are
tmpfs. The former is mounted with
rw,nosuid,nodev,relatime,size=8388608k,inode64 and the latter with
rw,nosuid,nodev,inode64.

I doubt that it fails for lack of system memory unless you have less than 3.7
GB of RAM.

Maybe permissions are somehow whacky? Both my /tmp and my /dev/shm are set to
drwxrwxrwt (so there is a sticky bit set) and owned by root:root.

What happens with other locations for TMPDIR?

> By the way, the old script that used guestfish (with all its limitations)
> was able to create a QEMU image in .qcow2 format and my /dev/shm was
> enough for it to work correctly.

It should be enough. Your /dev/shm is 8G large and it works with my 2G.

> Why does the current mmdebstrap-autopkgtest-build-qemu create a QEMU
> image in .img format? Isn't the .qcow2 format better?

Maybe. It's currently .img because it's easier to debug stuff and use kpartx
with it without having to extract it first. :)

> Please clarify and/or improve mmdebstrap-autopkgtest-build-qemu .
> Thanks for your time!

Thanks for your bug! Lets hope we get to the bottom of this.

cheers, josch

signature.asc
Description: signature

Bug#1054137: apt: let dh_installsystemd choose the location of units

[Johannes Schauer Marin Rodrigues]

Control: affects -1 + mmdebstrap

Hi,

Quoting Chris Hofstaedtler (2024-01-09 18:46:14)
> On Tue, Oct 17, 2023 at 04:36:19PM +0200, Helmut Grohne wrote:
> [Move systemd files to /usr]
> > I'm attaching a patch for your convenience.
> 
> Could I ask you to apply this patch some time soon, please?
> 
> > Once dh_installsystemd is updated and apt
> > binNMUed, it will have no files left in aliased locations.
> 
> debhelper is already changed, so dh_installsystemd will already
> install into /usr.

this bug affects my package mmdebstrap. For your convenience, I created a MR on
salsa:

https://salsa.debian.org/apt-team/apt/-/merge_requests/318

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Johannes Schauer Marin Rodrigues]

Hi Francesco,

On 2023-12-28 15:47, Francesco Poli wrote:

I stayed tuned and I saw that mmdebstrap/1.4.0-1 ships a
mmdebstrap-autopkgtest-build-qemu that does not require superuser
privileges (according to its man page).

Do I understand correctly that this new script is one of the building
blocks that can make the above-summarized "daydream" come true?

Could you please give me a status update on the "daydream"?

Looking forward to hearing back from you.
Thanks for all the good stuff!



Hello again,
a gentle ping about the above-quoted daydream.

Has anyone received my message?
Could someone please reply?

Thanks a lot for your time and patience.


Well.. thanks a lot for *your* patience! Sorry for letting you hang for 
this long. ‍♂️ And thank you for the ping -- I often loose track of 
projects that I'm involved in.


So... mmdebstrap-autopkgtest-build-qemu might be what you want *if* you 
don't care about ppc64el or mips. The latter was (I think) never 
supported my autopkgtest-qemu because there is no good option for a 
bootloader. In case of ppc64el, there is ieee1275 booting but that is 
not supported by mmdebstrap-autopkgtest-build-qemu.


Could you give mmdebstrap-autopkgtest-build-qemu a try and see if it 
does what you want? I'm preparing another mmdebstrap release and if you 
find any issues with that script I can incorporate fixes in the next 
release.


Thanks!

cheers, josch

Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency

[Johannes Schauer Marin Rodrigues]

Quoting Paul Gevers (2024-01-06 13:20:11)
> Thanks for being elaborate in your reply, it matches what I was thinking. (I
> wasn't aware of the other examples though).

there are certainly more examples. For example I maintain the package box64
which allows running amd64 binaries on arm64 but requires amd64 libc to
operate. Because pkg:amd64 doesn't work for britney, I used this:

Depends: libgcc-s1:amd64 | libgcc-s1-amd64-cross, libstdc++6:amd64 | 
libstdc++6-amd64-cross

I had to patch the software to also look into the paths that
libgcc-s1-amd64-cross and libstdc++6-amd64-cross use to make this work. Those
two packages are Architecture:all but they do contain amd64 shared libraries.
Helmut probably has a much better idea whether, in an ideal world, Arch:all
packages like libgcc-s1-amd64-cross should go away and be replaced by
corresponding architecture-qualified dependencies on the architecture-specific
libraries.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency

[Johannes Schauer Marin Rodrigues]

Quoting Paul Gevers (2024-01-05 20:15:22)
> Thanks for reaching out.

Thank Helmut for poking me in #debian-apt :)

> For britney2, the Sources stanza would also be needed; then we could use this
> to generate britney2 testcases. I created 10 of those yesterday by hand [1].
> 
> The simplest for of tests is a tree with
> var/data/unstable/Sources
> # i386 is the default archive of the testsuite, which can be overruled
> # if that makes sense
> var/data/unstable/Packages_i386
> var/data/testing/Sources (may be empty)
> var/data/testing/Packages_i386
> expected
> 
> expected is in Heidi format (if I understand correctly) of what britney2 
> will allow to be in testing after the run, it will only migrate packages 
> that are installable.
> 
> Would it make sense to you to generate a branch in that archive with a bunch
> of tests that you know the answer too?

I think it's a bit more complicated than that. Currently, the tool creates 8624
package relationships. If I remember correctly, britney is unable to analyze
cross-architecture relationships? In that case that number goes down to 2352.
One could reduce that number even further and only check those cases where apt,
dpkg and dose3 agree on a solution but that would then rather be a
documentation of the status quo than a list of the intended ground truth. Maybe
it would make sense to analyze the archive and only add those cases that
currently exist as real package relationships?

What the tool never received since its inception was somebody to look over
these cases and write down what the ground-truth actually is supposed to look
like. For the britney tests you likely want some kind of ground-truth and I
fear that tool can give you the status quo but not necessarily the truth as
intended by the spec.

If that is fine for you, do you still want to add thousands of test-cases? Or
do you want to hand-pick them?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1059929: release.debian.org: gobject-introspection_1.78.1-9 is said to have an unsatisfiable dependency

[Johannes Schauer Marin Rodrigues]

Hi,

On Thu, 4 Jan 2024 21:04:57 +0100 Paul Gevers  wrote:
> [20:21:54]  agreed, but britney2 doesn't handle :any on virtual 
> packages in any way (neither binary dep nor build dep)
> [20:22:10]  (and I'm not sure whether that's legal in any way)
> [20:23:08]  agreed
> [20:23:28]  which I'm fixing right now
> [20:23:39]  apparently there's builds with :native
> [20:23:43]  so I'll support that
> [20:24:04]  but I wanted to know what happens with :any (and 
> virtual B-D)
> [20:25:07]  well, this is not something that existed in the 
> archive before gobject-introspection, so we're about to extend what is 
> defined
> [20:25:15]  according to smcv this is legal to dpkg and apt
> [20:25:35]  but we know that resolvers 
> (dpkg/apt/dose/britney2/...) disagree on corner cases
> [20:25:50]  and I'm guilty of having written yet another 
> resolver in dumat
> [20:26:26]  britney2 just has to follow what dpkg and apt happen 
> to agree on
> [20:26:45]  or maybe even, what apt says
> [20:27:01]  if that works

back in 2015 I wrote a small tool which is able to create artificial dependency
situations involving two real packages and (optionally) a third virtual
packages to make sure that the multi-arch implementations of apt, dpkg and
dose3 agree with each other in all cases (spoiler: they do not).

https://gitlab.mister-muffin.de/josch/deb-m-a-dep-check/

The tool is meant to make mass comparisons (it generates 8624 test cases) and
thus its interface is a bit clunky but if what you want to do is to see whether
apt, dose3 and dpkg agree on a situation where one package depends on a virtual
package with :any which is provided by a real m-a: allowed package, then you
would write this:

./check.sh binary pkgc amd64 amd64 no allowed depends pkgc:any

It would generate the following two stubs (shortened here for brevity):

Package: pkga
Version: 1
Architecture: amd64
Depends: pkgc:any
Multi-Arch: no

Package: pkgb
Version: 1
Architecture: amd64
Provides: pkgc
Multi-Arch: allowed

And yes, all three tools agree on this situation: it is satisfiable. If you
make pkgb m-a:no, then all tools agree that the situation is unsatisfiable.

We can do the same checks for pkga being a source package:

./check.sh source pkgc none amd64 none allowed depends pkgc:any

Again, all three tools agree that this situation is satisfiable.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1059885: makes dkms package postinst fail with "post-installation script subprocess returned error exit status 127"

[Johannes Schauer Marin Rodrigues]

Control: severity -1 serious

Hi,

now that I know that BUILT_MODULE_NAME[0] and DEST_MODULE_LOCATION[0] break it,
I found other packages via codesearch.d.n that use that syntax and I was
wondering whether they break as well.

The following packages fail to install in unstable right now:

acpi-call-dkms adv-17v35x-dkms bbswitch-dkms dahdi-dkms ddcci-dkms
digimend-dkms dpdk-kmods-dkms evdi-dkms falcosecurity-scap-dkms failed
gost-crypto-dkms iptables-netflow-dkms jool-dkms langford-dkms
librem-ec-acpi-dkms vpoll-dkms lime-forensics-dkms apfs-dkms lttng-modules-dkms
nat-rtsp-dkms openafs-modules-dkms openrazer-driver-dkms openvpn-dco-dkms
rapiddisk-dkms rtpengine-kernel-dkms tp-smapi-dkms v4l2loopback-dkms
west-chamber-dkms xtables-addons-dkms xtrx-dkms

But all of those succeed to install in testing.

I also tried installing all of the above packages with dkms from unstable
patched with my patch from my last email and then they succeed to install
again, thus indicating that this is all the same problem class.

Since this problem is making several dozen packages in the archive
uninstallable, I'm raising the severity of this bug to serious.  Apologies if
I'm wrong but better safe then sorry, right?  Maybe this prevents users of
apt-listbugs from installing dkms from unstable and have their dkms modules
fail. Please lower the severity if I raised it in error.  Apologies in that
case.

I'd also like to suggest the following patch to be included into the dkms
package which would've spotted this problem early on:

--- dkms-3.0.12.orig/test/dkms_test-1.0/dkms.conf
+++ dkms-3.0.12/test/dkms_test-1.0/dkms.conf
@@ -1,7 +1,7 @@
 PACKAGE_NAME="dkms_test"
 PACKAGE_VERSION="1.0"
-BUILT_MODULE_NAME="dkms_test"
+BUILT_MODULE_NAME[0]="dkms_test"
 
 AUTOINSTALL="yes"
 
-DEST_MODULE_LOCATION="/kernel/extra"
+DEST_MODULE_LOCATION[0]="/kernel/extra"

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#229775: apt 2.7.7: build-dep doesn't work if priority is 499

[Johannes Schauer Marin Rodrigues]

Quoting Askar Safin (2024-01-02 16:05:03)
> It follows from my interpretation of the phrase "Subject to
> dependency constraints" from "man apt_preferences", that
> first apt should exclude logically inconsistent solutions. So it
> should first exclude "libuv1-dev 1.9.1-3", because it is
> inconsistent with the constraint ">= 1.10". Then apt should
> choose version with highest priority from versions, which
> are left. I. e. apt should choose "1.34.2-1~bpo9+1". But in
> practice I see this error message:
> 
> ===
> # apt-get build-dep cmake
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Some packages could not be installed. This may mean that you have
> requested an impossible situation or if you are using the unstable
> distribution that some required packages have not yet been created
> or been moved out of Incoming.
> The following information may help to resolve the situation:
> 
> The following packages have unmet dependencies:
>  builddeps:cmake : Depends: libuv1-dev (>= 1.10) but 1.9.1-3 is to be 
> installed
> E: Unable to correct problems, you have held broken packages.
> ===
> 
> So, apt for whatever reasons prefers logically inconsistent
> solution over consistent. Chooses wrong solution with high
> priority over correct solution with low priority. Chooses
> libuv1-dev 1.9.1-3 and then (of course) fails to install it,
> because the constraints are violated.
> 
> Okay, so I hope now I convinced you that this is a bug.

This is not how the apt dependency solver works. The apt solver uses pin
priorities to decide which package is the candidate version. What you want is a
different solver that is able to ignore pin values and find a solution
independent on what package has the highest pin priority. If you want that,
then why don't you use a solver that is not apt for your problem?

It feels to me like what you are describing as a bug is like saying that dash
is buggy because it does not support arrays. If you want arrays when writing
shell, use bash instead of dash. This is not a bug in dash as dash is
intentionally made to not support arrays.

In your case, maybe you should consider using aptitude or switch out the apt
dependency resolver to aspcud?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1059885: makes dkms package postinst fail with "post-installation script subprocess returned error exit status 127"

[Johannes Schauer Marin Rodrigues]

Hi Andreas,

thank you so much for your fast reply!

Quoting Andreas Beckmann (2024-01-03 02:53:20)
> Which dkms package version was used to build the failing package? Does█
> it make a difference (or even ftbfs) if it gets rebuilt with 3.0.12-3?

According to the Installed-Build-Depends field of the .buildinfo, the package
was built with dh-dkms (= 3.0.12-3):

https://source.mnt.re/reform/reform-debian-packages/-/jobs/2724/raw

Quoting Andreas Beckmann (2024-01-03 03:00:15)
> Can you install dkms-test-dkms and/or dkms-noautoinstall-test-dkms ?

Oh I didn't know that those packages existed, thank you! If I replace my
reform2-lpc-dkms package with either of those, it works fine. This suggests
that my package does something that used to work in the past but fails with the
new dkms version?

In an effort to get a bit further I patched the dkms package and added a "set
+x" to dkms_common.postinst which seems to be the part that failed. Now I get
this in my log:

+ [ -z reform2_lpc ]
+ [ -z 1.34 ]
+ [ -f /etc/dkms/no-autoinstall ]
+ [ -r /etc/dkms/framework.conf ]
+ . /etc/dkms/framework.conf
+ + cut -d/ -f4
ls -dv /lib/modules/6.6.9-reform2-arm64/build
+ KERNELS=6.6.9-reform2-arm64
+ uname -r
+ CURRENT_KERNEL=6.5.0-0.deb12.4-reform2-arm64
+ [ -e /var/lib/dkms/reform2_lpc/1.34 ]
+ [ -f /usr/share/reform2_lpc-dkms/reform2_lpc-1.34.dkms.tar.gz ]
+ [ -d /usr/src/reform2_lpc-1.34 ]
+ echo Loading new reform2_lpc-1.34 DKMS files...
Loading new reform2_lpc-1.34 DKMS files...
+ dkms add -m reform2_lpc -v 1.34
+ AUTOINSTALL=
+ . /var/lib/dkms/reform2_lpc/1.34/source/dkms.conf
+ autoinstall=
dpkg: error processing package reform2-lpc-dkms (--configure):
 installed reform2-lpc-dkms package post-installation script subprocess 
returned error exit status 127


The contents of /var/lib/dkms/reform2_lpc/1.34/source/dkms.conf are:

PACKAGE_NAME="reform2_lpc"
PACKAGE_VERSION="1.34"
BUILT_MODULE_NAME[0]="reform2_lpc"
DEST_MODULE_LOCATION[0]="/extra"
AUTOINSTALL="yes"

When I source that file in a dash shell I get:

sh: 3: /var/lib/dkms/reform2_lpc/1.34/source/dkms.conf: 
BUILT_MODULE_NAME[0]=reform2_lpc: not found
sh: 4: /var/lib/dkms/reform2_lpc/1.34/source/dkms.conf: 
DEST_MODULE_LOCATION[0]=/extra: not found

This makes sense because the [0] syntax is a bashism and not compatible with
POSIX shell. This also explains the exit status 127 I initially received which
indicates "command not found" as dash tries to run 'BUILT_MODULE_NAME[0]' and
fails to find an executable named like that.

I also see that dkms version 3.0.12-1 is missing this line in its
/usr/lib/dkms/common.postinst and thus does not trigger this problem:

autoinstall=$(AUTOINSTALL=; . "/var/lib/dkms/$NAME/$VERSION/source/dkms.conf" 
>/dev/null 2>&1; echo $AUTOINSTALL)

That line was added in
debian/patches/only-autobuild-AUTOINSTALL-yes-modules.patch in

https://salsa.debian.org/debian/dkms/-/commit/5cd92a30769dfaa2223f160ee83e8afcd9aacb9c

Is the [0] suffix not allowed anymore or are the dkms.conf files expected to be
sourced only in a bash shell? The following patch fixes my problem by sourcing
the dkms.conf in a bash sub-shell:

--- dkms-3.0.12/debian/patches/only-autobuild-AUTOINSTALL-yes-modules.patch 
2024-01-02 14:10:36.0 +0100
+++ dkms-3.0.12/debian/patches/only-autobuild-AUTOINSTALL-yes-modules.patch 
2024-01-03 06:26:30.0 +0100
@@ -4,7 +4,7 @@
  dkms add -m $NAME -v $VERSION > /dev/null
  fi
  
-+autoinstall=$(AUTOINSTALL=; . "/var/lib/dkms/$NAME/$VERSION/source/dkms.conf" 
>/dev/null 2>&1; echo $AUTOINSTALL)
++autoinstall=$(bash -c 'AUTOINSTALL=; . "/var/lib/dkms/$1/$2/source/dkms.conf" 
>/dev/null 2>&1; echo $AUTOINSTALL' -- "$NAME" "$VERSION")
 +if [ -z "$autoinstall" ]; then
 +echo "Not building the $NAME module which does not have AUTOINSTALL 
enabled."
 +exit 0


What do you think?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1059885: makes dkms package postinst fail with "post-installation script subprocess returned error exit status 127"

[Johannes Schauer Marin Rodrigues]

Package: dkms
Version: 3.0.12-3
Severity: normal

Hi,

the problem appears since 3.0.12-3 and not with 3.0.12-1. The package that
fails since this change in the dkms package is not part of Debian but its
packaging source can be found here:

https://source.mnt.re/reform/reform-tools/-/tree/main/debian?ref_type=heads

It is entirely possible that the *.dkms files in that package are faulty and
happened to work with 3.0.12-1 but fail with 3.0.12-3 and that it's not the
dkms package at fault.

Anyways, the installation fails since dkms 3.0.12-3 with the following message:

Setting up reform2-lpc-dkms (1.34) ...
Loading new reform2_lpc-1.34 DKMS files...
dpkg: error processing package reform2-lpc-dkms (--configure):
 installed reform2-lpc-dkms package post-installation script subprocess 
returned error exit status 127

Can you give me some hints how to debug this? How do I get a better error
message than this to try and figure out what went wrong?

Thanks!

cheers, josch

Bug#1059869: autopkgtest fails

[Johannes Schauer Marin Rodrigues]

Control: severity -1 serious

Hi Bastian,

thank you for your bug!

I'm raising its severity as this makes the package have a RC bug.

Quoting Bastian Germann (2024-01-02 18:02:45)
> The tests fail with the latest pymupdf: AttributeError: 'Shape' object has no
> attribute 'insertText'
> 
> https://ci.debian.net/packages/p/plakativ/testing/amd64/41418631/
> 
> The methon name has changed to insert_text.

Yes, this has been fixed upstream here:

https://gitlab.mister-muffin.de/josch/plakativ/commit/fab69256741b1b025d305f536941e71e21b9f1e0

I have to make a new release and package that.

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1059438: ITP: ruby-deb-version, A port of "Debian Version" comparison function to Ruby

[Johannes Schauer Marin Rodrigues]

Hi,

On Mon, 25 Dec 2023 13:25:19 +0100 "Ajayi Olatunji O." 
 wrote:
> * Package name: ruby-deb-version
> 
> Version : 1.0.2
> Upstream Author : Nemo
> 
> * URL : https://github.com/captn3m0/ruby-deb-version#readme
> * License : MIT
> Programming Lang:Ruby
> Description : A port of Debian Version comparison to Ruby.
> 
>   This package is a build dependency for ruby-arr-pm, a package required
> in gitlab 16.6.2

since this seems to be a new parser, could you please provide an implementation
that tests ruby-deb-version against existing parsers to assure it is correct?

https://gitlab.mister-muffin.de/josch/debversioncomp

Let me know if you need any assistance. Unfortunately I do not speak Ruby, so I
cannot do this myself.

Thanks!

cheers, josch

signature.asc
Description: signature