Bug#1068775: sbuild-qemu: fails to work, if piuparts is requested, but not installed on the host

[Richard Lewis]

On Fri, 12 Apr 2024, 19:00 Johannes Schauer Marin Rodrigues, <
jo...@debian.org> wrote:

> Hi Francesco,
>
> Quoting Francesco Poli (wintermute) (2024-04-11 00:13:51)
> >
> >   sbuild --dist unstable --purge-build=never --purge-deps=never
> --chroot-mode=autopkgtest --autopkgtest-virt-server=qemu
> --autopkgtest-virt-server-opt --overlay-dir=/dev/shm
> --autopkgtest-virt-server-opt --qemu-architecture=x86_64
> --autopkgtest-virt-server-opt --ram-size=2048 --autopkgtest-virt-server-opt
> --cpus=2 --autopkgtest-virt-server-opt --boot=efi
> --autopkgtest-virt-server-opt
> /home/$USER/.cache/sbuild/unstable-autopkgtest-amd64.img
> --bd-uninstallable-explainer apt
> > Error reading configuration: PIUPARTS binary 'piuparts' does not exist
> or is not executable at /usr/share/perl5/Sbuild/Conf.pm line 76.
> >
> >



> > However, I thought that setting $run_piuparts = 1 in ~/.sbuildrc caused
> > piuparts to be run inside the virtual machine guest system, not on the
> > host system!
>
> What made you think that? Is the error message above not clear enough? Do
> you
> think it should be improved to make things clearer? Do you have
> suggestions for
> a message that would've better explained that piuparts needs to be
> installed on
> the host system?
>

Adding a data point, i would also like the error message to be clearer - i
know that once you understand how sbuild and piuparts work it becomes
"obvious" that piuparts needs to be run from the host -- but this is not at
all clear for new users: would be great if the documentation and errors
could clearly distinguish between which things inside/outside the chroot


This particular error message could be improved in at least 3 ways:
1. "Error reading configuration" -- suggests it couldnt parse the config
file, which doesnt seem to be the case here (suggest either dropping this
from the error message or saying more clearly what the issue is)

2. "PIUPARTS binary 'piuparts' does not exist or is not executable"

not sure why the capitials and repetition?  just say more directly what the
issue is: eg : "Error: piuparts not found (it needs to be installed on the
host)"

3.  "at /usr/share/perl5/Sbuild/Conf.pm line 76." -- not sure this
implementation detail is helpful for a user -- it makes it look a bug
rather than a message for the user: could it be dropped from the error
message or hidden behind a "debug"/"verbose" setting?


thanks for working on sbuild, it's really great.

Bug#1068775: sbuild-qemu: fails to work, if piuparts is requested, but not installed on the host

[Francesco Poli]

Control: severity -1 wishlist


On Fri, 12 Apr 2024 19:57:23 +0200 Johannes Schauer Marin Rodrigues
wrote:

[...]
> Quoting Francesco Poli (wintermute) (2024-04-11 00:13:51)
[...]
> > I can update the virtual machine (if I create a symlink, see bug 
> > [#1061816]):
> >
> >   $ sbuild-qemu-update --boot=efi unstable-autopkgtest-amd64.img
> > 
> > [#1061816]: 
> 
> I think 1061816 was fixed with 0.85.7 and the changelog was just missing the
> "closes" entry:
> 
> https://tracker.debian.org/news/1518576/accepted-sbuild-0857-source-into-unstable/

Yes, I have just tested sbuild-qemu/0.85.7 from unstable, and I can
confirm that the symlink is no longer needed.
I have just sent a message to close bug report #1061816 ...

> 
> > But, when I try to build a package from withing the unpacked source
> > tree:
[...]
> > Error reading configuration: PIUPARTS binary 'piuparts' does not exist or 
> > is not executable at /usr/share/perl5/Sbuild/Conf.pm line 76.
> > 
> > Now, piuparts is indeed not installed on the host:
[...]
> > However, I thought that setting $run_piuparts = 1 in ~/.sbuildrc caused
> > piuparts to be run inside the virtual machine guest system, not on the
> > host system!
> 
> What made you think that? Is the error message above not clear enough? Do you
> think it should be improved to make things clearer? Do you have suggestions 
> for
> a message that would've better explained that piuparts needs to be installed 
> on
> the host system?

My point is not that the error message should be improved.
Actually, I think the error message clearness is basically OK.

> 
> > Hence, I thought that piuparts was going to be automatically installed
> > and executed on the guest system (actually on the throw-away overlay).
> > And I thought that the same was going to happen for $run_autopkgtest = 1
> > and for $run_lintian = 1 ...
> > 
> > Am I completely off-track?
> > What did I fail to understand?
> > 
> > Why does sbuild-qemu insist that piuparts be installed on the *host*
> > system?
> 
> Because it needs to be installed on the host. In the same way as autopkgtest
> needs to be on the host. What can sbuild improve?

My point is that sbuild{,-qemu} should install piuparts inside the build
environment (chroot or VM guest system) and run it from within the
build environment, if this is possible.

Does sbuild{,-qemu} do so for lintian? I think this is even more
important for lintian: if the host system runs Debian testing (or maybe
even Debian stable) and the package is being built for Debian unstable
(as upload target), then the package should be checked by the version
of lintian which is currently in Debian unstable. If it were checked by
the version of lintian from Debian testing (or even from Debian
stable), many undeserved "unknown Policy version" complaints could pop
up...

Do you agree with the reasoning?

Thanks for your time and patience!   :-)

-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgpli10LtTDY8.pgp
Description: PGP signature

Bug#1068775: sbuild-qemu: fails to work, if piuparts is requested, but not installed on the host

[Johannes Schauer Marin Rodrigues]

Hi Francesco,

Quoting Francesco Poli (wintermute) (2024-04-11 00:13:51)
> I am trying to set up sbuild-qemu to build (and test) Debian packages.
> 
> After creating the virtual machine image:
> 
>   $ mkdir -p ~/.cache/sbuild/build
>   $ cd /dev/shm
>   $ TMPDIR=/dev/shm mmdebstrap-autopkgtest-build-qemu \
> --size=25G --boot=efi sid unstable-autopkgtest-amd64.img
>   $ mv -i unstable-autopkgtest-amd64.img ~/.cache/sbuild/
> 
> I prepared the following configuration file:
> 
>   $ cat ~/.sbuildrc 
>   $source_only_changes = 1;
>   $autopkgtest_require_success = 1;
>   $lintian_require_success = 0;
>   $piuparts_require_success = 1;
>   $run_autopkgtest = 1;
>   $run_lintian = 1;
>   $run_piuparts = 1;
>   $build_dir = "$HOME/.cache/sbuild/build";
>   
>   # don't remove this, Perl needs it:
>   1;
> 
> I can update the virtual machine (if I create a symlink, see bug [#1061816]):
>
>   $ sbuild-qemu-update --boot=efi unstable-autopkgtest-amd64.img
> 
> [#1061816]: 

I think 1061816 was fixed with 0.85.7 and the changelog was just missing the
"closes" entry:

https://tracker.debian.org/news/1518576/accepted-sbuild-0857-source-into-unstable/

> But, when I try to build a package from withing the unpacked source
> tree:
> 
>   $ sbuild-qemu --boot=efi --overlay-dir=/dev/shm
>   sbuild --dist unstable --purge-build=never --purge-deps=never 
> --chroot-mode=autopkgtest --autopkgtest-virt-server=qemu 
> --autopkgtest-virt-server-opt --overlay-dir=/dev/shm 
> --autopkgtest-virt-server-opt --qemu-architecture=x86_64 
> --autopkgtest-virt-server-opt --ram-size=2048 --autopkgtest-virt-server-opt 
> --cpus=2 --autopkgtest-virt-server-opt --boot=efi 
> --autopkgtest-virt-server-opt 
> /home/$USER/.cache/sbuild/unstable-autopkgtest-amd64.img 
> --bd-uninstallable-explainer apt
> Error reading configuration: PIUPARTS binary 'piuparts' does not exist or is 
> not executable at /usr/share/perl5/Sbuild/Conf.pm line 76.
> 
> Now, piuparts is indeed not installed on the host:
> 
>   $ apt policy piuparts
>   piuparts:
> Installed: (none)
> Candidate: 1.4.1
> Version table:
>1.4.1 800
>   800 http://deb.debian.org/debian testing/main amd64 Packages
>   500 http://deb.debian.org/debian unstable/main amd64 Packages
> 
> However, I thought that setting $run_piuparts = 1 in ~/.sbuildrc caused
> piuparts to be run inside the virtual machine guest system, not on the
> host system!

What made you think that? Is the error message above not clear enough? Do you
think it should be improved to make things clearer? Do you have suggestions for
a message that would've better explained that piuparts needs to be installed on
the host system?

> Hence, I thought that piuparts was going to be automatically installed
> and executed on the guest system (actually on the throw-away overlay).
> And I thought that the same was going to happen for $run_autopkgtest = 1
> and for $run_lintian = 1 ...
> 
> Am I completely off-track?
> What did I fail to understand?
> 
> Why does sbuild-qemu insist that piuparts be installed on the *host*
> system?

Because it needs to be installed on the host. In the same way as autopkgtest
needs to be on the host. What can sbuild improve?

Thanks!

cheers, josch

signature.asc
Description: signature

Bug#1068775: sbuild-qemu: fails to work, if piuparts is requested, but not installed on the host

[Francesco Poli (wintermute)]

Package: sbuild-qemu
Version: 0.85.6
Severity: normal
X-Debbugs-Cc: invernom...@paranoici.org

Hi!

I am trying to set up sbuild-qemu to build (and test) Debian packages.

After creating the virtual machine image:

  $ mkdir -p ~/.cache/sbuild/build
  $ cd /dev/shm
  $ TMPDIR=/dev/shm mmdebstrap-autopkgtest-build-qemu \
--size=25G --boot=efi sid unstable-autopkgtest-amd64.img
  $ mv -i unstable-autopkgtest-amd64.img ~/.cache/sbuild/

I prepared the following configuration file:

  $ cat ~/.sbuildrc 
  $source_only_changes = 1;
  $autopkgtest_require_success = 1;
  $lintian_require_success = 0;
  $piuparts_require_success = 1;
  $run_autopkgtest = 1;
  $run_lintian = 1;
  $run_piuparts = 1;
  $build_dir = "$HOME/.cache/sbuild/build";
  
  # don't remove this, Perl needs it:
  1;

I can update the virtual machine (if I create a symlink, see bug [#1061816]):

  $ sbuild-qemu-update --boot=efi unstable-autopkgtest-amd64.img

[#1061816]: 

But, when I try to build a package from withing the unpacked source
tree:

  $ sbuild-qemu --boot=efi --overlay-dir=/dev/shm
  sbuild --dist unstable --purge-build=never --purge-deps=never 
--chroot-mode=autopkgtest --autopkgtest-virt-server=qemu 
--autopkgtest-virt-server-opt --overlay-dir=/dev/shm 
--autopkgtest-virt-server-opt --qemu-architecture=x86_64 
--autopkgtest-virt-server-opt --ram-size=2048 --autopkgtest-virt-server-opt 
--cpus=2 --autopkgtest-virt-server-opt --boot=efi --autopkgtest-virt-server-opt 
/home/$USER/.cache/sbuild/unstable-autopkgtest-amd64.img 
--bd-uninstallable-explainer apt
Error reading configuration: PIUPARTS binary 'piuparts' does not exist or is 
not executable at /usr/share/perl5/Sbuild/Conf.pm line 76.

Now, piuparts is indeed not installed on the host:

  $ apt policy piuparts
  piuparts:
Installed: (none)
Candidate: 1.4.1
Version table:
   1.4.1 800
  800 http://deb.debian.org/debian testing/main amd64 Packages
  500 http://deb.debian.org/debian unstable/main amd64 Packages

However, I thought that setting $run_piuparts = 1 in ~/.sbuildrc caused
piuparts to be run inside the virtual machine guest system, not on the
host system!
Hence, I thought that piuparts was going to be automatically installed
and executed on the guest system (actually on the throw-away overlay).
And I thought that the same was going to happen for $run_autopkgtest = 1
and for $run_lintian = 1 ...

Am I completely off-track?
What did I fail to understand?

Why does sbuild-qemu insist that piuparts be installed on the *host*
system?

Please clarify and/or fix this issue.
Thanks for your kind assistance!


-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (800, 'testing'), (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.6.15-amd64 (SMP w/4 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_US:en
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages sbuild-qemu depends on:
ii  autopkgtest  5.34
ii  python3  3.11.6-1
ii  python3-pexpect  4.9-2
ii  python3-psutil   5.9.8-2
ii  qemu-system-x86  1:8.2.1+ds-2
ii  qemu-utils   1:8.2.1+ds-2
ii  sbuild   0.85.6
ii  vmdb20.28-2

Versions of packages sbuild-qemu recommends:
ii  qemu-system-arm [qemu-system-arm]  1:8.2.1+ds-2
ii  qemu-system-ppc [qemu-system-ppc]  1:8.2.1+ds-2

sbuild-qemu suggests no packages.

-- no debconf information