Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Francesco Poli]

On Tue, 09 Jan 2024 11:41:09 +0100 Johannes Schauer Marin Rodrigues wrote:

[...]
> So... mmdebstrap-autopkgtest-build-qemu might be what you want *if* you 
> don't care about ppc64el or mips. The latter was (I think) never 
> supported my autopkgtest-qemu because there is no good option for a 
> bootloader. In case of ppc64el, there is ieee1275 booting but that is 
> not supported by mmdebstrap-autopkgtest-build-qemu.

Well, ppc64el or mips would be nice, but not a must.

I hope mips will be supported autopkgtest-qemu in the future.

As far as ppc64el is concerned, I hope a good solution for booting with
mmdebstrap-autopkgtest-build-qemu can be found, but first I would like
to see mmdebstrap-autopkgtest-build-qemu work correctly for the
supported architectures...

> 
> Could you give mmdebstrap-autopkgtest-build-qemu a try and see if it 
> does what you want? I'm preparing another mmdebstrap release and if you 
> find any issues with that script I can incorporate fixes in the next 
> release.

I have just filed bug report [#1060416].
Thanks for being willing to improve the script!

[#1060416]: 


-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgpyQ_zHw_dqA.pgp
Description: PGP signature

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Johannes Schauer Marin Rodrigues]

Hi Francesco,

On 2023-12-28 15:47, Francesco Poli wrote:

I stayed tuned and I saw that mmdebstrap/1.4.0-1 ships a
mmdebstrap-autopkgtest-build-qemu that does not require superuser
privileges (according to its man page).

Do I understand correctly that this new script is one of the building
blocks that can make the above-summarized "daydream" come true?

Could you please give me a status update on the "daydream"?

Looking forward to hearing back from you.
Thanks for all the good stuff!



Hello again,
a gentle ping about the above-quoted daydream.

Has anyone received my message?
Could someone please reply?

Thanks a lot for your time and patience.


Well.. thanks a lot for *your* patience! Sorry for letting you hang for 
this long. ‍♂️ And thank you for the ping -- I often loose track of 
projects that I'm involved in.


So... mmdebstrap-autopkgtest-build-qemu might be what you want *if* you 
don't care about ppc64el or mips. The latter was (I think) never 
supported my autopkgtest-qemu because there is no good option for a 
bootloader. In case of ppc64el, there is ieee1275 booting but that is 
not supported by mmdebstrap-autopkgtest-build-qemu.


Could you give mmdebstrap-autopkgtest-build-qemu a try and see if it 
does what you want? I'm preparing another mmdebstrap release and if you 
find any issues with that script I can incorporate fixes in the next 
release.


Thanks!

cheers, josch

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Francesco Poli]

On Tue, 31 Oct 2023 11:25:43 +0100 Francesco Poli wrote:

> On Mon, 26 Jun 2023 21:20:15 +0200 Johannes Schauer Marin Rodrigues
> wrote:
> 
> [...]
> > Quoting Francesco Poli (2023-06-26 20:20:47)
> > > If I may share my thoughts (daydreams?!?) about this issue, I was hoping 
> > > to
> > > see a (relatively) simple command able to create a QEMU/KVM image for
> > > autopkgtest without requiring superuser privileges. An image that could be
> > > easily upgraded after creation (without the need to re-create it from
> > > scratch). Bonus, if the image can then be used also for interactive 
> > > testing
> > > of packages and for package building.
> > > 
> > > Basically, I was hoping to see [sbuild-qemu-create] drop the requirement
> > > of superuser privileges (thus dropping the behind-the-scenes use of
> > > vmdb2 and switching to some other backend)...
> > > 
> > > [sbuild-qemu-create]: 
> > > 
> > > I am not sure I clearly understand whether this may happen or whether
> > > this is actually going to happen soon...
> > 
> > this is not a daydream and I think we have nearly all building blocks in 
> > place
> > to make all of this happen very soon!
> [...]
> > Of course once either MR !236 or !237 get merged
> [...]
> > So I don't think it's a daydream.
> [...]
> > So stay tuned! :D
> 
> 
> Hello!   :-)
> 
> I stayed tuned and I saw that mmdebstrap/1.4.0-1 ships a
> mmdebstrap-autopkgtest-build-qemu that does not require superuser
> privileges (according to its man page).
> 
> Do I understand correctly that this new script is one of the building
> blocks that can make the above-summarized "daydream" come true?
> 
> Could you please give me a status update on the "daydream"?
> 
> Looking forward to hearing back from you.
> Thanks for all the good stuff!


Hello again,
a gentle ping about the above-quoted daydream.

Has anyone received my message?
Could someone please reply?

Thanks a lot for your time and patience.



-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgplIkuJzcy6q.pgp
Description: PGP signature

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Francesco Poli]

On Mon, 26 Jun 2023 21:20:15 +0200 Johannes Schauer Marin Rodrigues
wrote:

[...]
> Quoting Francesco Poli (2023-06-26 20:20:47)
> > If I may share my thoughts (daydreams?!?) about this issue, I was hoping to
> > see a (relatively) simple command able to create a QEMU/KVM image for
> > autopkgtest without requiring superuser privileges. An image that could be
> > easily upgraded after creation (without the need to re-create it from
> > scratch). Bonus, if the image can then be used also for interactive testing
> > of packages and for package building.
> > 
> > Basically, I was hoping to see [sbuild-qemu-create] drop the requirement
> > of superuser privileges (thus dropping the behind-the-scenes use of
> > vmdb2 and switching to some other backend)...
> > 
> > [sbuild-qemu-create]: 
> > 
> > I am not sure I clearly understand whether this may happen or whether
> > this is actually going to happen soon...
> 
> this is not a daydream and I think we have nearly all building blocks in place
> to make all of this happen very soon!
[...]
> Of course once either MR !236 or !237 get merged
[...]
> So I don't think it's a daydream.
[...]
> So stay tuned! :D


Hello!   :-)

I stayed tuned and I saw that mmdebstrap/1.4.0-1 ships a
mmdebstrap-autopkgtest-build-qemu that does not require superuser
privileges (according to its man page).

Do I understand correctly that this new script is one of the building
blocks that can make the above-summarized "daydream" come true?

Could you please give me a status update on the "daydream"?

Looking forward to hearing back from you.
Thanks for all the good stuff!


-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgpxDDUrEBwZA.pgp
Description: PGP signature

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Christian Kastner]

On 2023-06-27 00:15, Christian Kastner wrote:
> I just have one question about debvm though (hence Helmut in CC), which
> is beyond my depth: is that kernel direct boot something akin to EFI, or
> BIOS, or of its own kind?
> 
> As I've managed to land on a use case where EFI boot seems to be needed
> (PCI device passthrough), at least according to the various docs I found.

Having slept over this, I realize this isn't a big issue for sbuild-qemu
after all. It's only needed if sbuild-qemu wants to run build-time tests.

That would be a nice-to-have, but we also ship (or will ship) upstream's
extensive test suites as autopkgtests, so we have testing covered, and
that nice-to-have shouldn't be a blocker for anything.

Best,
Christian

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Christian Kastner]

On 2023-06-26 21:20, Johannes Schauer Marin Rodrigues wrote:
> this is not a daydream and I think we have nearly all building blocks in place
> to make all of this happen very soon! Here is a summary:
> 
>  1. You use `debvm-create --arch $foo` to create a filesystem image for any
> architecture supported by QEMU without superuser privileges. The
> debvm-create utility is a thin wrapper around mmdebstrap and passes all 
> the
> right arguments to create this filesystem image.
> 
>  2. You use either autopkgtest-virt-qemu or autopkgtest-virt-ssh (depending on
> whether Simon prefers merge request !236 or !237) to run your autopkgtest
> through qemu with that disk image
> 
>  3. ???
> 
>  4. Profit!

As somebody who uses QEMU images for a lot of things, I'm really glad to
hear this. Finally! Getting rid of root is a such a big win, in many ways.

> If you want to upgrade the disk image you can do so by using `debvm-run` which
> gives you a root shell for that disk image where you can just run "apt
> upgrade".
> 
> Now lets get to your "bonus". Once either MR !236 or !237 is accepted I will
> rewrite sbuild-qemu-create to not call autopkgtest-build-qemu (which uses 
> vmdb2
> which requires root) but debvm-create instead. Similarly, the sbuild-qemu
> script will be adapted to call sbuild with the autopkgtest backend and the
> right option (again depending on whether MR !236 or !237 get merged). And the
> sbuild-qemu-update tool will automate the updating so that you don't have to
> run debvm-run yourself.
> 
> Of course once either MR !236 or !237 get merged, you do not have to wait for
> me to change sbuild-qemu. Even before I have done this work you can always
> manually call sbuild with the autopkgtest backend and do your package builds 
> in
> the QEMU image created by debvm-create without superuser privileges.

For FWIW, as stated earlier I fully support this -- in the sense that
whatever changes are necessary or convenient, just go for it. e.g.: I
chose Python for those scripts, but if Perl is more convenient, just
drop the old stuff. Or if I can help, let me know.

I just have one question about debvm though (hence Helmut in CC), which
is beyond my depth: is that kernel direct boot something akin to EFI, or
BIOS, or of its own kind?

As I've managed to land on a use case where EFI boot seems to be needed
(PCI device passthrough), at least according to the various docs I found.

Best,
Christia

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Francesco Poli (2023-06-26 20:20:47)
> If I may share my thoughts (daydreams?!?) about this issue, I was hoping to
> see a (relatively) simple command able to create a QEMU/KVM image for
> autopkgtest without requiring superuser privileges. An image that could be
> easily upgraded after creation (without the need to re-create it from
> scratch). Bonus, if the image can then be used also for interactive testing
> of packages and for package building.
> 
> Basically, I was hoping to see [sbuild-qemu-create] drop the requirement
> of superuser privileges (thus dropping the behind-the-scenes use of
> vmdb2 and switching to some other backend)...
> 
> [sbuild-qemu-create]: 
> 
> I am not sure I clearly understand whether this may happen or whether
> this is actually going to happen soon...

this is not a daydream and I think we have nearly all building blocks in place
to make all of this happen very soon! Here is a summary:

 1. You use `debvm-create --arch $foo` to create a filesystem image for any
architecture supported by QEMU without superuser privileges. The
debvm-create utility is a thin wrapper around mmdebstrap and passes all the
right arguments to create this filesystem image.

 2. You use either autopkgtest-virt-qemu or autopkgtest-virt-ssh (depending on
whether Simon prefers merge request !236 or !237) to run your autopkgtest
through qemu with that disk image

 3. ???

 4. Profit!

If you want to upgrade the disk image you can do so by using `debvm-run` which
gives you a root shell for that disk image where you can just run "apt
upgrade".

Now lets get to your "bonus". Once either MR !236 or !237 is accepted I will
rewrite sbuild-qemu-create to not call autopkgtest-build-qemu (which uses vmdb2
which requires root) but debvm-create instead. Similarly, the sbuild-qemu
script will be adapted to call sbuild with the autopkgtest backend and the
right option (again depending on whether MR !236 or !237 get merged). And the
sbuild-qemu-update tool will automate the updating so that you don't have to
run debvm-run yourself.

Of course once either MR !236 or !237 get merged, you do not have to wait for
me to change sbuild-qemu. Even before I have done this work you can always
manually call sbuild with the autopkgtest backend and do your package builds in
the QEMU image created by debvm-create without superuser privileges.

So I don't think it's a daydream. Two merge requests exist that provide the
necessary functionality for debvm support and we hopefully only have to go
through a few more iterations with Simon to get either of them into a state
where they can get merged (or in case of !237, shipped by debvm itself).

So stay tuned! :D

cheers, josch

signature.asc
Description: signature

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Francesco Poli]

On Mon, 26 Jun 2023 08:33:09 +0200 Johannes Schauer Marin Rodrigues wrote:

> Francesco, you initially reported this issue. Do you share my conclusion that
> debvm solves the problem you raised about creating autopktest disk images
> without superuser privileges?
> 
> Thanks!

Hi Johannes,
thanks for following up on this bug report!

I really appreciate all the investigation and development work done by
you and others, but I must confess that I somewhat lost track of all
the details.:-p

If I may share my thoughts (daydreams?!?) about this issue, I was hoping
to see a (relatively) simple command able to create a QEMU/KVM image
for autopkgtest without requiring superuser privileges. An image
that could be easily upgraded after creation (without the need to
re-create it from scratch). Bonus, if the image can then be used
also for interactive testing of packages and for package building.

Basically, I was hoping to see [sbuild-qemu-create] drop the requirement
of superuser privileges (thus dropping the behind-the-scenes use of
vmdb2 and switching to some other backend)...

[sbuild-qemu-create]: 

I am not sure I clearly understand whether this may happen or whether
this is actually going to happen soon...



-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgp6NOd08LU3d.pgp
Description: PGP signature

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Johannes Schauer Marin Rodrigues]

Hi Simon,

if you are short on time, you might skip right to the end of this mail and
ignore my other comments as they are not of any high priority since at least
for me, this issue is solved with somehow integrating debvm or adding an
interface that allows using debvm output with autopkgtest.

Quoting Simon McVittie (2022-06-08 11:57:58)
> On Sun, 05 Jun 2022 at 17:04:02 +0200, Johannes Schauer Marin Rodrigues wrote:
> > I am offering to either rewrite autopkgtest-build-qemu with a --method 
> > option
> > defaulting either to "guestfish" if a native image is requested, 
> > "initramfs" if
> > a foreign architecture is requested and "kernel" if the image is neither 
> > i386,
> > amd64, armhf, arm64 or ppc64el. If the user requires a system as close to a
> > real system as possible, they can manually use --method=debian-installer.
> 
> This is extremely comprehensive, but I'm concerned that if we have this
> many code paths for building a qemu VM, at least one of them will be
> broken at any given time. We already have more virtualization backends
> than maintainer bandwidth or routine testing (see my recent efforts to
> get all our tests passing, and I haven't even got as far as qemu yet).
> 
> I appreciate your offer to add more code, but if we add all these code
> paths and document them as something that should work, I'm concerned that
> we'll end up in a situation where we can't remove any of them because
> someone might be relying on them, but we don't actually know that they
> *work* either, and nobody among the autopkgtest maintainers has the time
> or knowledge to fix them when they don't. (This could be mitigated by
> outsourcing as much as possible to some other system builder, like we
> already do to some extent with vmdb2.)

those are very valid concerns. This makes sense.

> As I believe I've already said in an earlier message to this bug report,
> one way to get autopkgtest out of the business of being yet another
> system-building tool is to start from a "cloud" VM image produced by
> someone else (ideally, "have a qemu-runnable cloud image" would be one
> of the jobs for the porting team for each architecture). One approach to
> this is to take a generic cloud image, boot it with qemu and modify it
> to provide the serial ports that autopkgtest-virt-qemu currently needs,
> as autopkgtest-buildvm-ubuntu-cloud does.

My concern with using downloadable cloud images is that what I download from
some server is not verified by the archive GPG keys. I much rather run one of
the existing system-building tools which only downloads data verified by
debian-archive-keyring. But in another threat you already said that you'd like
the input to autopkgutils just being a tarball created by some other tool and
I'd be completely happy with that idea.

> However, the serial-port-based command execution is really quite
> fragile: there are arbitrary timeouts that cause testbed startup to
> fail randomly while under load (which is a large part of why I haven't
> been able to get it systematically passing tests), and it's vulnerable
> to architectures having different conventions for how the virtualized
> "hardware" is represented in the guest user-space. I've tried to reduce
> that problem by using the virtio hypervisor serial consoles /dev/hvc*, but
> as you can see from the ppc64le special case in lib/autopkgtest_qemu.py,
> even that is not completely predictable, because on ppc64le the virtio
> serial consoles start from hvc1.
> 
> I think a less fragile approach than the serial ports would be to use
> ssh, which in practice needs to work anyway, because it's how porters
> typically interact with their test hardware. My goal in refactoring
> part of autopkgtest-virt-qemu into lib/autopkgtest_qemu.py was to get an
> autopkgtest-virt-ssh setup script that runs and manages a qemu VM, using
> cloud-init to provision a trusted public key. Unfortunately, getting that
> tested and working has been delayed by trying to clear enough technical
> debt to make the tests pass on the backends we already have...

Is the argument here, that you are not happy with how autopkgtest-virt-qemu
operates in general and that's an additional reason (probably in connection
with the maintenance burden) why you don't want to add new features to it?

> I think the proliferation of system-building tools that you have noted in
> the past is partly because until recently they have always needed root,
> partly because none of the existing ones is particularly comprehensive,
> and partly because of the number of fiddly details involved in getting
> the correct bootloader for each architecture and poking it into place.
> If we had something analogous to grub-cloud (install on and boot from the
> first disk, no attempt to support dual-booting or special configurations,
> hard-code a serial console, etc.) as part of the minimal requirements
> for supporting an architecture, ideally with a metapackage available on
> all architectures that 

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Simon McVittie]

On Sun, 05 Jun 2022 at 17:04:02 +0200, Johannes Schauer Marin Rodrigues wrote:
> I am offering to either rewrite autopkgtest-build-qemu with a --method option
> defaulting either to "guestfish" if a native image is requested, "initramfs" 
> if
> a foreign architecture is requested and "kernel" if the image is neither i386,
> amd64, armhf, arm64 or ppc64el. If the user requires a system as close to a
> real system as possible, they can manually use --method=debian-installer.

This is extremely comprehensive, but I'm concerned that if we have this
many code paths for building a qemu VM, at least one of them will be
broken at any given time. We already have more virtualization backends
than maintainer bandwidth or routine testing (see my recent efforts to
get all our tests passing, and I haven't even got as far as qemu yet).

I appreciate your offer to add more code, but if we add all these code
paths and document them as something that should work, I'm concerned that
we'll end up in a situation where we can't remove any of them because
someone might be relying on them, but we don't actually know that they
*work* either, and nobody among the autopkgtest maintainers has the time
or knowledge to fix them when they don't. (This could be mitigated by
outsourcing as much as possible to some other system builder, like we
already do to some extent with vmdb2.)

As I believe I've already said in an earlier message to this bug report,
one way to get autopkgtest out of the business of being yet another
system-building tool is to start from a "cloud" VM image produced by
someone else (ideally, "have a qemu-runnable cloud image" would be one
of the jobs for the porting team for each architecture). One approach to
this is to take a generic cloud image, boot it with qemu and modify it
to provide the serial ports that autopkgtest-virt-qemu currently needs,
as autopkgtest-buildvm-ubuntu-cloud does.

However, the serial-port-based command execution is really quite
fragile: there are arbitrary timeouts that cause testbed startup to
fail randomly while under load (which is a large part of why I haven't
been able to get it systematically passing tests), and it's vulnerable
to architectures having different conventions for how the virtualized
"hardware" is represented in the guest user-space. I've tried to reduce
that problem by using the virtio hypervisor serial consoles /dev/hvc*, but
as you can see from the ppc64le special case in lib/autopkgtest_qemu.py,
even that is not completely predictable, because on ppc64le the virtio
serial consoles start from hvc1.

I think a less fragile approach than the serial ports would be to use
ssh, which in practice needs to work anyway, because it's how porters
typically interact with their test hardware. My goal in refactoring
part of autopkgtest-virt-qemu into lib/autopkgtest_qemu.py was to get an
autopkgtest-virt-ssh setup script that runs and manages a qemu VM, using
cloud-init to provision a trusted public key. Unfortunately, getting that
tested and working has been delayed by trying to clear enough technical
debt to make the tests pass on the backends we already have...

I think the proliferation of system-building tools that you have noted in
the past is partly because until recently they have always needed root,
partly because none of the existing ones is particularly comprehensive,
and partly because of the number of fiddly details involved in getting
the correct bootloader for each architecture and poking it into place.
If we had something analogous to grub-cloud (install on and boot from the
first disk, no attempt to support dual-booting or special configurations,
hard-code a serial console, etc.) as part of the minimal requirements
for supporting an architecture, ideally with a metapackage available on
all architectures that pulls in the correct grub-cloud-equivalent, then I
think that would help a lot - but unfortunately grub-cloud is amd64-only.

It also occurs to me that thanks to debootstrap and its clones (including
mmdebstrap), making a non-bootable Debian rootfs tree (in some transport
format like a tarball) is a solved problem; but one of the reasons people
have quality-of-implementation preferences between system-building tools
is that all of them try to implement the entire procedure, starting from
nothing and ending up with a bootable image. Might it help to re-scope
the problem to: start from a non-bootable rootfs tarball as produced by
debootstrap/mmdebstrap, and do whatever architecture-specific things
are necessary to turn it into a bootable disk image with a kernel,
a bootloader, whatever architecture-specific boot partitions are required,
and an expandable root partition populated from the tarball?

> Option 1 is to run guestfish inside another qemu instance. This is
> extremely slow though.

I agree this doesn't seem like a great trade-off.

> Option 2 is to do the partition setup and bootloader installation as an
> initramfs hook script

I think this has 

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Johannes Schauer Marin Rodrigues (2022-05-12 22:16:16)
> On Fri, 08 Nov 2019 23:59:48 +0100 "Francesco Poli (wintermute)" 
>  wrote:
> > Hello and thanks for maintaining autopkgtest!
> > 
> > I wanted to give the QEMU/KVM testbed a try. Hence I tried to create
> > a VM image by using autopkgtest-build-qemu. Its man page states:
> > 
> > [...]
> > |  Note that you need to call this as root.
> > [...]
> > 
> > And indeed the command
> > 
> >   $ autopkgtest-build-qemu unstable ~/var/cache/autopkgtest/sid.img
> > 
> > fails, when issued by a regular user, as it cannot even find parted
> > in the search PATH.
> > 
> > I guess this is because of vmdb2, which requires superuser privileges.
> > But why?
> > Is there any hope to improve vmdb2 or to use another tool, in order
> > to create a KVM testbed without requiring superuser privileges?
> 
> since you reported #944485 against mmdebstrap you already know that the answer
> is "Yes" but I just stumbled across this bug and wanted to share one possible
> solution with the readers here as well.
> 
> How to do it is described in the mmdebstrap man page under "Use as replacement
> for autopkgtest-build-qemu and vmdb2" and the TLDR is:
> 
> $ mmdebstrap --variant=important --include=linux-image-amd64 \
> --customize-hook='chroot "$1" passwd --delete root' \
> --customize-hook='chroot "$1" useradd --home-dir /home/user 
> --create-home user' \
> --customize-hook='chroot "$1" passwd --delete user' \
> --customize-hook='echo host > "$1/etc/hostname"' \
> --customize-hook='echo "127.0.0.1 localhost host" > "$1/etc/hosts"' \
> --customize-hook=/usr/share/autopkgtest/setup-commands/setup-testbed \
> unstable debian-unstable.tar
> $ cat << END > extlinux.conf
> > default linux
> > timeout 0
> >
> > label linux
> > kernel /vmlinuz
> > append initrd=/initrd.img root=/dev/vda1 console=ttyS0
> END
> $ guestfish -N debian-unstable.img=disk:8G -- \
> part-disk /dev/sda mbr : \
> part-set-bootable /dev/sda 1 true : \
> mkfs ext2 /dev/sda1 : mount /dev/sda1 / : \
> tar-in debian-unstable.tar / xattrs:true : \
> extlinux / : copy-in extlinux.conf / : \
> sync : umount / : shutdown
> $ qemu-img convert -O qcow2 debian-unstable.img debian-unstable.qcow2
> 
> Unfortunately, extlinux is only available on amd64 and i386. To support more
> architectures we have to boot using grub. Because this becomes a bit more
> complicated I put the whole thing into a script:
> 
> https://salsa.debian.org/debian/mmdebstrap/-/blob/master/mmdebstrap-autopkgtest-build-qemu
> 
> This now works on amd64, arm64, armhf, i386 and ppc64el. But it only works for
> building *native* qemu images because guestfish cannot handle foreign
> architecture emulation. I see five options that allow avoiding guestfish for
> building foreign architecture bootable images without being root.
> 
> Option 1 is to run guestfish inside another qemu instance. This is extremely
> slow though. Since it's also easiest to set-up I won't go further into this
> option here.
> 
> Option 2 is to do the partition setup and bootloader installation as an
> initramfs hook script:
> 
> main script: http://paste.debian.net/1235312/
> initramfs-hook: http://paste.debian.net/1235313/
> initramfs-script: http://paste.debian.net/1235314/
> 
> Option 3 is to use debian-installer and use pre-seeding to make the whole
> process unattended. I managed to get this working for amd64, i386, arm64 and
> ppc64el. Here is how to do it for ppc64el:
> 
> $ qemu-img create -f qcow2 disk.qcow 4G
> $ curl 
> https://deb.debian.org/debian/dists/bullseye/main/installer-ppc64el/current/images/netboot/debian-installer/ppc64el/initrd.gz
>  > initrd.gz
> $ curl 
> https://deb.debian.org/debian/dists/bullseye/main/installer-ppc64el/current/images/netboot/debian-installer/ppc64el/vmlinux
>  > linux
> $ qemu-system-ppc64le -no-reboot -machine pseries,graphics=off -serial 
> stdio -display none -monitor none -m 1G -initrd initrd.gz -kernel linux 
> -append 'console=hvc0 auto-install/enable=true debconf/priority=critical 
> preseed/url=http://www.debian.org/releases/stable/example-preseed.txt 
> netcfg/get_hostname=hostname netcfg/get_domain=domain 
> passwd/root-password=r00tme passwd/root-password-again=r00tme 
> passwd/user-fullname=user passwd/username=user passwd/user-password=insecure 
> passwd/user-password-again=insecure pkgsel/run_tasksel=false 
> popularity-contest:popularity-contest/participate=false 
> grub-installer/bootdev=/dev/sda1' -hda disk.qcow
> 
> Option 4 is to use u-boot-qemu like this (in case of riscv64):
> 
> $ mmdebstrap --arch=riscv64 --variant=apt \
> --include=linux-image-riscv64,u-boot-menu,systemd-sysv
> --customize-hook='printf "U_BOOT_PARAMETERS=\"rw noquiet 
> root=/dev/vda1\"\nU_BOOT_FDT_DIR=\"noexist\"\n" > "$1"/etc/default/u-boot' \
> 

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Johannes Schauer Marin Rodrigues]

Hi Francesco,

On Fri, 08 Nov 2019 23:59:48 +0100 "Francesco Poli (wintermute)" 
 wrote:
> Hello and thanks for maintaining autopkgtest!
> 
> I wanted to give the QEMU/KVM testbed a try. Hence I tried to create
> a VM image by using autopkgtest-build-qemu. Its man page states:
> 
> [...]
> |  Note that you need to call this as root.
> [...]
> 
> And indeed the command
> 
>   $ autopkgtest-build-qemu unstable ~/var/cache/autopkgtest/sid.img
> 
> fails, when issued by a regular user, as it cannot even find parted
> in the search PATH.
> 
> I guess this is because of vmdb2, which requires superuser privileges.
> But why?
> Is there any hope to improve vmdb2 or to use another tool, in order
> to create a KVM testbed without requiring superuser privileges?

since you reported #944485 against mmdebstrap you already know that the answer
is "Yes" but I just stumbled across this bug and wanted to share one possible
solution with the readers here as well.

How to do it is described in the mmdebstrap man page under "Use as replacement
for autopkgtest-build-qemu and vmdb2" and the TLDR is:

$ mmdebstrap --variant=important --include=linux-image-amd64 \
--customize-hook='chroot "$1" passwd --delete root' \
--customize-hook='chroot "$1" useradd --home-dir /home/user 
--create-home user' \
--customize-hook='chroot "$1" passwd --delete user' \
--customize-hook='echo host > "$1/etc/hostname"' \
--customize-hook='echo "127.0.0.1 localhost host" > "$1/etc/hosts"' \
--customize-hook=/usr/share/autopkgtest/setup-commands/setup-testbed \
unstable debian-unstable.tar
$ cat << END > extlinux.conf
> default linux
> timeout 0
>
> label linux
> kernel /vmlinuz
> append initrd=/initrd.img root=/dev/vda1 console=ttyS0
END
$ guestfish -N debian-unstable.img=disk:8G -- \
part-disk /dev/sda mbr : \
part-set-bootable /dev/sda 1 true : \
mkfs ext2 /dev/sda1 : mount /dev/sda1 / : \
tar-in debian-unstable.tar / xattrs:true : \
extlinux / : copy-in extlinux.conf / : \
sync : umount / : shutdown
$ qemu-img convert -O qcow2 debian-unstable.img debian-unstable.qcow2

Unfortunately, extlinux is only available on amd64 and i386. To support more
architectures we have to boot using grub. Because this becomes a bit more
complicated I put the whole thing into a script:

https://salsa.debian.org/debian/mmdebstrap/-/blob/master/mmdebstrap-autopkgtest-build-qemu

This now works on amd64, arm64, armhf, i386 and ppc64el. But it only works for
building *native* qemu images because guestfish cannot handle foreign
architecture emulation. I see five options that allow avoiding guestfish for
building foreign architecture bootable images without being root.

Option 1 is to run guestfish inside another qemu instance. This is extremely
slow though. Since it's also easiest to set-up I won't go further into this
option here.

Option 2 is to do the partition setup and bootloader installation as an
initramfs hook script:

main script: http://paste.debian.net/1235312/
initramfs-hook: http://paste.debian.net/1235313/
initramfs-script: http://paste.debian.net/1235314/

Option 3 is to use debian-installer and use pre-seeding to make the whole
process unattended. I managed to get this working for amd64, i386, arm64 and
ppc64el. Here is how to do it for ppc64el:

$ qemu-img create -f qcow2 disk.qcow 4G
$ curl 
https://deb.debian.org/debian/dists/bullseye/main/installer-ppc64el/current/images/netboot/debian-installer/ppc64el/initrd.gz
 > initrd.gz
$ curl 
https://deb.debian.org/debian/dists/bullseye/main/installer-ppc64el/current/images/netboot/debian-installer/ppc64el/vmlinux
 > linux
$ qemu-system-ppc64le -no-reboot -machine pseries,graphics=off -serial 
stdio -display none -monitor none -m 1G -initrd initrd.gz -kernel linux -append 
'console=hvc0 auto-install/enable=true debconf/priority=critical 
preseed/url=http://www.debian.org/releases/stable/example-preseed.txt 
netcfg/get_hostname=hostname netcfg/get_domain=domain 
passwd/root-password=r00tme passwd/root-password-again=r00tme 
passwd/user-fullname=user passwd/username=user passwd/user-password=insecure 
passwd/user-password-again=insecure pkgsel/run_tasksel=false 
popularity-contest:popularity-contest/participate=false 
grub-installer/bootdev=/dev/sda1' -hda disk.qcow

Option 4 is to use u-boot-qemu like this (in case of riscv64):

$ mmdebstrap --arch=riscv64 --variant=apt \
--include=linux-image-riscv64,u-boot-menu,systemd-sysv
--customize-hook='printf "U_BOOT_PARAMETERS=\"rw noquiet 
root=/dev/vda1\"\nU_BOOT_FDT_DIR=\"noexist\"\n" > "$1"/etc/default/u-boot' \
--customize-hook='chroot "$1" u-boot-update' \
unstable chroot.tar \
"deb http://deb.debian.org/debian-ports/ sid main" \
"deb http://deb.debian.org/debian-ports/ unreleased main"
$ genext2fs --block-size 1024 --size-in-blocks 2097152 

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Simon McVittie]

On Fri, 08 Nov 2019 at 23:59:48 +0100, Francesco Poli (wintermute) wrote:
> Its man page states:
> |  Note that you need to call this as root.
> 
> I guess this is because of vmdb2, which requires superuser privileges.

As a stopgap solution for this, the latest version in git will try to use
fakemachine(1) if you are not uid 0. fakemachine is currently only available
on amd64 (it has to make non-portable assumptions while building the
initramfs that it will use for the VM).

I think the long-term solution is to bootstrap from an official cloud
image or by running -build-qemu as root, then build any subsequent,
more minimal images by booting a pre-existing VM image and running
vmdb2 inside it (or by running whatever tool we use instead of vmdb2,
if the implementation of -build-qemu changes). This is how I do it in
.

smcv

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Simon McVittie]

On Wed, 20 Nov 2019 at 17:36:39 +0100, Martin Pitt wrote:
> Francesco Poli (wintermute) [2019-11-08 23:59 +0100]:
> > Is there any hope to improve vmdb2 or to use another tool, in order
> > to create a KVM testbed without requiring superuser privileges?
> 
> A much better approach to this would be what autopkgtest-buildvm-ubuntu-cloud
> does, and use a cloud image.

I think there's a place for both options. Starting from a cloud image
definitely seems better for a lowest-common-denominator, "I just want some
sort of testing" requirement set, and I would certainly prefer support
for new architectures to start from "first, make the cloud image work"
rather than always having to know how to build an autopkgtest-specific
bootable image from first principles.

One approach to this is to take a generic cloud image and modify it to
provide the serial ports that autopkgtest-virt-qemu currently needs,
like autopkgtest-buildvm-ubuntu-cloud does.

Another approach that I've been experimenting with is to make use of
the fact that cloud images already include cloud-init and a ssh server,
and use cloud-init to bring up the ssh server with a known public key
accepted. This seems less fragile than injecting commands on serial ports.

What building our own images *does* give us is total control. This is
partly bad, because we have to know all the tricks for making images
bootable on assorted architectures, but also partly good, because it lets
us test the different sorts of image that we want to support (EFI vs BIOS
boot, GPT vs MBR, systemd vs sysv-rc, merged-/usr vs unmerged-/usr).
This is particularly important when there are irreversible conversions
involved (you can go from unmerged /usr to merged /usr, but you can't
go from merged /usr to unmerged /usr).

I think the future I'd prefer to live in is that we *mostly* test on
cloud-derived images, but we also have the ability to build our own
autopkgtest-specific images for at least x86_64, and we can use those
to test the configurations that are not used on cloud images.

If that's how things work, then having autopkgtest-build-qemu be runnable
as non-root becomes a lot less important, because casual users with no
particularly special requirements will be using the cloud images anyway.

Another thing that has occurred to me when working on autopkgtest is
that it seems wasteful that we have so many wrappers around debootstrap,
which requires root (or playing with namespaces like mmdebstrap does,
or some other special action) and is not very fast. I would prefer to
be able to debootstrap a minbase tarball *once*, and then use that same
tarball as a basis for a bootable qemu image, lxc, lxd and Docker root
filesystems, and so on.

smcv

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Martin Pitt]

Hello Francesco,

Francesco Poli (wintermute) [2019-11-08 23:59 +0100]:
> I wanted to give the QEMU/KVM testbed a try. Hence I tried to create
> a VM image by using autopkgtest-build-qemu. Its man page states:
> 
> [...]
> |  Note that you need to call this as root.
> [...]
> 
> And indeed the command
> 
>   $ autopkgtest-build-qemu unstable ~/var/cache/autopkgtest/sid.img
> 
> fails, when issued by a regular user, as it cannot even find parted
> in the search PATH.
> 
> I guess this is because of vmdb2, which requires superuser privileges.
> But why?

Antonio already replied to this part.

> Is there any hope to improve vmdb2 or to use another tool, in order
> to create a KVM testbed without requiring superuser privileges?

A much better approach to this would be what autopkgtest-buildvm-ubuntu-cloud
does, and use a cloud image. Debian has these
(https://cloud.debian.org/images/cloud/), we use them for e. g. Cockpit CI as
well. There aren't any for testing, so you would need to start with stable and
upgrade. Perhaps we can talk the cloud image maintainers into creating images
for testing regularly.

Then adjusting this to an autopkgtest-buildvm-debian-cloud script, or even
generalizing/unifying the two, should not be too hard. Certainly much simpler
than trying to teach vmdb2 to run as normal user.

Martin

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Antonio Terceiro]

Control: tag -1 + confirmed help

On Fri, Nov 08, 2019 at 11:59:48PM +0100, Francesco Poli (wintermute) wrote:
> Package: autopkgtest
> Version: 5.11
> Severity: wishlist
> 
> Hello and thanks for maintaining autopkgtest!
> 
> I wanted to give the QEMU/KVM testbed a try. Hence I tried to create
> a VM image by using autopkgtest-build-qemu. Its man page states:
> 
> [...]
> |  Note that you need to call this as root.
> [...]
> 
> And indeed the command
> 
>   $ autopkgtest-build-qemu unstable ~/var/cache/autopkgtest/sid.img
> 
> fails, when issued by a regular user, as it cannot even find parted
> in the search PATH.
> 
> I guess this is because of vmdb2, which requires superuser privileges.
> But why?
> Is there any hope to improve vmdb2 or to use another tool, in order
> to create a KVM testbed without requiring superuser privileges?

Most disk images creation tools do it by mounting an empty image as a
loop device and then debootstrapping into it, which requires root.

There may be ways of creating an image without root, but to be honest
this is very low on my priority list.  I will take patches though.


signature.asc
Description: PGP signature

Bug#944386: autopkgtest: can autopkgtest-build-qemu create a QEMU/KVM image without requiring superuser privileges?

[Francesco Poli (wintermute)]

Package: autopkgtest
Version: 5.11
Severity: wishlist

Hello and thanks for maintaining autopkgtest!

I wanted to give the QEMU/KVM testbed a try. Hence I tried to create
a VM image by using autopkgtest-build-qemu. Its man page states:

[...]
|  Note that you need to call this as root.
[...]

And indeed the command

  $ autopkgtest-build-qemu unstable ~/var/cache/autopkgtest/sid.img

fails, when issued by a regular user, as it cannot even find parted
in the search PATH.

I guess this is because of vmdb2, which requires superuser privileges.
But why?
Is there any hope to improve vmdb2 or to use another tool, in order
to create a KVM testbed without requiring superuser privileges?



-- System Information:
Debian Release: bullseye/sid
  APT prefers testing
  APT policy: (800, 'testing'), (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.2.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages autopkgtest depends on:
ii  apt-utils   1.8.4
ii  libdpkg-perl1.19.7
ii  procps  2:3.3.15-2+b1
ii  python3 3.7.5-1
ii  python3-debian  0.1.36

Versions of packages autopkgtest recommends:
ii  autodep8  0.19

Versions of packages autopkgtest suggests:
pn  lxc   
pn  lxd   
ii  ovmf  0~20190828.37eef910-3
ii  qemu-efi-aarch64  0~20190828.37eef910-3
ii  qemu-efi-arm  0~20190828.37eef910-3
ii  qemu-system   1:4.1-1+b4
ii  qemu-utils1:4.1-1+b4
pn  schroot   
ii  vmdb2 0.13.2+git20190215-1

-- no debconf information