Bug#1018893: support for unshare in some form
Hi Johannes, On Mon, Sep 05, 2022 at 07:52:13AM +0200, Johannes Schauer Marin Rodrigues wrote: > I just uploaded mmdebstrap 1.2.0 which adds a few more --skip options. The > ones > that might be useful here are: > > - --skip=chroot/mount -- don't mount anything > - --skip=chroot/mount/proc -- only don't mount /proc > - --skip=chroot/mount/sys -- only don't mount /sys > > Maybe this makes implementing this easier? While that may make the combination simpler, the extra copy involved here is making things inefficient. Why would piuparts need two temporary chroots? It should just be using the one mmdebstrap prepared and be happy with that. So I do think we need a new piuparts option that explains how --existing-chroot should work: * Copy the chroot as a template. * Actually use it directly. While we often want the former, in this case, we want the latter. Without an extra option, piuparts will be unable to tell those two cases apart and always use it as a template. Helmut
Bug#1018893: support for unshare in some form
Hi, Quoting Helmut Grohne (2022-09-01 20:31:56) > piuparts has a --existing-chroot option. Unfortunately, it doesn't exactly do > what we need here. It uses the given directory as a template and tries to > copy it. That is bound to fail as mmdebstrap has kindly mounted /sys and > /proc and such. It would be nice if piuparts got some --use-existing option > that would make it just use that chroot directly. I just uploaded mmdebstrap 1.2.0 which adds a few more --skip options. The ones that might be useful here are: - --skip=chroot/mount -- don't mount anything - --skip=chroot/mount/proc -- only don't mount /proc - --skip=chroot/mount/sys -- only don't mount /sys Maybe this makes implementing this easier? Thanks! cheers, josch signature.asc Description: signature
Bug#1018893: support for unshare in some form
Hi Jelmer,
On Thu, Sep 01, 2022 at 03:51:19PM +, Jelmer Vernooij wrote:
> It would be great if piuparts supported root-less operation, ideally in a less
> complicated way than via podman+docker.
>
> Conversation in #debian-qa suggests the are various options for building on
> top of infrastructure that's provided by other packages, e.g. sbuild,
> autopkgtest or mmdebootstrap.
>
> Jelmer, h01ger: I'd second what helmut said. With mmdebstrap you get
> the equivalent of "lxc-usernsexec -- lxc-unshare -s 'MOUNT|PID|UTSNAME|IPC'
> -- /usr/sbin/chroot ./debian-rootfs /bin/bash" but without having to depend
> on lxc -- You can see a variant of this in the mmdebstrap man page where
> mmdebstrap is used as a wrapper of debootstrap to fix #829134. That way you
> can run debootstrap without
> needing root: mmdebstrap --variant=custom --mode=unshare --setup-hook='env
> container=lxc debootstrap unstable "$1"' - debian-debootstrap.tar
Yeah, I think we're 99% there.
piuparts has a --existing-chroot option. Unfortunately, it doesn't
exactly do what we need here. It uses the given directory as a template
and tries to copy it. That is bound to fail as mmdebstrap has kindly
mounted /sys and /proc and such. It would be nice if piuparts got some
--use-existing option that would make it just use that chroot directly.
--use-existing is relatively easy to implement. I'm attaching a patch
for your convenience. I'm not sure whether this is acceptable in
piuparts. I do find the flag, its semantics and its implementation quite
suboptimal. I'd prefer if you use it as inspiration rather than
solution.
So we're doing something like piuparts --existing-chroot=...
--use-existing and this is going to be our --customize-hook for
mmdebstrap. The whole thing is not entirely trivial to assemble, but
this is how it looks:
mmdebstrap \
--verbose \
--mode=unshare \
--variant=apt \
--customize-hook='mv $1/sbin/start-stop-daemon.REAL
$1/sbin/start-stop-daemon && ./piuparts --use-existing --existing-chroot=$1
.../somepackage.changes' \
sid \
/dev/null \
http://deb.debian.org/debian
I suppose the most tricky part is the one about start-stop-daemon. It's
mangled by mmdebstrap for historical reasons. It's a problem, because
piuparts runs debsums and debsums doesn't like that.
So I tried this with a simple package (e.g. buffer) and it passed
completely in an entirely unprivileged way without podman.
Helmut
--- /usr/sbin/piuparts 2021-10-14 15:23:26.0 +0200
+++ ./piuparts 2022-09-01 20:09:26.195314473 +0200
@@ -199,6 +199,7 @@
self.debfoster_options = None
self.docker_image = None
self.merged_usr = False
+self.use_existing = False
# tests and checks
self.no_install_purge_test = False
self.no_upgrade_test = False
@@ -782,7 +783,8 @@
def create(self, temp_tgz=None):
"""Create a chroot according to user's wishes."""
self.panic_handler_id = do_on_panic(self.remove)
-if not settings.schroot and not settings.docker_image:
+if (not settings.schroot and not settings.docker_image and
+not (settings.existing_chroot and settings.use_existing)):
self.create_temp_dir()
if temp_tgz:
@@ -792,7 +794,10 @@
elif settings.lvm_volume:
self.setup_from_lvm(settings.lvm_volume)
elif settings.existing_chroot:
-self.setup_from_dir(settings.existing_chroot)
+if settings.use_existing:
+self.name = settings.existing_chroot
+else:
+self.setup_from_dir(settings.existing_chroot)
elif settings.schroot:
self.setup_from_schroot(settings.schroot)
elif settings.docker_image:
@@ -800,7 +805,8 @@
else:
self.setup_minimal_chroot()
-if not settings.schroot and not settings.docker_image:
+if (not settings.schroot and not settings.docker_image and
+not (settings.existing_chroot and settings.use_existing)):
self.mount_proc()
self.configure_chroot()
@@ -851,7 +857,8 @@
if settings.docker_image:
logging.debug("Destroy docker container '%s'" % self.docker_container)
run(['docker', 'rm', '-f', self.docker_container])
-if not settings.schroot and not settings.docker_image:
+if (not settings.schroot and not settings.docker_image and
+not (settings.existing_chroot and settings.use_existing)):
run(['rm', '-rf', '--one-file-system', self.name])
if os.path.exists(self.name):
create_file(os.path.join(self.name, ".piuparts.tmpdir"), "removal failed")
@@ -2761,6 +2768,8 @@
help="Use DIR as the contents of the initial " +
"chroot, instead of building a new one with " +
Bug#1018893: support for unshare in some form
Package: piuparts Severity: wishlist It would be great if piuparts supported root-less operation, ideally in a less complicated way than via podman+docker. Conversation in #debian-qa suggests the are various options for building on top of infrastructure that's provided by other packages, e.g. sbuild, autopkgtest or mmdebootstrap. Jelmer, h01ger: I'd second what helmut said. With mmdebstrap you get the equivalent of "lxc-usernsexec -- lxc-unshare -s 'MOUNT|PID|UTSNAME|IPC' -- /usr/sbin/chroot ./debian-rootfs /bin/bash" but without having to depend on lxc -- You can see a variant of this in the mmdebstrap man page where mmdebstrap is used as a wrapper of debootstrap to fix #829134. That way you can run debootstrap without needing root: mmdebstrap --variant=custom --mode=unshare --setup-hook='env container=lxc debootstrap unstable "$1"' - debian-debootstrap.tar Alternatively, if you want to depend on neither lxc nor mmdebstrap, a number of tools implemented a simple unshare backend already using code like this: https://salsa.debian.org/debian/sbuild/-/blob/main/lib/Sbuild/Utility.pm#L382 or this: https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/virt/autopkgtest-virt-unshare#L131 re-using the unshare functionality of either mmdebstrap, sbuild or autopkgtest would probably be best there was some discussion whether those three tools could share some code here: https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/138#note_306768 unfortunately i don't see how if somebody wants to work on unshare support for piuparts, feel free to ask me questions about unshare or its implementation in mmdebstrap, sbuild or autopkgtest the other people in the know are smcv and jochensp oh and there is this as a standalone replacement: https://gitlab.mister-muffin.de/josch/user-unshare/src/branch/main/user-unshare -- System Information: Debian Release: bookworm/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing') merged-usr: no Architecture: amd64 (x86_64) Kernel: Linux 5.18.0-4-amd64 (SMP w/2 CPU threads; PREEMPT) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages piuparts depends on: ii debootstrap 1.0.127 pn debsums ii libjs-sphinxdoc 4.5.0-4 ii lsb-release 11.2 ii lsof 4.95.0-1 ii mount2.38.1-1 pn piuparts-common ii python3 3.10.6-1 ii python3-debian 0.1.47 Versions of packages piuparts recommends: pn adequate Versions of packages piuparts suggests: pn docker.io pn schroot
