Bug#949843: sbuild: add systemd-nspawn chroot mode
On Mon, 4 Apr 2022 15:28:01 -0700 Daniel Schepler wrote: > On Mon, Apr 4, 2022 at 3:03 PM Luca Boccassi wrote: > > > > On Sat, 25 Jan 2020 11:36:09 -0800 Daniel Schepler > > wrote: > > > Package: sbuild > > > Version: 0.78.1-2 > > > Severity: wishlist > > > > > > Here's my initial version of the cleaned up patch for adding a > > > --chroot-mode=systemd-nspawn. Some things I'm not sure about: > > > - Should we maybe ping upstream and/or Debian maintainers on > > > https://github.com/systemd/systemd/issues/13297 to see how hard it > > > would be to get it fixed so I could remove that whole ugly > > workaround? > > > (The workaround also only handles bind mount settings at present - > > > and for example, I've found that a lot of package builds will require > > > SystemCallFilter=@memlock due to a lot of crypto libraries and > > > utilities giving errors if they're denied access to mlock. So I > > would > > > probably want to add that to my > > > /etc/systemd/nspawn/unstable-amd64-sbuild.nspawn config file.) > > > > As mentioned on https://github.com/systemd/systemd/issues/13297 adding > > --ephemeral means the machine name has a randomized suffix. Passing -- > > machine=$chroot should ensure the config files are picked up as > > expected. > > OK, if I did that, then would that mean that it's no longer possible > to have two sbuild processes using the same base chroot at the same > time? (Not that that would be too much of an issue in practice. > Though I do admit it's convenient to be able to have my micro_buildd > script running one sbuild instance, while on another terminal I can > run a manual build with e.g. DEB_BUILD_OPTIONS=nocheck sbuild ... > --profiles=nocheck tobootstrap_1.0 .) Ok, sounds like it's worth supporting: https://github.com/systemd/systemd/pull/23110 > > > - It currently requires giving sudo access for systemd-run, which > > > essentially would open up execution of anything desired. And the > > fact > > > that it requires NOPASSWD (because some of the commands redirect > > > stdin/stdout) makes things even worse. And even if you restrict it > > to > > > e.g. "systemd-run -M unstable-amd64-sbuild*" it still seems it would > > > be possible to fool that with something like "sudo systemd-run -M > > > unstable-amd64-sbuild -M .host ~/myevilcmd". > > > > This seems to be used to implement manual synchronization, but this is > > not necessary as it's already implemented, see: > > > > https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html#--notify-ready= > > That's just one use of systemd-run, and a minor one at that. The main > use is to run the commands that sbuild needs to invoke, from "apt-get > update", "apt-get dist-upgrade", "apt-get source package=ver", > "dpkg-source -x filename.dsc", "dpkg-buildpackage" (with > --property=PrivateNetwork=yes on this one), "cat *.changes" into a > pipe, etc. > > And as for synchronization, I think I do remember seeing the > --notify-ready option. But the man page said the notification would > be going to systemd, and I didn't immediately see any way for the Any reason to boot the chroot instead of just running commands in it? That would remove the need for all of this, no? -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
Bug#949843: sbuild: add systemd-nspawn chroot mode
On Mon, Apr 4, 2022 at 3:03 PM Luca Boccassi wrote: > > On Sat, 25 Jan 2020 11:36:09 -0800 Daniel Schepler > wrote: > > Package: sbuild > > Version: 0.78.1-2 > > Severity: wishlist > > > > Here's my initial version of the cleaned up patch for adding a > > --chroot-mode=systemd-nspawn. Some things I'm not sure about: > > - Should we maybe ping upstream and/or Debian maintainers on > > https://github.com/systemd/systemd/issues/13297 to see how hard it > > would be to get it fixed so I could remove that whole ugly > workaround? > > (The workaround also only handles bind mount settings at present - > > and for example, I've found that a lot of package builds will require > > SystemCallFilter=@memlock due to a lot of crypto libraries and > > utilities giving errors if they're denied access to mlock. So I > would > > probably want to add that to my > > /etc/systemd/nspawn/unstable-amd64-sbuild.nspawn config file.) > > As mentioned on https://github.com/systemd/systemd/issues/13297 adding > --ephemeral means the machine name has a randomized suffix. Passing -- > machine=$chroot should ensure the config files are picked up as > expected. OK, if I did that, then would that mean that it's no longer possible to have two sbuild processes using the same base chroot at the same time? (Not that that would be too much of an issue in practice. Though I do admit it's convenient to be able to have my micro_buildd script running one sbuild instance, while on another terminal I can run a manual build with e.g. DEB_BUILD_OPTIONS=nocheck sbuild ... --profiles=nocheck tobootstrap_1.0 .) > > - It currently requires giving sudo access for systemd-run, which > > essentially would open up execution of anything desired. And the > fact > > that it requires NOPASSWD (because some of the commands redirect > > stdin/stdout) makes things even worse. And even if you restrict it > to > > e.g. "systemd-run -M unstable-amd64-sbuild*" it still seems it would > > be possible to fool that with something like "sudo systemd-run -M > > unstable-amd64-sbuild -M .host ~/myevilcmd". > > This seems to be used to implement manual synchronization, but this is > not necessary as it's already implemented, see: > > https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html#--notify-ready= That's just one use of systemd-run, and a minor one at that. The main use is to run the commands that sbuild needs to invoke, from "apt-get update", "apt-get dist-upgrade", "apt-get source package=ver", "dpkg-source -x filename.dsc", "dpkg-buildpackage" (with --property=PrivateNetwork=yes on this one), "cat *.changes" into a pipe, etc. And as for synchronization, I think I do remember seeing the --notify-ready option. But the man page said the notification would be going to systemd, and I didn't immediately see any way for the sbuild parent process to get that notification or to wait for it. -- Daniel Schepler
Bug#949843: sbuild: add systemd-nspawn chroot mode
On Sat, 25 Jan 2020 11:36:09 -0800 Daniel Schepler wrote: > Package: sbuild > Version: 0.78.1-2 > Severity: wishlist > > Here's my initial version of the cleaned up patch for adding a > --chroot-mode=systemd-nspawn. Some things I'm not sure about: > - Should we maybe ping upstream and/or Debian maintainers on > https://github.com/systemd/systemd/issues/13297 to see how hard it > would be to get it fixed so I could remove that whole ugly workaround? > (The workaround also only handles bind mount settings at present - > and for example, I've found that a lot of package builds will require > SystemCallFilter=@memlock due to a lot of crypto libraries and > utilities giving errors if they're denied access to mlock. So I would > probably want to add that to my > /etc/systemd/nspawn/unstable-amd64-sbuild.nspawn config file.) As mentioned on https://github.com/systemd/systemd/issues/13297 adding --ephemeral means the machine name has a randomized suffix. Passing -- machine=$chroot should ensure the config files are picked up as expected. > - It currently requires giving sudo access for systemd-run, which > essentially would open up execution of anything desired. And the fact > that it requires NOPASSWD (because some of the commands redirect > stdin/stdout) makes things even worse. And even if you restrict it to > e.g. "systemd-run -M unstable-amd64-sbuild*" it still seems it would > be possible to fool that with something like "sudo systemd-run -M > unstable-amd64-sbuild -M .host ~/myevilcmd". This seems to be used to implement manual synchronization, but this is not necessary as it's already implemented, see: https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html#--notify-ready= -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
Bug#949843: sbuild: add systemd-nspawn chroot mode
Package: sbuild
Version: 0.78.1-2
Severity: wishlist
Here's my initial version of the cleaned up patch for adding a
--chroot-mode=systemd-nspawn. Some things I'm not sure about:
- Should we maybe ping upstream and/or Debian maintainers on
https://github.com/systemd/systemd/issues/13297 to see how hard it
would be to get it fixed so I could remove that whole ugly workaround?
(The workaround also only handles bind mount settings at present -
and for example, I've found that a lot of package builds will require
SystemCallFilter=@memlock due to a lot of crypto libraries and
utilities giving errors if they're denied access to mlock. So I would
probably want to add that to my
/etc/systemd/nspawn/unstable-amd64-sbuild.nspawn config file.)
- It currently requires giving sudo access for systemd-run, which
essentially would open up execution of anything desired. And the fact
that it requires NOPASSWD (because some of the commands redirect
stdin/stdout) makes things even worse. And even if you restrict it to
e.g. "systemd-run -M unstable-amd64-sbuild*" it still seems it would
be possible to fool that with something like "sudo systemd-run -M
unstable-amd64-sbuild -M .host ~/myevilcmd".
- If you want to ignore the small patch in lib/Sbuild/Chroot.pm that's
fine. It's not really related to the systemd-nspawn chroot mode.
- It does add a dependency on libipc-run-perl.
- It would be nice (as a future enhancement) if it would be possible
to configure this backend to start the container in --network-veth
mode and set up the host's side of the veth to forward only traffic
to/from apt-cacher-ng on localhost:3142. Not sure how hard that would
be to accomplish.
--
Daniel Schepler
Description: Implement systemd-nspawn chroot mode
Adds a chroot mode using systemd-nspawn to start a container to use
for builds.
Author: Daniel Schepler
--- sbuild-0.78.1.orig/lib/Sbuild/Build.pm
+++ sbuild-0.78.1/lib/Sbuild/Build.pm
@@ -53,6 +53,7 @@ use Sbuild::ChrootInfoSchroot;
use Sbuild::ChrootInfoUnshare;
use Sbuild::ChrootInfoSudo;
use Sbuild::ChrootInfoAutopkgtest;
+use Sbuild::ChrootInfoNspawn;
use Sbuild::ChrootRoot;
use Sbuild::Sysconfig qw($version $release_date);
use Sbuild::Sysconfig;
@@ -422,6 +423,8 @@ sub run_chroot_session {
$chroot_info = Sbuild::ChrootInfoAutopkgtest->new($self->get('Config'));
} elsif ($self->get_conf('CHROOT_MODE') eq 'unshare') {
$chroot_info = Sbuild::ChrootInfoUnshare->new($self->get('Config'));
+ } elsif ($self->get_conf('CHROOT_MODE') eq 'systemd-nspawn') {
+ $chroot_info = Sbuild::ChrootInfoNspawn->new($self->get('Config'));
} else {
$chroot_info = Sbuild::ChrootInfoSudo->new($self->get('Config'));
}
--- sbuild-0.78.1.orig/lib/Sbuild/Chroot.pm
+++ sbuild-0.78.1/lib/Sbuild/Chroot.pm
@@ -244,10 +244,8 @@ sub get_read_file_handle {
my $dir = "/";
$dir = $options->{'DIR'} if defined $options->{'DIR'};
-my $escapedsource = shellescape $source;
-
my $pipe = $self->pipe_command({
- COMMAND => [ "sh", "-c", "cat $escapedsource" ],
+ COMMAND => [ "cat", "--", $source ],
DIR => $dir,
USER => $user,
PIPE => 'in'
--- /dev/null
+++ sbuild-0.78.1/lib/Sbuild/ChrootInfoNspawn.pm
@@ -0,0 +1,68 @@
+package Sbuild::ChrootInfoNspawn;
+
+use Sbuild::ChrootInfo;
+use Sbuild::ChrootNspawn;
+
+use strict;
+use warnings;
+
+BEGIN {
+use Exporter ();
+our (@ISA, @EXPORT);
+
+@ISA=qw(Exporter Sbuild::ChrootInfo);
+
+@EXPORT = qw();
+}
+
+sub new {
+my $class = shift;
+my $conf = shift;
+
+my $self = $class->SUPER::new($conf);
+bless($self, $class);
+
+return $self;
+}
+
+sub get_info_all {
+my $self = shift;
+
+my $chroots = {};
+
+open CHROOTS, '-|', $self->get_conf('MACHINECTL'), 'list-images'
+ or die 'Can\'t run machinectl list-images';
+
+# skip header line
+;
+
+while (($_ = ) ne "\n") {
+ chomp;
+ my @fields = split /\s+/, $_;
+ my $chroot = $fields[0];
+ $chroots->{'chroot'}->{$chroot} = 1;
+ $chroots->{'source'}->{$chroot} = 1;
+}
+
+# skip "N images listed"
+;
+
+close CHROOTS or die "Can't close machinectl list-images pipe";
+
+$self->set('Chroots', $chroots);
+}
+
+sub _create {
+my $self = shift;
+my $chroot_id = shift;
+
+my $chroot = undef;
+
+if (defined($chroot_id)) {
+ $chroot = Sbuild::ChrootNspawn->new($self->get('Config'), $chroot_id);
+}
+
+return $chroot;
+}
+
+1;
--- /dev/null
+++ sbuild-0.78.1/lib/Sbuild/ChrootNspawn.pm
@@ -0,0 +1,238 @@
+package Sbuild::ChrootNspawn;
+
+use strict;
+use warnings;
+
+use IPC::Run qw(run start finish);
+use IO::Handle;
+use File::Basename qw(basename);
+
+BEGIN {
+use Exporter ();
+use Sbuild::Chroot;
+our (@ISA, @EXPORT);
+
+@ISA = qw(Exporter Sbuild::Chroot);
+
+@EXPORT = qw();
+}
+
+sub new {
+my $class = shift;
+my $conf = shift;
+my $chroot_id = shift;
+
+my $self = $class->SUPER::new($conf, $chroot_id);
+bless($self, $class);
+
+
