Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Hi, Pierre-Elliott Bécue: > See the two latest commits for lxc: > https://salsa.debian.org/lxc-team/lxc/commits/master > Tell me what you think about them, Sweet, thanks! :) > and if needed don't hesitate to submit a MR! :) There you go: https://salsa.debian.org/lxc-team/lxc/merge_requests/4 Cheers, -- intrigeri
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Le dimanche 31 mars 2019 à 14:55:52+0200, intrigeri a écrit : > Hi, > > Regis Smith: > >> > lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use > > generated profile: apparmor_parser not available > > I've reproduced this problem and I could fix it with: > > lxc.apparmor.profile = unconfined > > Regis, can you please confirm this fix works for you as well? > > Pierre-Elliott Bécue: > > Cc-ing intrigeri: I'm reconsidering the /etc/lxc/default.conf setting > > regarding apparmor.profile. Putting generated breaks many unpriv > > containers as they have no apparmor.profile set in their configuration. > > Considering kernel.unprivileged_userns_clone is disabled by default > on Debian, IMO we should: > > - Optimize for the Debian defaults, i.e. privileged containers: > - Keep the settings we added recently in /etc/lxc/default.conf > - Replace "Suggests: apparmor" with "Depends: apparmor", because > the default config will create containers that fail to start > if the apparmor package is not installed. > > - Document how to use unprivileged containers on Debian. It's not as >if they were previously working fine by default and AppArmor broke >them — regardless of AppArmor, on current sid with the default >kernel settings and lxc.apparmor.profile = unconfined, trying to >start an unprivileged container fails in a very much user >unfriendly way: > > conf.c: chown_mapped_root: 3250 lxc-usernsexec failed: Permission denied > - Failed to open tt > >That's a first usability stumbling block. The new >lxc.apparmor.profile default setting merely adds a second one. > >So I think README.Debian should document the need for >kernel.unprivileged_userns_clone=1 and for >lxc.apparmor.profile = unconfined > > - Take care of the Stretch→Buster upgrade path for unprivileged >containers, by mentioning in NEWS.Debian that previously working >unprivileged containers now need lxc.apparmor.profile = unconfined. > > Thoughts? See the two latest commits for lxc: https://salsa.debian.org/lxc-team/lxc/commits/master Tell me what you think about them, and if needed don't hesitate to submit a MR! :) -- Pierre-Elliott Bécue GPG: 9AE0 4D98 6400 E3B6 7528 F493 0D44 2664 1949 74E2 It's far easier to fight for one's principles than to live up to them. signature.asc Description: PGP signature
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
On Sun, 31 Mar 2019 14:55:52 +0200 intrigeri wrote: > Hi, > > Regis Smith: > >> > lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use > > generated profile: apparmor_parser not available > > I've reproduced this problem and I could fix it with: > > lxc.apparmor.profile = unconfined > > Regis, can you please confirm this fix works for you as well? > Yes, this works. Thanks! This was the old solution required in Jessie or early Stretch (w/o backports), so people may still complain after Buster is released :) But I'm happpy if I can reliably start unprivileged containers again, which "unconfined" does. Regis
Bug#925899: [pkg-lxc-devel] Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Unprivileged containers is the only thing I use in Stretch atm, so call me biased. But it's also the thing that does not work out of the box (you need a sysctl, you need to add the user to lxc-net config and you need to adjust the users lxc.conf for the uid map). As such, I'd say having to set another config var is not a big deal as long it's properly documented and announced in NEWS.Debian. My 0.02€ On 31 March 2019 16:11:53 CEST, "Pierre-Elliott Bécue" wrote: >Le dimanche 31 mars 2019 à 14:55:52+0200, intrigeri a écrit : >> Pierre-Elliott Bécue: >> > Cc-ing intrigeri: I'm reconsidering the /etc/lxc/default.conf >setting >> > regarding apparmor.profile. Putting generated breaks many unpriv >> > containers as they have no apparmor.profile set in their >configuration. >> >> Considering kernel.unprivileged_userns_clone is disabled by default >> on Debian, IMO we should: >> >> - Optimize for the Debian defaults, i.e. privileged containers: >> - Keep the settings we added recently in /etc/lxc/default.conf >> - Replace "Suggests: apparmor" with "Depends: apparmor", because >> the default config will create containers that fail to start >> if the apparmor package is not installed. >> >> - Document how to use unprivileged containers on Debian. It's not as >>if they were previously working fine by default and AppArmor broke >>them — regardless of AppArmor, on current sid with the default >>kernel settings and lxc.apparmor.profile = unconfined, trying to >>start an unprivileged container fails in a very much user >>unfriendly way: >> >> conf.c: chown_mapped_root: 3250 lxc-usernsexec failed: >Permission denied - Failed to open tt >> >>That's a first usability stumbling block. The new >>lxc.apparmor.profile default setting merely adds a second one. >> >>So I think README.Debian should document the need for >>kernel.unprivileged_userns_clone=1 and for >>lxc.apparmor.profile = unconfined > >Yes and no, the issue is that we introduced a new problem which made >the >unpriviledged containers of user having already set the kernel properly >not working either. > >There is a tradeoff in your current solution, and I wonder if it is the >appropriate one. > >I'm Cc-ing Antonio and Evgeni to get their POV. > >> - Take care of the Stretch→Buster upgrade path for unprivileged >>containers, by mentioning in NEWS.Debian that previously working >>unprivileged containers now need lxc.apparmor.profile = >unconfined. > >The apparmor.profile = lxc-container-default-cgns works too, but will >prevent systemd isolation features to work. > >-- >Pierre-Elliott Bécue >GPG: 9AE0 4D98 6400 E3B6 7528 F493 0D44 2664 1949 74E2 >It's far easier to fight for one's principles than to live up to them. > >___ >Pkg-lxc-devel mailing list >pkg-lxc-de...@alioth-lists.debian.net >https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-lxc-devel
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Le dimanche 31 mars 2019 à 14:55:52+0200, intrigeri a écrit : > Pierre-Elliott Bécue: > > Cc-ing intrigeri: I'm reconsidering the /etc/lxc/default.conf setting > > regarding apparmor.profile. Putting generated breaks many unpriv > > containers as they have no apparmor.profile set in their configuration. > > Considering kernel.unprivileged_userns_clone is disabled by default > on Debian, IMO we should: > > - Optimize for the Debian defaults, i.e. privileged containers: > - Keep the settings we added recently in /etc/lxc/default.conf > - Replace "Suggests: apparmor" with "Depends: apparmor", because > the default config will create containers that fail to start > if the apparmor package is not installed. > > - Document how to use unprivileged containers on Debian. It's not as >if they were previously working fine by default and AppArmor broke >them — regardless of AppArmor, on current sid with the default >kernel settings and lxc.apparmor.profile = unconfined, trying to >start an unprivileged container fails in a very much user >unfriendly way: > > conf.c: chown_mapped_root: 3250 lxc-usernsexec failed: Permission denied > - Failed to open tt > >That's a first usability stumbling block. The new >lxc.apparmor.profile default setting merely adds a second one. > >So I think README.Debian should document the need for >kernel.unprivileged_userns_clone=1 and for >lxc.apparmor.profile = unconfined Yes and no, the issue is that we introduced a new problem which made the unpriviledged containers of user having already set the kernel properly not working either. There is a tradeoff in your current solution, and I wonder if it is the appropriate one. I'm Cc-ing Antonio and Evgeni to get their POV. > - Take care of the Stretch→Buster upgrade path for unprivileged >containers, by mentioning in NEWS.Debian that previously working >unprivileged containers now need lxc.apparmor.profile = unconfined. The apparmor.profile = lxc-container-default-cgns works too, but will prevent systemd isolation features to work. -- Pierre-Elliott Bécue GPG: 9AE0 4D98 6400 E3B6 7528 F493 0D44 2664 1949 74E2 It's far easier to fight for one's principles than to live up to them.
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Hi, Regis Smith: >> > lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use > generated profile: apparmor_parser not available I've reproduced this problem and I could fix it with: lxc.apparmor.profile = unconfined Regis, can you please confirm this fix works for you as well? Pierre-Elliott Bécue: > Cc-ing intrigeri: I'm reconsidering the /etc/lxc/default.conf setting > regarding apparmor.profile. Putting generated breaks many unpriv > containers as they have no apparmor.profile set in their configuration. Considering kernel.unprivileged_userns_clone is disabled by default on Debian, IMO we should: - Optimize for the Debian defaults, i.e. privileged containers: - Keep the settings we added recently in /etc/lxc/default.conf - Replace "Suggests: apparmor" with "Depends: apparmor", because the default config will create containers that fail to start if the apparmor package is not installed. - Document how to use unprivileged containers on Debian. It's not as if they were previously working fine by default and AppArmor broke them — regardless of AppArmor, on current sid with the default kernel settings and lxc.apparmor.profile = unconfined, trying to start an unprivileged container fails in a very much user unfriendly way: conf.c: chown_mapped_root: 3250 lxc-usernsexec failed: Permission denied - Failed to open tt That's a first usability stumbling block. The new lxc.apparmor.profile default setting merely adds a second one. So I think README.Debian should document the need for kernel.unprivileged_userns_clone=1 and for lxc.apparmor.profile = unconfined - Take care of the Stretch→Buster upgrade path for unprivileged containers, by mentioning in NEWS.Debian that previously working unprivileged containers now need lxc.apparmor.profile = unconfined. Thoughts? Cheers, -- intrigeri
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Hi! Thanks for replying. On Sat, 30 Mar 2019 14:51:47 +0100 Pierre-Elliott =?utf-8?B?QsOpY3Vl?= wrote: > Le mercredi 27 mars 2019 à 22:08:49-0700, Regis Smith a écrit : > > Package: lxc > > Version: 1:3.1.0+really3.0.3-6 > > Severity: important > > > > Dear Maintainer, > > > >* What led up to the situation? > > > > apt update; apt upgrade > > > >* What exactly did you do (or not do) that was effective (or > > ineffective)? > > > > As a normal user: > > $ lxc-start -n test > > > >* What was the outcome of this action? > > > > lxc-start: test: lxccontainer.c: wait_on_daemonized_start: 833 No such file or directory - Failed to receive the container state > > lxc-start: test: tools/lxc_start.c: main: 330 The container failed to start > > lxc-start: test: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode > > lxc-start: test: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options > > > > If I run it in the foreground instead I get > > > > $ lxc-start -n test -F > > lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use generated profile: apparmor_parser not available > > lxc-start: test: start.c: lxc_init: 899 Failed to initialize LSM > > lxc-start: test: start.c: __lxc_start: 1917 Failed to initialize container "test" > > lxc-start: test: tools/lxc_start.c: main: 330 The container failed to start > > lxc-start: test: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options > > > >* What outcome did you expect instead? > > > > A running container. These used to work up until recently. Now I can't stop > > already running containers because I won't be able to restart them. > > Hi, > > Thanks for submitting this bug. > > As you can see, it is possible to get more debug via the --logfile and > the --logpriority options. > > That said, the first line with the -F option says it all: > > > lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use > > generated profile: apparmor_parser not available > > It means that you're lacking the apparmor_parser command, which is > shipped by apparmor. It probably means that you refused to install > apparmor on your host. Actually, I do have apparmor installed, and I can run apparmor_parser as root. aa-status shows all the related "lxc-container-*" in enforce mode. Priveleged containers work fine, but I can not start unprivileged containers. Both privileged and unpriveleged worked fine before the updates over the past several weeks. > > You have multiple choices. The first one being installing apparmor, and > the second one being to edit your container's configuration (or the > /etc/lxc/default.conf file) to change the lxc.apparmor.profile > parameter. > > This bugreport raises an interesting question regarding the tradeoff I attached the log from running $ lxc-start -n test --logpriority DEBUG --logfile lxc.log I commented out "apparmor.profile = generated" and it still doesn't work. I'd like to get this working with apparmor, since it's the default. However, I'd love to hear from anyone who has unprivileged containers working on an up-to-date Buster. The fickleness of LXC in Stretch wore me out, so I was quite pleased when it worked reliably in Buster, up until now. Regis lxc-start test 20190330180301.167 INFO confile - confile.c:set_config_idmaps:1605 - Read uid map: type u nsid 0 hostid 10 range 65536 lxc-start test 20190330180301.167 INFO confile - confile.c:set_config_idmaps:1605 - Read uid map: type g nsid 0 hostid 10 range 65536 lxc-start test 20190330180301.168 INFO lxccontainer - lxccontainer.c:do_lxcapi_start:961 - Set process title to [lxc monitor] /home/rsmith/.local/share/lxc test lxc-start test 20190330180301.168 INFO lsm - lsm/lsm.c:lsm_init:50 - LSM security driver AppArmor lxc-start test 20190330180301.169 INFO seccomp - seccomp.c:parse_config_v2:759 - Processing "reject_force_umount # comment this to allow umount -f; not recommended" lxc-start test 20190330180301.169 INFO seccomp - seccomp.c:do_resolve_add_rule:505 - Set seccomp rule to reject force umounts lxc-start test 20190330180301.169 INFO seccomp - seccomp.c:parse_config_v2:937 - Added native rule for arch 0 for reject_force_umount action 0(kill) lxc-start test 20190330180301.169 INFO seccomp - seccomp.c:do_resolve_add_rule:505 - Set seccomp rule to reject force umounts lxc-start test 20190330180301.169 INFO seccomp - seccomp.c:parse_config_v2:946 - Added compat rule for arch 1073741827 for reject_force_umount action 0(kill) lxc-start test 20190330180301.169 INFO seccomp - seccomp.c:do_resolve_add_rule:505 - Set seccomp rule to reject force umounts lxc-start test 20190330180301.169 INFO seccomp - seccomp.c:parse_config_v2:956 - Added compat rule for arch 1073741886 for reject_force_umount action 0(kill) lxc-start
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Le 30 mars 2019 15:29:52 GMT+01:00, intrigeri a écrit : >Hi, > >Pierre-Elliott Bécue: >> This bugreport raises an interesting question regarding the tradeoff >> between the solution we implemented to fix bug #916639. > >> Cc-ing intrigeri: I'm reconsidering the /etc/lxc/default.conf setting >> regarding apparmor.profile. Putting generated breaks many unpriv >> containers as they have no apparmor.profile set in their >configuration. > >I'd love to help but I'll need more info to understand why the current >setup breaks "many unpriv containers", e.g.: > > - Is this specific to unprivileged containers? > > - Is it because "apparmor.profile = generated" is not suitable > for unprivileged containers? > >Finally, I wonder if "Suggests: apparmor" expresses strongly enough >the current status of the LXC + AppArmor integration in Debian. >Thankfully the Linux images will pull apparmor via Recommends… >except on systems where the administrator has disabled installation >of Recommends. > >Cheers, It is specific to unpriviledged containers and due to the fact that non root users don't seem to have the ability to use the generated profile. PEB (from my phone)
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Hi, Pierre-Elliott Bécue: > This bugreport raises an interesting question regarding the tradeoff > between the solution we implemented to fix bug #916639. > Cc-ing intrigeri: I'm reconsidering the /etc/lxc/default.conf setting > regarding apparmor.profile. Putting generated breaks many unpriv > containers as they have no apparmor.profile set in their configuration. I'd love to help but I'll need more info to understand why the current setup breaks "many unpriv containers", e.g.: - Is this specific to unprivileged containers? - Is it because "apparmor.profile = generated" is not suitable for unprivileged containers? Finally, I wonder if "Suggests: apparmor" expresses strongly enough the current status of the LXC + AppArmor integration in Debian. Thankfully the Linux images will pull apparmor via Recommends… except on systems where the administrator has disabled installation of Recommends. Cheers, -- intrigeri
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Le mercredi 27 mars 2019 à 22:08:49-0700, Regis Smith a écrit : > Package: lxc > Version: 1:3.1.0+really3.0.3-6 > Severity: important > > Dear Maintainer, > >* What led up to the situation? > > apt update; apt upgrade > >* What exactly did you do (or not do) that was effective (or > ineffective)? > > As a normal user: > $ lxc-start -n test > >* What was the outcome of this action? > > lxc-start: test: lxccontainer.c: wait_on_daemonized_start: 833 No such file > or directory - Failed to receive the container state > lxc-start: test: tools/lxc_start.c: main: 330 The container failed to start > lxc-start: test: tools/lxc_start.c: main: 333 To get more details, run the > container in foreground mode > lxc-start: test: tools/lxc_start.c: main: 336 Additional information can be > obtained by setting the --logfile and --logpriority options > > If I run it in the foreground instead I get > > $ lxc-start -n test -F > lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use generated > profile: apparmor_parser not available > lxc-start: test: start.c: lxc_init: 899 Failed to initialize LSM > lxc-start: test: start.c: __lxc_start: 1917 Failed to initialize container > "test" > lxc-start: test: tools/lxc_start.c: main: 330 The container failed to start > lxc-start: test: tools/lxc_start.c: main: 336 Additional information can be > obtained by setting the --logfile and --logpriority options > >* What outcome did you expect instead? > > A running container. These used to work up until recently. Now I can't stop > already running containers because I won't be able to restart them. Hi, Thanks for submitting this bug. As you can see, it is possible to get more debug via the --logfile and the --logpriority options. That said, the first line with the -F option says it all: > lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use > generated profile: apparmor_parser not available It means that you're lacking the apparmor_parser command, which is shipped by apparmor. It probably means that you refused to install apparmor on your host. You have multiple choices. The first one being installing apparmor, and the second one being to edit your container's configuration (or the /etc/lxc/default.conf file) to change the lxc.apparmor.profile parameter. This bugreport raises an interesting question regarding the tradeoff between the solution we implemented to fix bug #916639. Cc-ing intrigeri: I'm reconsidering the /etc/lxc/default.conf setting regarding apparmor.profile. Putting generated breaks many unpriv containers as they have no apparmor.profile set in their configuration. [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916639 -- Pierre-Elliott Bécue GPG: 9AE0 4D98 6400 E3B6 7528 F493 0D44 2664 1949 74E2 It's far easier to fight for one's principles than to live up to them. signature.asc Description: PGP signature
Bug#925899: lxc: Unprivileged containers fail to start after recent updates
Package: lxc Version: 1:3.1.0+really3.0.3-6 Severity: important Dear Maintainer, * What led up to the situation? apt update; apt upgrade * What exactly did you do (or not do) that was effective (or ineffective)? As a normal user: $ lxc-start -n test * What was the outcome of this action? lxc-start: test: lxccontainer.c: wait_on_daemonized_start: 833 No such file or directory - Failed to receive the container state lxc-start: test: tools/lxc_start.c: main: 330 The container failed to start lxc-start: test: tools/lxc_start.c: main: 333 To get more details, run the container in foreground mode lxc-start: test: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options If I run it in the foreground instead I get $ lxc-start -n test -F lxc-start: test: lsm/apparmor.c: apparmor_prepare: 974 Cannot use generated profile: apparmor_parser not available lxc-start: test: start.c: lxc_init: 899 Failed to initialize LSM lxc-start: test: start.c: __lxc_start: 1917 Failed to initialize container "test" lxc-start: test: tools/lxc_start.c: main: 330 The container failed to start lxc-start: test: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options * What outcome did you expect instead? A running container. These used to work up until recently. Now I can't stop already running containers because I won't be able to restart them. -- System Information: Debian Release: buster/sid APT prefers testing APT policy: (500, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 4.19.0-4-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages lxc depends on: ii debconf [debconf-2.0] 1.5.71 ii libc6 2.28-8 ii libcap21:2.25-2 ii libgnutls303.6.6-2 ii liblxc11:3.1.0+really3.0.3-6 ii libseccomp22.3.3-4 ii libselinux12.8-1+b1 ii lsb-base 10.2019031300 Versions of packages lxc recommends: ii bridge-utils 1.6-2 ii debootstrap 1.0.114 ii dirmngr 2.2.12-1 ii dnsmasq-base [dnsmasq-base] 2.80-1 ii gnupg2.2.12-1 ii iproute2 4.20.0-2 ii iptables 1.8.2-4 ii libpam-cgfs 1:3.1.0+really3.0.3-6 ii lxc-templates3.0.3-1 ii lxcfs3.0.3-2 ii nftables 0.9.0-2 ii openssl 1.1.1b-1 ii rsync3.1.3-6 ii uidmap 1:4.5-1.1 Versions of packages lxc suggests: ii apparmor 2.13.2-9 ii btrfs-progs 4.20.1-2 ii lvm2 2.03.02-2 pn python3-lxc -- Configuration Files: /etc/lxc/default.conf changed: lxc.net.0.type = veth lxc.net.0.link = lxcbr0 lxc.net.0.flags = up lxc.apparmor.profile = generated lxc.apparmor.allow_nesting = 1 -- debconf information: * lxc/auto_update_config: true
