Hi Serge,
> On May 8, 2017, at 11:55 AM, Serge E. Hallyn <se...@hallyn.com> wrote:
>
> Quoting Ben Warren (b...@skyportsystems.com <mailto:b...@skyportsystems.com>):
>> Hi Serge,
>>
>>> On May 4, 2017, at 9:00 AM, Serge E. Hallyn <serge at hallyn.com> wrote:
>>>
>>> Quoting Ben Warren (ben at skyportsystems.com):
>>>> Hi,
>>>>
>>>> I’m stuck with Ubuntu 14.04 for now and would like to be able to run
>>>> unprivileged containers that are systemd-based. I’ve found lots of
>>>> examples of problems that are close, but nothing exactly matches. I got
>>>> the lxc packages from trusty-backports.
>>>>
>>>> Versions:
>>>>
>>>> ben at ben-sc:~$ lxc-ls --version
>>>> 2.0.7
>>>> ben at ben-sc:~$ cat /etc/lsb-release
>>>> DISTRIB_ID=Ubuntu
>>>> DISTRIB_RELEASE=14.04
>>>> DISTRIB_CODENAME=trusty
>>>> DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"
>>>>
>>>> To keep it simple, I created an unprivileged container of ‘trusty’ using
>>>> the download method:
>>>>
>>>> ben at ben-sc:~$ lxc-create -n cd-build -t download
>>>>
>>>>
>>>> When I try to start the container, it won’t work:
>>>>
>>>> ben at ben-sc:~$ lxc-start -n cd-build -d --logfile cd-build.log
>>>> lxc-start: tools/lxc_start.c: main: 366 The container failed to start.
>>>> lxc-start: tools/lxc_start.c: main: 368 To get more details, run the
>>>> container in foreground mode.
>>>> lxc-start: tools/lxc_start.c: main: 370 Additional information can be
>>>> obtained by setting the --logfile and --logpriority options.
>>>>
>>>> Logfile contents:
>>>>
>>>> lxc-start 20170503225525.382 ERROR lxc_cgfsng -
>>>> cgroups/cgfsng.c:do_secondstage_mounts_if_needed:1557 - Operation not
>>>> permitted - Error remounting
>>>> /usr/lib/x86_64-linux-gnu/lxc/sys/fs/cgroup/cpu read-only
>>>
>>> This is odd, not the error I would have expected.
>>>
>>> Can you tell me the exact version and from which ppa?
>>>
>> $ dpkg -s lxc
>> Package: lxc
>> Status: install ok installed
>> Priority: extra
>> Section: oldlibs
>> Installed-Size: 77
>> Maintainer: Ubuntu Developers <ubuntu-devel-disc...@lists.ubuntu.com>
>> Architecture: all
>> Version: 2.0.7-0ubuntu1~14.04.1
>> Depends: lxc1 (>= 2.0.7-0ubuntu1~14.04.1)
>>
>> I got it from here:
>>
>> http://us.archive.ubuntu.com/ubuntu/ trusty-backports
>>
>> Here’s what gets installed:
>>
>> $ sudo apt-get install -t trusty-backports lxc
>> Reading package lists... Done
>> Building dependency tree
>> Reading state information... Done
>> The following extra packages will be installed:
>> bridge-utils cgroup-lite cloud-image-utils debootstrap distro-info euca2ools
>> libgnutls28 libhogweed2 liblxc1 libseccomp2 lxc-common lxc-templates lxc1
>> python-distro-info python-requestbuilder python3-lxc uidmap
>> Suggested packages:
>> shunit2 gnutls-bin btrfs-tools lvm2 lxctl
>> Recommended packages:
>> lxcfs libpam-cgfs
>> The following NEW packages will be installed:
>> bridge-utils cgroup-lite cloud-image-utils debootstrap distro-info euca2ools
>> libgnutls28 libhogweed2 liblxc1 libseccomp2 lxc lxc-common lxc-templates
>> lxc1 python-distro-info python-requestbuilder python3-lxc uidmap
>>
>> As for the overall environment, this is a VM that was originally set up
>> almost 3 years ago, and as a lab machine has only been piecemeal updated
>> over time as needed. The problem is that I have probably a hundred identical
>> instances and am concerned that the package dependencies are maybe not quite
>> right. I’m certainly willing to update whatever individual packages are
>> necessary to get this going. I have the VM snapshotted before trying this,
>> so it’s trivial to reproduce.
>>
>>> Is there anything in syslog about the failed mount?
>>>
>> This is all I see. It’s at lxc install time, now when trying to start the
>> container:
>>
>> May 7 21:01:01 ben-sc kernel: [ 103.486718] type=1400
>> audit(1494216061.420:68): apparmor="STATUS" operation="profile_load"
>> profile="unconfined" name="lxc-container-default" pid=5801
>> comm="apparmor_parser"
>> May 7 21:01:01 ben-sc kernel: [ 103.486925] type=1400
>> audit(1494216061.420:69): apparmor="STATUS" operation="profile_load"
>> profile="unconfined" name="lxc-container-default-cgns" pid=5801
>> comm="apparmor_parser"
>> May 7 21:01:01 ben-sc kernel: [ 103.487100] type=1400
>> audit(1494216061.420:70): apparmor="STATUS" operation="profile_load"
>> profile="unconfined" name="lxc-container-default-with-mounting" pid=5801
>> comm="apparmor_parser"
>> May 7 21:01:01 ben-sc kernel: [ 103.487292] type=1400
>> audit(1494216061.420:71): apparmor="STATUS" operation="profile_load"
>> profile="unconfined" name="lxc-container-default-with-nesting" pid=5801
>> comm="apparmor_parser"
>> May 7 21:01:01 ben-sc kernel: [ 103.519003] type=1400
>> audit(1494216061.452:72): apparmor="STATUS" operation="profile_load"
>> profile="unconfined" name="/usr/bin/lxc-start" pid=5835
>> comm="apparmor_parser"
>>
>>> You might try some of the other cgroup auto-mount settings (see
>>> lxc.container.conf(5)0, maybe
>>>
>>> lxc.mount.auto = cgroup:rw
>>>
>> I tried that, and get:
>>
>> lxc-start 20170508041726.340 ERROR lxc_cgfsng -
>> cgroups/cgfsng.c:do_secondstage_mounts_if_needed:1557 - Operation not
>> permitted - Error remounting /usr/lib/x86_64-linux-gnu/lxc/sys/fs/cgroup/cpu
>> read-only
>> lxc-start 20170508041726.340 ERROR lxc_conf -
>> conf.c:lxc_mount_auto_mounts:839 - Operation not permitted - error mounting
>> /sys/fs/cgroup
>> lxc-start 20170508041726.340 ERROR lxc_conf - conf.c:lxc_setup:3885
>> - failed to setup the automatic mounts for 'cd-build'
>> lxc-start 20170508041726.340 ERROR lxc_start - start.c:do_start:811
>> - Failed to setup container "cd-build".
>> lxc-start 20170508041726.340 ERROR lxc_sync - sync.c:__sync_wait:57
>> - An error occurred in another process (expected sequence number 3)
>> lxc-start 20170508041726.340 ERROR lxc_start -
>> start.c:__lxc_start:1346 - Failed to spawn container "cd-build".
>>
>>>> lxc-start 20170503225525.382 ERROR lxc_conf -
>>>> conf.c:lxc_mount_auto_mounts:839 - Operation not permitted - error
>>>> mounting /sys/fs/cgroup
>>>> lxc-start 20170503225525.382 ERROR lxc_conf - conf.c:lxc_setup:3885
>>>> - failed to setup the automatic mounts for 'cd-build'
>>>> lxc-start 20170503225525.382 ERROR lxc_start - start.c:do_start:811
>>>> - Failed to setup container "cd-build".
>>>> lxc-start 20170503225525.382 ERROR lxc_sync - sync.c:__sync_wait:57
>>>> - An error occurred in another process (expected sequence number 3)
>>>> lxc-start 20170503225525.382 ERROR lxc_start -
>>>> start.c:__lxc_start:1346 - Failed to spawn container "cd-build".
>>>> lxc-start 20170503225530.922 ERROR lxc_start_ui -
>>>> tools/lxc_start.c:main:366 - The container failed to start.
>>>> lxc-start 20170503225530.923 ERROR lxc_start_ui -
>>>> tools/lxc_start.c:main:368 - To get more details, run the container in
>>>> foreground mode.
>>>> lxc-start 20170503225530.923 ERROR lxc_start_ui -
>>>> tools/lxc_start.c:main:370 - Additional information can be obtained by
>>>> setting the --logfile and --logpriority options.
>>>>
>>>> Also:
>>>>
>>>> ————————————
>>>>
>>>> ben at ben-sc:~$ cat /proc/self/cgroup
>>>> 12:name=dsystemd:/
>>>> 11:name=systemd:/user/1001.user/c2.session
>>>> 10:hugetlb:/user/1001.user/c2.session
>>>> 9:perf_event:/user/1001.user/c2.session
>>>> 8:blkio:/user/1001.user/c2.session
>>>> 7:freezer:/user/1001.user/c2.session
>>>> 6:devices:/user/1001.user/c2.session
>>>> 5:memory:/user/1001.user/c2.session
>>>> 4:cpuacct:/user/1001.user/c2.session
>>>> 3:cpu:/user/1001.user/c2.session
>>>> 2:cpuset:/
>
> What the heck is cuasing this? When I log in on a trusty+backports system,
> I get:
>
> ubuntu@trusty:~$ cat /proc/self/cgroup
> 11:hugetlb:/user/1000.user/2.session
> 10:perf_event:/user/1000.user/2.session
> 9:blkio:/user/1000.user/2.session
> 8:freezer:/user/1000.user/2.session
> 7:devices:/user/1000.user/2.session
> 6:memory:/user/1000.user/2.session
> 5:cpuacct:/user/1000.user/2.session
> 4:cpu:/user/1000.user/2.session
> 3:cpuset:/user/1000.user/2.session
> 2:name=systemd:/user/1000.user/2.session
>
> (on second login)
>
> ubuntu@trusty:~$ dpkg -l | egrep -e "(cgroup|lxc|cgfs)"
> ii cgmanager 0.24-0ubuntu7.5
> amd64 Central cgroup manager daemon
> ii cgroup-lite 1.11~ubuntu14.04.2
> all Light-weight package to set up cgroups at system boot
> ii libcgmanager0:amd64 0.24-0ubuntu7.5
> amd64 Central cgroup manager daemon (client library)
> ii liblxc1 2.0.7-0ubuntu1~14.04.1
> amd64 Linux Containers userspace tools (library)
> ii libpam-cgfs 2.0.6-0ubuntu1~14.04.1
> amd64 PAM module for managing cgroups for LXC
> ii lxc 2.0.7-0ubuntu1~14.04.1
> all Transitional package for lxc1
> ii lxc-common 2.0.7-0ubuntu1~14.04.1
> amd64 Linux Containers userspace tools (common tools)
> ii lxc-templates 2.0.7-0ubuntu1~14.04.1
> amd64 Linux Containers userspace tools (templates)
> ii lxc1 2.0.7-0ubuntu1~14.04.1
> amd64 Linux Containers userspace tools
> ii lxcfs 2.0.6-0ubuntu1~14.04.1
> amd64 FUSE based filesystem for LXC
> ii python3-lxc 2.0.7-0ubuntu1~14.04.1
> amd64 Linux Containers userspace tools (Python 3.x bindings)
>
> Ah, now if I purge cgmanager, then upon login I get:
>
> ubuntu@trusty:~$ cat /proc/self/cgroup
> 11:name=systemd:/user/1000.user/1.session
> 10:hugetlb:/user/1000.user/1.session
> 9:perf_event:/user/1000.user/1.session
> 8:blkio:/user/1000.user/1.session
> 7:freezer:/user/1000.user/1.session
> 6:devices:/user/1000.user/1.session
> 5:memory:/user/1000.user/1.session
> 4:cpuacct:/user/1000.user/1.session
> 3:cpu:/user/1000.user/1.session
> 2:cpuset:/
>
> which looks more like yours, but my container still starts...
I’ve made some progress, but still don’t fully know what’s going on. When I
build lxc from source (top-of-tree github.com:lxc/lxc) and compile with full
cgmanager and libcap support, the generated binaries work, and I can start not
only my ‘trusty’ container, but also ones that are farther from the host, such
as ‘delian-stretch’, which is systemd-based.
The difference I see in the log is which cgroup driver is used.
When I build using the binaries from ’trusty-backports’, I see this:
lxc-start 20170509054154.989 INFO lxc_cgroup -
cgroups/cgroup.c:cgroup_init:68 - cgroup driver cgroupfs-ng initing for cd-build
When using the binaries I built from source, I see this:
lxc-start 20170509053256.861 INFO lxc_cgroup -
cgroups/cgroup.c:cgroup_init:68 - cgroup driver cgmanager initing for cd-build
Assuming cgmanager support is compiled in to the ‘trusty-backports’ version,
the following code determines if the cgmanager driver is used (non-NULL return
code means cgmanager is to be used):
struct cgroup_ops *cgm_ops_init(void)
{
check_supports_multiple_controllers(-1);
if (!collect_subsystems())
return NULL;
if (api_version < CGM_SUPPORTS_MULT_CONTROLLERS)
cgm_all_controllers_same = false;
// if root, try to escape to root cgroup
if (geteuid() == 0 && !cgm_escape(NULL)) {
free_subsystems();
return NULL;
}
return &cgmanager_ops;
}
I have no context for how any of this is dependent on the environment, although
I’m sure you do :)
regards,
Ben
_______________________________________________
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
