Package: docker.io
Version: 19.03.4+dfsg2-1
Followup-For: Bug #840829
Hi,
systemd maintainer here.
As you might have heard, Fedora has switched to cgroupv2 in their latest
release [1]. The new upstream default in systemd is unified, i.e.
cgroupv2. While we will revert this change for now, we plan to release
bullseye with cgroupv2 as the default [2], making this bug report RC.
I can confirm, that docker still fails if booted with unified hierarchy
enabled. I used cockpit-docker and got the following error message:
OCI runtime create failed: container_linux.go:346: starting container process
caused "process_linux.go:297: applying cgroup configuration for process caused
\"open /sys/fs/cgroup/docker/cpuset.cpus.effective: no such file or
directory\"": unknown
I'm also attached the journal log from docker.service.
If you want to reproduce the failure, you can enable cgroupv2 by adding
"systemd.unified_cgroup_hierarchy=true" to the kernel command line.
Regards,
Michael
[1] https://fedoraproject.org/wiki/Changes/CGroupsV2
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=943981
-- System Information:
Debian Release: bullseye/sid
APT prefers unstable
APT policy: (500, 'unstable'), (200, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.3.0-1-amd64 (SMP w/4 CPU cores)
Kernel taint flags: TAINT_WARN
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8),
LANGUAGE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages docker.io depends on:
ii adduser 3.118
ii iptables 1.8.3-2
ii libc6 2.29-3
ii libdevmapper1.02.1 2:1.02.155-3
ii libltdl7 2.4.6-11
ii libnspr4 2:4.23-1
ii libnss3 2:3.47-1
ii libseccomp2 2.4.1-2
ii libsystemd0 243-3
ii lsb-base 11.1.0
ii runc 1.0.0~rc9+dfsg1-1
ii tini 0.18.0-1+b1
Versions of packages docker.io recommends:
ii ca-certificates 20190110
pn cgroupfs-mount <none>
ii git 1:2.24.0-1
pn needrestart <none>
ii xz-utils 5.2.4-1+b1
Versions of packages docker.io suggests:
pn aufs-tools <none>
ii btrfs-progs 5.3.1-1
ii debootstrap 1.0.116
pn docker-doc <none>
ii e2fsprogs 1.45.4-1
pn rinse <none>
ii xfsprogs 5.2.1-1
pn zfs-fuse | zfsutils <none>
-- no debconf information
-- Logs begin at Tue 2019-11-05 22:38:05 CET, end at Sat 2019-11-09 02:03:07
CET. --
Nov 09 01:40:47 pluto systemd[1]: Starting Docker Application Container
Engine...
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.665821447+01:00"
level=info msg="Starting up"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.670404791+01:00"
level=info msg="libcontainerd: started new docker-containerd process" pid=1378
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.670480555+01:00"
level=info msg="parsed scheme: \"unix\"" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.670493413+01:00"
level=info msg="scheme \"unix\" not registered, fallback to default scheme"
module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.670519044+01:00"
level=info msg="ccResolverWrapper: sending update to cc:
{[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}"
module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.670539576+01:00"
level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.670647704+01:00"
level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc00038b5c0,
CONNECTING" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.777299292+01:00"
level=info msg="starting containerd"
revision=7c1e88399ec0b0b077121d9d5ad97e647b11c870 version=19.03.4
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.778353477+01:00"
level=info msg="loading plugin \"io.containerd.content.v1.content\"..."
type=io.containerd.content.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.778633132+01:00"
level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..."
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.779236043+01:00"
level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..."
error="path
/var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a
btrfs filesystem to be used with the btrfs snapshotter: skip plugin"
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.779291267+01:00"
level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..."
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.779634957+01:00"
level=warning msg="failed to load plugin
io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.779683781+01:00"
level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..."
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.782129333+01:00"
level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..."
error="modprobe aufs failed: \"modprobe: FATAL: Module aufs not found in
directory /lib/modules/5.3.0-1-amd64\\n\": exit status 1: skip plugin"
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.782181603+01:00"
level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..."
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.783037170+01:00"
level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..."
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.783414488+01:00"
level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..."
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.783661410+01:00"
level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..."
error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs
must be a zfs filesystem to be used with the zfs snapshotter: skip plugin"
type=io.containerd.snapshotter.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.783701093+01:00"
level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..."
type=io.containerd.metadata.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.783752509+01:00"
level=warning msg="could not use snapshotter devmapper in metadata plugin"
error="devmapper not configured"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.783782598+01:00"
level=info msg="metadata content store policy set" policy=shared
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785398695+01:00"
level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..."
type=io.containerd.differ.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785453322+01:00"
level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..."
type=io.containerd.gc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785538670+01:00"
level=info msg="loading plugin
\"io.containerd.service.v1.containers-service\"..."
type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785578268+01:00"
level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..."
type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785634904+01:00"
level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..."
type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785677792+01:00"
level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..."
type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785717044+01:00"
level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..."
type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785753239+01:00"
level=info msg="loading plugin
\"io.containerd.service.v1.namespaces-service\"..."
type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785788370+01:00"
level=info msg="loading plugin
\"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.785850987+01:00"
level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..."
type=io.containerd.runtime.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.786070557+01:00"
level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..."
type=io.containerd.runtime.v2
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.786197266+01:00"
level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..."
type=io.containerd.monitor.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.787581206+01:00"
level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..."
type=io.containerd.service.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.787669849+01:00"
level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..."
type=io.containerd.internal.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.787944881+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788018240+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788066826+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788159315+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788261813+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788362154+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788401268+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788458924+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.788498376+01:00"
level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..."
type=io.containerd.internal.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.789243028+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.789295401+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.789343465+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.789438211+01:00"
level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..."
type=io.containerd.grpc.v1
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.790114358+01:00"
level=info msg=serving...
address=/var/run/docker/containerd/containerd-debug.sock
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.790224519+01:00"
level=info msg=serving...
address=/var/run/docker/containerd/containerd.sock.ttrpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.790303165+01:00"
level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.790330835+01:00"
level=info msg="containerd successfully booted in 0.014156s"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.798118591+01:00"
level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc00038b5c0,
READY" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.836477305+01:00"
level=info msg="parsed scheme: \"unix\"" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.836508528+01:00"
level=info msg="scheme \"unix\" not registered, fallback to default scheme"
module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.836537400+01:00"
level=info msg="ccResolverWrapper: sending update to cc:
{[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}"
module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.836551478+01:00"
level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.836624219+01:00"
level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0008e39b0,
CONNECTING" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.836982515+01:00"
level=info msg="blockingPicker: the picked transport is not ready, loop back to
repick" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.837375646+01:00"
level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0008e39b0,
READY" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.839439113+01:00"
level=info msg="parsed scheme: \"unix\"" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.839473295+01:00"
level=info msg="scheme \"unix\" not registered, fallback to default scheme"
module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.839495934+01:00"
level=info msg="ccResolverWrapper: sending update to cc:
{[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}"
module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.839507168+01:00"
level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.839578878+01:00"
level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0008e3ee0,
CONNECTING" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.840059793+01:00"
level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0008e3ee0,
READY" module=grpc
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.850647524+01:00"
level=info msg="[graphdriver] using prior storage driver: overlay2"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865420495+01:00"
level=warning msg="Your kernel does not support swap memory limit"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865481360+01:00"
level=warning msg="Your kernel does not support memory reservation"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865505729+01:00"
level=warning msg="Your kernel does not support oom control"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865526356+01:00"
level=warning msg="Your kernel does not support memory swappiness"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865547516+01:00"
level=warning msg="Your kernel does not support kernel memory limit"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865576242+01:00"
level=warning msg="Your kernel does not support kernel memory TCP limit"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865599021+01:00"
level=warning msg="Your kernel does not support cgroup cpu shares"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865618295+01:00"
level=warning msg="Your kernel does not support cgroup cfs period"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865642125+01:00"
level=warning msg="Your kernel does not support cgroup cfs quotas"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865659834+01:00"
level=warning msg="Your kernel does not support cgroup rt period"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865682564+01:00"
level=warning msg="Your kernel does not support cgroup rt runtime"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865699174+01:00"
level=warning msg="Your kernel does not support cgroup blkio weight"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865716608+01:00"
level=warning msg="Your kernel does not support cgroup blkio weight_device"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865735111+01:00"
level=warning msg="Your kernel does not support cgroup blkio
throttle.read_bps_device"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865758398+01:00"
level=warning msg="Your kernel does not support cgroup blkio
throttle.write_bps_device"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865779155+01:00"
level=warning msg="Your kernel does not support cgroup blkio
throttle.read_iops_device"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.865796843+01:00"
level=warning msg="Your kernel does not support cgroup blkio
throttle.write_iops_device"
Nov 09 01:40:47 pluto dockerd[1274]: time="2019-11-09T01:40:47.866050587+01:00"
level=info msg="Loading containers: start."
Nov 09 01:40:48 pluto dockerd[1274]: time="2019-11-09T01:40:48.282467305+01:00"
level=info msg="Default bridge (docker0) is assigned with an IP address
172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Nov 09 01:40:48 pluto dockerd[1274]: time="2019-11-09T01:40:48.342147058+01:00"
level=info msg="Loading containers: done."
Nov 09 01:40:48 pluto dockerd[1274]: time="2019-11-09T01:40:48.430530387+01:00"
level=info msg="Docker daemon" commit=9013bf5 graphdriver(s)=overlay2
version=19.03.4
Nov 09 01:40:48 pluto dockerd[1274]: time="2019-11-09T01:40:48.431143384+01:00"
level=info msg="Daemon has completed initialization"
Nov 09 01:40:48 pluto dockerd[1274]: time="2019-11-09T01:40:48.478364115+01:00"
level=info msg="API listen on /run/docker.sock"
Nov 09 01:40:48 pluto systemd[1]: Started Docker Application Container Engine.
Nov 09 01:53:13 pluto dockerd[1274]: time="2019-11-09T01:53:13.254344587+01:00"
level=info msg="shim docker-containerd-shim started"
address=/containerd-shim/7138903035037a5a562734df37524be6f0e70c292e351217bba842573360bfb3.sock
debug=false pid=4855
Nov 09 01:53:13 pluto dockerd[1274]: time="2019-11-09T01:53:13.325347551+01:00"
level=info msg="shim reaped"
id=b515c46f00ec16a31e1bc71349693118c4488aa2ac5b5774a685474d7e7e9bb8
Nov 09 01:53:13 pluto dockerd[1274]: time="2019-11-09T01:53:13.341976089+01:00"
level=error msg="stream copy error: reading from a closed fifo"
Nov 09 01:53:13 pluto dockerd[1274]: time="2019-11-09T01:53:13.436759354+01:00"
level=error
msg="b515c46f00ec16a31e1bc71349693118c4488aa2ac5b5774a685474d7e7e9bb8 cleanup:
failed to delete container from containerd: no such container"
Nov 09 01:53:13 pluto dockerd[1274]: time="2019-11-09T01:53:13.436825232+01:00"
level=error msg="Handler for POST
/v1.12/containers/b515c46f00ec16a31e1bc71349693118c4488aa2ac5b5774a685474d7e7e9bb8/start
returned error: OCI runtime create failed: container_linux.go:346: starting
container process caused \"process_linux.go:297: applying cgroup configuration
for process caused \\\"open /sys/fs/cgroup/docker/cpuset.cpus.effective: no
such file or directory\\\"\": unknown"
