Bug#887787: lxc: CentOS 7 amd64 container can't be stopped

[Mathias Gibbens]

Control: tags -1 + moreinfo

Hi Toni,

  Sorry that no one responded to this bug until now. Is this behavior
still happening on a bookworm or sid system? I've just spent some time
testing various containers, including a CentOS 7 one, and they are
properly shutting down, both internally (`shutdown` and `halt`) as well
as via `lxc-stop`.

On Fri, 19 Jan 2018 22:51:13 +0100 Toni Mueller 
wrote:
> Trying to 'lxc-stop -n centos' (name of the container) also does not
> work. [snip] It would be great if lxc could prevent a container
> misbehaving like that.

  `lxc-stop` does have the "--timeout" and "--kill" options which can
be used to force stop a container that doesn't gracefully shutdown.

Mathias


signature.asc
Description: This is a digitally signed message part

Bug#887787: lxc: CentOS 7 amd64 container can't be stopped

[Toni Mueller]

Hi,

the problem is not limited to CentOS. I just had a Debian container lock
up the same way, on the same host.


Cheers,
Toni

Bug#887787: lxc: CentOS 7 amd64 container can't be stopped

[Toni Mueller]

Package: lxc
Version: 1:2.0.7-2+deb9u1
Severity: normal


Hi,


I installed an unprivileged CentOS 7 container with

$ lxc-create -n centos -t download

after setting my system up according to the instructions given here:

https://wiki.debian.org/LXC


The resulting container starts as expected, but any attempt to shut it
down again, fails. The container remains responsive, but just does not
stop. Inside the container, the process list looks as follows:

$ lxc-attach -n centos
bash-4.2# ps auwwx
USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
root 1  0.0  0.0  42544  2332 ?Ds   Jan20   0:00 /sbin/init
root   132  0.0  0.1 113380 13416 ?Ss   Jan20   0:00 /sbin/dhclient 
-1 -q -lf /var/lib/dhclient/dhclient--eth0.lease -pf /var/run/dhclient-eth0.pid 
-H centos eth0
root   191  0.0  0.0 115396  3160 ?Ss   Jan20   0:00 /bin/bash
root   192  0.0  0.0 107912   616 ?D+   Jan20   0:00 sync
root   193  0.0  0.0 115396  3064 ?Ss   Jan20   0:00 /bin/bash
root   194  0.0  0.0 151072  3564 ?R+   Jan20   0:00 ps auwwx
bash-4.2# 


In a different shell, things look like this:

$ lxc-attach -n centos
bash-4.2# halt -n -f
c^C^Z^C


(ie, no reaction)


In the kernel log of the host, I get a lot of these:


Jan 19 22:20:37 debian kernel: [39269.678133] INFO: task systemd:6381 blocked 
for more than 120 seconds.
Jan 19 22:20:37 debian kernel: [39269.678144]   Tainted: G   O
4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2
Jan 19 22:20:37 debian kernel: [39269.678149] "echo 0 > 
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 22:20:37 debian kernel: [39269.678154] systemd D0  6381   
6371 0x0104
Jan 19 22:20:37 debian kernel: [39269.678163]  976453d38000 
976553415800 976504ffafc0 97655fc98940
Jan 19 22:20:37 debian kernel: [39269.678170]  976500564480 
a7a7040b3bc0 a6802923 976504ffafc0
Jan 19 22:20:37 debian kernel: [39269.678176]  00ff976504ffafc0 
97655fc98940 976504161dc0 976504ffafc0
Jan 19 22:20:37 debian kernel: [39269.678181] Call Trace:
Jan 19 22:20:37 debian kernel: [39269.678194]  [] ? 
__schedule+0x233/0x6d0
Jan 19 22:20:37 debian kernel: [39269.678200]  [] ? 
schedule+0x32/0x80
Jan 19 22:20:37 debian kernel: [39269.678207]  [] ? 
rwsem_down_write_failed+0x1f9/0x360
Jan 19 22:20:37 debian kernel: [39269.678214]  [] ? 
kernfs_sop_show_options+0x30/0x30
Jan 19 22:20:37 debian kernel: [39269.678220]  [] ? 
call_rwsem_down_write_failed+0x13/0x20
Jan 19 22:20:37 debian kernel: [39269.678225]  [] ? 
down_write+0x29/0x40
Jan 19 22:20:37 debian kernel: [39269.678231]  [] ? 
grab_super+0x2b/0x90
Jan 19 22:20:37 debian kernel: [39269.678237]  [] ? 
sget_userns+0x163/0x490
Jan 19 22:20:37 debian kernel: [39269.678242]  [] ? 
kernfs_sop_show_path+0x40/0x40
Jan 19 22:20:37 debian kernel: [39269.678246]  [] ? 
kernfs_mount_ns+0x7a/0x220
Jan 19 22:20:37 debian kernel: [39269.678252]  [] ? 
cgroup_mount+0x334/0x810
Jan 19 22:20:37 debian kernel: [39269.678259]  [] ? 
mount_fs+0x36/0x150
Jan 19 22:20:37 debian kernel: [39269.678264]  [] ? 
vfs_kern_mount+0x62/0x100
Jan 19 22:20:37 debian kernel: [39269.678268]  [] ? 
do_mount+0x1cf/0xc80
Jan 19 22:20:37 debian kernel: [39269.678273]  [] ? 
SyS_mount+0x7e/0xd0
Jan 19 22:20:37 debian kernel: [39269.678279]  [] ? 
do_syscall_64+0x7c/0xf0
Jan 19 22:20:37 debian kernel: [39269.678285]  [] ? 
entry_SYSCALL64_slow_path+0x25/0x25
Jan 19 22:20:37 debian kernel: [39269.678294] INFO: task sync:6958 blocked for 
more than 120 seconds.
Jan 19 22:20:37 debian kernel: [39269.678300]   Tainted: G   O
4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2
Jan 19 22:20:37 debian kernel: [39269.678303] "echo 0 > 
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 22:20:37 debian kernel: [39269.678307] syncD0  6958   
6951 0x0104
Jan 19 22:20:37 debian kernel: [39269.678312]  976553fc4800 
 97653103a140 97655fcd8940
Jan 19 22:20:37 debian kernel: [39269.678318]  97655f00 
a7a706043df0 a6802923 9765500a5760
Jan 19 22:20:37 debian kernel: [39269.678324]  0286 
97655fcd8940 a7a706043df8 97653103a140
Jan 19 22:20:37 debian kernel: [39269.678329] Call Trace:
Jan 19 22:20:37 debian kernel: [39269.678335]  [] ? 
__schedule+0x233/0x6d0
Jan 19 22:20:37 debian kernel: [39269.678340]  [] ? 
schedule+0x32/0x80
Jan 19 22:20:37 debian kernel: [39269.678345]  [] ? 
rwsem_down_read_failed+0xf0/0x150
Jan 19 22:20:37 debian kernel: [39269.678350]  [] ? 
iput+0x7e/0x210
Jan 19 22:20:37 debian kernel: [39269.678356]  [] ? 
SyS_tee+0x390/0x390
Jan 19 22:20:37 debian kernel: [39269.678361]  [] ? 
call_rwsem_down_read_failed+0x14/0x30
Jan 19 22:20:37 debian kernel: [39269.678366]  [] ? 
down_read+0x1c/0x30
Jan 19 22:20:37 debian kernel: [39269.678371]  [] ? 
iterate_supers+0x9c/0x100
Jan 19 22:20:37 debian kernel: [3