CC-ing Michael and Jason as I was suggested in OFTC:#virt forum. Thanks! ---------- Forwarded message ---------- From: Lukáš Kubín <[email protected]> Date: Wed, Nov 15, 2017 at 1:39 PM Subject: qemu VM cannot be killed To: [email protected]
Hi, we've experienced an issue with kvm instance which got stuck at reboot. It's an OpenStack environment, with OpenContrail networking (vrouter agent running on host), Ubuntu 16.04. Machine was first called to reboot from guest OS by user, had issues with NFS unmount during that, user sent a hard-reboot call from OpenStack again then. Then we (platform operator) got involved, tried to "virsh destroy" it with this output: error: Failed to destroy domain instance-00004243 error: Failed to terminate process 140529 with SIGKILL: Device or resource busy Neither "kill -9" sent to the qemu process helped. Good guys at OFTC:#virt have guided me to collect the following traces and ask for help here: # cat /proc/140529/wchan vhost_net_ubuf_put_and_wait # cat /proc/140529/stack [<ffffffffc03c2f74>] vhost_net_ubuf_put_and_wait+0x54/0xa0 [vhost_net] [<ffffffffc03c33b4>] vhost_net_ioctl+0x354/0x8a0 [vhost_net] [<ffffffffa26478c1>] do_vfs_ioctl+0xa1/0x5f0 [<ffffffffa2647e89>] SyS_ioctl+0x79/0x90 [<ffffffffa2c9b7f6>] entry_SYSCALL_64_fastpath+0x1e/0xa8 [<ffffffffffffffff>] 0xffffffffffffffff The versions we use are: - kernel 4.8.0-41-generic - qemu-kvm 1:2.5+dfsg-5ubuntu10.2~xenial0+contrail1 - libvirt-bin 1.3.1-1ubuntu10.1~xenial1+contrail1 What can be the cause for this error? What can we do in such a situation to destroy the VM - is physical server reboot the only option? Thanks and greetings, Lukáš
