Re: io_uring possibly the culprit for qemu hang (linux-5.4.y)
Hi Jens. On Sat, Oct 17, 2020 at 3:07 AM Jens Axboe wrote: > > Would be great if you could try 5.4.71 and see if that helps for your > issue. > Oh wow, yeah it did fix the issue. I'm able to reliably turn off and start the VM multiple times in a row. Double checked by confirming QEMU is dynamically linked to liburing.so.1. Looks like those 4 io_uring fixes helped. Thanks!
Re: io_uring possibly the culprit for qemu hang (linux-5.4.y)
A small update: As per Stefano's suggestion, disabling io_uring support from QEMU from the configuration step did fix the problem and I'm no longer having hangs. Looks like it __is__ an io_uring issue :( Btw, I used liburing fe50048 for linking QEMU. Thanks. On Fri, Oct 2, 2020 at 4:35 PM Stefano Garzarella wrote: > > Hi Ju, > > On Thu, Oct 01, 2020 at 11:30:14PM +0900, Ju Hyung Park wrote: > > Hi Stefano, > > > > On Thu, Oct 1, 2020 at 5:59 PM Stefano Garzarella > > wrote: > > > Please, can you share the qemu command line that you are using? > > > This can be useful for the analysis. > > > > Sure. > > Thanks for sharing. > > The issue seems related to io_uring and the new io_uring fd monitoring > implementation available from QEMU 5.0. > > I'll try to reproduce. > > For now, as a workaround, you can rebuild qemu by disabling io-uring support: > > ../configure --disable-linux-io-uring ... > > > Thanks, > Stefano > > > > > QEMU: > > /usr/bin/qemu-system-x86_64 -name guest=win10,debug-threads=on -S > > -object > > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-win10/master-key.aes > > -blockdev > > {"driver":"file","filename":"/usr/share/OVMF/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"} > > -blockdev > > {"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"} > > -blockdev > > {"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/win10_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"} > > -blockdev > > {"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"} > > -machine > > pc-q35-5.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off,mem-merge=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format > > -cpu > > Skylake-Client-IBRS,ss=on,vmx=on,hypervisor=on,tsc-adjust=on,clflushopt=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaves=on,pdpe1gb=on,ibpb=on,amd-ssbd=on,fma=off,avx=off,f16c=off,rdrand=off,bmi1=off,hle=off,avx2=off,bmi2=off,rtm=off,rdseed=off,adx=off,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-runtime,hv-synic,hv-stimer,hv-reset > > -m 8192 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu/1-win10 > > -overcommit mem-lock=off -smp 4,sockets=1,dies=1,cores=2,threads=2 > > -uuid 7ccc3031-1dab-4267-b72a-d60065b5ff7f -display none > > -no-user-config -nodefaults -chardev > > socket,id=charmonitor,fd=32,server,nowait -mon > > chardev=charmonitor,id=monitor,mode=control -rtc > > base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay > > -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global > > ICH9-LPC.disable_s4=1 -boot menu=off,strict=on -device > > pcie-root-port,port=0x8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 > > -device pcie-root-port,port=0x9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 > > -device pcie-root-port,port=0xa,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 > > -device pcie-root-port,port=0xb,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x3 > > -device pcie-pci-bridge,id=pci.5,bus=pci.2,addr=0x0 -device > > qemu-xhci,id=usb,bus=pci.1,addr=0x0 -blockdev > > {"driver":"host_device","filename":"/dev/disk/by-partuuid/05c3750b-060f-4703-95ea-6f5e546bf6e9","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":true},"auto-read-only":true,"discard":"unmap"} > > -blockdev > > {"node-name":"libvirt-1-format","read-only":false,"discard":"unmap","detect-zeroes":"unmap","cache":{"direct":false,"no-flush":true},"driver":"raw","file":"libvirt-1-storage"} > > -device > > virtio-blk-pci,scsi=off,bus=pcie.0,addr=0xa,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on > > -netdev tap,fd=34,id=hostnet0 -device > > e1000,netdev=hostnet0,id=net0,mac=52:54:00:c6:bb:bc,bus=pcie.0,add
Re: io_uring possibly the culprit for qemu hang (linux-5.4.y)
Hi Stefano, On Thu, Oct 1, 2020 at 5:59 PM Stefano Garzarella wrote: > Please, can you share the qemu command line that you are using? > This can be useful for the analysis. Sure. QEMU: /usr/bin/qemu-system-x86_64 -name guest=win10,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-win10/master-key.aes -blockdev {"driver":"file","filename":"/usr/share/OVMF/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"} -blockdev {"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/win10_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"} -machine pc-q35-5.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off,mem-merge=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format -cpu Skylake-Client-IBRS,ss=on,vmx=on,hypervisor=on,tsc-adjust=on,clflushopt=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaves=on,pdpe1gb=on,ibpb=on,amd-ssbd=on,fma=off,avx=off,f16c=off,rdrand=off,bmi1=off,hle=off,avx2=off,bmi2=off,rtm=off,rdseed=off,adx=off,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-runtime,hv-synic,hv-stimer,hv-reset -m 8192 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu/1-win10 -overcommit mem-lock=off -smp 4,sockets=1,dies=1,cores=2,threads=2 -uuid 7ccc3031-1dab-4267-b72a-d60065b5ff7f -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=32,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot menu=off,strict=on -device pcie-root-port,port=0x8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 -device pcie-root-port,port=0x9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 -device pcie-root-port,port=0xa,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 -device pcie-root-port,port=0xb,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x3 -device pcie-pci-bridge,id=pci.5,bus=pci.2,addr=0x0 -device qemu-xhci,id=usb,bus=pci.1,addr=0x0 -blockdev {"driver":"host_device","filename":"/dev/disk/by-partuuid/05c3750b-060f-4703-95ea-6f5e546bf6e9","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":true},"auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":false,"discard":"unmap","detect-zeroes":"unmap","cache":{"direct":false,"no-flush":true},"driver":"raw","file":"libvirt-1-storage"} -device virtio-blk-pci,scsi=off,bus=pcie.0,addr=0xa,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on -netdev tap,fd=34,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:c6:bb:bc,bus=pcie.0,addr=0x3 -device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device vfio-pci,host=:00:02.0,id=hostdev0,bus=pcie.0,addr=0x2,rombar=0 -device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x8 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pcie.0,addr=0x9 -msg timestamp=on And I use libvirt 6.3.0 to manage the VM. Here's an xml of my VM. win10 7ccc3031-1dab-4267-b72a-d60065b5ff7f http://libosinfo.org/xmlns/libvirt/domain/1.0;> http://microsoft.com/win/10"/> 8388608 8388608 4 hvm /usr/share/OVMF/OVMF_CODE.fd /var/lib/libvirt/qemu/nvram/win10_VARS.fd destroy restart destroy /usr/bin/qemu-system-x86_64 /dev/urandom