------- Comment From [email protected] 2018-04-04 08:50 EDT------- Hi Frank,
First of all, thanks for caring about this bug and accepting the out-of- the-tree patches. You are right, the motivation to include this patchset is to re-enable the HTM in the KVM guests. So, we just need to schedule the fixes on both side. Here are some assumptions I have: 1) If you need to revert the qemu-kvm "HTM off" patches right now, We can survive, since we have a internal kernel that contains the fix, and we can use this custom kernel in the mean time. Not a big deal. 2) On the other side, I understand that the Final Freeze for Ubuntu will be in April 19th, so, we still have some time to release qemu, how long can we wait without affecting the time to we spend testing this package? 3) Releasing the fixed kernel prior to the fixed qemu package would be better than the other way around. 4) Can't we fix fix qemu now and keep it in the proposed until the kernel is released? Thanks! ------- Comment From [email protected] 2018-04-04 08:55 EDT------- One other possibility could be to have the changes going into the kernel and, then, remove the workaround from QEMU. QEMU with the workaround should continue to work with the kernel with the proper changes. HTM will be disabled in the guests, which would not be needed anymore, but would not block a VM from running. Anyway, I am OK either way. We need to make progress here and I understand this came in late. We need to have the fixes in the kernel and the workaround out of QEMU. If that means we will have a broken QEMU for few days, that would be OK. We can continue our tests with previous versions of kernel and QEMU until everything is settled in bionic repositories or disable HTM by hand when running tests. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1752026 Title: Ubuntu18.04:POWER9:DD2.2 - Unable to start a KVM guest with default machine type(pseries-bionic) complaining "KVM implementation does not support Transactional Memory, try cap-htm=off" (kvm) Status in The Ubuntu-power-systems project: Fix Committed Status in linux package in Ubuntu: Fix Committed Status in qemu package in Ubuntu: Fix Released Bug description: == Comment: #0 - Satheesh Rajendran <[email protected]> - 2018-02-23 08:31:06 == ---Problem Description--- libvirt unable to start a KVM guest complaining about cap-htm machine property to be off Host Env: # lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 160 On-line CPU(s) list: 0-159 Thread(s) per core: 4 Core(s) per socket: 20 Socket(s): 2 NUMA node(s): 2 Model: 2.2 (pvr 004e 1202) Model name: POWER9 (raw), altivec supported CPU max MHz: 3800.0000 CPU min MHz: 2166.0000 L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 10240K NUMA node0 CPU(s): 0-79 NUMA node8 CPU(s): 80-159 ii qemu-kvm 1:2.11+dfsg-1ubuntu2 ppc64el QEMU Full virtualization on x86 hardware ii libvirt-bin 4.0.0-1ubuntu3 ppc64el programs for the libvirt library # lsmcode Version of System Firmware : Product Name : OpenPOWER Firmware Product Version : open-power-SUPERMICRO-P9DSU-V1.03-20180205-imp Product Extra : occ-577915f Product Extra : skiboot-v5.9-240-g081882690163-pcbedce4 Product Extra : petitboot-v1.6.6-p019c87e Product Extra : sbe-095e608 Product Extra : machine-xml-fb5f933 Product Extra : hostboot-9bfb201 Product Extra : linux-4.14.13-openpower1-p78d7eee Contact Information = [email protected] ---uname output--- 4.15.0-10-generic Machine Type = power9 boston 2.2 (pvr 004e 1202) ---Debugger--- A debugger is not configured ---Steps to Reproduce--- 1. Boot a guest from libvirt with default pseries machine type or pseries-bionic /usr/bin/virt-install --connect=qemu:///system --hvm --accelerate --name 'virt-tests-vm1' --machine pseries --memory=32768 --vcpu=32,sockets=1,cores=32,threads=1 --import --nographics --serial pty --memballoon model=virtio --controller type=scsi,model=virtio-scsi --disk path=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,bus=scsi,size=10,format=qcow2 --network=bridge=virbr0,model=virtio,mac=52:54:00:77:78:79 --noautoconsole WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results. Starting install... ERROR internal error: process exited while connecting to monitor: ,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not support Transactional Memory, try cap-htm=off Domain installation does not appear to have been successful. If it was, you can restart your domain by running: virsh --connect qemu:///system start virt-tests-vm1 otherwise, please restart your installation. 2. Fails to boot.. Note: if we specify machine type as pseries=2.12 it boots fine like below /usr/bin/virt-install --connect=qemu:///system --hvm --accelerate --name 'virt-tests-vm1' --machine pseries-2.12 --memory=32768 --vcpu=32,sockets=1,cores=32,threads=1 --import --nographics --serial pty --memballoon model=virtio --controller type=scsi,model=virtio-scsi --disk path=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,bus=scsi,size=10,format=qcow2 --network=bridge=virbr0,model=virtio,mac=52:54:00:77:78:79 --noautoconsole WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results. qemu-cmd line: libvirt+ 4283 1 99 09:26 ? 00:00:38 qemu-system-ppc64 -enable-kvm -name guest=virt-tests-vm1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4 -virt-tests-vm1/master-key.aes -machine pseries-2.12,accel=kvm,usb=off ,dump-guest-core=off -m 32768 -realtime mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid 108ac2b5-e8b2-4399-a925-a707e8020871 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-virt-tests- vm1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt- wrapper/data/avocado- vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive- scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi- id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr- vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio- balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on Userspace tool common name: ii libvirt-bin 4.0.0-1ubuntu3 ppc64el programs for the libvirt library The userspace tool has the following bit modes: both Userspace rpm: ii libvirt-bin 4.0.0-1ubuntu3 ppc64el programs for the libvirt library Userspace tool obtained from project website: na *Additional Instructions for [email protected]: -Post a private note with access information to the machine that the bug is occuring on. -Attach ltrace and strace of userspace application. == Comment: #1 - Satheesh Rajendran <[email protected]> - 2018-02-23 08:35:17 == vm qemu log for failed and passed cases: Failed:(pseries-bionic) 2018-02-23 14:21:10.806+0000: starting up libvirt version: 4.0.0, package: 1ubuntu3 (Christian Ehrhardt <[email protected]> Mon, 19 Feb 2018 14:18:44 +0100), qemu version: 2.11.1(Debian 1:2.11+dfsg-1ubuntu2), hostname: ltc-boston8.aus.stglabs.ibm.com LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -name guest=virt-tests-vm1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-virt-tests-vm1/master-key.aes -machine pseries-bionic,accel=kvm,usb=off,dump-guest-core=off -m 32768 -realtime mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid 36c37d3b-fb24-4350-94f9-3271b257f75c -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-virt-tests-vm1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:21:10.909242Z qemu-system-ppc64: -chardev pty,id=charserial0: char device redirected to /dev/pts/1 (label charserial0) 2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not support Transactional Memory, try cap-htm=off 2018-02-23 14:21:18.857+0000: shutting down, reason=failed Passed:(pseries-2.12) 2018-02-23 14:26:07.047+0000: starting up libvirt version: 4.0.0, package: 1ubuntu3 (Christian Ehrhardt <[email protected]> Mon, 19 Feb 2018 14:18:44 +0100), qemu version: 2.11.1(Debian 1:2.11+dfsg-1ubuntu2), hostname: ltc-boston8.aus.stglabs.ibm.com LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -name guest=virt-tests-vm1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-virt-tests-vm1/master-key.aes -machine pseries-2.12,accel=kvm,usb=off,dump-guest-core=off -m 32768 -realtime mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid 108ac2b5-e8b2-4399-a925-a707e8020871 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-virt-tests-vm1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:26:07.116991Z qemu-system-ppc64: -chardev pty,id=charserial0: char device redirected to /dev/pts/1 (label charserial0) Regards, -Satheesh == Comment: #8 - VIPIN K. PARASHAR <[email protected]> - 2018-02-25 23:38:29 == Starting install... ERROR internal error: process exited while connecting to monitor: ,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not support Transactional Memory, try cap-htm=off Domain installation does not appear to have been successful. If it was, you can restart your domain by running: virsh --connect qemu:///system start virt-tests-vm1 otherwise, please restart your installation. As per above message, qemu is reporting TM to be not supported by KVM on this hardware and thus recommending to turn off cap-htm. == Comment: #12 - Suraj Jitindar Singh <[email protected]> - 2018-02-26 23:35:02 == I don't know what a pseries-bionic is, there is no reference to it upstream. What you are seeing is expected behaviour as far as I can tell. POWER9 currently does not support HTM for a guest and thus it must not be turned on, otherwise qemu will fail to start. HTM can be disabled from the qemu command line by setting cap-htm=off, as stated in the the error message. The pseries-2.12 machine type has htm disabled by default and thus with that machine type there is no requirement to set cap-htm=off on the command line to get qemu to start. So depending on what machine pseries-bionic is based on it will be required to disable htm on the command line (with cap-htm=off) if it is not disabled by default for the machine. == Comment: #13 - Satheesh Rajendran <[email protected]> - 2018-02-27 00:05:12 == Had a chat with Suraj and here is the summary 1. Currently Power9 DD2.2(host kernel) does not support HTM, so guest should be booted with cap-htm=off, looks like host kernel patch rework in progress--> Initial patch, https://www.spinics.net/lists/kvm-ppc/msg13378.html 2. Libvirt does not know about this cap-htm yet and currently it does not set any default values? 3. pseries-2.12 does have cap-htm=off by default but not the older machine types, so we see the guest is booting from libvirt with pseries-2.12 4. Once 1 is fixed, we can boot cap-htm=on, I guess by that time pseries-2.12 to be changed to cap-htm=on bydefault. ---> 3. Immediate fix can be Canonical defaults their machine type(pseries-bioic) to pseries-2.12... ---> Future 1 and 4 to be addressed, not sure about 2? Needs a mirror to Canonical to address this. Regards, -Satheesh To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1752026/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : [email protected] Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
