Since the kernel team is now investigating the integration of the kernel patches into bionic (the kernel bits are already Fix Committed), we should plan to get the temporary workaround again removed from qemu-kvm. It's hard to time it in a way that the kernel changes are rolled-out and the qemu workaround got removed at the same time. So this could lead to a short time of the qemu mitigation being reverted, but the kernel not yet being released, which would make you need the cap-htm=0ff workaround. But that way it would be much safer that both changes are available prior to the release of 18.04 Bionic. Hence the question (to IBM) is if it would be okay to plan ahead and to get the qemu changes reverted back now?
-- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1752026 Title: Ubuntu18.04:POWER9:DD2.2 - Unable to start a KVM guest with default machine type(pseries-bionic) complaining "KVM implementation does not support Transactional Memory, try cap-htm=off" (kvm) Status in The Ubuntu-power-systems project: Fix Committed Status in linux package in Ubuntu: Fix Committed Status in qemu package in Ubuntu: Fix Released Bug description: == Comment: #0 - Satheesh Rajendran <sathe...@in.ibm.com> - 2018-02-23 08:31:06 == ---Problem Description--- libvirt unable to start a KVM guest complaining about cap-htm machine property to be off Host Env: # lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 160 On-line CPU(s) list: 0-159 Thread(s) per core: 4 Core(s) per socket: 20 Socket(s): 2 NUMA node(s): 2 Model: 2.2 (pvr 004e 1202) Model name: POWER9 (raw), altivec supported CPU max MHz: 3800.0000 CPU min MHz: 2166.0000 L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 10240K NUMA node0 CPU(s): 0-79 NUMA node8 CPU(s): 80-159 ii qemu-kvm 1:2.11+dfsg-1ubuntu2 ppc64el QEMU Full virtualization on x86 hardware ii libvirt-bin 4.0.0-1ubuntu3 ppc64el programs for the libvirt library # lsmcode Version of System Firmware : Product Name : OpenPOWER Firmware Product Version : open-power-SUPERMICRO-P9DSU-V1.03-20180205-imp Product Extra : occ-577915f Product Extra : skiboot-v5.9-240-g081882690163-pcbedce4 Product Extra : petitboot-v1.6.6-p019c87e Product Extra : sbe-095e608 Product Extra : machine-xml-fb5f933 Product Extra : hostboot-9bfb201 Product Extra : linux-4.14.13-openpower1-p78d7eee Contact Information = sathe...@in.ibm.com ---uname output--- 4.15.0-10-generic Machine Type = power9 boston 2.2 (pvr 004e 1202) ---Debugger--- A debugger is not configured ---Steps to Reproduce--- 1. Boot a guest from libvirt with default pseries machine type or pseries-bionic /usr/bin/virt-install --connect=qemu:///system --hvm --accelerate --name 'virt-tests-vm1' --machine pseries --memory=32768 --vcpu=32,sockets=1,cores=32,threads=1 --import --nographics --serial pty --memballoon model=virtio --controller type=scsi,model=virtio-scsi --disk path=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,bus=scsi,size=10,format=qcow2 --network=bridge=virbr0,model=virtio,mac=52:54:00:77:78:79 --noautoconsole WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results. Starting install... ERROR internal error: process exited while connecting to monitor: ,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not support Transactional Memory, try cap-htm=off Domain installation does not appear to have been successful. If it was, you can restart your domain by running: virsh --connect qemu:///system start virt-tests-vm1 otherwise, please restart your installation. 2. Fails to boot.. Note: if we specify machine type as pseries=2.12 it boots fine like below /usr/bin/virt-install --connect=qemu:///system --hvm --accelerate --name 'virt-tests-vm1' --machine pseries-2.12 --memory=32768 --vcpu=32,sockets=1,cores=32,threads=1 --import --nographics --serial pty --memballoon model=virtio --controller type=scsi,model=virtio-scsi --disk path=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,bus=scsi,size=10,format=qcow2 --network=bridge=virbr0,model=virtio,mac=52:54:00:77:78:79 --noautoconsole WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results. qemu-cmd line: libvirt+ 4283 1 99 09:26 ? 00:00:38 qemu-system-ppc64 -enable-kvm -name guest=virt-tests-vm1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4 -virt-tests-vm1/master-key.aes -machine pseries-2.12,accel=kvm,usb=off ,dump-guest-core=off -m 32768 -realtime mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid 108ac2b5-e8b2-4399-a925-a707e8020871 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-virt-tests- vm1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt- wrapper/data/avocado- vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive- scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi- id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net- pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr- vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio- balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on Userspace tool common name: ii libvirt-bin 4.0.0-1ubuntu3 ppc64el programs for the libvirt library The userspace tool has the following bit modes: both Userspace rpm: ii libvirt-bin 4.0.0-1ubuntu3 ppc64el programs for the libvirt library Userspace tool obtained from project website: na *Additional Instructions for sathe...@in.ibm.com: -Post a private note with access information to the machine that the bug is occuring on. -Attach ltrace and strace of userspace application. == Comment: #1 - Satheesh Rajendran <sathe...@in.ibm.com> - 2018-02-23 08:35:17 == vm qemu log for failed and passed cases: Failed:(pseries-bionic) 2018-02-23 14:21:10.806+0000: starting up libvirt version: 4.0.0, package: 1ubuntu3 (Christian Ehrhardt <christian.ehrha...@canonical.com> Mon, 19 Feb 2018 14:18:44 +0100), qemu version: 2.11.1(Debian 1:2.11+dfsg-1ubuntu2), hostname: ltc-boston8.aus.stglabs.ibm.com LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -name guest=virt-tests-vm1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-virt-tests-vm1/master-key.aes -machine pseries-bionic,accel=kvm,usb=off,dump-guest-core=off -m 32768 -realtime mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid 36c37d3b-fb24-4350-94f9-3271b257f75c -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-virt-tests-vm1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:21:10.909242Z qemu-system-ppc64: -chardev pty,id=charserial0: char device redirected to /dev/pts/1 (label charserial0) 2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not support Transactional Memory, try cap-htm=off 2018-02-23 14:21:18.857+0000: shutting down, reason=failed Passed:(pseries-2.12) 2018-02-23 14:26:07.047+0000: starting up libvirt version: 4.0.0, package: 1ubuntu3 (Christian Ehrhardt <christian.ehrha...@canonical.com> Mon, 19 Feb 2018 14:18:44 +0100), qemu version: 2.11.1(Debian 1:2.11+dfsg-1ubuntu2), hostname: ltc-boston8.aus.stglabs.ibm.com LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -name guest=virt-tests-vm1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-virt-tests-vm1/master-key.aes -machine pseries-2.12,accel=kvm,usb=off,dump-guest-core=off -m 32768 -realtime mlock=off -smp 32,sockets=1,cores=32,threads=1 -uuid 108ac2b5-e8b2-4399-a925-a707e8020871 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-virt-tests-vm1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x3 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -drive file=/var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/ubuntu-18.04-ppc64le.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:26:07.116991Z qemu-system-ppc64: -chardev pty,id=charserial0: char device redirected to /dev/pts/1 (label charserial0) Regards, -Satheesh == Comment: #8 - VIPIN K. PARASHAR <vipar...@in.ibm.com> - 2018-02-25 23:38:29 == Starting install... ERROR internal error: process exited while connecting to monitor: ,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:77:78:79,bus=pci.0,addr=0x1 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-02-23T14:21:11.081809Z qemu-system-ppc64: KVM implementation does not support Transactional Memory, try cap-htm=off Domain installation does not appear to have been successful. If it was, you can restart your domain by running: virsh --connect qemu:///system start virt-tests-vm1 otherwise, please restart your installation. As per above message, qemu is reporting TM to be not supported by KVM on this hardware and thus recommending to turn off cap-htm. == Comment: #12 - Suraj Jitindar Singh <sura...@au1.ibm.com> - 2018-02-26 23:35:02 == I don't know what a pseries-bionic is, there is no reference to it upstream. What you are seeing is expected behaviour as far as I can tell. POWER9 currently does not support HTM for a guest and thus it must not be turned on, otherwise qemu will fail to start. HTM can be disabled from the qemu command line by setting cap-htm=off, as stated in the the error message. The pseries-2.12 machine type has htm disabled by default and thus with that machine type there is no requirement to set cap-htm=off on the command line to get qemu to start. So depending on what machine pseries-bionic is based on it will be required to disable htm on the command line (with cap-htm=off) if it is not disabled by default for the machine. == Comment: #13 - Satheesh Rajendran <sathe...@in.ibm.com> - 2018-02-27 00:05:12 == Had a chat with Suraj and here is the summary 1. Currently Power9 DD2.2(host kernel) does not support HTM, so guest should be booted with cap-htm=off, looks like host kernel patch rework in progress--> Initial patch, https://www.spinics.net/lists/kvm-ppc/msg13378.html 2. Libvirt does not know about this cap-htm yet and currently it does not set any default values? 3. pseries-2.12 does have cap-htm=off by default but not the older machine types, so we see the guest is booting from libvirt with pseries-2.12 4. Once 1 is fixed, we can boot cap-htm=on, I guess by that time pseries-2.12 to be changed to cap-htm=on bydefault. ---> 3. Immediate fix can be Canonical defaults their machine type(pseries-bioic) to pseries-2.12... ---> Future 1 and 4 to be addressed, not sure about 2? Needs a mirror to Canonical to address this. Regards, -Satheesh To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1752026/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
