[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Hi Sven, as tom said on IRC - to some extend it is that Bionic didn't have a Rome chip definition yet, so the changes to add it to Rome do not 100% make sense there. You have as part of similar fixes there already: 1. all kind of intel chips which were faster to provide that for 2.11 (in the stable tree of qemu) root@b:~# qemu-system-x86_64 -cpu ? | grep -i ibrs x86 Broadwell-IBRS Intel Core Processor (Broadwell, IBRS) x86 Broadwell-noTSX-IBRS Intel Core Processor (Broadwell, no TSX, IBRS) x86 Haswell-IBRS Intel Core Processor (Haswell, IBRS) x86 Haswell-noTSX-IBRS Intel Core Processor (Haswell, no TSX, IBRS) x86 IvyBridge-IBRS Intel Xeon E3-12xx v2 (Ivy Bridge, IBRS) x86 Nehalem-IBRS Intel Core i7 9xx (Nehalem Core i7, IBRS update) x86 SandyBridge-IBRS Intel Xeon E312xx (Sandy Bridge, IBRS update) x86 Skylake-Client-IBRS Intel Core Processor (Skylake, IBRS) x86 Skylake-Server-IBRS Intel Xeon Processor (Skylake, IBRS) x86Westmere-IBRS Westmere E56xx/L56xx/X56xx (IBRS update) 2. a generic EPYC chip which got what AMD first pushed for that was IBPB (mind all the characters - not the same) root@b:~# qemu-system-x86_64 -cpu ? | grep -i epy x86EPYC-IBPB AMD EPYC Processor (with IBPB) x86 EPYC AMD EPYC Processor I rechecked if e.g. upstream stable of 2.11 since then changed things, but there is nothing in this regard. Nor did I see (in a very quick not thorough check) anyone else backporting it that far. So you might say "can't I use shiny new chips and all of their feature with Bionic :-/", but for some applications stacks this is a common issue (the balance between stability vs staying new). And for those there often is a way between the obvious first recommendation "use a newer LTS and you get the newer bits" in the form of https://wiki.ubuntu.com/OpenStack/CloudArchive or https://launchpad.net/~canonical-server/+archive/ubuntu/server-backports that allow you to get some newer stacks for older LTSes. Back then when this bug came up these patches did neither make sense nor where in any way applicable to Bionic, one could take some more time to deeply re-evaluate this, but as I said above there are options already ...; I'm tempted to say that this should be spawned into a separate bug then to keep this one as-is reflecting only the efforts of the past. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
this does seem to affect ubuntu 18.04 (bionic beaver) as well, meaning it's not possible to host guest vms with the IBRS Security Flag: qemu-system-x86_64 -cpu ? | grep EPYC-Rome is empty dpkg -l qemu-system Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version ArchitectureDescription +++-=-===-===-=== ii qemu-system 1:4.0+dfsg-0ubuntu9.8 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
This bug was fixed in the package libvirt - 6.0.0-0ubuntu8.9 --- libvirt (6.0.0-0ubuntu8.9) focal; urgency=medium * d/p/u/lp-1921754*: add EPYC-Rome-v2 as v1 missed IBRS and thereby fails on some HW/Guest combinations e.g. Windows 10 on Threadripper (LP: #1921754) * d/p/u/lp-1921880*: add EPYC-Milan features and named cpu type support (LP: #1921880) * d/p/u/lp-1922907: add ability to parse cpu stepping and thereby correctly differentiate skylake and cascadelake chips (LP: #1922907) -- Christian Ehrhardt Wed, 07 Apr 2021 13:33:46 +0200 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
This bug was fixed in the package qemu - 1:4.2-3ubuntu6.16 --- qemu (1:4.2-3ubuntu6.16) focal; urgency=medium * d/p/u/lp-1921754*: add EPYC-Rome-v2 as v1 missed IBRS and thereby fails on some HW/Guest combinations e.g. Windows 10 on Threadripper chips (LP: #1921754) * d/p/u/lp-1921880*: add EPYC-Milan features and named cpu type support (LP: #1921880) -- Christian Ehrhardt Wed, 07 Apr 2021 11:58:29 +0200 ** Changed in: qemu (Ubuntu Focal) Status: Fix Committed => Fix Released ** Changed in: libvirt (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
This bug was fixed in the package libvirt - 6.6.0-1ubuntu3.5 --- libvirt (6.6.0-1ubuntu3.5) groovy; urgency=medium * d/p/u/lp-1921754*: add EPYC-Rome-v2 as v1 missed IBRS and thereby fails on some HW/Guest combinations e.g. Windows 10 on Threadripper (LP: #1921754) * d/p/u/lp-1921880*: add EPYC-Milan features and named cpu type support (LP: #1921880) -- Christian Ehrhardt Wed, 07 Apr 2021 13:33:46 +0200 ** Changed in: libvirt (Ubuntu Groovy) Status: Fix Committed => Fix Released ** Changed in: qemu (Ubuntu Groovy) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
This bug was fixed in the package qemu - 1:5.0-5ubuntu9.8 --- qemu (1:5.0-5ubuntu9.8) groovy; urgency=medium * d/p/u/lp-1921754*: add EPYC-Rome-v2 as v1 missed IBRS and thereby fails on some HW/Guest combinations e.g. Windows 10 on Threadripper chips (LP: #1921754) * d/p/u/lp-1921880*: add EPYC-Milan features and named cpu type support (LP: #1921880) -- Christian Ehrhardt Wed, 07 Apr 2021 11:58:29 +0200 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
The regression tests are good as well with the only hiccups being the known postcopy-after-precopy denial and a known bug in hirsute guest kernels on s390x. Nothing new introduced by the changes as far as I can see. prep (x86_64): Pass 20 F/S/N 0/0/0 - RC 0 (10 min 38961 lin) migrate (x86_64) : Pass 720 F/S/N 36/0/0 - RC 36 (444 min 776248 lin) cross (x86_64) : Pass 46 F/S/N 0/0/1 - RC 0 (53 min 72466 lin) misc (x86_64): Pass 219 F/S/N 0/0/0 - RC 0 (87 min 120975 lin) prep (s390x) : Pass 20 F/S/N 0/0/0 - RC 0 (10 min 26102 lin) migrate (s390x) : Pass 718 F/S/N 22/16/0 - RC 22 (501 min 582924 lin) cross (s390x): Pass 46 F/S/N 0/0/1 - RC 0 (42 min 66256 lin) misc (s390x) : Pass 199 F/S/N 2/0/0 - RC 2 (74 min 95916 lin) ** Tags removed: verification-needed verification-needed-groovy ** Tags added: verification-done verification-done-groovy -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Thanks Markus. Also testing groovy here ... Before: ++-=---= ii libvirt-daemon-system 6.6.0-1ubuntu3.4 amd64Libvirt daemon configuration files ii qemu-system-x86 1:5.0-5ubuntu9.7 amd64QEMU full system emulation binaries (x86) $ qemu-system-x86_64 -cpu ? | grep EPYC-Rome x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor After +++-=---= ii libvirt-daemon-system 6.6.0-1ubuntu3.5 amd64Libvirt daemon configuration files ii qemu-system-x86 1:5.0-5ubuntu9.8 amd64QEMU full system emulation binaries (x86) $ qemu-system-x86_64 -cpu ? | grep EPYC-Rome x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor x86 EPYC-Rome-v2 AMD EPYC-Rome Processor Guest with Rome can now control IBRS, both following cpu definitions worked. EPYC-Rome AMD EPYC-Rome AMD Regression tests are still running (on proposed this time), I'll mark it overall verified once those completed as well. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
** Merge proposal unlinked: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/401771 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/401771 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
focal: qemu 1:4.2-3ubuntu6.15, libvirt 6.0.0-0ubuntu8.8 # qemu-system-x86_64 -cpu ? | grep EPYC x86 EPYC (alias configured by machine type) x86 EPYC-IBPB (alias of EPYC-v2) x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor x86 EPYC-v1 AMD EPYC Processor x86 EPYC-v2 AMD EPYC Processor (with IBPB) x86 EPYC-v3 AMD EPYC Processor # virsh domcapabilities EPYC-Rome AMD After: # qemu-system-x86_64 -cpu ? | grep EPYC x86 EPYC (alias configured by machine type) x86 EPYC-IBPB (alias of EPYC-v2) x86 EPYC-Milan(alias configured by machine type) x86 EPYC-Milan-v1 AMD EPYC-Milan Processor x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor x86 EPYC-Rome-v2 AMD EPYC-Rome Processor x86 EPYC-v1 AMD EPYC Processor x86 EPYC-v2 AMD EPYC Processor (with IBPB) x86 EPYC-v3 AMD EPYC Processor EPYC-Milan AMD Still working as expected -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Old: qemu 1:4.2-3ubuntu6.15, libvirt 6.0.0-0ubuntu8.8 # qemu-system-x86_64 -cpu ? | grep Rome x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor # virsh domcapabilities EPYC-Rome AMD New: # qemu-system-x86_64 -cpu ? | grep Rome x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor x86 EPYC-Rome-v2 AMD EPYC-Rome Processor # virsh domcapabilities EPYC-Rome AMD Instance with ibrs flag can now be run processor : 0 vendor_id : AuthenticAMD cpu family : 23 model : 49 model name : AMD EPYC-Rome Processor stepping: 0 microcode : 0x165 cpu MHz : 2495.312 cache size : 512 KB physical id : 0 siblings: 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid bugs: sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass bogomips: 4990.62 TLB size: 1024 4K pages clflush size: 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: Migration of guest (e.g. with EPYC-IBPB or EPYC-Rome) from previous version to proposed still works. Migration of guest (EPYC-Rome with ibrs) to previous version doesn't (as expected) ** Tags removed: verification-needed-focal ** Tags added: verification-done-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Hello Markus, or anyone else affected, Accepted qemu into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:5.0-5ubuntu9.8 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-groovy. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: qemu (Ubuntu Groovy) Status: Triaged => Fix Committed ** Tags removed: verification-done ** Tags added: verification-needed verification-needed-groovy ** Changed in: qemu (Ubuntu Focal) Status: Triaged => Fix Committed ** Tags added: verification-needed-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Thanks Lukasz! Thereby Hirsute completed - now I've uploaded the (more usual SRUs) for Groovy & Focal for review by the SRU Team. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
This bug was fixed in the package libvirt - 7.0.0-2ubuntu2 --- libvirt (7.0.0-2ubuntu2) hirsute; urgency=medium * d/p/u/lp-1921754*: add EPYC-Rome-v2 as v1 missed IBRS and thereby fails on some HW/Guest combinations e.g. Windows 10 on Threadripper (LP: #1921754) * d/p/u/lp-1921880*: add EPYC-Milan features and named cpu type support (LP: #1921880) -- Christian Ehrhardt Wed, 07 Apr 2021 13:33:46 +0200 ** Changed in: libvirt (Ubuntu Hirsute) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
This bug was fixed in the package qemu - 1:5.2+dfsg-9ubuntu3 --- qemu (1:5.2+dfsg-9ubuntu3) hirsute; urgency=medium * d/p/u/lp-1921754*: add EPYC-Rome-v2 as v1 missed IBRS and thereby fails on some HW/Guest combinations e.g. Windows 10 on Threadripper chips (LP: #1921754) * d/p/u/lp-1921880*: add EPYC-Milan features and named cpu type support (LP: #1921880) -- Christian Ehrhardt Wed, 07 Apr 2021 11:58:29 +0200 ** Changed in: qemu (Ubuntu Hirsute) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
As per our IRC discussion, I will be releasing those early, without the required SRU aging period. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Old ii libvirt-daemon-system 7.0.0-2ubuntu1 amd64Libvirt daemon configuration files ii qemu-system-x86 1:5.2+dfsg-9ubuntu2 amd64QEMU full system emulation binaries (x86) New ii libvirt-daemon-system 7.0.0-2ubuntu2 amd64Libvirt daemon configuration files ii qemu-system-x86 1:5.2+dfsg-9ubuntu3 amd64QEMU full system emulation binaries (x86) 1. regression tests ran and found now new issues triggered by this change 2. qemu types shows v2 ubuntu@riccioli:~$ qemu-system-x86_64 -cpu ? | grep EPYC-Rome x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor x86 EPYC-Rome-v2 AMD EPYC-Rome Processor 3. Libvirt with the fix has the ability to control ibrs for Rome chips. works now. Thereby confirming Markus report - thanks a lot for your ongoing activity and support Markus! ** Tags removed: verification-needed ** Tags added: verification-done -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
** Tags removed: verification-needed-hirsute ** Tags added: verification-done-hirsute -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Before testing with linux 5.11.0-16-generic, qemu 1:5.2+dfsg-9ubuntu2, libvirt 7.0.0-2ubuntu1: # virsh domcapabilities |grep Milan # qemu-system-x86_64 -cpu ? | grep Rome x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor After installing proposed for qemu/libvirt: # virsh domcapabilities EPYC-Milan AMD # qemu-system-x86_64 -cpu ? | grep Rome x86 EPYC-Rome (alias configured by machine type) x86 EPYC-Rome-v1 AMD EPYC-Rome Processor x86 EPYC-Rome-v2 AMD EPYC-Rome Processor Starting focal instance with: EPYC-Rome # grep -P 'ssbd|ibrs' /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Hello Markus, or anyone else affected, Accepted libvirt into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/7.0.0-2ubuntu2 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: libvirt (Ubuntu Hirsute) Status: Triaged => Fix Committed ** Tags added: verification-needed verification-needed-hirsute -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/401372 ** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/libvirt/+git/libvirt/+merge/401371 ** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/401373 ** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/libvirt/+git/libvirt/+merge/401374 ** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/401376 ** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/libvirt/+git/libvirt/+merge/401377 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
** Description changed: - The EPYC-Rome model is missing the IBRS and SSBD flags due to an - oversight + [Impact] + + * IBRS was forgotten in the initial Rome type definitions. +On one hand that makes the feature non exposed/controllable and on the +other it can lead to crashes like bug 1915063 + + * backport chnges to teach: +- qemu about the IBRS feature existing in Rome chips and making it the + new default (-v2 and unspecified) type +- libvirt about the IBRS feature existign for that CPU + + [Test Plan] + + * First of all we'll (and have in advance) run general regression tests + + * Qemu should list the new type (and the old one) +# qemu-system-x86_64 -cpu ? | grep EPYC-Rome + x86 EPYC-Rome (alias configured by machine type) + x86 EPYC-Rome-v1 AMD EPYC-Rome Processor + x86 EPYC-Rome-v2 AMD EPYC-Rome Processor + + * Libvirt with the fix has the ability to control ibrs for Rome chips. +For example on a post Naples Epyc chip this is what host-model expands +(in my example) to - the exact list depends on your chip: + + + EPYC-Rome + AMD + + + + + + + + + + + + + + + + + + + + + + + + + + +Neither the named type nor the added features define IBRS + +Even if one manually adds + +You'd get + error: unsupported configuration: unknown CPU feature: ibrs + +With the fixes in place libvirt knows what bit to check and qemu can +(on the new v2) control that flag. + +The above does no more return an error with the fix + + [Where problems could occur] + + * There are two areas to look at +a) compat behavior on old systems - e.g. libvirt would now detect IBRS + on such AMD chips and one might wonder about the change. + E.g. compatibility would exist between old-code/new-code/old->new + code; but any action (e.g. suspend resume) from new to old code + might run into trouble (not supported that way but worth to mention + for awareness) +b) Migrations between systems - this should be covered by chip + versioning but still is worth to mention. Versioning will recognize + a formerly started system as v1 and continue to handle it that way. + Only new started guests would become v2 and behave the new and + improved way. + + + [Other Info] + + * n/a + + --- + + + The EPYC-Rome model is missing the IBRS and SSBD flags due to an oversight https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg07378.html They have been added as a EPYC-Rome-v2 in https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg01020.html This patch depends on the IBRS and AMD SSBD definitions that were added as part of the EPYC-Milan patch: https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg03370.html - diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 8d599bb5b8..888c3a59e2 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -816,8 +816,12 @@ typedef uint64_t FeatureWordArray[FEATURE_WORDS]; - #define CPUID_8000_0008_EBX_WBNOINVD(1U << 9) - /* Indirect Branch Prediction Barrier */ - #define CPUID_8000_0008_EBX_IBPB(1U << 12) + #define CPUID_8000_0008_EBX_WBNOINVD(1U << 9) + /* Indirect Branch Prediction Barrier */ + #define CPUID_8000_0008_EBX_IBPB(1U << 12) +/* Indirect Branch Restricted Speculation */ +#define CPUID_8000_0008_EBX_IBRS(1U << 14) - /* Single Thread Indirect Branch Predictors */ - #define CPUID_8000_0008_EBX_STIBP (1U << 15) + /* Single Thread Indirect Branch Predictors */ + #define CPUID_8000_0008_EBX_STIBP (1U << 15) +/* Speculative Store Bypass Disable */ +#define CPUID_8000_0008_EBX_AMD_SSBD(1U << 24) ** Description changed: [Impact] - * IBRS was forgotten in the initial Rome type definitions. -On one hand that makes the feature non exposed/controllable and on the -other it can lead to crashes like bug 1915063 + * IBRS was forgotten in the initial Rome type definitions. + On one hand that makes the feature non exposed/controllable and on the + other it can lead to crashes like bug 1915063 - * backport chnges to teach: -- qemu about the IBRS feature existing in Rome chips and making it the - new default (-v2 and unspecified) type -- libvirt about the IBRS feature existign for that CPU + * backport changes to teach: + - qemu about the IBRS feature existing in Rome chips and making it the + new default (-v2 and unspecified) type + - libvirt about the IBRS feature existign for that CPU [Test Plan] - * First of all we'll (and have in advance) run general regression tests + * First of all we'll (and have in advance) run general regression tests - * Qemu should list the new type (and the old one) -# qemu-system-x86_64 -cpu ? | grep
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
** Also affects: qemu (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: libvirt (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: qemu (Ubuntu Groovy) Importance: Undecided Status: New ** Also affects: libvirt (Ubuntu Groovy) Importance: Undecided Status: New ** Also affects: qemu (Ubuntu Hirsute) Importance: Undecided Status: Triaged ** Also affects: libvirt (Ubuntu Hirsute) Importance: Undecided Status: Triaged ** Changed in: qemu (Ubuntu Groovy) Status: New => Triaged ** Changed in: libvirt (Ubuntu Groovy) Status: New => Triaged ** Changed in: qemu (Ubuntu Focal) Status: New => Triaged ** Changed in: libvirt (Ubuntu Focal) Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Now the change was upstream applied and will be in v6.0.0 commit cdeaed27782835a875df7365d56d69dbe7250481 Author: Babu Moger Date: Wed Mar 3 09:45:30 2021 -0600 i386: Add missing cpu feature bits in EPYC-Rome model -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
With 5.11.12-051112-generic in the guest error changes to: unchecked MSR access error: WRMSR to 0xda0 (tried to write 0x) at rIP: 0x9587fa54 (native_write_msr+0x4/0x30) Call Trace: ? fpu__init_cpu_xstate+0xb8/0xf0 fpu__init_system_xstate+0x143/0x354 fpu__init_system+0x6e/0x93 early_identify_cpu.constprop.0+0x112/0x144 early_cpu_init+0x77/0x7c setup_arch+0xc9/0x8cc ? lockdown_lsm_init+0x21/0x25 start_kernel+0x67/0x4a2 ? copy_bootdata+0x1d/0x5d x86_64_start_reservations+0x24/0x26 x86_64_start_kernel+0x8b/0x8f secondary_startup_64_no_verify+0xc2/0xcb -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
As mentioned in LP#1921880 I get an unchecked MSR access when running a focal guest with the EPYC-Rome model on a Milan CPU, but it did not crash. So if you have access to a P620, which is pretty much a Milan CPU, you should be able to reproduce this. It does not happen, if the guest has just the EPYC model. Anyway the MSR error got me on the path that this is related to the missing IBRS flag, which after trying it with the patch, the MSR error was gone. So yes, this is related to LP#1915063 and as you already concluded with this backport, it would be resolved as well. I don't think only the kernel commit will resolve this. I have seen a similar/same MSR error without IBRS on a Fedora 33 guest which has kernel 5.11.x. So I if I have time, I test the mainline ppa kernel to confirm this. For reference, this is the MSR error on the focal guest with 5.4.0-70 on both host and guest: unchecked MSR access error: WRMSR to 0x48 (tried to write 0x0006) at rIP: 0x89a73594 (native_write_msr+0x4/0x30) Call Trace: ? __switch_to_xtra+0x1ae/0x5e0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70 ? __switch_to_asm+0x34/0x70 __switch_to+0x3b0/0x470 ? __switch_to_asm+0x40/0x70 ? __switch_to_asm+0x34/0x70 __schedule+0x2e3/0x740 preempt_schedule_common+0x18/0x30 _cond_resched+0x22/0x30 stop_one_cpu+0x69/0xa0 ? sched_ttwu_pending+0xe0/0xe0 sched_exec+0x92/0xc0 __do_execve_file.isra.0+0x1fc/0x840 ? strncpy_from_user+0x4c/0x150 __x64_sys_execve+0x39/0x50 do_syscall_64+0x57/0x190 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x7f31e09ef2fb Code: 41 89 01 eb da 66 2e 0f 1f 84 00 00 00 00 00 f7 d8 64 41 89 01 eb d6 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 3b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 65 4b 10 00 f7 d8 64 89 01 48 RSP: 002b:7fff1cfd3b48 EFLAGS: 0246 ORIG_RAX: 003b RAX: ffda RBX: 55713f000370 RCX: 7f31e09ef2fb RDX: 55713f0d5010 RSI: 55713f069690 RDI: 55713f006070 RBP: 7fff1cfd3d50 R08: 55713f057cd0 R09: R10: 55713efe9980 R11: 0246 R12: R13: 55713f0d0f50 R14: R15: 55713f069690 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
FYI Out of a different discussion [1] I happened to learn about a kernel side fix [2] which unbreaks this case. It still would help to get the qmeu change landed, but the crashing guest is no more crashing with this. Therefore I wonder if we should add a kernel task to backport that to the Focal kernel. That being said, the issue there was a crash if you run SSBD without IBRS, was it a crash as well for you or was it a different symptom? [1]: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1915063/comments/35 [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=841c2be09fe4f495fe5224952a419bd8c7e5b455 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Report on giving this a first shot for a Focal backport to experiment with it. The patches are (as expected) not too messy or non-applicable. Qemu backporting 143c30d4 already applied due to bug 1882774 5cb287d2 noise for missing the less used/requested 353f98c9 avx512-vp2intersect b3c7344e tsx-ldtrk 5447089c applies as-is 623972ce applies as-is Libvirt backporting 6ea3bb19 already applied due to bug 1887490 892b7c70 already applied due to bug 1887490 96a39aad already applied due to bug 1887490 8c5c660b applies as-is 5c17a7ba applies as-is 5ac6ab2f applies as-is f321a482 some whitespace damage d3de79db neededs to be adapted meson->make b5abf9a1 sync_qemu_cpu_i386 didn't exist before v6.10 Started building in: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4524 @Markus - if you could test these as time permits that would be great. Do whatever you usually do with them, but if - in addition - you could check what happens on live migrations from old to the upgraded package, that would be awesome. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Furthermore while this isn't a duplicate, but since bug 1921880 and this one are so closely tied together that I'll keep the discussion on bug 1921880 for now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Sadly https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg01020.html nor an alternative is merged yet :-/ That is a further inhibitor to pull this asap :-/ I wonder if we should bump the thread again to get it into v6.0.0 ... It had a v2 to remove the model-id. But that v2 never landed. I gave the thread a ping this morning, we will see ... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Hi Markus, thanks for your (two) reports. Essentially the same I said in https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1921880/comments/1 applies here as well. TL;DR: I'll get back with PPA builds if I can get it working and then we need to work together to test and regression check this. ** Also affects: libvirt (Ubuntu) Importance: Undecided Status: New ** Changed in: libvirt (Ubuntu) Status: New => Triaged ** Changed in: qemu (Ubuntu) Status: New => Triaged ** Tags added: server-next -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1921754] Re: Add missing cpu feature bits in EPYC-Rome model
Libvirt support for IBRS https://gitlab.com/libvirt/libvirt/-/commit/5c17a7ba41670f3182186c06e621995b5d03fc95.patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921754 Title: Add missing cpu feature bits in EPYC-Rome model To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1921754/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
