[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Edward Hope-Morley]

** Tags removed: sts-sru-needed
** Tags added: sts-sru-done

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Ryan Beisner]

This bug was fixed in the package nova - 2:13.1.4-0ubuntu2~cloud0
---

 nova (2:13.1.4-0ubuntu2~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 nova (2:13.1.4-0ubuntu2) xenial; urgency=medium
 .
   * d/p/libvirt-fix-incorrect-host-cpus-giving-to-emulator-t.patch:
 Backport fix for cpu pinning libvirt config incorrect emulator
 pin cpuset (LP: #1614054).


** Changed in: cloud-archive/mitaka
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Launchpad Bug Tracker]

This bug was fixed in the package nova - 2:13.1.4-0ubuntu2

---
nova (2:13.1.4-0ubuntu2) xenial; urgency=medium

  * d/p/libvirt-fix-incorrect-host-cpus-giving-to-emulator-t.patch:
Backport fix for cpu pinning libvirt config incorrect emulator
pin cpuset (LP: #1614054).

 -- Edward Hope-Morley   Wed, 26 Jul
2017 06:46:18 +0100

** Changed in: nova (Ubuntu Xenial)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Brian Murray]

Will there be any messaging done about the need to recreate instances to
benefit from this fix?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Edward Hope-Morley]

** Description changed:

  [Impact]
  
  This bug affects users of Openstack Nova who want to create instances
  that will leverage the realtime functionality that libvirt/qemu offers
  by, amongst other things, pinning guest vcpus and qemu emulator threads
  to specific pcpus. Nova provides the means for the user to control, via
  the flavor hw:cpu_realtime_mask or image property hw_cpu_realtime_mask,
  which physical cpus these resources will pinned to. This mask allows you
  to mask the set of N pins that Nova selects such that 1 or more of your
  vcpus can be declared "real-time" by ensuring that they do not have
  emulator threads also pinned to them. The remaining "non-realtime" vcpus
  will have vcpu and emulator threads colocated. The bug fixes the case
  where e.g. you have a guest that has 2 vcpus (logically 0 and 1) and
  Nova selects pcpus 14 and 22 and you have mask ^0 to indicate that you
  want all but the first vcpu to be realtime. This should result in the
  following being present in your libvirt xml for the guest:
  
    
  
  
  
  
    
  
  But (current only Mitaka since it does not have this patch) you will get
  this:
  
    
  
  
  
  
    
  
  i.e. Nova will always set the emulator pin to be id of the vcpu instead
  of the corresponding pcpu that it is pinned to.
  
  In terms of actual impact this could result in vcpus that are supposed
  to be isolated not being so and therefore not behaving as expected.
  
  [Test Case]
  
   * deploy openstack mitaka and configure nova.conf with vcpu-pin-
  set=0,1,2,3
  
     https://pastebin.ubuntu.com/25133260/
  
   * configure compute host kernel opts with "isolcpus=0,1,2,3" + reboot
  
   * create flavor with:
  
     openstack flavor create --public --ram 2048 --disk 10 --vcpus 2 --swap 0 
test_flavor
     openstack flavor set --property hw:cpu_realtime_mask='^0' test_flavor
     openstack flavor set --property hw:cpu_policy=dedicated test_flavor
     openstack flavor set --property hw:cpu_thread_policy=prefer test_flavor
     openstack flavor set --property hw:cpu_realtime=yes test_flavor
  
   * boot instance with ^^ flavor
  
   * check that libvirt xml for vm has correct emulator pin cpuset #
  
  [Regression Potential]
  
- None expected.
+ Since the patch being backported only touches the specific aread of code
+ that was causing the original problem  and that code only serves to
+ select cpusets based on flavor filters, i can't think of any regressions
+ that it would introduce. However, one potential side effect/change to be
+ aware of is that once nova-compute is upgraded to this newer version,
+ any new instances created will have the correct/expected cpuset
+ assignments whereas instances created prior to upgrade will remain
+ unchanged i.e. they will all likely still have their emulation threads
+ pinned to the wrong pcpu. In terms of side effects this will mean less
+ load on the pcpu that was previously incorrectly chosen for existing
+ guests but it will mean that older instances will need to be recreated
+ in order to benefit from the fix.
  
  
  
  Description of problem:
  When using the cpu_realtime and cpu_realtim_mask flag to create new instance, 
the 'cpuset' of 'emulatorpin' option is using the id of vcpu which is 
incorrect. The id of host cpu should be used here.
  
  e.g.
    
  
  
    ### the cpuset should be '2' here, 
when cpu_realtime_mask=^0.
  
    
  
  How reproducible:
  Boot new instance with cpu_realtime_mask flavor.
  
  Steps to Reproduce:
  1. Create RT flavor
  nova flavor-create m1.small.performance 6 2048 20 2
  nova flavor-key m1.small.performance set hw:cpu_realtime=yes
  nova flavor-key m1.small.performance set hw:cpu_realtime_mask=^0
  nova flavor-key m1.small.performance set hw:cpu_policy=dedicated
  2. Boot a instance with this flavor
  3. Check the xml of the new instance
  
  Actual results:
    
  
  
  
  
    
  
  Expected results:
    
  
  
  
  
    

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Edward Hope-Morley]

Hi @racb I will provide a more informative regression potential.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Robie Basak]

> [Regression Potential]
> 
> None expected.

See https://wiki.ubuntu.com/StableReleaseUpdates#Procedure

If nobody can be bothered to think about what this update might regress,
and then look for that during Ubuntu's SRU verification process, then
I'm not prepared to risk user stability by releasing this update to
Ubuntu.

If you object to this, then please seek a policy change in Ubuntu
removing the requirement that this be checked.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Edward Hope-Morley]

Tested trusty-mitaka/proposed and lgtm:

  
2048




  


** Tags removed: verification-mitaka-needed
** Tags added: verification-mitaka-done

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Edward Hope-Morley]

Verified for xenial-proposed:

root@orgen:~# dpkg -l| grep nova-compute
ii  nova-compute   2:13.1.4-0ubuntu2
  all  OpenStack Compute - compute node base
ii  nova-compute-kvm   2:13.1.4-0ubuntu2
  all  OpenStack Compute - compute node (KVM)
ii  nova-compute-libvirt   2:13.1.4-0ubuntu2
  all  OpenStack Compute - compute node libvirt support
root@orgen:~# virsh dumpxml instance-0001| grep cpuset





** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Łukasz Zemczak]

Hello sahid, or anyone else affected,

Accepted nova into xenial-proposed. The package will build now and be
available at https://launchpad.net/ubuntu/+source/nova/2:13.1.4-0ubuntu2
in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested and change the tag from
verification-needed-xenial to verification-done-xenial. If it does not
fix the bug for you, please add a comment stating that, and change the
tag to verification-failed-xenial. In either case, details of your
testing will help us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: nova (Ubuntu Xenial)
   Status: Triaged => Fix Committed

** Tags added: verification-needed verification-needed-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Edward Hope-Morley]

Hi @sil2100, apologies for not providing an intuitive Impact field. I
have updated it and hope it will be more helpful. Please let me know if
you need more info.

** Description changed:

  [Impact]
  
- See bug description
+ This bug affects users of Openstack Nova who want to create instances
+ that will leverage the realtime functionality that libvirt/qemu offers
+ by, amongst other things, pinning guest vcpus and qemu emulator threads
+ to specific pcpus. Nova provides the means for the user to control, via
+ the flavor hw:cpu_realtime_mask or image property hw_cpu_realtime_mask,
+ which physical cpus these resources will pinned to. This mask allows you
+ to mask the set of N pins that Nova selects such that 1 or more of your
+ vcpus can be declared "real-time" by ensuring that they do not have
+ emulator threads also pinned to them. The remaining "non-realtime" vcpus
+ will have vcpu and emulator threads colocated. The bug fixes the case
+ where e.g. you have a guest that has 2 vcpus (logically 0 and 1) and
+ Nova selects pcpus 14 and 22 and you have mask ^0 to indicate that you
+ want all but the first vcpu to be realtime. This should result in the
+ following being present in your libvirt xml for the guest:
+ 
+   
+ 
+ 
+ 
+ 
+   
+ 
+ But (current only Mitaka since it does not have this patch) you will get
+ this:
+ 
+   
+ 
+ 
+ 
+ 
+   
+ 
+ i.e. Nova will always set the emulator pin to be id of the vcpu instead
+ of the corresponding pcpu that it is pinned to.
  
  [Test Case]
  
-  * deploy openstack mitaka and configure nova.conf with vcpu-pin-
+  * deploy openstack mitaka and configure nova.conf with vcpu-pin-
  set=0,1,2,3
  
-https://pastebin.ubuntu.com/25133260/
+    https://pastebin.ubuntu.com/25133260/
  
-  * configure compute host kernel opts with "isolcpus=0,1,2,3" + reboot
+  * configure compute host kernel opts with "isolcpus=0,1,2,3" + reboot
  
-  * create flavor with:
+  * create flavor with:
  
-openstack flavor create --public --ram 2048 --disk 10 --vcpus 2 --swap 0 
test_flavor
-openstack flavor set --property hw:cpu_realtime_mask='^0' test_flavor
-openstack flavor set --property hw:cpu_policy=dedicated test_flavor
-openstack flavor set --property hw:cpu_thread_policy=prefer test_flavor
-openstack flavor set --property hw:cpu_realtime=yes test_flavor
+    openstack flavor create --public --ram 2048 --disk 10 --vcpus 2 --swap 0 
test_flavor
+    openstack flavor set --property hw:cpu_realtime_mask='^0' test_flavor
+    openstack flavor set --property hw:cpu_policy=dedicated test_flavor
+    openstack flavor set --property hw:cpu_thread_policy=prefer test_flavor
+    openstack flavor set --property hw:cpu_realtime=yes test_flavor
  
-  * boot instance with ^^ flavor
+  * boot instance with ^^ flavor
  
-  * check that libvirt xml for vm has correct emulator pin cpuset #
+  * check that libvirt xml for vm has correct emulator pin cpuset #
  
  [Regression Potential]
  
  None expected.
- 
  
  
  
  Description of problem:
  When using the cpu_realtime and cpu_realtim_mask flag to create new instance, 
the 'cpuset' of 'emulatorpin' option is using the id of vcpu which is 
incorrect. The id of host cpu should be used here.
  
  e.g.
    
  
  
    ### the cpuset should be '2' here, 
when cpu_realtime_mask=^0.
  
    
  
  How reproducible:
  Boot new instance with cpu_realtime_mask flavor.
  
  Steps to Reproduce:
  1. Create RT flavor
  nova flavor-create m1.small.performance 6 2048 20 2
  nova flavor-key m1.small.performance set hw:cpu_realtime=yes
  nova flavor-key m1.small.performance set hw:cpu_realtime_mask=^0
  nova flavor-key m1.small.performance set hw:cpu_policy=dedicated
  2. Boot a instance with this flavor
  3. Check the xml of the new instance
  
  Actual results:
    
  
  
  
  
    
  
  Expected results:
    
  
  
  
  
    

** Description changed:

  [Impact]
  
  This bug affects users of Openstack Nova who want to create instances
  that will leverage the realtime functionality that libvirt/qemu offers
  by, amongst other things, pinning guest vcpus and qemu emulator threads
  to specific pcpus. Nova provides the means for the user to control, via
  the flavor hw:cpu_realtime_mask or image property hw_cpu_realtime_mask,
  which physical cpus these resources will pinned to. This mask allows you
  to mask the set of N pins that Nova selects such that 1 or more of your
  vcpus can be declared "real-time" by ensuring that they do not have
  emulator threads also pinned to them. The remaining "non-realtime" vcpus
  will have vcpu and emulator threads colocated. The bug fixes the case
  where e.g. you have a guest that has 2 vcpus (logically 0 and 1) and
  Nova selects pcpus 14 and 22 and you have mask ^0 to indicate that you
  want all but the first vcpu to be realtime. This should result in the
  following being present in your libvirt xml for 

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Łukasz Zemczak]

Thank you for submitting the bug and uploading the fix!

Could we please make sure that the [Impact] field of a bug is used to
nicely outline the impact the bug has on users? Without specific
knowledge of the package itself it's hard for an SRU reviewer to see the
actual impact of the bug and how important the fix is - and these we use
to determine if a fix/change is worth the risk of including it in a
stable release.

For instance here the [Impact] field is completely meaningless (the
title doesn't give too much info for me) - I get some overview when
seeing the patch description in the proposed upload, but this should be
present here. If I would accept it the way it is, another SRU member
that would like to release this change into -updates would have issues
getting an overview of the impact.

Please update the bug description appropriately. The SRU template is
there for something. I know it might sometimes look like that but it's
not just useless paperwork that needs to be copy-pasted blindly.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1614054] Re: [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag is set

[Edward Hope-Morley]

** Summary changed:

- Incorrect host cpu is given to emulator threads when cpu_realtime_mask flag 
is set
+ [SRU] Incorrect host cpu is given to emulator threads when cpu_realtime_mask 
flag is set

** Changed in: cloud-archive/mitaka
 Assignee: (unassigned) => Edward Hope-Morley (hopem)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1614054

Title:
  [SRU] Incorrect host cpu is given to emulator threads when
  cpu_realtime_mask flag is set

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1614054/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs