Public bug reported: today when ever you use a realtime guest you are required to enable cpu pinning and other feature such as spcifing a real time core mask via hw:cpu_realtime_mask or hw_cpu_realtime_mask.
in the victoria release this requriement was relaxed somewhat with the intoduction of mixed cpu policy guest that are assinged pinned and floating cores. https://github.com/openstack/nova/commit/9fc63c764429c10f9041e6b53659e0cbd595bf6b It is now possible to allocate all cores in an instance to realtime and omit the ``hw:cpu_realtime_mask`` extra spec. This requires specifying the ``hw:emulator_threads_policy`` extra spec. https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/releasenotes/notes/bug-1884231-16acf297d88b122e.yaml however while that works well it also possible to hw:cpu_realtime_mask but not specify hw:emulator_threads_policy which leads to sub optimal xml generation for the libvirt driver. this is reported downstream as https://bugzilla.redhat.com/show_bug.cgi?id=1700390 for older releas that predata the changes referenced above. though in revaluation of this a possible improvment can be made as detailed in https://bugzilla.redhat.com/show_bug.cgi?id=1700390#c11 today if we have a 2 core vm where guest cpu 0 is non realtime and guest cpu 1 is realtime we .e.g. hw:cpu_policy=dedicated hw:cpu_realtime=True hw:cpu_realtime_mask=^0 would generate the xml as follows <vcpupin vcpu="0" cpuset="0"/> <vcpupin vcpu="1" cpuset="1"/> <vcpusched vcpus='1' scheduler='fifo' priority='1'/> <emulatorpin cpuset="0,1"/> this is because the default behavior when no emulator_threads_policy is specifed is for the emulator thread to float over all the vm cores. but a slight modifcation to the xml could be made to have a more optimal default in this case useing the cpu_realtime_mask we can instead restrict the emulator thread to float over the non realtime cores with realtime priortiy. <vcpupin vcpu="0" cpuset="0"/> <vcpupin vcpu="1" cpuset="1"/> <vcpusched vcpus='1' scheduler='fifo' priority='1'/> <emulatorpin cpuset="0"/> <emulatorsched scheduler='fifo' priority='1'> this will ensure that if qemu need to process a request for a device attach for example that the emulator thread has higher priorty then the guest vcpus that deal with guest house keeping task but will not interupt the realtime cores. this would give many of the benifits of emulator_threads_policy=share or emulator_threads_policy=isolate without increase resource usage or requireing any config,flavor or image changes. this should also be a backporable solution to this problem. this is espically important given realtime host often are deplopy with the kernel isolcpus paramater which mean that the kernel will not load balance the emulator thread acrros the range and will instead leave it onthe core it intially spwaned on. today you coudl get lucky and it could be spawn on core 0 in which case the new behvior would be the same or it coudl get spwaned on core 1. wehn the emulatro thread is spawned on core 1 sicne it has less priority then the vcpu thread it will only run if the guest vcpu idels resulting in the iablity for qemu to process device attach and other qemu monitor commands form libvirt or the user. ** Affects: nova Importance: Wishlist Status: Triaged ** Tags: libvirt numa -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1930706 Title: nova allows suboptimal emulator tread pinning for realtime guests Status in OpenStack Compute (nova): Triaged Bug description: today when ever you use a realtime guest you are required to enable cpu pinning and other feature such as spcifing a real time core mask via hw:cpu_realtime_mask or hw_cpu_realtime_mask. in the victoria release this requriement was relaxed somewhat with the intoduction of mixed cpu policy guest that are assinged pinned and floating cores. https://github.com/openstack/nova/commit/9fc63c764429c10f9041e6b53659e0cbd595bf6b It is now possible to allocate all cores in an instance to realtime and omit the ``hw:cpu_realtime_mask`` extra spec. This requires specifying the ``hw:emulator_threads_policy`` extra spec. https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/releasenotes/notes/bug-1884231-16acf297d88b122e.yaml however while that works well it also possible to hw:cpu_realtime_mask but not specify hw:emulator_threads_policy which leads to sub optimal xml generation for the libvirt driver. this is reported downstream as https://bugzilla.redhat.com/show_bug.cgi?id=1700390 for older releas that predata the changes referenced above. though in revaluation of this a possible improvment can be made as detailed in https://bugzilla.redhat.com/show_bug.cgi?id=1700390#c11 today if we have a 2 core vm where guest cpu 0 is non realtime and guest cpu 1 is realtime we .e.g. hw:cpu_policy=dedicated hw:cpu_realtime=True hw:cpu_realtime_mask=^0 would generate the xml as follows <vcpupin vcpu="0" cpuset="0"/> <vcpupin vcpu="1" cpuset="1"/> <vcpusched vcpus='1' scheduler='fifo' priority='1'/> <emulatorpin cpuset="0,1"/> this is because the default behavior when no emulator_threads_policy is specifed is for the emulator thread to float over all the vm cores. but a slight modifcation to the xml could be made to have a more optimal default in this case useing the cpu_realtime_mask we can instead restrict the emulator thread to float over the non realtime cores with realtime priortiy. <vcpupin vcpu="0" cpuset="0"/> <vcpupin vcpu="1" cpuset="1"/> <vcpusched vcpus='1' scheduler='fifo' priority='1'/> <emulatorpin cpuset="0"/> <emulatorsched scheduler='fifo' priority='1'> this will ensure that if qemu need to process a request for a device attach for example that the emulator thread has higher priorty then the guest vcpus that deal with guest house keeping task but will not interupt the realtime cores. this would give many of the benifits of emulator_threads_policy=share or emulator_threads_policy=isolate without increase resource usage or requireing any config,flavor or image changes. this should also be a backporable solution to this problem. this is espically important given realtime host often are deplopy with the kernel isolcpus paramater which mean that the kernel will not load balance the emulator thread acrros the range and will instead leave it onthe core it intially spwaned on. today you coudl get lucky and it could be spawn on core 0 in which case the new behvior would be the same or it coudl get spwaned on core 1. wehn the emulatro thread is spawned on core 1 sicne it has less priority then the vcpu thread it will only run if the guest vcpu idels resulting in the iablity for qemu to process device attach and other qemu monitor commands form libvirt or the user. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1930706/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
