On Thu, Jul 16, 2026 at 10:16:07AM +0200, Peter Krempa wrote: > On Wed, Jul 15, 2026 at 16:11:06 +0100, Daniel P. Berrangé via Devel wrote: > > On Wed, Jul 15, 2026 at 02:56:50PM +0530, Partha Satapathy via Devel wrote: > > > The existing per-vCPU 'order' attribute has different semantics and > > > cannot be reused for this purpose. It is used to define the order and > > > grouping of vCPUs when they are enabled. It applies to online vCPUs and > > > is not a policy for selecting offline hotpluggable vCPUs. > > > > > > In particular, 'order' is not used to choose which vCPUs are removed by > > > a count-based `setvcpus` reduction. Therefore, it cannot express an > > > administrator-defined hot-unplug sequence. > > > > Aren't you simply describing a bug in the implementation of "order". > > > > I defined a guest with > > > > # virsh dumpxml --inactive 2 --xpath //vcpu > > <vcpu placement="static" current="4">8</vcpu> > > <vcpu id="0" enabled="yes" hotpluggable="no" order="1"/> > > <vcpu id="1" enabled="yes" hotpluggable="yes" order="2"/> > > <vcpu id="2" enabled="no" hotpluggable="yes" order="3"/> > > <vcpu id="3" enabled="no" hotpluggable="yes" order="4"/> > > <vcpu id="4" enabled="no" hotpluggable="yes" order="5"/> > > <vcpu id="5" enabled="no" hotpluggable="yes" order="6"/> > > <vcpu id="6" enabled="yes" hotpluggable="yes" order="7"/> > > <vcpu id="7" enabled="yes" hotpluggable="yes" order="8"/> > > > > Yet when I start this guest, libvirt throws away the order > > information I configured > > > > # virsh setvcpus rhel92 6 > > > > # virsh dumpxml rhel92 --xpath //vcpu > > <vcpu placement="static" current="4">8</vcpu> > > <vcpu id="0" enabled="yes" hotpluggable="no" order="1"/> > > <vcpu id="1" enabled="yes" hotpluggable="yes" order="2"/> > > <vcpu id="2" enabled="no" hotpluggable="yes"/> > > <vcpu id="3" enabled="no" hotpluggable="yes"/> > > <vcpu id="4" enabled="no" hotpluggable="yes"/> > > <vcpu id="5" enabled="no" hotpluggable="yes"/> > > <vcpu id="6" enabled="yes" hotpluggable="yes" order="3"/> > > <vcpu id="7" enabled="yes" hotpluggable="yes" order="4"/> > > > > > > and hotplugging CPUs ignores the order I defined > > > > # virsh setvcpus rhel92 6 > > > > # virsh dumpxml rhel92 --xpath //vcpu > > <vcpu placement="static" current="6">8</vcpu> > > <vcpu id="0" enabled="yes" hotpluggable="no" order="1"/> > > <vcpu id="1" enabled="yes" hotpluggable="yes" order="2"/> > > <vcpu id="2" enabled="yes" hotpluggable="yes" order="5"/> > > <vcpu id="3" enabled="yes" hotpluggable="yes" order="6"/> > > <vcpu id="4" enabled="no" hotpluggable="yes"/> > > <vcpu id="5" enabled="no" hotpluggable="yes"/> > > <vcpu id="6" enabled="yes" hotpluggable="yes" order="3"/> > > <vcpu id="7" enabled="yes" hotpluggable="yes" order="4"/> > > > > IMHO, that is just broken and we should fix "order" to be honoured, > > which would appear to address your use case. > > Order is currently the effective order that was used to hotplug the > devices in so that e.g. on migration the state can be properly restored, > so it's used only for active vcpus and ignored for inactive.
Hmmm, I'm surprised migration state cares about the order in which we hotplug the CPUs. I would have thought that if we setup a VM with 8 CPUs, and only had 6 online, the VM state ought to always reflect the max 8 and track the online state, such that libvirt just needed to make sure the same subset of CPUs are online. > Since we have API to hotplug specific vCPU too, this *must* update the > order (in order for the migration to stay compatible) so it'd effectively > change the "config" if users were able to set the order (e.g. un-set > order from another vCPU if it were plugged in out of order). > > Effective re-use of 'order' is thus not reallistically feasible in this > case. It is unfortunate that we accept the 'order' attrbiute in the inactive XML at all, if we're ignoring it and inserting new values for live VM state. That didn't seem to reflect what our domain XML docs suggest the 'order' attribute used for, which reads somewhat like it matches Partha's required use case. Our docs could be improved in this regard I think. > As I've said in the other thread we do allow hotplug vcpus in arbitrary > order, so this would be just syntax-sugar to support hotplug in > arbitrary order via the sequenital order API. > > I don't feel it's worth keeping this complex logic in libvirt as the use > cases are relatively niche and it's a lot of extra complexity to > maintain. Yeah, it is a bit of an awkward case. It also gets slightly on the wrong side of "policy" vs "mechanism", where we try to avoid defining policy and leave it to mgmt app to decide upon. I am slightly concerned that an "order" attribute may not capture policies beyond the one expressed in the use case. With regards, Daniel -- |: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
