On 18.08.21 21:42, Peter Xu wrote: > This is a long pending issue that we haven't fixed. The issue is in QEMU we > have implicit device ordering requirement when realizing, otherwise some of > the > device may not work properly. > > The initial requirement comes from when vfio-pci starts to work with vIOMMUs. > To make sure vfio-pci will get the correct DMA address space, the vIOMMU > device > needs to be created before vfio-pci otherwise vfio-pci will stop working when > the guest enables the vIOMMU and the device at the same time. > > AFAIU Libvirt should have code that guarantees that. For QEMU cmdline users, > they need to pay attention or things will stop working at some point. > > Recently there's a growing and similar requirement on vDPA. It's not a hard > requirement so far but vDPA has patches that try to workaround this issue. > > This patchset allows us to realize the devices in the order that e.g. platform > devices will be created first (bus device, IOMMU, etc.), then the rest of > normal devices. It's done simply by ordering the QemuOptsList of "device" > entries before realization. The priority so far comes from migration > priorities which could be a little bit odd, but that's really about the same > problem and we can clean that part up in the future. > > Libvirt can still keep its ordering for sure so old QEMU will still work, > however that won't be needed for new qemus after this patchset, so with the > new > binary we should be able to specify qemu cmdline as wish on '-device'. > > Logically this should also work for vDPA and the workaround code can be done > with more straightforward approaches. > > Please review, thanks.
Hi Peter, looks like I have another use case: vhost devices can heavily restrict the number of available memslots: e.g., upstream KVM ~64k, vhost-user usually 32 (!). With virtio-mem intending to make use of multiple memslots [1] and auto-detecting how many to use based on currently avilable memslots when plugging and realizing the virtio-mem device, this implies that realizing vhost devices (especially vhost-user device) after virtio-mem devices can similarly result in issues: when trying realization of the vhost device with restricted memslots, QEMU will bail out. So similarly, we'd want to realize any vhost-* before any virtio-mem device. Do you have any updated version of this patchset? Thanks! [1] https://lkml.kernel.org/r/20211013103330.26869-1-da...@redhat.com -- Thanks, David / dhildenb
