On 07/07/17 07:21, Tian, Kevin wrote: > sorry I didn't quite get this part, and here is my understanding: > > Guest programs vIOMMU to map a gIOVA (used by MSI to a GPA > of doorbell register of virtual irqchip. vIOMMU then > triggers VFIO map/unmap to update physical IOMMU page > table for gIOVA -> HPA of real doorbell of physical irqchip
At the moment (non-SVM), physical and virtual MSI doorbell are completely dissociated. VFIO itself maps the doorbell GPA->HPA during container initialization. The GPA, chosen arbitrarily by the host, is then removed from the guest GPA space. When the guest programs the vIOMMU to map a gIOVA to the virtual irqchip doorbell, I suppose Qemu will notice that the GPA doesn't correspond to RAM and will withhold sending a VFIO_IOMMU_MAP_DMA request. (For SVM I don't want to go into the details just now, but we will probably need a separate VFIO mechanism to update the physical MSI-X tables with whatever gIOVA the guest mapped in its private stage-1 page tables.) > (assume your irqchip will provide multiple doorbells so each > device can have its own channel). In existing irqchips the doorbell is shared by endpoints, which are differentiated by their device ID (generally the BDF). I'm not sure why this matters here? > then once this update is > done, later MSI interrupts from assigned device will go > through physical IOMMU (gIOVA->HPA) then reach irqchip > for irq remapping. vIOMMU is involved only in configuration > path instead of actual interrupt path. Yes the vIOMMU is used to correlate the IOVA written by the guest in its virtual MSI-X table with the MAP request received by the vIOMMU. That is probably used to setup IRQFD routes with KVM. But the vIOMMU is not involved further than that in MSIs. > If my understanding is correct, above will be the natural flow then > why is additional virtio-iommu change required? :-) The change is not *required* for ARM systems, I only proposed removing the doorbell address translation stage to make host implementation simpler (and since virtio-iommu on x86 won't translate the doorbell anyway, we have to add support for this to virtio-iommu). But for Qemu, since vSMMU needs to implement the natural flow anyway, it might not be a lot of effort to also do it for virtio-iommu. Other implementations (e.g. kvmtool) might piggy-back on the x86 way and declare the irqchip doorbell as untranslated. My proposal also breaks when confronted to virtual SVM in a physical ARM system, where the guest owns stage-1 page tables and *has* to map the doorbell if it wants MSIs to work, so you can disregard it :) Thanks, Jean
