On Fri, 18 Oct 2019 06:08:31 +
"Boeuf, Sebastien" wrote:
> Hi folks,
>
> We have been recently implementing a nested VFIO solution for our Cloud
> Hypervisor VMM. Thanks to virtio-iommu, we can now pass a device
> through nested virtualization.
>
> After some performances testing, we realized the device in L2 was
> slightly less performant than the same device in L1. The DMAR has
> nothing to do with it since it is programmed only at boot time with the
> entire guest RAM mapping for each device, but the problem comes from
> the interrupts. When using polling instead of interrupts, we can
> clearly see the device in L2 behaving very closely to what is expected
> in L1.
>
> So basically, because the interrupt has to bounce through each layer of
> virtualization, the more layers the less performance we will get out of
> this device, right?
Yes
> We were wondering if there was any on going work to allow posted
> interrupts to be delivered directly to the last layer of virtualization
> where the device is actually used. I might be missing something but in
> theory, it seems feasible, the problem seems to be that VFIO does not
> update the IRT of the IOMMU, so I don't know how we could manually
> update it, the same way we can update the DMAR with VFIO_MAP_DMA ioctl
> for instance.
> And the second part is about knowing which interrupt needs to be
> updated in the IRT, which means the virtio-iommu device should be able
> to provide those kind of information.
>
> It seems like there are some possibilities but I clearly don't have the
> whole picture in mind, and I'm interested in learning more about any
> existing work on this topic.
I agree with your assessment, it seems like it should be possible, but
I don't know of any work to support it. Note that vfio doesn't
directly manipulate posted interrupts, it's done via the irqbypass
producer/consumer mechanism, so perhaps this largely needs to be done
in KVM. I can't say I've spent many cycles thinking about it either
though. Thanks,
Alex
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
