On Mon, Jun 06, 2016 at 12:21:34PM -0600, Alex Williamson wrote: [...] > I'm not sure why you're so eager to avoid implementing a replay > callback for VT-d. What happens when VT-d is not enabled by the > guest? vfio/pci.c calls pci_device_iommu_address_space() to get the > address space for the device, which results in vtd_find_add_as() which > gives us a unique VTDAddressSpace.as for that device. With VT-d not in > use by the guest, when do steps 3-5 occur? I agree with your reasoning > when VT-d is enabled, but the BIOS/UEFI does not enable VT-d, so are we > going to exclude all use cases of an assigned device prior to the guest > enabling VT-d?
I think I got the point. I failed to consider the case that we can run IOMMU without enabling it, like BIOS (as you have mentioned), or we can disable IOMMU in kernel boot parameters (though, iommu=pt should still follow the case that IOMMU is enabled). Sounds like a replay callback is a good idea. For Intel version of the callback: when DMAR is enabled, we can just return directly. when DMAR is disabled, we just do whatever we need to do region_add() for the global address_space_memory. > > On that same topic, I'm really dubious that we have the flexibility in > our memory API to really support an IOMMU like VT-d and the layout of > having a VTDAddressSpace per device, rather than exposing shared > address spaces, has implications on the efficiency and locked memory > requirements for vfio. In the above case with VT-d disabled, the > VTDAddressSpace should alias to the system memory AddressSpace and > dynamically switch to a per device address space when VT-d is enabled. > AFAICT, we don't have anything resembling that sort of feature, so we > rely on the IOMMU driver to replay, perhaps even configuring its own > MemoryListener to IOMMU notifier gateway, which is also something that > doesn't currently exist. It sounds more like a notifier for "IOMMU enablement"? The notifier should be triggered when IOMMU switch between enabled <-> disabled? When this happens, we rebuild the mapping in some way. > > Additionally, if there are things unique to VT-d, for instance if VT-d > is enabled and we can rely on the sequence of events you've set forth, > then yes, the replay mechanism should do nothing. But only the VT-d > code can decide that, which implies that vfio always needs to call the > replay function and a new MemoryRegionIOMMUOps callback needs to decide > what to do given the current state of the vIOMMU. Thanks, Right. Thanks. -- peterx
