On Sun, 12 Jul 2026 at 02:02, Ackerley Tng <[email protected]> wrote: > > Jason Gunthorpe <[email protected]> writes: > > > On Thu, May 29, 2025 at 01:34:53PM +0800, Xu Yilun wrote: > >> Export vfio dma-buf specific info by attaching vfio_dma_buf_data in > >> struct dma_buf::priv. Provide a helper vfio_dma_buf_get_data() for > >> importers to fetch these data. Exporters identify VFIO dma-buf by > >> successfully getting these data. > >> > >> VFIO dma-buf supports disabling host access to these exported MMIO > >> regions when the device is converted to private. Exporters like KVM > >> need to identify this type of dma-buf to decide if it is good to use. > >> KVM only allows host unaccessible MMIO regions been mapped in private > >> roots. > >> > >> Export struct kvm * handler attached to the vfio device. This > >> allows KVM to do another sanity check. MMIO should only be assigned to > >> a CoCo VM if its owner device is already assigned to the same VM. > > > > This doesn't seem right, it should be encapsulated into the standard > > DMABUF API in some way. > > > > I'd like to propose an alternative. I've been working on guest_memfd and > new to the world of IO, please help me along! :) > > It seems like using dmabufs are used a little awkwardly here. IIUC > dmabufs were originally meant to expose memory of one device to another > device, mostly meant to share memory. Dmabufs do expose MMIO too, for > device to device communications. Without virtualization, userspace MMIO > would be done by mmap()-ing a VFIO fd and having the userspace program > write to the userspace addresses. > > Before CoCo, device passthrough (MMIO) is mostly handled by mmap()-ing a > VFIO fd and setting up the userspace address in a KVM memslot for the > guest. > > With CoCo, is the problem we're solving that we want KVM to know what > pfns to set up in stage 2 page tables, but not via userspace addresses? > > guest_memfd already does that for regular host memory, tracks the > private/shared-ness of the memory, tracks which struct kvm the memory > belongs to. > > guest_memfd functions as KVM's bridge to host memory. KVM already can > ask guest_memfd for the pfn to map into stage 2 page tables, and already > asks guest_memfd for the shared/private state of the memory. guest_memfd > already also blocks the host from faulting guest private memory > (mmap()-ing is always allowed). > > > Instead of using dmabuf as the intermediary between the MMIO PFNs and > KVM, why not use guest_memfd? > > What if we make guest_memfd accept a VFIO fd, or a dmabuf fd?
This is interesting for pKVM too, provided it covers more than MMIO. We need guest_memfd to be backable by a dmabuf for ordinary guest memory, not only for device MMIO. There is mobile hardware that doesn't tolerate scattered private memory (DMA engines that can't gather, IOMMU page-table size constraints), and a CMA-backed dmabuf heap is the practical way to get contiguous memory at runtime. HugeTLB doesn't help, it wants boot-time reservation. Those pages are struct-page backed, so it's a different problem from the non-struct-page MMIO case, and the shared parts still need to be GUP-able. More important for the API shape: conversions have to work on subsets of such a region, at page granularity. A pKVM guest doesn't know what backs its memory, so it will issue share/unshare hypercalls over arbitrary ranges of whatever it was given. If a dmabuf-backed guest_memfd can only be converted as a whole, we can't use it for memory, and the guest can't be taught to care. Quentin made both points on the in-place conversion series [1], and covered the wider framing at KVM Forum [2]. Cheers, /fuad [1] https://lore.kernel.org/all/od4dx6snqsl2qiocgf3jxm4dndxhrlvsfr22eveuno6nskgfdj@mxsywvku2jk5/ [2] https://www.youtube.com/watch?v=zaBxoyRepzA > > guest_memfd can then take the mmap() calls from userspace and .fault() > from mm, and then forward them to VFIO or dmabuf. This way, VFIO/dmabuf > can stick to their original functions, and the changes to VFIO/dmabuf > would probably revolve around disabling access. > > Disabling access would probably involve some of these: > > + When guest_memfd receives the fd, it could return error for existing > mappings, or perhaps it could just force-unmap. > + 1 extra flag or field to indicate that guest_memfd is controlling this > file, so that if userspace tries to take some actions with the > original VFIO or dmabuf fd, the request should be blocked. > + Perhaps just close the original fd, like dup2(oldfd, newfd) closes > newfd? > > > I'm about to restart work on guest_memfd HugeTLB and I'm thinking about > a similar approach for guest_memfd HugeTLB, where perhaps the interface > could be that userspace will give guest_memfd a HugeTLB fd at creation > time, and then the original HugeTLB fd would be rendered unusable in the > same way as above, perhaps like with the S_IMMUTABLE inode flag, but > also blocking reads, and not userspace-modifiable. > > > In the course of a CoCo guest's operation, will the guest need to > convert between private/shared MMIO? Will the guest need some pages > shared and others private? If these are required operations, guest_memfd > already provides the tracking and is going to have a conversion ioctl > very soon. Instead of further extending dmabuf to track more things, how > about letting guest_memfd track it? >
