On Mon, 13 Jul 2026 at 20:17, Jason Gunthorpe <[email protected]> wrote: > > On Mon, Jul 13, 2026 at 08:08:14PM +0100, Fuad Tabba wrote: > > On Sun, 12 Jul 2026 at 02:02, Ackerley Tng <[email protected]> wrote: > > > > > > Jason Gunthorpe <[email protected]> writes: > > > > > > > On Thu, May 29, 2025 at 01:34:53PM +0800, Xu Yilun wrote: > > > >> Export vfio dma-buf specific info by attaching vfio_dma_buf_data in > > > >> struct dma_buf::priv. Provide a helper vfio_dma_buf_get_data() for > > > >> importers to fetch these data. Exporters identify VFIO dma-buf by > > > >> successfully getting these data. > > > >> > > > >> VFIO dma-buf supports disabling host access to these exported MMIO > > > >> regions when the device is converted to private. Exporters like KVM > > > >> need to identify this type of dma-buf to decide if it is good to use. > > > >> KVM only allows host unaccessible MMIO regions been mapped in private > > > >> roots. > > > >> > > > >> Export struct kvm * handler attached to the vfio device. This > > > >> allows KVM to do another sanity check. MMIO should only be assigned to > > > >> a CoCo VM if its owner device is already assigned to the same VM. > > > > > > > > This doesn't seem right, it should be encapsulated into the standard > > > > DMABUF API in some way. > > > > > > > > > > I'd like to propose an alternative. I've been working on guest_memfd and > > > new to the world of IO, please help me along! :) > > > > > > It seems like using dmabufs are used a little awkwardly here. IIUC > > > dmabufs were originally meant to expose memory of one device to another > > > device, mostly meant to share memory. Dmabufs do expose MMIO too, for > > > device to device communications. Without virtualization, userspace MMIO > > > would be done by mmap()-ing a VFIO fd and having the userspace program > > > write to the userspace addresses. > > > > > > Before CoCo, device passthrough (MMIO) is mostly handled by mmap()-ing a > > > VFIO fd and setting up the userspace address in a KVM memslot for the > > > guest. > > > > > > With CoCo, is the problem we're solving that we want KVM to know what > > > pfns to set up in stage 2 page tables, but not via userspace addresses? > > > > > > guest_memfd already does that for regular host memory, tracks the > > > private/shared-ness of the memory, tracks which struct kvm the memory > > > belongs to. > > > > > > guest_memfd functions as KVM's bridge to host memory. KVM already can > > > ask guest_memfd for the pfn to map into stage 2 page tables, and already > > > asks guest_memfd for the shared/private state of the memory. guest_memfd > > > already also blocks the host from faulting guest private memory > > > (mmap()-ing is always allowed). > > > > > > > > > Instead of using dmabuf as the intermediary between the MMIO PFNs and > > > KVM, why not use guest_memfd? > > > > > > What if we make guest_memfd accept a VFIO fd, or a dmabuf fd? > > > > This is interesting for pKVM too, provided it covers more than MMIO. > > > > We need guest_memfd to be backable by a dmabuf for ordinary guest memory, > > not > > only for device MMIO. There is mobile hardware that doesn't tolerate > > scattered > > private memory (DMA engines that can't gather, IOMMU page-table size > > constraints), and a CMA-backed dmabuf heap is the practical way to get > > contiguous memory at runtime. > > Why can't guestmemfd allocate directly from CMA? Allocating struct > page memory through dmabuf just to put it back in a guestmemfd sounds > very ugly to me.
Fair, and I think you're right. If guest_memfd can allocate from CMA directly that covers what we need for contiguous guest memory, and it's cleaner than routing it through a dmabuf. It also keeps the shared pages struct-page backed and GUP-able, which the CMA heap's own mmap doesn't, since it sets VM_PFNMAP. So going through a dmabuf for plain guest RAM would have cost us the thing we need on the shared side anyway. Consider the request to be for guest_memfd to be able to give us physically contiguous memory. dmabuf was the mechanism I assumed, not the requirement. > > HugeTLB doesn't help, it wants boot-time > > reservation. Those pages are struct-page backed, so it's a different problem > > from the non-struct-page MMIO case, and the shared parts still need to be > > GUP-able. > > Isn't dmabuf pretty allergic to mmaping refcounted struct page backed > memory since that wrecks its lifetime model? Yes, and that's the same point. Agreed. > > More important for the API shape: conversions have to work on subsets of > > such a > > region, at page granularity. A pKVM guest doesn't know what backs its > > memory, so > > it will issue share/unshare hypercalls over arbitrary ranges of whatever it > > was > > given. If a dmabuf-backed guest_memfd can only be converted as a whole, we > > can't > > use it for memory, and the guest can't be taught to care. > > More reasons not to involve DMABUF since guestmemfd already does all > of this... Also agreed for guest RAM. Where I do still think a dmabuf is involved is the case where the buffer isn't guest_memfd's to allocate: it already belongs to another exporter, and the guest needs to see that same buffer. That's structurally what you're already handling for device memory rather than a separate guest_memfd-over-dmabuf path, so I don't think it argues for backing ordinary guest memory with a dmabuf. > Jason
