Hi Christian, > Subject: Re: [PATCH v4 1/5] PCI/P2PDMA: Don't enforce ACS check for device > functions of Intel GPUs > > On 23.09.25 15:38, Jason Gunthorpe wrote: > > On Tue, Sep 23, 2025 at 03:28:53PM +0200, Christian König wrote: > >> On 23.09.25 15:12, Jason Gunthorpe wrote: > >>>> When you want to communicate addresses in a device specific address > >>>> space you need a device specific type for that and not abuse > >>>> phys_addr_t. > >>> > >>> I'm not talking about abusing phys_addr_t, I'm talking about putting a > >>> legitimate CPU address in there. > >>> > >>> You can argue it is hack in Xe to reverse engineer the VRAM offset > >>> from a CPU physical, and I would be sympathetic, but it does allow > >>> VFIO to be general not specialized to Xe. > >> > >> No, exactly that doesn't work for all use cases. That's why I'm > >> pushing back so hard on using phys_addr_t or CPU addresses. > >> > >> See the CPU address is only valid temporary because the VF BAR is > >> only a window into the device memory. > > > > I know, generally yes. > > > > But there should be no way that a VFIO VF driver in the hypervisor > > knows what is currently mapped to the VF's BAR. The only way I can > > make sense of what Xe is doing here is if the VF BAR is a static > > aperture of the VRAM.. > > > > Would be nice to know the details. > > Yeah, that's why i asked how VFIO gets the information which parts of the > it's BAR should be part of the DMA-buf? > > That would be really interesting to know. As Jason guessed, we are relying on the GPU VF being a Large BAR device here. In other words, as you suggested, this will not work if the VF BAR size is not as big as its actual VRAM portion. We can certainly add this check but we have not seen either the GPU PF or VF getting detected as Small BAR devices in various test environments.
So, given the above, once a VF device is bound to vfio-pci driver and assigned to a Guest VM (launched via Qemu), Qemu's vfio layer maps all the VF's resources including the BARs. This mapping info (specifically HVA) is leveraged (by Qemu) to identity the offset at which the Guest VM's buffer is located (in the BAR) and this info is then provided to vfio-pci kernel driver which finally creates the dmabuf (with BAR Addresses). Thanks, Vivek > > Regards, > Christian. > > > > >> What Simona agreed on is exactly what I proposed as well, that you > >> get a private interface for exactly that use case. > > > > A "private" interface to exchange phys_addr_t between at least > > VFIO/KVM/iommufd - sure no complaint with that. > > > > Jason
