Hi Jason,
> Subject: Re: [PATCH v4 1/5] PCI/P2PDMA: Don't enforce ACS check for device
> functions of Intel GPUs
>
> On Mon, Sep 15, 2025 at 12:21:05AM -0700, Vivek Kasireddy wrote:
> > Typically, functions of the same PCI device (such as a PF and a VF)
> > share the same bus and have a common root port and the PF provisions
> > resources for the VF. Given this model, they can be considered
> > compatible as far as P2PDMA access is considered.
>
> Huh? I'm not sure I understand what this is about. Please be more
> clear what your use case is and what exactly is not working.
>
> If it is talking about internal loopback within a single function
> between PF and VF, then no, this is very expressly not something that
> should be expected to work by default!
>
> In fact I would consider any SRIOV capable device that had such a
> behavior by default to be catastrophically security broken.
>
> So this patch can't be talking about that, right?
>
> Yet that is what this code seems to do?!?!?
Here is my use-case:
- Xe Graphics driver, bound to GPU PF on the Host provisions its resources
including VRAM (aka device/graphics memory) among all the VFs.
- A GPU VF device is bound to vfio-pci and assigned to a Linux VM which
is launched via Qemu.
- The Xe Graphics driver running inside the Linux VM creates a buffer in
the VF's portion (or share) of the VRAM and this buffer is shared with
Qemu. Qemu then requests vfio-pci driver to create a dmabuf associated
with this buffer. Note that I am testing with Leon's vfio-pci series included
(vfio/pci: Allow MMIO regions to be exported through dma-buf)
- Next, Qemu requests the GPU PF (via the Xe driver) to import (or access)
the dmabuf (or buffer) located in VF's portion of the VRAM. This is where a
problem occurs.
The exporter (vfio-pci driver in this case) calls pci_p2pdma_map_type() to
determine the map type between both devices (GPU VF and PF) but it fails
due to the ACS enforcement check.
However, assuming that pci_p2pdma_map_type() did not fail, based on my
experiments, the GPU PF is still unable to access the buffer located in VF's
VRAM portion directly because it is represented using PCI BAR addresses.
The only way this seems to be working at the moment is if the BAR addresses
are translated into VRAM addresses that the GPU PF understands (this is done
done inside Xe driver on the Host using provisioning data). Note that this
buffer
is accessible by the CPU using BAR addresses but it is very slow.
So, given that the GPU PF does not need to use PCIe fabric/machinery in order
to access the buffer located in GPU VF's portion of the VRAM in this use-case,
I figured adding a quirk (to not enforce ACS check) here would make sense.
>
> > +static bool pci_devfns_support_p2pdma(struct pci_dev *provider,
> > + struct pci_dev *client)
> > +{
> > + if (provider->vendor == PCI_VENDOR_ID_INTEL &&
> > + client->vendor == PCI_VENDOR_ID_INTEL) {
> > + if ((pci_is_vga(provider) && pci_is_vga(client)) ||
> > + (pci_is_display(provider) && pci_is_display(client)))
> > + return pci_physfn(provider) == pci_physfn(client);
> > + }
>
> Do not open code quirks like this in random places, if this device
> supports some weird ACS behavior and does not include it in the ACS
> Caps the right place is to supply an ACS quirk in quirks.c so all the
> code knows about the device behavior, including the iommu grouping.
Ok, I'll move it to quirks.c.
>
> If your device supports P2P between VF and PF then iommu grouping must
> put VFs in the PF's group and you loose VFIO support.
On my test system, it looks like the VFs and the PF are put into different
iommu groups. I am checking with our hardware folks to understand how this
is expected to work but does it mean that P2P between PF and VF is not
supported in my case?
Also, we do need VFIO support. Otherwise, I don't see any other way as to
how a GPU VF device can be assigned (or passthrough'd) to a Guest VM.
Thanks,
Vivek
>
> Jason
