RE: [PATCH v2 1/6] hw/arm/smmuv3: Add support to associate a PCIe RC

[Shameerali Kolothum Thodi via]

> -----Original Message-----
> From: Markus Armbruster <arm...@redhat.com>
> Sent: Wednesday, May 7, 2025 8:17 AM
> To: Donald Dutile <ddut...@redhat.com>
> Cc: Shameer Kolothum via <qemu-devel@nongnu.org>; qemu-
> a...@nongnu.org; Shameerali Kolothum Thodi
> <shameerali.kolothum.th...@huawei.com>; eric.au...@redhat.com;
> peter.mayd...@linaro.org; j...@nvidia.com; nicol...@nvidia.com;
> berra...@redhat.com; nath...@nvidia.com; mo...@nvidia.com;
> smost...@google.com; Linuxarm <linux...@huawei.com>; Wangzhou (B)
> <wangzh...@hisilicon.com>; jiangkunkun <jiangkun...@huawei.com>;
> Jonathan Cameron <jonathan.came...@huawei.com>;
> zhangfei....@linaro.org
> Subject: Re: [PATCH v2 1/6] hw/arm/smmuv3: Add support to associate a
> PCIe RC
> 
> Donald Dutile <ddut...@redhat.com> writes:
> 
> [...]
> 
> > In this series, an iommu/smmu needs to be placed -BETWEEN- a sysbus
> and a PCIe-tree,
> > or step-wise, plug an smmuv3 into a sysbus, and a pcie tree/domain/RC
> into an SMMUv3.
> 
> RC = root complex?

Yes.

> 
> > So, an smmu needs to be associated with a bus (tree), i.e., pcie.0, 
> > pcie.1...
> > One could model it as a PCIe device, attached at the pcie-RC ... but that's
> not how it's modelled in ARM hw.
> 
> Physical ARM hardware?
> 
> Assuming the virtual devices and buses we're discussing model physical
> devices and buses:
> 
> * What are the physical devices of interest?
> 
> * How are they wired together?  Which of the wires are buses, in
>   particular PCI buses?

SMMUv3 is a platform device and its placement in a system is typically as below
for PCI devices,

+------------------+
|   PCIe Devices   |
+------------------+
         |
         v
  +-------------+      +---------------+
  |  PCIe RC A  |<---->| Interconnect  |
  +-------------+      +---------------+
                               |
                               |
                        +------v---+
                        | SMMUv3.A |
                        | (IOMMU)  |
                        +----+-----+
                             |
                             v
                     +-------+--------+
                     |   System RAM   |
                     +----------------+
 
This patch is attempting to establish that association between the PCIe RC and 
the SMMUv3 device so that Qemu can build the ACPI tables/DT iommu mappings  
for the SMMUv3 device.

> > SMMU's are discovered via ACPI tables.
> >
> > That leaves us back to the 'how to associate an SMMUv3 to a PCIe
> tree(RC)',
> > and that leads me to the other discussion & format I saw btwn Eric &
> Shameer:
> >  -device arm-smmv3,id=smmuv3.3
> >  -device xxxx,smmuv3= smmuv3.3
> > where one tags a (PCIe) device to an smmuv3(id), which is needed to build
> the (proper) IORT for (pcie-)device<->SMMUv3 associativity in a multi-
> SMMUv3 configuration.
> >
> > We could keep the bus=pcie.X option for the -device arm-smmuv3 to
> indicate that all PCIe devices connected to the pcie.0 tree go through that
> smmuv3;
> > qdev would model/config as the smmuv3 is 'attached to pcie.0'... which it
> sorta is...  and I think the IORT build could associate all devices on pcie.0 
> to
> be associated
> > with the proper smmuv3.
> 
> Device property "bus" is strictly for specifying into which the bus the
> device is to be plugged.  The device's type must match the bus: only a
> PCI device can plug into a PCI bus, and so forth.

The whole idea of reusing the "bus" property for SMMUv3 device was to make
it easier for libvirt. As I mentioned earlier we could go back and use a 
different
property name like "primary-bus" or "pci-bus" for SMMUv3 dev here.

Thanks,
Shameer