Hi All,
Changes from v6:
https://lore.kernel.org/qemu-devel/[email protected]/
1. Fixed the warning case for DT support, reported by Eric(patch #8).
2. Picked up R-by's and T-by's. Thanks!
Please take a look and let me know. I think this is in a good shape now
for 10.1.
Thanks,
Shameer
Changes from v5:
https://lore.kernel.org/qemu-devel/[email protected]/
1. Rebased to target-arm.next and resolved conflicts with the series
[PATCH-for-10.1 v6 0/9] hw/arm: GIC 'its=off'.
2. While at it, noticed an issue with RC id mappings creation
and patch #1 is a fix for that.
3. Patches 3 and 4 have changes because of the conflict resolution with the
above series. I have retained the R-by tags, but encourage all to take
another look in case I missed anything.
4. Collected R-by and T-by tags. Thanks!.
Changes from v4:
https://lore.kernel.org/qemu-devel/[email protected]/
Major changes from v4:
1. Added stricter validation for PCI buses associated with the SMMU.
Only the default PCIe Root Complex (pcie.0) and additional root
complexes created using pxb-pcie (see patch #1) are allowed.
2. While testing this series with a setup involving multiple PCIe root
complexes using pxb-pcie, I encountered an issue related to IOMMU
ops resolution. Consider the below configuration, where an
arm-smmuv3 device is associated with the default root complex pcie.0,
and an additional pxb-pcie-based root complex (pcie.1) is added
without any associated SMMU:
-device arm-smmuv3,primary-bus=pcie.0,id=smmuv3.1 \
...
-device pxb-pcie,id=pcie.1,bus_nr=8,bus=pcie.0 \
-device pcie-root-port,id=pcie.port1,chassis=2,bus=pcie.1 \
-device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.2 \
...
-device virtio-net-pci,bus=pcie.0,netdev=net0,id=virtionet.0 \
-device virtio-net-pci,bus=pcie.port1,netdev=net1,id=virtionet.1
The guest boots fine, and virtionet.0(behind the SMMUV3) bring up
is successful. However, attempting to bring up virtionet.1
(behind pcie.1, which lacks a connected SMMU) results in a failure:
root@ubuntu:/# dhclient enp9s0
arm-smmu-v3 arm-smmu-v3.0.auto: event 0x02 received:
arm-smmu-v3 arm-smmu-v3.0.auto: 0x0902
arm-smmu-v3 arm-smmu-v3.0.auto: 0x
arm-smmu-v3 arm-smmu-v3.0.auto: 0x
arm-smmu-v3 arm-smmu-v3.0.auto: 0x
arm-smmu-v3 arm-smmu-v3.0.auto: event: C_BAD_STREAMID client: (unassigned
sid) sid: 0x900 ssid: 0x0
virtio_net virtio1 enp9s0: NETDEV WATCHDOG: CPU: 2: transmit queue 0 timed
out 5172 ms
virtio_net virtio1 enp9s0: TX timeout on queue: 0, sq: output.0, vq: 0x1,
name: output.0, 5172000 usecs ago
...
Debug shows that QEMU currently registers IOMMU ops for bus using
pci_setup_iommu(). However, when retrieving IOMMU ops for a device
via pci_device_get_iommu_bus_devfn(), the function walks up to the
parent_dev and fetches the IOMMU ops from the parent, even if the
current root bus has none configured.
This works today because existing IOMMU models in QEMU are globally
scoped, and pxb-pcie based extra root complexes can use the
bypass_iommu property to skip translation as needed.
However, with this series introducing support for associating
arm-smmuv3 devices with specific PCIe root complexes, this
becomes problematic. In QEMU, pxb-pcie is implemented as a synthetic
root complex whose parent_dev is pcie.0. As a result, even though
pcie.1 has no SMMU attached, pci_device_get_iommu_bus_devfn()
incorrectly returns the IOMMU ops associated with pcie.0 due to
the fallback mechanism via parent_dev. This causes devices on
pcie.1 to erroneously use the address space from pcie.0's SMMU,
leading to failures like the one above.
To address this, patch #6 in the series introduces a new helper
function pci_setup_iommu_per_bus(), which explicitly sets the
iommu_per_bus field in the PCIBus structure. This allows
pci_device_get_iommu_bus_devfn() to retrieve IOMMU ops based
on the specific bus.
Not sure this is the correct approach or not. If there is a better
way to handle this, please let me know .
3. Picked up few R-by tags where the patch content has not changed much.
4. Dropped T-by from Nathan for some patches as things have changed a bit.
@Nathan, apprecaite if you have time to rerun the tests.
5. Added a bios table tests for both legacy SMMUv3 and new SMMMv3 devices.
See last few patches.
Cover letter:
This patch series introduces support for a user-creatable SMMUv3 device
(-device arm-smmuv3) in QEMU.
The implementation is based on feedback received from the RFCv2[0]:
"hw/arm/virt: Add support for user-creatable accelerated SMMUv3"
Currently, QEMU's SMMUv3 emulation (iommu=smmuv3) is tied to the machine
and does not support instantiating multiple SMMUv3 devices—ea
