Re: [PATCH v9 1/2] qemu: Add support for HW-accelerated nested SMMUv3

[Nathan Chen via Devel]

Hi Pavel,

On 7/15/2026 9:54 AM, Pavel Hrdina wrote:
On Tue, Jun 30, 2026 at 07:17:31PM -0700, Nathan Chen via Devel wrote:
From: Nathan Chen<nath...@nvidia.com>
Add support for enabling HW-accelerated nested SMMUv3 via <accel>
attribute and its additional attributes for ATS, SSID, RIL, and OAS
configuration.

Signed-off-by: Nathan Chen<nath...@nvidia.com>
---
  docs/formatdomain.rst             | 29 +++++++++++
  src/conf/domain_conf.c            | 80 +++++++++++++++++++++++++++++++
  src/conf/domain_conf.h            |  5 ++
  src/conf/domain_validate.c        | 27 ++++++++++-
  src/conf/schemas/domaincommon.rng | 25 ++++++++++
  src/qemu/qemu_command.c           | 15 ++++++
  6 files changed, 180 insertions(+), 1 deletion(-)

diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst
index a861f9f177..570395f586 100644
--- a/docs/formatdomain.rst
+++ b/docs/formatdomain.rst
@@ -9426,6 +9426,35 @@ Examples:
        The ``pciBus`` attribute notes the index of the controller that an
        IOMMU device is attached to. (QEMU/KVM and ``smmuv3`` model only)
  
+   ``accel``
+      The ``accel`` attribute with possible values ``on`` and ``off`` can be 
used
+      to enable hardware acceleration support for smmuv3 IOMMU devices.
+      (QEMU/KVM and ``smmuv3`` model only)
+
+   ``ats``
+      The ``ats`` attribute with possible values ``on`` and ``off`` can be used
+      to enable reporting Address Translation Services capability to the guest
+      for smmuv3 IOMMU devices with ``accel`` set to ``on``, if the host
+      SMMUv3 supports ATS and the associated passthrough device supports ATS.
+      (QEMU/KVM and ``smmuv3`` model only)
+
+   ``ril``
+      The ``ril`` attribute with possible values ``on`` and ``off`` can be used
+      to report whether Range Invalidation for IOMMU devices with
+      ``accel`` set to ``on`` is compatible with host SMMUv3 support.
+      (QEMU/KVM and ``smmuv3`` model only)
+
+   ``ssidSize``
+      The ``ssidSize`` attribute sets the number of bits used to represent
+      SubstreamIDs. A value of N allows SSIDs in the range [0 .. 2^N - 1].
+      The valid range is 0-20, and a value greater than 0 is required for
+      enabling PASID support, as doing so advertises PASID capability to
+      the vIOMMU. (QEMU/KVM and ``smmuv3`` model only)
I think we should rename this attribute to `ssidsize`, most attributes
with size use it with lower case as well.

Ok, I will rename it on the next revision.>> +
+   ``oas``
+      The ``oas`` attribute sets the output address size in units of bits.
+      (QEMU/KVM and ``smmuv3`` model only)
All of the options are missing `since 12.6.0` note to make sure users
know in which version these options were added.

All options except for `accel` should mention that if accel is enabled
and the option is not configured hypervisor default value is used.

Got it, I will document these in the next refresh.>> +
  In case of ``virtio`` IOMMU device, the ``driver`` element can optionally
  contain ``granule`` subelement that allows to choose which granule will be
  used by default. It is useful when running guests with different page size
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 17c4a57cd8..39cb6bc780 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -2863,6 +2863,10 @@ virDomainIOMMUDefNew(void)
  
      iommu->pci_bus = -1;
  
+    iommu->ssid_size = -1;
+
+    iommu->oas = -1;
+
      return g_steal_pointer(&iommu);
  }
  
@@ -14746,6 +14750,26 @@ virDomainIOMMUDefParseXML(virDomainXMLOption *xmlopt,
                            &iommu->pci_bus, -1) < 0)
              return NULL;
  
+        if (virXMLPropTristateSwitch(driver, "accel", VIR_XML_PROP_NONE,
+                                     &iommu->accel) < 0)
+            return NULL;
+
+        if (virXMLPropTristateSwitch(driver, "ats", VIR_XML_PROP_NONE,
+                                     &iommu->ats) < 0)
+            return NULL;
+
+        if (virXMLPropTristateSwitch(driver, "ril", VIR_XML_PROP_NONE,
+                                     &iommu->ril) < 0)
+            return NULL;
+
+        if (virXMLPropInt(driver, "ssidSize", 10, VIR_XML_PROP_NONE,
+                          &iommu->ssid_size, -1) < 0)
+            return NULL;
+
+        if (virXMLPropInt(driver, "oas", 10, VIR_XML_PROP_NONE,
+                          &iommu->oas, -1) < 0)
+            return NULL;
+
          if ((granule = virXPathNode("./driver/granule", ctxt))) {
              g_autofree char *mode = virXMLPropString(granule, "mode");
              unsigned long long size;
@@ -16849,6 +16873,12 @@ virDomainIOMMUDefEquals(const virDomainIOMMUDef *a,
          a->iotlb != b->iotlb ||
          a->aw_bits != b->aw_bits ||
          a->dma_translation != b->dma_translation ||
+        a->pci_bus != b->pci_bus ||
Unrelated to this commit because it fixes commit
4d6419b286afd9149f41d7cba5cd33ee3d435c63 where this changes was not done.
Make separate commit for this change.

Ok, I will separate this out.>> +        a->accel != b->accel ||
+        a->ats != b->ats ||
+        a->ril != b->ril ||
+        a->ssid_size != b->ssid_size ||
+        a->oas != b->oas ||
          a->xtsup != b->xtsup ||
          a->pt != b->pt ||
          a->granule != b->granule)
@@ -22733,6 +22763,36 @@ virDomainIOMMUDefCheckABIStability(virDomainIOMMUDef 
*src,
                         dst->pci_bus, src->pci_bus);
          return false;
      }
+    if (src->accel != dst->accel) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+                       _("Target domain IOMMU device accel value '%1$d' does not 
match source '%2$d'"),
+                       dst->accel, src->accel);
+        return false;
+    }
+    if (src->ats != dst->ats) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+                       _("Target domain IOMMU device ATS value '%1$d' does not 
match source '%2$d'"),
+                       dst->ats, src->ats);
+        return false;
+    }
+    if (src->ril != dst->ril) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+                       _("Target domain IOMMU device ril value '%1$d' does not 
match source '%2$d'"),
+                       dst->ril, src->ril);
+        return false;
+    }
+    if (src->ssid_size != dst->ssid_size) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+                       _("Target domain IOMMU device ssid_size value '%1$d' does 
not match source '%2$d'"),
+                       dst->ssid_size, src->ssid_size);
+        return false;
+    }
+    if (src->oas != dst->oas) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+                       _("Target domain IOMMU device oas value '%1$d' does not 
match source '%2$d'"),
+                       dst->oas, src->oas);
+        return false;
+    }
      if (src->dma_translation != dst->dma_translation) {
          virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
                         _("Target domain IOMMU device dma translation '%1$s' does 
not match source '%2$s'"),
@@ -29203,6 +29263,26 @@ virDomainIOMMUDefFormat(virBuffer *buf,
          virBufferAsprintf(&driverAttrBuf, " pciBus='%d'",
                            iommu->pci_bus);
      }
+    if (iommu->accel != VIR_TRISTATE_SWITCH_ABSENT) {
+        virBufferAsprintf(&driverAttrBuf, " accel='%s'",
+                          virTristateSwitchTypeToString(iommu->accel));
+    }
+    if (iommu->ats != VIR_TRISTATE_SWITCH_ABSENT) {
+        virBufferAsprintf(&driverAttrBuf, " ats='%s'",
+                          virTristateSwitchTypeToString(iommu->ats));
+    }
+    if (iommu->ril != VIR_TRISTATE_SWITCH_ABSENT) {
+        virBufferAsprintf(&driverAttrBuf, " ril='%s'",
+                          virTristateSwitchTypeToString(iommu->ril));
+    }
+    if (iommu->ssid_size >= 0) {
+        virBufferAsprintf(&driverAttrBuf, " ssidSize='%d'",
+                          iommu->ssid_size);
+    }
+    if (iommu->oas >= 0) {
+        virBufferAsprintf(&driverAttrBuf, " oas='%d'",
+                          iommu->oas);
+    }
      if (iommu->granule != 0) {
          if (iommu->granule == -1) {
              virBufferAddLit(&driverChildBuf, "<granule mode='host'/>\n");
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index 0c6c79c413..40ea6090f8 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -3105,6 +3105,11 @@ struct _virDomainIOMMUDef {
      virTristateSwitch xtsup;
      virTristateSwitch pt;
      int granule; /* -1 means 'host', 0 unset, page size in KiB otherwise */
+    virTristateSwitch accel;
+    virTristateSwitch ats;
+    virTristateSwitch ril;
+    int ssid_size;
+    int oas;
  };
  
  typedef enum {
diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c
index 3946f92182..40d47dab57 100644
--- a/src/conf/domain_validate.c
+++ b/src/conf/domain_validate.c
@@ -3203,6 +3203,16 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu)
                             virDomainIOMMUModelTypeToString(iommu->model));
              return -1;
          }
+        if (iommu->accel != VIR_TRISTATE_SWITCH_ON &&
+            (iommu->ats != VIR_TRISTATE_SWITCH_ABSENT ||
+             iommu->ril != VIR_TRISTATE_SWITCH_ABSENT ||
+             iommu->ssid_size >= 0 ||
+             iommu->oas >= 0)) {
+            virReportError(VIR_ERR_XML_ERROR,
+                           _("accel must be enabled for iommu model '%1$s' when 
setting ats, ril, ssidSize, or oas"),
+                           virDomainIOMMUModelTypeToString(iommu->model));
+            return -1;
+        }
          break;
      case VIR_DOMAIN_IOMMU_MODEL_VIRTIO:
          if (iommu->intremap != VIR_TRISTATE_SWITCH_ABSENT ||
@@ -3210,7 +3220,12 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu)
              iommu->eim != VIR_TRISTATE_SWITCH_ABSENT ||
              iommu->iotlb != VIR_TRISTATE_SWITCH_ABSENT ||
              iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT ||
-            iommu->pci_bus >= 0) {
+            iommu->pci_bus >= 0 ||
+            iommu->accel != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ats != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ril != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ssid_size >= 0 ||
+            iommu->oas >= 0) {
              virReportError(VIR_ERR_XML_ERROR,
                             _("iommu model '%1$s' doesn't support some additional 
attributes"),
                             virDomainIOMMUModelTypeToString(iommu->model));
@@ -3233,6 +3248,11 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu)
              iommu->aw_bits != 0 ||
              iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT ||
              iommu->pci_bus >= 0 ||
+            iommu->accel != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ats != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ril != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ssid_size >= 0 ||
+            iommu->oas >= 0 ||
              iommu->granule != 0) {
              virReportError(VIR_ERR_XML_ERROR,
                             _("iommu model '%1$s' doesn't support some additional 
attributes"),
@@ -3245,6 +3265,11 @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu)
          if (iommu->pt != VIR_TRISTATE_SWITCH_ABSENT ||
              iommu->xtsup != VIR_TRISTATE_SWITCH_ABSENT ||
              iommu->pci_bus >= 0 ||
+            iommu->accel != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ats != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ril != VIR_TRISTATE_SWITCH_ABSENT ||
+            iommu->ssid_size >= 0 ||
+            iommu->oas >= 0 ||
              iommu->granule != 0) {
              virReportError(VIR_ERR_XML_ERROR,
                             _("iommu model '%1$s' doesn't support some additional 
attributes"),
diff --git a/src/conf/schemas/domaincommon.rng 
b/src/conf/schemas/domaincommon.rng
index 121e4e06a6..ea38065764 100644
--- a/src/conf/schemas/domaincommon.rng
+++ b/src/conf/schemas/domaincommon.rng
@@ -6454,6 +6454,31 @@
                  <empty/>
                </element>
              </optional>
+            <optional>
+              <attribute name="accel">
+                <ref name="virOnOff"/>
+              </attribute>
+            </optional>
+            <optional>
+              <attribute name="ats">
+                <ref name="virOnOff"/>
+              </attribute>
+            </optional>
+            <optional>
+              <attribute name="ril">
+                <ref name="virOnOff"/>
+              </attribute>
+            </optional>
+            <optional>
+              <attribute name="ssidSize">
+                <data type="int"/>
+              </attribute>
+            </optional>
+            <optional>
+              <attribute name="oas">
+                <data type="int"/>
+              </attribute>
+            </optional>
            </element>
          </optional>
          <optional>
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index e726dc661c..1e66da2cd3 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -6276,6 +6276,8 @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def,
  {
      g_autoptr(virJSONValue) props = NULL;
      g_autofree char *bus = NULL;
+    g_autofree char *ssidsizeStr = NULL;
+    g_autofree char *oasStr = NULL;
      virPCIDeviceAddress addr = { .bus = iommu->pci_bus };
  
      bus = qemuBuildDeviceAddressPCIGetBus(def, &addr);
@@ -6286,10 +6288,23 @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef 
*def,
          return NULL;
      }
  
+    if (iommu->ssid_size >= 0) {
+        ssidsizeStr = g_strdup_printf("%u", iommu->ssid_size);
+    }
+
+    if (iommu->oas >= 0) {
+        oasStr = g_strdup_printf("%u", iommu->oas);
+    }
+
      if (virJSONValueObjectAdd(&props,
                                "s:driver", "arm-smmuv3",
                                "s:primary-bus", bus,
                                "s:id", iommu->info.alias,
+                              "B:accel", (iommu->accel == 
VIR_TRISTATE_SWITCH_ON),
Use `"T:accel", iommu->accel,` to make sure if accell is set to `off` we
propagate that to qemu cmdline as well.

Ok, I will change this to use T:>> + 
"S:ats", iommu->ats == VIR_TRISTATE_SWITCH_ON ? "on" : iommu->ats == 
VIR_TRISTATE_SWITCH_OFF ? "off" : NULL,
+                              "S:ril", iommu->ril == VIR_TRISTATE_SWITCH_ON ? "on" : 
iommu->ril == VIR_TRISTATE_SWITCH_OFF ? "off" : NULL,
Both of the parameters should use `qemuOnOffAuto()`.

Yes I will update this, thanks.>> + 
"S:ssidsize", ssidsizeStr,
+                              "S:oas", oasStr,
                                NULL) < 0)
          return NULL;
We should also add QEMU capability checks for arm-smmuv3 and the accel
attribute and validate it in qemu_validate.c that the accel attribute is
supported by QEMU or error out. Adding check fo accel is sufficient because
all the other attributes were introduced at the same time in QEMU.

You can check these two commits where similar code was added for
amd-iommu device:

14192af47ab8eda8bb9b9eee14fd1090addbb731
33a5d9dcea59b28e9869e876536d75c748e27176

I will reference these commits and add the QEMU capability checks.

Thanks,
Nathan