Re: [PATCH v4 1/5] iommu: Add optional subdev in aux_at(de)tach ops

[Lu Baolu]

Hi Alex,

On 9/11/20 6:05 AM, Alex Williamson wrote:
On Tue,  1 Sep 2020 11:34:18 +0800
Lu Baolu <baolu...@linux.intel.com> wrote:
In the vfio/mdev use case of aux-domain, the subdevices are created from
the physical devices with IOMMU_DEV_FEAT_AUX enabled and the aux-domains
are attached to the subdevices through the iommu_ops.aux_attach_dev()
interface.

Current iommu_ops.aux_at(de)tach_dev() design only takes the aux-domain
and the physical device as the parameters, this is insufficient if we
want the vendor iommu drivers to learn the knowledge about relationships
between the aux-domains and the subdevices. Add a @subdev parameter to
iommu_ops.aux_at(de)tach_dev() interfaces so that a subdevice could be
opt-in.

Signed-off-by: Lu Baolu <baolu...@linux.intel.com>
---
  drivers/iommu/intel/iommu.c | 10 ++++++----
  drivers/iommu/iommu.c       |  4 ++--
  include/linux/iommu.h       |  6 ++++--
  3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index bce158468abf..3c12fd06856c 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -5338,8 +5338,9 @@ static int intel_iommu_attach_device(struct iommu_domain 
*domain,
        return domain_add_dev_info(to_dmar_domain(domain), dev);
  }
  
-static int intel_iommu_aux_attach_device(struct iommu_domain *domain,
-                                        struct device *dev)
+static int
+intel_iommu_aux_attach_device(struct iommu_domain *domain,
+                             struct device *dev, struct device *subdev)
  {
        int ret;
  
@@ -5359,8 +5360,9 @@ static void intel_iommu_detach_device(struct iommu_domain *domain,
        dmar_remove_one_dev_info(dev);
  }
  
-static void intel_iommu_aux_detach_device(struct iommu_domain *domain,
-                                         struct device *dev)
+static void
+intel_iommu_aux_detach_device(struct iommu_domain *domain, struct device *dev,
+                             struct device *subdev)
  {
        aux_domain_remove_dev(to_dmar_domain(domain), dev);
  }
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 609bd25bf154..38cdfeb887e1 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2728,7 +2728,7 @@ int iommu_aux_attach_device(struct iommu_domain *domain, 
struct device *dev)
        int ret = -ENODEV;
  
  	if (domain->ops->aux_attach_dev)
-               ret = domain->ops->aux_attach_dev(domain, dev);
+               ret = domain->ops->aux_attach_dev(domain, dev, NULL);
  
  	if (!ret)
                trace_attach_device_to_domain(dev);
@@ -2740,7 +2740,7 @@ EXPORT_SYMBOL_GPL(iommu_aux_attach_device);
  void iommu_aux_detach_device(struct iommu_domain *domain, struct device *dev)
  {
        if (domain->ops->aux_detach_dev) {
-               domain->ops->aux_detach_dev(domain, dev);
+               domain->ops->aux_detach_dev(domain, dev, NULL);
                trace_detach_device_from_domain(dev);
        }
  }
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index fee209efb756..871267104915 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -279,8 +279,10 @@ struct iommu_ops {
        int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f);
  
  	/* Aux-domain specific attach/detach entries */
-       int (*aux_attach_dev)(struct iommu_domain *domain, struct device *dev);
-       void (*aux_detach_dev)(struct iommu_domain *domain, struct device *dev);
+       int (*aux_attach_dev)(struct iommu_domain *domain, struct device *dev,
+                             struct device *subdev);
+       void (*aux_detach_dev)(struct iommu_domain *domain, struct device *dev,
+                              struct device *subdev);
        int (*aux_get_pasid)(struct iommu_domain *domain, struct device *dev);
  
  	struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm,

Would this be a good spot in the code to provide comments more formally
defining this subdevice concept?  For example, what exactly is the
relationship between the device and the subdevice and which device do
we use for the remaining aux domain functions, ex. aux_get_pasid().

Yes. Agreed. I will add below comments and adjust the function
parameters according to the naming rule.

/*
- * Aux-domain specific attach/detach.
+ * Aux-domain specific interfaces.
  *
  * Only works if iommu_dev_feature_enabled(dev, IOMMU_DEV_FEAT_AUX) 
returns
  * true. Also, as long as domains are attached to a device through this
@@ -2722,6 +2722,14 @@ EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled);
  * (iommu_detach_device() can't fail, so we fail when trying to 
re-attach).
  * This should make us safe against a device being attached to a guest 
as a
  * whole while there are still pasid users on it (aux and sva).
+ *
+ * Some physical devices can be configured to generate several subdevices.
+ * The modern IOMMUs support the identification and isolation of these
+ * subdevices. Hence they could be passed through to users space. VFIO/mdev
+ * provides a generic framework for subdevice passthrough. Below interfaces
+ * are extended to support such use case. Generally, among the 
parameters of
+ * the following aux-domain specific functions, @physdev represents a 
physical
+ * device and @subdev represents a subdevice.
  */

Please help to review.

Best regards,
baolu

Thanks,

Alex

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu