Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
On Fri, Mar 18, 2022 at 08:01:04PM +0800, Lu Baolu wrote: > On 2022/3/15 19:49, Tian, Kevin wrote: > > > From: Jean-Philippe Brucker > > > Sent: Tuesday, March 15, 2022 7:27 PM > > > > > > On Mon, Mar 14, 2022 at 10:07:06PM -0700, Jacob Pan wrote: > > > > From: Lu Baolu > > > > > > > > An IOMMU domain represents an address space which can be attached by > > > > devices that perform DMA within a domain. However, for platforms with > > > > PASID capability the domain attachment needs be handled at device+PASID > > > > level. There can be multiple PASIDs within a device and multiple devices > > > > attached to a given domain. > > > > This patch introduces a new IOMMU op which support device, PASID, and > > > > IOMMU domain attachment. The immediate use case is for PASID capable > > > > devices to perform DMA under DMA APIs. > > > > > > > > Signed-off-by: Lu Baolu > > > > Signed-off-by: Jacob Pan > > > > include/linux/iommu.h | 6 ++ > > > > 1 file changed, 6 insertions(+) > > > > > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > > > index 369f05c2a4e2..fde5b933dbe3 100644 > > > > +++ b/include/linux/iommu.h > > > > @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { > > > >* @aux_get_pasid: get the pasid given an aux-domain > > > >* @sva_bind: Bind process address space to device > > > >* @sva_unbind: Unbind process address space from device > > > > + * @attach_dev_pasid: attach an iommu domain to a pasid of device > > > > + * @detach_dev_pasid: detach an iommu domain from a pasid of device > > > Isn't that operation "assign a PASID to a domain" instead? In patch 5, > > > the domain is already attached to the device, so set_domain_pasid() might > > > be clearer and to the point. If the IOMMU driver did the allocation we > > > could also avoid patch 1. > > iiuc this API can also work for future SIOV usage where each mdev attached > > to the domain has its own pasid. "assigning a PASID to a domain" sounds > > like going back to the previous aux domain approach which has one PASID > > per domain and that PASID is used on all devices attached to the aux > > domain... > > > > This also works for SVA as far as I can see. The sva_bind essentially is > to attach an SVA domain to the PASID of a device. The sva_bind/unbind > ops could be removed with these two new callbacks. As we talked before I would like to see 'sva bind' go away and be replaced with: domain = alloc_sva_iommu_domain(device, current) attach_dev_pasid_domain(domain, device, pasid) Which composes better with everything else. SVA is just a special kind of iommu_domain Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
On Fri, Mar 18, 2022 at 07:52:33PM +0800, Lu Baolu wrote: > On 2022/3/15 13:07, Jacob Pan wrote: > > From: Lu Baolu > > > > An IOMMU domain represents an address space which can be attached by > > devices that perform DMA within a domain. However, for platforms with > > PASID capability the domain attachment needs be handled at device+PASID > > level. There can be multiple PASIDs within a device and multiple devices > > attached to a given domain. > > This patch introduces a new IOMMU op which support device, PASID, and > > IOMMU domain attachment. The immediate use case is for PASID capable > > devices to perform DMA under DMA APIs. > > > > Signed-off-by: Lu Baolu > > Signed-off-by: Jacob Pan > > include/linux/iommu.h | 6 ++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > index 369f05c2a4e2..fde5b933dbe3 100644 > > +++ b/include/linux/iommu.h > > @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { > >* @aux_get_pasid: get the pasid given an aux-domain > >* @sva_bind: Bind process address space to device > >* @sva_unbind: Unbind process address space from device > > + * @attach_dev_pasid: attach an iommu domain to a pasid of device > > + * @detach_dev_pasid: detach an iommu domain from a pasid of device > >* @sva_get_pasid: Get PASID associated to a SVA handle > >* @page_response: handle page request response > >* @cache_invalidate: invalidate translation caches > > @@ -296,6 +298,10 @@ struct iommu_ops { > > struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, > > void *drvdata); > > void (*sva_unbind)(struct iommu_sva *handle); > > + int (*attach_dev_pasid)(struct iommu_domain *domain, > > + struct device *dev, ioasid_t id); > > + void (*detach_dev_pasid)(struct iommu_domain *domain, > > +struct device *dev, ioasid_t id); > > As we have introduced iommu_domain_ops, these callbacks should be part > of the domain ops. +1 Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
On 2022/3/15 19:49, Tian, Kevin wrote: From: Jean-Philippe Brucker Sent: Tuesday, March 15, 2022 7:27 PM On Mon, Mar 14, 2022 at 10:07:06PM -0700, Jacob Pan wrote: From: Lu Baolu An IOMMU domain represents an address space which can be attached by devices that perform DMA within a domain. However, for platforms with PASID capability the domain attachment needs be handled at device+PASID level. There can be multiple PASIDs within a device and multiple devices attached to a given domain. This patch introduces a new IOMMU op which support device, PASID, and IOMMU domain attachment. The immediate use case is for PASID capable devices to perform DMA under DMA APIs. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan --- include/linux/iommu.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 369f05c2a4e2..fde5b933dbe3 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { * @aux_get_pasid: get the pasid given an aux-domain * @sva_bind: Bind process address space to device * @sva_unbind: Unbind process address space from device + * @attach_dev_pasid: attach an iommu domain to a pasid of device + * @detach_dev_pasid: detach an iommu domain from a pasid of device Isn't that operation "assign a PASID to a domain" instead? In patch 5, the domain is already attached to the device, so set_domain_pasid() might be clearer and to the point. If the IOMMU driver did the allocation we could also avoid patch 1. iiuc this API can also work for future SIOV usage where each mdev attached to the domain has its own pasid. "assigning a PASID to a domain" sounds like going back to the previous aux domain approach which has one PASID per domain and that PASID is used on all devices attached to the aux domain... This also works for SVA as far as I can see. The sva_bind essentially is to attach an SVA domain to the PASID of a device. The sva_bind/unbind ops could be removed with these two new callbacks. Best regards, baolu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
On 2022/3/15 13:07, Jacob Pan wrote: From: Lu Baolu An IOMMU domain represents an address space which can be attached by devices that perform DMA within a domain. However, for platforms with PASID capability the domain attachment needs be handled at device+PASID level. There can be multiple PASIDs within a device and multiple devices attached to a given domain. This patch introduces a new IOMMU op which support device, PASID, and IOMMU domain attachment. The immediate use case is for PASID capable devices to perform DMA under DMA APIs. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan --- include/linux/iommu.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 369f05c2a4e2..fde5b933dbe3 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { * @aux_get_pasid: get the pasid given an aux-domain * @sva_bind: Bind process address space to device * @sva_unbind: Unbind process address space from device + * @attach_dev_pasid: attach an iommu domain to a pasid of device + * @detach_dev_pasid: detach an iommu domain from a pasid of device * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @cache_invalidate: invalidate translation caches @@ -296,6 +298,10 @@ struct iommu_ops { struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, void *drvdata); void (*sva_unbind)(struct iommu_sva *handle); + int (*attach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t id); + void (*detach_dev_pasid)(struct iommu_domain *domain, +struct device *dev, ioasid_t id); As we have introduced iommu_domain_ops, these callbacks should be part of the domain ops. u32 (*sva_get_pasid)(struct iommu_sva *handle); int (*page_response)(struct device *dev, Best regards, baolu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
Hi Kevin, On Tue, 15 Mar 2022 11:49:57 +, "Tian, Kevin" wrote: > > From: Jean-Philippe Brucker > > Sent: Tuesday, March 15, 2022 7:27 PM > > > > On Mon, Mar 14, 2022 at 10:07:06PM -0700, Jacob Pan wrote: > > > From: Lu Baolu > > > > > > An IOMMU domain represents an address space which can be attached by > > > devices that perform DMA within a domain. However, for platforms with > > > PASID capability the domain attachment needs be handled at > > > device+PASID level. There can be multiple PASIDs within a device and > > > multiple devices attached to a given domain. > > > This patch introduces a new IOMMU op which support device, PASID, and > > > IOMMU domain attachment. The immediate use case is for PASID capable > > > devices to perform DMA under DMA APIs. > > > > > > Signed-off-by: Lu Baolu > > > Signed-off-by: Jacob Pan > > > --- > > > include/linux/iommu.h | 6 ++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > > index 369f05c2a4e2..fde5b933dbe3 100644 > > > --- a/include/linux/iommu.h > > > +++ b/include/linux/iommu.h > > > @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { > > > * @aux_get_pasid: get the pasid given an aux-domain > > > * @sva_bind: Bind process address space to device > > > * @sva_unbind: Unbind process address space from device > > > + * @attach_dev_pasid: attach an iommu domain to a pasid of device > > > + * @detach_dev_pasid: detach an iommu domain from a pasid of device > > > > Isn't that operation "assign a PASID to a domain" instead? In patch 5, > > the domain is already attached to the device, so set_domain_pasid() > > might be clearer and to the point. If the IOMMU driver did the > > allocation we could also avoid patch 1. I agree, we could let vendor driver do the allocation inside this op. On the other side, we could also keep the flexibility such that this op can be used for guest PASID bind, with a SVA domain. > > iiuc this API can also work for future SIOV usage where each mdev attached > to the domain has its own pasid. "assigning a PASID to a domain" sounds > like going back to the previous aux domain approach which has one PASID > per domain and that PASID is used on all devices attached to the aux > domain... > yes, that is the intention. I plan to lift the requirement in patch 5 such that device attachment will not be a prerequisite. That would be after mdev adoption. > > > > If I understand correctly this series is not about a generic PASID API > > that allows drivers to manage multiple DMA address spaces, because there > > still doesn't seem to be any interest in that. It's about the specific > > IDXD use-case, so let's focus on that. We can introduce a specialized > > call such as (iommu|dma)_set_device_pasid(), which will be easy to > > consolidate later into a more generic "dma_enable_pasid()" API if that > > ever seems useful. > > Right, at the moment it is still a single address space. i.e. the current domain of the device/group. But this limitation is at the driver facing API layer not limited in the IOMMU ops. > > Thanks, > > Jean > > > > > * @sva_get_pasid: Get PASID associated to a SVA handle > > > * @page_response: handle page request response > > > * @cache_invalidate: invalidate translation caches > > > @@ -296,6 +298,10 @@ struct iommu_ops { > > > struct iommu_sva *(*sva_bind)(struct device *dev, struct > > > mm_struct > > *mm, > > > void *drvdata); > > > void (*sva_unbind)(struct iommu_sva *handle); > > > + int (*attach_dev_pasid)(struct iommu_domain *domain, > > > + struct device *dev, ioasid_t id); > > > + void (*detach_dev_pasid)(struct iommu_domain *domain, > > > + struct device *dev, ioasid_t id); > > > u32 (*sva_get_pasid)(struct iommu_sva *handle); > > > > > > int (*page_response)(struct device *dev, > > > -- > > > 2.25.1 > > > Thanks, Jacob ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
> From: Jean-Philippe Brucker > Sent: Tuesday, March 15, 2022 7:27 PM > > On Mon, Mar 14, 2022 at 10:07:06PM -0700, Jacob Pan wrote: > > From: Lu Baolu > > > > An IOMMU domain represents an address space which can be attached by > > devices that perform DMA within a domain. However, for platforms with > > PASID capability the domain attachment needs be handled at device+PASID > > level. There can be multiple PASIDs within a device and multiple devices > > attached to a given domain. > > This patch introduces a new IOMMU op which support device, PASID, and > > IOMMU domain attachment. The immediate use case is for PASID capable > > devices to perform DMA under DMA APIs. > > > > Signed-off-by: Lu Baolu > > Signed-off-by: Jacob Pan > > --- > > include/linux/iommu.h | 6 ++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > > index 369f05c2a4e2..fde5b933dbe3 100644 > > --- a/include/linux/iommu.h > > +++ b/include/linux/iommu.h > > @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { > > * @aux_get_pasid: get the pasid given an aux-domain > > * @sva_bind: Bind process address space to device > > * @sva_unbind: Unbind process address space from device > > + * @attach_dev_pasid: attach an iommu domain to a pasid of device > > + * @detach_dev_pasid: detach an iommu domain from a pasid of device > > Isn't that operation "assign a PASID to a domain" instead? In patch 5, > the domain is already attached to the device, so set_domain_pasid() might > be clearer and to the point. If the IOMMU driver did the allocation we > could also avoid patch 1. iiuc this API can also work for future SIOV usage where each mdev attached to the domain has its own pasid. "assigning a PASID to a domain" sounds like going back to the previous aux domain approach which has one PASID per domain and that PASID is used on all devices attached to the aux domain... > > If I understand correctly this series is not about a generic PASID API > that allows drivers to manage multiple DMA address spaces, because there > still doesn't seem to be any interest in that. It's about the specific > IDXD use-case, so let's focus on that. We can introduce a specialized call > such as (iommu|dma)_set_device_pasid(), which will be easy to consolidate > later into a more generic "dma_enable_pasid()" API if that ever seems > useful. > > Thanks, > Jean > > > * @sva_get_pasid: Get PASID associated to a SVA handle > > * @page_response: handle page request response > > * @cache_invalidate: invalidate translation caches > > @@ -296,6 +298,10 @@ struct iommu_ops { > > struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct > *mm, > > void *drvdata); > > void (*sva_unbind)(struct iommu_sva *handle); > > + int (*attach_dev_pasid)(struct iommu_domain *domain, > > + struct device *dev, ioasid_t id); > > + void (*detach_dev_pasid)(struct iommu_domain *domain, > > +struct device *dev, ioasid_t id); > > u32 (*sva_get_pasid)(struct iommu_sva *handle); > > > > int (*page_response)(struct device *dev, > > -- > > 2.25.1 > > ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
On Mon, Mar 14, 2022 at 10:07:06PM -0700, Jacob Pan wrote: > From: Lu Baolu > > An IOMMU domain represents an address space which can be attached by > devices that perform DMA within a domain. However, for platforms with > PASID capability the domain attachment needs be handled at device+PASID > level. There can be multiple PASIDs within a device and multiple devices > attached to a given domain. > This patch introduces a new IOMMU op which support device, PASID, and > IOMMU domain attachment. The immediate use case is for PASID capable > devices to perform DMA under DMA APIs. > > Signed-off-by: Lu Baolu > Signed-off-by: Jacob Pan > --- > include/linux/iommu.h | 6 ++ > 1 file changed, 6 insertions(+) > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 369f05c2a4e2..fde5b933dbe3 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { > * @aux_get_pasid: get the pasid given an aux-domain > * @sva_bind: Bind process address space to device > * @sva_unbind: Unbind process address space from device > + * @attach_dev_pasid: attach an iommu domain to a pasid of device > + * @detach_dev_pasid: detach an iommu domain from a pasid of device Isn't that operation "assign a PASID to a domain" instead? In patch 5, the domain is already attached to the device, so set_domain_pasid() might be clearer and to the point. If the IOMMU driver did the allocation we could also avoid patch 1. If I understand correctly this series is not about a generic PASID API that allows drivers to manage multiple DMA address spaces, because there still doesn't seem to be any interest in that. It's about the specific IDXD use-case, so let's focus on that. We can introduce a specialized call such as (iommu|dma)_set_device_pasid(), which will be easy to consolidate later into a more generic "dma_enable_pasid()" API if that ever seems useful. Thanks, Jean > * @sva_get_pasid: Get PASID associated to a SVA handle > * @page_response: handle page request response > * @cache_invalidate: invalidate translation caches > @@ -296,6 +298,10 @@ struct iommu_ops { > struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, > void *drvdata); > void (*sva_unbind)(struct iommu_sva *handle); > + int (*attach_dev_pasid)(struct iommu_domain *domain, > + struct device *dev, ioasid_t id); > + void (*detach_dev_pasid)(struct iommu_domain *domain, > + struct device *dev, ioasid_t id); > u32 (*sva_get_pasid)(struct iommu_sva *handle); > > int (*page_response)(struct device *dev, > -- > 2.25.1 > ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
> From: Jacob Pan > Sent: Tuesday, March 15, 2022 1:07 PM > > From: Lu Baolu > > An IOMMU domain represents an address space which can be attached by > devices that perform DMA within a domain. However, for platforms with > PASID capability the domain attachment needs be handled at device+PASID > level. There can be multiple PASIDs within a device and multiple devices > attached to a given domain. > This patch introduces a new IOMMU op which support device, PASID, and > IOMMU domain attachment. The immediate use case is for PASID capable > devices to perform DMA under DMA APIs. > > Signed-off-by: Lu Baolu > Signed-off-by: Jacob Pan > --- > include/linux/iommu.h | 6 ++ > 1 file changed, 6 insertions(+) > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 369f05c2a4e2..fde5b933dbe3 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { > * @aux_get_pasid: get the pasid given an aux-domain > * @sva_bind: Bind process address space to device > * @sva_unbind: Unbind process address space from device > + * @attach_dev_pasid: attach an iommu domain to a pasid of device > + * @detach_dev_pasid: detach an iommu domain from a pasid of device > * @sva_get_pasid: Get PASID associated to a SVA handle > * @page_response: handle page request response > * @cache_invalidate: invalidate translation caches > @@ -296,6 +298,10 @@ struct iommu_ops { > struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct > *mm, > void *drvdata); > void (*sva_unbind)(struct iommu_sva *handle); > + int (*attach_dev_pasid)(struct iommu_domain *domain, > + struct device *dev, ioasid_t id); > + void (*detach_dev_pasid)(struct iommu_domain *domain, > + struct device *dev, ioasid_t id); > u32 (*sva_get_pasid)(struct iommu_sva *handle); > > int (*page_response)(struct device *dev, > -- > 2.25.1 Probably this can be combined with patch05 to demonstrate the usage together with the definition of the new ops. Then comes the vt-d specific change. Thanks Kevin ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops
From: Lu Baolu An IOMMU domain represents an address space which can be attached by devices that perform DMA within a domain. However, for platforms with PASID capability the domain attachment needs be handled at device+PASID level. There can be multiple PASIDs within a device and multiple devices attached to a given domain. This patch introduces a new IOMMU op which support device, PASID, and IOMMU domain attachment. The immediate use case is for PASID capable devices to perform DMA under DMA APIs. Signed-off-by: Lu Baolu Signed-off-by: Jacob Pan --- include/linux/iommu.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 369f05c2a4e2..fde5b933dbe3 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -227,6 +227,8 @@ struct iommu_iotlb_gather { * @aux_get_pasid: get the pasid given an aux-domain * @sva_bind: Bind process address space to device * @sva_unbind: Unbind process address space from device + * @attach_dev_pasid: attach an iommu domain to a pasid of device + * @detach_dev_pasid: detach an iommu domain from a pasid of device * @sva_get_pasid: Get PASID associated to a SVA handle * @page_response: handle page request response * @cache_invalidate: invalidate translation caches @@ -296,6 +298,10 @@ struct iommu_ops { struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm, void *drvdata); void (*sva_unbind)(struct iommu_sva *handle); + int (*attach_dev_pasid)(struct iommu_domain *domain, + struct device *dev, ioasid_t id); + void (*detach_dev_pasid)(struct iommu_domain *domain, +struct device *dev, ioasid_t id); u32 (*sva_get_pasid)(struct iommu_sva *handle); int (*page_response)(struct device *dev, -- 2.25.1 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu