Re: [RFC 2/3] iommu: Account for dma_mask and iommu aperture in IOVA reserved regions

2020-09-29 Thread Auger Eric
Hi Christoph,

On 9/29/20 8:03 AM, Christoph Hellwig wrote:
> On Mon, Sep 28, 2020 at 09:50:36PM +0200, Eric Auger wrote:
>> VFIO currently exposes the usable IOVA regions through the
>> VFIO_IOMMU_GET_INFO ioctl. However it fails to take into account
>> the dma_mask of the devices within the container. The top limit
>> currently is defined by the iommu aperture.
> 
> Can we take a step back here?  The dma_mask only has a meaning for
> the DMA API, and not the iommu API, it should have no relevance here.
> 
> More importantly if we are using vfio no dma_mask should be set to
> start with.

You will find more context in my reply to Alex.

Thanks

Eric
> 
>> +if (geo.aperture_end < ULLONG_MAX && geo.aperture_end != 
>> geo.aperture_start) {
> 
> Please avoid pointlessly overlong lines.
> 

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


Re: [RFC 2/3] iommu: Account for dma_mask and iommu aperture in IOVA reserved regions

2020-09-29 Thread Christoph Hellwig
On Mon, Sep 28, 2020 at 09:50:36PM +0200, Eric Auger wrote:
> VFIO currently exposes the usable IOVA regions through the
> VFIO_IOMMU_GET_INFO ioctl. However it fails to take into account
> the dma_mask of the devices within the container. The top limit
> currently is defined by the iommu aperture.

Can we take a step back here?  The dma_mask only has a meaning for
the DMA API, and not the iommu API, it should have no relevance here.

More importantly if we are using vfio no dma_mask should be set to
start with.

> + if (geo.aperture_end < ULLONG_MAX && geo.aperture_end != 
> geo.aperture_start) {

Please avoid pointlessly overlong lines.
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu


[RFC 2/3] iommu: Account for dma_mask and iommu aperture in IOVA reserved regions

2020-09-28 Thread Eric Auger
VFIO currently exposes the usable IOVA regions through the
VFIO_IOMMU_GET_INFO ioctl. However it fails to take into account
the dma_mask of the devices within the container. The top limit
currently is defined by the iommu aperture.

So, for instance, if the IOMMU supports up to 48bits, it may give
the impression the max IOVA is 48b while a device may have a
dma_mask of 42b. So this API cannot really be used to compute
the max usable IOVA.

This patch removes the IOVA region beyond the dma_mask's. As we
start to expose this reserved region in the sysfs file
/sys/kernel/iommu_groups//reserved_regions, we also need to
expose the IOVA range beyond the IOMMU aperture to handle the case
where the dma_mask would have a higher number of bits than the iommu
max input address. Those out-of-reach regions get the
IOMMU_RESV_RESERVED type.

This is a change to the ABI as this reserved region was not yet
exposed in sysfs /sys/kernel/iommu_groups//reserved_regions or
through the VFIO ioctl. Document that change.

Signed-off-by: Eric Auger 
---
 .../ABI/testing/sysfs-kernel-iommu_groups |  7 
 drivers/iommu/iommu.c | 39 +++
 2 files changed, 46 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-kernel-iommu_groups 
b/Documentation/ABI/testing/sysfs-kernel-iommu_groups
index 017f5bc3920c..2f316686c88b 100644
--- a/Documentation/ABI/testing/sysfs-kernel-iommu_groups
+++ b/Documentation/ABI/testing/sysfs-kernel-iommu_groups
@@ -33,3 +33,10 @@ Description:In case an RMRR is used only by graphics or 
USB devices
it is now exposed as "direct-relaxable" instead of "direct".
In device assignment use case, for instance, those RMRR
are considered to be relaxable and safe.
+
+What:  /sys/kernel/iommu_groups/reserved_regions
+Date:  Sept 2020
+KernelVersion:  v5.11
+Contact:   Eric Auger 
+Description:Regions beyond the device dma_mask and the iommu aperture
+   now are exposed as IOMMU_RESV_RESERVED reserved regions.
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index dd8cda340e62..d797f07b3625 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2511,9 +2511,48 @@ EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
 void iommu_get_resv_regions(struct device *dev, struct list_head *list)
 {
const struct iommu_ops *ops = dev->bus->iommu_ops;
+   struct iommu_resv_region *region;
+   struct iommu_domain *domain;
+
+   domain = iommu_get_domain_for_dev(dev);
+
+   if (domain) {
+   struct iommu_domain_geometry geo;
+
+   if (iommu_domain_get_attr(domain, DOMAIN_ATTR_GEOMETRY, ))
+   return;
+
+   if (geo.aperture_end < ULLONG_MAX && geo.aperture_end != 
geo.aperture_start) {
+   region = iommu_alloc_resv_region(geo.aperture_end + 1,
+ULLONG_MAX - 
geo.aperture_end,
+0, 
IOMMU_RESV_RESERVED);
+   if (!region)
+   return;
+   list_add_tail(>list, list);
+   }
+
+   if (geo.aperture_start > 0) {
+   region = iommu_alloc_resv_region(0, geo.aperture_start,
+0, 
IOMMU_RESV_RESERVED);
+   if (!region)
+   return;
+   list_add_tail(>list, list);
+   }
+   }
 
if (ops && ops->get_resv_regions)
ops->get_resv_regions(dev, list);
+
+   if (!dev->dma_mask || *dev->dma_mask == ULLONG_MAX)
+   return;
+
+   region = iommu_alloc_resv_region(*dev->dma_mask + 1,
+ULLONG_MAX - *dev->dma_mask,
+0, IOMMU_RESV_RESERVED);
+   if (!region)
+   return;
+
+   list_add_tail(>list, list);
 }
 
 void iommu_put_resv_regions(struct device *dev, struct list_head *list)
-- 
2.21.3

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu