Re: [PATCH v6 02/15] iommu: Report domain nesting info
On Sun, 16 Aug 2020 14:40:57 +0200
Auger Eric wrote:
> Hi Yi,
>
> On 8/14/20 9:15 AM, Liu, Yi L wrote:
> > Hi Eric,
> >
> >> From: Auger Eric
> >> Sent: Thursday, August 13, 2020 8:53 PM
> >>
> >> Yi,
> >> On 7/28/20 8:27 AM, Liu Yi L wrote:
> >>> IOMMUs that support nesting translation needs report the
> >>> capability info
> >> s/needs/need to
> >>> to userspace. It gives information about requirements the
> >>> userspace needs to implement plus other features characterizing
> >>> the physical implementation.
> >>>
> >>> This patch reports nesting info by DOMAIN_ATTR_NESTING. Caller
> >>> can get nesting info after setting DOMAIN_ATTR_NESTING. For VFIO,
> >>> it is after selecting VFIO_TYPE1_NESTING_IOMMU.
> >> This is not what this patch does ;-) It introduces a new IOMMU UAPI
> >> struct that gives information about the nesting capabilities and
> >> features. This struct is supposed to be returned by
> >> iommu_domain_get_attr() with DOMAIN_ATTR_NESTING attribute
> >> parameter, one a domain whose type has been set to
> >> DOMAIN_ATTR_NESTING.
> >
> > got it. let me apply your suggestion. thanks. :-)
> >
> >>>
> >>> Cc: Kevin Tian
> >>> CC: Jacob Pan
> >>> Cc: Alex Williamson
> >>> Cc: Eric Auger
> >>> Cc: Jean-Philippe Brucker
> >>> Cc: Joerg Roedel
> >>> Cc: Lu Baolu
> >>> Signed-off-by: Liu Yi L
> >>> Signed-off-by: Jacob Pan
> >>> ---
> >>> v5 -> v6:
> >>> *) rephrase the feature notes per comments from Eric Auger.
> >>> *) rename @size of struct iommu_nesting_info to @argsz.
> >>>
> >>> v4 -> v5:
> >>> *) address comments from Eric Auger.
> >>>
> >>> v3 -> v4:
> >>> *) split the SMMU driver changes to be a separate patch
> >>> *) move the @addr_width and @pasid_bits from vendor specific
> >>>part to generic part.
> >>> *) tweak the description for the @features field of struct
> >>>iommu_nesting_info.
> >>> *) add description on the @data[] field of struct
> >>> iommu_nesting_info
> >>>
> >>> v2 -> v3:
> >>> *) remvoe cap/ecap_mask in iommu_nesting_info.
> >>> *) reuse DOMAIN_ATTR_NESTING to get nesting info.
> >>> *) return an empty iommu_nesting_info for SMMU drivers per Jean'
> >>>suggestion.
> >>> ---
> >>> include/uapi/linux/iommu.h | 74
> >> ++
> >>> 1 file changed, 74 insertions(+)
> >>>
> >>> diff --git a/include/uapi/linux/iommu.h
> >>> b/include/uapi/linux/iommu.h index 7c8e075..5e4745a 100644
> >>> --- a/include/uapi/linux/iommu.h
> >>> +++ b/include/uapi/linux/iommu.h
> >>> @@ -332,4 +332,78 @@ struct iommu_gpasid_bind_data {
> >>> } vendor;
> >>> };
> >>>
> >>> +/*
> >>> + * struct iommu_nesting_info - Information for nesting-capable
> >>> IOMMU.
> >>> + * userspace should check it
> >>> before using
> >>> + * nesting capability.
> >>> + *
> >>> + * @argsz: size of the whole structure.
> >>> + * @flags: currently reserved for future extension. must
> >>> set to 0.
> >>> + * @format: PASID table entry format, the same definition
> >>> as struct
> >>> + * iommu_gpasid_bind_data @format.
> >>> + * @features:supported nesting features.
> >>> + * @addr_width: The output addr width of first
> >>> level/stage translation
> >>> + * @pasid_bits: Maximum supported PASID bits, 0
> >>> represents no PASID
> >>> + * support.
> >>> + * @data:vendor specific cap info. data[] structure type
> >>> can be deduced
> >>> + * from @format field.
> >>> + *
> >>> + *
> >> +===+===
> >> ===+
> >>> + * | feature |
> >>> Notes |
> >>> + *
> >> +===+===
> >> ===+
> >>> + * | SYSWIDE_PASID | IOMMU vendor driver sets it to mandate
> >>> userspace|
> >>> + * | | to allocate PASID from kernel. All PASID
> >>> allocation |
> >>> + * | | free must be mediated through the TBD
> >>> API. |
> >> s/TBD/IOMMU
> >
> > got it.
> >
> >>> + *
> >>> +---+--+
> >>> + * | BIND_PGTBL| IOMMU vendor driver sets it to mandate
> >>> userspace|
> >>> + * | | bind the first level/stage page table to
> >>> associated |
> >> s/bind/to bind
> >
> > got it.
> >
> >>> + * | | PASID (either the one specified in bind
> >>> request or |
> >>> + * | | the default PASID of iommu domain),
> >>> through IOMMU |
> >>> + * | |
> >>> UAPI. |
> >>> + *
> >>> +---+--+
> >>> + * | CACHE_INVLD | IOMMU vendor driver sets it to mandate
> >>> userspace|
> >>
> >>> + * | | explicitly invalidate the IOMMU cache
> >>> through IOMMU |
[PATCH] iommu/vt-d:Add support for probing ACPI device in RMRR
After acpi device in RMRR is detected,it is necessary
to establish a mapping for these devices.
In acpi_device_create_direct_mappings(),create a mapping
for the acpi device in RMRR.
Add a helper to achieve the acpi namespace device can
access the RMRR region.
Signed-off-by: FelixCuioc
---
drivers/iommu/intel/iommu.c | 27 +++
drivers/iommu/iommu.c | 6 ++
include/linux/iommu.h | 3 +++
3 files changed, 36 insertions(+)
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index f774ef63d473..b31f02f41c96 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -4797,6 +4797,20 @@ static int __init platform_optin_force_iommu(void)
return 1;
}
+static int acpi_device_create_direct_mappings(struct device *pn_dev, struct
device *acpi_device)
+{
+ struct iommu_group *group;
+
+ acpi_device->bus->iommu_ops = _iommu_ops;
+ group = iommu_group_get(pn_dev);
+ if (!group) {
+ pr_warn("ACPI name space devices create direct mappings
wrong!\n");
+ return -1;
+ }
+ __acpi_device_create_direct_mappings(group, acpi_device);
+
+ return 0;
+}
static int __init probe_acpi_namespace_devices(void)
{
@@ -4812,6 +4826,7 @@ static int __init probe_acpi_namespace_devices(void)
struct acpi_device_physical_node *pn;
struct iommu_group *group;
struct acpi_device *adev;
+ struct device *pn_dev = NULL;
if (dev->bus != _bus_type)
continue;
@@ -4822,6 +4837,7 @@ static int __init probe_acpi_namespace_devices(void)
>physical_node_list, node) {
group = iommu_group_get(pn->dev);
if (group) {
+ pn_dev = pn->dev;
iommu_group_put(group);
continue;
}
@@ -4830,7 +4846,18 @@ static int __init probe_acpi_namespace_devices(void)
ret = iommu_probe_device(pn->dev);
if (ret)
break;
+ pn_dev = pn->dev;
+ }
+ if (pn_dev == NULL) {
+ dev->bus->iommu_ops = _iommu_ops;
+ ret = iommu_probe_device(dev);
+ if (ret) {
+ pr_err("acpi_device probe fail!
ret:%d\n", ret);
+ return ret;
+ }
+ return 0;
}
+ ret = acpi_device_create_direct_mappings(pn_dev, dev);
mutex_unlock(>physical_node_lock);
if (ret)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 609bd25bf154..4f714a2d5ef7 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -779,6 +779,12 @@ static bool iommu_is_attach_deferred(struct iommu_domain
*domain,
return false;
}
+void __acpi_device_create_direct_mappings(struct iommu_group *group, struct
device *acpi_device)
+{
+ iommu_create_device_direct_mappings(group, acpi_device);
+}
+EXPORT_SYMBOL_GPL(__acpi_device_create_direct_mappings);
+
/**
* iommu_group_add_device - add a device to an iommu group
* @group: the group into which to add the device (reference should be held)
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index fee209efb756..9be134775886 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -514,6 +514,9 @@ extern void iommu_domain_window_disable(struct iommu_domain
*domain, u32 wnd_nr)
extern int report_iommu_fault(struct iommu_domain *domain, struct device *dev,
unsigned long iova, int flags);
+extern void __acpi_device_create_direct_mappings(struct iommu_group *group,
+ struct device *acpi_device);
+
static inline void iommu_flush_tlb_all(struct iommu_domain *domain)
{
if (domain->ops->flush_iotlb_all)
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH] iommu/vt-d:Add support for detecting ACPI device in RMRR
Some ACPI devices need to issue dma requests to access
the reserved memory area.BIOS uses the device scope type
ACPI_NAMESPACE_DEVICE in RMRR to report these ACPI devices.
This patch add support for detecting ACPI devices in RMRR.
Signed-off-by: FelixCuioc
---
drivers/iommu/intel/dmar.c | 74 -
drivers/iommu/intel/iommu.c | 22 ++-
include/linux/dmar.h| 12 +-
3 files changed, 72 insertions(+), 36 deletions(-)
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 93e6345f3414..024ca38dba12 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -215,7 +215,7 @@ static bool dmar_match_pci_path(struct dmar_pci_notify_info
*info, int bus,
}
/* Return: > 0 if match found, 0 if no match found, < 0 if error happens */
-int dmar_insert_dev_scope(struct dmar_pci_notify_info *info,
+int dmar_pci_insert_dev_scope(struct dmar_pci_notify_info *info,
void *start, void*end, u16 segment,
struct dmar_dev_scope *devices,
int devices_cnt)
@@ -304,7 +304,7 @@ static int dmar_pci_bus_add_dev(struct dmar_pci_notify_info
*info)
drhd = container_of(dmaru->hdr,
struct acpi_dmar_hardware_unit, header);
- ret = dmar_insert_dev_scope(info, (void *)(drhd + 1),
+ ret = dmar_pci_insert_dev_scope(info, (void *)(drhd + 1),
((void *)drhd) + drhd->header.length,
dmaru->segment,
dmaru->devices, dmaru->devices_cnt);
@@ -696,48 +696,56 @@ dmar_find_matched_drhd_unit(struct pci_dev *dev)
return dmaru;
}
-
-static void __init dmar_acpi_insert_dev_scope(u8 device_number,
- struct acpi_device *adev)
+int dmar_acpi_insert_dev_scope(u8 device_number,
+ struct acpi_device *adev,
+ void *start, void *end,
+ struct dmar_dev_scope *devices,
+ int devices_cnt)
{
- struct dmar_drhd_unit *dmaru;
- struct acpi_dmar_hardware_unit *drhd;
struct acpi_dmar_device_scope *scope;
struct device *tmp;
int i;
struct acpi_dmar_pci_path *path;
+ for (; start < end; start += scope->length) {
+ scope = start;
+ if (scope->entry_type != ACPI_DMAR_SCOPE_TYPE_NAMESPACE)
+ continue;
+ if (scope->enumeration_id != device_number)
+ continue;
+ path = (void *)(scope + 1);
+ for_each_dev_scope(devices, devices_cnt, i, tmp)
+ if (tmp == NULL) {
+ devices[i].bus = scope->bus;
+ devices[i].devfn = PCI_DEVFN(path->device,
path->function);
+ rcu_assign_pointer(devices[i].dev,
+ get_device(>dev));
+ return 1;
+ }
+ WARN_ON(i >= devices_cnt);
+ }
+ return 0;
+}
+static int dmar_acpi_bus_add_dev(u8 device_number, struct acpi_device *adev)
+{
+ struct dmar_drhd_unit *dmaru;
+ struct acpi_dmar_hardware_unit *drhd;
+ int ret = 0;
+
for_each_drhd_unit(dmaru) {
drhd = container_of(dmaru->hdr,
struct acpi_dmar_hardware_unit,
header);
+ ret = dmar_acpi_insert_dev_scope(device_number, adev, (void
*)(drhd+1),
+ ((void
*)drhd)+drhd->header.length,
+ dmaru->devices,
dmaru->devices_cnt);
+ if (ret)
+ break;
+ }
+ ret = dmar_rmrr_add_acpi_dev(device_number, adev);
- for (scope = (void *)(drhd + 1);
-(unsigned long)scope < ((unsigned long)drhd) +
drhd->header.length;
-scope = ((void *)scope) + scope->length) {
- if (scope->entry_type != ACPI_DMAR_SCOPE_TYPE_NAMESPACE)
- continue;
- if (scope->enumeration_id != device_number)
- continue;
+ return ret;
- path = (void *)(scope + 1);
- pr_info("ACPI device \"%s\" under DMAR at %llx as
%02x:%02x.%d\n",
- dev_name(>dev), dmaru->reg_base_addr,
- scope->bus, path->device, path->function);
- for_each_dev_scope(dmaru->devices, dmaru->devices_cnt,
i, tmp)
- if (tmp == NULL) {
- dmaru->devices[i].bus =
Re: [PATCH 1/2] Add new flush_iotlb_range and handle freelists when using iommu_unmap_fast
Hi Tom,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on tegra/for-next]
[also build test ERROR on vfio/next v5.9-rc1 next-20200817]
[cannot apply to iommu/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url:
https://github.com/0day-ci/linux/commits/Tom-Murphy/Add-new-flush_iotlb_range-and-handle-freelists-when-using-iommu_unmap_fast/20200818-052129
base: https://git.kernel.org/pub/scm/linux/kernel/git/tegra/linux.git for-next
config: arm-allyesconfig (attached as .config)
compiler: arm-linux-gnueabi-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O
~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot
All errors (new ones prefixed by >>):
>> drivers/iommu/mtk_iommu_v1.c:526:12: error: initialization of 'size_t
>> (*)(struct iommu_domain *, long unsigned int, size_t, struct
>> iommu_iotlb_gather *, struct page **)' {aka 'unsigned int (*)(struct
>> iommu_domain *, long unsigned int, unsigned int, struct iommu_iotlb_gather
>> *, struct page **)'} from incompatible pointer type 'size_t (*)(struct
>> iommu_domain *, long unsigned int, size_t, struct iommu_iotlb_gather *)'
>> {aka 'unsigned int (*)(struct iommu_domain *, long unsigned int, unsigned
>> int, struct iommu_iotlb_gather *)'} [-Werror=incompatible-pointer-types]
526 | .unmap = mtk_iommu_unmap,
|^~~
drivers/iommu/mtk_iommu_v1.c:526:12: note: (near initialization for
'mtk_iommu_ops.unmap')
cc1: some warnings being treated as errors
#
https://github.com/0day-ci/linux/commit/d64ecd192f5239ba9fbf8124e7b3069cb4145c47
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review
Tom-Murphy/Add-new-flush_iotlb_range-and-handle-freelists-when-using-iommu_unmap_fast/20200818-052129
git checkout d64ecd192f5239ba9fbf8124e7b3069cb4145c47
vim +526 drivers/iommu/mtk_iommu_v1.c
b17336c55d8928 Honghui Zhang 2016-06-08 519
b65f501627fffc Arvind Yadav 2018-10-18 520 static const struct iommu_ops
mtk_iommu_ops = {
b17336c55d8928 Honghui Zhang 2016-06-08 521.domain_alloc =
mtk_iommu_domain_alloc,
b17336c55d8928 Honghui Zhang 2016-06-08 522.domain_free=
mtk_iommu_domain_free,
b17336c55d8928 Honghui Zhang 2016-06-08 523.attach_dev =
mtk_iommu_attach_device,
b17336c55d8928 Honghui Zhang 2016-06-08 524.detach_dev =
mtk_iommu_detach_device,
b17336c55d8928 Honghui Zhang 2016-06-08 525.map= mtk_iommu_map,
b17336c55d8928 Honghui Zhang 2016-06-08 @526.unmap =
mtk_iommu_unmap,
b17336c55d8928 Honghui Zhang 2016-06-08 527.iova_to_phys =
mtk_iommu_iova_to_phys,
57dbf81f50c82a Joerg Roedel 2020-04-29 528.probe_device =
mtk_iommu_probe_device,
57dbf81f50c82a Joerg Roedel 2020-04-29 529.probe_finalize =
mtk_iommu_probe_finalize,
57dbf81f50c82a Joerg Roedel 2020-04-29 530.release_device =
mtk_iommu_release_device,
8bbe13f52cb796 Yong Wu 2020-05-15 531.def_domain_type =
mtk_iommu_def_domain_type,
57dbf81f50c82a Joerg Roedel 2020-04-29 532.device_group =
generic_device_group,
b17336c55d8928 Honghui Zhang 2016-06-08 533.pgsize_bitmap = ~0UL <<
MT2701_IOMMU_PAGE_SHIFT,
b17336c55d8928 Honghui Zhang 2016-06-08 534 };
b17336c55d8928 Honghui Zhang 2016-06-08 535
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-...@lists.01.org
.config.gz
Description: application/gzip
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 01/12] ACPI/IORT: Make iort_match_node_callback walk the ACPI namespace for NC
On 2020/7/2 16:22, Hanjun Guo wrote: As I said in previous email, I'm not against this patch, and seems have no regressions for platforms that using named component node such as D05/D06 (I will test it shortly to make sure), but it's better to update the wording of the spec (even after this patch set is merged). I can see that IORT revision E was updated to add IMP_DEF input IDs in ID mappings for Named Component nodes, thanks for that. Thanks Hanjun ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v7 5/7] iommu/uapi: Rename uapi functions
On Thu, 13 Aug 2020 10:58:53 +0200
Auger Eric wrote:
> Hi Jacob,
>
> On 7/30/20 2:21 AM, Jacob Pan wrote:
> > User APIs such as iommu_sva_unbind_gpasid() may also be used by the
> > kernel. Since we introduced user pointer to the UAPI functions,
> Practically this is done in the next patch. What about something like:
>
> We plan to have two flavors of the same API functions, one called
> through ioctls, carrying a user pointer and one called directly with
> valid IOMMU UAPI structs. To differentiate both, let's rename existing
> functions with an iommu_uapi_ prefix.
>
will do. Thanks!
> Besides
> Reviewed-by: Eric Auger
>
>
> Thanks
>
> Eric
> > in-kernel callers cannot share the same APIs. In-kernel callers are
> > also trusted, there is no need to validate the data.
> >
> > This patch renames all UAPI functions with iommu_uapi_ prefix such
> > that is clear to the intended callers.
> >
> > Suggested-by: Alex Williamson
> > Signed-off-by: Jacob Pan
> > ---
> > drivers/iommu/iommu.c | 18 +-
> > include/linux/iommu.h | 31 ---
> > 2 files changed, 25 insertions(+), 24 deletions(-)
> >
> > diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> > index b6858adc4f17..3a913ce94a3d 100644
> > --- a/drivers/iommu/iommu.c
> > +++ b/drivers/iommu/iommu.c
> > @@ -1950,35 +1950,35 @@ int iommu_attach_device(struct iommu_domain
> > *domain, struct device *dev) }
> > EXPORT_SYMBOL_GPL(iommu_attach_device);
> >
> > -int iommu_cache_invalidate(struct iommu_domain *domain, struct
> > device *dev,
> > - struct iommu_cache_invalidate_info
> > *inv_info) +int iommu_uapi_cache_invalidate(struct iommu_domain
> > *domain, struct device *dev,
> > + struct iommu_cache_invalidate_info
> > *inv_info) {
> > if (unlikely(!domain->ops->cache_invalidate))
> > return -ENODEV;
> >
> > return domain->ops->cache_invalidate(domain, dev,
> > inv_info); }
> > -EXPORT_SYMBOL_GPL(iommu_cache_invalidate);
> > +EXPORT_SYMBOL_GPL(iommu_uapi_cache_invalidate);
> >
> > -int iommu_sva_bind_gpasid(struct iommu_domain *domain,
> > - struct device *dev, struct
> > iommu_gpasid_bind_data *data) +int
> > iommu_uapi_sva_bind_gpasid(struct iommu_domain *domain,
> > + struct device *dev, struct
> > iommu_gpasid_bind_data *data) {
> > if (unlikely(!domain->ops->sva_bind_gpasid))
> > return -ENODEV;
> >
> > return domain->ops->sva_bind_gpasid(domain, dev, data);
> > }
> > -EXPORT_SYMBOL_GPL(iommu_sva_bind_gpasid);
> > +EXPORT_SYMBOL_GPL(iommu_uapi_sva_bind_gpasid);
> >
> > -int iommu_sva_unbind_gpasid(struct iommu_domain *domain, struct
> > device *dev,
> > -ioasid_t pasid)
> > +int iommu_uapi_sva_unbind_gpasid(struct iommu_domain *domain,
> > struct device *dev,
> > +ioasid_t pasid)
> > {
> > if (unlikely(!domain->ops->sva_unbind_gpasid))
> > return -ENODEV;
> >
> > return domain->ops->sva_unbind_gpasid(dev, pasid);
> > }
> > -EXPORT_SYMBOL_GPL(iommu_sva_unbind_gpasid);
> > +EXPORT_SYMBOL_GPL(iommu_uapi_sva_unbind_gpasid);
> >
> > static void __iommu_detach_device(struct iommu_domain *domain,
> > struct device *dev)
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> > index 5f0b7859d2eb..2dcc1a33f6dc 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -430,13 +430,13 @@ extern int iommu_attach_device(struct
> > iommu_domain *domain, struct device *dev);
> > extern void iommu_detach_device(struct iommu_domain *domain,
> > struct device *dev);
> > -extern int iommu_cache_invalidate(struct iommu_domain *domain,
> > - struct device *dev,
> > - struct
> > iommu_cache_invalidate_info *inv_info); -extern int
> > iommu_sva_bind_gpasid(struct iommu_domain *domain,
> > - struct device *dev, struct iommu_gpasid_bind_data
> > *data); -extern int iommu_sva_unbind_gpasid(struct iommu_domain
> > *domain,
> > - struct device *dev, ioasid_t
> > pasid); +extern int iommu_uapi_cache_invalidate(struct iommu_domain
> > *domain,
> > + struct device *dev,
> > + struct
> > iommu_cache_invalidate_info *inv_info); +extern int
> > iommu_uapi_sva_bind_gpasid(struct iommu_domain *domain,
> > + struct device *dev, struct
> > iommu_gpasid_bind_data *data); +extern int
> > iommu_uapi_sva_unbind_gpasid(struct iommu_domain *domain,
> > + struct device *dev,
> > ioasid_t pasid); extern struct iommu_domain
> > *iommu_get_domain_for_dev(struct device *dev); extern struct
> > iommu_domain *iommu_get_dma_domain(struct device *dev); extern int
> > iommu_map(struct iommu_domain *domain,
Re: [PATCH v5] PCI/ACS: Enable PCI_ACS_TB and disable only when needed for ATS
Hello Bjorn,
On Sat, Aug 1, 2020 at 5:30 PM Rajat Jain wrote:
>
> Hi Bjorn,
>
>
> On Tue, Jul 14, 2020 at 1:24 PM Rajat Jain wrote:
> >
> > On Tue, Jul 14, 2020 at 1:15 PM Rajat Jain wrote:
> > >
> > > The ACS "Translation Blocking" bit blocks the translated addresses from
> > > the devices. We don't expect such traffic from devices unless ATS is
> > > enabled on them. A device sending such traffic without ATS enabled,
> > > indicates malicious intent, and thus should be blocked.
> > >
> > > Enable PCI_ACS_TB by default for all devices, and it stays enabled until
> > > atleast one of the devices downstream wants to enable ATS. It gets
> > > disabled to enable ATS on a device downstream it, and then gets enabled
> > > back on once all the downstream devices don't need ATS.
> > >
> > > Signed-off-by: Rajat Jain
>
> Just checking to see if you got a chance to look at this V5 patch.
Any feedback on this patch?
Thanks & Best Regards,
Rajat
>
> Thanks & Best Regards,
>
> Rajat
>
> > > ---
> > > Note that I'm ignoring the devices that require quirks to enable or
> > > disable ACS, instead of using the standard way for ACS configuration.
> > > The reason is that it would require adding yet another quirk table or
> > > quirk function pointer, that I don't know how to implement for those
> > > devices, and will neither have the devices to test that code.
> > >
> > > v5: Enable TB and disable ATS for all devices on boot. Disable TB later
> > > only if needed to enable ATS on downstream devices.
> > > v4: Add braces to avoid warning from kernel robot
> > > print warning for only external-facing devices.
> > > v3: print warning if ACS_TB not supported on external-facing/untrusted
> > > ports.
> > > Minor code comments fixes.
> > > v2: Commit log change
> > >
> > > drivers/pci/ats.c | 5
> > > drivers/pci/pci.c | 57 +
> > > drivers/pci/pci.h | 2 ++
> > > drivers/pci/probe.c | 2 +-
> > > include/linux/pci.h | 2 ++
> > > 5 files changed, 67 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
> > > index b761c1f72f67..e2ea9083f30f 100644
> > > --- a/drivers/pci/ats.c
> > > +++ b/drivers/pci/ats.c
> > > @@ -28,6 +28,9 @@ void pci_ats_init(struct pci_dev *dev)
> > > return;
> > >
> > > dev->ats_cap = pos;
> > > +
> > > + dev->ats_enabled = 1; /* To avoid WARN_ON from pci_disable_ats()
> > > */
> > > + pci_disable_ats(dev);
> > > }
> > >
> > > /**
> > > @@ -82,6 +85,7 @@ int pci_enable_ats(struct pci_dev *dev, int ps)
> > > }
> > > pci_write_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, ctrl);
> > >
> > > + pci_disable_acs_trans_blocking(dev);
> > > dev->ats_enabled = 1;
> > > return 0;
> > > }
> > > @@ -102,6 +106,7 @@ void pci_disable_ats(struct pci_dev *dev)
> > > ctrl &= ~PCI_ATS_CTRL_ENABLE;
> > > pci_write_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, ctrl);
> > >
> > > + pci_enable_acs_trans_blocking(dev);
> > > dev->ats_enabled = 0;
> > > }
> > > EXPORT_SYMBOL_GPL(pci_disable_ats);
> > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > index 73a862782214..614e3c1e8c56 100644
> > > --- a/drivers/pci/pci.c
> > > +++ b/drivers/pci/pci.c
> > > @@ -876,6 +876,9 @@ static void pci_std_enable_acs(struct pci_dev *dev)
> > > /* Upstream Forwarding */
> > > ctrl |= (cap & PCI_ACS_UF);
> > >
> > > + /* Translation Blocking */
> > > + ctrl |= (cap & PCI_ACS_TB);
> > > +
> > > pci_write_config_word(dev, pos + PCI_ACS_CTRL, ctrl);
> > > }
> > >
> > > @@ -904,6 +907,60 @@ static void pci_enable_acs(struct pci_dev *dev)
> > > pci_disable_acs_redir(dev);
> > > }
> > >
> > > +void pci_disable_acs_trans_blocking(struct pci_dev *pdev)
> > > +{
> > > + u16 cap, ctrl, pos;
> > > + struct pci_dev *dev;
> > > +
> > > + if (!pci_acs_enable)
> > > + return;
> > > +
> > > + for (dev = pdev; dev; dev = pci_upstream_bridge(pdev)) {
> > > +
> > > + pos = dev->acs_cap;
> > > + if (!pos)
> > > + continue;
> > > +
> > > + /*
> > > +* Disable translation blocking when first downstream
> > > +* device that needs it (for ATS) wants to enable ATS
> > > +*/
> > > + if (++dev->ats_dependencies == 1) {
> >
> > I am a little worried about a potential race condition here. I know
> > that 2 PCI devices cannot be enumerating at the same time. Do we know
> > if multiple pci_enable_ats() and pci_disable_ats() function calls can
> > be simultaneously executing (even for different devices)? If so, we
> > may need an atomic_t variable for ats_dependencies.
> >
> > Thanks,
> >
> > Rajat
> >
> >
> > > + pci_read_config_word(dev, pos + PCI_ACS_CAP,
> > > );
> > > +
Re: [PATCH v7 2/7] iommu/uapi: Add argsz for user filled data
On Wed, 12 Aug 2020 18:59:14 +0200
Auger Eric wrote:
> Hi,
>
> On 7/30/20 2:21 AM, Jacob Pan wrote:
> > As IOMMU UAPI gets extended, user data size may increase. To support
> > backward compatibiliy, this patch introduces a size field to each
> > UAPI
> s/compatibiliy/compatibility
will fix
> > data structures. It is *always* the responsibility for the user to
> > fill in the correct size. Padding fields are adjusted to ensure 8
> > byte alignment.
> >
> > Specific scenarios for user data handling are documented in:
> > Documentation/userspace-api/iommu.rst
>
> you may mention the struct version does not need to be incremented as
> no IOCTL uses the structs yet.
Yes, good point. I also mentioned in the doc that:
"Version field is only reserved for the unlikely event of UAPI upgrade
at its entirety."
> >
> > Signed-off-by: Liu Yi L
> > Signed-off-by: Jacob Pan
> > ---
> > include/uapi/linux/iommu.h | 12 +---
> > 1 file changed, 9 insertions(+), 3 deletions(-)
> >
> > diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h
> > index e907b7091a46..d5e9014f690e 100644
> > --- a/include/uapi/linux/iommu.h
> > +++ b/include/uapi/linux/iommu.h
> > @@ -135,6 +135,7 @@ enum iommu_page_response_code {
> >
> > /**
> > * struct iommu_page_response - Generic page response information
> > + * @argsz: User filled size of this data
> > * @version: API version of this structure
> > * @flags: encodes whether the corresponding fields are valid
> > * (IOMMU_FAULT_PAGE_RESPONSE_* values)
> > @@ -143,6 +144,7 @@ enum iommu_page_response_code {
> > * @code: response code from iommu_page_response_code
> > */
> > struct iommu_page_response {
> > + __u32 argsz;
> > #define IOMMU_PAGE_RESP_VERSION_1 1
> > __u32 version;
> > #define IOMMU_PAGE_RESP_PASID_VALID(1 << 0)
> > @@ -218,6 +220,7 @@ struct iommu_inv_pasid_info {
> > /**
> > * struct iommu_cache_invalidate_info - First level/stage
> > invalidation
> > * information
> > + * @argsz: User filled size of this data
> > * @version: API version of this structure
> > * @cache: bitfield that allows to select which caches to
> > invalidate
> > * @granularity: defines the lowest granularity used for the
> > invalidation: @@ -246,6 +249,7 @@ struct iommu_inv_pasid_info {
> > * must support the used granularity.
> > */
> > struct iommu_cache_invalidate_info {
> > + __u32 argsz;
> > #define IOMMU_CACHE_INVALIDATE_INFO_VERSION_1 1
> > __u32 version;
> > /* IOMMU paging structure cache */
> > @@ -255,7 +259,7 @@ struct iommu_cache_invalidate_info {
> > #define IOMMU_CACHE_INV_TYPE_NR(3)
> > __u8cache;
> > __u8granularity;
> > - __u8padding[2];
> > + __u8padding[6];
> > union {
> > struct iommu_inv_pasid_info pasid_info;
> > struct iommu_inv_addr_info addr_info;
> > @@ -292,6 +296,7 @@ struct iommu_gpasid_bind_data_vtd {
> >
> > /**
> > * struct iommu_gpasid_bind_data - Information about device and
> > guest PASID binding
> > + * @argsz: User filled size of this data
> > * @version: Version of this data structure
> > * @format:PASID table entry format
> > * @flags: Additional information on guest bind request
> > @@ -309,17 +314,18 @@ struct iommu_gpasid_bind_data_vtd {
> > * PASID to host PASID based on this bind data.
> > */
> > struct iommu_gpasid_bind_data {
> > + __u32 argsz;
> > #define IOMMU_GPASID_BIND_VERSION_11
> > __u32 version;
> > #define IOMMU_PASID_FORMAT_INTEL_VTD 1
> > __u32 format;
> > + __u32 addr_width;
> > #define IOMMU_SVA_GPASID_VAL (1 << 0) /* guest PASID valid
> > */ __u64 flags;
> > __u64 gpgd;
> > __u64 hpasid;
> > __u64 gpasid;
> > - __u32 addr_width;
> > - __u8 padding[12];
> > + __u8 padding[8];
> > /* Vendor specific data */
> > union {
> > struct iommu_gpasid_bind_data_vtd vtd;
> >
> Reviewed-by: Eric Auger
>
> Thanks
>
> Eric
>
[Jacob Pan]
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 20/20] drm/msm: show process names in gem_describe
From: Rob Clark
In $debugfs/gem we already show any vma(s) associated with an object.
Also show process names if the vma's address space is a per-process
address space.
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/msm_drv.c | 2 +-
drivers/gpu/drm/msm/msm_gem.c | 25 +
drivers/gpu/drm/msm/msm_gem.h | 5 +
drivers/gpu/drm/msm/msm_gem_vma.c | 1 +
drivers/gpu/drm/msm/msm_gpu.c | 8 +---
drivers/gpu/drm/msm/msm_gpu.h | 2 +-
6 files changed, 34 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
index 8e70d220bba8..8d5c4f98c332 100644
--- a/drivers/gpu/drm/msm/msm_drv.c
+++ b/drivers/gpu/drm/msm/msm_drv.c
@@ -597,7 +597,7 @@ static int context_init(struct drm_device *dev, struct
drm_file *file)
kref_init(>ref);
msm_submitqueue_init(dev, ctx);
- ctx->aspace = msm_gpu_create_private_address_space(priv->gpu);
+ ctx->aspace = msm_gpu_create_private_address_space(priv->gpu, current);
file->driver_priv = ctx;
return 0;
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index 3cb7aeb93fd3..76a6c5271e57 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -842,11 +842,28 @@ void msm_gem_describe(struct drm_gem_object *obj, struct
seq_file *m)
seq_puts(m, " vmas:");
- list_for_each_entry(vma, _obj->vmas, list)
- seq_printf(m, " [%s: %08llx,%s,inuse=%d]",
- vma->aspace != NULL ? vma->aspace->name : NULL,
- vma->iova, vma->mapped ? "mapped" : "unmapped",
+ list_for_each_entry(vma, _obj->vmas, list) {
+ const char *name, *comm;
+ if (vma->aspace) {
+ struct msm_gem_address_space *aspace =
vma->aspace;
+ struct task_struct *task =
+ get_pid_task(aspace->pid, PIDTYPE_PID);
+ if (task) {
+ comm = kstrdup(task->comm, GFP_KERNEL);
+ } else {
+ comm = NULL;
+ }
+ name = aspace->name;
+ } else {
+ name = comm = NULL;
+ }
+ seq_printf(m, " [%s%s%s: aspace=%p,
%08llx,%s,inuse=%d]",
+ name, comm ? ":" : "", comm ? comm : "",
+ vma->aspace, vma->iova,
+ vma->mapped ? "mapped" : "unmapped",
vma->inuse);
+ kfree(comm);
+ }
seq_puts(m, "\n");
}
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index 9c573c4269cb..7b1c7a5f8eef 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -24,6 +24,11 @@ struct msm_gem_address_space {
spinlock_t lock; /* Protects drm_mm node allocation/removal */
struct msm_mmu *mmu;
struct kref kref;
+
+ /* For address spaces associated with a specific process, this
+* will be non-NULL:
+*/
+ struct pid *pid;
};
struct msm_gem_vma {
diff --git a/drivers/gpu/drm/msm/msm_gem_vma.c
b/drivers/gpu/drm/msm/msm_gem_vma.c
index 29cc1305cf37..80a8a266d68f 100644
--- a/drivers/gpu/drm/msm/msm_gem_vma.c
+++ b/drivers/gpu/drm/msm/msm_gem_vma.c
@@ -17,6 +17,7 @@ msm_gem_address_space_destroy(struct kref *kref)
drm_mm_takedown(>mm);
if (aspace->mmu)
aspace->mmu->funcs->destroy(aspace->mmu);
+ put_pid(aspace->pid);
kfree(aspace);
}
diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index 951850804d77..ac8961187a73 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -825,10 +825,9 @@ static int get_clocks(struct platform_device *pdev, struct
msm_gpu *gpu)
/* Return a new address space for a msm_drm_private instance */
struct msm_gem_address_space *
-msm_gpu_create_private_address_space(struct msm_gpu *gpu)
+msm_gpu_create_private_address_space(struct msm_gpu *gpu, struct task_struct
*task)
{
struct msm_gem_address_space *aspace = NULL;
-
if (!gpu)
return NULL;
@@ -836,8 +835,11 @@ msm_gpu_create_private_address_space(struct msm_gpu *gpu)
* If the target doesn't support private address spaces then return
* the global one
*/
- if (gpu->funcs->create_private_address_space)
+ if (gpu->funcs->create_private_address_space) {
aspace = gpu->funcs->create_private_address_space(gpu);
+ if (!IS_ERR(aspace))
+ aspace->pid =
[PATCH 19/20] iommu/arm-smmu: add a way for implementations to influence SCTLR
From: Rob Clark
For the Adreno GPU's SMMU, we want SCTLR.HUPCF set to ensure that
pending translations are not terminated on iova fault. Otherwise
a terminated CP read could hang the GPU by returning invalid
command-stream data.
Signed-off-by: Rob Clark
---
drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 6 ++
drivers/iommu/arm/arm-smmu/arm-smmu.c | 3 +++
drivers/iommu/arm/arm-smmu/arm-smmu.h | 3 +++
3 files changed, 12 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
index 5640d9960610..2aa6249050ff 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
@@ -127,6 +127,12 @@ static int qcom_adreno_smmu_init_context(struct
arm_smmu_domain *smmu_domain,
(smmu_domain->cfg.fmt == ARM_SMMU_CTX_FMT_AARCH64))
pgtbl_cfg->quirks |= IO_PGTABLE_QUIRK_ARM_TTBR1;
+ /*
+* On the GPU device we want to process subsequent transactions after a
+* fault to keep the GPU from hanging
+*/
+ smmu_domain->cfg.sctlr_set |= ARM_SMMU_SCTLR_HUPCF;
+
/*
* Initialize private interface with GPU:
*/
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c
b/drivers/iommu/arm/arm-smmu/arm-smmu.c
index e63a480d7f71..bbec5793faf8 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
@@ -617,6 +617,9 @@ void arm_smmu_write_context_bank(struct arm_smmu_device
*smmu, int idx)
if (IS_ENABLED(CONFIG_CPU_BIG_ENDIAN))
reg |= ARM_SMMU_SCTLR_E;
+ reg |= cfg->sctlr_set;
+ reg &= ~cfg->sctlr_clr;
+
arm_smmu_cb_write(smmu, idx, ARM_SMMU_CB_SCTLR, reg);
}
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h
b/drivers/iommu/arm/arm-smmu/arm-smmu.h
index cd75a33967bb..2df3a70a8a41 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.h
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h
@@ -144,6 +144,7 @@ enum arm_smmu_cbar_type {
#define ARM_SMMU_CB_SCTLR 0x0
#define ARM_SMMU_SCTLR_S1_ASIDPNE BIT(12)
#define ARM_SMMU_SCTLR_CFCFG BIT(7)
+#define ARM_SMMU_SCTLR_HUPCF BIT(8)
#define ARM_SMMU_SCTLR_CFIEBIT(6)
#define ARM_SMMU_SCTLR_CFREBIT(5)
#define ARM_SMMU_SCTLR_E BIT(4)
@@ -341,6 +342,8 @@ struct arm_smmu_cfg {
u16 asid;
u16 vmid;
};
+ u32 sctlr_set;/* extra bits to set in
SCTLR */
+ u32 sctlr_clr;/* bits to mask in SCTLR
*/
enum arm_smmu_cbar_type cbar;
enum arm_smmu_context_fmt fmt;
};
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 17/20] arm: dts: qcom: sm845: Set the compatible string for the GPU SMMU
From: Jordan Crouse
Set the qcom,adreno-smmu compatible string for the GPU SMMU to enable
split pagetables and per-instance pagetables for drm/msm.
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
arch/arm64/boot/dts/qcom/sdm845.dtsi | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi
b/arch/arm64/boot/dts/qcom/sdm845.dtsi
index 2884577dcb77..6a9adaa401a9 100644
--- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
@@ -4058,7 +4058,7 @@ opp-25700 {
};
adreno_smmu: iommu@504 {
- compatible = "qcom,sdm845-smmu-v2", "qcom,smmu-v2";
+ compatible = "qcom,adreno-smmu", "qcom,smmu-v2";
reg = <0 0x504 0 0x1>;
#iommu-cells = <1>;
#global-interrupts = <2>;
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 16/20] drm/msm/a6xx: Add support for per-instance pagetables
From: Jordan Crouse
Add support for using per-instance pagetables if all the dependencies are
available.
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 63 +++
drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 +
drivers/gpu/drm/msm/msm_ringbuffer.h | 1 +
3 files changed, 65 insertions(+)
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
index 5eabb0109577..d7ad6c78d787 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
@@ -81,6 +81,49 @@ static void get_stats_counter(struct msm_ringbuffer *ring,
u32 counter,
OUT_RING(ring, upper_32_bits(iova));
}
+static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,
+ struct msm_ringbuffer *ring, struct msm_file_private *ctx)
+{
+ phys_addr_t ttbr;
+ u32 asid;
+ u64 memptr = rbmemptr(ring, ttbr0);
+
+ if (ctx == a6xx_gpu->cur_ctx)
+ return;
+
+ if (msm_iommu_pagetable_params(ctx->aspace->mmu, , ))
+ return;
+
+ /* Execute the table update */
+ OUT_PKT7(ring, CP_SMMU_TABLE_UPDATE, 4);
+ OUT_RING(ring, CP_SMMU_TABLE_UPDATE_0_TTBR0_LO(lower_32_bits(ttbr)));
+
+ OUT_RING(ring,
+ CP_SMMU_TABLE_UPDATE_1_TTBR0_HI(upper_32_bits(ttbr)) |
+ CP_SMMU_TABLE_UPDATE_1_ASID(asid));
+ OUT_RING(ring, CP_SMMU_TABLE_UPDATE_2_CONTEXTIDR(0));
+ OUT_RING(ring, CP_SMMU_TABLE_UPDATE_3_CONTEXTBANK(0));
+
+ /*
+* Write the new TTBR0 to the memstore. This is good for debugging.
+*/
+ OUT_PKT7(ring, CP_MEM_WRITE, 4);
+ OUT_RING(ring, CP_MEM_WRITE_0_ADDR_LO(lower_32_bits(memptr)));
+ OUT_RING(ring, CP_MEM_WRITE_1_ADDR_HI(upper_32_bits(memptr)));
+ OUT_RING(ring, lower_32_bits(ttbr));
+ OUT_RING(ring, (asid << 16) | upper_32_bits(ttbr));
+
+ /*
+* And finally, trigger a uche flush to be sure there isn't anything
+* lingering in that part of the GPU
+*/
+
+ OUT_PKT7(ring, CP_EVENT_WRITE, 1);
+ OUT_RING(ring, 0x31);
+
+ a6xx_gpu->cur_ctx = ctx;
+}
+
static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT;
@@ -90,6 +133,8 @@ static void a6xx_submit(struct msm_gpu *gpu, struct
msm_gem_submit *submit)
struct msm_ringbuffer *ring = submit->ring;
unsigned int i;
+ a6xx_set_pagetable(a6xx_gpu, ring, submit->queue->ctx);
+
get_stats_counter(ring, REG_A6XX_RBBM_PERFCTR_CP_0_LO,
rbmemptr_stats(ring, index, cpcycles_start));
@@ -696,6 +741,8 @@ static int a6xx_hw_init(struct msm_gpu *gpu)
/* Always come up on rb 0 */
a6xx_gpu->cur_ring = gpu->rb[0];
+ a6xx_gpu->cur_ctx = NULL;
+
/* Enable the SQE_to start the CP engine */
gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1);
@@ -1008,6 +1055,21 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)
return (unsigned long)busy_time;
}
+static struct msm_gem_address_space *
+a6xx_create_private_address_space(struct msm_gpu *gpu)
+{
+ struct msm_gem_address_space *aspace = NULL;
+ struct msm_mmu *mmu;
+
+ mmu = msm_iommu_pagetable_create(gpu->aspace->mmu);
+
+ if (!IS_ERR(mmu))
+ aspace = msm_gem_address_space_create(mmu,
+ "gpu", 0x1ULL, 0x1ULL);
+
+ return aspace;
+}
+
static const struct adreno_gpu_funcs funcs = {
.base = {
.get_param = adreno_get_param,
@@ -1031,6 +1093,7 @@ static const struct adreno_gpu_funcs funcs = {
.gpu_state_put = a6xx_gpu_state_put,
#endif
.create_address_space = adreno_iommu_create_address_space,
+ .create_private_address_space =
a6xx_create_private_address_space,
},
.get_timestamp = a6xx_get_timestamp,
};
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
index 03ba60d5b07f..da22d7549d9b 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
@@ -19,6 +19,7 @@ struct a6xx_gpu {
uint64_t sqe_iova;
struct msm_ringbuffer *cur_ring;
+ struct msm_file_private *cur_ctx;
struct a6xx_gmu gmu;
};
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h
b/drivers/gpu/drm/msm/msm_ringbuffer.h
index 7764373d0ed2..0987d6bf848c 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.h
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.h
@@ -31,6 +31,7 @@ struct msm_rbmemptrs {
volatile uint32_t fence;
volatile struct msm_gpu_submit_stats stats[MSM_GPU_SUBMIT_STATS_COUNT];
+ volatile u64 ttbr0;
};
struct msm_ringbuffer {
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
[PATCH 14/20] drm/msm: Add support to create a local pagetable
From: Jordan Crouse
Add support to create a io-pgtable for use by targets that support
per-instance pagetables. In order to support per-instance pagetables the
GPU SMMU device needs to have the qcom,adreno-smmu compatible string and
split pagetables enabled.
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/msm_gpummu.c | 2 +-
drivers/gpu/drm/msm/msm_iommu.c | 199 ++-
drivers/gpu/drm/msm/msm_mmu.h| 16 ++-
3 files changed, 214 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gpummu.c b/drivers/gpu/drm/msm/msm_gpummu.c
index 310a31b05faa..aab121f4beb7 100644
--- a/drivers/gpu/drm/msm/msm_gpummu.c
+++ b/drivers/gpu/drm/msm/msm_gpummu.c
@@ -102,7 +102,7 @@ struct msm_mmu *msm_gpummu_new(struct device *dev, struct
msm_gpu *gpu)
}
gpummu->gpu = gpu;
- msm_mmu_init(>base, dev, );
+ msm_mmu_init(>base, dev, , MSM_MMU_GPUMMU);
return >base;
}
diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c
index 1b6635504069..697cc0a059d6 100644
--- a/drivers/gpu/drm/msm/msm_iommu.c
+++ b/drivers/gpu/drm/msm/msm_iommu.c
@@ -4,15 +4,210 @@
* Author: Rob Clark
*/
+#include
+#include
#include "msm_drv.h"
#include "msm_mmu.h"
struct msm_iommu {
struct msm_mmu base;
struct iommu_domain *domain;
+ atomic_t pagetables;
};
+
#define to_msm_iommu(x) container_of(x, struct msm_iommu, base)
+struct msm_iommu_pagetable {
+ struct msm_mmu base;
+ struct msm_mmu *parent;
+ struct io_pgtable_ops *pgtbl_ops;
+ phys_addr_t ttbr;
+ u32 asid;
+};
+static struct msm_iommu_pagetable *to_pagetable(struct msm_mmu *mmu)
+{
+ return container_of(mmu, struct msm_iommu_pagetable, base);
+}
+
+static int msm_iommu_pagetable_unmap(struct msm_mmu *mmu, u64 iova,
+ size_t size)
+{
+ struct msm_iommu_pagetable *pagetable = to_pagetable(mmu);
+ struct io_pgtable_ops *ops = pagetable->pgtbl_ops;
+ size_t unmapped = 0;
+
+ /* Unmap the block one page at a time */
+ while (size) {
+ unmapped += ops->unmap(ops, iova, 4096, NULL);
+ iova += 4096;
+ size -= 4096;
+ }
+
+ iommu_flush_tlb_all(to_msm_iommu(pagetable->parent)->domain);
+
+ return (unmapped == size) ? 0 : -EINVAL;
+}
+
+static int msm_iommu_pagetable_map(struct msm_mmu *mmu, u64 iova,
+ struct sg_table *sgt, size_t len, int prot)
+{
+ struct msm_iommu_pagetable *pagetable = to_pagetable(mmu);
+ struct io_pgtable_ops *ops = pagetable->pgtbl_ops;
+ struct scatterlist *sg;
+ size_t mapped = 0;
+ u64 addr = iova;
+ unsigned int i;
+
+ for_each_sg(sgt->sgl, sg, sgt->nents, i) {
+ size_t size = sg->length;
+ phys_addr_t phys = sg_phys(sg);
+
+ /* Map the block one page at a time */
+ while (size) {
+ if (ops->map(ops, addr, phys, 4096, prot, GFP_KERNEL)) {
+ msm_iommu_pagetable_unmap(mmu, iova, mapped);
+ return -EINVAL;
+ }
+
+ phys += 4096;
+ addr += 4096;
+ size -= 4096;
+ mapped += 4096;
+ }
+ }
+
+ return 0;
+}
+
+static void msm_iommu_pagetable_destroy(struct msm_mmu *mmu)
+{
+ struct msm_iommu_pagetable *pagetable = to_pagetable(mmu);
+ struct msm_iommu *iommu = to_msm_iommu(pagetable->parent);
+ struct adreno_smmu_priv *adreno_smmu =
+ dev_get_drvdata(pagetable->parent->dev);
+
+ /*
+* If this is the last attached pagetable for the parent,
+* disable TTBR0 in the arm-smmu driver
+*/
+ if (atomic_dec_return(>pagetables) == 0)
+ adreno_smmu->set_ttbr0_cfg(adreno_smmu->cookie, NULL);
+
+ free_io_pgtable_ops(pagetable->pgtbl_ops);
+ kfree(pagetable);
+}
+
+int msm_iommu_pagetable_params(struct msm_mmu *mmu,
+ phys_addr_t *ttbr, int *asid)
+{
+ struct msm_iommu_pagetable *pagetable;
+
+ if (mmu->type != MSM_MMU_IOMMU_PAGETABLE)
+ return -EINVAL;
+
+ pagetable = to_pagetable(mmu);
+
+ if (ttbr)
+ *ttbr = pagetable->ttbr;
+
+ if (asid)
+ *asid = pagetable->asid;
+
+ return 0;
+}
+
+static const struct msm_mmu_funcs pagetable_funcs = {
+ .map = msm_iommu_pagetable_map,
+ .unmap = msm_iommu_pagetable_unmap,
+ .destroy = msm_iommu_pagetable_destroy,
+};
+
+static void msm_iommu_tlb_flush_all(void *cookie)
+{
+}
+
+static void msm_iommu_tlb_flush_walk(unsigned long iova, size_t size,
+ size_t granule, void *cookie)
+{
+}
+
+static void msm_iommu_tlb_add_page(struct iommu_iotlb_gather *gather,
+ unsigned
[PATCH 18/20] arm: dts: qcom: sc7180: Set the compatible string for the GPU SMMU
From: Rob Clark
Set the qcom,adreno-smmu compatible string for the GPU SMMU to enable
split pagetables and per-instance pagetables for drm/msm.
Signed-off-by: Rob Clark
---
arch/arm64/boot/dts/qcom/sc7180.dtsi | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/boot/dts/qcom/sc7180.dtsi
b/arch/arm64/boot/dts/qcom/sc7180.dtsi
index d46b3833e52f..61ae67186691 100644
--- a/arch/arm64/boot/dts/qcom/sc7180.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180.dtsi
@@ -1937,7 +1937,7 @@ opp-18000 {
};
adreno_smmu: iommu@504 {
- compatible = "qcom,sc7180-smmu-v2", "qcom,smmu-v2";
+ compatible = "qcom,adreno-smmu", "qcom,smmu-v2";
reg = <0 0x0504 0 0x1>;
#iommu-cells = <1>;
#global-interrupts = <2>;
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 15/20] drm/msm: Add support for private address space instances
From: Jordan Crouse
Add support for allocating private address space instances. Targets that
support per-context pagetables should implement their own function to
allocate private address spaces.
The default will return a pointer to the global address space.
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/msm_drv.c | 13 +++--
drivers/gpu/drm/msm/msm_drv.h | 5 +
drivers/gpu/drm/msm/msm_gem_vma.c | 9 +
drivers/gpu/drm/msm/msm_gpu.c | 22 ++
drivers/gpu/drm/msm/msm_gpu.h | 5 +
5 files changed, 48 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
index 01845a3b8d52..8e70d220bba8 100644
--- a/drivers/gpu/drm/msm/msm_drv.c
+++ b/drivers/gpu/drm/msm/msm_drv.c
@@ -597,7 +597,7 @@ static int context_init(struct drm_device *dev, struct
drm_file *file)
kref_init(>ref);
msm_submitqueue_init(dev, ctx);
- ctx->aspace = priv->gpu ? priv->gpu->aspace : NULL;
+ ctx->aspace = msm_gpu_create_private_address_space(priv->gpu);
file->driver_priv = ctx;
return 0;
@@ -780,18 +780,19 @@ static int msm_ioctl_gem_cpu_fini(struct drm_device *dev,
void *data,
}
static int msm_ioctl_gem_info_iova(struct drm_device *dev,
- struct drm_gem_object *obj, uint64_t *iova)
+ struct drm_file *file, struct drm_gem_object *obj,
+ uint64_t *iova)
{
- struct msm_drm_private *priv = dev->dev_private;
+ struct msm_file_private *ctx = file->driver_priv;
- if (!priv->gpu)
+ if (!ctx->aspace)
return -EINVAL;
/*
* Don't pin the memory here - just get an address so that userspace can
* be productive
*/
- return msm_gem_get_iova(obj, priv->gpu->aspace, iova);
+ return msm_gem_get_iova(obj, ctx->aspace, iova);
}
static int msm_ioctl_gem_info(struct drm_device *dev, void *data,
@@ -830,7 +831,7 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void
*data,
args->value = msm_gem_mmap_offset(obj);
break;
case MSM_INFO_GET_IOVA:
- ret = msm_ioctl_gem_info_iova(dev, obj, >value);
+ ret = msm_ioctl_gem_info_iova(dev, file, obj, >value);
break;
case MSM_INFO_SET_NAME:
/* length check should leave room for terminating null: */
diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h
index 4561bfb5e745..2ca9c3c03845 100644
--- a/drivers/gpu/drm/msm/msm_drv.h
+++ b/drivers/gpu/drm/msm/msm_drv.h
@@ -249,6 +249,10 @@ int msm_gem_map_vma(struct msm_gem_address_space *aspace,
void msm_gem_close_vma(struct msm_gem_address_space *aspace,
struct msm_gem_vma *vma);
+
+struct msm_gem_address_space *
+msm_gem_address_space_get(struct msm_gem_address_space *aspace);
+
void msm_gem_address_space_put(struct msm_gem_address_space *aspace);
struct msm_gem_address_space *
@@ -434,6 +438,7 @@ static inline void __msm_file_private_destroy(struct kref
*kref)
struct msm_file_private *ctx = container_of(kref,
struct msm_file_private, ref);
+ msm_gem_address_space_put(ctx->aspace);
kfree(ctx);
}
diff --git a/drivers/gpu/drm/msm/msm_gem_vma.c
b/drivers/gpu/drm/msm/msm_gem_vma.c
index 5f6a11211b64..29cc1305cf37 100644
--- a/drivers/gpu/drm/msm/msm_gem_vma.c
+++ b/drivers/gpu/drm/msm/msm_gem_vma.c
@@ -27,6 +27,15 @@ void msm_gem_address_space_put(struct msm_gem_address_space
*aspace)
kref_put(>kref, msm_gem_address_space_destroy);
}
+struct msm_gem_address_space *
+msm_gem_address_space_get(struct msm_gem_address_space *aspace)
+{
+ if (!IS_ERR_OR_NULL(aspace))
+ kref_get(>kref);
+
+ return aspace;
+}
+
/* Actually unmap memory for the vma */
void msm_gem_purge_vma(struct msm_gem_address_space *aspace,
struct msm_gem_vma *vma)
diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index e1a3cbe25a0c..951850804d77 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -823,6 +823,28 @@ static int get_clocks(struct platform_device *pdev, struct
msm_gpu *gpu)
return 0;
}
+/* Return a new address space for a msm_drm_private instance */
+struct msm_gem_address_space *
+msm_gpu_create_private_address_space(struct msm_gpu *gpu)
+{
+ struct msm_gem_address_space *aspace = NULL;
+
+ if (!gpu)
+ return NULL;
+
+ /*
+* If the target doesn't support private address spaces then return
+* the global one
+*/
+ if (gpu->funcs->create_private_address_space)
+ aspace = gpu->funcs->create_private_address_space(gpu);
+
+ if (IS_ERR_OR_NULL(aspace))
+ aspace = msm_gem_address_space_get(gpu->aspace);
+
+ return aspace;
+}
+
int msm_gpu_init(struct
[PATCH 07/20] drm/msm: set adreno_smmu as gpu's drvdata
From: Rob Clark
This will be populated by adreno-smmu, to provide a way for coordinating
enabling/disabling TTBR0 translation.
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/adreno/adreno_device.c | 2 --
drivers/gpu/drm/msm/msm_gpu.c | 2 +-
drivers/gpu/drm/msm/msm_gpu.h | 6 +-
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c
b/drivers/gpu/drm/msm/adreno/adreno_device.c
index 26664e1b30c0..58e03b20e1c7 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_device.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_device.c
@@ -417,8 +417,6 @@ static int adreno_bind(struct device *dev, struct device
*master, void *data)
return PTR_ERR(gpu);
}
- dev_set_drvdata(dev, gpu);
-
return 0;
}
diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index 6aa9e04e52e7..806eb0957280 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -892,7 +892,7 @@ int msm_gpu_init(struct drm_device *drm, struct
platform_device *pdev,
gpu->gpu_cx = NULL;
gpu->pdev = pdev;
- platform_set_drvdata(pdev, gpu);
+ platform_set_drvdata(pdev, >adreno_smmu);
msm_devfreq_init(gpu);
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index 8bda7beaed4b..f91b141add75 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -7,6 +7,7 @@
#ifndef __MSM_GPU_H__
#define __MSM_GPU_H__
+#include
#include
#include
#include
@@ -73,6 +74,8 @@ struct msm_gpu {
struct platform_device *pdev;
const struct msm_gpu_funcs *funcs;
+ struct adreno_smmu_priv adreno_smmu;
+
/* performance counters (hw & sw): */
spinlock_t perf_lock;
bool perfcntr_active;
@@ -143,7 +146,8 @@ struct msm_gpu {
static inline struct msm_gpu *dev_to_gpu(struct device *dev)
{
- return dev_get_drvdata(dev);
+ struct adreno_smmu_priv *adreno_smmu = dev_get_drvdata(dev);
+ return container_of(adreno_smmu, struct msm_gpu, adreno_smmu);
}
/* It turns out that all targets use the same ringbuffer size */
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 11/20] drm/msm: Add a context pointer to the submitqueue
From: Jordan Crouse
Each submitqueue is attached to a context. Add a pointer to the
context to the submitqueue at create time and refcount it so
that it stays around through the life of the queue.
Co-developed-by: Rob Clark
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/msm_drv.c | 3 ++-
drivers/gpu/drm/msm/msm_drv.h | 20
drivers/gpu/drm/msm/msm_gem.h | 1 +
drivers/gpu/drm/msm/msm_gem_submit.c | 6 +++---
drivers/gpu/drm/msm/msm_gpu.h | 1 +
drivers/gpu/drm/msm/msm_submitqueue.c | 3 +++
6 files changed, 30 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
index 7d641c7e3514..01845a3b8d52 100644
--- a/drivers/gpu/drm/msm/msm_drv.c
+++ b/drivers/gpu/drm/msm/msm_drv.c
@@ -594,6 +594,7 @@ static int context_init(struct drm_device *dev, struct
drm_file *file)
if (!ctx)
return -ENOMEM;
+ kref_init(>ref);
msm_submitqueue_init(dev, ctx);
ctx->aspace = priv->gpu ? priv->gpu->aspace : NULL;
@@ -615,7 +616,7 @@ static int msm_open(struct drm_device *dev, struct drm_file
*file)
static void context_close(struct msm_file_private *ctx)
{
msm_submitqueue_close(ctx);
- kfree(ctx);
+ msm_file_private_put(ctx);
}
static void msm_postclose(struct drm_device *dev, struct drm_file *file)
diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h
index af259b0573ea..4561bfb5e745 100644
--- a/drivers/gpu/drm/msm/msm_drv.h
+++ b/drivers/gpu/drm/msm/msm_drv.h
@@ -57,6 +57,7 @@ struct msm_file_private {
struct list_head submitqueues;
int queueid;
struct msm_gem_address_space *aspace;
+ struct kref ref;
};
enum msm_mdp_plane_property {
@@ -428,6 +429,25 @@ void msm_submitqueue_close(struct msm_file_private *ctx);
void msm_submitqueue_destroy(struct kref *kref);
+static inline void __msm_file_private_destroy(struct kref *kref)
+{
+ struct msm_file_private *ctx = container_of(kref,
+ struct msm_file_private, ref);
+
+ kfree(ctx);
+}
+
+static inline void msm_file_private_put(struct msm_file_private *ctx)
+{
+ kref_put(>ref, __msm_file_private_destroy);
+}
+
+static inline struct msm_file_private *msm_file_private_get(
+ struct msm_file_private *ctx)
+{
+ kref_get(>ref);
+ return ctx;
+}
#define DBG(fmt, ...) DRM_DEBUG_DRIVER(fmt"\n", ##__VA_ARGS__)
#define VERB(fmt, ...) if (0) DRM_DEBUG_DRIVER(fmt"\n", ##__VA_ARGS__)
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index 972490b14ba5..9c573c4269cb 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -142,6 +142,7 @@ struct msm_gem_submit {
bool valid; /* true if no cmdstream patching needed */
bool in_rb; /* "sudo" mode, copy cmds into RB */
struct msm_ringbuffer *ring;
+ struct msm_file_private *ctx;
unsigned int nr_cmds;
unsigned int nr_bos;
u32 ident; /* A "identifier" for the submit for logging */
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c
b/drivers/gpu/drm/msm/msm_gem_submit.c
index 8cb9aa15ff90..1464b04d25d3 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -27,7 +27,7 @@
#define BO_PINNED 0x2000
static struct msm_gem_submit *submit_create(struct drm_device *dev,
- struct msm_gpu *gpu, struct msm_gem_address_space *aspace,
+ struct msm_gpu *gpu,
struct msm_gpu_submitqueue *queue, uint32_t nr_bos,
uint32_t nr_cmds)
{
@@ -43,7 +43,7 @@ static struct msm_gem_submit *submit_create(struct drm_device
*dev,
return NULL;
submit->dev = dev;
- submit->aspace = aspace;
+ submit->aspace = queue->ctx->aspace;
submit->gpu = gpu;
submit->fence = NULL;
submit->cmd = (void *)>bos[nr_bos];
@@ -677,7 +677,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
}
}
- submit = submit_create(dev, gpu, ctx->aspace, queue, args->nr_bos,
+ submit = submit_create(dev, gpu, queue, args->nr_bos,
args->nr_cmds);
if (!submit) {
ret = -ENOMEM;
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index f91b141add75..97c527e98391 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -190,6 +190,7 @@ struct msm_gpu_submitqueue {
u32 flags;
u32 prio;
int faults;
+ struct msm_file_private *ctx;
struct list_head node;
struct kref ref;
};
diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c
b/drivers/gpu/drm/msm/msm_submitqueue.c
index 90c9d84e6155..c3d206105d28 100644
--- a/drivers/gpu/drm/msm/msm_submitqueue.c
+++ b/drivers/gpu/drm/msm/msm_submitqueue.c
@@ -12,6 +12,8 @@ void
[PATCH 08/20] iommu/arm-smmu: constify some helpers
From: Rob Clark
Sprinkle a few `const`s where helpers don't need write access.
Signed-off-by: Rob Clark
---
drivers/iommu/arm/arm-smmu/arm-smmu.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h
b/drivers/iommu/arm/arm-smmu/arm-smmu.h
index 59ff3fc5c6c8..27c8fc50 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.h
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h
@@ -377,7 +377,7 @@ struct arm_smmu_master_cfg {
s16 smendx[];
};
-static inline u32 arm_smmu_lpae_tcr(struct io_pgtable_cfg *cfg)
+static inline u32 arm_smmu_lpae_tcr(const struct io_pgtable_cfg *cfg)
{
u32 tcr = FIELD_PREP(ARM_SMMU_TCR_TG0, cfg->arm_lpae_s1_cfg.tcr.tg) |
FIELD_PREP(ARM_SMMU_TCR_SH0, cfg->arm_lpae_s1_cfg.tcr.sh) |
@@ -398,13 +398,13 @@ static inline u32 arm_smmu_lpae_tcr(struct io_pgtable_cfg
*cfg)
return tcr;
}
-static inline u32 arm_smmu_lpae_tcr2(struct io_pgtable_cfg *cfg)
+static inline u32 arm_smmu_lpae_tcr2(const struct io_pgtable_cfg *cfg)
{
return FIELD_PREP(ARM_SMMU_TCR2_PASIZE, cfg->arm_lpae_s1_cfg.tcr.ips) |
FIELD_PREP(ARM_SMMU_TCR2_SEP, ARM_SMMU_TCR2_SEP_UPSTREAM);
}
-static inline u32 arm_smmu_lpae_vtcr(struct io_pgtable_cfg *cfg)
+static inline u32 arm_smmu_lpae_vtcr(const struct io_pgtable_cfg *cfg)
{
return ARM_SMMU_VTCR_RES1 |
FIELD_PREP(ARM_SMMU_VTCR_PS, cfg->arm_lpae_s2_cfg.vtcr.ps) |
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 09/20] iommu/arm-smmu-qcom: Add implementation for the adreno GPU SMMU
From: Jordan Crouse
Add a special implementation for the SMMU attached to most Adreno GPU
target triggered from the qcom,adreno-smmu compatible string.
The new Adreno SMMU implementation will enable split pagetables
(TTBR1) for the domain attached to the GPU device (SID 0) and
hard code it context bank 0 so the GPU hardware can implement
per-instance pagetables.
Co-developed-by: Rob Clark
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/iommu/arm/arm-smmu/arm-smmu-impl.c | 3 +
drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 149 -
drivers/iommu/arm/arm-smmu/arm-smmu.h | 1 +
3 files changed, 151 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
b/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
index 88f17cc33023..d199b4bff15d 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
@@ -223,6 +223,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct
arm_smmu_device *smmu)
of_device_is_compatible(np, "qcom,sm8250-smmu-500"))
return qcom_smmu_impl_init(smmu);
+ if (of_device_is_compatible(smmu->dev->of_node, "qcom,adreno-smmu"))
+ return qcom_adreno_smmu_impl_init(smmu);
+
if (of_device_is_compatible(np, "marvell,ap806-smmu-500"))
smmu->impl = _mmu500_impl;
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
index be4318044f96..5640d9960610 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c
@@ -3,6 +3,7 @@
* Copyright (c) 2019, The Linux Foundation. All rights reserved.
*/
+#include
#include
#include
@@ -12,6 +13,132 @@ struct qcom_smmu {
struct arm_smmu_device smmu;
};
+#define QCOM_ADRENO_SMMU_GPU_SID 0
+
+static bool qcom_adreno_smmu_is_gpu_device(struct device *dev)
+{
+ struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+ int i;
+
+ /*
+* The GPU will always use SID 0 so that is a handy way to uniquely
+* identify it and configure it for per-instance pagetables
+*/
+ for (i = 0; i < fwspec->num_ids; i++) {
+ u16 sid = FIELD_GET(ARM_SMMU_SMR_ID, fwspec->ids[i]);
+
+ if (sid == QCOM_ADRENO_SMMU_GPU_SID)
+ return true;
+ }
+
+ return false;
+}
+
+static const struct io_pgtable_cfg *qcom_adreno_smmu_get_ttbr1_cfg(
+ const void *cookie)
+{
+ struct arm_smmu_domain *smmu_domain = (void *)cookie;
+ struct io_pgtable *pgtable =
+ io_pgtable_ops_to_pgtable(smmu_domain->pgtbl_ops);
+ return >cfg;
+}
+
+/*
+ * Local implementation to configure TTBR0 with the specified pagetable config.
+ * The GPU driver will call this to enable TTBR0 when per-instance pagetables
+ * are active
+ */
+
+static int qcom_adreno_smmu_set_ttbr0_cfg(const void *cookie,
+ const struct io_pgtable_cfg *pgtbl_cfg)
+{
+ struct arm_smmu_domain *smmu_domain = (void *)cookie;
+ struct io_pgtable *pgtable =
io_pgtable_ops_to_pgtable(smmu_domain->pgtbl_ops);
+ struct arm_smmu_cfg *cfg = _domain->cfg;
+ struct arm_smmu_cb *cb = _domain->smmu->cbs[cfg->cbndx];
+
+ /* The domain must have split pagetables already enabled */
+ if (cb->tcr[0] & ARM_SMMU_TCR_EPD1)
+ return -EINVAL;
+
+ /* If the pagetable config is NULL, disable TTBR0 */
+ if (!pgtbl_cfg) {
+ /* Do nothing if it is already disabled */
+ if ((cb->tcr[0] & ARM_SMMU_TCR_EPD0))
+ return -EINVAL;
+
+ /* Set TCR to the original configuration */
+ cb->tcr[0] = arm_smmu_lpae_tcr(>cfg);
+ cb->ttbr[0] = FIELD_PREP(ARM_SMMU_TTBRn_ASID, cb->cfg->asid);
+ } else {
+ u32 tcr = cb->tcr[0];
+
+ /* Don't call this again if TTBR0 is already enabled */
+ if (!(cb->tcr[0] & ARM_SMMU_TCR_EPD0))
+ return -EINVAL;
+
+ tcr |= arm_smmu_lpae_tcr(pgtbl_cfg);
+ tcr &= ~(ARM_SMMU_TCR_EPD0 | ARM_SMMU_TCR_EPD1);
+
+ cb->tcr[0] = tcr;
+ cb->ttbr[0] = pgtbl_cfg->arm_lpae_s1_cfg.ttbr;
+ cb->ttbr[0] |= FIELD_PREP(ARM_SMMU_TTBRn_ASID, cb->cfg->asid);
+ }
+
+ arm_smmu_write_context_bank(smmu_domain->smmu, cb->cfg->cbndx);
+
+ return 0;
+}
+
+static int qcom_adreno_smmu_alloc_context_bank(struct arm_smmu_domain
*smmu_domain,
+ struct device *dev, int start, int count)
+{
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ /*
+* Assign context bank 0 to the GPU device so the GPU hardware can
+* switch pagetables
+*/
+ if (qcom_adreno_smmu_is_gpu_device(dev)) {
+ start = 0;
+ count = 1;
+ } else {
+ start
[PATCH 12/20] drm/msm: Drop context arg to gpu->submit()
From: Jordan Crouse
Now that we can get the ctx from the submitqueue, the extra arg is
redundant.
Signed-off-by: Jordan Crouse
[split out of previous patch to reduce churny noise]
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 12 +---
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 5 ++---
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 5 ++---
drivers/gpu/drm/msm/adreno/adreno_gpu.h | 3 +--
drivers/gpu/drm/msm/msm_gem_submit.c| 2 +-
drivers/gpu/drm/msm/msm_gpu.c | 9 -
drivers/gpu/drm/msm/msm_gpu.h | 6 ++
7 files changed, 17 insertions(+), 25 deletions(-)
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
index 9e63a190642c..eff2439ea57b 100644
--- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
@@ -43,8 +43,7 @@ static void a5xx_flush(struct msm_gpu *gpu, struct
msm_ringbuffer *ring)
gpu_write(gpu, REG_A5XX_CP_RB_WPTR, wptr);
}
-static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct msm_gem_submit
*submit,
- struct msm_file_private *ctx)
+static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct msm_gem_submit
*submit)
{
struct msm_drm_private *priv = gpu->dev->dev_private;
struct msm_ringbuffer *ring = submit->ring;
@@ -57,7 +56,7 @@ static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct
msm_gem_submit *submit
case MSM_SUBMIT_CMD_IB_TARGET_BUF:
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
- if (priv->lastctx == ctx)
+ if (priv->lastctx == submit->queue->ctx)
break;
/* fall-thru */
case MSM_SUBMIT_CMD_BUF:
@@ -103,8 +102,7 @@ static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct
msm_gem_submit *submit
msm_gpu_retire(gpu);
}
-static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
- struct msm_file_private *ctx)
+static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu);
@@ -114,7 +112,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct
msm_gem_submit *submit,
if (IS_ENABLED(CONFIG_DRM_MSM_GPU_SUDO) && submit->in_rb) {
priv->lastctx = NULL;
- a5xx_submit_in_rb(gpu, submit, ctx);
+ a5xx_submit_in_rb(gpu, submit);
return;
}
@@ -148,7 +146,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct
msm_gem_submit *submit,
case MSM_SUBMIT_CMD_IB_TARGET_BUF:
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
- if (priv->lastctx == ctx)
+ if (priv->lastctx == submit->queue->ctx)
break;
/* fall-thru */
case MSM_SUBMIT_CMD_BUF:
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
index c5a3e4d4c007..5eabb0109577 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
@@ -81,8 +81,7 @@ static void get_stats_counter(struct msm_ringbuffer *ring,
u32 counter,
OUT_RING(ring, upper_32_bits(iova));
}
-static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
- struct msm_file_private *ctx)
+static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT;
struct msm_drm_private *priv = gpu->dev->dev_private;
@@ -115,7 +114,7 @@ static void a6xx_submit(struct msm_gpu *gpu, struct
msm_gem_submit *submit,
case MSM_SUBMIT_CMD_IB_TARGET_BUF:
break;
case MSM_SUBMIT_CMD_CTX_RESTORE_BUF:
- if (priv->lastctx == ctx)
+ if (priv->lastctx == submit->queue->ctx)
break;
/* fall-thru */
case MSM_SUBMIT_CMD_BUF:
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index d2dbb6968cba..533a34b4cce2 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -457,8 +457,7 @@ void adreno_recover(struct msm_gpu *gpu)
}
}
-void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
- struct msm_file_private *ctx)
+void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
struct msm_drm_private *priv = gpu->dev->dev_private;
@@ -472,7 +471,7 @@ void adreno_submit(struct msm_gpu *gpu, struct
msm_gem_submit *submit,
break;
case
[PATCH 10/20] dt-bindings: arm-smmu: Add compatible string for Adreno GPU SMMU
From: Jordan Crouse Every Qcom Adreno GPU has an embedded SMMU for its own use. These devices depend on unique features such as split pagetables, different stall/halt requirements and other settings. Identify them with a compatible string so that they can be identified in the arm-smmu implementation specific code. Signed-off-by: Jordan Crouse Reviewed-by: Rob Herring Signed-off-by: Rob Clark --- Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 4 1 file changed, 4 insertions(+) diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.yaml b/Documentation/devicetree/bindings/iommu/arm,smmu.yaml index 503160a7b9a0..5ec5d0d691f6 100644 --- a/Documentation/devicetree/bindings/iommu/arm,smmu.yaml +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.yaml @@ -40,6 +40,10 @@ properties: - qcom,sm8150-smmu-500 - qcom,sm8250-smmu-500 - const: arm,mmu-500 + - description: Qcom Adreno GPUs implementing "arm,smmu-v2" +items: + - const: qcom,adreno-smmu + - const: qcom,smmu-v2 - description: Marvell SoCs implementing "arm,mmu-500" items: - const: marvell,ap806-smmu-500 -- 2.26.2 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 13/20] drm/msm: Set the global virtual address range from the IOMMU domain
From: Jordan Crouse
Use the aperture settings from the IOMMU domain to set up the virtual
address range for the GPU. This allows us to transparently deal with
IOMMU side features (like split pagetables).
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 13 +++--
drivers/gpu/drm/msm/msm_iommu.c | 7 +++
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index 533a34b4cce2..34e6242c1767 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -192,9 +192,18 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu,
struct iommu_domain *iommu = iommu_domain_alloc(_bus_type);
struct msm_mmu *mmu = msm_iommu_new(>dev, iommu);
struct msm_gem_address_space *aspace;
+ u64 start, size;
- aspace = msm_gem_address_space_create(mmu, "gpu", SZ_16M,
- 0x - SZ_16M);
+ /*
+* Use the aperture start or SZ_16M, whichever is greater. This will
+* ensure that we align with the allocated pagetable range while still
+* allowing room in the lower 32 bits for GMEM and whatnot
+*/
+ start = max_t(u64, SZ_16M, iommu->geometry.aperture_start);
+ size = iommu->geometry.aperture_end - start + 1;
+
+ aspace = msm_gem_address_space_create(mmu, "gpu",
+ start & GENMASK(48, 0), size);
if (IS_ERR(aspace) && !IS_ERR(mmu))
mmu->funcs->destroy(mmu);
diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c
index 3a381a9674c9..1b6635504069 100644
--- a/drivers/gpu/drm/msm/msm_iommu.c
+++ b/drivers/gpu/drm/msm/msm_iommu.c
@@ -36,6 +36,10 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova,
struct msm_iommu *iommu = to_msm_iommu(mmu);
size_t ret;
+ /* The arm-smmu driver expects the addresses to be sign extended */
+ if (iova & BIT_ULL(48))
+ iova |= GENMASK_ULL(63, 49);
+
ret = iommu_map_sg(iommu->domain, iova, sgt->sgl, sgt->nents, prot);
WARN_ON(!ret);
@@ -46,6 +50,9 @@ static int msm_iommu_unmap(struct msm_mmu *mmu, uint64_t
iova, size_t len)
{
struct msm_iommu *iommu = to_msm_iommu(mmu);
+ if (iova & BIT_ULL(48))
+ iova |= GENMASK_ULL(63, 49);
+
iommu_unmap(iommu->domain, iova, len);
return 0;
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 05/20] iommu: add private interface for adreno-smmu
From: Rob Clark
This interface will be used for drm/msm to coordinate with the
qcom_adreno_smmu_impl to enable/disable TTBR0 translation.
Once TTBR0 translation is enabled, the GPU's CP (Command Processor)
will directly switch TTBR0 pgtables (and do the necessary TLB inv)
synchronized to the GPU's operation. But help from the SMMU driver
is needed to initially bootstrap TTBR0 translation, which cannot be
done from the GPU.
Since this is a very special case, a private interface is used to
avoid adding highly driver specific things to the public iommu
interface.
Signed-off-by: Rob Clark
---
include/linux/adreno-smmu-priv.h | 36
1 file changed, 36 insertions(+)
create mode 100644 include/linux/adreno-smmu-priv.h
diff --git a/include/linux/adreno-smmu-priv.h b/include/linux/adreno-smmu-priv.h
new file mode 100644
index ..a889f28afb42
--- /dev/null
+++ b/include/linux/adreno-smmu-priv.h
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2020 Google, Inc
+ */
+
+#ifndef __ADRENO_SMMU_PRIV_H
+#define __ADRENO_SMMU_PRIV_H
+
+#include
+
+/**
+ * struct adreno_smmu_priv - private interface between adreno-smmu and GPU
+ *
+ * @cookie:An opque token provided by adreno-smmu and passed
+ * back into the callbacks
+ * @get_ttbr1_cfg: Get the TTBR1 config for the GPUs context-bank
+ * @set_ttbr0_cfg: Set the TTBR0 config for the GPUs context bank. A
+ * NULL config disables TTBR0 translation, otherwise
+ * TTBR0 translation is enabled with the specified cfg
+ *
+ * The GPU driver (drm/msm) and adreno-smmu work together for controlling
+ * the GPU's SMMU instance. This is by necessity, as the GPU is directly
+ * updating the SMMU for context switches, while on the other hand we do
+ * not want to duplicate all of the initial setup logic from arm-smmu.
+ *
+ * This private interface is used for the two drivers to coordinate. The
+ * cookie and callback functions are populated when the GPU driver attaches
+ * it's domain.
+ */
+struct adreno_smmu_priv {
+const void *cookie;
+const struct io_pgtable_cfg *(*get_ttbr1_cfg)(const void *cookie);
+int (*set_ttbr0_cfg)(const void *cookie, const struct io_pgtable_cfg *cfg);
+};
+
+#endif /* __ADRENO_SMMU_PRIV_H */
\ No newline at end of file
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 06/20] drm/msm/gpu: add dev_to_gpu() helper
From: Rob Clark
In a later patch, the drvdata will not directly be 'struct msm_gpu *',
so add a helper to reduce the churn.
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/adreno/adreno_device.c | 10 --
drivers/gpu/drm/msm/msm_gpu.c | 6 +++---
drivers/gpu/drm/msm/msm_gpu.h | 5 +
3 files changed, 12 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c
b/drivers/gpu/drm/msm/adreno/adreno_device.c
index 9eeb46bf2a5d..26664e1b30c0 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_device.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_device.c
@@ -282,7 +282,7 @@ struct msm_gpu *adreno_load_gpu(struct drm_device *dev)
int ret;
if (pdev)
- gpu = platform_get_drvdata(pdev);
+ gpu = dev_to_gpu(>dev);
if (!gpu) {
dev_err_once(dev->dev, "no GPU device was found\n");
@@ -425,7 +425,7 @@ static int adreno_bind(struct device *dev, struct device
*master, void *data)
static void adreno_unbind(struct device *dev, struct device *master,
void *data)
{
- struct msm_gpu *gpu = dev_get_drvdata(dev);
+ struct msm_gpu *gpu = dev_to_gpu(dev);
pm_runtime_force_suspend(dev);
gpu->funcs->destroy(gpu);
@@ -490,16 +490,14 @@ static const struct of_device_id dt_match[] = {
#ifdef CONFIG_PM
static int adreno_resume(struct device *dev)
{
- struct platform_device *pdev = to_platform_device(dev);
- struct msm_gpu *gpu = platform_get_drvdata(pdev);
+ struct msm_gpu *gpu = dev_to_gpu(dev);
return gpu->funcs->pm_resume(gpu);
}
static int adreno_suspend(struct device *dev)
{
- struct platform_device *pdev = to_platform_device(dev);
- struct msm_gpu *gpu = platform_get_drvdata(pdev);
+ struct msm_gpu *gpu = dev_to_gpu(dev);
return gpu->funcs->pm_suspend(gpu);
}
diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index d5645472b25d..6aa9e04e52e7 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -24,7 +24,7 @@
static int msm_devfreq_target(struct device *dev, unsigned long *freq,
u32 flags)
{
- struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
+ struct msm_gpu *gpu = dev_to_gpu(dev);
struct dev_pm_opp *opp;
opp = devfreq_recommended_opp(dev, freq, flags);
@@ -45,7 +45,7 @@ static int msm_devfreq_target(struct device *dev, unsigned
long *freq,
static int msm_devfreq_get_dev_status(struct device *dev,
struct devfreq_dev_status *status)
{
- struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
+ struct msm_gpu *gpu = dev_to_gpu(dev);
ktime_t time;
if (gpu->funcs->gpu_get_freq)
@@ -64,7 +64,7 @@ static int msm_devfreq_get_dev_status(struct device *dev,
static int msm_devfreq_get_cur_freq(struct device *dev, unsigned long *freq)
{
- struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
+ struct msm_gpu *gpu = dev_to_gpu(dev);
if (gpu->funcs->gpu_get_freq)
*freq = gpu->funcs->gpu_get_freq(gpu);
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index 0db117a7339b..8bda7beaed4b 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -141,6 +141,11 @@ struct msm_gpu {
struct msm_gpu_state *crashstate;
};
+static inline struct msm_gpu *dev_to_gpu(struct device *dev)
+{
+ return dev_get_drvdata(dev);
+}
+
/* It turns out that all targets use the same ringbuffer size */
#define MSM_GPU_RINGBUFFER_SZ SZ_32K
#define MSM_GPU_RINGBUFFER_BLKSIZE 32
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 01/20] drm/msm: remove dangling submitqueue references
From: Rob Clark
Currently it doesn't matter, since we free the ctx immediately. But
when we start refcnt'ing the ctx, we don't want old dangling list
entries to hang around.
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/msm_submitqueue.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c
b/drivers/gpu/drm/msm/msm_submitqueue.c
index a1d94be7883a..90c9d84e6155 100644
--- a/drivers/gpu/drm/msm/msm_submitqueue.c
+++ b/drivers/gpu/drm/msm/msm_submitqueue.c
@@ -49,8 +49,10 @@ void msm_submitqueue_close(struct msm_file_private *ctx)
* No lock needed in close and there won't
* be any more user ioctls coming our way
*/
- list_for_each_entry_safe(entry, tmp, >submitqueues, node)
+ list_for_each_entry_safe(entry, tmp, >submitqueues, node) {
+ list_del(>node);
msm_submitqueue_put(entry);
+ }
}
int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private
*ctx,
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 04/20] iommu/arm-smmu: Prepare for the adreno-smmu implementation
From: Jordan Crouse
Do a bit of prep work to add the upcoming adreno-smmu implementation.
Add an hook to allow the implementation to choose which context banks
to allocate.
Move some of the common structs to arm-smmu.h in anticipation of them
being used by the implementations and update some of the existing hooks
to pass more information that the implementation will need.
These modifications will be used by the upcoming Adreno SMMU
implementation to identify the GPU device and properly configure it
for pagetable switching.
Co-developed-by: Rob Clark
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/iommu/arm/arm-smmu/arm-smmu-impl.c | 2 +-
drivers/iommu/arm/arm-smmu/arm-smmu.c | 69 ++
drivers/iommu/arm/arm-smmu/arm-smmu.h | 51 +++-
3 files changed, 68 insertions(+), 54 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
b/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
index a9861dcd0884..88f17cc33023 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
@@ -69,7 +69,7 @@ static int cavium_cfg_probe(struct arm_smmu_device *smmu)
}
static int cavium_init_context(struct arm_smmu_domain *smmu_domain,
- struct io_pgtable_cfg *pgtbl_cfg)
+ struct io_pgtable_cfg *pgtbl_cfg, struct device *dev)
{
struct cavium_smmu *cs = container_of(smmu_domain->smmu,
struct cavium_smmu, smmu);
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c
b/drivers/iommu/arm/arm-smmu/arm-smmu.c
index 976d43a7f2ff..e63a480d7f71 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
@@ -65,41 +65,10 @@ module_param(disable_bypass, bool, S_IRUGO);
MODULE_PARM_DESC(disable_bypass,
"Disable bypass streams such that incoming transactions from devices
that are not attached to an iommu domain will report an abort back to the
device and will not be allowed to pass through the SMMU.");
-struct arm_smmu_s2cr {
- struct iommu_group *group;
- int count;
- enum arm_smmu_s2cr_type type;
- enum arm_smmu_s2cr_privcfg privcfg;
- u8 cbndx;
-};
-
#define s2cr_init_val (struct arm_smmu_s2cr){ \
.type = disable_bypass ? S2CR_TYPE_FAULT : S2CR_TYPE_BYPASS,\
}
-struct arm_smmu_smr {
- u16 mask;
- u16 id;
- boolvalid;
-};
-
-struct arm_smmu_cb {
- u64 ttbr[2];
- u32 tcr[2];
- u32 mair[2];
- struct arm_smmu_cfg *cfg;
-};
-
-struct arm_smmu_master_cfg {
- struct arm_smmu_device *smmu;
- s16 smendx[];
-};
-#define INVALID_SMENDX -1
-#define cfg_smendx(cfg, fw, i) \
- (i >= fw->num_ids ? INVALID_SMENDX : cfg->smendx[i])
-#define for_each_cfg_sme(cfg, fw, i, idx) \
- for (i = 0; idx = cfg_smendx(cfg, fw, i), i < fw->num_ids; ++i)
-
static bool using_legacy_binding, using_generic_binding;
static inline int arm_smmu_rpm_get(struct arm_smmu_device *smmu)
@@ -234,19 +203,6 @@ static int arm_smmu_register_legacy_master(struct device
*dev,
}
#endif /* CONFIG_ARM_SMMU_LEGACY_DT_BINDINGS */
-static int __arm_smmu_alloc_bitmap(unsigned long *map, int start, int end)
-{
- int idx;
-
- do {
- idx = find_next_zero_bit(map, end, start);
- if (idx == end)
- return -ENOSPC;
- } while (test_and_set_bit(idx, map));
-
- return idx;
-}
-
static void __arm_smmu_free_bitmap(unsigned long *map, int idx)
{
clear_bit(idx, map);
@@ -578,7 +534,7 @@ static void arm_smmu_init_context_bank(struct
arm_smmu_domain *smmu_domain,
}
}
-static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx)
+void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx)
{
u32 reg;
bool stage1;
@@ -665,7 +621,8 @@ static void arm_smmu_write_context_bank(struct
arm_smmu_device *smmu, int idx)
}
static int arm_smmu_init_domain_context(struct iommu_domain *domain,
- struct arm_smmu_device *smmu)
+ struct arm_smmu_device *smmu,
+ struct device *dev)
{
int irq, start, ret = 0;
unsigned long ias, oas;
@@ -780,10 +737,20 @@ static int arm_smmu_init_domain_context(struct
iommu_domain *domain,
ret = -EINVAL;
goto out_unlock;
}
- ret = __arm_smmu_alloc_bitmap(smmu->context_map, start,
+
+ smmu_domain->smmu = smmu;
+
+ if (smmu->impl && smmu->impl->alloc_context_bank)
+
[PATCH 02/20] iommu/arm-smmu: Pass io-pgtable config to implementation specific function
From: Jordan Crouse
Construct the io-pgtable config before calling the implementation specific
init_context function and pass it so the implementation specific function
can get a chance to change it before the io-pgtable is created.
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/iommu/arm/arm-smmu/arm-smmu-impl.c | 3 ++-
drivers/iommu/arm/arm-smmu/arm-smmu.c | 11 ++-
drivers/iommu/arm/arm-smmu/arm-smmu.h | 3 ++-
3 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
b/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
index f4ff124a1967..a9861dcd0884 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu-impl.c
@@ -68,7 +68,8 @@ static int cavium_cfg_probe(struct arm_smmu_device *smmu)
return 0;
}
-static int cavium_init_context(struct arm_smmu_domain *smmu_domain)
+static int cavium_init_context(struct arm_smmu_domain *smmu_domain,
+ struct io_pgtable_cfg *pgtbl_cfg)
{
struct cavium_smmu *cs = container_of(smmu_domain->smmu,
struct cavium_smmu, smmu);
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c
b/drivers/iommu/arm/arm-smmu/arm-smmu.c
index 09c42af9f31e..37d8d49299b4 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
@@ -795,11 +795,6 @@ static int arm_smmu_init_domain_context(struct
iommu_domain *domain,
cfg->asid = cfg->cbndx;
smmu_domain->smmu = smmu;
- if (smmu->impl && smmu->impl->init_context) {
- ret = smmu->impl->init_context(smmu_domain);
- if (ret)
- goto out_unlock;
- }
pgtbl_cfg = (struct io_pgtable_cfg) {
.pgsize_bitmap = smmu->pgsize_bitmap,
@@ -810,6 +805,12 @@ static int arm_smmu_init_domain_context(struct
iommu_domain *domain,
.iommu_dev = smmu->dev,
};
+ if (smmu->impl && smmu->impl->init_context) {
+ ret = smmu->impl->init_context(smmu_domain, _cfg);
+ if (ret)
+ goto out_clear_smmu;
+ }
+
if (smmu_domain->non_strict)
pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT;
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h
b/drivers/iommu/arm/arm-smmu/arm-smmu.h
index d890a4a968e8..83294516ac08 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.h
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h
@@ -386,7 +386,8 @@ struct arm_smmu_impl {
u64 val);
int (*cfg_probe)(struct arm_smmu_device *smmu);
int (*reset)(struct arm_smmu_device *smmu);
- int (*init_context)(struct arm_smmu_domain *smmu_domain);
+ int (*init_context)(struct arm_smmu_domain *smmu_domain,
+ struct io_pgtable_cfg *cfg);
void (*tlb_sync)(struct arm_smmu_device *smmu, int page, int sync,
int status);
int (*def_domain_type)(struct device *dev);
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 03/20] iommu/arm-smmu: Add support for split pagetables
From: Jordan Crouse
Enable TTBR1 for a context bank if IO_PGTABLE_QUIRK_ARM_TTBR1 is selected
by the io-pgtable configuration.
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/iommu/arm/arm-smmu/arm-smmu.c | 21 -
drivers/iommu/arm/arm-smmu/arm-smmu.h | 25 +++--
2 files changed, 35 insertions(+), 11 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c
b/drivers/iommu/arm/arm-smmu/arm-smmu.c
index 37d8d49299b4..976d43a7f2ff 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
@@ -552,11 +552,15 @@ static void arm_smmu_init_context_bank(struct
arm_smmu_domain *smmu_domain,
cb->ttbr[0] = pgtbl_cfg->arm_v7s_cfg.ttbr;
cb->ttbr[1] = 0;
} else {
- cb->ttbr[0] = pgtbl_cfg->arm_lpae_s1_cfg.ttbr;
- cb->ttbr[0] |= FIELD_PREP(ARM_SMMU_TTBRn_ASID,
- cfg->asid);
+ cb->ttbr[0] = FIELD_PREP(ARM_SMMU_TTBRn_ASID,
+ cfg->asid);
cb->ttbr[1] = FIELD_PREP(ARM_SMMU_TTBRn_ASID,
-cfg->asid);
+ cfg->asid);
+
+ if (pgtbl_cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ cb->ttbr[1] |= pgtbl_cfg->arm_lpae_s1_cfg.ttbr;
+ else
+ cb->ttbr[0] |= pgtbl_cfg->arm_lpae_s1_cfg.ttbr;
}
} else {
cb->ttbr[0] = pgtbl_cfg->arm_lpae_s2_cfg.vttbr;
@@ -822,7 +826,14 @@ static int arm_smmu_init_domain_context(struct
iommu_domain *domain,
/* Update the domain's page sizes to reflect the page table format */
domain->pgsize_bitmap = pgtbl_cfg.pgsize_bitmap;
- domain->geometry.aperture_end = (1UL << ias) - 1;
+
+ if (pgtbl_cfg.quirks & IO_PGTABLE_QUIRK_ARM_TTBR1) {
+ domain->geometry.aperture_start = ~0UL << ias;
+ domain->geometry.aperture_end = ~0UL;
+ } else {
+ domain->geometry.aperture_end = (1UL << ias) - 1;
+ }
+
domain->geometry.force_aperture = true;
/* Initialise the context bank with our page table cfg */
diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h
b/drivers/iommu/arm/arm-smmu/arm-smmu.h
index 83294516ac08..f3e456893f28 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.h
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h
@@ -169,10 +169,12 @@ enum arm_smmu_cbar_type {
#define ARM_SMMU_CB_TCR0x30
#define ARM_SMMU_TCR_EAE BIT(31)
#define ARM_SMMU_TCR_EPD1 BIT(23)
+#define ARM_SMMU_TCR_A1BIT(22)
#define ARM_SMMU_TCR_TG0 GENMASK(15, 14)
#define ARM_SMMU_TCR_SH0 GENMASK(13, 12)
#define ARM_SMMU_TCR_ORGN0 GENMASK(11, 10)
#define ARM_SMMU_TCR_IRGN0 GENMASK(9, 8)
+#define ARM_SMMU_TCR_EPD0 BIT(7)
#define ARM_SMMU_TCR_T0SZ GENMASK(5, 0)
#define ARM_SMMU_VTCR_RES1 BIT(31)
@@ -350,12 +352,23 @@ struct arm_smmu_domain {
static inline u32 arm_smmu_lpae_tcr(struct io_pgtable_cfg *cfg)
{
- return ARM_SMMU_TCR_EPD1 |
- FIELD_PREP(ARM_SMMU_TCR_TG0, cfg->arm_lpae_s1_cfg.tcr.tg) |
- FIELD_PREP(ARM_SMMU_TCR_SH0, cfg->arm_lpae_s1_cfg.tcr.sh) |
- FIELD_PREP(ARM_SMMU_TCR_ORGN0, cfg->arm_lpae_s1_cfg.tcr.orgn) |
- FIELD_PREP(ARM_SMMU_TCR_IRGN0, cfg->arm_lpae_s1_cfg.tcr.irgn) |
- FIELD_PREP(ARM_SMMU_TCR_T0SZ, cfg->arm_lpae_s1_cfg.tcr.tsz);
+ u32 tcr = FIELD_PREP(ARM_SMMU_TCR_TG0, cfg->arm_lpae_s1_cfg.tcr.tg) |
+ FIELD_PREP(ARM_SMMU_TCR_SH0, cfg->arm_lpae_s1_cfg.tcr.sh) |
+ FIELD_PREP(ARM_SMMU_TCR_ORGN0, cfg->arm_lpae_s1_cfg.tcr.orgn) |
+ FIELD_PREP(ARM_SMMU_TCR_IRGN0, cfg->arm_lpae_s1_cfg.tcr.irgn) |
+ FIELD_PREP(ARM_SMMU_TCR_T0SZ, cfg->arm_lpae_s1_cfg.tcr.tsz);
+
+ /*
+ * When TTBR1 is selected shift the TCR fields by 16 bits and disable
+ * translation in TTBR0
+ */
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1) {
+ tcr = (tcr << 16) & ~ARM_SMMU_TCR_A1;
+ tcr |= ARM_SMMU_TCR_EPD0;
+ } else
+ tcr |= ARM_SMMU_TCR_EPD1;
+
+ return tcr;
}
static inline u32 arm_smmu_lpae_tcr2(struct io_pgtable_cfg *cfg)
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 00/20] iommu/arm-smmu + drm/msm: per-process GPU pgtables
From: Rob Clark This series adds an Adreno SMMU implementation to arm-smmu to allow GPU hardware pagetable switching. The Adreno GPU has built in capabilities to switch the TTBR0 pagetable during runtime to allow each individual instance or application to have its own pagetable. In order to take advantage of the HW capabilities there are certain requirements needed of the SMMU hardware. This series adds support for an Adreno specific arm-smmu implementation. The new implementation 1) ensures that the GPU domain is always assigned context bank 0, 2) enables split pagetable support (TTBR1) so that the instance specific pagetable can be swapped while the global memory remains in place and 3) shares the current pagetable configuration with the GPU driver to allow it to create its own io-pgtable instances. The series then adds the drm/msm code to enable these features. For targets that support it allocate new pagetables using the io-pgtable configuration shared by the arm-smmu driver and swap them in during runtime. This version of the series merges the previous patchset(s) [1] and [2] with the following improvements: v14: (Respin by Rob) - Minor update to 16/20 (only force ASID to zero in one place) - Addition of sc7180 dtsi patch. v13: (Respin by Rob) - Switch to a private interface between adreno-smmu and GPU driver, dropping the custom domain attr (Will Deacon) - Rework the SCTLR.HUPCF patch to add new fields in smmu_domain->cfg rather than adding new impl hook (Will Deacon) - Drop for_each_cfg_sme() in favor of plain for() loop (Will Deacon) - Fix context refcnt'ing issue which was causing problems with GPU crash recover stress testing. - Spiff up $debugfs/gem to show process information associated with VMAs v12: - Nitpick cleanups in gpu/drm/msm/msm_iommu.c (Rob Clark) - Reorg in gpu/drm/msm/msm_gpu.c (Rob Clark) - Use the default asid for the context bank so that iommu_tlb_flush_all works - Flush the UCHE after a page switch - Add the SCTLR.HUPCF patch at the end of the series v11: - Add implementation specific get_attr/set_attr functions (per Rob Clark) - Fix context bank allocation (per Bjorn Andersson) v10: - arm-smmu: add implementation hook to allocate context banks - arm-smmu: Match the GPU domain by stream ID instead of compatible string - arm-smmu: Make DOMAIN_ATTR_PGTABLE_CFG bi-directional. The leaf driver queries the configuration to create a pagetable and then sends the newly created configuration back to the smmu-driver to enable TTBR0 - drm/msm: Add context reference counting for submissions - drm/msm: Use dummy functions to skip TLB operations on per-instance pagetables [1] https://lists.linuxfoundation.org/pipermail/iommu/2020-June/045653.html [2] https://lists.linuxfoundation.org/pipermail/iommu/2020-June/045659.html Jordan Crouse (12): iommu/arm-smmu: Pass io-pgtable config to implementation specific function iommu/arm-smmu: Add support for split pagetables iommu/arm-smmu: Prepare for the adreno-smmu implementation iommu/arm-smmu-qcom: Add implementation for the adreno GPU SMMU dt-bindings: arm-smmu: Add compatible string for Adreno GPU SMMU drm/msm: Add a context pointer to the submitqueue drm/msm: Drop context arg to gpu->submit() drm/msm: Set the global virtual address range from the IOMMU domain drm/msm: Add support to create a local pagetable drm/msm: Add support for private address space instances drm/msm/a6xx: Add support for per-instance pagetables arm: dts: qcom: sm845: Set the compatible string for the GPU SMMU Rob Clark (8): drm/msm: remove dangling submitqueue references iommu: add private interface for adreno-smmu drm/msm/gpu: add dev_to_gpu() helper drm/msm: set adreno_smmu as gpu's drvdata iommu/arm-smmu: constify some helpers arm: dts: qcom: sc7180: Set the compatible string for the GPU SMMU iommu/arm-smmu: add a way for implementations to influence SCTLR drm/msm: show process names in gem_describe .../devicetree/bindings/iommu/arm,smmu.yaml | 4 + arch/arm64/boot/dts/qcom/sc7180.dtsi | 2 +- arch/arm64/boot/dts/qcom/sdm845.dtsi | 2 +- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 12 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 68 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 + drivers/gpu/drm/msm/adreno/adreno_device.c| 12 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 18 +- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 3 +- drivers/gpu/drm/msm/msm_drv.c | 16 +- drivers/gpu/drm/msm/msm_drv.h | 25 +++ drivers/gpu/drm/msm/msm_gem.c | 25 ++- drivers/gpu/drm/msm/msm_gem.h | 6 + drivers/gpu/drm/msm/msm_gem_submit.c | 8 +- drivers/gpu/drm/msm/msm_gem_vma.c | 10 + drivers/gpu/drm/msm/msm_gpu.c | 41 +++- drivers/gpu/drm/msm/msm_gpu.h | 21 +-
Re: [PATCH v7 1/7] docs: IOMMU user API
On Wed, 12 Aug 2020 18:38:50 +0200 Auger Eric wrote: > Hi Jacob, > > On 7/30/20 2:21 AM, Jacob Pan wrote: > > IOMMU UAPI is newly introduced to support communications between > > guest virtual IOMMU and host IOMMU. There has been lots of > > discussions on how it should work with VFIO UAPI and userspace in > > general. > > > > This document is intended to clarify the UAPI design and usage. The > > mechanics of how future extensions should be achieved are also > > covered in this documentation. > > > > Signed-off-by: Liu Yi L > > Signed-off-by: Jacob Pan > > --- > > Documentation/userspace-api/iommu.rst | 212 > > ++ > > MAINTAINERS | 1 + 2 files changed, 213 > > insertions(+) create mode 100644 > > Documentation/userspace-api/iommu.rst > > > > diff --git a/Documentation/userspace-api/iommu.rst > > b/Documentation/userspace-api/iommu.rst new file mode 100644 > > index ..b2f5b3256d85 > > --- /dev/null > > +++ b/Documentation/userspace-api/iommu.rst > > @@ -0,0 +1,212 @@ > > +.. SPDX-License-Identifier: GPL-2.0 > > +.. iommu: > > + > > += > > +IOMMU Userspace API > > += > > + > > +IOMMU UAPI is used for virtualization cases where communications > > are +needed between physical and virtual IOMMU drivers. For > > baremetal +usage, the IOMMU is a system device which does not need > > to communicate +with user space directly. > > + > > +The primary use cases are guest Shared Virtual Address (SVA) and > > +guest IO virtual address (IOVA), wherin the vIOMMU implementation > > +relies on the physical IOMMU and for this reason requires > > interactions +with the host driver. > > + > > +.. contents:: :local: > > + > > +Functionalities > > +=== > > +Communications of user and kernel involve both directions. The > > +supported user-kernel APIs are as follows: > > + > > +1. Alloc/Free PASID > > +2. Bind/unbind guest PASID (e.g. Intel VT-d) > nit: s/unbind/Unbind to match above Free Sounds good, same below. > > +3. Bind/unbind guest PASID table (e.g. ARM SMMU) > > +4. Invalidate IOMMU caches requested by guests > s/requested by guests/upon guest requests Will do. > > +5. Report errors to the guest and serve page requests > > + > > +Requirements > > + > > +The IOMMU UAPIs are generic and extensible to meet the following > > +requirements: > > + > > +1. Emulated and para-virtualised vIOMMUs > > +2. Multiple vendors (Intel VT-d, ARM SMMU, etc.) > > +3. Extensions to the UAPI shall not break existing user space > > + > > +Interfaces > > +== > > +Although the data structures defined in IOMMU UAPI are > > self-contained, +there is no user API functions introduced. > > Instead, IOMMU UAPI is +designed to work with existing user driver > > frameworks such as VFIO. + > > +Extension Rules & Precautions > > +- > > +When IOMMU UAPI gets extended, the data structures can *only* be > > +modified in two ways: > > + > > +1. Adding new fields by re-purposing the padding[] field. No size > > change. +2. Adding new union members at the end. May increase the > > structure sizes. + > > +No new fields can be added *after* the variable sized union in > > that it +will break backward compatibility when offset moves. A new > > flag must +be introduced whenever a change affects the structure > > using either +method. The IOMMU driver processes the data based on > > flags which +ensures backward compatibility. > > + > > +Version field is only reserved for the unlikely event of UAPI > > upgrade +at its entirety. > > + > > +It's *always* the caller's responsibility to indicate the size of > > the +structure passed by setting argsz appropriately. > > +Though at the same time, argsz is user provided data which is not > > +trusted. The argsz field allows the user app to indicate how much > > data +it is providing, it's still the kernel's responsibility to > > validate +whether it's correct and sufficient for the requested > > operation. + > > +Compatibility Checking > > +-- > > +When IOMMU UAPI extension results in some structure size increase, > > +IOMMU UAPI code shall handle the following cases: > > + > > +1. User and kernel has exact size match > > +2. An older user with older kernel header (smaller UAPI size) > > running on a > > + newer kernel (larger UAPI size) > > +3. A newer user with newer kernel header (larger UAPI size) running > > + on an older kernel. > > +4. A malicious/misbehaving user pass illegal/invalid size but > > within > > + range. The data may contain garbage. > > + > > +Feature Checking > > + > > +While launching a guest with vIOMMU, it is important to ensure > > that host +can support the UAPI data structures to be used for > > vIOMMU-pIOMMU +communications. Without upfront compatibility > > checking, the future errors +can lead to catastrophic failures for > > the users.
[PATCH RESEND v10 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
The new field 'dma_range_map' in struct device is used to facilitate the
use of single or multiple offsets between mapping regions of cpu addrs and
dma addrs. It subsumes the role of "dev->dma_pfn_offset" which was only
capable of holding a single uniform offset and had no region bounds
checking.
The function of_dma_get_range() has been modified so that it takes a single
argument -- the device node -- and returns a map, NULL, or an error code.
The map is an array that holds the information regarding the DMA regions.
Each range entry contains the address offset, the cpu_start address, the
dma_start address, and the size of the region.
of_dma_configure() is the typical manner to set range offsets but there are
a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
driver code. These cases now invoke the function
dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
Signed-off-by: Jim Quinlan
---
arch/arm/include/asm/dma-mapping.h| 10 +--
arch/arm/mach-keystone/keystone.c | 17 +++--
arch/sh/drivers/pci/pcie-sh7786.c | 9 +--
arch/sh/kernel/dma-coherent.c | 15 ++--
arch/x86/pci/sta2x11-fixup.c | 7 +-
drivers/acpi/arm64/iort.c | 5 +-
drivers/base/core.c | 2 +
drivers/gpu/drm/sun4i/sun4i_backend.c | 5 +-
drivers/iommu/io-pgtable-arm.c| 2 +-
.../platform/sunxi/sun4i-csi/sun4i_csi.c | 5 +-
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 4 +-
drivers/of/address.c | 72 +--
drivers/of/device.c | 43 ++-
drivers/of/of_private.h | 10 +--
drivers/of/unittest.c | 31 +---
drivers/remoteproc/remoteproc_core.c | 8 ++-
.../staging/media/sunxi/cedrus/cedrus_hw.c| 7 +-
drivers/usb/core/message.c| 9 ++-
drivers/usb/core/usb.c| 7 +-
include/linux/device.h| 4 +-
include/linux/dma-direct.h| 8 +--
include/linux/dma-mapping.h | 36 ++
kernel/dma/coherent.c | 10 +--
kernel/dma/mapping.c | 65 +
24 files changed, 269 insertions(+), 122 deletions(-)
diff --git a/arch/arm/include/asm/dma-mapping.h
b/arch/arm/include/asm/dma-mapping.h
index bdd80ddbca34..2405afeb7957 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,8 +35,11 @@ static inline const struct dma_map_ops
*get_arch_dma_ops(struct bus_type *bus)
#ifndef __arch_pfn_to_dma
static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
{
- if (dev)
- pfn -= dev->dma_pfn_offset;
+ if (dev) {
+ phys_addr_t paddr = PFN_PHYS(pfn);
+
+ pfn -= (dma_offset_from_phys_addr(dev, paddr) >> PAGE_SHIFT);
+ }
return (dma_addr_t)__pfn_to_bus(pfn);
}
@@ -45,8 +48,7 @@ static inline unsigned long dma_to_pfn(struct device *dev,
dma_addr_t addr)
unsigned long pfn = __bus_to_pfn(addr);
if (dev)
- pfn += dev->dma_pfn_offset;
-
+ pfn += (dma_offset_from_dma_addr(dev, addr) >> PAGE_SHIFT);
return pfn;
}
diff --git a/arch/arm/mach-keystone/keystone.c
b/arch/arm/mach-keystone/keystone.c
index 638808c4e122..78808942ad1c 100644
--- a/arch/arm/mach-keystone/keystone.c
+++ b/arch/arm/mach-keystone/keystone.c
@@ -8,6 +8,7 @@
*/
#include
#include
+#include
#include
#include
#include
@@ -24,8 +25,6 @@
#include "keystone.h"
-static unsigned long keystone_dma_pfn_offset __read_mostly;
-
static int keystone_platform_notifier(struct notifier_block *nb,
unsigned long event, void *data)
{
@@ -38,9 +37,12 @@ static int keystone_platform_notifier(struct notifier_block
*nb,
return NOTIFY_BAD;
if (!dev->of_node) {
- dev->dma_pfn_offset = keystone_dma_pfn_offset;
- dev_err(dev, "set dma_pfn_offset%08lx\n",
- dev->dma_pfn_offset);
+ int ret = dma_set_offset_range(dev, KEYSTONE_HIGH_PHYS_START,
+ KEYSTONE_LOW_PHYS_START,
+ KEYSTONE_HIGH_PHYS_SIZE);
+ dev_err(dev, "set dma_offset%08llx%s\n",
+ KEYSTONE_HIGH_PHYS_START - KEYSTONE_LOW_PHYS_START,
+ ret ? " failed" : "");
}
return NOTIFY_OK;
}
@@ -51,11 +53,8 @@ static struct notifier_block platform_nb = {
static void __init keystone_init(void)
{
- if (PHYS_OFFSET >= KEYSTONE_HIGH_PHYS_START) {
- keystone_dma_pfn_offset = PFN_DOWN(KEYSTONE_HIGH_PHYS_START -
- KEYSTONE_LOW_PHYS_START);
+ if
[PATCH RESEND v10 00/11] PCI: brcmstb: enable PCIe for STB chips
Patchset Summary:
Enhance a PCIe host controller driver. Because of its unusual design
we are foced to change dev->dma_pfn_offset into a more general role
allowing multiple offsets. See the 'v1' notes below for more info.
v10:
Commit: "device-mapping: Introduce DMA range map, supplanting ..."
-- change title of commit; "bus core:" => "device-mapping:"
-- instead of allocating the DMA map with devm, use kcalloc
and call kfree() during device_release(). (RobH) Also,
for three cases that want to use the same DMA map, copy
the dma_range_map using a helper function.
-- added a missing 'return = 0;' to of_dma_get_range(). (Nicolas)
-- removed dma_range_overlaps(); instead return error if there
is an existing DMA map. (Christoph).
Commit: "PCI: brcmstb: Set additional internal memory DMA ..."
-- Changed constant 1 to 1ULL. (Nicolas)
Commit: "ata: ahci_brcm: Fix use of BCM7216 reset controller"
This commit has been removed from this patchset and will be
submitted on its own.
v9:
Commit: "device core: Introduce DMA range map, supplanting ..."
-- A number of code improvements were implemented as suggested by
ChristophH. Unfortunately, some of these changes reversed the
implemented suggestions of other reviewers; for example, the new
macros PFN_DMA_ADDR(), DMA_ADDR_PFN() have been pulled.
v8:
Commit: "device core: Introduce DMA range map, supplanting ..."
-- To satisfy a specific m68 compile configuration, I moved the 'struct
bus_dma_region; definition out of #ifdef CONFIG_HAS_DMA and also defined
three inline functions for !CONFIG_HAS_DMA (kernel test robot).
-- The sunXi drivers -- suc4i_csi, sun6i_csi, cedrus_hw -- set
a pfn_offset outside of_dma_configure() but the code offers no
insight on the size of the translation window. V7 had me using
SIZE_MAX as the size. I have since contacted the sunXi maintainer and
he said that using a size of SZ_4G would cover sunXi configurations.
v7:
Commit: "device core: Introduce DMA range map, supplanting ..."
-- remove second kcalloc/copy in device.c (AndyS)
-- use PTR_ERR_OR_ZERO() and PHYS_PFN() (AndyS)
-- indentation, sizeof(struct ...) => sizeof(*r) (AndyS)
-- add pfn.h definitions: PFN_DMA_ADDR(), DMA_ADDR_PFN() (AndyS)
-- Fixed compile error in "sun6i_csi.c" (kernel test robot)
Commit "ata: ahci_brcm: Fix use of BCM7216 reset controller"
-- correct name of function in the commit msg (SergeiS)
v6:
Commit "device core: Introduce DMA range map":
-- of_dma_get_range() now takes a single argument and returns either
NULL, a valid map, or an ERR_PTR. (Robin)
-- offsets are no longer a PFN value but an actual address. (Robin)
-- the bus_dma_region struct stores the range size instead of
the cpu_end and pci_end values. (Robin)
-- devices that were setting a single offset with no boundaries
have been modified to have boundaries; in a few places
where this information was unavilable a /* FIXME: ... */
comment was added. (Robin)
-- dma_attach_offset_range() can be called when an offset
map already exists; if it's range is already present
nothing is done and success is returned. (Robin)
All commits:
-- Man name/style/corrections/etc changed (Bjorn)
-- rebase to Torvalds master
v5:
Commit "device core: Introduce multiple dma pfn offsets"
-- in of/address.c: "map_size = 0" => "*map_size = 0"
-- use kcalloc instead of kzalloc (AndyS)
-- use PHYS_ADDR_MAX instead of "~(phys_addr_t)0"
Commit "PCI: brcmstb: Set internal memory viewport sizes"
-- now gives error on missing dma-ranges property.
Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
-- removed "Allof:" from brcm,scb-sizes definition (RobH)
All Commits:
-- indentation style, use max chars 100 (AndyS)
-- rebased to torvalds master
v4:
Commit "device core: Introduce multiple dma pfn offsets"
-- of_dma_get_range() does not take a dev param but instead
takes two "out" params: map and map_size. We do this so
that the code that parses dma-ranges is separate from
the code that modifies 'dev'. (Nicolas)
-- the separate case of having a single pfn offset has
been removed and is now processed by going through the
map array. (Nicolas)
-- move attach_uniform_dma_pfn_offset() from of/address.c to
dma/mapping.c so that it does not depend on CONFIG_OF. (Nicolas)
-- devm_kcalloc => devm_kzalloc (DanC)
-- add/fix assignment to dev->dma_pfn_offset_map for func
attach_uniform_dma_pfn_offset() (DanC, Nicolas)
-- s/struct dma_pfn_offset_region/struct bus_dma_region/ (Nicolas)
-- s/attach_uniform_dma_pfn_offset/dma_attach_uniform_pfn_offset/
-- s/attach_dma_pfn_offset_map/dma_attach_pfn_offset_map/
-- More use of PFN_{PHYS,DOWN,UP}. (AndyS)
Commit "of: Include a dev param in of_dma_get_range()"
-- this commit was sqaushed with "device core: Introduce ..."
v3:
Re: [PATCH] swiotlb: Allow allocating buffer anywhere in memory
Hello Christoph, Christoph Hellwig writes: > On Sat, Aug 15, 2020 at 05:45:36PM -0300, Thiago Jung Bauermann wrote: >> POWER secure guests (i.e., guests which use the Protection Execution >> Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but >> they don't need the SWIOTLB memory to be in low addresses since the >> hypervisor doesn't have any addressing limitation. >> >> This solves a SWIOTLB initialization problem we are seeing in secure guests >> with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved >> memory, which leaves no space for SWIOTLB in low addresses. > > What about just open coding the allocation and using > swiotlb_init_with_tbl? Yes, that works too. I just sent a v2 implementing that change. I just had to add a small accessor function so that I could set no_iotlb_memory from outside swiotlb.c. Thank you for the quick review. -- Thiago Jung Bauermann IBM Linux Technology Center ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2] powerpc/pseries/svm: Allocate SWIOTLB buffer anywhere in memory
POWER secure guests (i.e., guests which use the Protection Execution
Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but
they don't need the SWIOTLB memory to be in low addresses since the
hypervisor doesn't have any addressing limitation.
This solves a SWIOTLB initialization problem we are seeing in secure guests
with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved
memory, which leaves no space for SWIOTLB in low addresses.
To do this, we use mostly the same code as swiotlb_init(), but allocate the
buffer using memblock_alloc() instead of memblock_alloc_low().
We also need to add swiotlb_set_no_iotlb_memory() in order to set the
no_iotlb_memory flag if initialization fails.
Signed-off-by: Thiago Jung Bauermann
---
arch/powerpc/include/asm/svm.h | 4
arch/powerpc/mm/mem.c| 6 +-
arch/powerpc/platforms/pseries/svm.c | 27 +++
include/linux/swiotlb.h | 1 +
kernel/dma/swiotlb.c | 5 +
5 files changed, 42 insertions(+), 1 deletion(-)
Changes from v1:
- Open-code swiotlb_init() in arch-specific code, as suggested by
Christoph.
diff --git a/arch/powerpc/include/asm/svm.h b/arch/powerpc/include/asm/svm.h
index 85580b30aba4..7546402d796a 100644
--- a/arch/powerpc/include/asm/svm.h
+++ b/arch/powerpc/include/asm/svm.h
@@ -15,6 +15,8 @@ static inline bool is_secure_guest(void)
return mfmsr() & MSR_S;
}
+void __init svm_swiotlb_init(void);
+
void dtl_cache_ctor(void *addr);
#define get_dtl_cache_ctor() (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -25,6 +27,8 @@ static inline bool is_secure_guest(void)
return false;
}
+static inline void svm_swiotlb_init(void) {}
+
#define get_dtl_cache_ctor() NULL
#endif /* CONFIG_PPC_SVM */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index c2c11eb8dcfc..0f21bcb16405 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -50,6 +50,7 @@
#include
#include
#include
+#include
#include
@@ -290,7 +291,10 @@ void __init mem_init(void)
* back to to-down.
*/
memblock_set_bottom_up(true);
- swiotlb_init(0);
+ if (is_secure_guest())
+ svm_swiotlb_init();
+ else
+ swiotlb_init(0);
#endif
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
diff --git a/arch/powerpc/platforms/pseries/svm.c
b/arch/powerpc/platforms/pseries/svm.c
index 40c0637203d5..d592e663a8d6 100644
--- a/arch/powerpc/platforms/pseries/svm.c
+++ b/arch/powerpc/platforms/pseries/svm.c
@@ -7,6 +7,7 @@
*/
#include
+#include
#include
#include
#include
@@ -34,6 +35,32 @@ static int __init init_svm(void)
}
machine_early_initcall(pseries, init_svm);
+/*
+ * Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it
+ * can allocate the buffer anywhere in memory. Since the hypervisor doesn't
have
+ * any addressing limitation, we don't need to allocate it in low addresses.
+ */
+void __init svm_swiotlb_init(void)
+{
+ unsigned char *vstart;
+ unsigned long bytes, io_tlb_nslabs;
+
+ io_tlb_nslabs = (swiotlb_size_or_default() >> IO_TLB_SHIFT);
+ io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
+
+ bytes = io_tlb_nslabs << IO_TLB_SHIFT;
+
+ vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
+ if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, false))
+ return;
+
+ if (io_tlb_start)
+ memblock_free_early(io_tlb_start,
+ PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
+ pr_warn("Cannot allocate SWIOTLB buffer");
+ swiotlb_set_no_iotlb_memory(true);
+}
+
int set_memory_encrypted(unsigned long addr, int numpages)
{
if (!PAGE_ALIGNED(addr))
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 046bb94bd4d6..991e9f13e663 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -35,6 +35,7 @@ extern unsigned long swiotlb_nr_tbl(void);
unsigned long swiotlb_size_or_default(void);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
extern void __init swiotlb_update_mem_attributes(void);
+void __init swiotlb_set_no_iotlb_memory(bool value);
/*
* Enumeration for sync targets
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index c19379fabd20..ed2b8818ff67 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -132,6 +132,11 @@ early_param("swiotlb", setup_io_tlb_npages);
static bool no_iotlb_memory;
+void __init swiotlb_set_no_iotlb_memory(bool value)
+{
+ no_iotlb_memory = value;
+}
+
unsigned long swiotlb_nr_tbl(void)
{
return unlikely(no_iotlb_memory) ? 0 : io_tlb_nslabs;
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 2/2] Handle init_iova_flush_queue failure in dma-iommu path
init_iova_flush_queue can fail if we run out of memory. Fall back to no
flush queue if it fails.
Signed-off-by: Tom Murphy
---
drivers/iommu/dma-iommu.c | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 7433f74d921a..5445e2be08b5 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -356,9 +356,11 @@ static int iommu_dma_init_domain(struct iommu_domain
*domain, dma_addr_t base,
if (!cookie->fq_domain && !iommu_domain_get_attr(domain,
DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE, ) && attr) {
- cookie->fq_domain = domain;
- init_iova_flush_queue(iovad, iommu_dma_flush_iotlb_all,
- iommu_dma_entry_dtor);
+ if (init_iova_flush_queue(iovad, iommu_dma_flush_iotlb_all,
+ iommu_dma_entry_dtor))
+ pr_warn("iova flush queue initialization failed\n");
+ else
+ cookie->fq_domain = domain;
}
if (!dev)
--
2.20.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Rename iommu_tlb_* functions to iommu_iotlb_*
To keep naming consistent we should stick with *iotlb*. This patch
renames a few remaining functions.
Signed-off-by: Tom Murphy
---
drivers/iommu/dma-iommu.c | 2 +-
drivers/iommu/iommu.c | 4 ++--
drivers/vfio/vfio_iommu_type1.c | 2 +-
include/linux/io-pgtable.h | 2 +-
include/linux/iommu.h | 10 +-
5 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 79e6d8d799a3..59adb1a0aefc 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -503,7 +503,7 @@ static void __iommu_dma_unmap(struct device *dev,
dma_addr_t dma_addr,
domain->ops->flush_iotlb_range(domain, dma_addr, size,
freelist);
else
- iommu_tlb_sync(domain, _gather);
+ iommu_iotlb_sync(domain, _gather);
}
iommu_dma_free_iova(cookie, dma_addr, size, freelist);
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 9065127d7e9c..70a85f41876f 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -762,7 +762,7 @@ static int iommu_create_device_direct_mappings(struct
iommu_group *group,
}
- iommu_flush_tlb_all(domain);
+ iommu_flush_iotlb_all(domain);
out:
iommu_put_resv_regions(dev, );
@@ -2317,7 +2317,7 @@ size_t iommu_unmap(struct iommu_domain *domain,
if (ops->flush_iotlb_range)
ops->flush_iotlb_range(domain, iova, ret, freelist);
else
- iommu_tlb_sync(domain, _gather);
+ iommu_iotlb_sync(domain, _gather);
return ret;
}
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 570ebf878fea..d550ceb7b2aa 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -774,7 +774,7 @@ static long vfio_sync_unpin(struct vfio_dma *dma, struct
vfio_domain *domain,
long unlocked = 0;
struct vfio_regions *entry, *next;
- iommu_tlb_sync(domain->domain, iotlb_gather);
+ iommu_iotlb_sync(domain->domain, iotlb_gather);
list_for_each_entry_safe(entry, next, regions, list) {
unlocked += vfio_unpin_pages_remote(dma,
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 53d53c6c2be9..d3f2bd4a3ac4 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -31,7 +31,7 @@ enum io_pgtable_fmt {
* single page. IOMMUs that cannot batch TLB invalidation
* operations efficiently will typically issue them here, but
* others may decide to update the iommu_iotlb_gather
structure
- * and defer the invalidation until iommu_tlb_sync() instead.
+ * and defer the invalidation until iommu_iotlb_sync()
instead.
*
* Note that these can all be called in atomic context and must therefore
* not block.
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 77e773d03f22..7b363f24bf99 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -542,7 +542,7 @@ extern void iommu_domain_window_disable(struct iommu_domain
*domain, u32 wnd_nr)
extern int report_iommu_fault(struct iommu_domain *domain, struct device *dev,
unsigned long iova, int flags);
-static inline void iommu_flush_tlb_all(struct iommu_domain *domain)
+static inline void iommu_flush_iotlb_all(struct iommu_domain *domain)
{
if (domain->ops->flush_iotlb_all)
domain->ops->flush_iotlb_all(domain);
@@ -556,7 +556,7 @@ static inline void flush_iotlb_range(struct iommu_domain
*domain,
domain->ops->flush_iotlb_range(domain, iova, size, freelist);
}
-static inline void iommu_tlb_sync(struct iommu_domain *domain,
+static inline void iommu_iotlb_sync(struct iommu_domain *domain,
struct iommu_iotlb_gather *iotlb_gather)
{
if (domain->ops->iotlb_sync)
@@ -579,7 +579,7 @@ static inline void iommu_iotlb_gather_add_page(struct
iommu_domain *domain,
if (gather->pgsize != size ||
end < gather->start || start > gather->end) {
if (gather->pgsize)
- iommu_tlb_sync(domain, gather);
+ iommu_iotlb_sync(domain, gather);
gather->pgsize = size;
}
@@ -762,11 +762,11 @@ static inline size_t iommu_map_sg_atomic(struct
iommu_domain *domain,
return 0;
}
-static inline void iommu_flush_tlb_all(struct iommu_domain *domain)
+static inline void iommu_flush_iotlb_all(struct iommu_domain *domain)
{
}
-static inline void iommu_tlb_sync(struct iommu_domain *domain,
+static inline void iommu_iotlb_sync(struct iommu_domain *domain,
struct iommu_iotlb_gather *iotlb_gather)
{
}
--
2.20.1
[PATCH 1/2] Add new flush_iotlb_range and handle freelists when using iommu_unmap_fast
Add a flush_iotlb_range to allow flushing of an iova range instead of a
full flush in the dma-iommu path.
Allow the iommu_unmap_fast to return newly freed page table pages and
pass the freelist to queue_iova in the dma-iommu ops path.
This patch is useful for iommu drivers (in this case the intel iommu
driver) which need to wait for the ioTLB to be flushed before newly
free/unmapped page table pages can be freed. This way we can still batch
ioTLB free operations and handle the freelists.
Signed-off-by: Tom Murphy
---
drivers/iommu/amd/iommu.c | 14 -
drivers/iommu/arm-smmu-v3.c | 3 +-
drivers/iommu/arm-smmu.c| 3 +-
drivers/iommu/dma-iommu.c | 45 ---
drivers/iommu/exynos-iommu.c| 3 +-
drivers/iommu/intel/iommu.c | 54 +
drivers/iommu/iommu.c | 25 +++
drivers/iommu/ipmmu-vmsa.c | 3 +-
drivers/iommu/msm_iommu.c | 3 +-
drivers/iommu/mtk_iommu.c | 3 +-
drivers/iommu/omap-iommu.c | 3 +-
drivers/iommu/qcom_iommu.c | 3 +-
drivers/iommu/rockchip-iommu.c | 3 +-
drivers/iommu/s390-iommu.c | 3 +-
drivers/iommu/sun50i-iommu.c| 3 +-
drivers/iommu/tegra-gart.c | 3 +-
drivers/iommu/tegra-smmu.c | 3 +-
drivers/iommu/virtio-iommu.c| 3 +-
drivers/vfio/vfio_iommu_type1.c | 2 +-
include/linux/iommu.h | 21 +++--
20 files changed, 148 insertions(+), 55 deletions(-)
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 2f22326ee4df..25fbacab23c3 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -2513,7 +2513,8 @@ static int amd_iommu_map(struct iommu_domain *dom,
unsigned long iova,
static size_t amd_iommu_unmap(struct iommu_domain *dom, unsigned long iova,
size_t page_size,
- struct iommu_iotlb_gather *gather)
+ struct iommu_iotlb_gather *gather,
+ struct page **freelist)
{
struct protection_domain *domain = to_pdomain(dom);
struct domain_pgtable pgtable;
@@ -2636,6 +2637,16 @@ static void amd_iommu_flush_iotlb_all(struct
iommu_domain *domain)
spin_unlock_irqrestore(>lock, flags);
}
+static void amd_iommu_flush_iotlb_range(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ struct page *freelist)
+{
+ struct protection_domain *dom = to_pdomain(domain);
+
+ domain_flush_pages(dom, iova, size);
+ domain_flush_complete(dom);
+}
+
static void amd_iommu_iotlb_sync(struct iommu_domain *domain,
struct iommu_iotlb_gather *gather)
{
@@ -2675,6 +2686,7 @@ const struct iommu_ops amd_iommu_ops = {
.is_attach_deferred = amd_iommu_is_attach_deferred,
.pgsize_bitmap = AMD_IOMMU_PGSIZES,
.flush_iotlb_all = amd_iommu_flush_iotlb_all,
+ .flush_iotlb_range = amd_iommu_flush_iotlb_range,
.iotlb_sync = amd_iommu_iotlb_sync,
.def_domain_type = amd_iommu_def_domain_type,
};
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index f578677a5c41..8d328dc25326 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2854,7 +2854,8 @@ static int arm_smmu_map(struct iommu_domain *domain,
unsigned long iova,
}
static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova,
-size_t size, struct iommu_iotlb_gather *gather)
+size_t size, struct iommu_iotlb_gather *gather,
+struct page **freelist)
{
struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index 243bc4cb2705..0cd0dfc89875 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -1234,7 +1234,8 @@ static int arm_smmu_map(struct iommu_domain *domain,
unsigned long iova,
}
static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova,
-size_t size, struct iommu_iotlb_gather *gather)
+size_t size, struct iommu_iotlb_gather *gather,
+struct page **freelist)
{
struct io_pgtable_ops *ops = to_smmu_domain(domain)->pgtbl_ops;
struct arm_smmu_device *smmu = to_smmu_domain(domain)->smmu;
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 4959f5df21bd..7433f74d921a 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -50,6 +50,19 @@ struct iommu_dma_cookie {
struct iommu_domain *fq_domain;
};
+
+static void iommu_dma_entry_dtor(unsigned long data)
+{
+ struct page *freelist = (struct page *)data;
+
+ while (freelist !=
[PATCH RESEND v9 10/13] iommu/arm-smmu-v3: Check for SVA features
Aggregate all sanity-checks for sharing CPU page tables with the SMMU
under a single ARM_SMMU_FEAT_SVA bit. For PCIe SVA, users also need to
check FEAT_ATS and FEAT_PRI. For platform SVA, they will have to check
FEAT_STALLS.
Introduce ARM_SMMU_FEAT_BTM (Broadcast TLB Maintenance), but don't
enable it at the moment. Since the entire VMID space is shared with the
CPU, enabling DVM (by clearing SMMU_CR2.PTM) could result in
over-invalidation and affect performance of stage-2 mappings.
Cc: Suzuki K Poulose
Signed-off-by: Jean-Philippe Brucker
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 10 +
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 43 +++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 ++
3 files changed, 56 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 90c08f156b43..7b14b48a26c7 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -602,6 +602,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_STALL_FORCE (1 << 13)
#define ARM_SMMU_FEAT_VAX (1 << 14)
#define ARM_SMMU_FEAT_RANGE_INV(1 << 15)
+#define ARM_SMMU_FEAT_BTM (1 << 16)
+#define ARM_SMMU_FEAT_SVA (1 << 17)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
@@ -683,4 +685,12 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain
*smmu_domain, int ssid,
void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid);
bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd);
+#ifdef CONFIG_ARM_SMMU_V3_SVA
+bool arm_smmu_sva_supported(struct arm_smmu_device *smmu);
+#else /* CONFIG_ARM_SMMU_V3_SVA */
+static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
+{
+ return false;
+}
+#endif /* CONFIG_ARM_SMMU_V3_SVA */
#endif /* _ARM_SMMU_V3_H */
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index e919ce894dd1..bf81d91ce71e 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -153,3 +153,46 @@ static void arm_smmu_free_shared_cd(struct
arm_smmu_ctx_desc *cd)
kfree(cd);
}
}
+
+bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
+{
+ unsigned long reg, fld;
+ unsigned long oas;
+ unsigned long asid_bits;
+
+ u32 feat_mask = ARM_SMMU_FEAT_BTM | ARM_SMMU_FEAT_COHERENCY;
+
+ if ((smmu->features & feat_mask) != feat_mask)
+ return false;
+
+ if (!(smmu->pgsize_bitmap & PAGE_SIZE))
+ return false;
+
+ /*
+* Get the smallest PA size of all CPUs (sanitized by cpufeature). We're
+* not even pretending to support AArch32 here. Abort if the MMU outputs
+* addresses larger than what we support.
+*/
+ reg = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);
+ fld = cpuid_feature_extract_unsigned_field(reg,
ID_AA64MMFR0_PARANGE_SHIFT);
+ oas = id_aa64mmfr0_parange_to_phys_shift(fld);
+ if (smmu->oas < oas)
+ return false;
+
+ /* We can support bigger ASIDs than the CPU, but not smaller */
+ fld = cpuid_feature_extract_unsigned_field(reg,
ID_AA64MMFR0_ASID_SHIFT);
+ asid_bits = fld ? 16 : 8;
+ if (smmu->asid_bits < asid_bits)
+ return false;
+
+ /*
+* See max_pinned_asids in arch/arm64/mm/context.c. The following is
+* generally the maximum number of bindable processes.
+*/
+ if (IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0))
+ asid_bits--;
+ dev_dbg(smmu->dev, "%d shared contexts\n", (1 << asid_bits) -
+ num_possible_cpus() - 2);
+
+ return true;
+}
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9e755caea525..15cb3d9c1a5d 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3258,6 +3258,9 @@ static int arm_smmu_device_hw_probe(struct
arm_smmu_device *smmu)
smmu->ias = max(smmu->ias, smmu->oas);
+ if (arm_smmu_sva_supported(smmu))
+ smmu->features |= ARM_SMMU_FEAT_SVA;
+
dev_info(smmu->dev, "ias %lu-bit, oas %lu-bit (features 0x%08x)\n",
smmu->ias, smmu->oas, smmu->features);
return 0;
--
2.28.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH RESEND v9 07/13] iommu/arm-smmu-v3: Move definitions to a header
Allow sharing structure definitions with the upcoming SVA support for Arm SMMUv3, by moving them to a separate header. We could surgically extract only what is needed but keeping all definitions in one place looks nicer. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 675 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 660 +-- 2 files changed, 676 insertions(+), 659 deletions(-) create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h new file mode 100644 index ..51a9ce07b2d6 --- /dev/null +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -0,0 +1,675 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * IOMMU API for ARM architected SMMUv3 implementations. + * + * Copyright (C) 2015 ARM Limited + */ + +#ifndef _ARM_SMMU_V3_H +#define _ARM_SMMU_V3_H + +#include +#include +#include +#include +#include + +/* MMIO registers */ +#define ARM_SMMU_IDR0 0x0 +#define IDR0_ST_LVLGENMASK(28, 27) +#define IDR0_ST_LVL_2LVL 1 +#define IDR0_STALL_MODEL GENMASK(25, 24) +#define IDR0_STALL_MODEL_STALL 0 +#define IDR0_STALL_MODEL_FORCE 2 +#define IDR0_TTENDIAN GENMASK(22, 21) +#define IDR0_TTENDIAN_MIXED0 +#define IDR0_TTENDIAN_LE 2 +#define IDR0_TTENDIAN_BE 3 +#define IDR0_CD2L (1 << 19) +#define IDR0_VMID16(1 << 18) +#define IDR0_PRI (1 << 16) +#define IDR0_SEV (1 << 14) +#define IDR0_MSI (1 << 13) +#define IDR0_ASID16(1 << 12) +#define IDR0_ATS (1 << 10) +#define IDR0_HYP (1 << 9) +#define IDR0_COHACC(1 << 4) +#define IDR0_TTF GENMASK(3, 2) +#define IDR0_TTF_AARCH64 2 +#define IDR0_TTF_AARCH32_643 +#define IDR0_S1P (1 << 1) +#define IDR0_S2P (1 << 0) + +#define ARM_SMMU_IDR1 0x4 +#define IDR1_TABLES_PRESET (1 << 30) +#define IDR1_QUEUES_PRESET (1 << 29) +#define IDR1_REL (1 << 28) +#define IDR1_CMDQS GENMASK(25, 21) +#define IDR1_EVTQS GENMASK(20, 16) +#define IDR1_PRIQS GENMASK(15, 11) +#define IDR1_SSIDSIZE GENMASK(10, 6) +#define IDR1_SIDSIZE GENMASK(5, 0) + +#define ARM_SMMU_IDR3 0xc +#define IDR3_RIL (1 << 10) + +#define ARM_SMMU_IDR5 0x14 +#define IDR5_STALL_MAX GENMASK(31, 16) +#define IDR5_GRAN64K (1 << 6) +#define IDR5_GRAN16K (1 << 5) +#define IDR5_GRAN4K(1 << 4) +#define IDR5_OAS GENMASK(2, 0) +#define IDR5_OAS_32_BIT0 +#define IDR5_OAS_36_BIT1 +#define IDR5_OAS_40_BIT2 +#define IDR5_OAS_42_BIT3 +#define IDR5_OAS_44_BIT4 +#define IDR5_OAS_48_BIT5 +#define IDR5_OAS_52_BIT6 +#define IDR5_VAX GENMASK(11, 10) +#define IDR5_VAX_52_BIT1 + +#define ARM_SMMU_CR0 0x20 +#define CR0_ATSCHK (1 << 4) +#define CR0_CMDQEN (1 << 3) +#define CR0_EVTQEN (1 << 2) +#define CR0_PRIQEN (1 << 1) +#define CR0_SMMUEN (1 << 0) + +#define ARM_SMMU_CR0ACK0x24 + +#define ARM_SMMU_CR1 0x28 +#define CR1_TABLE_SH GENMASK(11, 10) +#define CR1_TABLE_OC GENMASK(9, 8) +#define CR1_TABLE_IC GENMASK(7, 6) +#define CR1_QUEUE_SH GENMASK(5, 4) +#define CR1_QUEUE_OC GENMASK(3, 2) +#define CR1_QUEUE_IC GENMASK(1, 0) +/* CR1 cacheability fields don't quite follow the usual TCR-style encoding */ +#define CR1_CACHE_NC 0 +#define CR1_CACHE_WB 1 +#define CR1_CACHE_WT 2 + +#define ARM_SMMU_CR2 0x2c +#define CR2_PTM(1 << 2) +#define CR2_RECINVSID (1 << 1) +#define CR2_E2H(1 << 0) + +#define ARM_SMMU_GBPA 0x44 +#define GBPA_UPDATE(1 << 31) +#define GBPA_ABORT (1 << 20) + +#define ARM_SMMU_IRQ_CTRL 0x50 +#define IRQ_CTRL_EVTQ_IRQEN(1 << 2) +#define IRQ_CTRL_PRIQ_IRQEN(1 << 1) +#define IRQ_CTRL_GERROR_IRQEN (1 <<
[PATCH RESEND v9 11/13] iommu/arm-smmu-v3: Add SVA device feature
Implement the IOMMU device feature callbacks to support the SVA feature.
At the moment dev_has_feat() returns false since I/O Page Faults isn't
yet implemented.
Signed-off-by: Jean-Philippe Brucker
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 26 ++
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 49
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 79 +++
3 files changed, 154 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 7b14b48a26c7..ba34914813ff 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -646,6 +646,8 @@ struct arm_smmu_master {
u32 *sids;
unsigned intnum_sids;
boolats_enabled;
+ boolsva_enabled;
+ struct list_headbonds;
unsigned intssid_bits;
};
@@ -687,10 +689,34 @@ bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd);
#ifdef CONFIG_ARM_SMMU_V3_SVA
bool arm_smmu_sva_supported(struct arm_smmu_device *smmu);
+bool arm_smmu_master_sva_supported(struct arm_smmu_master *master);
+bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master);
+int arm_smmu_master_enable_sva(struct arm_smmu_master *master);
+int arm_smmu_master_disable_sva(struct arm_smmu_master *master);
#else /* CONFIG_ARM_SMMU_V3_SVA */
static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
{
return false;
}
+
+static inline bool arm_smmu_master_sva_supported(struct arm_smmu_master
*master)
+{
+ return false;
+}
+
+static inline bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master)
+{
+ return false;
+}
+
+static inline int arm_smmu_master_enable_sva(struct arm_smmu_master *master)
+{
+ return -ENODEV;
+}
+
+static inline int arm_smmu_master_disable_sva(struct arm_smmu_master *master)
+{
+ return -ENODEV;
+}
#endif /* CONFIG_ARM_SMMU_V3_SVA */
#endif /* _ARM_SMMU_V3_H */
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index bf81d91ce71e..28027620cf2e 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -10,6 +10,8 @@
#include "arm-smmu-v3.h"
#include "../../io-pgtable-arm.h"
+static DEFINE_MUTEX(sva_lock);
+
/*
* Try to reserve this ASID in the SMMU. If it is in use, try to steal it from
* the private entry. Careful here, we may be modifying the context tables of
@@ -196,3 +198,50 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
return true;
}
+
+static bool arm_smmu_iopf_supported(struct arm_smmu_master *master)
+{
+ return false;
+}
+
+bool arm_smmu_master_sva_supported(struct arm_smmu_master *master)
+{
+ if (!(master->smmu->features & ARM_SMMU_FEAT_SVA))
+ return false;
+
+ /* SSID and IOPF support are mandatory for the moment */
+ return master->ssid_bits && arm_smmu_iopf_supported(master);
+}
+
+bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master)
+{
+ bool enabled;
+
+ mutex_lock(_lock);
+ enabled = master->sva_enabled;
+ mutex_unlock(_lock);
+ return enabled;
+}
+
+int arm_smmu_master_enable_sva(struct arm_smmu_master *master)
+{
+ mutex_lock(_lock);
+ master->sva_enabled = true;
+ mutex_unlock(_lock);
+
+ return 0;
+}
+
+int arm_smmu_master_disable_sva(struct arm_smmu_master *master)
+{
+ mutex_lock(_lock);
+ if (!list_empty(>bonds)) {
+ dev_err(master->dev, "cannot disable SVA, device is bound\n");
+ mutex_unlock(_lock);
+ return -EBUSY;
+ }
+ master->sva_enabled = false;
+ mutex_unlock(_lock);
+
+ return 0;
+}
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 15cb3d9c1a5d..5ed5bb42298f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2163,6 +2163,16 @@ static int arm_smmu_attach_dev(struct iommu_domain
*domain, struct device *dev)
master = dev_iommu_priv_get(dev);
smmu = master->smmu;
+ /*
+* Checking that SVA is disabled ensures that this device isn't bound to
+* any mm, and can be safely detached from its old domain. Bonds cannot
+* be removed concurrently since we're holding the group mutex.
+*/
+ if (arm_smmu_master_sva_enabled(master)) {
+ dev_err(dev, "cannot attach - SVA enabled\n");
+ return -EBUSY;
+ }
+
arm_smmu_detach_dev(master);
mutex_lock(_domain->init_mutex);
@@ -2310,6 +2320,7 @@ static struct iommu_device *arm_smmu_probe_device(struct
device *dev)
master->smmu = smmu;
master->sids =
[PATCH RESEND v9 09/13] iommu/arm-smmu-v3: Seize private ASID
The SMMU has a single ASID space, the union of shared and private ASID
sets. This means that the SMMU driver competes with the arch allocator
for ASIDs. Shared ASIDs are those of Linux processes, allocated by the
arch, and contribute in broadcast TLB maintenance. Private ASIDs are
allocated by the SMMU driver and used for "classic" map/unmap DMA. They
require command-queue TLB invalidations.
When we pin down an mm_context and get an ASID that is already in use by
the SMMU, it belongs to a private context. We used to simply abort the
bind, but this is unfair to users that would be unable to bind a few
seemingly random processes. Try to allocate a new private ASID for the
context, and make the old ASID shared.
Signed-off-by: Jean-Philippe Brucker
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 ++
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 36 +--
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 34 +++---
3 files changed, 58 insertions(+), 15 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 6b06a6f19604..90c08f156b43 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -678,6 +678,9 @@ struct arm_smmu_domain {
extern struct xarray arm_smmu_asid_xa;
extern struct mutex arm_smmu_asid_lock;
+int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
+ struct arm_smmu_ctx_desc *cd);
+void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid);
bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd);
#endif /* _ARM_SMMU_V3_H */
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index 7a4f40565e06..e919ce894dd1 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -10,10 +10,19 @@
#include "arm-smmu-v3.h"
#include "../../io-pgtable-arm.h"
+/*
+ * Try to reserve this ASID in the SMMU. If it is in use, try to steal it from
+ * the private entry. Careful here, we may be modifying the context tables of
+ * another SMMU!
+ */
static struct arm_smmu_ctx_desc *
arm_smmu_share_asid(struct mm_struct *mm, u16 asid)
{
+ int ret;
+ u32 new_asid;
struct arm_smmu_ctx_desc *cd;
+ struct arm_smmu_device *smmu;
+ struct arm_smmu_domain *smmu_domain;
cd = xa_load(_smmu_asid_xa, asid);
if (!cd)
@@ -27,8 +36,31 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid)
return cd;
}
- /* Ouch, ASID is already in use for a private cd. */
- return ERR_PTR(-EBUSY);
+ smmu_domain = container_of(cd, struct arm_smmu_domain, s1_cfg.cd);
+ smmu = smmu_domain->smmu;
+
+ ret = xa_alloc(_smmu_asid_xa, _asid, cd,
+ XA_LIMIT(1, 1 << smmu->asid_bits), GFP_KERNEL);
+ if (ret)
+ return ERR_PTR(-ENOSPC);
+ /*
+* Race with unmap: TLB invalidations will start targeting the new ASID,
+* which isn't assigned yet. We'll do an invalidate-all on the old ASID
+* later, so it doesn't matter.
+*/
+ cd->asid = new_asid;
+ /*
+* Update ASID and invalidate CD in all associated masters. There will
+* be some overlap between use of both ASIDs, until we invalidate the
+* TLB.
+*/
+ arm_smmu_write_ctx_desc(smmu_domain, 0, cd);
+
+ /* Invalidate TLB entries previously associated with that context */
+ arm_smmu_tlb_inv_asid(smmu, asid);
+
+ xa_erase(_smmu_asid_xa, asid);
+ return NULL;
}
__maybe_unused
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9e81615744de..9e755caea525 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -873,6 +873,17 @@ static int arm_smmu_cmdq_batch_submit(struct
arm_smmu_device *smmu,
}
/* Context descriptor manipulation functions */
+void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid)
+{
+ struct arm_smmu_cmdq_ent cmd = {
+ .opcode = CMDQ_OP_TLBI_NH_ASID,
+ .tlbi.asid = asid,
+ };
+
+ arm_smmu_cmdq_issue_cmd(smmu, );
+ arm_smmu_cmdq_issue_sync(smmu);
+}
+
static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain,
int ssid, bool leaf)
{
@@ -953,8 +964,8 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain
*smmu_domain,
return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS;
}
-static int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain,
- int ssid, struct arm_smmu_ctx_desc *cd)
+int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
+ struct arm_smmu_ctx_desc *cd)
{
/*
* This function
[PATCH RESEND v9 12/13] iommu/arm-smmu-v3: Implement iommu_sva_bind/unbind()
The sva_bind() function allows devices to access process address spaces
using a PASID (aka SSID).
(1) bind() allocates or gets an existing MMU notifier tied to the
(domain, mm) pair. Each mm gets one PASID.
(2) Any change to the address space calls invalidate_range() which sends
ATC invalidations (in a subsequent patch).
(3) When the process address space dies, the release() notifier disables
the CD to allow reclaiming the page tables. Since release() has to
be light we do not instruct device drivers to stop DMA here, we just
ignore incoming page faults from this point onwards.
To avoid any event 0x0a print (C_BAD_CD) we disable translation
without clearing CD.V. PCIe Translation Requests and Page Requests
are silently denied. Don't clear the R bit because the S bit can't
be cleared when STALL_MODEL==0b10 (forced), and clearing R without
clearing S is useless. Faulting transactions will stall and will be
aborted by the IOPF handler.
(4) After stopping DMA, the device driver releases the bond by calling
unbind(). We release the MMU notifier, free the PASID and the bond.
Three structures keep track of bonds:
* arm_smmu_bond: one per {device, mm} pair, the handle returned to the
device driver for a bind() request.
* arm_smmu_mmu_notifier: one per {domain, mm} pair, deals with ATS/TLB
invalidations and clearing the context descriptor on mm exit.
* arm_smmu_ctx_desc: one per mm, holds the pinned ASID and pgd.
Signed-off-by: Jean-Philippe Brucker
---
drivers/iommu/Kconfig | 2 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 +++
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 230 +-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 32 ++-
4 files changed, 282 insertions(+), 10 deletions(-)
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index b1d592cd9984..a8eb8b7f35f7 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -316,6 +316,8 @@ config ARM_SMMU_V3
config ARM_SMMU_V3_SVA
bool "Shared Virtual Addressing support for the ARM SMMUv3"
depends on ARM_SMMU_V3
+ select IOMMU_SVA_LIB
+ select MMU_NOTIFIER
help
Support for sharing process address spaces with devices using the
SMMUv3.
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index ba34914813ff..6365c81a4614 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -677,10 +677,18 @@ struct arm_smmu_domain {
struct list_headdevices;
spinlock_t devices_lock;
+
+ struct list_headmmu_notifiers;
};
+static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom)
+{
+ return container_of(dom, struct arm_smmu_domain, domain);
+}
+
extern struct xarray arm_smmu_asid_xa;
extern struct mutex arm_smmu_asid_lock;
+extern struct arm_smmu_ctx_desc quiet_cd;
int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
struct arm_smmu_ctx_desc *cd);
@@ -693,6 +701,11 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master
*master);
bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master);
int arm_smmu_master_enable_sva(struct arm_smmu_master *master);
int arm_smmu_master_disable_sva(struct arm_smmu_master *master);
+struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm,
+ void *drvdata);
+void arm_smmu_sva_unbind(struct iommu_sva *handle);
+int arm_smmu_sva_get_pasid(struct iommu_sva *handle);
+void arm_smmu_sva_notifier_synchronize(void);
#else /* CONFIG_ARM_SMMU_V3_SVA */
static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
{
@@ -718,5 +731,20 @@ static inline int arm_smmu_master_disable_sva(struct
arm_smmu_master *master)
{
return -ENODEV;
}
+
+static inline struct iommu_sva *
+arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata)
+{
+ return ERR_PTR(-ENODEV);
+}
+
+static inline void arm_smmu_sva_unbind(struct iommu_sva *handle) {}
+
+static inline int arm_smmu_sva_get_pasid(struct iommu_sva *handle)
+{
+ return IOMMU_PASID_INVALID;
+}
+
+static inline void arm_smmu_sva_notifier_synchronize(void) {}
#endif /* CONFIG_ARM_SMMU_V3_SVA */
#endif /* _ARM_SMMU_V3_H */
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index 28027620cf2e..fb9bfda2d192 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -5,11 +5,35 @@
#include
#include
+#include
#include
#include "arm-smmu-v3.h"
+#include "../../iommu-sva-lib.h"
#include "../../io-pgtable-arm.h"
+struct arm_smmu_mmu_notifier {
+ struct mmu_notifier mn;
+ struct arm_smmu_ctx_desc
[PATCH RESEND v9 13/13] iommu/arm-smmu-v3: Hook up ATC invalidation to mm ops
The invalidate_range() notifier is called for any change to the address
space. Perform the required ATC invalidations.
Signed-off-by: Jean-Philippe Brucker
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h| 2 ++
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c| 16 +++-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c| 18 --
3 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 6365c81a4614..baa80498ad9f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -694,6 +694,8 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain
*smmu_domain, int ssid,
struct arm_smmu_ctx_desc *cd);
void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid);
bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd);
+int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid,
+ unsigned long iova, size_t size);
#ifdef CONFIG_ARM_SMMU_V3_SVA
bool arm_smmu_sva_supported(struct arm_smmu_device *smmu);
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index fb9bfda2d192..1687aac2d96b 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -178,6 +178,16 @@ static void arm_smmu_free_shared_cd(struct
arm_smmu_ctx_desc *cd)
}
}
+static void arm_smmu_mm_invalidate_range(struct mmu_notifier *mn,
+struct mm_struct *mm,
+unsigned long start, unsigned long end)
+{
+ struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn);
+
+ arm_smmu_atc_inv_domain(smmu_mn->domain, mm->pasid, start,
+ end - start + 1);
+}
+
static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm)
{
struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn);
@@ -196,6 +206,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn,
struct mm_struct *mm)
arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, _cd);
arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid);
+ arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0);
smmu_mn->cleared = true;
mutex_unlock(_lock);
@@ -207,6 +218,7 @@ static void arm_smmu_mmu_notifier_free(struct mmu_notifier
*mn)
}
static struct mmu_notifier_ops arm_smmu_mmu_notifier_ops = {
+ .invalidate_range = arm_smmu_mm_invalidate_range,
.release= arm_smmu_mm_release,
.free_notifier = arm_smmu_mmu_notifier_free,
};
@@ -279,8 +291,10 @@ static void arm_smmu_mmu_notifier_put(struct
arm_smmu_mmu_notifier *smmu_mn)
* If we went through clear(), we've already invalidated, and no
* new TLB entry can have been formed.
*/
- if (!smmu_mn->cleared)
+ if (!smmu_mn->cleared) {
arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid);
+ arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0);
+ }
/* Frees smmu_mn */
mmu_notifier_put(_mn->mn);
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index ae93cdd09eff..d85a49b41a15 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1515,6 +1515,20 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova,
size_t size,
size_t inval_grain_shift = 12;
unsigned long page_start, page_end;
+ /*
+* ATS and PASID:
+*
+* If substream_valid is clear, the PCIe TLP is sent without a PASID
+* prefix. In that case all ATC entries within the address range are
+* invalidated, including those that were requested with a PASID! There
+* is no way to invalidate only entries without PASID.
+*
+* When using STRTAB_STE_1_S1DSS_SSID0 (reserving CD 0 for non-PASID
+* traffic), translation requests without PASID create ATC entries
+* without PASID, which must be invalidated with substream_valid clear.
+* This has the unpleasant side-effect of invalidating all PASID-tagged
+* ATC entries within the address range.
+*/
*cmd = (struct arm_smmu_cmdq_ent) {
.opcode = CMDQ_OP_ATC_INV,
.substream_valid= !!ssid,
@@ -1573,8 +1587,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master
*master)
return arm_smmu_cmdq_issue_sync(master->smmu);
}
-static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain,
- int ssid, unsigned long iova, size_t size)
+int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid,
+
[PATCH RESEND v9 08/13] iommu/arm-smmu-v3: Share process page tables
With Shared Virtual Addressing (SVA), we need to mirror CPU TTBR, TCR,
MAIR and ASIDs in SMMU contexts. Each SMMU has a single ASID space split
into two sets, shared and private. Shared ASIDs correspond to those
obtained from the arch ASID allocator, and private ASIDs are used for
"classic" map/unmap DMA.
A possible conflict happens when trying to use a shared ASID that has
already been allocated for private use by the SMMU driver. This will be
addressed in a later patch by replacing the private ASID. At the
moment we return -EBUSY.
Each mm_struct shared with the SMMU will have a single context
descriptor. Add a refcount to keep track of this. It will be protected
by the global SVA lock.
Introduce a new arm-smmu-v3-sva.c file and the CONFIG_ARM_SMMU_V3_SVA
option to let users opt in SVA support.
Signed-off-by: Jean-Philippe Brucker
---
v9: Move to arm-smmu-v3-sva.c
---
drivers/iommu/Kconfig | 10 ++
drivers/iommu/arm/arm-smmu-v3/Makefile| 5 +-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 8 ++
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 123 ++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 34 -
5 files changed, 172 insertions(+), 8 deletions(-)
create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index fb1787377eb6..b1d592cd9984 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -313,6 +313,16 @@ config ARM_SMMU_V3
Say Y here if your system includes an IOMMU device implementing
the ARM SMMUv3 architecture.
+config ARM_SMMU_V3_SVA
+ bool "Shared Virtual Addressing support for the ARM SMMUv3"
+ depends on ARM_SMMU_V3
+ help
+ Support for sharing process address spaces with devices using the
+ SMMUv3.
+
+ Say Y here if your system supports SVA extensions such as PCIe PASID
+ and PRI.
+
config S390_IOMMU
def_bool y if S390 && PCI
depends on S390 && PCI
diff --git a/drivers/iommu/arm/arm-smmu-v3/Makefile
b/drivers/iommu/arm/arm-smmu-v3/Makefile
index 569e24e9f162..54feb1ecccad 100644
--- a/drivers/iommu/arm/arm-smmu-v3/Makefile
+++ b/drivers/iommu/arm/arm-smmu-v3/Makefile
@@ -1,2 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o
+obj-$(CONFIG_ARM_SMMU_V3) += arm_smmu_v3.o
+arm_smmu_v3-objs-y += arm-smmu-v3.o
+arm_smmu_v3-objs-$(CONFIG_ARM_SMMU_V3_SVA) += arm-smmu-v3-sva.o
+arm_smmu_v3-objs := $(arm_smmu_v3-objs-y)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 51a9ce07b2d6..6b06a6f19604 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -540,6 +540,9 @@ struct arm_smmu_ctx_desc {
u64 ttbr;
u64 tcr;
u64 mair;
+
+ refcount_t refs;
+ struct mm_struct*mm;
};
struct arm_smmu_l1_ctx_desc {
@@ -672,4 +675,9 @@ struct arm_smmu_domain {
spinlock_t devices_lock;
};
+extern struct xarray arm_smmu_asid_xa;
+extern struct mutex arm_smmu_asid_lock;
+
+bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd);
+
#endif /* _ARM_SMMU_V3_H */
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
new file mode 100644
index ..7a4f40565e06
--- /dev/null
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -0,0 +1,123 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Implementation of the IOMMU SVA API for the ARM SMMUv3
+ */
+
+#include
+#include
+#include
+
+#include "arm-smmu-v3.h"
+#include "../../io-pgtable-arm.h"
+
+static struct arm_smmu_ctx_desc *
+arm_smmu_share_asid(struct mm_struct *mm, u16 asid)
+{
+ struct arm_smmu_ctx_desc *cd;
+
+ cd = xa_load(_smmu_asid_xa, asid);
+ if (!cd)
+ return NULL;
+
+ if (cd->mm) {
+ if (WARN_ON(cd->mm != mm))
+ return ERR_PTR(-EINVAL);
+ /* All devices bound to this mm use the same cd struct. */
+ refcount_inc(>refs);
+ return cd;
+ }
+
+ /* Ouch, ASID is already in use for a private cd. */
+ return ERR_PTR(-EBUSY);
+}
+
+__maybe_unused
+static struct arm_smmu_ctx_desc *arm_smmu_alloc_shared_cd(struct mm_struct *mm)
+{
+ u16 asid;
+ int err = 0;
+ u64 tcr, par, reg;
+ struct arm_smmu_ctx_desc *cd;
+ struct arm_smmu_ctx_desc *ret = NULL;
+
+ asid = arm64_mm_context_get(mm);
+ if (!asid)
+ return ERR_PTR(-ESRCH);
+
+ cd = kzalloc(sizeof(*cd), GFP_KERNEL);
+ if (!cd) {
+ err = -ENOMEM;
+ goto out_put_context;
+ }
+
+ refcount_set(>refs, 1);
+
+ mutex_lock(_smmu_asid_lock);
+
[PATCH RESEND v9 05/13] iommu/io-pgtable-arm: Move some definitions to a header
Extract some of the most generic TCR defines, so they can be reused by the page table sharing code. Acked-by: Will Deacon Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/io-pgtable-arm.h | 30 ++ drivers/iommu/io-pgtable-arm.c | 27 ++- MAINTAINERS| 3 +-- 3 files changed, 33 insertions(+), 27 deletions(-) create mode 100644 drivers/iommu/io-pgtable-arm.h diff --git a/drivers/iommu/io-pgtable-arm.h b/drivers/iommu/io-pgtable-arm.h new file mode 100644 index ..ba7cfdf7afa0 --- /dev/null +++ b/drivers/iommu/io-pgtable-arm.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef IO_PGTABLE_ARM_H_ +#define IO_PGTABLE_ARM_H_ + +#define ARM_LPAE_TCR_TG0_4K0 +#define ARM_LPAE_TCR_TG0_64K 1 +#define ARM_LPAE_TCR_TG0_16K 2 + +#define ARM_LPAE_TCR_TG1_16K 1 +#define ARM_LPAE_TCR_TG1_4K2 +#define ARM_LPAE_TCR_TG1_64K 3 + +#define ARM_LPAE_TCR_SH_NS 0 +#define ARM_LPAE_TCR_SH_OS 2 +#define ARM_LPAE_TCR_SH_IS 3 + +#define ARM_LPAE_TCR_RGN_NC0 +#define ARM_LPAE_TCR_RGN_WBWA 1 +#define ARM_LPAE_TCR_RGN_WT2 +#define ARM_LPAE_TCR_RGN_WB3 + +#define ARM_LPAE_TCR_PS_32_BIT 0x0ULL +#define ARM_LPAE_TCR_PS_36_BIT 0x1ULL +#define ARM_LPAE_TCR_PS_40_BIT 0x2ULL +#define ARM_LPAE_TCR_PS_42_BIT 0x3ULL +#define ARM_LPAE_TCR_PS_44_BIT 0x4ULL +#define ARM_LPAE_TCR_PS_48_BIT 0x5ULL +#define ARM_LPAE_TCR_PS_52_BIT 0x6ULL + +#endif /* IO_PGTABLE_ARM_H_ */ diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index dc7bcf858b6d..534810b6be77 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -20,6 +20,8 @@ #include +#include "io-pgtable-arm.h" + #define ARM_LPAE_MAX_ADDR_BITS 52 #define ARM_LPAE_S2_MAX_CONCAT_PAGES 16 #define ARM_LPAE_MAX_LEVELS4 @@ -100,23 +102,6 @@ #define ARM_LPAE_PTE_MEMATTR_DEV (((arm_lpae_iopte)0x1) << 2) /* Register bits */ -#define ARM_LPAE_TCR_TG0_4K0 -#define ARM_LPAE_TCR_TG0_64K 1 -#define ARM_LPAE_TCR_TG0_16K 2 - -#define ARM_LPAE_TCR_TG1_16K 1 -#define ARM_LPAE_TCR_TG1_4K2 -#define ARM_LPAE_TCR_TG1_64K 3 - -#define ARM_LPAE_TCR_SH_NS 0 -#define ARM_LPAE_TCR_SH_OS 2 -#define ARM_LPAE_TCR_SH_IS 3 - -#define ARM_LPAE_TCR_RGN_NC0 -#define ARM_LPAE_TCR_RGN_WBWA 1 -#define ARM_LPAE_TCR_RGN_WT2 -#define ARM_LPAE_TCR_RGN_WB3 - #define ARM_LPAE_VTCR_SL0_MASK 0x3 #define ARM_LPAE_TCR_T0SZ_SHIFT0 @@ -124,14 +109,6 @@ #define ARM_LPAE_VTCR_PS_SHIFT 16 #define ARM_LPAE_VTCR_PS_MASK 0x7 -#define ARM_LPAE_TCR_PS_32_BIT 0x0ULL -#define ARM_LPAE_TCR_PS_36_BIT 0x1ULL -#define ARM_LPAE_TCR_PS_40_BIT 0x2ULL -#define ARM_LPAE_TCR_PS_42_BIT 0x3ULL -#define ARM_LPAE_TCR_PS_44_BIT 0x4ULL -#define ARM_LPAE_TCR_PS_48_BIT 0x5ULL -#define ARM_LPAE_TCR_PS_52_BIT 0x6ULL - #define ARM_LPAE_MAIR_ATTR_SHIFT(n)((n) << 3) #define ARM_LPAE_MAIR_ATTR_MASK0xff #define ARM_LPAE_MAIR_ATTR_DEVICE 0x04 diff --git a/MAINTAINERS b/MAINTAINERS index deaafb617361..4470769a9833 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1506,8 +1506,7 @@ L:linux-arm-ker...@lists.infradead.org (moderated for non-subscribers) S: Maintained F: Documentation/devicetree/bindings/iommu/arm,smmu* F: drivers/iommu/arm/ -F: drivers/iommu/io-pgtable-arm-v7s.c -F: drivers/iommu/io-pgtable-arm.c +F: drivers/iommu/io-pgtable-arm* ARM SUB-ARCHITECTURES L: linux-arm-ker...@lists.infradead.org (moderated for non-subscribers) -- 2.28.0 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH RESEND v9 06/13] arm64: cpufeature: Export symbol read_sanitised_ftr_reg()
The SMMUv3 driver would like to read the MMFR0 PARANGE field in order to share CPU page tables with devices. Allow the driver to be built as module by exporting the read_sanitized_ftr_reg() cpufeature symbol. Acked-by: Suzuki K Poulose Signed-off-by: Jean-Philippe Brucker --- arch/arm64/kernel/cpufeature.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index a389b999482e..c53666a47e1d 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -,6 +,7 @@ u64 read_sanitised_ftr_reg(u32 id) return 0; return regp->sys_val; } +EXPORT_SYMBOL_GPL(read_sanitised_ftr_reg); #define read_sysreg_case(r)\ case r: return read_sysreg_s(r) -- 2.28.0 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH RESEND v9 03/13] iommu/sva: Add PASID helpers
Let IOMMU drivers allocate a single PASID per mm. Store the mm in the
IOASID set to allow refcounting and searching mm by PASID, when handling
an I/O page fault.
Reviewed-by: Lu Baolu
Signed-off-by: Jean-Philippe Brucker
---
drivers/iommu/Kconfig | 5 +++
drivers/iommu/Makefile| 1 +
drivers/iommu/iommu-sva-lib.h | 15 +++
drivers/iommu/iommu-sva-lib.c | 85 +++
4 files changed, 106 insertions(+)
create mode 100644 drivers/iommu/iommu-sva-lib.h
create mode 100644 drivers/iommu/iommu-sva-lib.c
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index bef5d75e306b..fb1787377eb6 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -103,6 +103,11 @@ config IOMMU_DMA
select IRQ_MSI_IOMMU
select NEED_SG_DMA_LENGTH
+# Shared Virtual Addressing library
+config IOMMU_SVA_LIB
+ bool
+ select IOASID
+
config FSL_PAMU
bool "Freescale IOMMU support"
depends on PCI
diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile
index 11f1771104f3..61bd30cd8369 100644
--- a/drivers/iommu/Makefile
+++ b/drivers/iommu/Makefile
@@ -27,3 +27,4 @@ obj-$(CONFIG_FSL_PAMU) += fsl_pamu.o fsl_pamu_domain.o
obj-$(CONFIG_S390_IOMMU) += s390-iommu.o
obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o
obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o
+obj-$(CONFIG_IOMMU_SVA_LIB) += iommu-sva-lib.o
diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h
new file mode 100644
index ..b40990aef3fd
--- /dev/null
+++ b/drivers/iommu/iommu-sva-lib.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * SVA library for IOMMU drivers
+ */
+#ifndef _IOMMU_SVA_LIB_H
+#define _IOMMU_SVA_LIB_H
+
+#include
+#include
+
+int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max);
+void iommu_sva_free_pasid(struct mm_struct *mm);
+struct mm_struct *iommu_sva_find(ioasid_t pasid);
+
+#endif /* _IOMMU_SVA_LIB_H */
diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c
new file mode 100644
index ..db7e6c104d6b
--- /dev/null
+++ b/drivers/iommu/iommu-sva-lib.c
@@ -0,0 +1,85 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Helpers for IOMMU drivers implementing SVA
+ */
+#include
+#include
+
+#include "iommu-sva-lib.h"
+
+static DEFINE_MUTEX(iommu_sva_lock);
+static DECLARE_IOASID_SET(iommu_sva_pasid);
+
+/**
+ * iommu_sva_alloc_pasid - Allocate a PASID for the mm
+ * @mm: the mm
+ * @min: minimum PASID value (inclusive)
+ * @max: maximum PASID value (inclusive)
+ *
+ * Try to allocate a PASID for this mm, or take a reference to the existing one
+ * provided it fits within the [min, max] range. On success the PASID is
+ * available in mm->pasid, and must be released with iommu_sva_free_pasid().
+ *
+ * Returns 0 on success and < 0 on error.
+ */
+int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max)
+{
+ int ret = 0;
+ ioasid_t pasid;
+
+ if (min == INVALID_IOASID || max == INVALID_IOASID ||
+ min == 0 || max < min)
+ return -EINVAL;
+
+ mutex_lock(_sva_lock);
+ if (mm->pasid) {
+ if (mm->pasid >= min && mm->pasid <= max)
+ ioasid_get(mm->pasid);
+ else
+ ret = -EOVERFLOW;
+ } else {
+ pasid = ioasid_alloc(_sva_pasid, min, max, mm);
+ if (pasid == INVALID_IOASID)
+ ret = -ENOMEM;
+ else
+ mm->pasid = pasid;
+ }
+ mutex_unlock(_sva_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_sva_alloc_pasid);
+
+/**
+ * iommu_sva_free_pasid - Release the mm's PASID
+ * @mm: the mm.
+ *
+ * Drop one reference to a PASID allocated with iommu_sva_alloc_pasid()
+ */
+void iommu_sva_free_pasid(struct mm_struct *mm)
+{
+ mutex_lock(_sva_lock);
+ if (ioasid_put(mm->pasid))
+ mm->pasid = 0;
+ mutex_unlock(_sva_lock);
+}
+EXPORT_SYMBOL_GPL(iommu_sva_free_pasid);
+
+/* ioasid wants a void * argument */
+static bool __mmget_not_zero(void *mm)
+{
+ return mmget_not_zero(mm);
+}
+
+/**
+ * iommu_sva_find() - Find mm associated to the given PASID
+ * @pasid: Process Address Space ID assigned to the mm
+ *
+ * On success a reference to the mm is taken, and must be released with
mmput().
+ *
+ * Returns the mm corresponding to this PASID, or an error if not found.
+ */
+struct mm_struct *iommu_sva_find(ioasid_t pasid)
+{
+ return ioasid_find(_sva_pasid, pasid, __mmget_not_zero);
+}
+EXPORT_SYMBOL_GPL(iommu_sva_find);
--
2.28.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH RESEND v9 04/13] arm64: mm: Pin down ASIDs for sharing mm with devices
To enable address space sharing with the IOMMU, introduce
arm64_mm_context_get() and arm64_mm_context_put(), that pin down a
context and ensure that it will keep its ASID after a rollover. Export
the symbols to let the modular SMMUv3 driver use them.
Pinning is necessary because a device constantly needs a valid ASID,
unlike tasks that only require one when running. Without pinning, we would
need to notify the IOMMU when we're about to use a new ASID for a task,
and it would get complicated when a new task is assigned a shared ASID.
Consider the following scenario with no ASID pinned:
1. Task t1 is running on CPUx with shared ASID (gen=1, asid=1)
2. Task t2 is scheduled on CPUx, gets ASID (1, 2)
3. Task tn is scheduled on CPUy, a rollover occurs, tn gets ASID (2, 1)
We would now have to immediately generate a new ASID for t1, notify
the IOMMU, and finally enable task tn. We are holding the lock during
all that time, since we can't afford having another CPU trigger a
rollover. The IOMMU issues invalidation commands that can take tens of
milliseconds.
It gets needlessly complicated. All we wanted to do was schedule task tn,
that has no business with the IOMMU. By letting the IOMMU pin tasks when
needed, we avoid stalling the slow path, and let the pinning fail when
we're out of shareable ASIDs.
After a rollover, the allocator expects at least one ASID to be available
in addition to the reserved ones (one per CPU). So (NR_ASIDS - NR_CPUS -
1) is the maximum number of ASIDs that can be shared with the IOMMU.
Signed-off-by: Jean-Philippe Brucker
---
v9:
* Make mm->context.pinned a refcount_t.
* Prepend exported symbols with arm64_.
* Initialize pinned_asid_map with kernel ASID bits if necessary.
* Allow pinned_asid_map to be NULL. It could also depend on
CONFIG_ARM_SMMU_V3_SVA since it adds an overhead on rollover (memcpy()
asid_map instead of memset()), but that can be changed later.
* Only set the USER_ASID_BIT if kpti is enabled (previously it was set
if CONFIG_UNMAP_KERNEL_AT_EL0).
---
arch/arm64/include/asm/mmu.h | 3 +
arch/arm64/include/asm/mmu_context.h | 11 ++-
arch/arm64/mm/context.c | 105 +--
3 files changed, 112 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
index a7a5ecaa2e83..0fda85b2cc1b 100644
--- a/arch/arm64/include/asm/mmu.h
+++ b/arch/arm64/include/asm/mmu.h
@@ -17,11 +17,14 @@
#ifndef __ASSEMBLY__
+#include
+
typedef struct {
atomic64_t id;
#ifdef CONFIG_COMPAT
void*sigpage;
#endif
+ refcount_t pinned;
void*vdso;
unsigned long flags;
} mm_context_t;
diff --git a/arch/arm64/include/asm/mmu_context.h
b/arch/arm64/include/asm/mmu_context.h
index f2d7537d6f83..0672236e1aea 100644
--- a/arch/arm64/include/asm/mmu_context.h
+++ b/arch/arm64/include/asm/mmu_context.h
@@ -177,7 +177,13 @@ static inline void cpu_replace_ttbr1(pgd_t *pgdp)
#define destroy_context(mm)do { } while(0)
void check_and_switch_context(struct mm_struct *mm);
-#define init_new_context(tsk,mm) ({ atomic64_set(&(mm)->context.id, 0);
0; })
+static inline int
+init_new_context(struct task_struct *tsk, struct mm_struct *mm)
+{
+ atomic64_set(>context.id, 0);
+ refcount_set(>context.pinned, 0);
+ return 0;
+}
#ifdef CONFIG_ARM64_SW_TTBR0_PAN
static inline void update_saved_ttbr0(struct task_struct *tsk,
@@ -248,6 +254,9 @@ switch_mm(struct mm_struct *prev, struct mm_struct *next,
void verify_cpu_asid_bits(void);
void post_ttbr_update_workaround(void);
+unsigned long arm64_mm_context_get(struct mm_struct *mm);
+void arm64_mm_context_put(struct mm_struct *mm);
+
#endif /* !__ASSEMBLY__ */
#endif /* !__ASM_MMU_CONTEXT_H */
diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c
index a206655a39a5..373d87370899 100644
--- a/arch/arm64/mm/context.c
+++ b/arch/arm64/mm/context.c
@@ -27,6 +27,10 @@ static DEFINE_PER_CPU(atomic64_t, active_asids);
static DEFINE_PER_CPU(u64, reserved_asids);
static cpumask_t tlb_flush_pending;
+static unsigned long max_pinned_asids;
+static unsigned long nr_pinned_asids;
+static unsigned long *pinned_asid_map;
+
#define ASID_MASK (~GENMASK(asid_bits - 1, 0))
#define ASID_FIRST_VERSION (1UL << asid_bits)
@@ -72,7 +76,7 @@ void verify_cpu_asid_bits(void)
}
}
-static void set_kpti_asid_bits(void)
+static void set_kpti_asid_bits(unsigned long *map)
{
unsigned int len = BITS_TO_LONGS(NUM_USER_ASIDS) * sizeof(unsigned
long);
/*
@@ -81,13 +85,15 @@ static void set_kpti_asid_bits(void)
* is set, then the ASID will map only userspace. Thus
* mark even as reserved for kernel.
*/
- memset(asid_map, 0xaa, len);
+ memset(map, 0xaa, len);
}
static void set_reserved_asid_bits(void)
{
- if (arm64_kernel_unmapped_at_el0())
-
[PATCH RESEND v9 00/13] iommu: Shared Virtual Addressing for SMMUv3 (PT sharing part)
This is version 9 of the page table sharing support for SMMUv3, rebased onto v5.9-rc1. v9 resend: * Rebased onto v5.9-rc1. Moved to drivers/iommu/arm/arm-smmu-v3/ as a result. v9: https://lore.kernel.org/linux-iommu/20200723145724.3014766-1-jean-phili...@linaro.org/ * Moved most of the SVA code to arm-smmu-v3-sva.c. This required moving struct definitions and macros to arm-smmu-v3.h (patch 7), hence the new 700 insertions/deletions in the diffstat. * Updated patches 4 and 8 following review. * Fixed a bug when replacing a private ASID. v8: https://lore.kernel.org/linux-iommu/20200618155125.1548969-1-jean-phili...@linaro.org/ * Split SVA series into three parts: page table sharing, I/O page faults, and additional features (DVM, VHE and HTTU). Fenghua Yu (1): mm: Define pasid in mm Jean-Philippe Brucker (12): iommu/ioasid: Add ioasid references iommu/sva: Add PASID helpers arm64: mm: Pin down ASIDs for sharing mm with devices iommu/io-pgtable-arm: Move some definitions to a header arm64: cpufeature: Export symbol read_sanitised_ftr_reg() iommu/arm-smmu-v3: Move definitions to a header iommu/arm-smmu-v3: Share process page tables iommu/arm-smmu-v3: Seize private ASID iommu/arm-smmu-v3: Check for SVA features iommu/arm-smmu-v3: Add SVA device feature iommu/arm-smmu-v3: Implement iommu_sva_bind/unbind() iommu/arm-smmu-v3: Hook up ATC invalidation to mm ops drivers/iommu/Kconfig | 17 + drivers/iommu/Makefile| 1 + drivers/iommu/arm/arm-smmu-v3/Makefile| 5 +- arch/arm64/include/asm/mmu.h | 3 + arch/arm64/include/asm/mmu_context.h | 11 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 752 +++ drivers/iommu/io-pgtable-arm.h| 30 + drivers/iommu/iommu-sva-lib.h | 15 + include/linux/ioasid.h| 10 +- include/linux/mm_types.h | 4 + arch/arm64/kernel/cpufeature.c| 1 + arch/arm64/mm/context.c | 105 ++- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 487 ++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 860 -- drivers/iommu/intel/iommu.c | 4 +- drivers/iommu/intel/svm.c | 6 +- drivers/iommu/io-pgtable-arm.c| 27 +- drivers/iommu/ioasid.c| 38 +- drivers/iommu/iommu-sva-lib.c | 85 ++ MAINTAINERS | 3 +- 20 files changed, 1729 insertions(+), 735 deletions(-) create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h create mode 100644 drivers/iommu/io-pgtable-arm.h create mode 100644 drivers/iommu/iommu-sva-lib.h create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c create mode 100644 drivers/iommu/iommu-sva-lib.c -- 2.28.0 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH RESEND v9 02/13] iommu/ioasid: Add ioasid references
Let IOASID users take references to existing ioasids with ioasid_get().
ioasid_put() drops a reference and only frees the ioasid when its
reference number is zero. It returns true if the ioasid was freed.
For drivers that don't call ioasid_get(), ioasid_put() is the same as
ioasid_free().
Reviewed-by: Lu Baolu
Signed-off-by: Jean-Philippe Brucker
---
include/linux/ioasid.h | 10 --
drivers/iommu/intel/iommu.c | 4 ++--
drivers/iommu/intel/svm.c | 6 +++---
drivers/iommu/ioasid.c | 38 +
4 files changed, 47 insertions(+), 11 deletions(-)
diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h
index 6f000d7a0ddc..e9dacd4b9f6b 100644
--- a/include/linux/ioasid.h
+++ b/include/linux/ioasid.h
@@ -34,7 +34,8 @@ struct ioasid_allocator_ops {
#if IS_ENABLED(CONFIG_IOASID)
ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max,
void *private);
-void ioasid_free(ioasid_t ioasid);
+void ioasid_get(ioasid_t ioasid);
+bool ioasid_put(ioasid_t ioasid);
void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
bool (*getter)(void *));
int ioasid_register_allocator(struct ioasid_allocator_ops *allocator);
@@ -48,10 +49,15 @@ static inline ioasid_t ioasid_alloc(struct ioasid_set *set,
ioasid_t min,
return INVALID_IOASID;
}
-static inline void ioasid_free(ioasid_t ioasid)
+static inline void ioasid_get(ioasid_t ioasid)
{
}
+static inline bool ioasid_put(ioasid_t ioasid)
+{
+ return false;
+}
+
static inline void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
bool (*getter)(void *))
{
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index e9864e52b0e9..152fc2dc17e0 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -5149,7 +5149,7 @@ static void auxiliary_unlink_device(struct dmar_domain
*domain,
domain->auxd_refcnt--;
if (!domain->auxd_refcnt && domain->default_pasid > 0)
- ioasid_free(domain->default_pasid);
+ ioasid_put(domain->default_pasid);
}
static int aux_domain_add_dev(struct dmar_domain *domain,
@@ -5210,7 +5210,7 @@ static int aux_domain_add_dev(struct dmar_domain *domain,
spin_unlock(>lock);
spin_unlock_irqrestore(_domain_lock, flags);
if (!domain->auxd_refcnt && domain->default_pasid > 0)
- ioasid_free(domain->default_pasid);
+ ioasid_put(domain->default_pasid);
return ret;
}
diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c
index 95c3164a2302..50897a2bd1da 100644
--- a/drivers/iommu/intel/svm.c
+++ b/drivers/iommu/intel/svm.c
@@ -565,7 +565,7 @@ intel_svm_bind_mm(struct device *dev, int flags, struct
svm_dev_ops *ops,
if (mm) {
ret = mmu_notifier_register(>notifier, mm);
if (ret) {
- ioasid_free(svm->pasid);
+ ioasid_put(svm->pasid);
kfree(svm);
kfree(sdev);
goto out;
@@ -583,7 +583,7 @@ intel_svm_bind_mm(struct device *dev, int flags, struct
svm_dev_ops *ops,
if (ret) {
if (mm)
mmu_notifier_unregister(>notifier, mm);
- ioasid_free(svm->pasid);
+ ioasid_put(svm->pasid);
kfree(svm);
kfree(sdev);
goto out;
@@ -652,7 +652,7 @@ static int intel_svm_unbind_mm(struct device *dev, int
pasid)
kfree_rcu(sdev, rcu);
if (list_empty(>devs)) {
- ioasid_free(svm->pasid);
+ ioasid_put(svm->pasid);
if (svm->mm)
mmu_notifier_unregister(>notifier,
svm->mm);
list_del(>list);
diff --git a/drivers/iommu/ioasid.c b/drivers/iommu/ioasid.c
index 0f8dd377aada..50ee27bbd04e 100644
--- a/drivers/iommu/ioasid.c
+++ b/drivers/iommu/ioasid.c
@@ -2,7 +2,7 @@
/*
* I/O Address Space ID allocator. There is one global IOASID space, split into
* subsets. Users create a subset with DECLARE_IOASID_SET, then allocate and
- * free IOASIDs with ioasid_alloc and ioasid_free.
+ * free IOASIDs with ioasid_alloc and ioasid_put.
*/
#include
#include
@@ -15,6 +15,7 @@ struct ioasid_data {
struct ioasid_set *set;
void *private;
struct rcu_head rcu;
+ refcount_t refs;
};
/*
@@ -314,6 +315,7 @@ ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min,
ioasid_t max,
data->set = set;
data->private = private;
+ refcount_set(>refs, 1);
/*
* Custom allocator needs allocator data to perform
[PATCH RESEND v9 01/13] mm: Define pasid in mm
From: Fenghua Yu
PASID is shared by all threads in a process. So the logical place to keep
track of it is in the "mm". Both ARM and X86 need to use the PASID in the
"mm".
Suggested-by: Christoph Hellwig
Signed-off-by: Fenghua Yu
Reviewed-by: Tony Luck
---
https://lore.kernel.org/linux-iommu/1594684087-61184-8-git-send-email-fenghua...@intel.com/
---
---
include/linux/mm_types.h | 4
1 file changed, 4 insertions(+)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 496c3ff97cce..1ff0615ef19f 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -542,6 +542,10 @@ struct mm_struct {
atomic_long_t hugetlb_usage;
#endif
struct work_struct async_put_work;
+
+#ifdef CONFIG_IOMMU_SUPPORT
+ u32 pasid;
+#endif
} __randomize_layout;
/*
--
2.28.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Freedreno] [PATCH 19/19] drm/msm: show process names in gem_describe
On Thu, Aug 13, 2020 at 07:41:14PM -0700, Rob Clark wrote:
> From: Rob Clark
>
> In $debugfs/gem we already show any vma(s) associated with an object.
> Also show process names if the vma's address space is a per-process
> address space.
Reviewed-by: Jordan Crouse
> Signed-off-by: Rob Clark
> ---
> drivers/gpu/drm/msm/msm_drv.c | 2 +-
> drivers/gpu/drm/msm/msm_gem.c | 25 +
> drivers/gpu/drm/msm/msm_gem.h | 5 +
> drivers/gpu/drm/msm/msm_gem_vma.c | 1 +
> drivers/gpu/drm/msm/msm_gpu.c | 8 +---
> drivers/gpu/drm/msm/msm_gpu.h | 2 +-
> 6 files changed, 34 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
> index 8e70d220bba8..8d5c4f98c332 100644
> --- a/drivers/gpu/drm/msm/msm_drv.c
> +++ b/drivers/gpu/drm/msm/msm_drv.c
> @@ -597,7 +597,7 @@ static int context_init(struct drm_device *dev, struct
> drm_file *file)
> kref_init(>ref);
> msm_submitqueue_init(dev, ctx);
>
> - ctx->aspace = msm_gpu_create_private_address_space(priv->gpu);
> + ctx->aspace = msm_gpu_create_private_address_space(priv->gpu, current);
> file->driver_priv = ctx;
>
> return 0;
> diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
> index 3cb7aeb93fd3..76a6c5271e57 100644
> --- a/drivers/gpu/drm/msm/msm_gem.c
> +++ b/drivers/gpu/drm/msm/msm_gem.c
> @@ -842,11 +842,28 @@ void msm_gem_describe(struct drm_gem_object *obj,
> struct seq_file *m)
>
> seq_puts(m, " vmas:");
>
> - list_for_each_entry(vma, _obj->vmas, list)
> - seq_printf(m, " [%s: %08llx,%s,inuse=%d]",
> - vma->aspace != NULL ? vma->aspace->name : NULL,
> - vma->iova, vma->mapped ? "mapped" : "unmapped",
> + list_for_each_entry(vma, _obj->vmas, list) {
> + const char *name, *comm;
> + if (vma->aspace) {
> + struct msm_gem_address_space *aspace =
> vma->aspace;
> + struct task_struct *task =
> + get_pid_task(aspace->pid, PIDTYPE_PID);
> + if (task) {
> + comm = kstrdup(task->comm, GFP_KERNEL);
> + } else {
> + comm = NULL;
> + }
> + name = aspace->name;
> + } else {
> + name = comm = NULL;
> + }
> + seq_printf(m, " [%s%s%s: aspace=%p,
> %08llx,%s,inuse=%d]",
> + name, comm ? ":" : "", comm ? comm : "",
> + vma->aspace, vma->iova,
> + vma->mapped ? "mapped" : "unmapped",
> vma->inuse);
> + kfree(comm);
> + }
>
> seq_puts(m, "\n");
> }
> diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
> index 9c573c4269cb..7b1c7a5f8eef 100644
> --- a/drivers/gpu/drm/msm/msm_gem.h
> +++ b/drivers/gpu/drm/msm/msm_gem.h
> @@ -24,6 +24,11 @@ struct msm_gem_address_space {
> spinlock_t lock; /* Protects drm_mm node allocation/removal */
> struct msm_mmu *mmu;
> struct kref kref;
> +
> + /* For address spaces associated with a specific process, this
> + * will be non-NULL:
> + */
> + struct pid *pid;
> };
>
> struct msm_gem_vma {
> diff --git a/drivers/gpu/drm/msm/msm_gem_vma.c
> b/drivers/gpu/drm/msm/msm_gem_vma.c
> index 29cc1305cf37..80a8a266d68f 100644
> --- a/drivers/gpu/drm/msm/msm_gem_vma.c
> +++ b/drivers/gpu/drm/msm/msm_gem_vma.c
> @@ -17,6 +17,7 @@ msm_gem_address_space_destroy(struct kref *kref)
> drm_mm_takedown(>mm);
> if (aspace->mmu)
> aspace->mmu->funcs->destroy(aspace->mmu);
> + put_pid(aspace->pid);
> kfree(aspace);
> }
>
> diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
> index 951850804d77..ac8961187a73 100644
> --- a/drivers/gpu/drm/msm/msm_gpu.c
> +++ b/drivers/gpu/drm/msm/msm_gpu.c
> @@ -825,10 +825,9 @@ static int get_clocks(struct platform_device *pdev,
> struct msm_gpu *gpu)
>
> /* Return a new address space for a msm_drm_private instance */
> struct msm_gem_address_space *
> -msm_gpu_create_private_address_space(struct msm_gpu *gpu)
> +msm_gpu_create_private_address_space(struct msm_gpu *gpu, struct task_struct
> *task)
> {
> struct msm_gem_address_space *aspace = NULL;
> -
> if (!gpu)
> return NULL;
>
> @@ -836,8 +835,11 @@ msm_gpu_create_private_address_space(struct msm_gpu *gpu)
>* If the target doesn't support private address spaces then return
>* the global one
>*/
> - if
Re: [PATCH 07/19] drm/msm: set adreno_smmu as gpu's drvdata
On Thu, Aug 13, 2020 at 07:41:02PM -0700, Rob Clark wrote:
> From: Rob Clark
>
> This will be populated by adreno-smmu, to provide a way for coordinating
> enabling/disabling TTBR0 translation.
>
Reviewed-by: Jordan Crouse
> Signed-off-by: Rob Clark
> ---
> drivers/gpu/drm/msm/adreno/adreno_device.c | 2 --
> drivers/gpu/drm/msm/msm_gpu.c | 2 +-
> drivers/gpu/drm/msm/msm_gpu.h | 6 +-
> 3 files changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c
> b/drivers/gpu/drm/msm/adreno/adreno_device.c
> index 26664e1b30c0..58e03b20e1c7 100644
> --- a/drivers/gpu/drm/msm/adreno/adreno_device.c
> +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c
> @@ -417,8 +417,6 @@ static int adreno_bind(struct device *dev, struct device
> *master, void *data)
> return PTR_ERR(gpu);
> }
>
> - dev_set_drvdata(dev, gpu);
> -
> return 0;
> }
>
> diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
> index 6aa9e04e52e7..806eb0957280 100644
> --- a/drivers/gpu/drm/msm/msm_gpu.c
> +++ b/drivers/gpu/drm/msm/msm_gpu.c
> @@ -892,7 +892,7 @@ int msm_gpu_init(struct drm_device *drm, struct
> platform_device *pdev,
> gpu->gpu_cx = NULL;
>
> gpu->pdev = pdev;
> - platform_set_drvdata(pdev, gpu);
> + platform_set_drvdata(pdev, >adreno_smmu);
>
> msm_devfreq_init(gpu);
>
> diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
> index 8bda7beaed4b..f91b141add75 100644
> --- a/drivers/gpu/drm/msm/msm_gpu.h
> +++ b/drivers/gpu/drm/msm/msm_gpu.h
> @@ -7,6 +7,7 @@
> #ifndef __MSM_GPU_H__
> #define __MSM_GPU_H__
>
> +#include
> #include
> #include
> #include
> @@ -73,6 +74,8 @@ struct msm_gpu {
> struct platform_device *pdev;
> const struct msm_gpu_funcs *funcs;
>
> + struct adreno_smmu_priv adreno_smmu;
> +
> /* performance counters (hw & sw): */
> spinlock_t perf_lock;
> bool perfcntr_active;
> @@ -143,7 +146,8 @@ struct msm_gpu {
>
> static inline struct msm_gpu *dev_to_gpu(struct device *dev)
> {
> - return dev_get_drvdata(dev);
> + struct adreno_smmu_priv *adreno_smmu = dev_get_drvdata(dev);
> + return container_of(adreno_smmu, struct msm_gpu, adreno_smmu);
> }
>
> /* It turns out that all targets use the same ringbuffer size */
> --
> 2.26.2
>
--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Freedreno] [PATCH 05/19] iommu: add private interface for adreno-smmu
On Thu, Aug 13, 2020 at 07:41:00PM -0700, Rob Clark wrote:
> From: Rob Clark
>
> This interface will be used for drm/msm to coordinate with the
> qcom_adreno_smmu_impl to enable/disable TTBR0 translation.
>
> Once TTBR0 translation is enabled, the GPU's CP (Command Processor)
> will directly switch TTBR0 pgtables (and do the necessary TLB inv)
> synchronized to the GPU's operation. But help from the SMMU driver
> is needed to initially bootstrap TTBR0 translation, which cannot be
> done from the GPU.
>
> Since this is a very special case, a private interface is used to
> avoid adding highly driver specific things to the public iommu
> interface.
>
Reviewed-by: Jordan Crouse
> Signed-off-by: Rob Clark
> ---
> include/linux/adreno-smmu-priv.h | 36
> 1 file changed, 36 insertions(+)
> create mode 100644 include/linux/adreno-smmu-priv.h
>
> diff --git a/include/linux/adreno-smmu-priv.h
> b/include/linux/adreno-smmu-priv.h
> new file mode 100644
> index ..a889f28afb42
> --- /dev/null
> +++ b/include/linux/adreno-smmu-priv.h
> @@ -0,0 +1,36 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2020 Google, Inc
> + */
> +
> +#ifndef __ADRENO_SMMU_PRIV_H
> +#define __ADRENO_SMMU_PRIV_H
> +
> +#include
> +
> +/**
> + * struct adreno_smmu_priv - private interface between adreno-smmu and GPU
> + *
> + * @cookie:An opque token provided by adreno-smmu and passed
> + * back into the callbacks
> + * @get_ttbr1_cfg: Get the TTBR1 config for the GPUs context-bank
> + * @set_ttbr0_cfg: Set the TTBR0 config for the GPUs context bank. A
> + * NULL config disables TTBR0 translation, otherwise
> + * TTBR0 translation is enabled with the specified cfg
> + *
> + * The GPU driver (drm/msm) and adreno-smmu work together for controlling
> + * the GPU's SMMU instance. This is by necessity, as the GPU is directly
> + * updating the SMMU for context switches, while on the other hand we do
> + * not want to duplicate all of the initial setup logic from arm-smmu.
> + *
> + * This private interface is used for the two drivers to coordinate. The
> + * cookie and callback functions are populated when the GPU driver attaches
> + * it's domain.
> + */
> +struct adreno_smmu_priv {
> +const void *cookie;
> +const struct io_pgtable_cfg *(*get_ttbr1_cfg)(const void *cookie);
> +int (*set_ttbr0_cfg)(const void *cookie, const struct io_pgtable_cfg
> *cfg);
> +};
> +
> +#endif /* __ADRENO_SMMU_PRIV_H */
> \ No newline at end of file
> --
> 2.26.2
>
> ___
> Freedreno mailing list
> freedr...@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/freedreno
--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Freedreno] [PATCH 06/19] drm/msm/gpu: add dev_to_gpu() helper
On Thu, Aug 13, 2020 at 07:41:01PM -0700, Rob Clark wrote:
> From: Rob Clark
>
> In a later patch, the drvdata will not directly be 'struct msm_gpu *',
> so add a helper to reduce the churn.
>
Reviewed-by: Jordan Crouse
> Signed-off-by: Rob Clark
> ---
> drivers/gpu/drm/msm/adreno/adreno_device.c | 10 --
> drivers/gpu/drm/msm/msm_gpu.c | 6 +++---
> drivers/gpu/drm/msm/msm_gpu.h | 5 +
> 3 files changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c
> b/drivers/gpu/drm/msm/adreno/adreno_device.c
> index 9eeb46bf2a5d..26664e1b30c0 100644
> --- a/drivers/gpu/drm/msm/adreno/adreno_device.c
> +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c
> @@ -282,7 +282,7 @@ struct msm_gpu *adreno_load_gpu(struct drm_device *dev)
> int ret;
>
> if (pdev)
> - gpu = platform_get_drvdata(pdev);
> + gpu = dev_to_gpu(>dev);
>
> if (!gpu) {
> dev_err_once(dev->dev, "no GPU device was found\n");
> @@ -425,7 +425,7 @@ static int adreno_bind(struct device *dev, struct device
> *master, void *data)
> static void adreno_unbind(struct device *dev, struct device *master,
> void *data)
> {
> - struct msm_gpu *gpu = dev_get_drvdata(dev);
> + struct msm_gpu *gpu = dev_to_gpu(dev);
>
> pm_runtime_force_suspend(dev);
> gpu->funcs->destroy(gpu);
> @@ -490,16 +490,14 @@ static const struct of_device_id dt_match[] = {
> #ifdef CONFIG_PM
> static int adreno_resume(struct device *dev)
> {
> - struct platform_device *pdev = to_platform_device(dev);
> - struct msm_gpu *gpu = platform_get_drvdata(pdev);
> + struct msm_gpu *gpu = dev_to_gpu(dev);
>
> return gpu->funcs->pm_resume(gpu);
> }
>
> static int adreno_suspend(struct device *dev)
> {
> - struct platform_device *pdev = to_platform_device(dev);
> - struct msm_gpu *gpu = platform_get_drvdata(pdev);
> + struct msm_gpu *gpu = dev_to_gpu(dev);
>
> return gpu->funcs->pm_suspend(gpu);
> }
> diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
> index d5645472b25d..6aa9e04e52e7 100644
> --- a/drivers/gpu/drm/msm/msm_gpu.c
> +++ b/drivers/gpu/drm/msm/msm_gpu.c
> @@ -24,7 +24,7 @@
> static int msm_devfreq_target(struct device *dev, unsigned long *freq,
> u32 flags)
> {
> - struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
> + struct msm_gpu *gpu = dev_to_gpu(dev);
> struct dev_pm_opp *opp;
>
> opp = devfreq_recommended_opp(dev, freq, flags);
> @@ -45,7 +45,7 @@ static int msm_devfreq_target(struct device *dev, unsigned
> long *freq,
> static int msm_devfreq_get_dev_status(struct device *dev,
> struct devfreq_dev_status *status)
> {
> - struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
> + struct msm_gpu *gpu = dev_to_gpu(dev);
> ktime_t time;
>
> if (gpu->funcs->gpu_get_freq)
> @@ -64,7 +64,7 @@ static int msm_devfreq_get_dev_status(struct device *dev,
>
> static int msm_devfreq_get_cur_freq(struct device *dev, unsigned long *freq)
> {
> - struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev));
> + struct msm_gpu *gpu = dev_to_gpu(dev);
>
> if (gpu->funcs->gpu_get_freq)
> *freq = gpu->funcs->gpu_get_freq(gpu);
> diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
> index 0db117a7339b..8bda7beaed4b 100644
> --- a/drivers/gpu/drm/msm/msm_gpu.h
> +++ b/drivers/gpu/drm/msm/msm_gpu.h
> @@ -141,6 +141,11 @@ struct msm_gpu {
> struct msm_gpu_state *crashstate;
> };
>
> +static inline struct msm_gpu *dev_to_gpu(struct device *dev)
> +{
> + return dev_get_drvdata(dev);
> +}
> +
> /* It turns out that all targets use the same ringbuffer size */
> #define MSM_GPU_RINGBUFFER_SZ SZ_32K
> #define MSM_GPU_RINGBUFFER_BLKSIZE 32
> --
> 2.26.2
>
> ___
> Freedreno mailing list
> freedr...@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/freedreno
--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 01/19] drm/msm: remove dangling submitqueue references
On Thu, Aug 13, 2020 at 07:40:56PM -0700, Rob Clark wrote:
> From: Rob Clark
>
> Currently it doesn't matter, since we free the ctx immediately. But
> when we start refcnt'ing the ctx, we don't want old dangling list
> entries to hang around.
Reviewed-by: Jordan Crouse
> Signed-off-by: Rob Clark
> ---
> drivers/gpu/drm/msm/msm_submitqueue.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c
> b/drivers/gpu/drm/msm/msm_submitqueue.c
> index a1d94be7883a..90c9d84e6155 100644
> --- a/drivers/gpu/drm/msm/msm_submitqueue.c
> +++ b/drivers/gpu/drm/msm/msm_submitqueue.c
> @@ -49,8 +49,10 @@ void msm_submitqueue_close(struct msm_file_private *ctx)
>* No lock needed in close and there won't
>* be any more user ioctls coming our way
>*/
> - list_for_each_entry_safe(entry, tmp, >submitqueues, node)
> + list_for_each_entry_safe(entry, tmp, >submitqueues, node) {
> + list_del(>node);
> msm_submitqueue_put(entry);
> + }
> }
>
> int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private
> *ctx,
> --
> 2.26.2
>
> ___
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu
--
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 00/19] iommu/arm-smmu + drm/msm: per-process GPU pgtables
On Thu, Aug 13, 2020 at 07:40:55PM -0700, Rob Clark wrote: > From: Rob Clark > > NOTE: Since Jordan was out today, and I wanted to keep things moving on > this, I took the liberty of respinning his series (originally > titled "iommu/arm-smmu: Add Adreno SMMU specific implementation") > with updates based on Will's review comments, and some fixes and > extra bits that I found in stress testing the series. Original > commit msg and updated version history below. > > In general I like the private interface between adreno-smmu and > the GPU driver. It should make for a more straightforward way > to extend things to optimize TLB invalidation in the future, for > example, rather than shoe-horning everything thru domain attrs. > And it lets us describe the get_ttbr1_cfg/set_ttrb0_cfg dance > more clearly. Although it is going to make landing this via > iommu vs drm tree a bit more difficult. Maybe there are some > arm-smmu parts of this series that could be pulled out to make > it not conflicty to land the private interface and adreno-smmu > bits via the drm tree? (But I'm jumping a bit ahead here. Just > wanted to point out that issue.) > > The complete series can be found at: > https://gitlab.freedesktop.org/drm/msm/-/commits/msm-next-pgtables > > This series adds an Adreno SMMU implementation to arm-smmu to allow GPU > hardware > pagetable switching. > > The Adreno GPU has built in capabilities to switch the TTBR0 pagetable during > runtime to allow each individual instance or application to have its own > pagetable. In order to take advantage of the HW capabilities there are > certain > requirements needed of the SMMU hardware. > > This series adds support for an Adreno specific arm-smmu implementation. The > new > implementation 1) ensures that the GPU domain is always assigned context bank > 0, > 2) enables split pagetable support (TTBR1) so that the instance specific > pagetable can be swapped while the global memory remains in place and 3) > shares > the current pagetable configuration with the GPU driver to allow it to create > its own io-pgtable instances. > > The series then adds the drm/msm code to enable these features. For targets > that > support it allocate new pagetables using the io-pgtable configuration shared > by > the arm-smmu driver and swap them in during runtime. > > This version of the series merges the previous patchset(s) [1] and [2] > with the following improvements: > > v13: (Respin by Rob) > - Switch to a private interface between adreno-smmu and GPU driver, > dropping the custom domain attr (Will Deacon) > - Rework the SCTLR.HUPCF patch to add new fields in smmu_domain->cfg > rather than adding new impl hook (Will Deacon) > - Drop for_each_cfg_sme() in favor of plain for() loop (Will Deacon) > - Fix context refcnt'ing issue which was causing problems with GPU > crash recover stress testing. > - Spiff up $debugfs/gem to show process information associated with > VMAs I'll add the tags to Rob's code but in general I ack all these changes. I also like the private interface - it gives us the most flexibility without either changing the IOMMU API or giving up entirely and making an internal SMMU implementation inside drm/msm. Jordan -- The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 16/19] drm/msm/a6xx: Add support for per-instance pagetables
On Mon, Aug 17, 2020 at 09:10:46PM +0530, Akhil P Oommen wrote:
> On 8/14/2020 8:11 AM, Rob Clark wrote:
> >From: Jordan Crouse
> >
> >Add support for using per-instance pagetables if all the dependencies are
> >available.
> >
> >Signed-off-by: Jordan Crouse
> >Signed-off-by: Rob Clark
> >---
> > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 70 +++
> > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 +
> > drivers/gpu/drm/msm/msm_ringbuffer.h | 1 +
> > 3 files changed, 72 insertions(+)
> >
> >diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> >b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> >index 5eabb0109577..9653ac9b3cb8 100644
> >--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> >+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> >@@ -81,6 +81,56 @@ static void get_stats_counter(struct msm_ringbuffer
> >*ring, u32 counter,
> > OUT_RING(ring, upper_32_bits(iova));
> > }
> >+static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,
> >+struct msm_ringbuffer *ring, struct msm_file_private *ctx)
> >+{
> >+phys_addr_t ttbr;
> >+u32 asid;
> >+u64 memptr = rbmemptr(ring, ttbr0);
> >+
> >+if (ctx == a6xx_gpu->cur_ctx)
> >+return;
> >+
> >+if (msm_iommu_pagetable_params(ctx->aspace->mmu, , ))
> >+return;
> >+
> >+/* Execute the table update */
> >+OUT_PKT7(ring, CP_SMMU_TABLE_UPDATE, 4);
> >+OUT_RING(ring, CP_SMMU_TABLE_UPDATE_0_TTBR0_LO(lower_32_bits(ttbr)));
> >+
> >+/*
> >+ * For now ignore the asid since the smmu driver uses a TLBIASID to
> >+ * flush the TLB when we use iommu_flush_tlb_all() and the smmu driver
> >+ * isn't aware that the asid changed. Instead, keep the default asid
> >+ * (0, same as the context bank) to make sure the TLB is properly
> >+ * flushed.
> >+ */
> >+OUT_RING(ring,
> >+CP_SMMU_TABLE_UPDATE_1_TTBR0_HI(upper_32_bits(ttbr)) |
> >+CP_SMMU_TABLE_UPDATE_1_ASID(0));
> >+OUT_RING(ring, CP_SMMU_TABLE_UPDATE_2_CONTEXTIDR(0));
> >+OUT_RING(ring, CP_SMMU_TABLE_UPDATE_3_CONTEXTBANK(0));
> >+
> >+/*
> >+ * Write the new TTBR0 to the memstore. This is good for debugging.
> >+ */
> >+OUT_PKT7(ring, CP_MEM_WRITE, 4);
> >+OUT_RING(ring, CP_MEM_WRITE_0_ADDR_LO(lower_32_bits(memptr)));
> >+OUT_RING(ring, CP_MEM_WRITE_1_ADDR_HI(upper_32_bits(memptr)));
> >+OUT_RING(ring, lower_32_bits(ttbr));
> >+OUT_RING(ring, (0 << 16) | upper_32_bits(ttbr));
> why (0 << 16) is required here?
Because that is the ASID we are using and we would want the debug TTBR0 to match
the hardware as closely as possible.
> >+
> >+/*
> >+ * And finally, trigger a uche flush to be sure there isn't anything
> >+ * lingering in that part of the GPU
> >+ */
> >+
> >+OUT_PKT7(ring, CP_EVENT_WRITE, 1);
> >+OUT_RING(ring, 0x31);
> This may be unnecessary, but no harm in keeping it. SMMU_TABLE_UPDATE is
> supposed to do a UCHE flush.
Correct but I think it is wise to try to match the downstream sequence as much
as possible.
Jordan
> -Akhil
> >+
> >+a6xx_gpu->cur_ctx = ctx;
> >+}
> >+
> > static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
> > {
> > unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT;
> >@@ -90,6 +140,8 @@ static void a6xx_submit(struct msm_gpu *gpu, struct
> >msm_gem_submit *submit)
> > struct msm_ringbuffer *ring = submit->ring;
> > unsigned int i;
> >+a6xx_set_pagetable(a6xx_gpu, ring, submit->queue->ctx);
> >+
> > get_stats_counter(ring, REG_A6XX_RBBM_PERFCTR_CP_0_LO,
> > rbmemptr_stats(ring, index, cpcycles_start));
> >@@ -696,6 +748,8 @@ static int a6xx_hw_init(struct msm_gpu *gpu)
> > /* Always come up on rb 0 */
> > a6xx_gpu->cur_ring = gpu->rb[0];
> >+a6xx_gpu->cur_ctx = NULL;
> >+
> > /* Enable the SQE_to start the CP engine */
> > gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1);
> >@@ -1008,6 +1062,21 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu
> >*gpu)
> > return (unsigned long)busy_time;
> > }
> >+static struct msm_gem_address_space *
> >+a6xx_create_private_address_space(struct msm_gpu *gpu)
> >+{
> >+struct msm_gem_address_space *aspace = NULL;
> >+struct msm_mmu *mmu;
> >+
> >+mmu = msm_iommu_pagetable_create(gpu->aspace->mmu);
> >+
> >+if (!IS_ERR(mmu))
> >+aspace = msm_gem_address_space_create(mmu,
> >+"gpu", 0x1ULL, 0x1ULL);
> >+
> >+return aspace;
> >+}
> >+
> > static const struct adreno_gpu_funcs funcs = {
> > .base = {
> > .get_param = adreno_get_param,
> >@@ -1031,6 +1100,7 @@ static const struct adreno_gpu_funcs funcs = {
> > .gpu_state_put = a6xx_gpu_state_put,
> > #endif
> > .create_address_space = adreno_iommu_create_address_space,
> >+.create_private_address_space =
> >a6xx_create_private_address_space,
> > },
> > .get_timestamp = a6xx_get_timestamp,
> >
Re: [PATCH 16/19] drm/msm/a6xx: Add support for per-instance pagetables
On 8/14/2020 8:11 AM, Rob Clark wrote:
From: Jordan Crouse
Add support for using per-instance pagetables if all the dependencies are
available.
Signed-off-by: Jordan Crouse
Signed-off-by: Rob Clark
---
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 70 +++
drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 +
drivers/gpu/drm/msm/msm_ringbuffer.h | 1 +
3 files changed, 72 insertions(+)
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
index 5eabb0109577..9653ac9b3cb8 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
@@ -81,6 +81,56 @@ static void get_stats_counter(struct msm_ringbuffer *ring,
u32 counter,
OUT_RING(ring, upper_32_bits(iova));
}
+static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,
+ struct msm_ringbuffer *ring, struct msm_file_private *ctx)
+{
+ phys_addr_t ttbr;
+ u32 asid;
+ u64 memptr = rbmemptr(ring, ttbr0);
+
+ if (ctx == a6xx_gpu->cur_ctx)
+ return;
+
+ if (msm_iommu_pagetable_params(ctx->aspace->mmu, , ))
+ return;
+
+ /* Execute the table update */
+ OUT_PKT7(ring, CP_SMMU_TABLE_UPDATE, 4);
+ OUT_RING(ring, CP_SMMU_TABLE_UPDATE_0_TTBR0_LO(lower_32_bits(ttbr)));
+
+ /*
+* For now ignore the asid since the smmu driver uses a TLBIASID to
+* flush the TLB when we use iommu_flush_tlb_all() and the smmu driver
+* isn't aware that the asid changed. Instead, keep the default asid
+* (0, same as the context bank) to make sure the TLB is properly
+* flushed.
+*/
+ OUT_RING(ring,
+ CP_SMMU_TABLE_UPDATE_1_TTBR0_HI(upper_32_bits(ttbr)) |
+ CP_SMMU_TABLE_UPDATE_1_ASID(0));
+ OUT_RING(ring, CP_SMMU_TABLE_UPDATE_2_CONTEXTIDR(0));
+ OUT_RING(ring, CP_SMMU_TABLE_UPDATE_3_CONTEXTBANK(0));
+
+ /*
+* Write the new TTBR0 to the memstore. This is good for debugging.
+*/
+ OUT_PKT7(ring, CP_MEM_WRITE, 4);
+ OUT_RING(ring, CP_MEM_WRITE_0_ADDR_LO(lower_32_bits(memptr)));
+ OUT_RING(ring, CP_MEM_WRITE_1_ADDR_HI(upper_32_bits(memptr)));
+ OUT_RING(ring, lower_32_bits(ttbr));
+ OUT_RING(ring, (0 << 16) | upper_32_bits(ttbr));
why (0 << 16) is required here?
+
+ /*
+* And finally, trigger a uche flush to be sure there isn't anything
+* lingering in that part of the GPU
+*/
+
+ OUT_PKT7(ring, CP_EVENT_WRITE, 1);
+ OUT_RING(ring, 0x31);
This may be unnecessary, but no harm in keeping it. SMMU_TABLE_UPDATE is
supposed to do a UCHE flush.
-Akhil
+
+ a6xx_gpu->cur_ctx = ctx;
+}
+
static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
{
unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT;
@@ -90,6 +140,8 @@ static void a6xx_submit(struct msm_gpu *gpu, struct
msm_gem_submit *submit)
struct msm_ringbuffer *ring = submit->ring;
unsigned int i;
+ a6xx_set_pagetable(a6xx_gpu, ring, submit->queue->ctx);
+
get_stats_counter(ring, REG_A6XX_RBBM_PERFCTR_CP_0_LO,
rbmemptr_stats(ring, index, cpcycles_start));
@@ -696,6 +748,8 @@ static int a6xx_hw_init(struct msm_gpu *gpu)
/* Always come up on rb 0 */
a6xx_gpu->cur_ring = gpu->rb[0];
+ a6xx_gpu->cur_ctx = NULL;
+
/* Enable the SQE_to start the CP engine */
gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1);
@@ -1008,6 +1062,21 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)
return (unsigned long)busy_time;
}
+static struct msm_gem_address_space *
+a6xx_create_private_address_space(struct msm_gpu *gpu)
+{
+ struct msm_gem_address_space *aspace = NULL;
+ struct msm_mmu *mmu;
+
+ mmu = msm_iommu_pagetable_create(gpu->aspace->mmu);
+
+ if (!IS_ERR(mmu))
+ aspace = msm_gem_address_space_create(mmu,
+ "gpu", 0x1ULL, 0x1ULL);
+
+ return aspace;
+}
+
static const struct adreno_gpu_funcs funcs = {
.base = {
.get_param = adreno_get_param,
@@ -1031,6 +1100,7 @@ static const struct adreno_gpu_funcs funcs = {
.gpu_state_put = a6xx_gpu_state_put,
#endif
.create_address_space = adreno_iommu_create_address_space,
+ .create_private_address_space =
a6xx_create_private_address_space,
},
.get_timestamp = a6xx_get_timestamp,
};
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
index 03ba60d5b07f..da22d7549d9b 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h
@@ -19,6 +19,7 @@ struct a6xx_gpu {
uint64_t sqe_iova;
struct msm_ringbuffer *cur_ring;
+ struct msm_file_private *cur_ctx;
struct a6xx_gmu gmu;
};
diff --git
Re: [PATCH 16/19] drm/msm/a6xx: Add support for per-instance pagetables
On Mon, Aug 17, 2020 at 8:41 AM Akhil P Oommen wrote:
>
> On 8/14/2020 8:11 AM, Rob Clark wrote:
> > From: Jordan Crouse
> >
> > Add support for using per-instance pagetables if all the dependencies are
> > available.
> >
> > Signed-off-by: Jordan Crouse
> > Signed-off-by: Rob Clark
> > ---
> > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 70 +++
> > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 +
> > drivers/gpu/drm/msm/msm_ringbuffer.h | 1 +
> > 3 files changed, 72 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> > b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> > index 5eabb0109577..9653ac9b3cb8 100644
> > --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> > @@ -81,6 +81,56 @@ static void get_stats_counter(struct msm_ringbuffer
> > *ring, u32 counter,
> > OUT_RING(ring, upper_32_bits(iova));
> > }
> >
> > +static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,
> > + struct msm_ringbuffer *ring, struct msm_file_private *ctx)
> > +{
> > + phys_addr_t ttbr;
> > + u32 asid;
> > + u64 memptr = rbmemptr(ring, ttbr0);
> > +
> > + if (ctx == a6xx_gpu->cur_ctx)
> > + return;
> > +
> > + if (msm_iommu_pagetable_params(ctx->aspace->mmu, , ))
> > + return;
> > +
> > + /* Execute the table update */
> > + OUT_PKT7(ring, CP_SMMU_TABLE_UPDATE, 4);
> > + OUT_RING(ring, CP_SMMU_TABLE_UPDATE_0_TTBR0_LO(lower_32_bits(ttbr)));
> > +
> > + /*
> > + * For now ignore the asid since the smmu driver uses a TLBIASID to
> > + * flush the TLB when we use iommu_flush_tlb_all() and the smmu driver
> > + * isn't aware that the asid changed. Instead, keep the default asid
> > + * (0, same as the context bank) to make sure the TLB is properly
> > + * flushed.
> > + */
> > + OUT_RING(ring,
> > + CP_SMMU_TABLE_UPDATE_1_TTBR0_HI(upper_32_bits(ttbr)) |
> > + CP_SMMU_TABLE_UPDATE_1_ASID(0));
> > + OUT_RING(ring, CP_SMMU_TABLE_UPDATE_2_CONTEXTIDR(0));
> > + OUT_RING(ring, CP_SMMU_TABLE_UPDATE_3_CONTEXTBANK(0));
> > +
> > + /*
> > + * Write the new TTBR0 to the memstore. This is good for debugging.
> > + */
> > + OUT_PKT7(ring, CP_MEM_WRITE, 4);
> > + OUT_RING(ring, CP_MEM_WRITE_0_ADDR_LO(lower_32_bits(memptr)));
> > + OUT_RING(ring, CP_MEM_WRITE_1_ADDR_HI(upper_32_bits(memptr)));
> > + OUT_RING(ring, lower_32_bits(ttbr));
> > + OUT_RING(ring, (0 << 16) | upper_32_bits(ttbr));
> why (0 << 16) is required here?
That would normally be (asid << 16), except due to the way we
currently flush tlb, we need to use asid==0..
(Although msm_iommu_pagetable_params() should return asid==0, I fixed
that in msm_iommu_pagetable_create() and didn't notice that Jordan had
fixed that here instead)
BR,
-R
> > +
> > + /*
> > + * And finally, trigger a uche flush to be sure there isn't anything
> > + * lingering in that part of the GPU
> > + */
> > +
> > + OUT_PKT7(ring, CP_EVENT_WRITE, 1);
> > + OUT_RING(ring, 0x31);
> This may be unnecessary, but no harm in keeping it. SMMU_TABLE_UPDATE is
> supposed to do a UCHE flush.
>
> -Akhil
> > +
> > + a6xx_gpu->cur_ctx = ctx;
> > +}
> > +
> > static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit
> > *submit)
> > {
> > unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT;
> > @@ -90,6 +140,8 @@ static void a6xx_submit(struct msm_gpu *gpu, struct
> > msm_gem_submit *submit)
> > struct msm_ringbuffer *ring = submit->ring;
> > unsigned int i;
> >
> > + a6xx_set_pagetable(a6xx_gpu, ring, submit->queue->ctx);
> > +
> > get_stats_counter(ring, REG_A6XX_RBBM_PERFCTR_CP_0_LO,
> > rbmemptr_stats(ring, index, cpcycles_start));
> >
> > @@ -696,6 +748,8 @@ static int a6xx_hw_init(struct msm_gpu *gpu)
> > /* Always come up on rb 0 */
> > a6xx_gpu->cur_ring = gpu->rb[0];
> >
> > + a6xx_gpu->cur_ctx = NULL;
> > +
> > /* Enable the SQE_to start the CP engine */
> > gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1);
> >
> > @@ -1008,6 +1062,21 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu
> > *gpu)
> > return (unsigned long)busy_time;
> > }
> >
> > +static struct msm_gem_address_space *
> > +a6xx_create_private_address_space(struct msm_gpu *gpu)
> > +{
> > + struct msm_gem_address_space *aspace = NULL;
> > + struct msm_mmu *mmu;
> > +
> > + mmu = msm_iommu_pagetable_create(gpu->aspace->mmu);
> > +
> > + if (!IS_ERR(mmu))
> > + aspace = msm_gem_address_space_create(mmu,
> > + "gpu", 0x1ULL, 0x1ULL);
> > +
> > + return aspace;
> > +}
> > +
> > static const struct adreno_gpu_funcs funcs = {
> > .base = {
> > .get_param = adreno_get_param,
> > @@ -1031,6 +1100,7 @@ static const struct adreno_gpu_funcs funcs = {
> >
Re: [PATCH 00/16] IOMMU driver for Kirin 960/970
On Mon, Aug 17, 2020 at 12:53:45PM +0200, Greg Kroah-Hartman wrote: > You can always do this just fine, as one single patch. You do know > about the co-developed-by: line, right? Agreed. Please keep the main iommu driver in one patch and use co-developed-by. This makes it easier for me to review it and provide feedback. And please Cc me on the whole patch-set for v2. Thanks, Joerg ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 00/16] IOMMU driver for Kirin 960/970
On Mon, Aug 17, 2020 at 12:46:17PM +0200, Mauro Carvalho Chehab wrote: > The main reason of submitting via staging is that I need to preserve > the patch that added this driver as-is, in order to preserve its > SoB and not causing legal issues. > > It it is OK for iommu to accept a submission like that, I can > re-submit it, doing the changes at drivers/iommu. You can always do this just fine, as one single patch. You do know about the co-developed-by: line, right? thanks, greg k-h ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 00/16] IOMMU driver for Kirin 960/970
Em Mon, 17 Aug 2020 11:37:03 +0200
Greg Kroah-Hartman escreveu:
> On Mon, Aug 17, 2020 at 11:27:25AM +0200, Mauro Carvalho Chehab wrote:
> > Hi Christoph,
> >
> > Em Mon, 17 Aug 2020 09:21:06 +0100
> > Christoph Hellwig escreveu:
> >
> > > On Mon, Aug 17, 2020 at 09:49:59AM +0200, Mauro Carvalho Chehab wrote:
> > > > Add a driver for the Kirin 960/970 iommu.
> > > >
> > > > As on the past series, this starts from the original 4.9 driver from
> > > > the 96boards tree:
> > > >
> > > > https://github.com/96boards-hikey/linux/tree/hikey970-v4.9
> > > >
> > > > The remaining patches add SPDX headers and make it build and run with
> > > > the upstream Kernel.
> > >
> > > Please don't add iommu drivers to staging, and just work with the
> > > maintainers to properly clean it up.
> >
> > I need to start from the original patch in order to preserve its
> > authorship.
> >
> > My plan is to work with the iommu subsystem maintainers after
> > have this (and another pending patch series for DRM) merged.
> >
> > > I also don't think adding a totally out of date not compiling version
> > > is a good idea. Please do a proper rollup, and if required (probably
> > > not in this case), split it into useful chunks.
> >
> > This series make this driver working as expected.
> >
> > I mean, while patch 01/16 is against Kernel 4.9, the other patches
> > on this series ports it to upstream, cleans up the driver and
> > address several issues on it.
> >
> > This specific IOMMU seems to be an specific part of the SoC dedicated for
> > the display engine and by the encoding/decoding images via the ISP.
> > With this series, this driver builds and runs as expected, providing
> > IOMMU support needed by the upcoming KMS/DRM driver.
> >
> > The only issue on it (as far as I can tell) is that the DT bindings
> > require some work, as, instead of using dma-ranges, the DRM driver binds
> > into it with:
> >
> > smmu_lpae {
> > compatible = "hisilicon,smmu-lpae";
> > };
> >
> > dpe: dpe@e860 {
> > compatible = "hisilicon,kirin970-dpe";
> > ...
> > iommu_info {
> > start-addr = <0x8000>;
> > size = <0xbfff8000>;
> > };
> > };
> >
> > In order to properly address it, the best would be to also have the
> > DRM driver merged upstream, as it relies on it. So, a change in DT will
> > also mean a change at the way the DRM uses it.
> >
> > The DRM itself should go via staging, as it has some bugs that I'd
> > like to fix before moving it to drivers/gpu/drm. Those are more
> > tricky to solve, as they seem to require using different settings for
> > some hardware registers, and the downstream driver also have the same
> > issues. Fixing them will likely require some time.
>
> DRM drivers can't go through staging unless you get the DRM developers
> to agree with it, and last I heard, they were strongly against it.
Ok, I'll ping them. There's already a thread I opened at the DRM devel
ML about this driver. IMHO, there's nothing on this specific driver
that would prevent having it on staging for a while, until the two
or three remaining bugs gets fixed.
On the other hand, the code already follows the current DRM policies,
as far as I can tell. The bugs are related to some specific register
settings that would need to be better tuned (and maybe some delay
when waiting for EDID data at the already-existing adv7535 driver).
Maybe it would be preferred to have it at drivers/gpu even with
such bugs.
> It's _always_ faster to just do the work out-of-tree for a week or so
> and then merge it correctly to the proper part of the kernel tree. I'd
> recommend doing that here for the iommu driver, as well as the DRM
> driver.
It is OK for me to working for a week or so, until the iommu people
become happy with that.
The main reason of submitting via staging is that I need to preserve
the patch that added this driver as-is, in order to preserve its
SoB and not causing legal issues.
It it is OK for iommu to accept a submission like that, I can
re-submit it, doing the changes at drivers/iommu.
If not, besides this series, the only alternative I can think of is to
merge it first at the staging, with the incremental changes, and with
a final patch moving the code out of staging.
Thanks,
Mauro
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] swiotlb: Allow allocating buffer anywhere in memory
On Sat, Aug 15, 2020 at 05:45:36PM -0300, Thiago Jung Bauermann wrote:
> POWER secure guests (i.e., guests which use the Protection Execution
> Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but
> they don't need the SWIOTLB memory to be in low addresses since the
> hypervisor doesn't have any addressing limitation.
>
> This solves a SWIOTLB initialization problem we are seeing in secure guests
> with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved
> memory, which leaves no space for SWIOTLB in low addresses.
What about just open coding the allocation and using
swiotlb_init_with_tbl?
>
> Signed-off-by: Thiago Jung Bauermann
> ---
> arch/powerpc/mm/mem.c | 7 ++-
> include/linux/swiotlb.h | 8 +++-
> kernel/dma/swiotlb.c| 10 +++---
> 3 files changed, 20 insertions(+), 5 deletions(-)
>
> Normally I would split changes like this into one patch touching generic
> code and another for the arch-specific part, but in this case I thought it
> would be unneeded complexity. I can split though if people prefer it that
> way.
>
> diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
> index c2c11eb8dcfc..13f2e3aff8b5 100644
> --- a/arch/powerpc/mm/mem.c
> +++ b/arch/powerpc/mm/mem.c
> @@ -50,6 +50,7 @@
> #include
> #include
> #include
> +#include
>
> #include
>
> @@ -290,7 +291,11 @@ void __init mem_init(void)
>* back to to-down.
>*/
> memblock_set_bottom_up(true);
> - swiotlb_init(0);
> + /*
> + * SVM guests can use the SWIOTLB wherever it is in memory,
> + * even if not DMA-able.
> + */
> + swiotlb_init_anywhere(0, is_secure_guest());
> #endif
>
> high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
> diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
> index 046bb94bd4d6..433f3dbb35b5 100644
> --- a/include/linux/swiotlb.h
> +++ b/include/linux/swiotlb.h
> @@ -29,7 +29,13 @@ enum swiotlb_force {
> */
> #define IO_TLB_SHIFT 11
>
> -extern void swiotlb_init(int verbose);
> +void __init swiotlb_init_anywhere(int verbose, bool allocate_anywhere);
> +
> +static inline void swiotlb_init(int verbose)
> +{
> + swiotlb_init_anywhere(verbose, false);
> +}
> +
> int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
> extern unsigned long swiotlb_nr_tbl(void);
> unsigned long swiotlb_size_or_default(void);
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index c19379fabd20..27070aa59e34 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -244,7 +244,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long
> nslabs, int verbose)
> * structures for the software IO TLB used to implement the DMA API.
> */
> void __init
> -swiotlb_init(int verbose)
> +swiotlb_init_anywhere(int verbose, bool allocate_anywhere)
> {
> size_t default_size = IO_TLB_DEFAULT_SIZE;
> unsigned char *vstart;
> @@ -257,8 +257,12 @@ swiotlb_init(int verbose)
>
> bytes = io_tlb_nslabs << IO_TLB_SHIFT;
>
> - /* Get IO TLB memory from the low pages */
> - vstart = memblock_alloc_low(PAGE_ALIGN(bytes), PAGE_SIZE);
> + if (allocate_anywhere)
> + vstart = memblock_alloc(PAGE_ALIGN(bytes), PAGE_SIZE);
> + else
> + /* Get IO TLB memory from the low pages */
> + vstart = memblock_alloc_low(PAGE_ALIGN(bytes), PAGE_SIZE);
> +
> if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, verbose))
> return;
>
---end quoted text---
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 00/16] IOMMU driver for Kirin 960/970
On Mon, Aug 17, 2020 at 11:27:25AM +0200, Mauro Carvalho Chehab wrote:
> Hi Christoph,
>
> Em Mon, 17 Aug 2020 09:21:06 +0100
> Christoph Hellwig escreveu:
>
> > On Mon, Aug 17, 2020 at 09:49:59AM +0200, Mauro Carvalho Chehab wrote:
> > > Add a driver for the Kirin 960/970 iommu.
> > >
> > > As on the past series, this starts from the original 4.9 driver from
> > > the 96boards tree:
> > >
> > > https://github.com/96boards-hikey/linux/tree/hikey970-v4.9
> > >
> > > The remaining patches add SPDX headers and make it build and run with
> > > the upstream Kernel.
> >
> > Please don't add iommu drivers to staging, and just work with the
> > maintainers to properly clean it up.
>
> I need to start from the original patch in order to preserve its
> authorship.
>
> My plan is to work with the iommu subsystem maintainers after
> have this (and another pending patch series for DRM) merged.
>
> > I also don't think adding a totally out of date not compiling version
> > is a good idea. Please do a proper rollup, and if required (probably
> > not in this case), split it into useful chunks.
>
> This series make this driver working as expected.
>
> I mean, while patch 01/16 is against Kernel 4.9, the other patches
> on this series ports it to upstream, cleans up the driver and
> address several issues on it.
>
> This specific IOMMU seems to be an specific part of the SoC dedicated for
> the display engine and by the encoding/decoding images via the ISP.
> With this series, this driver builds and runs as expected, providing
> IOMMU support needed by the upcoming KMS/DRM driver.
>
> The only issue on it (as far as I can tell) is that the DT bindings
> require some work, as, instead of using dma-ranges, the DRM driver binds
> into it with:
>
> smmu_lpae {
> compatible = "hisilicon,smmu-lpae";
> };
>
> dpe: dpe@e860 {
> compatible = "hisilicon,kirin970-dpe";
> ...
> iommu_info {
> start-addr = <0x8000>;
> size = <0xbfff8000>;
> };
> };
>
> In order to properly address it, the best would be to also have the
> DRM driver merged upstream, as it relies on it. So, a change in DT will
> also mean a change at the way the DRM uses it.
>
> The DRM itself should go via staging, as it has some bugs that I'd
> like to fix before moving it to drivers/gpu/drm. Those are more
> tricky to solve, as they seem to require using different settings for
> some hardware registers, and the downstream driver also have the same
> issues. Fixing them will likely require some time.
DRM drivers can't go through staging unless you get the DRM developers
to agree with it, and last I heard, they were strongly against it.
It's _always_ faster to just do the work out-of-tree for a week or so
and then merge it correctly to the proper part of the kernel tree. I'd
recommend doing that here for the iommu driver, as well as the DRM
driver.
There's no issues with authorship and the like, just properly attribute
it when you submit it and you are fine.
Again, merging in staging always takes more work and energy, don't do it
unless there is no other way.
thanks,
greg k-h
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 00/16] IOMMU driver for Kirin 960/970
On Mon, Aug 17, 2020 at 11:27:25AM +0200, Mauro Carvalho Chehab wrote: > I need to start from the original patch in order to preserve its > authorship. Nom you don't. We have plenty of example where people submit code written by other, especially when it is significantly reworked, > My plan is to work with the iommu subsystem maintainers after > have this (and another pending patch series for DRM) merged. Please submit it to the iommu list directly. Seriously - you did not even cc the relevant list, and you don't even have a comment from the maintainers. Don't do this kind of crap. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 00/16] IOMMU driver for Kirin 960/970
Hi Christoph,
Em Mon, 17 Aug 2020 09:21:06 +0100
Christoph Hellwig escreveu:
> On Mon, Aug 17, 2020 at 09:49:59AM +0200, Mauro Carvalho Chehab wrote:
> > Add a driver for the Kirin 960/970 iommu.
> >
> > As on the past series, this starts from the original 4.9 driver from
> > the 96boards tree:
> >
> > https://github.com/96boards-hikey/linux/tree/hikey970-v4.9
> >
> > The remaining patches add SPDX headers and make it build and run with
> > the upstream Kernel.
>
> Please don't add iommu drivers to staging, and just work with the
> maintainers to properly clean it up.
I need to start from the original patch in order to preserve its
authorship.
My plan is to work with the iommu subsystem maintainers after
have this (and another pending patch series for DRM) merged.
> I also don't think adding a totally out of date not compiling version
> is a good idea. Please do a proper rollup, and if required (probably
> not in this case), split it into useful chunks.
This series make this driver working as expected.
I mean, while patch 01/16 is against Kernel 4.9, the other patches
on this series ports it to upstream, cleans up the driver and
address several issues on it.
This specific IOMMU seems to be an specific part of the SoC dedicated for
the display engine and by the encoding/decoding images via the ISP.
With this series, this driver builds and runs as expected, providing
IOMMU support needed by the upcoming KMS/DRM driver.
The only issue on it (as far as I can tell) is that the DT bindings
require some work, as, instead of using dma-ranges, the DRM driver binds
into it with:
smmu_lpae {
compatible = "hisilicon,smmu-lpae";
};
dpe: dpe@e860 {
compatible = "hisilicon,kirin970-dpe";
...
iommu_info {
start-addr = <0x8000>;
size = <0xbfff8000>;
};
};
In order to properly address it, the best would be to also have the
DRM driver merged upstream, as it relies on it. So, a change in DT will
also mean a change at the way the DRM uses it.
The DRM itself should go via staging, as it has some bugs that I'd
like to fix before moving it to drivers/gpu/drm. Those are more
tricky to solve, as they seem to require using different settings for
some hardware registers, and the downstream driver also have the same
issues. Fixing them will likely require some time.
Thanks,
Mauro
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 00/16] IOMMU driver for Kirin 960/970
On Mon, Aug 17, 2020 at 09:49:59AM +0200, Mauro Carvalho Chehab wrote: > Add a driver for the Kirin 960/970 iommu. > > As on the past series, this starts from the original 4.9 driver from > the 96boards tree: > > https://github.com/96boards-hikey/linux/tree/hikey970-v4.9 > > The remaining patches add SPDX headers and make it build and run with > the upstream Kernel. Please don't add iommu drivers to staging, and just work with the maintainers to properly clean it up. I also don't think adding a totally out of date not compiling version is a good idea. Please do a proper rollup, and if required (probably not in this case), split it into useful chunks. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 14/15] vfio: Document dual stage control
Hi Yi, On 8/17/20 9:00 AM, Liu, Yi L wrote: > Hi Eric, > >> From: Auger Eric >> Sent: Sunday, August 16, 2020 7:52 PM >> >> Hi Yi, >> >> On 7/28/20 8:27 AM, Liu Yi L wrote: >>> From: Eric Auger >>> >>> The VFIO API was enhanced to support nested stage control: a bunch of> new >> ioctls and usage guideline. >>> >>> Let's document the process to follow to set up nested mode. >>> >>> Cc: Kevin Tian >>> CC: Jacob Pan >>> Cc: Alex Williamson >>> Cc: Eric Auger >>> Cc: Jean-Philippe Brucker >>> Cc: Joerg Roedel >>> Cc: Lu Baolu >>> Reviewed-by: Stefan Hajnoczi >>> Signed-off-by: Eric Auger >>> Signed-off-by: Liu Yi L >>> --- >>> v5 -> v6: >>> *) tweak per Eric's comments. >>> >>> v3 -> v4: >>> *) add review-by from Stefan Hajnoczi >>> >>> v2 -> v3: >>> *) address comments from Stefan Hajnoczi >>> >>> v1 -> v2: >>> *) new in v2, compared with Eric's original version, pasid table bind >>>and fault reporting is removed as this series doesn't cover them. >>>Original version from Eric. >>>https://lkml.org/lkml/2020/3/20/700 >>> --- >>> Documentation/driver-api/vfio.rst | 75 >> +++ >>> 1 file changed, 75 insertions(+) >>> >>> diff --git a/Documentation/driver-api/vfio.rst >>> b/Documentation/driver-api/vfio.rst >>> index f1a4d3c..c0d43f0 100644 >>> --- a/Documentation/driver-api/vfio.rst >>> +++ b/Documentation/driver-api/vfio.rst >>> @@ -239,6 +239,81 @@ group and can access them as follows:: >>> /* Gratuitous device reset and go... */ >>> ioctl(device, VFIO_DEVICE_RESET); >>> >>> +IOMMU Dual Stage Control >>> + >>> + >>> +Some IOMMUs support 2 stages/levels of translation. Stage corresponds >>> +to the ARM terminology while level corresponds to Intel's terminology. >>> +In the following text we use either without distinction. >>> + >>> +This is useful when the guest is exposed with a virtual IOMMU and some >>> +devices are assigned to the guest through VFIO. Then the guest OS can >>> +use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor uses stage >>> +2 for VM isolation (GPA -> HPA). >>> + >>> +Under dual stage translation, the guest gets ownership of the stage-1 page >>> +tables and also owns stage-1 configuration structures. The hypervisor owns >>> +the root configuration structure (for security reason), including stage-2 >>> +configuration. >> This is only true for vtd. On ARM the stage2 cfg is the Context >> Descriptor table (aka PASID table). root cfg only store the GPA of the >> CD table. > > I've a check with you on the meaning of "configuration structures". > For Vt-d, does it mean the root table/context table/pasid table? if > I'm correct, then how about below description? Yes I agree > > "Under dual stage translation, the guest gets ownership of the stage-1 > configuration structures or page tables. Actually on ARM the guest both owns the S1 configuration (CD table) and S1 page tables ;-) on Intel I understand the guest only owns the S1 page tables. If confirmed, you may use such kind of explicit statement. Thanks Eric This depends on vendor. The > hypervisor owns the root configuration structure (for security reason), > including stage-2 configuration." > >> This works as long as configuration structures and page table >>> +formats are compatible between the virtual IOMMU and the physical IOMMU. >>> + >>> +Assuming the HW supports it, this nested mode is selected by choosing the >>> +VFIO_TYPE1_NESTING_IOMMU type through: >>> + >>> +ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); >>> + >>> +This forces the hypervisor to use the stage-2, leaving stage-1 available >>> +for guest usage. The stage-1 format and binding method are vendor specific >> . There are reported in the nesting capability ... > > got it. > > "The stage-1 format and binding method are reported in nesting capability." > >>> +and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) through >>> +VFIO_IOMMU_GET_INFO: >>> + >>> +ioctl(container->fd, VFIO_IOMMU_GET_INFO, _info); >>> + >>> +The nesting cap info is available only after NESTING_IOMMU is selected. >>> +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails and >> If the underlying > > got it. > >>> +userspace should try other IOMMU types. Details of the nesting cap info >>> +can be found in Documentation/userspace-api/iommu.rst. >>> + >>> +The stage-1 page table can be bound to the IOMMU in two methods: directly> >> +or indirectly. Direct binding requires userspace to notify VFIO of every >> Not sure we shall use this direct/indirect terminology. I don't think >> this is part of either ARM or Intel SPEC. >> >> Suggestion: On Intel, the stage1 page table info are mediated by the >> userspace for each PASID. On ARM, the userspace directly passes the GPA >> of the whole PASID table. Currently only Intel's binding is supported. > > got it. this is what we want to say by ditect/indirect terminology. > > Regards, > Yi Liu > >>>
[PATCH 14/16] dt: add an spec for the Kirin36x0 SMMU
Describe the properties expected by the IOMMU driver used on
Hikey960 and Hikey970 boards.
Signed-off-by: Mauro Carvalho Chehab
---
.../iommu/hisilicon,kirin36x0-smmu.yaml | 55 +++
1 file changed, 55 insertions(+)
create mode 100644
Documentation/devicetree/bindings/iommu/hisilicon,kirin36x0-smmu.yaml
diff --git
a/Documentation/devicetree/bindings/iommu/hisilicon,kirin36x0-smmu.yaml
b/Documentation/devicetree/bindings/iommu/hisilicon,kirin36x0-smmu.yaml
new file mode 100644
index ..ec4c98faf3a5
--- /dev/null
+++ b/Documentation/devicetree/bindings/iommu/hisilicon,kirin36x0-smmu.yaml
@@ -0,0 +1,55 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iommu/hisilicon,kirin36x0-smmu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Hisilicon support for HI3660/HI3670 SMMU
+
+maintainers:
+ - Mauro Carvalho Chehab
+
+description: |+
+ Huawei's Hisilicon Kirin 3660/3670 contains a System MMU that enables
+ scattered physical memory chunks visible as a contiguous region to
+ DMA-capable peripheral devices like GPU and ISP.
+
+ The IOMMU domains are described via iommu_info settings.
+
+properties:
+ compatible:
+const: hisilicon,hisi-smmu-lpae
+
+ iommu_info:
+type: object
+
+properties:
+ start-addr:
+maxItems: 1
+description: Memory start address (32 bits)
+
+ size:
+maxItems: 1
+description: size of the I/O MMU block (32 bits)
+
+ iova-align:
+minItems: 2
+maxItems: 2
+description: DMA address alignment of the mapped memory (64 bits)
+
+required:
+ - compatible
+
+additionalProperties: false
+
+examples:
+ - |
+smmu_lpae {
+ compatible = "hisilicon,smmu-lpae";
+
+ iommu_info {
+start-addr = <0x4>;
+size = <0xbffc>;
+iova-align = <0x0 0x8000>;
+ };
+};
--
2.26.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 00/16] IOMMU driver for Kirin 960/970
Add a driver for the Kirin 960/970 iommu. As on the past series, this starts from the original 4.9 driver from the 96boards tree: https://github.com/96boards-hikey/linux/tree/hikey970-v4.9 The remaining patches add SPDX headers and make it build and run with the upstream Kernel. Chenfeng (1): iommu: add support for HiSilicon Kirin 960/970 iommu Mauro Carvalho Chehab (15): iommu: hisilicon: remove default iommu_map_sg handler iommu: hisilicon: map and unmap ops gained new arguments iommu: hisi_smmu_lpae: rebase it to work with upstream iommu: hisi_smmu: remove linux/hisi/hisi-iommu.h iommu: hisilicon: cleanup its code style iommu: hisi_smmu_lpae: get rid of IOMMU_SEC and IOMMU_DEVICE iommu: get rid of map/unmap tile functions iommu: hisi_smmu_lpae: use the right code to get domain-priv data iommu: hisi_smmu_lpae: convert it to probe_device iommu: add Hisilicon Kirin970 iommu at the building system iommu: hisi_smmu_lpae: cleanup printk macros iommu: hisi_smmu_lpae: make OF compatible more standard dt: add an spec for the Kirin36x0 SMMU dt: hi3670-hikey970.dts: load the SMMU driver on Hikey970 staging: hikey9xx: add an item about the iommu driver .../iommu/hisilicon,kirin36x0-smmu.yaml | 55 ++ .../boot/dts/hisilicon/hi3670-hikey970.dts| 3 + drivers/staging/hikey9xx/Kconfig | 9 + drivers/staging/hikey9xx/Makefile | 1 + drivers/staging/hikey9xx/TODO | 1 + drivers/staging/hikey9xx/hisi_smmu.h | 196 ++ drivers/staging/hikey9xx/hisi_smmu_lpae.c | 648 ++ 7 files changed, 913 insertions(+) create mode 100644 Documentation/devicetree/bindings/iommu/hisilicon,kirin36x0-smmu.yaml create mode 100644 drivers/staging/hikey9xx/hisi_smmu.h create mode 100644 drivers/staging/hikey9xx/hisi_smmu_lpae.c -- 2.26.2 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 15/15] iommu/vt-d: Support reporting nesting capability info
Eric,
> From: Auger Eric
> Sent: Monday, August 17, 2020 3:43 PM
>
> On 8/17/20 9:05 AM, Liu, Yi L wrote:
> > Hi Eric,
> >
> >> Auger Eric
> >> Sent: Sunday, August 16, 2020 8:01 PM
> >>
> >> Hi Yi,
> >>
> >> On 7/28/20 8:27 AM, Liu Yi L wrote:
> >>> This patch reports nesting info, and only supports the case where all
> >>> the physical iomms have the same CAP/ECAP MASKS.
> >> s/iomms/iommus
> >
> > yep.
> >
> >>>
> >>> Cc: Kevin Tian
> >>> CC: Jacob Pan
> >>> Cc: Alex Williamson
> >>> Cc: Eric Auger
> >>> Cc: Jean-Philippe Brucker
> >>> Cc: Joerg Roedel
> >>> Cc: Lu Baolu
> >>> Signed-off-by: Liu Yi L
> >>> Signed-off-by: Jacob Pan
> >>> ---
> >>> v2 -> v3:
> >>> *) remove cap/ecap_mask in iommu_nesting_info.
> >>> ---
> >>> drivers/iommu/intel/iommu.c | 81
> >> +++--
> >>> include/linux/intel-iommu.h | 16 +
> >>> 2 files changed, 95 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> >>> index 88f4647..0835804 100644
> >>> --- a/drivers/iommu/intel/iommu.c
> >>> +++ b/drivers/iommu/intel/iommu.c
> >>> @@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)
> >>> static inline bool nested_mode_support(void)
> >>> {
> >>> struct dmar_drhd_unit *drhd;
> >>> - struct intel_iommu *iommu;
> >>> + struct intel_iommu *iommu, *prev = NULL;
> >>> bool ret = true;
> >>>
> >>> rcu_read_lock();
> >>> for_each_active_iommu(iommu, drhd) {
> >>> - if (!sm_supported(iommu) || !ecap_nest(iommu->ecap)) {
> >>> + if (!prev)
> >>> + prev = iommu;
> >>> + if (!sm_supported(iommu) || !ecap_nest(iommu->ecap) ||
> >>> + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) ||
> >>> + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) {
> >>> ret = false;
> >>> break;
> >> So this changes the behavior of DOMAIN_ATTR_NESTING. Shouldn't it have a
> >> Fixes tag as well? And maybe add the capability getter in a separate patch?
> >
> > yes, this changed the behavior. so it would be better to be a separate patch
> > and upstream along? how about your idea? @Lu, Baolu :-)
> >
> >>> }
> >>> @@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct
> iommu_domain
> >> *domain,
> >>> return ret;
> >>> }
> >>>
> >>> +static int intel_iommu_get_nesting_info(struct iommu_domain *domain,
> >>> + struct iommu_nesting_info *info)
> >>> +{
> >>> + struct dmar_domain *dmar_domain = to_dmar_domain(domain);
> >>> + u64 cap = VTD_CAP_MASK, ecap = VTD_ECAP_MASK;
> >>> + struct device_domain_info *domain_info;
> >>> + struct iommu_nesting_info_vtd vtd;
> >>> + unsigned long flags;
> >>> + unsigned int size;
> >>> +
> >
> > perhaps better to acquire the lock here. [1]
> >
> >>> + if (domain->type != IOMMU_DOMAIN_UNMANAGED ||
> >>> + !(dmar_domain->flags & DOMAIN_FLAG_NESTING_MODE))
> >>> + return -ENODEV;
> >>> +
> >>> + if (!info)
> >>> + return -EINVAL;
> >>> +
> >>> + size = sizeof(struct iommu_nesting_info) +
> >>> + sizeof(struct iommu_nesting_info_vtd);
> >>> + /*
> >>> + * if provided buffer size is smaller than expected, should
> >>> + * return 0 and also the expected buffer size to caller.
> >>> + */
> >>> + if (info->argsz < size) {
> >>> + info->argsz = size;
> >>> + return 0;
> >>> + }
> >>> +
> >>> + spin_lock_irqsave(_domain_lock, flags);
> >>> + /*
> >>> + * arbitrary select the first domain_info as all nesting
> >>> + * related capabilities should be consistent across iommu
> >>> + * units.
> >>> + */
> >>> + domain_info = list_first_entry(_domain->devices,
> >>> +struct device_domain_info, link);
> >>> + cap &= domain_info->iommu->cap;
> >>> + ecap &= domain_info->iommu->ecap;
> >>> + spin_unlock_irqrestore(_domain_lock, flags);
> >>> +
> >>> + info->format = IOMMU_PASID_FORMAT_INTEL_VTD;
> >>> + info->features = IOMMU_NESTING_FEAT_SYSWIDE_PASID |
> >>> + IOMMU_NESTING_FEAT_BIND_PGTBL |
> >>> + IOMMU_NESTING_FEAT_CACHE_INVLD;
> >>> + info->addr_width = dmar_domain->gaw;
> >>> + info->pasid_bits = ilog2(intel_pasid_max_id);
> >>> + info->padding = 0;
> >>> + vtd.flags = 0;
> >>> + vtd.padding = 0;
> >>> + vtd.cap_reg = cap;
> >>> + vtd.ecap_reg = ecap;
> >>> +
> >>> + memcpy(info->data, , sizeof(vtd));
> >>> + return 0;
> >>> +}
> >>> +
> >>> +static int intel_iommu_domain_get_attr(struct iommu_domain *domain,
> >>> +enum iommu_attr attr, void *data)
> >>> +{
> >>> + switch (attr) {
> >>> + case DOMAIN_ATTR_NESTING:
> >>> + {
> >>> + struct iommu_nesting_info *info =
> >>> + (struct iommu_nesting_info *)data;
> >>
> >> don't you need to hold a device_domain_lock earlier to make sure domain
> >> attributes can't change behind your back (unmanaged type and nested mode)?
> >
> > do you
RE: [PATCH v6 14/15] vfio: Document dual stage control
Hi Eric, > From: Eric Auger > Sent: Monday, August 17, 2020 3:41 PM > > Hi Yi, > > On 8/17/20 9:00 AM, Liu, Yi L wrote: > > Hi Eric, > > > >> From: Auger Eric > >> Sent: Sunday, August 16, 2020 7:52 PM > >> > >> Hi Yi, > >> > >> On 7/28/20 8:27 AM, Liu Yi L wrote: > >>> From: Eric Auger > >>> > >>> The VFIO API was enhanced to support nested stage control: a bunch > >>> of> new > >> ioctls and usage guideline. > >>> > >>> Let's document the process to follow to set up nested mode. > >>> > >>> Cc: Kevin Tian > >>> CC: Jacob Pan > >>> Cc: Alex Williamson > >>> Cc: Eric Auger > >>> Cc: Jean-Philippe Brucker > >>> Cc: Joerg Roedel > >>> Cc: Lu Baolu > >>> Reviewed-by: Stefan Hajnoczi > >>> Signed-off-by: Eric Auger > >>> Signed-off-by: Liu Yi L > >>> --- > >>> v5 -> v6: > >>> *) tweak per Eric's comments. > >>> > >>> v3 -> v4: > >>> *) add review-by from Stefan Hajnoczi > >>> > >>> v2 -> v3: > >>> *) address comments from Stefan Hajnoczi > >>> > >>> v1 -> v2: > >>> *) new in v2, compared with Eric's original version, pasid table bind > >>>and fault reporting is removed as this series doesn't cover them. > >>>Original version from Eric. > >>>https://lkml.org/lkml/2020/3/20/700 > >>> --- > >>> Documentation/driver-api/vfio.rst | 75 > >> +++ > >>> 1 file changed, 75 insertions(+) > >>> > >>> diff --git a/Documentation/driver-api/vfio.rst > >>> b/Documentation/driver-api/vfio.rst > >>> index f1a4d3c..c0d43f0 100644 > >>> --- a/Documentation/driver-api/vfio.rst > >>> +++ b/Documentation/driver-api/vfio.rst > >>> @@ -239,6 +239,81 @@ group and can access them as follows:: > >>> /* Gratuitous device reset and go... */ > >>> ioctl(device, VFIO_DEVICE_RESET); > >>> > >>> +IOMMU Dual Stage Control > >>> + > >>> + > >>> +Some IOMMUs support 2 stages/levels of translation. Stage > >>> +corresponds to the ARM terminology while level corresponds to Intel's > terminology. > >>> +In the following text we use either without distinction. > >>> + > >>> +This is useful when the guest is exposed with a virtual IOMMU and > >>> +some devices are assigned to the guest through VFIO. Then the guest > >>> +OS can use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor > >>> +uses stage > >>> +2 for VM isolation (GPA -> HPA). > >>> + > >>> +Under dual stage translation, the guest gets ownership of the > >>> +stage-1 page tables and also owns stage-1 configuration structures. > >>> +The hypervisor owns the root configuration structure (for security > >>> +reason), including stage-2 configuration. > >> This is only true for vtd. On ARM the stage2 cfg is the Context > >> Descriptor table (aka PASID table). root cfg only store the GPA of > >> the CD table. > > > > I've a check with you on the meaning of "configuration structures". > > For Vt-d, does it mean the root table/context table/pasid table? if > > I'm correct, then how about below description? > Yes I agree thanks. > > > > "Under dual stage translation, the guest gets ownership of the stage-1 > > configuration structures or page tables. > Actually on ARM the guest both owns the S1 configuration (CD table) and > S1 page tables ;-) I see. so on ARM platform, guest owns both configuration and page table. > on Intel I understand the guest only owns the S1 page tables. yes, on Intel, guest only owns the S1 page tables. > If confirmed, you may use such kind of explicit statement. will do. Regards, Yi Liu > Thanks > > Eric > > This depends on vendor. The > > hypervisor owns the root configuration structure (for security > > reason), including stage-2 configuration." > > > >> This works as long as configuration structures and page table > >>> +formats are compatible between the virtual IOMMU and the physical IOMMU. > >>> + > >>> +Assuming the HW supports it, this nested mode is selected by > >>> +choosing the VFIO_TYPE1_NESTING_IOMMU type through: > >>> + > >>> +ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); > >>> + > >>> +This forces the hypervisor to use the stage-2, leaving stage-1 > >>> +available for guest usage. The stage-1 format and binding method > >>> +are vendor specific > >> . There are reported in the nesting capability ... > > > > got it. > > > > "The stage-1 format and binding method are reported in nesting capability." > > > >>> +and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) > >>> +through > >>> +VFIO_IOMMU_GET_INFO: > >>> + > >>> +ioctl(container->fd, VFIO_IOMMU_GET_INFO, _info); > >>> + > >>> +The nesting cap info is available only after NESTING_IOMMU is selected. > >>> +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails > >>> +and > >> If the underlying > > > > got it. > > > >>> +userspace should try other IOMMU types. Details of the nesting cap > >>> +info can be found in Documentation/userspace-api/iommu.rst. > >>> + > >>> +The stage-1 page table can be bound to the IOMMU in two methods: > >>> +directly> >
Re: [PATCH v6 15/15] iommu/vt-d: Support reporting nesting capability info
On 8/17/20 9:05 AM, Liu, Yi L wrote:
> Hi Eric,
>
>> Auger Eric
>> Sent: Sunday, August 16, 2020 8:01 PM
>>
>> Hi Yi,
>>
>> On 7/28/20 8:27 AM, Liu Yi L wrote:
>>> This patch reports nesting info, and only supports the case where all
>>> the physical iomms have the same CAP/ECAP MASKS.
>> s/iomms/iommus
>
> yep.
>
>>>
>>> Cc: Kevin Tian
>>> CC: Jacob Pan
>>> Cc: Alex Williamson
>>> Cc: Eric Auger
>>> Cc: Jean-Philippe Brucker
>>> Cc: Joerg Roedel
>>> Cc: Lu Baolu
>>> Signed-off-by: Liu Yi L
>>> Signed-off-by: Jacob Pan
>>> ---
>>> v2 -> v3:
>>> *) remove cap/ecap_mask in iommu_nesting_info.
>>> ---
>>> drivers/iommu/intel/iommu.c | 81
>> +++--
>>> include/linux/intel-iommu.h | 16 +
>>> 2 files changed, 95 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>>> index 88f4647..0835804 100644
>>> --- a/drivers/iommu/intel/iommu.c
>>> +++ b/drivers/iommu/intel/iommu.c
>>> @@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)
>>> static inline bool nested_mode_support(void)
>>> {
>>> struct dmar_drhd_unit *drhd;
>>> - struct intel_iommu *iommu;
>>> + struct intel_iommu *iommu, *prev = NULL;
>>> bool ret = true;
>>>
>>> rcu_read_lock();
>>> for_each_active_iommu(iommu, drhd) {
>>> - if (!sm_supported(iommu) || !ecap_nest(iommu->ecap)) {
>>> + if (!prev)
>>> + prev = iommu;
>>> + if (!sm_supported(iommu) || !ecap_nest(iommu->ecap) ||
>>> + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) ||
>>> + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) {
>>> ret = false;
>>> break;
>> So this changes the behavior of DOMAIN_ATTR_NESTING. Shouldn't it have a
>> Fixes tag as well? And maybe add the capability getter in a separate patch?
>
> yes, this changed the behavior. so it would be better to be a separate patch
> and upstream along? how about your idea? @Lu, Baolu :-)
>
>>> }
>>> @@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct iommu_domain
>> *domain,
>>> return ret;
>>> }
>>>
>>> +static int intel_iommu_get_nesting_info(struct iommu_domain *domain,
>>> + struct iommu_nesting_info *info)
>>> +{
>>> + struct dmar_domain *dmar_domain = to_dmar_domain(domain);
>>> + u64 cap = VTD_CAP_MASK, ecap = VTD_ECAP_MASK;
>>> + struct device_domain_info *domain_info;
>>> + struct iommu_nesting_info_vtd vtd;
>>> + unsigned long flags;
>>> + unsigned int size;
>>> +
>
> perhaps better to acquire the lock here. [1]
>
>>> + if (domain->type != IOMMU_DOMAIN_UNMANAGED ||
>>> + !(dmar_domain->flags & DOMAIN_FLAG_NESTING_MODE))
>>> + return -ENODEV;
>>> +
>>> + if (!info)
>>> + return -EINVAL;
>>> +
>>> + size = sizeof(struct iommu_nesting_info) +
>>> + sizeof(struct iommu_nesting_info_vtd);
>>> + /*
>>> +* if provided buffer size is smaller than expected, should
>>> +* return 0 and also the expected buffer size to caller.
>>> +*/
>>> + if (info->argsz < size) {
>>> + info->argsz = size;
>>> + return 0;
>>> + }
>>> +
>>> + spin_lock_irqsave(_domain_lock, flags);
>>> + /*
>>> +* arbitrary select the first domain_info as all nesting
>>> +* related capabilities should be consistent across iommu
>>> +* units.
>>> +*/
>>> + domain_info = list_first_entry(_domain->devices,
>>> + struct device_domain_info, link);
>>> + cap &= domain_info->iommu->cap;
>>> + ecap &= domain_info->iommu->ecap;
>>> + spin_unlock_irqrestore(_domain_lock, flags);
>>> +
>>> + info->format = IOMMU_PASID_FORMAT_INTEL_VTD;
>>> + info->features = IOMMU_NESTING_FEAT_SYSWIDE_PASID |
>>> +IOMMU_NESTING_FEAT_BIND_PGTBL |
>>> +IOMMU_NESTING_FEAT_CACHE_INVLD;
>>> + info->addr_width = dmar_domain->gaw;
>>> + info->pasid_bits = ilog2(intel_pasid_max_id);
>>> + info->padding = 0;
>>> + vtd.flags = 0;
>>> + vtd.padding = 0;
>>> + vtd.cap_reg = cap;
>>> + vtd.ecap_reg = ecap;
>>> +
>>> + memcpy(info->data, , sizeof(vtd));
>>> + return 0;
>>> +}
>>> +
>>> +static int intel_iommu_domain_get_attr(struct iommu_domain *domain,
>>> + enum iommu_attr attr, void *data)
>>> +{
>>> + switch (attr) {
>>> + case DOMAIN_ATTR_NESTING:
>>> + {
>>> + struct iommu_nesting_info *info =
>>> + (struct iommu_nesting_info *)data;
>>
>> don't you need to hold a device_domain_lock earlier to make sure domain
>> attributes can't change behind your back (unmanaged type and nested mode)?
>
> do you mean acquire lock at [1]?
yep either at [1] or before calling intel_iommu_get_nesting_info and
adding a comment saying intel_iommu_get_nesting_info() shall be called
with the lock held
Thanks
Eric
>
>
RE: [PATCH v6 15/15] iommu/vt-d: Support reporting nesting capability info
Hi Eric,
> Auger Eric
> Sent: Sunday, August 16, 2020 8:01 PM
>
> Hi Yi,
>
> On 7/28/20 8:27 AM, Liu Yi L wrote:
> > This patch reports nesting info, and only supports the case where all
> > the physical iomms have the same CAP/ECAP MASKS.
> s/iomms/iommus
yep.
> >
> > Cc: Kevin Tian
> > CC: Jacob Pan
> > Cc: Alex Williamson
> > Cc: Eric Auger
> > Cc: Jean-Philippe Brucker
> > Cc: Joerg Roedel
> > Cc: Lu Baolu
> > Signed-off-by: Liu Yi L
> > Signed-off-by: Jacob Pan
> > ---
> > v2 -> v3:
> > *) remove cap/ecap_mask in iommu_nesting_info.
> > ---
> > drivers/iommu/intel/iommu.c | 81
> +++--
> > include/linux/intel-iommu.h | 16 +
> > 2 files changed, 95 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > index 88f4647..0835804 100644
> > --- a/drivers/iommu/intel/iommu.c
> > +++ b/drivers/iommu/intel/iommu.c
> > @@ -5660,12 +5660,16 @@ static inline bool iommu_pasid_support(void)
> > static inline bool nested_mode_support(void)
> > {
> > struct dmar_drhd_unit *drhd;
> > - struct intel_iommu *iommu;
> > + struct intel_iommu *iommu, *prev = NULL;
> > bool ret = true;
> >
> > rcu_read_lock();
> > for_each_active_iommu(iommu, drhd) {
> > - if (!sm_supported(iommu) || !ecap_nest(iommu->ecap)) {
> > + if (!prev)
> > + prev = iommu;
> > + if (!sm_supported(iommu) || !ecap_nest(iommu->ecap) ||
> > + (VTD_CAP_MASK & (iommu->cap ^ prev->cap)) ||
> > + (VTD_ECAP_MASK & (iommu->ecap ^ prev->ecap))) {
> > ret = false;
> > break;
> So this changes the behavior of DOMAIN_ATTR_NESTING. Shouldn't it have a
> Fixes tag as well? And maybe add the capability getter in a separate patch?
yes, this changed the behavior. so it would be better to be a separate patch
and upstream along? how about your idea? @Lu, Baolu :-)
> > }
> > @@ -6081,6 +6085,78 @@ intel_iommu_domain_set_attr(struct iommu_domain
> *domain,
> > return ret;
> > }
> >
> > +static int intel_iommu_get_nesting_info(struct iommu_domain *domain,
> > + struct iommu_nesting_info *info)
> > +{
> > + struct dmar_domain *dmar_domain = to_dmar_domain(domain);
> > + u64 cap = VTD_CAP_MASK, ecap = VTD_ECAP_MASK;
> > + struct device_domain_info *domain_info;
> > + struct iommu_nesting_info_vtd vtd;
> > + unsigned long flags;
> > + unsigned int size;
> > +
perhaps better to acquire the lock here. [1]
> > + if (domain->type != IOMMU_DOMAIN_UNMANAGED ||
> > + !(dmar_domain->flags & DOMAIN_FLAG_NESTING_MODE))
> > + return -ENODEV;
> > +
> > + if (!info)
> > + return -EINVAL;
> > +
> > + size = sizeof(struct iommu_nesting_info) +
> > + sizeof(struct iommu_nesting_info_vtd);
> > + /*
> > +* if provided buffer size is smaller than expected, should
> > +* return 0 and also the expected buffer size to caller.
> > +*/
> > + if (info->argsz < size) {
> > + info->argsz = size;
> > + return 0;
> > + }
> > +
> > + spin_lock_irqsave(_domain_lock, flags);
> > + /*
> > +* arbitrary select the first domain_info as all nesting
> > +* related capabilities should be consistent across iommu
> > +* units.
> > +*/
> > + domain_info = list_first_entry(_domain->devices,
> > + struct device_domain_info, link);
> > + cap &= domain_info->iommu->cap;
> > + ecap &= domain_info->iommu->ecap;
> > + spin_unlock_irqrestore(_domain_lock, flags);
> > +
> > + info->format = IOMMU_PASID_FORMAT_INTEL_VTD;
> > + info->features = IOMMU_NESTING_FEAT_SYSWIDE_PASID |
> > +IOMMU_NESTING_FEAT_BIND_PGTBL |
> > +IOMMU_NESTING_FEAT_CACHE_INVLD;
> > + info->addr_width = dmar_domain->gaw;
> > + info->pasid_bits = ilog2(intel_pasid_max_id);
> > + info->padding = 0;
> > + vtd.flags = 0;
> > + vtd.padding = 0;
> > + vtd.cap_reg = cap;
> > + vtd.ecap_reg = ecap;
> > +
> > + memcpy(info->data, , sizeof(vtd));
> > + return 0;
> > +}
> > +
> > +static int intel_iommu_domain_get_attr(struct iommu_domain *domain,
> > + enum iommu_attr attr, void *data)
> > +{
> > + switch (attr) {
> > + case DOMAIN_ATTR_NESTING:
> > + {
> > + struct iommu_nesting_info *info =
> > + (struct iommu_nesting_info *)data;
>
> don't you need to hold a device_domain_lock earlier to make sure domain
> attributes can't change behind your back (unmanaged type and nested mode)?
do you mean acquire lock at [1]?
Regards,
Yi Liu
> > +
> > + return intel_iommu_get_nesting_info(domain, info);
> > + }
> > + default:
> > + return -ENOENT;
> > + }
> > +}
> > +
> > /*
> > * Check that the device does not live on an external facing PCI port
RE: [PATCH v6 14/15] vfio: Document dual stage control
Hi Eric, > From: Auger Eric > Sent: Sunday, August 16, 2020 7:52 PM > > Hi Yi, > > On 7/28/20 8:27 AM, Liu Yi L wrote: > > From: Eric Auger > > > > The VFIO API was enhanced to support nested stage control: a bunch of> new > ioctls and usage guideline. > > > > Let's document the process to follow to set up nested mode. > > > > Cc: Kevin Tian > > CC: Jacob Pan > > Cc: Alex Williamson > > Cc: Eric Auger > > Cc: Jean-Philippe Brucker > > Cc: Joerg Roedel > > Cc: Lu Baolu > > Reviewed-by: Stefan Hajnoczi > > Signed-off-by: Eric Auger > > Signed-off-by: Liu Yi L > > --- > > v5 -> v6: > > *) tweak per Eric's comments. > > > > v3 -> v4: > > *) add review-by from Stefan Hajnoczi > > > > v2 -> v3: > > *) address comments from Stefan Hajnoczi > > > > v1 -> v2: > > *) new in v2, compared with Eric's original version, pasid table bind > >and fault reporting is removed as this series doesn't cover them. > >Original version from Eric. > >https://lkml.org/lkml/2020/3/20/700 > > --- > > Documentation/driver-api/vfio.rst | 75 > +++ > > 1 file changed, 75 insertions(+) > > > > diff --git a/Documentation/driver-api/vfio.rst > > b/Documentation/driver-api/vfio.rst > > index f1a4d3c..c0d43f0 100644 > > --- a/Documentation/driver-api/vfio.rst > > +++ b/Documentation/driver-api/vfio.rst > > @@ -239,6 +239,81 @@ group and can access them as follows:: > > /* Gratuitous device reset and go... */ > > ioctl(device, VFIO_DEVICE_RESET); > > > > +IOMMU Dual Stage Control > > + > > + > > +Some IOMMUs support 2 stages/levels of translation. Stage corresponds > > +to the ARM terminology while level corresponds to Intel's terminology. > > +In the following text we use either without distinction. > > + > > +This is useful when the guest is exposed with a virtual IOMMU and some > > +devices are assigned to the guest through VFIO. Then the guest OS can > > +use stage-1 (GIOVA -> GPA or GVA->GPA), while the hypervisor uses stage > > +2 for VM isolation (GPA -> HPA). > > + > > +Under dual stage translation, the guest gets ownership of the stage-1 page > > +tables and also owns stage-1 configuration structures. The hypervisor owns > > +the root configuration structure (for security reason), including stage-2 > > +configuration. > This is only true for vtd. On ARM the stage2 cfg is the Context > Descriptor table (aka PASID table). root cfg only store the GPA of the > CD table. I've a check with you on the meaning of "configuration structures". For Vt-d, does it mean the root table/context table/pasid table? if I'm correct, then how about below description? "Under dual stage translation, the guest gets ownership of the stage-1 configuration structures or page tables. This depends on vendor. The hypervisor owns the root configuration structure (for security reason), including stage-2 configuration." > This works as long as configuration structures and page table > > +formats are compatible between the virtual IOMMU and the physical IOMMU. > > + > > +Assuming the HW supports it, this nested mode is selected by choosing the > > +VFIO_TYPE1_NESTING_IOMMU type through: > > + > > +ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); > > + > > +This forces the hypervisor to use the stage-2, leaving stage-1 available > > +for guest usage. The stage-1 format and binding method are vendor specific > . There are reported in the nesting capability ... got it. "The stage-1 format and binding method are reported in nesting capability." > > +and reported in nesting cap (VFIO_IOMMU_TYPE1_INFO_CAP_NESTING) through > > +VFIO_IOMMU_GET_INFO: > > + > > +ioctl(container->fd, VFIO_IOMMU_GET_INFO, _info); > > + > > +The nesting cap info is available only after NESTING_IOMMU is selected. > > +If underlying IOMMU doesn't support nesting, VFIO_SET_IOMMU fails and > If the underlying got it. > > +userspace should try other IOMMU types. Details of the nesting cap info > > +can be found in Documentation/userspace-api/iommu.rst. > > + > > +The stage-1 page table can be bound to the IOMMU in two methods: directly> > +or indirectly. Direct binding requires userspace to notify VFIO of every > Not sure we shall use this direct/indirect terminology. I don't think > this is part of either ARM or Intel SPEC. > > Suggestion: On Intel, the stage1 page table info are mediated by the > userspace for each PASID. On ARM, the userspace directly passes the GPA > of the whole PASID table. Currently only Intel's binding is supported. got it. this is what we want to say by ditect/indirect terminology. Regards, Yi Liu > > +guest stage-1 page table binding, while indirect binding allows userspace > > +to bind once with an intermediate structure (e.g. PASID table) which > > +indirectly links to guest stage-1 page tables. The actual binding method > > +depends on IOMMU vendor. Currently only the direct binding capability ( > > +IOMMU_NESTING_FEAT_BIND_PGTBL) is supported: >
RE: [PATCH v6 11/15] vfio/type1: Allow invalidating first-level/stage IOMMU cache
Hi Eric,
> From: Auger Eric
> Sent: Sunday, August 16, 2020 7:35 PM
>
> Hi Yi,
>
> On 7/28/20 8:27 AM, Liu Yi L wrote:
> > This patch provides an interface allowing the userspace to invalidate
> > IOMMU cache for first-level page table. It is required when the first
> > level IOMMU page table is not managed by the host kernel in the nested
> > translation setup.
> >
> > Cc: Kevin Tian
> > CC: Jacob Pan
> > Cc: Alex Williamson
> > Cc: Eric Auger
> > Cc: Jean-Philippe Brucker
> > Cc: Joerg Roedel
> > Cc: Lu Baolu
> > Signed-off-by: Liu Yi L
> > Signed-off-by: Eric Auger
> > Signed-off-by: Jacob Pan
> > ---
> > v1 -> v2:
> > *) rename from "vfio/type1: Flush stage-1 IOMMU cache for nesting type"
> > *) rename vfio_cache_inv_fn() to vfio_dev_cache_invalidate_fn()
> > *) vfio_dev_cache_inv_fn() always successful
> > *) remove VFIO_IOMMU_CACHE_INVALIDATE, and reuse
> VFIO_IOMMU_NESTING_OP
> > ---
> > drivers/vfio/vfio_iommu_type1.c | 42
> +
> > include/uapi/linux/vfio.h | 3 +++
> > 2 files changed, 45 insertions(+)
> >
> > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > b/drivers/vfio/vfio_iommu_type1.c index 245436e..bf95a0f 100644
> > --- a/drivers/vfio/vfio_iommu_type1.c
> > +++ b/drivers/vfio/vfio_iommu_type1.c
> > @@ -3056,6 +3056,45 @@ static long vfio_iommu_handle_pgtbl_op(struct
> vfio_iommu *iommu,
> > return ret;
> > }
> >
> > +static int vfio_dev_cache_invalidate_fn(struct device *dev, void
> > +*data) {
> > + struct domain_capsule *dc = (struct domain_capsule *)data;
> > + unsigned long arg = *(unsigned long *)dc->data;
> > +
> > + iommu_uapi_cache_invalidate(dc->domain, dev, (void __user *)arg);
> > + return 0;
> > +}
> > +
> > +static long vfio_iommu_invalidate_cache(struct vfio_iommu *iommu,
> > + unsigned long arg)
> > +{
> > + struct domain_capsule dc = { .data = };
> > + struct iommu_nesting_info *info;
> > + int ret;
> > +
> > + mutex_lock(>lock);
> > + /*
> > +* Cache invalidation is required for any nesting IOMMU,
> So why do we expose the IOMMU_NESTING_FEAT_CACHE_INVLD capability? :-)
it's a stale comment. should be removed. :-)
> > +* so no need to check system-wide PASID support.
> > +*/
> > + info = iommu->nesting_info;
> > + if (!info || !(info->features & IOMMU_NESTING_FEAT_CACHE_INVLD)) {
> > + ret = -EOPNOTSUPP;
> > + goto out_unlock;
> > + }
> > +
> > + ret = vfio_get_nesting_domain_capsule(iommu, );
> > + if (ret)
> > + goto out_unlock;
> > +
> > + iommu_group_for_each_dev(dc.group->iommu_group, ,
> > +vfio_dev_cache_invalidate_fn);
> > +
> > +out_unlock:
> > + mutex_unlock(>lock);
> > + return ret;
> > +}
> > +
> > static long vfio_iommu_type1_nesting_op(struct vfio_iommu *iommu,
> > unsigned long arg)
> > {
> > @@ -3078,6 +3117,9 @@ static long vfio_iommu_type1_nesting_op(struct
> vfio_iommu *iommu,
> > case VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL:
> > ret = vfio_iommu_handle_pgtbl_op(iommu, false, arg + minsz);
> > break;
> > + case VFIO_IOMMU_NESTING_OP_CACHE_INVLD:
> > + ret = vfio_iommu_invalidate_cache(iommu, arg + minsz);
> > + break;
> > default:
> > ret = -EINVAL;
> > }
> > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > index 9501cfb..48e2fb5 100644
> > --- a/include/uapi/linux/vfio.h
> > +++ b/include/uapi/linux/vfio.h
> > @@ -1225,6 +1225,8 @@ struct vfio_iommu_type1_pasid_request {
> > * +-+---+
> > * | UNBIND_PGTBL| struct iommu_gpasid_bind_data|
> > *
> > +-+---+
> > + * | CACHE_INVLD | struct iommu_cache_invalidate_info |
> > + *
> > + +-+---+
> > *
> > * returns: 0 on success, -errno on failure.
> > */
> > @@ -1237,6 +1239,7 @@ struct vfio_iommu_type1_nesting_op {
> >
> > #define VFIO_IOMMU_NESTING_OP_BIND_PGTBL (0)
> > #define VFIO_IOMMU_NESTING_OP_UNBIND_PGTBL (1)
> > +#define VFIO_IOMMU_NESTING_OP_CACHE_INVLD (2)
> According to my previous comment, you may refine VFIO_NESTING_OP_MASK too
yes, I've noticed it. also replied in patch 10/15.
Regards,
Yi Liu
> Thanks
>
> Eric
> >
> > #define VFIO_IOMMU_NESTING_OP _IO(VFIO_TYPE, VFIO_BASE + 19)
> >
> >
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 10/15] vfio/type1: Support binding guest page tables to PASID
Hi Eric,
> From: Auger Eric
> Sent: Sunday, August 16, 2020 7:29 PM
>
> Hi Yi,
>
> On 7/28/20 8:27 AM, Liu Yi L wrote:
> > Nesting translation allows two-levels/stages page tables, with 1st level
> > for guest translations (e.g. GVA->GPA), 2nd level for host translations
> > (e.g. GPA->HPA). This patch adds interface for binding guest page tables
> > to a PASID. This PASID must have been allocated by the userspace before
> > the binding request.
> >
> > Cc: Kevin Tian
> > CC: Jacob Pan
> > Cc: Alex Williamson
> > Cc: Eric Auger
> > Cc: Jean-Philippe Brucker
> > Cc: Joerg Roedel
> > Cc: Lu Baolu
> > Signed-off-by: Jean-Philippe Brucker
> > Signed-off-by: Liu Yi L
> > Signed-off-by: Jacob Pan
> > ---
> > v5 -> v6:
> > *) dropped vfio_find_nesting_group() and add
> > vfio_get_nesting_domain_capsule().
> >per comment from Eric.
> > *) use iommu_uapi_sva_bind/unbind_gpasid() and iommu_sva_unbind_gpasid() in
> >linux/iommu.h for userspace operation and in-kernel operation.
> >
> > v3 -> v4:
> > *) address comments from Alex on v3
> >
> > v2 -> v3:
> > *) use __iommu_sva_unbind_gpasid() for unbind call issued by VFIO
> >https://lore.kernel.org/linux-iommu/1592931837-58223-6-git-send-email-
> jacob.jun@linux.intel.com/
> >
> > v1 -> v2:
> > *) rename subject from "vfio/type1: Bind guest page tables to host"
> > *) remove VFIO_IOMMU_BIND, introduce VFIO_IOMMU_NESTING_OP to support
> bind/
> >unbind guet page table
> > *) replaced vfio_iommu_for_each_dev() with a group level loop since this
> >series enforces one group per container w/ nesting type as start.
> > *) rename vfio_bind/unbind_gpasid_fn() to vfio_dev_bind/unbind_gpasid_fn()
> > *) vfio_dev_unbind_gpasid() always successful
> > *) use vfio_mm->pasid_lock to avoid race between PASID free and page table
> >bind/unbind
> > ---
> > drivers/vfio/vfio_iommu_type1.c | 161
>
> > drivers/vfio/vfio_pasid.c | 26 +++
> > include/linux/vfio.h| 20 +
> > include/uapi/linux/vfio.h | 31
> > 4 files changed, 238 insertions(+)
> >
> > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > b/drivers/vfio/vfio_iommu_type1.c
> > index ea89c7c..245436e 100644
> > --- a/drivers/vfio/vfio_iommu_type1.c
> > +++ b/drivers/vfio/vfio_iommu_type1.c
> > @@ -149,6 +149,36 @@ struct vfio_regions {
> > #define DIRTY_BITMAP_PAGES_MAX ((u64)INT_MAX)
> > #define DIRTY_BITMAP_SIZE_MAX
> DIRTY_BITMAP_BYTES(DIRTY_BITMAP_PAGES_MAX)
> >
> > +struct domain_capsule {
> > + struct vfio_group *group;
> > + struct iommu_domain *domain;
> > + void *data;
> You may add a bool indicating whether the data is a user pointer or the
> direct IOMMU UAPI struct.
yes, it is helpful.
> > +};
> > +
> > +/* iommu->lock must be held */
> > +static int vfio_get_nesting_domain_capsule(struct vfio_iommu *iommu,
> > + struct domain_capsule *dc)
> I would rename the function into vfio_prepare_nesting_domain_capsule
got it. :-)
> > +{
> > + struct vfio_domain *domain = NULL;
> > + struct vfio_group *group = NULL;
> > +
> > + if (!iommu->nesting_info)
> > + return -EINVAL;
> > +
> > + /*
> > +* Only support singleton container with nesting type.
> > +* If nesting_info is non-NULL, the conatiner should
> s/should/is here and below
> s/conatiner/container
got it. thanks.
> > +* be non-empty. Also domain should be non-empty.
> > +*/
> > + domain = list_first_entry(>domain_list,
> > + struct vfio_domain, next);
> > + group = list_first_entry(>group_list,
> > +struct vfio_group, next);
> > + dc->group = group;
> > + dc->domain = domain->domain;
> dc->user = true;?
yep.
> > + return 0;
> > +}
> > +
> > static int put_pfn(unsigned long pfn, int prot);
> >
> > static struct vfio_group *vfio_iommu_find_iommu_group(struct vfio_iommu
> *iommu,
> > @@ -2349,6 +2379,48 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu
> *iommu,
> > return ret;
> > }
> >
> > +static int vfio_dev_bind_gpasid_fn(struct device *dev, void *data)
> > +{
> > + struct domain_capsule *dc = (struct domain_capsule *)data;
> > + unsigned long arg = *(unsigned long *)dc->data;
> > +
> > + return iommu_uapi_sva_bind_gpasid(dc->domain, dev, (void __user *)arg);
> > +}
> > +
> > +static int vfio_dev_unbind_gpasid_fn(struct device *dev, void *data)
> > +{
> > + struct domain_capsule *dc = (struct domain_capsule *)data;
> > + unsigned long arg = *(unsigned long *)dc->data;
> > +
> > + iommu_uapi_sva_unbind_gpasid(dc->domain, dev, (void __user *)arg);
> > + return 0;
> > +}
> > +
> > +static int __vfio_dev_unbind_gpasid_fn(struct device *dev, void *data)
> can be removed if dc->user flag gets added
yes.
> > +{
> > + struct domain_capsule *dc = (struct domain_capsule *)data;
> > + struct iommu_gpasid_bind_data *unbind_data =
> > +
