Re: [RFC/PATCH 0/9] IOMMU probe deferral support
On 05/14/2015 04:00 PM, Laurent Pinchart wrote: Hello, This patch series attempts to implement support for deferring probe of both IOMMU drivers and bus master drivers. The relationship between bus masters and IOMMUs creates a strong ordering during initialization of devices. As in the general case IOMMUs are hidden behind the DMA mapping API, IOMMU support relies on the automatic setup of DMA operations without any direct intervention of bus master drivers. DMA operations are set up when platform devices are added to the system. This requires IOMMUs to be available at that time. On systems where ordering of device add and probe can't be guaranteed (such as, but not limited to, DT-based systems) this caused incorrect DMA operation setup. This has been addressed by a patches series [1] that introduced a DT-based early registration mechanism for IOMMUs. However, that mechanism fails to address all issues. Various dependencies exist between IOMMU devices and other devices, in particular on clocks and on power domains (as mentioned by Marek in [2]). While there are mechanisms to handle some of them without probe deferral (for instance by using the OF_DECLARE macros to register clock drivers), generalizing those mechanisms would essentially recreate a probe ordering mechanism similar to link order probe ordering and couldn't really scale. Additionally, IOMMUs could also be present hot-pluggable devices and depend on resources that are thus hot-plugged. OF_DECLARE wouldn't help in that case. For all those reasons probe deferral for IOMMUs has been considered as desired if it can be implemented cleanly. For more in-depth information see [3]. This RFC series is a first attempt at implementing IOMMU probe deferral support cleanly. The core idea is to move setup of DMA operations from device add time to device probe time, implemented in patch 6/9. It could be possible to move setup of other DMA parameters (namely masks and offset) to probe time as well, but that change would be more intrusive and has a higher risk of introducing regressions. For that reason I've decided to keep DMA masks and offset setup at device add time and thus split DMA configuration in masks and operations (patch 5/9). This can be revisited if we decide that the DMA mapping API shouldn't require masks and offset to be set before probe time. Patch 8/9 then defers probe of bus master drivers when required IOMMUs are not available yet. This requires knowing when a failed IOMMU lookup should be considered as permanent or temporary. I've reused the OF_DECLARE_IOMMU for this purpose, considering that the presence of a driver compatible with the IOMMU DT node indicates that the failure is temporary and probing of the bus master device should be deferred. Note that only IOMMU drivers using the recent .of_xlate() mechanism for DT-based IOMMU reference can cause probe deferral of bus master devices. The .add_device() mechanism isn't supported in this case. As an example I've converted the ipmmu-vmsa driver to the new API in patch 9/9. At this point many enhancements are possible, but I'd like to receive feedback on the proposed approach before basing more patches on this series. One particular point I would like to address (or see being addressed) in the future is the use of struct iommu_ops with of_iommu_get_ops() and of_iommu_set_ops(). I believe we should introduct a struct iommu and register IOMMU instances instead of IOMMU operations. That should bring us one step closer to removing bus_set_iommu(). [1] http://www.spinics.net/lists/arm-kernel/msg382787.html [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-February/323238.html [3] https://lkml.org/lkml/2015/2/16/345 Laurent Pinchart (9): arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops() arm: dma-mapping: Support IOMMU mappings spanning the full 32 bits range of: dma: Move range size workaround to of_dma_get_range() of: dma: Make of_dma_deconfigure() public of: dma: Split of_configure_dma() into mask and ops configuration drivers: platform: Configure dma operations at probe time iommu: of: Document the of_iommu_configure() function iommu: of: Handle IOMMU lookup failure with deferred probing or error iommu/ipmmu-vmsa: Use DT-based instantiation arch/arm/include/asm/dma-iommu.h | 2 +- arch/arm/mm/dma-mapping.c| 21 +++-- drivers/base/platform.c | 9 ++ drivers/iommu/ipmmu-vmsa.c | 189 +-- drivers/iommu/of_iommu.c | 29 +- drivers/of/address.c | 20 - drivers/of/device.c | 77 ++-- drivers/of/of_pci.c | 3 +- drivers/of/platform.c| 16 ++-- include/linux/of_device.h| 14 ++- 10 files changed, 195 insertions(+), 185 deletions(-) I no longer have hardware to test this on but the entire approach looks reasonable to me. Reviewed-by: Laura Abbott
Re: [PATCH v7 00/25] Exynos SYSMMU (IOMMU) integration with DT andDMA-mapping subsystem
On 23.05.2015 12:56, Kukjin Kim wrote: On 05/20/15 16:31, Javier Martinez Canillas wrote: Hello Marek, On Tue, May 19, 2015 at 3:20 PM, Marek Szyprowski m.szyprow...@samsung.com wrote: Hello Everyone, This is yet another attempt to get Exynos SYSMMU driver with integrated with IOMMU DMA-mapping subsystems. This version includes minor fixes suggested by Joerg Roedel, Cho KyongHo and Robin Murphy. All patches are also available in the following git repository: https://git.linaro.org/people/marek.szyprowski/linux-srpol.git branch v4.1-exynos-iommu-v7. My plan for merging this patchset is as follows: - DTS changes and power domain changes should go via Samsung tree to avoid conflicts with other pending DTS patches - all exynos-iommu patches and dma-mapping/reserved iommu region should go via IOMMU tree, - all Exynos DRM patches should go via DRM/Exynos tree. There are no build cross-subsystem dependencies and IOMMU on Exynos is already nonfunctional (and disabled in defconfig), so merging patches in parts doesn't break anything. There are knowns issues with Exynos DRM driver and IOMMU support (i.e. Xorg freeze reported by Javier Martinez Canillas: http://www.spinics.net/lists/linux-samsung-soc/msg44350.html ). They will be handled by a separate fixes to Exynos DRM drivers. This patchset fixes only those issues in Exynos DRM FIMD driver, which prevents booting to console. I tested your v7 using exynos_defconfig + CONFIG_EXYNOS_IOMMU on both Exynos5250 Snow and Exynos5420 Peach Pit Chromebooks. I can only reproduce the complete system hang on Xorg if I use the xf86-video-armsoc DDX. If I use the -fbdev DDX instead, X starts correctly and the system does not freeze. Also, this only happens on Exynos5420 Peach Pit, Exynos5250 Snow is working correctly even when using the -armsoc DDX. So as you said, this is a separate issue with the Exynos DRM driver IOMMU support and is not related to your series. For the whole series: Tested-by: Javier Martinez Canillas javier.marti...@collabora.co.uk Thanks for your test on the boards. I'll sort out the arch/ side changes into samsung tree in this weekend. Hi Kukjin, How is the progress of sorting out this patchset? There will be some conflicts between this and DTS-label-rework so maybe you want something to be rebased? Best regards, Krzysztpf ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 2/4] iommu: Implement common IOMMU ops for DMA mapping
Hi Robin,
Thanks.
While we test venc in v4l2, we get a problem:
When we enter the funtion[0], it will be break unexpectedly in the
funcion[1] while the offset of sg table is not zero. It is ok if the
offset is zero. Then I add more log in dma-iommu.c, please help check
below.
All we tested it based on dma v2. and have not tested it on v3 yet.
The code of iommu-map-sg seems the same. if it's fixed in v3, I'm very
sorry. The map_sg in mtk-iommu use default_iommu_map_sg.
Any question please tell me, Thanks very much.
[0]http://lxr.free-electrons.com/source/drivers/media/v4l2-core/videobuf2-dma-contig.c#L564
[1]http://lxr.free-electrons.com/source/drivers/media/v4l2-core/videobuf2-dma-contig.c#L70
On Wed, 2015-05-27 at 15:09 +0100, Robin Murphy wrote:
Taking inspiration from the existing arch/arm code, break out some
generic functions to interface the DMA-API to the IOMMU-API. This will
do the bulk of the heavy lifting for IOMMU-backed dma-mapping.
Signed-off-by: Robin Murphy robin.mur...@arm.com
---
drivers/iommu/Kconfig | 7 +
drivers/iommu/Makefile| 1 +
drivers/iommu/dma-iommu.c | 560
++
include/linux/dma-iommu.h | 94
4 files changed, 662 insertions(+)
create mode 100644 drivers/iommu/dma-iommu.c
create mode 100644 include/linux/dma-iommu.h
[snip]
+static int __finalise_sg(struct device *dev, struct scatterlist *sg, int
nents,
+ dma_addr_t dma_addr)
+{
+ struct scatterlist *s, *seg = sg;
+ unsigned long seg_mask = dma_get_seg_boundary(dev);
+ unsigned int max_len = dma_get_max_seg_size(dev);
+ unsigned int seg_len = 0, seg_dma = 0;
+ int i, count = 1;
+
+ for_each_sg(sg, s, nents, i) {
+ /* Un-swizzling the fields here, hence the naming mismatch */
+ unsigned int s_offset = sg_dma_address(s);
+ unsigned int s_length = sg_dma_len(s);
+ unsigned int s_dma_len = s-length;
+
+ s-offset = s_offset;
+ s-length = s_length;
+ sg_dma_address(s) = DMA_ERROR_CODE;
+ sg_dma_len(s) = 0;
+
+ if (seg_len (seg_dma + seg_len == dma_addr + s_offset)
+ (seg_len + s_dma_len = max_len)
+ ((seg_dma seg_mask) = seg_mask - (seg_len + s_length))
+) {
+ sg_dma_len(seg) += s_dma_len;
+ } else {
+ if (seg_len) {
+ seg = sg_next(seg);
+ count++;
+ }
+ sg_dma_len(seg) = s_dma_len;
+ sg_dma_address(seg) = dma_addr + s_offset;
Here the value of sg_dma_address have added s_offset, but
sg_dma_len(seg) still is s_dma_len.
In the first loop, s_dma_len is from s-length which is alignd by
s_length = iova_align(iovad, s_length + s_offset); in
the interface iommu_dma_map_sg.
+
+ seg_len = s_offset;
+ seg_dma = dma_addr + s_offset;
+ }
+ seg_len += s_length;
+ dma_addr += s_dma_len;
+ }
+ return count;
+}
+
+static void __invalidate_sg(struct scatterlist *sg, int nents)
+{
+ struct scatterlist *s;
+ int i;
+
+ for_each_sg(sg, s, nents, i) {
+ if (sg_dma_address(s) != DMA_ERROR_CODE)
+ s-offset = sg_dma_address(s);
+ if (sg_dma_len(s))
+ s-length = sg_dma_len(s);
+ sg_dma_address(s) = DMA_ERROR_CODE;
+ sg_dma_len(s) = 0;
+ }
+}
+
+int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg,
+ int nents, int prot, bool coherent)
+{
+ struct iommu_dma_domain *dom = arch_get_dma_domain(dev);
+ struct iova_domain *iovad = dom-iovad;
+ struct iova *iova;
+ struct scatterlist *s;
+ dma_addr_t dma_addr;
+ size_t iova_len = 0;
+ int i;
+
+ /*
+ * Work out how much IOVA space we need, and align the segments to
+ * IOVA granules for the IOMMU driver to handle. With some clever
+ * trickery we can modify the list in a reversible manner.
+ */
+ for_each_sg(sg, s, nents, i) {
+ size_t s_offset = iova_offset(iovad, s-offset);
+ size_t s_length = s-length;
+
+ sg_dma_address(s) = s-offset;
+ sg_dma_len(s) = s_length;
+ s-offset -= s_offset;
+ s_length = iova_align(iovad, s_length + s_offset);
+ s-length = s_length;
At the begging, s-length is the length of valid data. but it's aligned
here.
+
+ iova_len += s_length;
+ }
+
+ iova = __alloc_iova(dev, iova_len, coherent);
+ if (!iova)
+ goto out_restore_sg;
+
+ /*
+ * We'll leave any physical concatenation to the IOMMU driver's
+ * implementation - it
Re: [RFC/PATCH 7/9] iommu: of: Document the of_iommu_configure() function
On Fri, May 15, 2015 at 12:00:08AM +0100, Laurent Pinchart wrote:
The function isn't trivial, document its behaviour.
Signed-off-by: Laurent Pinchart laurent.pinchart+rene...@ideasonboard.com
---
drivers/iommu/of_iommu.c | 13 +
1 file changed, 13 insertions(+)
Thanks for doing this.
Acked-by: Will Deacon will.dea...@arm.com
Will
diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index 43429ab62228..b922ed4f9fb3 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -133,6 +133,19 @@ struct iommu_ops *of_iommu_get_ops(struct device_node
*np)
return ops;
}
+/**
+ * of_iommu_configure - Configure and return the IOMMU for a device
+ * @dev: device for which to configure the IOMMU
+ * @master_np: device node of the bus master connected to the IOMMU
+ *
+ * The master_np parameter specifies the device node of the bus master seen
by
+ * the IOMMU. This is usually the device node of the dev device, but can be
the
+ * device node of a bridge when the device is dynamically discovered and
+ * instantiated and thus has no device node (such as PCI devices for
instance).
+ *
+ * Return a pointer to the iommu_ops for the device, NULL if the device isn't
+ * connected to an IOMMU, or a negative value if an error occurs.
+ */
struct iommu_ops *of_iommu_configure(struct device *dev,
struct device_node *master_np)
{
--
2.3.6
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC/PATCH 4/9] of: dma: Make of_dma_deconfigure() public
(sorry for the mid-review delay, I got side-tracked yesterday evening)
On Fri, May 15, 2015 at 12:00:05AM +0100, Laurent Pinchart wrote:
As part of moving DMA initializing to probe time the
of_dma_deconfigure() function will need to be called from different
source files. Make it public and move it to drivers/of/device.c where
the of_dma_configure() function is.
Signed-off-by: Laurent Pinchart laurent.pinchart+rene...@ideasonboard.com
---
drivers/of/device.c | 12
drivers/of/platform.c | 5 -
include/linux/of_device.h | 3 +++
3 files changed, 15 insertions(+), 5 deletions(-)
Acked-by: Will Deacon will.dea...@arm.com
Cheers,
Will
diff --git a/drivers/of/device.c b/drivers/of/device.c
index 530aa1ed3e1b..f1b84f464fe1 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -135,6 +135,18 @@ void of_dma_configure(struct device *dev, struct
device_node *np)
}
EXPORT_SYMBOL_GPL(of_dma_configure);
+/**
+ * of_dma_deconfigure - Clean up DMA configuration
+ * @dev: Device for which to clean up DMA configuration
+ *
+ * Clean up all configuration performed by of_dma_configure_ops() and free
all
+ * resources that have been allocated.
+ */
+void of_dma_deconfigure(struct device *dev)
+{
+ arch_teardown_dma_ops(dev);
+}
+
int of_device_register(struct platform_device *pdev)
{
device_initialize(pdev-dev);
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index a01f57c9e34e..7a660c79ff84 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -149,11 +149,6 @@ struct platform_device *of_device_alloc(struct
device_node *np,
}
EXPORT_SYMBOL(of_device_alloc);
-static void of_dma_deconfigure(struct device *dev)
-{
- arch_teardown_dma_ops(dev);
-}
-
/**
* of_platform_device_create_pdata - Alloc, initialize and register an
of_device
* @np: pointer to node to create device for
diff --git a/include/linux/of_device.h b/include/linux/of_device.h
index 22801b10cef5..6710807b0653 100644
--- a/include/linux/of_device.h
+++ b/include/linux/of_device.h
@@ -54,6 +54,7 @@ static inline struct device_node
*of_cpu_device_node_get(int cpu)
}
void of_dma_configure(struct device *dev, struct device_node *np);
+void of_dma_deconfigure(struct device *dev);
#else /* CONFIG_OF */
static inline int of_driver_match_device(struct device *dev,
@@ -93,6 +94,8 @@ static inline struct device_node
*of_cpu_device_node_get(int cpu)
}
static inline void of_dma_configure(struct device *dev, struct device_node
*np)
{}
+static inline void of_dma_deconfigure(struct device *dev)
+{}
#endif /* CONFIG_OF */
#endif /* _LINUX_OF_DEVICE_H */
--
2.3.6
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC/PATCH 5/9] of: dma: Split of_configure_dma() into mask and ops configuration
On Fri, May 15, 2015 at 12:00:06AM +0100, Laurent Pinchart wrote:
The of_configure_dma() function configures both the DMA masks and ops.
Moving DMA ops configuration to probe time would thus also delay
configuration of the DMA masks, which might not be safe. To avoid issues
split the configuration in two to allow keeping masks configuration at
device add time and move ops configuration to device probe time.
Signed-off-by: Laurent Pinchart laurent.pinchart+rene...@ideasonboard.com
---
drivers/of/device.c | 48
++-
drivers/of/of_pci.c | 3 ++-
drivers/of/platform.c | 6 --
include/linux/of_device.h | 11 +--
4 files changed, 50 insertions(+), 18 deletions(-)
diff --git a/drivers/of/device.c b/drivers/of/device.c
index f1b84f464fe1..3cb3f78a6d13 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -70,7 +70,7 @@ int of_device_add(struct platform_device *ofdev)
}
/**
- * of_dma_configure - Setup DMA configuration
+ * of_dma_configure - Setup DMA masks and offset
* @dev: Device to apply DMA configuration
* @np: Pointer to OF node having DMA configuration
*
@@ -81,13 +81,11 @@ int of_device_add(struct platform_device *ofdev)
* can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
* to fix up DMA configuration.
*/
-void of_dma_configure(struct device *dev, struct device_node *np)
+void of_dma_configure_masks(struct device *dev, struct device_node *np)
{
- u64 dma_addr, paddr, size;
- int ret;
- bool coherent;
+ u64 dma_addr, paddr, size, range_mask;
unsigned long offset;
- struct iommu_ops *iommu;
+ int ret;
/*
* Set default coherent_dma_mask to 32 bit. Drivers are expected to
@@ -105,9 +103,10 @@ void of_dma_configure(struct device *dev, struct
device_node *np)
ret = of_dma_get_range(np, dma_addr, paddr, size);
if (ret 0) {
- dma_addr = offset = 0;
- size = dev-coherent_dma_mask + 1;
+ range_mask = dev-coherent_dma_mask + 1;
Hmm, do we still want this '+ 1'...
+ offset = 0;
} else {
+ range_mask = DMA_BIT_MASK(ilog2(dma_addr + size));
offset = PFN_DOWN(paddr - dma_addr);
dev_dbg(dev, dma_pfn_offset(%#08lx)\n, offset);
}
@@ -118,10 +117,31 @@ void of_dma_configure(struct device *dev, struct
device_node *np)
* Limit coherent and dma mask based on size and default mask
* set by the driver.
*/
- dev-coherent_dma_mask = min(dev-coherent_dma_mask,
- DMA_BIT_MASK(ilog2(dma_addr + size)));
- *dev-dma_mask = min((*dev-dma_mask),
- DMA_BIT_MASK(ilog2(dma_addr + size)));
+ dev-coherent_dma_mask = min(dev-coherent_dma_mask, range_mask);
... if we now compare directly with the mask here? In fact, this setting
of the coherent_dma_mask could just belong in the else clause, no?
+ *dev-dma_mask = min((*dev-dma_mask), range_mask);
Similar comment here, but we need to do this for both cases I guess.
+}
+EXPORT_SYMBOL_GPL(of_dma_configure_masks);
+
+/**
+ * of_dma_configure_ops - Setup DMA operations
+ * @dev: Device to apply DMA configuration
+ * @np: Pointer to OF node having DMA configuration
+ *
+ * Try to get devices's DMA configuration from DT and update it
Weird spelling.
Will
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC/PATCH 8/9] iommu: of: Handle IOMMU lookup failure with deferred probing or error
Hi Laurent, On Fri, May 15, 2015 at 12:00:09AM +0100, Laurent Pinchart wrote: Failures to look up an IOMMU when parsing the DT iommus property need to be handled separately from the .of_xlate() failures to support deferred probing. The lack of a registered IOMMU can be caused by the lack of a driver for the IOMMU, the IOMMU device probe not having been performed yet, having been deferred, or having failed. The first case occurs when the device tree describes the bus master and IOMMU topology correctly but no device driver exists for the IOMMU yet or the device driver has not been compiled in. Return NULL, the caller will configure the device without an IOMMU. The second and third cases are handled by deferring the probe of the bus master device which will eventually get reprobed after the IOMMU. The last case is currently handled by deferring the probe of the bus master device as well. A mechanism to either configure the bus master device without an IOMMU or to fail the bus master device probe depending on whether the IOMMU is optional or mandatory would be a good enhancement. I appreciate that you're just looking to handle early initialisation failures here, but do you have any thoughts on how to deal with failures later on when e.g. the DMA-mapping API is trying to create IOMMU domains. One potential problem I foresee is if we try to add all devices to a common DMA domain, we may get -ENOSPC-style failures due to limited resources on the IOMMU. In this case, we'd probably want to fall-back to non-IOMMU DMA ops, but that in-turn could have consequences on things like dma-coherent. It's all a bit murky, so I'd be glad to hear any thoughts you might have around this. Anyway, this patch looks fine: Acked-by: Will Deacon will.dea...@arm.com but we should consider how all of this will get used too. Will ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 01/22] iommu: Remove function name from pr_fmt()
From: Joerg Roedel jroe...@suse.de Including the function name is only useful for debugging messages. They don't belong into other messages from the iommu core. Signed-off-by: Joerg Roedel jroe...@suse.de --- drivers/iommu/iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index d4f527e..c31bfd0 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -16,7 +16,7 @@ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ -#define pr_fmt(fmt)%s: fmt, __func__ +#define pr_fmt(fmt)iommu: fmt #include linux/device.h #include linux/kernel.h -- 1.9.1 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 00/22 v2] Introduce default domains for iommu groups
Hi, here is the second version of my patch-set to introduce default domains into the iommu core. This time it has a lot more patches, mostly because I added a proof of concept implementation by converting the AMD IOMMU driver to make use of it. Converting the first driver to the new concept triggered a lot of changes and extensions in the patch-set to fit all the needs of a more complex iommu driver. Converting other drivers might need further changes, but that is something for the future. A major change is that now the default domain has to be allocated by the code that allocates the iommu group. For PCI devices this happens in the IOMMU core, but drivers allocating the group on their own can now implement a policy that fits their needs (e.g. not allocate one domain per group but let multiple groups share one domain). The new core code is changed in a way to stay compatible with the old behavior. All IOMMU drivers that are not yet converted to default domains should behave as without this patch-set. I tested the patches on AMD systems with IOMMUv1 and IOMMUv2, did boot-testing and also successfully tested device assignment. I did the same tests on an Intel VT-d machine to make sure the changes do not introduce regressions on unconverted drivers. If you prefer a git branch for testing, please look here: git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux.git iommu-default-domains Any feedback and further testing welcome! Thanks, Joerg Joerg Roedel (22): iommu: Remove function name from pr_fmt() iommu: Add a few printk messages to group handling code iommu: Propagate error in add_iommu_group iommu: Clean up after a failed bus initialization iommu: Call remove_device call-back after driver release iommu: Allocate a default domain for iommu groups iommu: Limit iommu_attach/detach_device to devices with their own group iommu: Make sure a device is always attached to a domain iommu: Add iommu_get_domain_for_dev function iommu: Introduce direct mapped region handling iommu: Create direct mappings in default domains iommu: Add function to query the default domain of a group iommu: Introduce iommu_request_dm_for_dev() iommu/amd: Implement dm_region call-backs iommu/amd: Use default domain if available for DMA-API iommu/amd: Implement add_device and remove_device iommu/amd: Support IOMMU_DOMAIN_DMA type allocation iommu/amd: Support IOMMU_DOMAIN_IDENTITY type allocation iommu/amd: Put IOMMUv2 devices in a direct mapped domain iommu/amd: Get rid of device_dma_ops_init() iommu/amd: Remove unused fields from struct dma_ops_domain iommu/amd: Propagate errors from amd_iommu_init_api drivers/iommu/amd_iommu.c | 568 drivers/iommu/amd_iommu_init.c | 34 +-- drivers/iommu/amd_iommu_proto.h | 2 +- drivers/iommu/amd_iommu_types.h | 11 - drivers/iommu/iommu.c | 369 -- include/linux/iommu.h | 44 6 files changed, 562 insertions(+), 466 deletions(-) -- 1.9.1 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 08/22] iommu: Make sure a device is always attached to a domain
From: Joerg Roedel jroe...@suse.de
Make use of the default domain and re-attach a device to it
when it is detached from another domain. Also enforce that a
device has to be in the default domain before it can be
attached to a different domain.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 83 ++-
1 file changed, 75 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index adeedd2..7bce522 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -52,6 +52,7 @@ struct iommu_group {
char *name;
int id;
struct iommu_domain *default_domain;
+ struct iommu_domain *domain;
};
struct iommu_device {
@@ -78,6 +79,12 @@ struct iommu_group_attribute iommu_group_attr_##_name =
\
static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
unsigned type);
+static int __iommu_attach_device(struct iommu_domain *domain,
+struct device *dev);
+static int __iommu_attach_group(struct iommu_domain *domain,
+ struct iommu_group *group);
+static void __iommu_detach_group(struct iommu_domain *domain,
+struct iommu_group *group);
static ssize_t iommu_group_attr_show(struct kobject *kobj,
struct attribute *__attr, char *buf)
@@ -376,6 +383,8 @@ rename:
mutex_lock(group-mutex);
list_add_tail(device-list, group-devices);
+ if (group-domain)
+ __iommu_attach_device(group-domain, dev);
mutex_unlock(group-mutex);
/* Notify any listeners about change to group. */
@@ -455,19 +464,30 @@ static int iommu_group_device_count(struct iommu_group
*group)
* The group-mutex is held across callbacks, which will block calls to
* iommu_group_add/remove_device.
*/
-int iommu_group_for_each_dev(struct iommu_group *group, void *data,
-int (*fn)(struct device *, void *))
+static int __iommu_group_for_each_dev(struct iommu_group *group, void *data,
+ int (*fn)(struct device *, void *))
{
struct iommu_device *device;
int ret = 0;
- mutex_lock(group-mutex);
list_for_each_entry(device, group-devices, list) {
ret = fn(device-dev, data);
if (ret)
break;
}
+ return ret;
+}
+
+
+int iommu_group_for_each_dev(struct iommu_group *group, void *data,
+int (*fn)(struct device *, void *))
+{
+ int ret;
+
+ mutex_lock(group-mutex);
+ ret = __iommu_group_for_each_dev(group, data, fn);
mutex_unlock(group-mutex);
+
return ret;
}
EXPORT_SYMBOL_GPL(iommu_group_for_each_dev);
@@ -1013,7 +1033,7 @@ int iommu_attach_device(struct iommu_domain *domain,
struct device *dev)
if (iommu_group_device_count(group) != 1)
goto out_unlock;
- ret = __iommu_attach_device(domain, dev);
+ ret = __iommu_attach_group(domain, group);
out_unlock:
mutex_unlock(group-mutex);
@@ -1048,7 +1068,7 @@ void iommu_detach_device(struct iommu_domain *domain,
struct device *dev)
goto out_unlock;
}
- __iommu_detach_device(domain, dev);
+ __iommu_detach_group(domain, group);
out_unlock:
mutex_unlock(group-mutex);
@@ -1073,10 +1093,31 @@ static int iommu_group_do_attach_device(struct device
*dev, void *data)
return __iommu_attach_device(domain, dev);
}
+static int __iommu_attach_group(struct iommu_domain *domain,
+ struct iommu_group *group)
+{
+ int ret;
+
+ if (group-default_domain group-domain != group-default_domain)
+ return -EBUSY;
+
+ ret = __iommu_group_for_each_dev(group, domain,
+iommu_group_do_attach_device);
+ if (ret == 0)
+ group-domain = domain;
+
+ return ret;
+}
+
int iommu_attach_group(struct iommu_domain *domain, struct iommu_group *group)
{
- return iommu_group_for_each_dev(group, domain,
- iommu_group_do_attach_device);
+ int ret;
+
+ mutex_lock(group-mutex);
+ ret = __iommu_attach_group(domain, group);
+ mutex_unlock(group-mutex);
+
+ return ret;
}
EXPORT_SYMBOL_GPL(iommu_attach_group);
@@ -1089,9 +1130,35 @@ static int iommu_group_do_detach_device(struct device
*dev, void *data)
return 0;
}
+static void __iommu_detach_group(struct iommu_domain *domain,
+struct iommu_group *group)
+{
+ int ret;
+
+ if (!group-default_domain) {
+ __iommu_group_for_each_dev(group, domain,
+ iommu_group_do_detach_device);
+
[PATCH 07/22] iommu: Limit iommu_attach/detach_device to devices with their own group
From: Joerg Roedel jroe...@suse.de
This patch changes the behavior of the iommu_attach_device
and iommu_detach_device functions. With this change these
functions only work on devices that have their own group.
For all other devices the iommu_group_attach/detach
functions must be used.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 71 ---
1 file changed, 67 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index fbfc015..adeedd2 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -433,6 +433,17 @@ void iommu_group_remove_device(struct device *dev)
}
EXPORT_SYMBOL_GPL(iommu_group_remove_device);
+static int iommu_group_device_count(struct iommu_group *group)
+{
+ struct iommu_device *entry;
+ int ret = 0;
+
+ list_for_each_entry(entry, group-devices, list)
+ ret++;
+
+ return ret;
+}
+
/**
* iommu_group_for_each_dev - iterate over each device in the group
* @group: the group
@@ -970,7 +981,8 @@ void iommu_domain_free(struct iommu_domain *domain)
}
EXPORT_SYMBOL_GPL(iommu_domain_free);
-int iommu_attach_device(struct iommu_domain *domain, struct device *dev)
+static int __iommu_attach_device(struct iommu_domain *domain,
+struct device *dev)
{
int ret;
if (unlikely(domain-ops-attach_dev == NULL))
@@ -981,9 +993,38 @@ int iommu_attach_device(struct iommu_domain *domain,
struct device *dev)
trace_attach_device_to_domain(dev);
return ret;
}
+
+int iommu_attach_device(struct iommu_domain *domain, struct device *dev)
+{
+ struct iommu_group *group;
+ int ret;
+
+ group = iommu_group_get(dev);
+ /* FIXME: Remove this when groups a mandatory for iommu drivers */
+ if (group == NULL)
+ return __iommu_attach_device(domain, dev);
+
+ /*
+* We have a group - lock it to make sure the device-count doesn't
+* change while we are attaching
+*/
+ mutex_lock(group-mutex);
+ ret = -EINVAL;
+ if (iommu_group_device_count(group) != 1)
+ goto out_unlock;
+
+ ret = __iommu_attach_device(domain, dev);
+
+out_unlock:
+ mutex_unlock(group-mutex);
+ iommu_group_put(group);
+
+ return ret;
+}
EXPORT_SYMBOL_GPL(iommu_attach_device);
-void iommu_detach_device(struct iommu_domain *domain, struct device *dev)
+static void __iommu_detach_device(struct iommu_domain *domain,
+ struct device *dev)
{
if (unlikely(domain-ops-detach_dev == NULL))
return;
@@ -991,6 +1032,28 @@ void iommu_detach_device(struct iommu_domain *domain,
struct device *dev)
domain-ops-detach_dev(domain, dev);
trace_detach_device_from_domain(dev);
}
+
+void iommu_detach_device(struct iommu_domain *domain, struct device *dev)
+{
+ struct iommu_group *group;
+
+ group = iommu_group_get(dev);
+ /* FIXME: Remove this when groups a mandatory for iommu drivers */
+ if (group == NULL)
+ return __iommu_detach_device(domain, dev);
+
+ mutex_lock(group-mutex);
+ if (iommu_group_device_count(group) != 1) {
+ WARN_ON(1);
+ goto out_unlock;
+ }
+
+ __iommu_detach_device(domain, dev);
+
+out_unlock:
+ mutex_unlock(group-mutex);
+ iommu_group_put(group);
+}
EXPORT_SYMBOL_GPL(iommu_detach_device);
/*
@@ -1007,7 +1070,7 @@ static int iommu_group_do_attach_device(struct device
*dev, void *data)
{
struct iommu_domain *domain = data;
- return iommu_attach_device(domain, dev);
+ return __iommu_attach_device(domain, dev);
}
int iommu_attach_group(struct iommu_domain *domain, struct iommu_group *group)
@@ -1021,7 +1084,7 @@ static int iommu_group_do_detach_device(struct device
*dev, void *data)
{
struct iommu_domain *domain = data;
- iommu_detach_device(domain, dev);
+ __iommu_detach_device(domain, dev);
return 0;
}
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 05/22] iommu: Call remove_device call-back after driver release
From: Joerg Roedel jroe...@suse.de
Do not remove the device from the IOMMU while the driver is
still attached.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index f0e0a23..d69e0ca 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -779,7 +779,7 @@ static int iommu_bus_notifier(struct notifier_block *nb,
if (action == BUS_NOTIFY_ADD_DEVICE) {
if (ops-add_device)
return ops-add_device(dev);
- } else if (action == BUS_NOTIFY_DEL_DEVICE) {
+ } else if (action == BUS_NOTIFY_REMOVED_DEVICE) {
if (ops-remove_device dev-iommu_group) {
ops-remove_device(dev);
return 0;
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 02/22] iommu: Add a few printk messages to group handling code
From: Joerg Roedel jroe...@suse.de
Write a message to the kernel log when a device is added or
removed from a group and add debug messages to group
allocation and release routines.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index c31bfd0..755e488 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -128,6 +128,8 @@ static void iommu_group_release(struct kobject *kobj)
{
struct iommu_group *group = to_iommu_group(kobj);
+ pr_debug(Releasing group %d\n, group-id);
+
if (group-iommu_data_release)
group-iommu_data_release(group-iommu_data);
@@ -207,6 +209,8 @@ again:
*/
kobject_put(group-kobj);
+ pr_debug(Allocated group %d\n, group-id);
+
return group;
}
EXPORT_SYMBOL_GPL(iommu_group_alloc);
@@ -372,6 +376,9 @@ rename:
IOMMU_GROUP_NOTIFY_ADD_DEVICE, dev);
trace_add_device_to_group(group-id, dev);
+
+ pr_info(Adding device %s to group %d\n, dev_name(dev), group-id);
+
return 0;
}
EXPORT_SYMBOL_GPL(iommu_group_add_device);
@@ -388,6 +395,8 @@ void iommu_group_remove_device(struct device *dev)
struct iommu_group *group = dev-iommu_group;
struct iommu_device *tmp_device, *device = NULL;
+ pr_info(Removing device %s from group %d\n, dev_name(dev), group-id);
+
/* Pre-notify listeners that a device is being removed. */
blocking_notifier_call_chain(group-notifier,
IOMMU_GROUP_NOTIFY_DEL_DEVICE, dev);
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 09/22] iommu: Add iommu_get_domain_for_dev function
From: Joerg Roedel jroe...@suse.de
This function can be used to request the current domain a
device is attached to.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 18 ++
include/linux/iommu.h | 6 ++
2 files changed, 24 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 7bce522..a0a38bd 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1076,6 +1076,24 @@ out_unlock:
}
EXPORT_SYMBOL_GPL(iommu_detach_device);
+struct iommu_domain *iommu_get_domain_for_dev(struct device *dev)
+{
+ struct iommu_domain *domain;
+ struct iommu_group *group;
+
+ group = iommu_group_get(dev);
+ /* FIXME: Remove this when groups a mandatory for iommu drivers */
+ if (group == NULL)
+ return NULL;
+
+ domain = group-domain;
+
+ iommu_group_put(group);
+
+ return domain;
+}
+EXPORT_SYMBOL_GPL(iommu_get_domain_for_dev);
+
/*
* IOMMU groups are really the natrual working unit of the IOMMU, but
* the IOMMU API works on domains and devices. Bridge that gap by
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 0546b87..683a1c4 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -193,6 +193,7 @@ extern int iommu_attach_device(struct iommu_domain *domain,
struct device *dev);
extern void iommu_detach_device(struct iommu_domain *domain,
struct device *dev);
+extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev);
extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
phys_addr_t paddr, size_t size, int prot);
extern size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova,
@@ -332,6 +333,11 @@ static inline void iommu_detach_device(struct iommu_domain
*domain,
{
}
+static inline struct iommu_domain *iommu_get_domain_for_dev(struct device *dev)
+{
+ return NULL;
+}
+
static inline int iommu_map(struct iommu_domain *domain, unsigned long iova,
phys_addr_t paddr, int gfp_order, int prot)
{
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
ACPI crash with big gart IOMMU area (stack overflow?)
I have a computer where I had noticed that I must not turn on IOMMU in the BIOS, or Linux would crash on boot. The computer is Sun Ultra 20 workstation with dual-core 1st gen Opteron (175) and Nvidia CK804 chipset and 4G RAM. So IOMMU never worked for me before. I investigated it more today - set higher screen resolution from grub and tried different BIOS options for GART. It appears that IOMMU GART size of 64M..256M works fine but GART size 512M causes the crash. Crash screenshots from GART 512M (two different crashes, with and without an interrupt in the backtrace): https://drive.google.com/file/d/0Bzm3pOFVnfg-em0taW5RUHRKdUk https://drive.google.com/file/d/0Bzm3pOFVnfg-UEM5dWlWUG43WkU The shorter crash message suggested that ACPI DSDT has been overwritten (because of the stack overflow?) and suggested acpi=copy_dsdt. Tried it, also crashes with stack overflow: https://drive.google.com/file/d/0Bzm3pOFVnfg-MlQ5eGszSW9WYnc Kernel config and full dmesg from working 256M GART option are included below for reference. # # Automatically generated file; DO NOT EDIT. # Linux/x86 4.1.0-rc3 Kernel Configuration # CONFIG_64BIT=y CONFIG_X86_64=y CONFIG_X86=y CONFIG_INSTRUCTION_DECODER=y CONFIG_PERF_EVENTS_INTEL_UNCORE=y CONFIG_OUTPUT_FORMAT=elf64-x86-64 CONFIG_ARCH_DEFCONFIG=arch/x86/configs/x86_64_defconfig CONFIG_LOCKDEP_SUPPORT=y CONFIG_STACKTRACE_SUPPORT=y CONFIG_HAVE_LATENCYTOP_SUPPORT=y CONFIG_MMU=y CONFIG_NEED_DMA_MAP_STATE=y CONFIG_NEED_SG_DMA_LENGTH=y CONFIG_GENERIC_ISA_DMA=y CONFIG_GENERIC_BUG=y CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y CONFIG_GENERIC_HWEIGHT=y CONFIG_ARCH_MAY_HAVE_PC_FDC=y CONFIG_RWSEM_XCHGADD_ALGORITHM=y CONFIG_GENERIC_CALIBRATE_DELAY=y CONFIG_ARCH_HAS_CPU_RELAX=y CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y CONFIG_HAVE_SETUP_PER_CPU_AREA=y CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_SUSPEND_POSSIBLE=y CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y CONFIG_ARCH_WANT_GENERAL_HUGETLB=y CONFIG_ZONE_DMA32=y CONFIG_AUDIT_ARCH=y CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y CONFIG_X86_64_SMP=y CONFIG_X86_HT=y CONFIG_ARCH_HWEIGHT_CFLAGS=-fcall-saved-rdi -fcall-saved-rsi -fcall-saved-rdx -fcall-saved-rcx -fcall-saved-r8 -fcall-saved-r9 -fcall-saved-r10 -fcall-saved-r11 CONFIG_ARCH_SUPPORTS_UPROBES=y CONFIG_FIX_EARLYCON_MEM=y CONFIG_PGTABLE_LEVELS=4 CONFIG_DEFCONFIG_LIST=/lib/modules/$UNAME_RELEASE/.config CONFIG_IRQ_WORK=y CONFIG_BUILDTIME_EXTABLE_SORT=y # # General setup # CONFIG_INIT_ENV_ARG_LIMIT=32 CONFIG_CROSS_COMPILE= # CONFIG_COMPILE_TEST is not set CONFIG_LOCALVERSION= CONFIG_LOCALVERSION_AUTO=y CONFIG_HAVE_KERNEL_GZIP=y CONFIG_HAVE_KERNEL_BZIP2=y CONFIG_HAVE_KERNEL_LZMA=y CONFIG_HAVE_KERNEL_XZ=y CONFIG_HAVE_KERNEL_LZO=y CONFIG_HAVE_KERNEL_LZ4=y # CONFIG_KERNEL_GZIP is not set # CONFIG_KERNEL_BZIP2 is not set # CONFIG_KERNEL_LZMA is not set # CONFIG_KERNEL_XZ is not set CONFIG_KERNEL_LZO=y # CONFIG_KERNEL_LZ4 is not set CONFIG_DEFAULT_HOSTNAME=(none) CONFIG_SWAP=y CONFIG_SYSVIPC=y CONFIG_SYSVIPC_SYSCTL=y CONFIG_POSIX_MQUEUE=y CONFIG_POSIX_MQUEUE_SYSCTL=y # CONFIG_CROSS_MEMORY_ATTACH is not set # CONFIG_FHANDLE is not set # CONFIG_USELIB is not set # CONFIG_AUDIT is not set CONFIG_HAVE_ARCH_AUDITSYSCALL=y # # IRQ subsystem # CONFIG_GENERIC_IRQ_PROBE=y CONFIG_GENERIC_IRQ_SHOW=y CONFIG_GENERIC_IRQ_LEGACY_ALLOC_HWIRQ=y CONFIG_GENERIC_PENDING_IRQ=y CONFIG_IRQ_DOMAIN=y CONFIG_GENERIC_MSI_IRQ=y # CONFIG_IRQ_DOMAIN_DEBUG is not set CONFIG_IRQ_FORCED_THREADING=y CONFIG_SPARSE_IRQ=y CONFIG_CLOCKSOURCE_WATCHDOG=y CONFIG_ARCH_CLOCKSOURCE_DATA=y CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE=y CONFIG_GENERIC_TIME_VSYSCALL=y CONFIG_GENERIC_CLOCKEVENTS=y CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST=y CONFIG_GENERIC_CMOS_UPDATE=y # # Timers subsystem # CONFIG_TICK_ONESHOT=y CONFIG_NO_HZ_COMMON=y # CONFIG_HZ_PERIODIC is not set CONFIG_NO_HZ_IDLE=y # CONFIG_NO_HZ_FULL is not set # CONFIG_NO_HZ is not set CONFIG_HIGH_RES_TIMERS=y # # CPU/Task time and stats accounting # CONFIG_VIRT_CPU_ACCOUNTING=y # CONFIG_TICK_CPU_ACCOUNTING is not set CONFIG_VIRT_CPU_ACCOUNTING_GEN=y # CONFIG_IRQ_TIME_ACCOUNTING is not set # CONFIG_BSD_PROCESS_ACCT is not set CONFIG_TASKSTATS=y CONFIG_TASK_DELAY_ACCT=y CONFIG_TASK_XACCT=y CONFIG_TASK_IO_ACCOUNTING=y # # RCU Subsystem # CONFIG_TREE_RCU=y CONFIG_SRCU=y # CONFIG_TASKS_RCU is not set CONFIG_RCU_STALL_COMMON=y CONFIG_CONTEXT_TRACKING=y CONFIG_RCU_USER_QS=y CONFIG_CONTEXT_TRACKING_FORCE=y CONFIG_RCU_FANOUT=64 CONFIG_RCU_FANOUT_LEAF=16 # CONFIG_RCU_FANOUT_EXACT is not set CONFIG_RCU_FAST_NO_HZ=y # CONFIG_TREE_RCU_TRACE is not set CONFIG_RCU_KTHREAD_PRIO=0 CONFIG_RCU_NOCB_CPU=y # CONFIG_RCU_NOCB_CPU_NONE is not set # CONFIG_RCU_NOCB_CPU_ZERO is not set CONFIG_RCU_NOCB_CPU_ALL=y # CONFIG_RCU_EXPEDITE_BOOT is not set # CONFIG_BUILD_BIN2C is not set # CONFIG_IKCONFIG is not set CONFIG_LOG_BUF_SHIFT=17 CONFIG_LOG_CPU_MAX_BUF_SHIFT=12
[PATCH 16/22] iommu/amd: Implement add_device and remove_device
From: Joerg Roedel jroe...@suse.de
Implement these two iommu-ops call-backs to make use of the
initialization and notifier features of the iommu core.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 210 +++--
drivers/iommu/amd_iommu_init.c | 31 ++
2 files changed, 63 insertions(+), 178 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index dc8e44b..44eeca4 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -119,7 +119,7 @@ struct iommu_cmd {
struct kmem_cache *amd_iommu_irq_cache;
static void update_domain(struct protection_domain *domain);
-static int __init alloc_passthrough_domain(void);
+static int alloc_passthrough_domain(void);
/
*
@@ -434,64 +434,15 @@ static void iommu_uninit_device(struct device *dev)
/* Unlink from alias, it may change if another device is re-plugged */
dev_data-alias_data = NULL;
+ /* Remove dma-ops */
+ dev-archdata.dma_ops = NULL;
+
/*
* We keep dev_data around for unplugged devices and reuse it when the
* device is re-plugged - not doing so would introduce a ton of races.
*/
}
-void __init amd_iommu_uninit_devices(void)
-{
- struct iommu_dev_data *dev_data, *n;
- struct pci_dev *pdev = NULL;
-
- for_each_pci_dev(pdev) {
-
- if (!check_device(pdev-dev))
- continue;
-
- iommu_uninit_device(pdev-dev);
- }
-
- /* Free all of our dev_data structures */
- list_for_each_entry_safe(dev_data, n, dev_data_list, dev_data_list)
- free_dev_data(dev_data);
-}
-
-int __init amd_iommu_init_devices(void)
-{
- struct pci_dev *pdev = NULL;
- int ret = 0;
-
- for_each_pci_dev(pdev) {
-
- if (!check_device(pdev-dev))
- continue;
-
- ret = iommu_init_device(pdev-dev);
- if (ret == -ENOTSUPP)
- iommu_ignore_device(pdev-dev);
- else if (ret)
- goto out_free;
- }
-
- /*
-* Initialize IOMMU groups only after iommu_init_device() has
-* had a chance to populate any IVRS defined aliases.
-*/
- for_each_pci_dev(pdev) {
- if (check_device(pdev-dev))
- init_iommu_group(pdev-dev);
- }
-
- return 0;
-
-out_free:
-
- amd_iommu_uninit_devices();
-
- return ret;
-}
#ifdef CONFIG_AMD_IOMMU_STATS
/*
@@ -2402,81 +2353,79 @@ static struct protection_domain
*domain_for_device(struct device *dev)
return dom;
}
-static int device_change_notifier(struct notifier_block *nb,
- unsigned long action, void *data)
+static int amd_iommu_add_device(struct device *dev)
{
struct dma_ops_domain *dma_domain;
struct protection_domain *domain;
struct iommu_dev_data *dev_data;
- struct device *dev = data;
struct amd_iommu *iommu;
unsigned long flags;
u16 devid;
+ int ret;
- if (!check_device(dev))
+ if (!check_device(dev) || get_dev_data(dev))
return 0;
- devid= get_device_id(dev);
- iommu= amd_iommu_rlookup_table[devid];
- dev_data = get_dev_data(dev);
-
- switch (action) {
- case BUS_NOTIFY_ADD_DEVICE:
-
- iommu_init_device(dev);
- init_iommu_group(dev);
+ devid = get_device_id(dev);
+ iommu = amd_iommu_rlookup_table[devid];
- /*
-* dev_data is still NULL and
-* got initialized in iommu_init_device
-*/
- dev_data = get_dev_data(dev);
+ ret = iommu_init_device(dev);
+ if (ret == -ENOTSUPP) {
+ iommu_ignore_device(dev);
+ goto out;
+ }
+ init_iommu_group(dev);
- if (iommu_pass_through || dev_data-iommu_v2) {
- dev_data-passthrough = true;
- attach_device(dev, pt_domain);
- break;
- }
+ dev_data = get_dev_data(dev);
- domain = domain_for_device(dev);
+ if (iommu_pass_through || dev_data-iommu_v2) {
+ /* Make sure passthrough domain is allocated */
+ alloc_passthrough_domain();
+ dev_data-passthrough = true;
+ attach_device(dev, pt_domain);
+ goto out;
+ }
- /* allocate a protection domain if a device is added */
- dma_domain = find_protection_domain(devid);
- if (!dma_domain) {
- dma_domain = dma_ops_domain_alloc();
- if (!dma_domain)
- goto out;
-
[PATCH 18/22] iommu/amd: Support IOMMU_DOMAIN_IDENTITY type allocation
From: Joerg Roedel jroe...@suse.de Add support to allocate direct mapped domains through the IOMMU-API. Signed-off-by: Joerg Roedel jroe...@suse.de --- drivers/iommu/amd_iommu.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 06d19e8..33bbb4d 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -2996,6 +2996,13 @@ static struct iommu_domain *amd_iommu_domain_alloc(unsigned type) } pdomain = dma_domain-domain; break; + case IOMMU_DOMAIN_IDENTITY: + pdomain = protection_domain_alloc(); + if (!pdomain) + return NULL; + + pdomain-mode = PAGE_MODE_NONE; + break; default: return NULL; } -- 1.9.1 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 17/22] iommu/amd: Support IOMMU_DOMAIN_DMA type allocation
From: Joerg Roedel jroe...@suse.de
This enables allocation of DMA-API default domains from the
IOMMU core and switches allocation of domain dma-api domain
to the IOMMU core too.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 311 ++--
drivers/iommu/amd_iommu_types.h | 3 -
2 files changed, 73 insertions(+), 241 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 44eeca4..06d19e8 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -64,10 +64,6 @@
static DEFINE_RWLOCK(amd_iommu_devtable_lock);
-/* A list of preallocated protection domains */
-static LIST_HEAD(iommu_pd_list);
-static DEFINE_SPINLOCK(iommu_pd_list_lock);
-
/* List of all available dev_data structures */
static LIST_HEAD(dev_data_list);
static DEFINE_SPINLOCK(dev_data_list_lock);
@@ -234,31 +230,38 @@ static bool pdev_pri_erratum(struct pci_dev *pdev, u32
erratum)
}
/*
- * In this function the list of preallocated protection domains is traversed to
- * find the domain for a specific device
+ * This function actually applies the mapping to the page table of the
+ * dma_ops domain.
*/
-static struct dma_ops_domain *find_protection_domain(u16 devid)
+static void alloc_unity_mapping(struct dma_ops_domain *dma_dom,
+ struct unity_map_entry *e)
{
- struct dma_ops_domain *entry, *ret = NULL;
- unsigned long flags;
- u16 alias = amd_iommu_alias_table[devid];
-
- if (list_empty(iommu_pd_list))
- return NULL;
-
- spin_lock_irqsave(iommu_pd_list_lock, flags);
+ u64 addr;
- list_for_each_entry(entry, iommu_pd_list, list) {
- if (entry-target_dev == devid ||
- entry-target_dev == alias) {
- ret = entry;
- break;
- }
+ for (addr = e-address_start; addr e-address_end;
+addr += PAGE_SIZE) {
+ if (addr dma_dom-aperture_size)
+ __set_bit(addr PAGE_SHIFT,
+ dma_dom-aperture[0]-bitmap);
}
+}
+
+/*
+ * Inits the unity mappings required for a specific device
+ */
+static void init_unity_mappings_for_device(struct device *dev,
+ struct dma_ops_domain *dma_dom)
+{
+ struct unity_map_entry *e;
+ u16 devid;
- spin_unlock_irqrestore(iommu_pd_list_lock, flags);
+ devid = get_device_id(dev);
- return ret;
+ list_for_each_entry(e, amd_iommu_unity_map, list) {
+ if (!(devid = e-devid_start devid = e-devid_end))
+ continue;
+ alloc_unity_mapping(dma_dom, e);
+ }
}
/*
@@ -290,11 +293,23 @@ static bool check_device(struct device *dev)
static void init_iommu_group(struct device *dev)
{
+ struct dma_ops_domain *dma_domain;
+ struct iommu_domain *domain;
struct iommu_group *group;
group = iommu_group_get_for_dev(dev);
- if (!IS_ERR(group))
- iommu_group_put(group);
+ if (IS_ERR(group))
+ return;
+
+ domain = iommu_group_default_domain(group);
+ if (!domain)
+ goto out;
+
+ dma_domain = to_pdomain(domain)-priv;
+
+ init_unity_mappings_for_device(dev, dma_domain);
+out:
+ iommu_group_put(group);
}
static int __last_alias(struct pci_dev *pdev, u16 alias, void *data)
@@ -1414,94 +1429,6 @@ static unsigned long iommu_unmap_page(struct
protection_domain *dom,
return unmapped;
}
-/*
- * This function checks if a specific unity mapping entry is needed for
- * this specific IOMMU.
- */
-static int iommu_for_unity_map(struct amd_iommu *iommu,
- struct unity_map_entry *entry)
-{
- u16 bdf, i;
-
- for (i = entry-devid_start; i = entry-devid_end; ++i) {
- bdf = amd_iommu_alias_table[i];
- if (amd_iommu_rlookup_table[bdf] == iommu)
- return 1;
- }
-
- return 0;
-}
-
-/*
- * This function actually applies the mapping to the page table of the
- * dma_ops domain.
- */
-static int dma_ops_unity_map(struct dma_ops_domain *dma_dom,
-struct unity_map_entry *e)
-{
- u64 addr;
- int ret;
-
- for (addr = e-address_start; addr e-address_end;
-addr += PAGE_SIZE) {
- ret = iommu_map_page(dma_dom-domain, addr, addr, e-prot,
-PAGE_SIZE);
- if (ret)
- return ret;
- /*
-* if unity mapping is in aperture range mark the page
-* as allocated in the aperture
-*/
- if (addr dma_dom-aperture_size)
- __set_bit(addr PAGE_SHIFT,
- dma_dom-aperture[0]-bitmap);
[PATCH 11/22] iommu: Create direct mappings in default domains
From: Joerg Roedel jroe...@suse.de
Use the information exported by the IOMMU drivers to create
direct mapped regions in the default domains.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 48
1 file changed, 48 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 6b8d6e7..ffad1ea 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -325,6 +325,52 @@ int iommu_group_set_name(struct iommu_group *group, const
char *name)
}
EXPORT_SYMBOL_GPL(iommu_group_set_name);
+static int iommu_group_create_direct_mappings(struct iommu_group *group,
+ struct device *dev)
+{
+ struct iommu_domain *domain = group-default_domain;
+ struct iommu_dm_region *entry;
+ struct list_head mappings;
+ unsigned long pg_size;
+ int ret = 0;
+
+ if (!domain || domain-type != IOMMU_DOMAIN_DMA)
+ return 0;
+
+ BUG_ON(!domain-ops-pgsize_bitmap);
+
+ pg_size = 1UL __ffs(domain-ops-pgsize_bitmap);
+ INIT_LIST_HEAD(mappings);
+
+ iommu_get_dm_regions(dev, mappings);
+
+ /* We need to consider overlapping regions for different devices */
+ list_for_each_entry(entry, mappings, list) {
+ dma_addr_t start, end, addr;
+
+ start = ALIGN(entry-start, pg_size);
+ end = ALIGN(entry-start + entry-length, pg_size);
+
+ for (addr = start; addr end; addr += pg_size) {
+ phys_addr_t phys_addr;
+
+ phys_addr = iommu_iova_to_phys(domain, addr);
+ if (phys_addr)
+ continue;
+
+ ret = iommu_map(domain, addr, addr, pg_size,
entry-prot);
+ if (ret)
+ goto out;
+ }
+
+ }
+
+out:
+ iommu_put_dm_regions(dev, mappings);
+
+ return ret;
+}
+
/**
* iommu_group_add_device - add a device to an iommu group
* @group: the group into which to add the device (reference should be held)
@@ -381,6 +427,8 @@ rename:
dev-iommu_group = group;
+ iommu_group_create_direct_mappings(group, dev);
+
mutex_lock(group-mutex);
list_add_tail(device-list, group-devices);
if (group-domain)
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 06/22] iommu: Allocate a default domain for iommu groups
From: Joerg Roedel jroe...@suse.de
The default domain will be used (if supported by the iommu
driver) when the devices in the iommu group are not attached
to any other domain.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 32
1 file changed, 28 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index d69e0ca..fbfc015 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -51,6 +51,7 @@ struct iommu_group {
void (*iommu_data_release)(void *iommu_data);
char *name;
int id;
+ struct iommu_domain *default_domain;
};
struct iommu_device {
@@ -75,6 +76,9 @@ struct iommu_group_attribute iommu_group_attr_##_name =
\
#define to_iommu_group(_kobj) \
container_of(_kobj, struct iommu_group, kobj)
+static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
+unsigned type);
+
static ssize_t iommu_group_attr_show(struct kobject *kobj,
struct attribute *__attr, char *buf)
{
@@ -137,6 +141,9 @@ static void iommu_group_release(struct kobject *kobj)
ida_remove(iommu_group_ida, group-id);
mutex_unlock(iommu_group_mutex);
+ if (group-default_domain)
+ iommu_domain_free(group-default_domain);
+
kfree(group-name);
kfree(group);
}
@@ -701,7 +708,18 @@ static struct iommu_group
*iommu_group_get_for_pci_dev(struct pci_dev *pdev)
return group;
/* No shared group found, allocate new */
- return iommu_group_alloc();
+ group = iommu_group_alloc();
+ if (group) {
+ /*
+* Try to allocate a default domain - needs support from the
+* IOMMU driver.
+*/
+ group-default_domain = __iommu_domain_alloc(pdev-dev.bus,
+IOMMU_DOMAIN_DMA);
+ group-domain = group-default_domain;
+ }
+
+ return group;
}
/**
@@ -922,22 +940,28 @@ void iommu_set_fault_handler(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_set_fault_handler);
-struct iommu_domain *iommu_domain_alloc(struct bus_type *bus)
+static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
+unsigned type)
{
struct iommu_domain *domain;
if (bus == NULL || bus-iommu_ops == NULL)
return NULL;
- domain = bus-iommu_ops-domain_alloc(IOMMU_DOMAIN_UNMANAGED);
+ domain = bus-iommu_ops-domain_alloc(type);
if (!domain)
return NULL;
domain-ops = bus-iommu_ops;
- domain-type = IOMMU_DOMAIN_UNMANAGED;
+ domain-type = type;
return domain;
}
+
+struct iommu_domain *iommu_domain_alloc(struct bus_type *bus)
+{
+ return __iommu_domain_alloc(bus, IOMMU_DOMAIN_UNMANAGED);
+}
EXPORT_SYMBOL_GPL(iommu_domain_alloc);
void iommu_domain_free(struct iommu_domain *domain)
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 04/22] iommu: Clean up after a failed bus initialization
From: Joerg Roedel jroe...@suse.de
Make sure we call the -remove_device call-back on all
devices already initialized with -add_device when the bus
initialization fails.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 35 ++-
1 file changed, 26 insertions(+), 9 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 9c9336a..f0e0a23 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -753,6 +753,17 @@ static int add_iommu_group(struct device *dev, void *data)
return ops-add_device(dev);
}
+static int remove_iommu_group(struct device *dev, void *data)
+{
+ struct iommu_callback_data *cb = data;
+ const struct iommu_ops *ops = cb-ops;
+
+ if (ops-remove_device dev-iommu_group)
+ ops-remove_device(dev);
+
+ return 0;
+}
+
static int iommu_bus_notifier(struct notifier_block *nb,
unsigned long action, void *data)
{
@@ -821,19 +832,25 @@ static int iommu_bus_init(struct bus_type *bus, const
struct iommu_ops *ops)
nb-notifier_call = iommu_bus_notifier;
err = bus_register_notifier(bus, nb);
- if (err) {
- kfree(nb);
- return err;
- }
+ if (err)
+ goto out_free;
err = bus_for_each_dev(bus, NULL, cb, add_iommu_group);
- if (err) {
- bus_unregister_notifier(bus, nb);
- kfree(nb);
- return err;
- }
+ if (err)
+ goto out_err;
+
return 0;
+
+out_err:
+ /* Clean up */
+ bus_for_each_dev(bus, NULL, cb, remove_iommu_group);
+ bus_unregister_notifier(bus, nb);
+
+out_free:
+ kfree(nb);
+
+ return err;
}
/**
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 10/22] iommu: Introduce direct mapped region handling
From: Joerg Roedel jroe...@suse.de
Add two new functions to the IOMMU-API to allow the IOMMU
drivers to export the requirements for direct mapped regions
per device.
This is useful for exporting the information in Intel VT-d's
RMRR entries or AMD-Vi's unity mappings.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 16
include/linux/iommu.h | 31 +++
2 files changed, 47 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index a0a38bd..6b8d6e7 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1469,3 +1469,19 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
return ret;
}
EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
+
+void iommu_get_dm_regions(struct device *dev, struct list_head *list)
+{
+ const struct iommu_ops *ops = dev-bus-iommu_ops;
+
+ if (ops ops-get_dm_regions)
+ ops-get_dm_regions(dev, list);
+}
+
+void iommu_put_dm_regions(struct device *dev, struct list_head *list)
+{
+ const struct iommu_ops *ops = dev-bus-iommu_ops;
+
+ if (ops ops-put_dm_regions)
+ ops-put_dm_regions(dev, list);
+}
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 683a1c4..6894999 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -114,6 +114,20 @@ enum iommu_attr {
DOMAIN_ATTR_MAX,
};
+/**
+ * struct iommu_dm_region - descriptor for a direct mapped memory region
+ * @list: Linked list pointers
+ * @start: System physical start address of the region
+ * @length: Length of the region in bytes
+ * @prot: IOMMU Protection flags (READ/WRITE/...)
+ */
+struct iommu_dm_region {
+ struct list_headlist;
+ phys_addr_t start;
+ size_t length;
+ int prot;
+};
+
#ifdef CONFIG_IOMMU_API
/**
@@ -159,6 +173,10 @@ struct iommu_ops {
int (*domain_set_attr)(struct iommu_domain *domain,
enum iommu_attr attr, void *data);
+ /* Request/Free a list of direct mapping requirements for a device */
+ void (*get_dm_regions)(struct device *dev, struct list_head *list);
+ void (*put_dm_regions)(struct device *dev, struct list_head *list);
+
/* Window handling functions */
int (*domain_window_enable)(struct iommu_domain *domain, u32 wnd_nr,
phys_addr_t paddr, u64 size, int prot);
@@ -205,6 +223,9 @@ extern phys_addr_t iommu_iova_to_phys(struct iommu_domain
*domain, dma_addr_t io
extern void iommu_set_fault_handler(struct iommu_domain *domain,
iommu_fault_handler_t handler, void *token);
+extern void iommu_get_dm_regions(struct device *dev, struct list_head *list);
+extern void iommu_put_dm_regions(struct device *dev, struct list_head *list);
+
extern int iommu_attach_group(struct iommu_domain *domain,
struct iommu_group *group);
extern void iommu_detach_group(struct iommu_domain *domain,
@@ -379,6 +400,16 @@ static inline void iommu_set_fault_handler(struct
iommu_domain *domain,
{
}
+static inline void iommu_get_dm_regions(struct device *dev,
+ struct list_head *list)
+{
+}
+
+static inline void iommu_put_dm_regions(struct device *dev,
+ struct list_head *list)
+{
+}
+
static inline int iommu_attach_group(struct iommu_domain *domain,
struct iommu_group *group)
{
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 03/22] iommu: Propagate error in add_iommu_group
From: Joerg Roedel jroe...@suse.de Make sure any errors reported from the IOMMU drivers get progapated back to the IOMMU core. Signed-off-by: Joerg Roedel jroe...@suse.de --- drivers/iommu/iommu.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 755e488..9c9336a 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -750,9 +750,7 @@ static int add_iommu_group(struct device *dev, void *data) WARN_ON(dev-iommu_group); - ops-add_device(dev); - - return 0; + return ops-add_device(dev); } static int iommu_bus_notifier(struct notifier_block *nb, -- 1.9.1 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 14/22] iommu/amd: Implement dm_region call-backs
From: Joerg Roedel jroe...@suse.de
Add the get_dm_regions and put_dm_regions callbacks to the
iommu_ops of the AMD IOMMU driver.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 37 +
1 file changed, 37 insertions(+)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index e43d489..9da3f0e 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -3412,6 +3412,41 @@ static bool amd_iommu_capable(enum iommu_cap cap)
return false;
}
+static void amd_iommu_get_dm_regions(struct device *dev,
+struct list_head *head)
+{
+ struct unity_map_entry *entry;
+ u16 devid;
+
+ devid = get_device_id(dev);
+
+ list_for_each_entry(entry, amd_iommu_unity_map, list) {
+ struct iommu_dm_region *region;
+
+ if (devid entry-devid_start || devid entry-devid_end)
+ continue;
+
+ region = kzalloc(sizeof(*region), GFP_KERNEL);
+ region-start = entry-address_start;
+ region-length = entry-address_end - entry-address_start;
+ if (entry-prot IOMMU_PROT_IR)
+ region-prot |= IOMMU_READ;
+ if (entry-prot IOMMU_PROT_IW)
+ region-prot |= IOMMU_WRITE;
+
+ list_add_tail(region-list, head);
+ }
+}
+
+static void amd_iommu_put_dm_regions(struct device *dev,
+struct list_head *head)
+{
+ struct iommu_dm_region *entry, *next;
+
+ list_for_each_entry_safe(entry, next, head, list)
+ kfree(entry);
+}
+
static const struct iommu_ops amd_iommu_ops = {
.capable = amd_iommu_capable,
.domain_alloc = amd_iommu_domain_alloc,
@@ -3422,6 +3457,8 @@ static const struct iommu_ops amd_iommu_ops = {
.unmap = amd_iommu_unmap,
.map_sg = default_iommu_map_sg,
.iova_to_phys = amd_iommu_iova_to_phys,
+ .get_dm_regions = amd_iommu_get_dm_regions,
+ .put_dm_regions = amd_iommu_put_dm_regions,
.pgsize_bitmap = AMD_IOMMU_PGSIZES,
};
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 12/22] iommu: Add function to query the default domain of a group
From: Joerg Roedel jroe...@suse.de
This will be used to handle unity mappings in the iommu
drivers.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 5 +
include/linux/iommu.h | 1 +
2 files changed, 6 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index ffad1ea..224c6dd 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -837,6 +837,11 @@ struct iommu_group *iommu_group_get_for_dev(struct device
*dev)
return group;
}
+struct iommu_domain *iommu_group_default_domain(struct iommu_group *group)
+{
+ return group-default_domain;
+}
+
static int add_iommu_group(struct device *dev, void *data)
{
struct iommu_callback_data *cb = data;
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 6894999..b944b2b 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -249,6 +249,7 @@ extern int iommu_group_unregister_notifier(struct
iommu_group *group,
struct notifier_block *nb);
extern int iommu_group_id(struct iommu_group *group);
extern struct iommu_group *iommu_group_get_for_dev(struct device *dev);
+extern struct iommu_domain *iommu_group_default_domain(struct iommu_group *);
extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 15/22] iommu/amd: Use default domain if available for DMA-API
From: Joerg Roedel jroe...@suse.de
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 7 +++
1 file changed, 7 insertions(+)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 9da3f0e..dc8e44b 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2495,12 +2495,19 @@ void amd_iommu_init_notifier(void)
static struct protection_domain *get_domain(struct device *dev)
{
struct protection_domain *domain;
+ struct iommu_domain *io_domain;
struct dma_ops_domain *dma_dom;
u16 devid = get_device_id(dev);
if (!check_device(dev))
return ERR_PTR(-EINVAL);
+ io_domain = iommu_get_domain_for_dev(dev);
+ if (io_domain) {
+ domain = to_pdomain(io_domain);
+ return domain;
+ }
+
domain = domain_for_device(dev);
if (domain != NULL !dma_ops_domain(domain))
return ERR_PTR(-EBUSY);
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 20/22] iommu/amd: Get rid of device_dma_ops_init()
From: Joerg Roedel jroe...@suse.de
With device intialization done in the add_device call-back
now there is no reason for this function anymore.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 40 +---
1 file changed, 1 insertion(+), 39 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 8682786..978741b 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2268,6 +2268,7 @@ static int amd_iommu_add_device(struct device *dev)
ret = iommu_init_device(dev);
if (ret == -ENOTSUPP) {
iommu_ignore_device(dev);
+ dev-archdata.dma_ops = nommu_dma_ops;
goto out;
}
init_iommu_group(dev);
@@ -2840,36 +2841,6 @@ static struct dma_map_ops amd_iommu_dma_ops = {
.dma_supported = amd_iommu_dma_supported,
};
-static unsigned device_dma_ops_init(void)
-{
- struct iommu_dev_data *dev_data;
- struct pci_dev *pdev = NULL;
- unsigned unhandled = 0;
-
- for_each_pci_dev(pdev) {
- if (!check_device(pdev-dev)) {
-
- iommu_ignore_device(pdev-dev);
-
- unhandled += 1;
- continue;
- }
-
- dev_data = get_dev_data(pdev-dev);
-
- if (!dev_data-passthrough)
- pdev-dev.archdata.dma_ops = amd_iommu_dma_ops;
- else
- pdev-dev.archdata.dma_ops = nommu_dma_ops;
- }
-
- return unhandled;
-}
-
-/*
- * The function which clues the AMD IOMMU driver into dma_ops.
- */
-
void __init amd_iommu_init_api(void)
{
bus_set_iommu(pci_bus_type, amd_iommu_ops);
@@ -2877,18 +2848,9 @@ void __init amd_iommu_init_api(void)
int __init amd_iommu_init_dma_ops(void)
{
- int unhandled;
-
iommu_detected = 1;
swiotlb = 0;
- /* Make the driver finally visible to the drivers */
- unhandled = device_dma_ops_init();
- if (unhandled max_pfn MAX_DMA32_PFN) {
- /* There are unhandled devices - initialize swiotlb for them */
- swiotlb = 1;
- }
-
amd_iommu_stats_init();
if (amd_iommu_unmap_flush)
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 22/22] iommu/amd: Propagate errors from amd_iommu_init_api
From: Joerg Roedel jroe...@suse.de
This function can fail. Propagate any errors back to the
initialization state machine.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 4 ++--
drivers/iommu/amd_iommu_init.c | 5 +++--
drivers/iommu/amd_iommu_proto.h | 2 +-
3 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 6e73fa1..6659b51 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2840,9 +2840,9 @@ static struct dma_map_ops amd_iommu_dma_ops = {
.dma_supported = amd_iommu_dma_supported,
};
-void __init amd_iommu_init_api(void)
+int __init amd_iommu_init_api(void)
{
- bus_set_iommu(pci_bus_type, amd_iommu_ops);
+ return bus_set_iommu(pci_bus_type, amd_iommu_ops);
}
int __init amd_iommu_init_dma_ops(void)
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index e4a6e40..dbac49c 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -1391,9 +1391,10 @@ static int __init amd_iommu_init_pci(void)
for_each_iommu(iommu)
iommu_flush_all_caches(iommu);
- amd_iommu_init_api();
+ ret = amd_iommu_init_api();
- print_iommu_info();
+ if (!ret)
+ print_iommu_info();
return ret;
}
diff --git a/drivers/iommu/amd_iommu_proto.h b/drivers/iommu/amd_iommu_proto.h
index 72b0fd4..9ed1c43 100644
--- a/drivers/iommu/amd_iommu_proto.h
+++ b/drivers/iommu/amd_iommu_proto.h
@@ -30,7 +30,7 @@ extern void amd_iommu_reset_cmd_buffer(struct amd_iommu
*iommu);
extern int amd_iommu_init_devices(void);
extern void amd_iommu_uninit_devices(void);
extern void amd_iommu_init_notifier(void);
-extern void amd_iommu_init_api(void);
+extern int amd_iommu_init_api(void);
/* Needed for interrupt remapping */
extern int amd_iommu_prepare(void);
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 21/22] iommu/amd: Remove unused fields from struct dma_ops_domain
From: Joerg Roedel jroe...@suse.de
The list_head and target_dev members are not used anymore.
Remove them.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 1 -
drivers/iommu/amd_iommu_types.h | 8
2 files changed, 9 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 978741b..6e73fa1 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -1886,7 +1886,6 @@ static struct dma_ops_domain *dma_ops_domain_alloc(void)
goto free_dma_dom;
dma_dom-need_flush = false;
- dma_dom-target_dev = 0x;
add_domain_to_list(dma_dom-domain);
diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h
index fe796cf..bb56560 100644
--- a/drivers/iommu/amd_iommu_types.h
+++ b/drivers/iommu/amd_iommu_types.h
@@ -446,8 +446,6 @@ struct aperture_range {
* Data container for a dma_ops specific protection domain
*/
struct dma_ops_domain {
- struct list_head list;
-
/* generic protection domain information */
struct protection_domain domain;
@@ -462,12 +460,6 @@ struct dma_ops_domain {
/* This will be set to true when TLB needs to be flushed */
bool need_flush;
-
- /*
-* if this is a preallocated domain, keep the device for which it was
-* preallocated in this variable
-*/
- u16 target_dev;
};
/*
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 13/22] iommu: Introduce iommu_request_dm_for_dev()
From: Joerg Roedel jroe...@suse.de
This function can be called by an IOMMU driver to request
that a device's default domain is direct mapped.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/iommu.c | 52 +++
include/linux/iommu.h | 6 ++
2 files changed, 58 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 224c6dd..cf6ea95 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1538,3 +1538,55 @@ void iommu_put_dm_regions(struct device *dev, struct
list_head *list)
if (ops ops-put_dm_regions)
ops-put_dm_regions(dev, list);
}
+
+/* Request that a device is direct mapped by the IOMMU */
+int iommu_request_dm_for_dev(struct device *dev)
+{
+ struct iommu_domain *dm_domain;
+ struct iommu_group *group;
+ int ret;
+
+ /* Device must already be in a group before calling this function */
+ group = iommu_group_get_for_dev(dev);
+ if (!group)
+ return -EINVAL;
+
+ mutex_lock(group-mutex);
+
+ /* Check if the default domain is already direct mapped */
+ ret = 0;
+ if (group-default_domain
+ group-default_domain-type == IOMMU_DOMAIN_IDENTITY)
+ goto out;
+
+ /* Don't change mappings of existing devices */
+ ret = -EBUSY;
+ if (iommu_group_device_count(group) != 1)
+ goto out;
+
+ /* Allocate a direct mapped domain */
+ ret = -ENOMEM;
+ dm_domain = __iommu_domain_alloc(dev-bus, IOMMU_DOMAIN_IDENTITY);
+ if (!dm_domain)
+ goto out;
+
+ /* Attach the device to the domain */
+ ret = __iommu_attach_group(dm_domain, group);
+ if (ret) {
+ iommu_domain_free(dm_domain);
+ goto out;
+ }
+
+ /* Make the direct mapped domain the default for this group */
+ iommu_domain_free(group-default_domain);
+ group-default_domain = dm_domain;
+
+ pr_info(Using direct mapping for device %s\n, dev_name(dev));
+
+ ret = 0;
+out:
+ mutex_unlock(group-mutex);
+ iommu_group_put(group);
+
+ return ret;
+}
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index b944b2b..dc767f7 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -225,6 +225,7 @@ extern void iommu_set_fault_handler(struct iommu_domain
*domain,
extern void iommu_get_dm_regions(struct device *dev, struct list_head *list);
extern void iommu_put_dm_regions(struct device *dev, struct list_head *list);
+extern int iommu_request_dm_for_dev(struct device *dev);
extern int iommu_attach_group(struct iommu_domain *domain,
struct iommu_group *group);
@@ -411,6 +412,11 @@ static inline void iommu_put_dm_regions(struct device *dev,
{
}
+static inline int iommu_request_dm_for_dev(struct device *dev)
+{
+ return -ENODEV;
+}
+
static inline int iommu_attach_group(struct iommu_domain *domain,
struct iommu_group *group)
{
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 19/22] iommu/amd: Put IOMMUv2 devices in a direct mapped domain
From: Joerg Roedel jroe...@suse.de
A device that might be used for HSA needs to be in a direct
mapped domain so that all DMA-API mappings stay alive when
the IOMMUv2 stack is used.
Signed-off-by: Joerg Roedel jroe...@suse.de
---
drivers/iommu/amd_iommu.c | 15 ++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 33bbb4d..8682786 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2253,6 +2253,8 @@ static void detach_device(struct device *dev)
static int amd_iommu_add_device(struct device *dev)
{
+ struct iommu_dev_data *dev_data;
+ struct iommu_domain *domain;
struct amd_iommu *iommu;
u16 devid;
int ret;
@@ -2270,7 +2272,18 @@ static int amd_iommu_add_device(struct device *dev)
}
init_iommu_group(dev);
- dev-archdata.dma_ops = amd_iommu_dma_ops;
+ dev_data = get_dev_data(dev);
+ if (dev_data dev_data-iommu_v2)
+ iommu_request_dm_for_dev(dev);
+
+ /* Domains are initialized for this device - have a look what we ended
up with */
+ domain = iommu_get_domain_for_dev(dev);
+ if (domain-type == IOMMU_DOMAIN_IDENTITY) {
+ dev_data-passthrough = true;
+ dev-archdata.dma_ops = nommu_dma_ops;
+ } else {
+ dev-archdata.dma_ops = amd_iommu_dma_ops;
+ }
out:
iommu_completion_wait(iommu);
--
1.9.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
