Re: [PATCH 5/5] dma-direct: always allow dma mask <= physiscal memory size
On Wed, 2018-10-03 at 16:10 -0700, Alexander Duyck wrote: > > -* Because 32-bit DMA masks are so common we expect every > > architecture > > -* to be able to satisfy them - either by not supporting more > > physical > > -* memory, or by providing a ZONE_DMA32. If neither is the case, > > the > > -* architecture needs to use an IOMMU instead of the direct mapping. > > -*/ > > - if (mask < phys_to_dma(dev, DMA_BIT_MASK(32))) > > + u64 min_mask; > > + > > + if (IS_ENABLED(CONFIG_ZONE_DMA)) > > + min_mask = DMA_BIT_MASK(ARCH_ZONE_DMA_BITS); > > + else > > + min_mask = DMA_BIT_MASK(32); > > + > > + min_mask = min_t(u64, min_mask, (max_pfn - 1) << PAGE_SHIFT); > > + > > + if (mask >= phys_to_dma(dev, min_mask)) > > return 0; > > -#endif > > return 1; > > } > > So I believe I have run into the same issue that Guenter reported. On > an x86_64 system w/ Intel IOMMU. I wasn't able to complete boot and > all probe attempts for various devices were failing with -EIO errors. > > I believe the last mask check should be "if (mask < phys_to_dma(dev, > min_mask))" not a ">=" check. Right, that test is backwards. I needed to change it here too (powermac with the rest of the powerpc series). Cheers, Ben. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] dma-direct: document the zone selection logic
On Mon, 2018-10-08 at 09:03 +0200, Christoph Hellwig wrote: > Ben, does this resolve your issues with the confusing zone selection? The comment does make things a tad clearer yes :) Thanks ! Cheers, Ben. > On Mon, Oct 01, 2018 at 01:10:16PM -0700, Christoph Hellwig wrote: > > What we are doing here isn't quite obvious, so add a comment explaining > > it. > > > > Signed-off-by: Christoph Hellwig > > --- > > kernel/dma/direct.c | 9 - > > 1 file changed, 8 insertions(+), 1 deletion(-) > > > > diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c > > index ba6f5956a291..14b966e2349a 100644 > > --- a/kernel/dma/direct.c > > +++ b/kernel/dma/direct.c > > @@ -84,7 +84,14 @@ static gfp_t __dma_direct_optimal_gfp_mask(struct device > > *dev, u64 dma_mask, > > else > > *phys_mask = dma_to_phys(dev, dma_mask); > > > > - /* GFP_DMA32 and GFP_DMA are no ops without the corresponding zones: */ > > + /* > > +* Optimistically try the zone that the physicall address mask falls > > +* into first. If that returns memory that isn't actually addressable > > +* we will fallback to the next lower zone and try again. > > +* > > +* Note that GFP_DMA32 and GFP_DMA are no ops without the corresponding > > +* zones. > > +*/ > > if (*phys_mask <= DMA_BIT_MASK(ARCH_ZONE_DMA_BITS)) > > return GFP_DMA; > > if (*phys_mask <= DMA_BIT_MASK(32)) > > -- > > 2.19.0 > > > > ___ > > iommu mailing list > > iommu@lists.linux-foundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/iommu > > ---end quoted text--- ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 4.18 156/168] iommu/amd: Clear memory encryption mask from physical address
4.18-stable review patch. If anyone has any objections, please let me know.
--
From: Singh, Brijesh
commit b3e9b515b08e407ab3a026dc2e4d935c48d05f69 upstream.
Boris Ostrovsky reported a memory leak with device passthrough when SME
is active.
The VFIO driver uses iommu_iova_to_phys() to get the physical address for
an iova. This physical address is later passed into vfio_unmap_unpin() to
unpin the memory. The vfio_unmap_unpin() uses pfn_valid() before unpinning
the memory. The pfn_valid() check was failing because encryption mask was
part of the physical address returned. This resulted in the memory not
being unpinned and therefore leaked after the guest terminates.
The memory encryption mask must be cleared from the physical address in
iommu_iova_to_phys().
Fixes: 2543a786aa25 ("iommu/amd: Allow the AMD IOMMU to work with memory
encryption")
Reported-by: Boris Ostrovsky
Cc: Tom Lendacky
Cc: Joerg Roedel
Cc:
Cc: Borislav Petkov
Cc: Paolo Bonzini
Cc: Radim Krčmář
Cc: k...@vger.kernel.org
Cc: Boris Ostrovsky
Cc: # 4.14+
Signed-off-by: Brijesh Singh
Signed-off-by: Joerg Roedel
Signed-off-by: Greg Kroah-Hartman
---
drivers/iommu/amd_iommu.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -3073,7 +3073,7 @@ static phys_addr_t amd_iommu_iova_to_phy
return 0;
offset_mask = pte_pgsize - 1;
- __pte = *pte & PM_ADDR_MASK;
+ __pte = __sme_clr(*pte & PM_ADDR_MASK);
return (__pte & ~offset_mask) | (iova & offset_mask);
}
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 4.14 84/94] iommu/amd: Clear memory encryption mask from physical address
4.14-stable review patch. If anyone has any objections, please let me know.
--
From: Singh, Brijesh
commit b3e9b515b08e407ab3a026dc2e4d935c48d05f69 upstream.
Boris Ostrovsky reported a memory leak with device passthrough when SME
is active.
The VFIO driver uses iommu_iova_to_phys() to get the physical address for
an iova. This physical address is later passed into vfio_unmap_unpin() to
unpin the memory. The vfio_unmap_unpin() uses pfn_valid() before unpinning
the memory. The pfn_valid() check was failing because encryption mask was
part of the physical address returned. This resulted in the memory not
being unpinned and therefore leaked after the guest terminates.
The memory encryption mask must be cleared from the physical address in
iommu_iova_to_phys().
Fixes: 2543a786aa25 ("iommu/amd: Allow the AMD IOMMU to work with memory
encryption")
Reported-by: Boris Ostrovsky
Cc: Tom Lendacky
Cc: Joerg Roedel
Cc:
Cc: Borislav Petkov
Cc: Paolo Bonzini
Cc: Radim Krčmář
Cc: k...@vger.kernel.org
Cc: Boris Ostrovsky
Cc: # 4.14+
Signed-off-by: Brijesh Singh
Signed-off-by: Joerg Roedel
Signed-off-by: Greg Kroah-Hartman
---
drivers/iommu/amd_iommu.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -3071,7 +3071,7 @@ static phys_addr_t amd_iommu_iova_to_phy
return 0;
offset_mask = pte_pgsize - 1;
- __pte = *pte & PM_ADDR_MASK;
+ __pte = __sme_clr(*pte & PM_ADDR_MASK);
return (__pte & ~offset_mask) | (iova & offset_mask);
}
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Patch "iommu/amd: Clear memory encryption mask from physical address" has been added to the 4.18-stable tree
This is a note to let you know that I've just added the patch titled
iommu/amd: Clear memory encryption mask from physical address
to the 4.18-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
iommu-amd-clear-memory-encryption-mask-from-physical-address.patch
and it can be found in the queue-4.18 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree,
please let know about it.
From b3e9b515b08e407ab3a026dc2e4d935c48d05f69 Mon Sep 17 00:00:00 2001
From: "Singh, Brijesh"
Date: Thu, 4 Oct 2018 21:40:23 +
Subject: iommu/amd: Clear memory encryption mask from physical address
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
From: Singh, Brijesh
commit b3e9b515b08e407ab3a026dc2e4d935c48d05f69 upstream.
Boris Ostrovsky reported a memory leak with device passthrough when SME
is active.
The VFIO driver uses iommu_iova_to_phys() to get the physical address for
an iova. This physical address is later passed into vfio_unmap_unpin() to
unpin the memory. The vfio_unmap_unpin() uses pfn_valid() before unpinning
the memory. The pfn_valid() check was failing because encryption mask was
part of the physical address returned. This resulted in the memory not
being unpinned and therefore leaked after the guest terminates.
The memory encryption mask must be cleared from the physical address in
iommu_iova_to_phys().
Fixes: 2543a786aa25 ("iommu/amd: Allow the AMD IOMMU to work with memory
encryption")
Reported-by: Boris Ostrovsky
Cc: Tom Lendacky
Cc: Joerg Roedel
Cc:
Cc: Borislav Petkov
Cc: Paolo Bonzini
Cc: Radim Krčmář
Cc: k...@vger.kernel.org
Cc: Boris Ostrovsky
Cc: # 4.14+
Signed-off-by: Brijesh Singh
Signed-off-by: Joerg Roedel
Signed-off-by: Greg Kroah-Hartman
---
drivers/iommu/amd_iommu.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -3073,7 +3073,7 @@ static phys_addr_t amd_iommu_iova_to_phy
return 0;
offset_mask = pte_pgsize - 1;
- __pte = *pte & PM_ADDR_MASK;
+ __pte = __sme_clr(*pte & PM_ADDR_MASK);
return (__pte & ~offset_mask) | (iova & offset_mask);
}
Patches currently in stable-queue which might be from brijesh.si...@amd.com are
queue-4.18/iommu-amd-clear-memory-encryption-mask-from-physical-address.patch
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Patch "iommu/amd: Clear memory encryption mask from physical address" has been added to the 4.14-stable tree
This is a note to let you know that I've just added the patch titled
iommu/amd: Clear memory encryption mask from physical address
to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
iommu-amd-clear-memory-encryption-mask-from-physical-address.patch
and it can be found in the queue-4.14 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree,
please let know about it.
From b3e9b515b08e407ab3a026dc2e4d935c48d05f69 Mon Sep 17 00:00:00 2001
From: "Singh, Brijesh"
Date: Thu, 4 Oct 2018 21:40:23 +
Subject: iommu/amd: Clear memory encryption mask from physical address
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
From: Singh, Brijesh
commit b3e9b515b08e407ab3a026dc2e4d935c48d05f69 upstream.
Boris Ostrovsky reported a memory leak with device passthrough when SME
is active.
The VFIO driver uses iommu_iova_to_phys() to get the physical address for
an iova. This physical address is later passed into vfio_unmap_unpin() to
unpin the memory. The vfio_unmap_unpin() uses pfn_valid() before unpinning
the memory. The pfn_valid() check was failing because encryption mask was
part of the physical address returned. This resulted in the memory not
being unpinned and therefore leaked after the guest terminates.
The memory encryption mask must be cleared from the physical address in
iommu_iova_to_phys().
Fixes: 2543a786aa25 ("iommu/amd: Allow the AMD IOMMU to work with memory
encryption")
Reported-by: Boris Ostrovsky
Cc: Tom Lendacky
Cc: Joerg Roedel
Cc:
Cc: Borislav Petkov
Cc: Paolo Bonzini
Cc: Radim Krčmář
Cc: k...@vger.kernel.org
Cc: Boris Ostrovsky
Cc: # 4.14+
Signed-off-by: Brijesh Singh
Signed-off-by: Joerg Roedel
Signed-off-by: Greg Kroah-Hartman
---
drivers/iommu/amd_iommu.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -3071,7 +3071,7 @@ static phys_addr_t amd_iommu_iova_to_phy
return 0;
offset_mask = pte_pgsize - 1;
- __pte = *pte & PM_ADDR_MASK;
+ __pte = __sme_clr(*pte & PM_ADDR_MASK);
return (__pte & ~offset_mask) | (iova & offset_mask);
}
Patches currently in stable-queue which might be from brijesh.si...@amd.com are
queue-4.14/iommu-amd-clear-memory-encryption-mask-from-physical-address.patch
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: source-id verification failures
On Fri Oct 05 18, Raj, Ashok wrote: On Thu, Oct 04, 2018 at 03:07:46PM -0700, Jacob Pan wrote: On Thu, 4 Oct 2018 13:57:24 -0700 Jerry Snitselaar wrote: > On Thu Oct 04 18, Joerg Roedel wrote: > >Hi Jerry, > > > >thanks for the report. > > > >On Tue, Oct 02, 2018 at 10:25:29AM -0700, Jerry Snitselaar wrote: > >> I've been trying to track down a problem where an hp dl380 gen8 > >> with a Cavium QLogic BR-1860 Fabric Adapter is getting source-id > >> verification failures when running dhclient against that > >> interface. This started showing up when I backported the iova > >> deferred flushing patches. So far this has only been seen on this > >> one system, but I'm trying to understand why it appears with the > >> new deferred flushing code. I also see it with both 4.18.10, and > >> 4.19.0-rc6 kernels. Weird.. IRC, these were there to accomodate phantom functions. Thought PCIe allowed 8bit tag, so if the device needs to allow more than 256 outstanding transactions, one could use the extra functions to account for. I assumed Linux didn't enable phantom functions. If that's the case we also need to ensure all the DMA is aliased properly. I'm assuming if interrupts are generated by other aliases we could block them. Is this device one such? Cheers, Ashok > >> > >> [35645.282021] bna :24:00.1 ens5f1: link down > >> [35645.298396] bna :24:00.0 ens5f0: link down > >> [35650.313210] DMAR: DRHD: handling fault status reg 2 > >> [35650.332477] DMAR: [INTR-REMAP] Request device [24:00.0] fault > >> index 14 [fault reason 38] Blocked an interrupt request due to > >> source-id verification failure [35655.137667] bna :24:00.0 > >> ens5f0: link up [35657.532454] bna :24:00.1 ens5f1: link up > >> [35664.281563] bna :24:00.1 ens5f1: link down [35664.298103] > >> bna :24:00.0 ens5f0: link down [35669.313568] DMAR: DRHD: > >> handling fault status reg 102 [35669.333198] DMAR: [INTR-REMAP] > >> Request device [24:00.0] fault index 14 [fault reason 38] Blocked > >> an interrupt request due to source-id verification failure > >> [35674.081212] bna :24:00.0 ens5f0: link up [35674.981280] bna > >> :24:00.1 ens5f1: link up > >> > >> > >> Any ideas? > > > >No, not yet. Can you please post the output of lscpi -vvv? > > > >Jacob, can you or someone from your team please also have a look into > >this problem report? > > yep. +Ashok Jerry, Could you also dump the interrupt remapping table with this patchset? https://lkml.org/lkml/2018/9/12/44 Thanks, Sorry, I've been on dad duty the past few days. I should be back working on this tonight or tomorrow. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 1/1] iommu/amd: Add default branch in amd_iommu_capable()
On Mon, Oct 08, 2018 at 10:24:19AM +0800, Lu Baolu wrote: > Recent gcc warns about switching on an enumeration, but not having > an explicit case statement for all members of the enumeration. To > show the compiler this is intentional, we simply add a default case > with nothing more than a break statement. > > Signed-off-by: Lu Baolu > --- > drivers/iommu/amd_iommu.c | 2 ++ > 1 file changed, 2 insertions(+) Applied, thanks. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] dt-bindings: iommu: ipmmu-vmsa: Add r8a7744 support
On Thu, Oct 04, 2018 at 05:25:47PM +0100, Biju Das wrote: > Document RZ/G1N (R8A7744) SoC bindings. > > Signed-off-by: Biju Das > Reviewed-by: Chris Paterson Applied, thanks. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v3 03/10] iommu/sva: Manage process address spaces
Hi Jean, > From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com] > Sent: Thursday, September 27, 2018 9:38 PM > To: Liu, Yi L ; Joerg Roedel > Subject: Re: [PATCH v3 03/10] iommu/sva: Manage process address spaces > > On 27/09/2018 04:22, Liu, Yi L wrote: > >> For the "classic" vfio-pci case, "SVA in guest" still means giving the > >> guest control over the whole PASID table. > > > > No, if giving guest control over the whole PASID table, it means guest may > > have > > its own PASID namespace. right? And for vfio-mdev case, it gets PASID from > > host. > > So there would be multiple PASID namespaces. Thinking about the following > scenario: > > > > A PF/VF assigned to a VM via "classic" vfio-pci. And an > > assignable-device-interface > > assigned to this VM via vfio-mdev. If an application in this VM tries to > > manipulate > > these two "devices", it should have the same PASID programmed to them. > > right? > > But as the above comments mentioned, for vfio-pci case, it would get a PASID > from > > its own PASID namespace. While the vfio-mdev case would get a PASID from > > host. > > This would result in conflict. > > Ah I see, if the host assigns an ADI via vfio-mdev and a PCI function > via vfio-pci to the same VM, the guest needs to use the paravirtualized > PASID allocator for the PCI device as well, not just the ADI. In fact > all guest PASIDs need to be allocated through one PV channel, even if > the VM has other vIOMMUs that don't support PV. But I suppose that kind > of VM is unrealistic. yes, such kind of VM is unrealistic. :) > However for SMMUv3 we'll still use the > bind_pasid_table for vfio-pci and let the guest allocate PASIDs, since > the PASID table lives in guest-physical space. I think it's ok. This doesn’t result in any conflict. > > In any case, for the patch series at hand, it means that iommu-sva will > need assistance from the vt-d driver to allocate PASIDs: host uses the > generic allocator, guest uses the PV one. Exactly. > I guess the mm_alloc() op could do that? Do you mean the io_mm_alloc in your SVA patch series? We've got some patch for the PV one. Allen (Baolu Lu) is preparing to send it out for review. I guess we can have more alignment during that patch reviewing. Thanks, Yi Liu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 10/10] arm64: use the generic swiotlb_dma_ops
Now that the generic swiotlb code supports non-coherent DMA we can switch
to it for arm64. For that we need to refactor the existing
alloc/free/mmap/pgprot helpers to be used as the architecture hooks,
and implement the standard arch_sync_dma_for_{device,cpu} hooks for
cache maintaincance in the streaming dma hooks, which also implies
using the generic dma_coherent flag in struct device.
Note that we need to keep the old is_device_dma_coherent function around
for now, so that the shared arm/arm64 Xen code keeps working.
Signed-off-by: Christoph Hellwig
---
arch/arm64/Kconfig | 4 +
arch/arm64/include/asm/device.h | 1 -
arch/arm64/include/asm/dma-mapping.h | 7 +-
arch/arm64/mm/dma-mapping.c | 257 +--
4 files changed, 56 insertions(+), 213 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1b1a0e95c751..c4db5131d837 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -11,6 +11,8 @@ config ARM64
select ARCH_CLOCKSOURCE_DATA
select ARCH_HAS_DEBUG_VIRTUAL
select ARCH_HAS_DEVMEM_IS_ALLOWED
+ select ARCH_HAS_DMA_COHERENT_TO_PFN
+ select ARCH_HAS_DMA_MMAP_PGPROT
select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
select ARCH_HAS_ELF_RANDOMIZE
select ARCH_HAS_FAST_MULTIPLIER
@@ -24,6 +26,8 @@ config ARM64
select ARCH_HAS_SG_CHAIN
select ARCH_HAS_STRICT_KERNEL_RWX
select ARCH_HAS_STRICT_MODULE_RWX
+ select ARCH_HAS_SYNC_DMA_FOR_DEVICE
+ select ARCH_HAS_SYNC_DMA_FOR_CPU
select ARCH_HAS_SYSCALL_WRAPPER
select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
select ARCH_HAVE_NMI_SAFE_CMPXCHG
diff --git a/arch/arm64/include/asm/device.h b/arch/arm64/include/asm/device.h
index 5a5fa47a6b18..3dd3d664c5c5 100644
--- a/arch/arm64/include/asm/device.h
+++ b/arch/arm64/include/asm/device.h
@@ -23,7 +23,6 @@ struct dev_archdata {
#ifdef CONFIG_XEN
const struct dma_map_ops *dev_dma_ops;
#endif
- bool dma_coherent;
};
struct pdev_archdata {
diff --git a/arch/arm64/include/asm/dma-mapping.h
b/arch/arm64/include/asm/dma-mapping.h
index b7847eb8a7bb..c41f3fb1446c 100644
--- a/arch/arm64/include/asm/dma-mapping.h
+++ b/arch/arm64/include/asm/dma-mapping.h
@@ -44,10 +44,13 @@ void arch_teardown_dma_ops(struct device *dev);
#define arch_teardown_dma_ops arch_teardown_dma_ops
#endif
-/* do not use this function in a driver */
+/*
+ * Do not use this function in a driver, it is only provided for
+ * arch/arm/mm/xen.c, which is used by arm64 as well.
+ */
static inline bool is_device_dma_coherent(struct device *dev)
{
- return dev->archdata.dma_coherent;
+ return dev->dma_coherent;
}
#endif /* __KERNEL__ */
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index eee6cfcfde9e..3c75d69b54e7 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -25,6 +25,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -32,16 +33,6 @@
#include
-static int swiotlb __ro_after_init;
-
-static pgprot_t __get_dma_pgprot(unsigned long attrs, pgprot_t prot,
-bool coherent)
-{
- if (!coherent || (attrs & DMA_ATTR_WRITE_COMBINE))
- return pgprot_writecombine(prot);
- return prot;
-}
-
static struct gen_pool *atomic_pool __ro_after_init;
#define DEFAULT_DMA_COHERENT_POOL_SIZE SZ_256K
@@ -91,18 +82,16 @@ static int __free_from_pool(void *start, size_t size)
return 1;
}
-static void *__dma_alloc(struct device *dev, size_t size,
-dma_addr_t *dma_handle, gfp_t flags,
-unsigned long attrs)
+void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
+ gfp_t flags, unsigned long attrs)
{
struct page *page;
void *ptr, *coherent_ptr;
- bool coherent = is_device_dma_coherent(dev);
- pgprot_t prot = __get_dma_pgprot(attrs, PAGE_KERNEL, false);
+ pgprot_t prot = pgprot_writecombine(PAGE_KERNEL);
size = PAGE_ALIGN(size);
- if (!coherent && !gfpflags_allow_blocking(flags)) {
+ if (!gfpflags_allow_blocking(flags)) {
struct page *page = NULL;
void *addr = __alloc_from_pool(size, , flags);
@@ -116,10 +105,6 @@ static void *__dma_alloc(struct device *dev, size_t size,
if (!ptr)
goto no_mem;
- /* no need for non-cacheable mapping if coherent */
- if (coherent)
- return ptr;
-
/* remove any dirty cache lines on the kernel alias */
__dma_flush_area(ptr, size);
@@ -138,125 +123,50 @@ static void *__dma_alloc(struct device *dev, size_t size,
return NULL;
}
-static void __dma_free(struct device *dev, size_t size,
- void *vaddr, dma_addr_t dma_handle,
- unsigned long attrs)
-{
-
[PATCH 09/10] swiotlb: add support for non-coherent DMA
Handle architectures that are not cache coherent directly in the main
swiotlb code by calling arch_sync_dma_for_{device,cpu} in all the right
places from the various dma_map/unmap/sync methods when the device is
non-coherent.
Because swiotlb now uses dma_direct_alloc for the coherent allocation
that side is already taken care of by the dma-direct code calling into
arch_dma_{alloc,free} for devices that are non-coherent.
Signed-off-by: Christoph Hellwig
---
kernel/dma/swiotlb.c | 23 +--
1 file changed, 17 insertions(+), 6 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 475a41eff3dc..52885b274368 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -21,6 +21,7 @@
#include
#include
+#include
#include
#include
#include
@@ -677,6 +678,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct
page *page,
dma_addr = swiotlb_bounce_page(dev, , size, dir, attrs);
}
+ if (!dev_is_dma_coherent(dev) &&
+ (attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
+ arch_sync_dma_for_device(dev, phys, size, dir);
+
return dma_addr;
}
@@ -696,6 +701,10 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t
dev_addr,
BUG_ON(dir == DMA_NONE);
+ if (!dev_is_dma_coherent(hwdev) &&
+ (attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
+ arch_sync_dma_for_cpu(hwdev, paddr, size, dir);
+
if (is_swiotlb_buffer(paddr)) {
swiotlb_tbl_unmap_single(hwdev, paddr, size, dir, attrs);
return;
@@ -732,15 +741,17 @@ swiotlb_sync_single(struct device *hwdev, dma_addr_t
dev_addr,
BUG_ON(dir == DMA_NONE);
- if (is_swiotlb_buffer(paddr)) {
+ if (!dev_is_dma_coherent(hwdev) && target == SYNC_FOR_CPU)
+ arch_sync_dma_for_cpu(hwdev, paddr, size, dir);
+
+ if (is_swiotlb_buffer(paddr))
swiotlb_tbl_sync_single(hwdev, paddr, size, dir, target);
- return;
- }
- if (dir != DMA_FROM_DEVICE)
- return;
+ if (!dev_is_dma_coherent(hwdev) && target == SYNC_FOR_DEVICE)
+ arch_sync_dma_for_device(hwdev, paddr, size, dir);
- dma_mark_clean(phys_to_virt(paddr), size);
+ if (!is_swiotlb_buffer(paddr) && dir == DMA_FROM_DEVICE)
+ dma_mark_clean(phys_to_virt(paddr), size);
}
void
--
2.19.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 07/10] swiotlb: refactor swiotlb_map_page
Remove the somewhat useless map_single function, and replace it with a
swiotlb_bounce_page handler that handles everything related to actually
bouncing a page.
Signed-off-by: Christoph Hellwig
---
kernel/dma/swiotlb.c | 77 +---
1 file changed, 36 insertions(+), 41 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 15755d7a5242..4d7a4d85d71e 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -543,26 +543,6 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
return tlb_addr;
}
-/*
- * Allocates bounce buffer and returns its physical address.
- */
-static phys_addr_t
-map_single(struct device *hwdev, phys_addr_t phys, size_t size,
- enum dma_data_direction dir, unsigned long attrs)
-{
- dma_addr_t start_dma_addr;
-
- if (swiotlb_force == SWIOTLB_NO_FORCE) {
- dev_warn_ratelimited(hwdev, "Cannot do DMA to address %pa\n",
-);
- return SWIOTLB_MAP_ERROR;
- }
-
- start_dma_addr = __phys_to_dma(hwdev, io_tlb_start);
- return swiotlb_tbl_map_single(hwdev, start_dma_addr, phys, size,
- dir, attrs);
-}
-
/*
* tlb_addr is the physical address of the bounce buffer to unmap.
*/
@@ -714,6 +694,34 @@ static bool swiotlb_free_buffer(struct device *dev, size_t
size,
return true;
}
+static dma_addr_t swiotlb_bounce_page(struct device *dev, phys_addr_t *phys,
+ size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+ dma_addr_t dma_addr;
+
+ if (unlikely(swiotlb_force == SWIOTLB_NO_FORCE)) {
+ dev_warn_ratelimited(dev,
+ "Cannot do DMA to address %pa\n", phys);
+ return DIRECT_MAPPING_ERROR;
+ }
+
+ /* Oh well, have to allocate and map a bounce buffer. */
+ *phys = swiotlb_tbl_map_single(dev, __phys_to_dma(dev, io_tlb_start),
+ *phys, size, dir, attrs);
+ if (*phys == SWIOTLB_MAP_ERROR)
+ return DIRECT_MAPPING_ERROR;
+
+ /* Ensure that the address returned is DMA'ble */
+ dma_addr = __phys_to_dma(dev, *phys);
+ if (unlikely(!dma_capable(dev, dma_addr, size))) {
+ swiotlb_tbl_unmap_single(dev, *phys, size, dir,
+ attrs | DMA_ATTR_SKIP_CPU_SYNC);
+ return DIRECT_MAPPING_ERROR;
+ }
+
+ return dma_addr;
+}
+
/*
* Map a single buffer of the indicated size for DMA in streaming mode. The
* physical address to use is returned.
@@ -726,8 +734,8 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page
*page,
enum dma_data_direction dir,
unsigned long attrs)
{
- phys_addr_t map, phys = page_to_phys(page) + offset;
- dma_addr_t dev_addr = phys_to_dma(dev, phys);
+ phys_addr_t phys = page_to_phys(page) + offset;
+ dma_addr_t dma_addr = phys_to_dma(dev, phys);
BUG_ON(dir == DMA_NONE);
/*
@@ -735,26 +743,13 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct
page *page,
* we can safely return the device addr and not worry about bounce
* buffering it.
*/
- if (dma_capable(dev, dev_addr, size) && swiotlb_force != SWIOTLB_FORCE)
- return dev_addr;
-
- trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
-
- /* Oh well, have to allocate and map a bounce buffer. */
- map = map_single(dev, phys, size, dir, attrs);
- if (map == SWIOTLB_MAP_ERROR)
- return DIRECT_MAPPING_ERROR;
-
- dev_addr = __phys_to_dma(dev, map);
-
- /* Ensure that the address returned is DMA'ble */
- if (dma_capable(dev, dev_addr, size))
- return dev_addr;
-
- attrs |= DMA_ATTR_SKIP_CPU_SYNC;
- swiotlb_tbl_unmap_single(dev, map, size, dir, attrs);
+ if (!dma_capable(dev, dma_addr, size) ||
+ swiotlb_force == SWIOTLB_FORCE) {
+ trace_swiotlb_bounced(dev, dma_addr, size, swiotlb_force);
+ dma_addr = swiotlb_bounce_page(dev, , size, dir, attrs);
+ }
- return DIRECT_MAPPING_ERROR;
+ return dma_addr;
}
/*
--
2.19.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 08/10] swiotlb: don't dip into swiotlb pool for coherent allocations
All architectures that support swiotlb also have a zone that backs up
these less than full addressing allocations (usually ZONE_DMA32).
Because of that it is rather pointless to fall back to the global swiotlb
buffer if the normal dma direct allocation failed - the only thing this
will do is to eat up bounce buffers that would be more useful to serve
streaming mappings.
Signed-off-by: Christoph Hellwig
---
arch/arm64/mm/dma-mapping.c | 6 +--
include/linux/swiotlb.h | 5 --
kernel/dma/swiotlb.c| 105 +---
3 files changed, 5 insertions(+), 111 deletions(-)
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index 8d91b927e09e..eee6cfcfde9e 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -112,7 +112,7 @@ static void *__dma_alloc(struct device *dev, size_t size,
return addr;
}
- ptr = swiotlb_alloc(dev, size, dma_handle, flags, attrs);
+ ptr = dma_direct_alloc_pages(dev, size, dma_handle, flags, attrs);
if (!ptr)
goto no_mem;
@@ -133,7 +133,7 @@ static void *__dma_alloc(struct device *dev, size_t size,
return coherent_ptr;
no_map:
- swiotlb_free(dev, size, ptr, *dma_handle, attrs);
+ dma_direct_free_pages(dev, size, ptr, *dma_handle, attrs);
no_mem:
return NULL;
}
@@ -151,7 +151,7 @@ static void __dma_free(struct device *dev, size_t size,
return;
vunmap(vaddr);
}
- swiotlb_free(dev, size, swiotlb_addr, dma_handle, attrs);
+ dma_direct_free_pages(dev, size, swiotlb_addr, dma_handle, attrs);
}
static dma_addr_t __swiotlb_map_page(struct device *dev, struct page *page,
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index f847c1b265c4..a387b59640a4 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -67,11 +67,6 @@ extern void swiotlb_tbl_sync_single(struct device *hwdev,
/* Accessory functions. */
-void *swiotlb_alloc(struct device *hwdev, size_t size, dma_addr_t *dma_handle,
- gfp_t flags, unsigned long attrs);
-void swiotlb_free(struct device *dev, size_t size, void *vaddr,
- dma_addr_t dma_addr, unsigned long attrs);
-
extern dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size,
enum dma_data_direction dir,
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 4d7a4d85d71e..475a41eff3dc 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -622,78 +622,6 @@ void swiotlb_tbl_sync_single(struct device *hwdev,
phys_addr_t tlb_addr,
}
}
-static inline bool dma_coherent_ok(struct device *dev, dma_addr_t addr,
- size_t size)
-{
- u64 mask = DMA_BIT_MASK(32);
-
- if (dev && dev->coherent_dma_mask)
- mask = dev->coherent_dma_mask;
- return addr + size - 1 <= mask;
-}
-
-static void *
-swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
- unsigned long attrs)
-{
- phys_addr_t phys_addr;
-
- if (swiotlb_force == SWIOTLB_NO_FORCE)
- goto out_warn;
-
- phys_addr = swiotlb_tbl_map_single(dev,
- __phys_to_dma(dev, io_tlb_start),
- 0, size, DMA_FROM_DEVICE, attrs);
- if (phys_addr == SWIOTLB_MAP_ERROR)
- goto out_warn;
-
- *dma_handle = __phys_to_dma(dev, phys_addr);
- if (!dma_coherent_ok(dev, *dma_handle, size))
- goto out_unmap;
-
- memset(phys_to_virt(phys_addr), 0, size);
- return phys_to_virt(phys_addr);
-
-out_unmap:
- dev_warn(dev, "hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n",
- (unsigned long long)dev->coherent_dma_mask,
- (unsigned long long)*dma_handle);
-
- /*
-* DMA_TO_DEVICE to avoid memcpy in unmap_single.
-* DMA_ATTR_SKIP_CPU_SYNC is optional.
-*/
- swiotlb_tbl_unmap_single(dev, phys_addr, size, DMA_TO_DEVICE,
- DMA_ATTR_SKIP_CPU_SYNC);
-out_warn:
- if (!(attrs & DMA_ATTR_NO_WARN) && printk_ratelimit()) {
- dev_warn(dev,
- "swiotlb: coherent allocation failed, size=%zu\n",
- size);
- dump_stack();
- }
- return NULL;
-}
-
-static bool swiotlb_free_buffer(struct device *dev, size_t size,
- dma_addr_t dma_addr)
-{
- phys_addr_t phys_addr = dma_to_phys(dev, dma_addr);
-
- WARN_ON_ONCE(irqs_disabled());
-
- if (!is_swiotlb_buffer(phys_addr))
- return false;
-
- /*
-* DMA_TO_DEVICE to avoid memcpy in swiotlb_tbl_unmap_single.
-* DMA_ATTR_SKIP_CPU_SYNC is optional.
-*/
- swiotlb_tbl_unmap_single(dev, phys_addr, size, DMA_TO_DEVICE,
-
[PATCH 06/10] swiotlb: use swiotlb_map_page in swiotlb_map_sg_attrs
No need to duplicate the code - map_sg is equivalent to map_page
for each page in the scatterlist.
Signed-off-by: Christoph Hellwig
---
kernel/dma/swiotlb.c | 34 --
1 file changed, 12 insertions(+), 22 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 15335f3a1bf3..15755d7a5242 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -845,37 +845,27 @@ swiotlb_sync_single_for_device(struct device *hwdev,
dma_addr_t dev_addr,
* same here.
*/
int
-swiotlb_map_sg_attrs(struct device *hwdev, struct scatterlist *sgl, int nelems,
+swiotlb_map_sg_attrs(struct device *dev, struct scatterlist *sgl, int nelems,
enum dma_data_direction dir, unsigned long attrs)
{
struct scatterlist *sg;
int i;
- BUG_ON(dir == DMA_NONE);
-
for_each_sg(sgl, sg, nelems, i) {
- phys_addr_t paddr = sg_phys(sg);
- dma_addr_t dev_addr = phys_to_dma(hwdev, paddr);
-
- if (swiotlb_force == SWIOTLB_FORCE ||
- !dma_capable(hwdev, dev_addr, sg->length)) {
- phys_addr_t map = map_single(hwdev, sg_phys(sg),
-sg->length, dir, attrs);
- if (map == SWIOTLB_MAP_ERROR) {
- /* Don't panic here, we expect map_sg users
- to do proper error handling. */
- attrs |= DMA_ATTR_SKIP_CPU_SYNC;
- swiotlb_unmap_sg_attrs(hwdev, sgl, i, dir,
- attrs);
- sg_dma_len(sgl) = 0;
- return 0;
- }
- sg->dma_address = __phys_to_dma(hwdev, map);
- } else
- sg->dma_address = dev_addr;
+ sg->dma_address = swiotlb_map_page(dev, sg_page(sg), sg->offset,
+ sg->length, dir, attrs);
+ if (sg->dma_address == DIRECT_MAPPING_ERROR)
+ goto out_error;
sg_dma_len(sg) = sg->length;
}
+
return nelems;
+
+out_error:
+ swiotlb_unmap_sg_attrs(dev, sgl, i, dir,
+ attrs | DMA_ATTR_SKIP_CPU_SYNC);
+ sg_dma_len(sgl) = 0;
+ return 0;
}
/*
--
2.19.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 05/10] swiotlb: merge swiotlb_unmap_page and unmap_single
Signed-off-by: Christoph Hellwig
---
kernel/dma/swiotlb.c | 15 ---
1 file changed, 4 insertions(+), 11 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 11dbcd80b4a6..15335f3a1bf3 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -765,9 +765,9 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page
*page,
* After this call, reads by the cpu to the buffer are guaranteed to see
* whatever the device wrote there.
*/
-static void unmap_single(struct device *hwdev, dma_addr_t dev_addr,
-size_t size, enum dma_data_direction dir,
-unsigned long attrs)
+void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs)
{
phys_addr_t paddr = dma_to_phys(hwdev, dev_addr);
@@ -790,13 +790,6 @@ static void unmap_single(struct device *hwdev, dma_addr_t
dev_addr,
dma_mark_clean(phys_to_virt(paddr), size);
}
-void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
- size_t size, enum dma_data_direction dir,
- unsigned long attrs)
-{
- unmap_single(hwdev, dev_addr, size, dir, attrs);
-}
-
/*
* Make physical memory consistent for a single streaming mode DMA translation
* after a transfer.
@@ -900,7 +893,7 @@ swiotlb_unmap_sg_attrs(struct device *hwdev, struct
scatterlist *sgl,
BUG_ON(dir == DMA_NONE);
for_each_sg(sgl, sg, nelems, i)
- unmap_single(hwdev, sg->dma_address, sg_dma_len(sg), dir,
+ swiotlb_unmap_page(hwdev, sg->dma_address, sg_dma_len(sg), dir,
attrs);
}
--
2.19.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 02/10] swiotlb: mark is_swiotlb_buffer static
Signed-off-by: Christoph Hellwig
---
include/linux/swiotlb.h | 1 -
kernel/dma/swiotlb.c| 2 +-
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 965be92c33b5..7ef541ce8f34 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -121,7 +121,6 @@ static inline unsigned int swiotlb_max_segment(void) {
return 0; }
#endif
extern void swiotlb_print_info(void);
-extern int is_swiotlb_buffer(phys_addr_t paddr);
extern void swiotlb_set_max_segment(unsigned int);
extern const struct dma_map_ops swiotlb_dma_ops;
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 9062b14bc7f4..26d3af52956f 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -429,7 +429,7 @@ void __init swiotlb_exit(void)
max_segment = 0;
}
-int is_swiotlb_buffer(phys_addr_t paddr)
+static int is_swiotlb_buffer(phys_addr_t paddr)
{
return paddr >= io_tlb_start && paddr < io_tlb_end;
}
--
2.19.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 04/10] swiotlb: remove the overflow buffer
Like all other dma mapping drivers just return an error code instead
of an actual memory buffer. The reason for the overflow buffer was
that at the time swiotlb was invented there was no way to check for
dma mapping errors, but this has long been fixed.
Signed-off-by: Christoph Hellwig
---
arch/arm64/mm/dma-mapping.c | 2 +-
arch/powerpc/kernel/dma-swiotlb.c | 4 +--
include/linux/dma-direct.h| 2 ++
include/linux/swiotlb.h | 3 --
kernel/dma/direct.c | 2 --
kernel/dma/swiotlb.c | 59 ++-
6 files changed, 8 insertions(+), 64 deletions(-)
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index 072c51fb07d7..8d91b927e09e 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -324,7 +324,7 @@ static int __swiotlb_dma_supported(struct device *hwdev,
u64 mask)
static int __swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t addr)
{
if (swiotlb)
- return swiotlb_dma_mapping_error(hwdev, addr);
+ return dma_direct_mapping_error(hwdev, addr);
return 0;
}
diff --git a/arch/powerpc/kernel/dma-swiotlb.c
b/arch/powerpc/kernel/dma-swiotlb.c
index 88f3963ca30f..5fc335f4d9cd 100644
--- a/arch/powerpc/kernel/dma-swiotlb.c
+++ b/arch/powerpc/kernel/dma-swiotlb.c
@@ -11,7 +11,7 @@
*
*/
-#include
+#include
#include
#include
#include
@@ -59,7 +59,7 @@ const struct dma_map_ops powerpc_swiotlb_dma_ops = {
.sync_single_for_device = swiotlb_sync_single_for_device,
.sync_sg_for_cpu = swiotlb_sync_sg_for_cpu,
.sync_sg_for_device = swiotlb_sync_sg_for_device,
- .mapping_error = swiotlb_dma_mapping_error,
+ .mapping_error = dma_direct_mapping_error,
.get_required_mask = swiotlb_powerpc_get_required,
};
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index fbca184ff5a0..bd73e7a91410 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -5,6 +5,8 @@
#include
#include
+#define DIRECT_MAPPING_ERROR 0
+
#ifdef CONFIG_ARCH_HAS_PHYS_TO_DMA
#include
#else
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 7ef541ce8f34..f847c1b265c4 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -106,9 +106,6 @@ extern void
swiotlb_sync_sg_for_device(struct device *hwdev, struct scatterlist *sg,
int nelems, enum dma_data_direction dir);
-extern int
-swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t dma_addr);
-
extern int
swiotlb_dma_supported(struct device *hwdev, u64 mask);
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 674a8da22844..12798abba55e 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -14,8 +14,6 @@
#include
#include
-#define DIRECT_MAPPING_ERROR 0
-
/*
* Most architectures use ZONE_DMA for the first 16 Megabytes, but
* some use it for entirely different regions:
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 69bf305ee5f8..11dbcd80b4a6 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -72,13 +72,6 @@ static phys_addr_t io_tlb_start, io_tlb_end;
*/
static unsigned long io_tlb_nslabs;
-/*
- * When the IOMMU overflows we return a fallback buffer. This sets the size.
- */
-static unsigned long io_tlb_overflow = 32*1024;
-
-static phys_addr_t io_tlb_overflow_buffer;
-
/*
* This is a free list describing the number of free entries available from
* each index
@@ -126,7 +119,6 @@ setup_io_tlb_npages(char *str)
return 0;
}
early_param("swiotlb", setup_io_tlb_npages);
-/* make io_tlb_overflow tunable too? */
unsigned long swiotlb_nr_tbl(void)
{
@@ -194,16 +186,10 @@ void __init swiotlb_update_mem_attributes(void)
bytes = PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT);
set_memory_decrypted((unsigned long)vaddr, bytes >> PAGE_SHIFT);
memset(vaddr, 0, bytes);
-
- vaddr = phys_to_virt(io_tlb_overflow_buffer);
- bytes = PAGE_ALIGN(io_tlb_overflow);
- set_memory_decrypted((unsigned long)vaddr, bytes >> PAGE_SHIFT);
- memset(vaddr, 0, bytes);
}
int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
{
- void *v_overflow_buffer;
unsigned long i, bytes;
bytes = nslabs << IO_TLB_SHIFT;
@@ -212,17 +198,6 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long
nslabs, int verbose)
io_tlb_start = __pa(tlb);
io_tlb_end = io_tlb_start + bytes;
- /*
-* Get the overflow emergency buffer
-*/
- v_overflow_buffer = memblock_virt_alloc_low_nopanic(
- PAGE_ALIGN(io_tlb_overflow),
- PAGE_SIZE);
- if (!v_overflow_buffer)
- return -ENOMEM;
-
- io_tlb_overflow_buffer = __pa(v_overflow_buffer);
-
/*
* Allocate and
[PATCH 01/10] swiotlb: remove a pointless comment
This comments describes an aspect of the map_sg interface that isn't
even exploited by swiotlb.
Signed-off-by: Christoph Hellwig
---
kernel/dma/swiotlb.c | 6 --
1 file changed, 6 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 4f8a6dbf0b60..9062b14bc7f4 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -925,12 +925,6 @@ swiotlb_sync_single_for_device(struct device *hwdev,
dma_addr_t dev_addr,
* appropriate dma address and length. They are obtained via
* sg_dma_{address,length}(SG).
*
- * NOTE: An implementation may be able to use a smaller number of
- * DMA address/length pairs than there are SG table elements.
- * (for example via virtual mapping capabilities)
- * The routine returns the number of addr/length pairs actually
- * used, at most nents.
- *
* Device ownership issues as mentioned above for swiotlb_map_page are the
* same here.
*/
--
2.19.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 03/10] swiotlb: do not panic on mapping failures
All properly written drivers now have error handling in the
dma_map_single / dma_map_page callers. As swiotlb_tbl_map_single already
prints a useful warning when running out of swiotlb pool swace we can
also remove swiotlb_full entirely as it serves no purpose now.
Signed-off-by: Christoph Hellwig
---
kernel/dma/swiotlb.c | 33 +
1 file changed, 1 insertion(+), 32 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 26d3af52956f..69bf305ee5f8 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -761,34 +761,6 @@ static bool swiotlb_free_buffer(struct device *dev, size_t
size,
return true;
}
-static void
-swiotlb_full(struct device *dev, size_t size, enum dma_data_direction dir,
-int do_panic)
-{
- if (swiotlb_force == SWIOTLB_NO_FORCE)
- return;
-
- /*
-* Ran out of IOMMU space for this operation. This is very bad.
-* Unfortunately the drivers cannot handle this operation properly.
-* unless they check for dma_mapping_error (most don't)
-* When the mapping is small enough return a static buffer to limit
-* the damage, or panic when the transfer is too big.
-*/
- dev_err_ratelimited(dev, "DMA: Out of SW-IOMMU space for %zu bytes\n",
- size);
-
- if (size <= io_tlb_overflow || !do_panic)
- return;
-
- if (dir == DMA_BIDIRECTIONAL)
- panic("DMA: Random memory could be DMA accessed\n");
- if (dir == DMA_FROM_DEVICE)
- panic("DMA: Random memory could be DMA written\n");
- if (dir == DMA_TO_DEVICE)
- panic("DMA: Random memory could be DMA read\n");
-}
-
/*
* Map a single buffer of the indicated size for DMA in streaming mode. The
* physical address to use is returned.
@@ -817,10 +789,8 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct
page *page,
/* Oh well, have to allocate and map a bounce buffer. */
map = map_single(dev, phys, size, dir, attrs);
- if (map == SWIOTLB_MAP_ERROR) {
- swiotlb_full(dev, size, dir, 1);
+ if (map == SWIOTLB_MAP_ERROR)
return __phys_to_dma(dev, io_tlb_overflow_buffer);
- }
dev_addr = __phys_to_dma(dev, map);
@@ -948,7 +918,6 @@ swiotlb_map_sg_attrs(struct device *hwdev, struct
scatterlist *sgl, int nelems,
if (map == SWIOTLB_MAP_ERROR) {
/* Don't panic here, we expect map_sg users
to do proper error handling. */
- swiotlb_full(hwdev, sg->length, dir, 0);
attrs |= DMA_ATTR_SKIP_CPU_SYNC;
swiotlb_unmap_sg_attrs(hwdev, sgl, i, dir,
attrs);
--
2.19.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
move swiotlb noncoherent dma support from arm64 to generic code V2
Hi all, this series starts with various swiotlb cleanups, then adds support for non-cache coherent devices to the generic swiotlb support, and finally switches arm64 to use the generic code. Given that this series depends on patches in the dma-mapping tree, or pending for it I've also published a git tree here: git://git.infradead.org/users/hch/misc.git swiotlb-noncoherent.2 Gitweb: http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-noncoherent.2 Changes since v1: - make sure arm64 always calls dma_direct_alloc_pages to avoid circular calling conventions - add a patch in to stop dipping into the swiotlb pool for coherent allocations - this just depletes the swiotlb pool for no good reason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3] dma-debug: Check for drivers mapping invalid addresses in dma_map_single()
Thanks, applied to the dma-mapping tree for 4.20. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] dt-bindings: iommu: ipmmu-vmsa: Add r8a7744 support
On Thu, Oct 04, 2018 at 05:25:47PM +0100, Biju Das wrote: > Document RZ/G1N (R8A7744) SoC bindings. > > Signed-off-by: Biju Das > Reviewed-by: Chris Paterson Reviewed-by: Simon Horman ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v3] dma-debug: Check for drivers mapping invalid addresses in dma_map_single()
I recently debugged a DMA mapping oops where a driver was trying to map
a buffer returned from request_firmware() with dma_map_single(). Memory
returned from request_firmware() is mapped into the vmalloc region and
this isn't a valid region to map with dma_map_single() per the DMA
documentation's "What memory is DMA'able?" section.
Unfortunately, we don't really check that in the DMA debugging code, so
enabling DMA debugging doesn't help catch this problem. Let's add a new
DMA debug function to check for a vmalloc address or an invalid virtual
address and print a warning if this happens. This makes it a little
easier to debug these sorts of problems, instead of seeing odd behavior
or crashes when drivers attempt to map the vmalloc space for DMA.
Cc: Marek Szyprowski
Reviewed-by: Robin Murphy
Signed-off-by: Stephen Boyd
---
Changes from v2:
* Rename function properly and actually test it!
* Add Robin's reviewed-by tag
Changes from v1:
* Update code to check for invalid virtual address too
* Rename function to debug_dma_map_single()
include/linux/dma-debug.h | 8
include/linux/dma-mapping.h | 1 +
kernel/dma/debug.c | 16
3 files changed, 25 insertions(+)
diff --git a/include/linux/dma-debug.h b/include/linux/dma-debug.h
index a785f2507159..30213adbb6b9 100644
--- a/include/linux/dma-debug.h
+++ b/include/linux/dma-debug.h
@@ -32,6 +32,9 @@ extern void dma_debug_add_bus(struct bus_type *bus);
extern int dma_debug_resize_entries(u32 num_entries);
+extern void debug_dma_map_single(struct device *dev, const void *addr,
+unsigned long len);
+
extern void debug_dma_map_page(struct device *dev, struct page *page,
size_t offset, size_t size,
int direction, dma_addr_t dma_addr,
@@ -103,6 +106,11 @@ static inline int dma_debug_resize_entries(u32 num_entries)
return 0;
}
+static inline void debug_dma_map_single(struct device *dev, const void *addr,
+ unsigned long len)
+{
+}
+
static inline void debug_dma_map_page(struct device *dev, struct page *page,
size_t offset, size_t size,
int direction, dma_addr_t dma_addr,
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 562af6b45f23..547a48bcfa3d 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -229,6 +229,7 @@ static inline dma_addr_t dma_map_single_attrs(struct device
*dev, void *ptr,
dma_addr_t addr;
BUG_ON(!valid_dma_direction(dir));
+ debug_dma_map_single(dev, ptr, size);
addr = ops->map_page(dev, virt_to_page(ptr),
offset_in_page(ptr), size,
dir, attrs);
diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c
index c007d25bee09..231ca4628062 100644
--- a/kernel/dma/debug.c
+++ b/kernel/dma/debug.c
@@ -1312,6 +1312,22 @@ static void check_sg_segment(struct device *dev, struct
scatterlist *sg)
#endif
}
+void debug_dma_map_single(struct device *dev, const void *addr,
+ unsigned long len)
+{
+ if (unlikely(dma_debug_disabled()))
+ return;
+
+ if (!virt_addr_valid(addr))
+ err_printk(dev, NULL, "DMA-API: device driver maps memory from
invalid area [addr=%p] [len=%lu]\n",
+ addr, len);
+
+ if (is_vmalloc_addr(addr))
+ err_printk(dev, NULL, "DMA-API: device driver maps memory from
vmalloc area [addr=%p] [len=%lu]\n",
+ addr, len);
+}
+EXPORT_SYMBOL(debug_dma_map_single);
+
void debug_dma_map_page(struct device *dev, struct page *page, size_t offset,
size_t size, int direction, dma_addr_t dma_addr,
bool map_single)
--
Sent by a computer through tubes
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] dma-direct: document the zone selection logic
Ben, does this resolve your issues with the confusing zone selection? On Mon, Oct 01, 2018 at 01:10:16PM -0700, Christoph Hellwig wrote: > What we are doing here isn't quite obvious, so add a comment explaining > it. > > Signed-off-by: Christoph Hellwig > --- > kernel/dma/direct.c | 9 - > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c > index ba6f5956a291..14b966e2349a 100644 > --- a/kernel/dma/direct.c > +++ b/kernel/dma/direct.c > @@ -84,7 +84,14 @@ static gfp_t __dma_direct_optimal_gfp_mask(struct device > *dev, u64 dma_mask, > else > *phys_mask = dma_to_phys(dev, dma_mask); > > - /* GFP_DMA32 and GFP_DMA are no ops without the corresponding zones: */ > + /* > + * Optimistically try the zone that the physicall address mask falls > + * into first. If that returns memory that isn't actually addressable > + * we will fallback to the next lower zone and try again. > + * > + * Note that GFP_DMA32 and GFP_DMA are no ops without the corresponding > + * zones. > + */ > if (*phys_mask <= DMA_BIT_MASK(ARCH_ZONE_DMA_BITS)) > return GFP_DMA; > if (*phys_mask <= DMA_BIT_MASK(32)) > -- > 2.19.0 > > ___ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu ---end quoted text--- ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: fix up nowarn confusion in the dma mapping layer
Any comments on these rather trivial patches? On Mon, Oct 01, 2018 at 01:12:55PM -0700, Christoph Hellwig wrote: > Hi all, > > this series sorts out how we deal with the nowarn flags in the dma > mapping code. We still support __GFP_NOWARN for the legacy APIs that > don't support passing the dma specific flags, but we generally want > every implementation to actually check DMA_ATTR_NO_WARN instead. > ___ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu ---end quoted text--- ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2] dma-debug: Check for drivers mapping invalid addresses in dma_map_single()
On Sun, Oct 07, 2018 at 09:32:30PM -0700, Stephen Boyd wrote: > > Reviewed-by: Robin Murphy > > Thanks for checking! ;) Can you resend the fixed version to make my life easier? ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
