Re: [PATCH] Revert "firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module"
On Thu, Nov 19, 2020 at 11:42 AM Thierry Reding
wrote:
>
> From: Thierry Reding
>
> Commit d0511b5496c0 ("firmware: QCOM_SCM: Allow qcom_scm driver to be
> loadable as a permenent module") causes the ARM SMMU driver to be built
> as a loadable module when using the Aarch64 default configuration. This
> in turn causes problems because if the loadable module is not shipped
> in an initial ramdisk, then the deferred probe timeout mechanism will
> cause all SMMU masters to probe without SMMU support and fall back to
> just plain DMA ops (not IOMMU-backed).
>
> Once the system has mounted the rootfs, the ARM SMMU driver will then
> be loaded, but since the ARM SMMU driver faults by default, this causes
> a slew of SMMU faults for the SMMU masters that have already been set
> up with plain DMA ops and cause these devices to malfunction.
>
FWIW I had the same issues on the Qualcomm platform and merged a patch
that turns QCOM_SCM=y in arm64 defconfig earlier today. So this should
hide the problem in next linux-next. (And it really should be =y in
defconfig regardless of this revert or not).
> Revert that commit to unbreak things while we look for an alternative
> solution.
>
I don't fancy the fact that we have a situation where if you're
unlucky to have probe deferrals lingering past late initcall things
will start to just break and this from a growing number of resource
types. But I also don't see any alternatives to fixing the kernel to
handle this gracefully.
Regards,
Bjorn
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2] iommu/amd: Enforce 4k mapping for certain IOMMU data structures
On 11/19/20 8:30 PM, Suravee Suthikulpanit wrote:
> Will,
>
> To answer your questions from v1 thread.
>
> On 11/18/20 5:57 AM, Will Deacon wrote:
> > On 11/5/20 9:58 PM, Suravee Suthikulpanit wrote:
> >> AMD IOMMU requires 4k-aligned pages for the event log, the PPR log,
> >> and the completion wait write-back regions. However, when allocating
> >> the pages, they could be part of large mapping (e.g. 2M) page.
> >> This causes #PF due to the SNP RMP hardware enforces the check based
> >> on the page level for these data structures.
> >
> > Please could you include an example backtrace here?
>
> Unfortunately, we don't actually have the backtrace available here.
> This information is based on the SEV-SNP specification.
>
> >> So, fix by calling set_memory_4k() on the allocated pages.
> >
> > I think I'm missing something here. set_memory_4k() will break the
> kernel
> > linear mapping up into page granular mappings, but the IOMMU isn't
> using
> > that mapping, right?
>
> That's correct. This does not affect the IOMMU, but it affects the PSP
> FW.
>
> > It's just using the physical address returned by
> iommu_virt_to_phys(), so why does it matter?
> >
> > Just be nice to capture some of this rationale in the log,
> especially as
> > I'm not familiar with this device.
>
> According to the AMD SEV-SNP white paper
> (https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf),
> the Reverse Map Table (RMP) contains one entry for every 4K page of
> DRAM that may be used by the VM. In this case, the pages allocated by
> the IOMMU driver are added as 4K entries in the RMP table by the
> SEV-SNP FW.
>
> During the page table walk, the RMP checks if the page is owned by the
> hypervisor. Without calling set_memory_4k() to break the mapping up
> into 4K pages, pages could end up being part of large mapping (e.g. 2M
> page), in which the page access would be denied and result in #PF.
Since the page is added as a 4K page in the RMP table by the SEV-SNP FW,
so we need to split the physmap to ensure that this page will be access
with a 4K mapping from the x86. If the page is part of large page then
write access will cause a RMP violation (i.e #PF), this is because SNP
hardware enforce that the CPU page level walk must match with page-level
programmed in the RMP table.
>
> >> Fixes: commit c69d89aff393 ("iommu/amd: Use 4K page for completion
> wait write-back semaphore")
> >
> > I couldn't figure out how that commit could cause this problem.
> Please can
> > you explain that to me?
>
> Hope this helps clarify. If so, I'll update the commit log and send
> out V3.
>
> Thanks,
> Suravee
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2] iommu/amd: Enforce 4k mapping for certain IOMMU data structures
Will,
To answer your questions from v1 thread.
On 11/18/20 5:57 AM, Will Deacon wrote:
> On 11/5/20 9:58 PM, Suravee Suthikulpanit wrote:
>> AMD IOMMU requires 4k-aligned pages for the event log, the PPR log,
>> and the completion wait write-back regions. However, when allocating
>> the pages, they could be part of large mapping (e.g. 2M) page.
>> This causes #PF due to the SNP RMP hardware enforces the check based
>> on the page level for these data structures.
>
> Please could you include an example backtrace here?
Unfortunately, we don't actually have the backtrace available here.
This information is based on the SEV-SNP specification.
>> So, fix by calling set_memory_4k() on the allocated pages.
>
> I think I'm missing something here. set_memory_4k() will break the kernel
> linear mapping up into page granular mappings, but the IOMMU isn't using
> that mapping, right?
That's correct. This does not affect the IOMMU, but it affects the PSP FW.
> It's just using the physical address returned by iommu_virt_to_phys(), so why
does it matter?
>
> Just be nice to capture some of this rationale in the log, especially as
> I'm not familiar with this device.
According to the AMD SEV-SNP white paper
(https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf),
the Reverse Map Table (RMP) contains one entry for every 4K page of DRAM that may be used by the VM. In this case, the
pages allocated by the IOMMU driver are added as 4K entries in the RMP table by the SEV-SNP FW.
During the page table walk, the RMP checks if the page is owned by the hypervisor. Without calling set_memory_4k() to
break the mapping up into 4K pages, pages could end up being part of large mapping (e.g. 2M page), in which the page
access would be denied and result in #PF.
>> Fixes: commit c69d89aff393 ("iommu/amd: Use 4K page for completion wait
write-back semaphore")
>
> I couldn't figure out how that commit could cause this problem. Please can
> you explain that to me?
Hope this helps clarify. If so, I'll update the commit log and send out V3.
Thanks,
Suravee
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Patch V8 3/3] iommu: Document usage of "/sys/kernel/iommu_groups//type" file
Hi Will On 11/19/20 4:55 PM, Will Deacon wrote: Hi Lu, On Thu, Nov 19, 2020 at 10:32:43AM +0800, Lu Baolu wrote: On 11/18/20 9:51 PM, Will Deacon wrote: On Fri, Sep 25, 2020 at 12:06:20PM -0700, Ashok Raj wrote: I can't figure out from this description what string is returned to userspace in the case that the group is configured as blocked or unmanaged. This series only enables switching a default domain in use between DMA and IDENTITY. Other cases will result in write failures. I understood that from the text, but what I couldn't figure out is what happens if you *read* the file when the default domain is not identity or DMA. I think that should be documented. Yes, agreed. I will add this. Best regards, baolu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Patch V8 1/3] iommu: Add support to change default domain of an iommu group
Hi Will, On 11/19/20 4:53 PM, Will Deacon wrote: On Thu, Nov 19, 2020 at 10:18:05AM +0800, Lu Baolu wrote: The original author of this patch series has left Intel. I am now the backup. Ok, thanks for letting me know. On 11/18/20 9:51 PM, Will Deacon wrote: On Fri, Sep 25, 2020 at 12:06:18PM -0700, Ashok Raj wrote: From: Sai Praneeth Prakhya [...] +free_new_domain: + iommu_domain_free(group->default_domain); + group->default_domain = prev_dom; + group->domain = prev_dom;i Hmm. This seems to rely on all users of group->default_domain holding the group->mutex. Have you confirmed that this is the case? There's a funny use of iommu_group_get() in the exynos IOMMU driver at least. Emm. This change happens within the area with group->mutex held. Or I am not getting your point? Yeah, sorry, I wasn't very clear. This code holds the group->mutex, and it relies on _anybody_ else who wants to inspect group->default_domain also holding that mutex, otherwise they could observe a transient domain pointer which we free on the failure path here. Clear to me now. Thanks for explanation. :-) Changing default domain through sysfs requires the users to ubind any driver from the devices in the group. There's a check code and return failure if this requirement doesn't meet. So we only need to consider the device release path. device_lock(dev) is used in this patch to guarantee that no device release happens at the same time. My question is whether or not there is code that inspects group->default_domain without group->mutex held? The exynos case doesn't obviously hold it, and I'd like to make sure that there aren't others that we need to worry about. I searched the code. The exynos is the only case that inspects group->default_domain without holding the mutex during run time. It's in the device release path, so I think it's safe. Does that make more sense? Yes. Thanks! Thanks, Will Best regards, baolu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v6] swiotlb: Adjust SWIOTBL bounce buffer size for SEV guests.
From: Ashish Kalra
For SEV, all DMA to and from guest has to use shared (un-encrypted) pages.
SEV uses SWIOTLB to make this happen without requiring changes to device
drivers. However, depending on workload being run, the default 64MB of
SWIOTLB might not be enough and SWIOTLB may run out of buffers to use
for DMA, resulting in I/O errors and/or performance degradation for
high I/O workloads.
Increase the default size of SWIOTLB for SEV guests using a minimum
value of 128MB and a maximum value of 512MB, determining on amount
of provisioned guest memory.
Using late_initcall() interface to invoke swiotlb_adjust() does not
work as the size adjustment needs to be done before mem_encrypt_init()
and reserve_crashkernel() which use the allocated SWIOTLB buffer size,
hence calling it explicitly from setup_arch().
The SWIOTLB default size adjustment is added as an architecture specific
interface/callback to allow architectures such as those supporting memory
encryption to adjust/expand SWIOTLB size for their use.
v5 fixed build errors and warnings as
Reported-by: kbuild test robot
Signed-off-by: Ashish Kalra
---
arch/x86/kernel/setup.c | 2 ++
arch/x86/mm/mem_encrypt.c | 32
include/linux/swiotlb.h | 6 ++
kernel/dma/swiotlb.c | 24
4 files changed, 64 insertions(+)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 3511736fbc74..b073d58dd4a3 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1166,6 +1166,8 @@ void __init setup_arch(char **cmdline_p)
if (boot_cpu_has(X86_FEATURE_GBPAGES))
hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT);
+ swiotlb_adjust();
+
/*
* Reserve memory for crash kernel after SRAT is parsed so that it
* won't consume hotpluggable memory.
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 3f248f0d0e07..c79a0d761db5 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -490,6 +490,38 @@ static void print_mem_encrypt_feature_info(void)
}
/* Architecture __weak replacement functions */
+unsigned long __init arch_swiotlb_adjust(unsigned long iotlb_default_size)
+{
+ unsigned long size = 0;
+
+ /*
+* For SEV, all DMA has to occur via shared/unencrypted pages.
+* SEV uses SWOTLB to make this happen without changing device
+* drivers. However, depending on the workload being run, the
+* default 64MB of SWIOTLB may not be enough & SWIOTLB may
+* run out of buffers for DMA, resulting in I/O errors and/or
+* performance degradation especially with high I/O workloads.
+* Increase the default size of SWIOTLB for SEV guests using
+* a minimum value of 128MB and a maximum value of 512MB,
+* depending on amount of provisioned guest memory.
+*/
+ if (sev_active()) {
+ phys_addr_t total_mem = memblock_phys_mem_size();
+
+ if (total_mem <= SZ_1G)
+ size = max(iotlb_default_size, (unsigned long) SZ_128M);
+ else if (total_mem <= SZ_4G)
+ size = max(iotlb_default_size, (unsigned long) SZ_256M);
+ else
+ size = max(iotlb_default_size, (unsigned long) SZ_512M);
+
+ pr_info("SWIOTLB bounce buffer size adjusted to %luMB for SEV
platform",
+ size >> 20);
+ }
+
+ return size;
+}
+
void __init mem_encrypt_init(void)
{
if (!sme_me_mask)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 046bb94bd4d6..46a693f76f1e 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -33,6 +33,7 @@ extern void swiotlb_init(int verbose);
int swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose);
extern unsigned long swiotlb_nr_tbl(void);
unsigned long swiotlb_size_or_default(void);
+unsigned long __init arch_swiotlb_adjust(unsigned long size);
extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
extern void __init swiotlb_update_mem_attributes(void);
@@ -80,6 +81,7 @@ void __init swiotlb_exit(void);
unsigned int swiotlb_max_segment(void);
size_t swiotlb_max_mapping_size(struct device *dev);
bool is_swiotlb_active(void);
+void __init swiotlb_adjust(void);
#else
#define swiotlb_force SWIOTLB_NO_FORCE
static inline bool is_swiotlb_buffer(phys_addr_t paddr)
@@ -102,6 +104,10 @@ static inline bool is_swiotlb_active(void)
{
return false;
}
+
+static inline void swiotlb_adjust(void)
+{
+}
#endif /* CONFIG_SWIOTLB */
extern void swiotlb_print_info(void);
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index c19379fabd20..3be9a19ea0a5 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -163,6 +163,30 @@ unsigned long swiotlb_size_or_default(void)
return size ? size : (IO_TLB_DEFAULT_SIZE);
}
+unsigned long __init __weak
Re: [PATCH] Revert "firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module"
On Thu, Nov 19, 2020 at 9:41 AM Thierry Reding wrote:
>
> From: Thierry Reding
>
> Commit d0511b5496c0 ("firmware: QCOM_SCM: Allow qcom_scm driver to be
> loadable as a permenent module") causes the ARM SMMU driver to be built
> as a loadable module when using the Aarch64 default configuration. This
> in turn causes problems because if the loadable module is not shipped
> in an initial ramdisk, then the deferred probe timeout mechanism will
> cause all SMMU masters to probe without SMMU support and fall back to
> just plain DMA ops (not IOMMU-backed).
>
> Once the system has mounted the rootfs, the ARM SMMU driver will then
> be loaded, but since the ARM SMMU driver faults by default, this causes
> a slew of SMMU faults for the SMMU masters that have already been set
> up with plain DMA ops and cause these devices to malfunction.
>
> Revert that commit to unbreak things while we look for an alternative
> solution.
>
> Reported-by: Jon Hunter
> Signed-off-by: Thierry Reding
Acked-by: John Stultz
Thanks for sending this Thierry, and sorry again for the troubles.
thanks
-john
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] Revert "firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module"
On Thu, Nov 19, 2020 at 06:41:49PM +0100, Thierry Reding wrote:
> From: Thierry Reding
>
> Commit d0511b5496c0 ("firmware: QCOM_SCM: Allow qcom_scm driver to be
> loadable as a permenent module") causes the ARM SMMU driver to be built
> as a loadable module when using the Aarch64 default configuration. This
> in turn causes problems because if the loadable module is not shipped
> in an initial ramdisk, then the deferred probe timeout mechanism will
> cause all SMMU masters to probe without SMMU support and fall back to
> just plain DMA ops (not IOMMU-backed).
>
> Once the system has mounted the rootfs, the ARM SMMU driver will then
> be loaded, but since the ARM SMMU driver faults by default, this causes
> a slew of SMMU faults for the SMMU masters that have already been set
> up with plain DMA ops and cause these devices to malfunction.
>
> Revert that commit to unbreak things while we look for an alternative
> solution.
>
> Reported-by: Jon Hunter
> Signed-off-by: Thierry Reding
> ---
> drivers/firmware/Kconfig| 4 ++--
> drivers/firmware/Makefile | 3 +--
> drivers/firmware/qcom_scm.c | 4
> drivers/iommu/Kconfig | 2 --
> drivers/net/wireless/ath/ath10k/Kconfig | 1 -
> 5 files changed, 3 insertions(+), 11 deletions(-)
Acked-by: Will Deacon
John previously agreed to this here:
https://lore.kernel.org/r/calaqxlvdm923wrsb+dvxfacmetmeps7qeq+rw_jbcdmnmww...@mail.gmail.com
so might be worth putting in a Link: tag when it gets applied.
Will
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 1/7] arm64: mm: Move reserve_crashkernel() into mem_init()
Hi, (sorry for the late response) On 06/11/2020 18:46, Nicolas Saenz Julienne wrote: > On Thu, 2020-11-05 at 16:11 +, James Morse wrote:>> We also depend on > this when skipping the checksum code in purgatory, which can be >> exceedingly slow. > > This one I don't fully understand, so I'll lazily assume the prerequisite is > the same WRT how memory is mapped. :) The aim is its never normally mapped by the kernel. This is so that if we can't get rid of the secondary CPUs (e.g. they have IRQs masked), but they are busy scribbling all over memory, we have a rough guarantee that they aren't scribbling over the kdump kernel. We can skip the checksum in purgatory, as there is very little risk of the memory having been corrupted. > Ultimately there's also /sys/kernel/kexec_crash_size's handling. Same > prerequisite. Yeah, this lets you release PAGE_SIZEs back to the allocator, which means the marked-invalid page tables we have hidden there need to be PAGE_SIZE mappings. Thanks, James > Keeping in mind acpi_table_upgrade() and unflatten_device_tree() depend on > having the linear mappings available. I don't see any simple way of solving > this. Both moving the firmware description routines to use fixmap or > correcting > the linear mapping further down the line so as to include kdump's regions, > seem > excessive/impossible (feel free to correct me here). I'd be happy to hear > suggestions. Otherwise we're back to hard-coding the information as we > initially did. > > Let me stress that knowing the DMA constraints in the system before reserving > crashkernel's regions is necessary if we ever want it to work seamlessly on > all > platforms. Be it small stuff like the Raspberry Pi or huge servers with TB of > memory. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v7 7/7] mm: Remove examples from enum zone_type comment
We can't really list every setup in common code. On top of that they are
unlikely to stay true for long as things change in the arch trees
independently of this comment.
Suggested-by: Christoph Hellwig
Signed-off-by: Nicolas Saenz Julienne
Reviewed-by: Christoph Hellwig
---
include/linux/mmzone.h | 20
1 file changed, 20 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 8b074e2ba12c..15132adaa233 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -355,26 +355,6 @@ enum zone_type {
* DMA mask is assumed when ZONE_DMA32 is defined. Some 64-bit
* platforms may need both zones as they support peripherals with
* different DMA addressing limitations.
-*
-* Some examples:
-*
-* - i386 and x86_64 have a fixed 16M ZONE_DMA and ZONE_DMA32 for the
-*rest of the lower 4G.
-*
-* - arm only uses ZONE_DMA, the size, up to 4G, may vary depending on
-*the specific device.
-*
-* - arm64 has a fixed 1G ZONE_DMA and ZONE_DMA32 for the rest of the
-*lower 4G.
-*
-* - powerpc only uses ZONE_DMA, the size, up to 2G, may vary
-*depending on the specific device.
-*
-* - s390 uses ZONE_DMA fixed to the lower 2G.
-*
-* - ia64 and riscv only use ZONE_DMA32.
-*
-* - parisc uses neither.
*/
#ifdef CONFIG_ZONE_DMA
ZONE_DMA,
--
2.29.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v7 5/7] arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges
We recently introduced a 1 GB sized ZONE_DMA to cater for platforms
incorporating masters that can address less than 32 bits of DMA, in
particular the Raspberry Pi 4, which has 4 or 8 GB of DRAM, but has
peripherals that can only address up to 1 GB (and its PCIe host
bridge can only access the bottom 3 GB)
The DMA layer also needs to be able to allocate memory that is
guaranteed to meet those DMA constraints, for bounce buffering as well
as allocating the backing for consistent mappings. This is why the 1 GB
ZONE_DMA was introduced recently. Unfortunately, it turns out the having
a 1 GB ZONE_DMA as well as a ZONE_DMA32 causes problems with kdump, and
potentially in other places where allocations cannot cross zone
boundaries. Therefore, we should avoid having two separate DMA zones
when possible.
So, with the help of of_dma_get_max_cpu_address() get the topmost
physical address accessible to all DMA masters in system and use that
information to fine-tune ZONE_DMA's size. In the absence of addressing
limited masters ZONE_DMA will span the whole 32-bit address space,
otherwise, in the case of the Raspberry Pi 4 it'll only span the 30-bit
address space, and have ZONE_DMA32 cover the rest of the 32-bit address
space.
Signed-off-by: Nicolas Saenz Julienne
---
Changes since v4:
- Use fls64 as we're now using the max address (as opposed to the
limit)
Changes since v3:
- Simplify code for readability.
Changes since v2:
- Updated commit log by shamelessly copying Ard's ACPI commit log
arch/arm64/mm/init.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 0954ea736987..a96d3fbbd12c 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -42,8 +42,6 @@
#include
#include
-#define ARM64_ZONE_DMA_BITS30
-
/*
* We need to be able to catch inadvertent references to memstart_addr
* that occur (potentially in generic code) before arm64_memblock_init()
@@ -188,9 +186,11 @@ static phys_addr_t __init max_zone_phys(unsigned int
zone_bits)
static void __init zone_sizes_init(unsigned long min, unsigned long max)
{
unsigned long max_zone_pfns[MAX_NR_ZONES] = {0};
+ unsigned int __maybe_unused dt_zone_dma_bits;
#ifdef CONFIG_ZONE_DMA
- zone_dma_bits = ARM64_ZONE_DMA_BITS;
+ dt_zone_dma_bits = fls64(of_dma_get_max_cpu_address(NULL));
+ zone_dma_bits = min(32U, dt_zone_dma_bits);
arm64_dma_phys_limit = max_zone_phys(zone_dma_bits);
max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit);
#endif
--
2.29.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v7 4/7] of: unittest: Add test for of_dma_get_max_cpu_address()
Introduce a test for of_dma_get_max_cup_address(), it uses the same DT
data as the rest of dma-ranges unit tests.
Signed-off-by: Nicolas Saenz Julienne
Reviewed-by: Rob Herring
---
Changes since v5:
- Update address expected by test
Changes since v3:
- Remove HAS_DMA guards
drivers/of/unittest.c | 18 ++
1 file changed, 18 insertions(+)
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 06cc988faf78..98cc0163301b 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -869,6 +869,23 @@ static void __init of_unittest_changeset(void)
#endif
}
+static void __init of_unittest_dma_get_max_cpu_address(void)
+{
+ struct device_node *np;
+ phys_addr_t cpu_addr;
+
+ np = of_find_node_by_path("/testcase-data/address-tests");
+ if (!np) {
+ pr_err("missing testcase data\n");
+ return;
+ }
+
+ cpu_addr = of_dma_get_max_cpu_address(np);
+ unittest(cpu_addr == 0x4fff,
+"of_dma_get_max_cpu_address: wrong CPU addr %pad (expecting
%x)\n",
+_addr, 0x4fff);
+}
+
static void __init of_unittest_dma_ranges_one(const char *path,
u64 expect_dma_addr, u64 expect_paddr)
{
@@ -3266,6 +3283,7 @@ static int __init of_unittest(void)
of_unittest_changeset();
of_unittest_parse_interrupts();
of_unittest_parse_interrupts_extended();
+ of_unittest_dma_get_max_cpu_address();
of_unittest_parse_dma_ranges();
of_unittest_pci_dma_ranges();
of_unittest_match_node();
--
2.29.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v7 6/7] arm64: mm: Set ZONE_DMA size based on early IORT scan
From: Ard Biesheuvel
We recently introduced a 1 GB sized ZONE_DMA to cater for platforms
incorporating masters that can address less than 32 bits of DMA, in
particular the Raspberry Pi 4, which has 4 or 8 GB of DRAM, but has
peripherals that can only address up to 1 GB (and its PCIe host
bridge can only access the bottom 3 GB)
Instructing the DMA layer about these limitations is straight-forward,
even though we had to fix some issues regarding memory limits set in
the IORT for named components, and regarding the handling of ACPI _DMA
methods. However, the DMA layer also needs to be able to allocate
memory that is guaranteed to meet those DMA constraints, for bounce
buffering as well as allocating the backing for consistent mappings.
This is why the 1 GB ZONE_DMA was introduced recently. Unfortunately,
it turns out the having a 1 GB ZONE_DMA as well as a ZONE_DMA32 causes
problems with kdump, and potentially in other places where allocations
cannot cross zone boundaries. Therefore, we should avoid having two
separate DMA zones when possible.
So let's do an early scan of the IORT, and only create the ZONE_DMA
if we encounter any devices that need it. This puts the burden on
the firmware to describe such limitations in the IORT, which may be
redundant (and less precise) if _DMA methods are also being provided.
However, it should be noted that this situation is highly unusual for
arm64 ACPI machines. Also, the DMA subsystem still gives precedence to
the _DMA method if implemented, and so we will not lose the ability to
perform streaming DMA outside the ZONE_DMA if the _DMA method permits
it.
Cc: Jeremy Linton
Cc: Lorenzo Pieralisi
Cc: Nicolas Saenz Julienne
Cc: Rob Herring
Cc: Christoph Hellwig
Cc: Robin Murphy
Cc: Hanjun Guo
Cc: Sudeep Holla
Cc: Anshuman Khandual
Signed-off-by: Ard Biesheuvel
[nsaenz: unified implementation with DT's counterpart]
Signed-off-by: Nicolas Saenz Julienne
Tested-by: Jeremy Linton
Acked-by: Lorenzo Pieralisi
Acked-by: Hanjun Guo
---
Changes since v3:
- Use min_not_zero()
- Check revision
- Remove unnecessary #ifdef in zone_sizes_init()
arch/arm64/mm/init.c | 5 +++-
drivers/acpi/arm64/iort.c | 55 +++
include/linux/acpi_iort.h | 4 +++
3 files changed, 63 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index a96d3fbbd12c..99741ba63cb8 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -29,6 +29,7 @@
#include
#include
#include
+#include
#include
#include
@@ -186,11 +187,13 @@ static phys_addr_t __init max_zone_phys(unsigned int
zone_bits)
static void __init zone_sizes_init(unsigned long min, unsigned long max)
{
unsigned long max_zone_pfns[MAX_NR_ZONES] = {0};
+ unsigned int __maybe_unused acpi_zone_dma_bits;
unsigned int __maybe_unused dt_zone_dma_bits;
#ifdef CONFIG_ZONE_DMA
+ acpi_zone_dma_bits = fls64(acpi_iort_dma_get_max_cpu_address());
dt_zone_dma_bits = fls64(of_dma_get_max_cpu_address(NULL));
- zone_dma_bits = min(32U, dt_zone_dma_bits);
+ zone_dma_bits = min3(32U, dt_zone_dma_bits, acpi_zone_dma_bits);
arm64_dma_phys_limit = max_zone_phys(zone_dma_bits);
max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit);
#endif
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 9929ff50c0c0..1787406684aa 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1718,3 +1718,58 @@ void __init acpi_iort_init(void)
iort_init_platform_devices();
}
+
+#ifdef CONFIG_ZONE_DMA
+/*
+ * Extract the highest CPU physical address accessible to all DMA masters in
+ * the system. PHYS_ADDR_MAX is returned when no constrained device is found.
+ */
+phys_addr_t __init acpi_iort_dma_get_max_cpu_address(void)
+{
+ phys_addr_t limit = PHYS_ADDR_MAX;
+ struct acpi_iort_node *node, *end;
+ struct acpi_table_iort *iort;
+ acpi_status status;
+ int i;
+
+ if (acpi_disabled)
+ return limit;
+
+ status = acpi_get_table(ACPI_SIG_IORT, 0,
+ (struct acpi_table_header **));
+ if (ACPI_FAILURE(status))
+ return limit;
+
+ node = ACPI_ADD_PTR(struct acpi_iort_node, iort, iort->node_offset);
+ end = ACPI_ADD_PTR(struct acpi_iort_node, iort, iort->header.length);
+
+ for (i = 0; i < iort->node_count; i++) {
+ if (node >= end)
+ break;
+
+ switch (node->type) {
+ struct acpi_iort_named_component *ncomp;
+ struct acpi_iort_root_complex *rc;
+ phys_addr_t local_limit;
+
+ case ACPI_IORT_NODE_NAMED_COMPONENT:
+ ncomp = (struct acpi_iort_named_component
*)node->node_data;
+ local_limit = DMA_BIT_MASK(ncomp->memory_address_limit);
+ limit =
[PATCH v7 0/7] arm64: Default to 32-bit wide ZONE_DMA
Using two distinct DMA zones turned out to be problematic. Here's an attempt go back to a saner default. I tested this on both a RPi4 and QEMU. --- Changes since v6: - Update patch #1 so we reserve crashkernel before request_standard_resources() - Tested on top of Catalin's mem_init() patches. Changes since v5: - Unify ACPI/DT functions Changes since v4: - Fix of_dma_get_max_cpu_address() so it returns the last addressable addres, not the limit Changes since v3: - Drop patch adding define in dma-mapping - Address small review changes - Update Ard's patch - Add new patch removing examples from mmzone.h Changes since v2: - Introduce Ard's patch - Improve OF dma-ranges parsing function - Add unit test for OF function - Address small changes - Move crashkernel reservation later in boot process Changes since v1: - Parse dma-ranges instead of using machine compatible string Ard Biesheuvel (1): arm64: mm: Set ZONE_DMA size based on early IORT scan Nicolas Saenz Julienne (6): arm64: mm: Move reserve_crashkernel() into mem_init() arm64: mm: Move zone_dma_bits initialization into zone_sizes_init() of/address: Introduce of_dma_get_max_cpu_address() of: unittest: Add test for of_dma_get_max_cpu_address() arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges mm: Remove examples from enum zone_type comment arch/arm64/mm/init.c | 22 +--- drivers/acpi/arm64/iort.c | 55 +++ drivers/of/address.c | 42 ++ drivers/of/unittest.c | 18 + include/linux/acpi_iort.h | 4 +++ include/linux/mmzone.h| 20 -- include/linux/of.h| 7 + 7 files changed, 139 insertions(+), 29 deletions(-) -- 2.29.2 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v7 1/7] arm64: mm: Move reserve_crashkernel() into mem_init()
crashkernel might reserve memory located in ZONE_DMA. We plan to delay ZONE_DMA's initialization after unflattening the devicetree and ACPI's boot table initialization, so move it later in the boot process. Specifically into bootmem_init() since request_standard_resources() depends on it. Signed-off-by: Nicolas Saenz Julienne Tested-by: Jeremy Linton --- Changes since v6: - Move crashkernel reserve placement earlier, in bootmem_init() arch/arm64/mm/init.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 71d463544400..fafdf992fd32 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -389,8 +389,6 @@ void __init arm64_memblock_init(void) else arm64_dma32_phys_limit = PHYS_MASK + 1; - reserve_crashkernel(); - reserve_elfcorehdr(); high_memory = __va(memblock_end_of_DRAM() - 1) + 1; @@ -430,6 +428,12 @@ void __init bootmem_init(void) sparse_init(); zone_sizes_init(min, max); + /* +* request_standard_resources() depends on crashkernel's memory being +* reserved, so do it here. +*/ + reserve_crashkernel(); + memblock_dump_all(); } -- 2.29.2 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v7 2/7] arm64: mm: Move zone_dma_bits initialization into zone_sizes_init()
zone_dma_bits's initialization happens earlier that it's actually
needed, in arm64_memblock_init(). So move it into the more suitable
zone_sizes_init().
Signed-off-by: Nicolas Saenz Julienne
Tested-by: Jeremy Linton
---
arch/arm64/mm/init.c | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index fafdf992fd32..0954ea736987 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -190,6 +190,8 @@ static void __init zone_sizes_init(unsigned long min,
unsigned long max)
unsigned long max_zone_pfns[MAX_NR_ZONES] = {0};
#ifdef CONFIG_ZONE_DMA
+ zone_dma_bits = ARM64_ZONE_DMA_BITS;
+ arm64_dma_phys_limit = max_zone_phys(zone_dma_bits);
max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit);
#endif
#ifdef CONFIG_ZONE_DMA32
@@ -379,11 +381,6 @@ void __init arm64_memblock_init(void)
early_init_fdt_scan_reserved_mem();
- if (IS_ENABLED(CONFIG_ZONE_DMA)) {
- zone_dma_bits = ARM64_ZONE_DMA_BITS;
- arm64_dma_phys_limit = max_zone_phys(ARM64_ZONE_DMA_BITS);
- }
-
if (IS_ENABLED(CONFIG_ZONE_DMA32))
arm64_dma32_phys_limit = max_zone_phys(32);
else
--
2.29.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 1/7] arm64: mm: Move reserve_crashkernel() into mem_init()
On Thu, Nov 19, 2020 at 06:25:29PM +0100, Nicolas Saenz Julienne wrote:
> On Thu, 2020-11-19 at 17:10 +, Catalin Marinas wrote:
> > On Thu, Nov 19, 2020 at 03:09:58PM +0100, Nicolas Saenz Julienne wrote:
> > > On Fri, 2020-11-13 at 11:29 +, Catalin Marinas wrote:
> > > [...]
> > > > > > > Let me stress that knowing the DMA constraints in the system
> > > > > > > before reserving
> > > > > > > crashkernel's regions is necessary if we ever want it to work
> > > > > > > seamlessly on all
> > > > > > > platforms. Be it small stuff like the Raspberry Pi or huge
> > > > > > > servers with TB of
> > > > > > > memory.
> > > > > >
> > > > > > Indeed. So we have 3 options (so far):
> > > > > >
> > > > > > 1. Allow the crashkernel reservation to go into the linear map but
> > > > > > set
> > > > > >it to invalid once allocated.
> > > > > >
> > > > > > 2. Parse the flattened DT (not sure what we do with ACPI) before
> > > > > >creating the linear map. We may have to rely on some SoC ID here
> > > > > >instead of actual DMA ranges.
> > > > > >
> > > > > > 3. Assume the smallest ZONE_DMA possible on arm64 (1GB) for
> > > > > > crashkernel
> > > > > >reservations and not rely on arm64_dma_phys_limit in
> > > > > >reserve_crashkernel().
> > > > > >
> > > > > > I think (2) we tried hard to avoid. Option (3) brings us back to the
> > > > > > issues we had on large crashkernel reservations regressing on some
> > > > > > platforms (though it's been a while since, they mostly went quiet
> > > > > > ;)).
> > > > > > However, with Chen's crashkernel patches we end up with two
> > > > > > reservations, one in the low DMA zone and one higher, potentially
> > > > > > above
> > > > > > 4GB. Having a fixed 1GB limit wouldn't be any worse for crashkernel
> > > > > > reservations than what we have now.
> > > > > >
> > > > > > If (1) works, I'd go for it (James knows this part better than me),
> > > > > > otherwise we can go for (3).
> > > > >
> > > > > Overall, I'd prefer (1) as well, and I'd be happy to have a got at
> > > > > it. If not
> > > > > I'll append (3) in this series.
> > > >
> > > > I think for 1 we could also remove the additional KEXEC_CORE checks,
> > > > something like below, untested:
> > > >
> > > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> > > > index 3e5a6913acc8..27ab609c1c0c 100644
> > > > --- a/arch/arm64/mm/mmu.c
> > > > +++ b/arch/arm64/mm/mmu.c
> > > > @@ -477,7 +477,8 @@ static void __init map_mem(pgd_t *pgdp)
> > > > int flags = 0;
> > > > u64 i;
> > > >
> > > > - if (rodata_full || debug_pagealloc_enabled())
> > > > + if (rodata_full || debug_pagealloc_enabled() ||
> > > > + IS_ENABLED(CONFIG_KEXEC_CORE))
> > > > flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
> > > >
> > > > /*
> > > > @@ -487,11 +488,6 @@ static void __init map_mem(pgd_t *pgdp)
> > > > * the following for-loop
> > > > */
> > > > memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
> > > > -#ifdef CONFIG_KEXEC_CORE
> > > > - if (crashk_res.end)
> > > > - memblock_mark_nomap(crashk_res.start,
> > > > - resource_size(_res));
> > > > -#endif
> > > >
> > > > /* map all the memory banks */
> > > > for_each_mem_range(i, , ) {
> > > > @@ -518,21 +514,6 @@ static void __init map_mem(pgd_t *pgdp)
> > > > __map_memblock(pgdp, kernel_start, kernel_end,
> > > >PAGE_KERNEL, NO_CONT_MAPPINGS);
> > > > memblock_clear_nomap(kernel_start, kernel_end - kernel_start);
> > > > -
> > > > -#ifdef CONFIG_KEXEC_CORE
> > > > - /*
> > > > -* Use page-level mappings here so that we can shrink the region
> > > > -* in page granularity and put back unused memory to buddy
> > > > system
> > > > -* through /sys/kernel/kexec_crash_size interface.
> > > > -*/
> > > > - if (crashk_res.end) {
> > > > - __map_memblock(pgdp, crashk_res.start, crashk_res.end +
> > > > 1,
> > > > - PAGE_KERNEL,
> > > > - NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS);
> > > > - memblock_clear_nomap(crashk_res.start,
> > > > -resource_size(_res));
> > > > - }
> > > > -#endif
> > > > }
> > > >
> > > > void mark_rodata_ro(void)
> > >
> > > So as far as I'm concerned this is good enough for me. I took the time to
> > > properly test crashkernel on RPi4 using the series, this patch, and
> > > another
> > > small fix to properly update /proc/iomem.
> > >
> > > I'll send v7 soon, but before, James (or anyone for that matter) any
> > > obvious
> > > push-back to Catalin's solution?
> >
> > I talked to James earlier and he was suggesting that we check the
> > command line for any crashkernel reservations and only disable block
> > mappings in that case, see the
[PATCH] Revert "firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module"
From: Thierry Reding
Commit d0511b5496c0 ("firmware: QCOM_SCM: Allow qcom_scm driver to be
loadable as a permenent module") causes the ARM SMMU driver to be built
as a loadable module when using the Aarch64 default configuration. This
in turn causes problems because if the loadable module is not shipped
in an initial ramdisk, then the deferred probe timeout mechanism will
cause all SMMU masters to probe without SMMU support and fall back to
just plain DMA ops (not IOMMU-backed).
Once the system has mounted the rootfs, the ARM SMMU driver will then
be loaded, but since the ARM SMMU driver faults by default, this causes
a slew of SMMU faults for the SMMU masters that have already been set
up with plain DMA ops and cause these devices to malfunction.
Revert that commit to unbreak things while we look for an alternative
solution.
Reported-by: Jon Hunter
Signed-off-by: Thierry Reding
---
drivers/firmware/Kconfig| 4 ++--
drivers/firmware/Makefile | 3 +--
drivers/firmware/qcom_scm.c | 4
drivers/iommu/Kconfig | 2 --
drivers/net/wireless/ath/ath10k/Kconfig | 1 -
5 files changed, 3 insertions(+), 11 deletions(-)
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index 5e369928bc56..3315e3c21586 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -235,8 +235,8 @@ config INTEL_STRATIX10_RSU
Say Y here if you want Intel RSU support.
config QCOM_SCM
- tristate "Qcom SCM driver"
- depends on (ARM && HAVE_ARM_SMCCC) || ARM64
+ bool
+ depends on ARM || ARM64
select RESET_CONTROLLER
config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 523173cbff33..5e013b6a3692 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -17,8 +17,7 @@ obj-$(CONFIG_ISCSI_IBFT) += iscsi_ibft.o
obj-$(CONFIG_FIRMWARE_MEMMAP) += memmap.o
obj-$(CONFIG_RASPBERRYPI_FIRMWARE) += raspberrypi.o
obj-$(CONFIG_FW_CFG_SYSFS) += qemu_fw_cfg.o
-obj-$(CONFIG_QCOM_SCM) += qcom-scm.o
-qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
+obj-$(CONFIG_QCOM_SCM) += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
obj-$(CONFIG_TI_SCI_PROTOCOL) += ti_sci.o
obj-$(CONFIG_TRUSTED_FOUNDATIONS) += trusted_foundations.o
obj-$(CONFIG_TURRIS_MOX_RWTM) += turris-mox-rwtm.o
diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c
index 6f431b73e617..7be48c1bec96 100644
--- a/drivers/firmware/qcom_scm.c
+++ b/drivers/firmware/qcom_scm.c
@@ -1280,7 +1280,6 @@ static const struct of_device_id qcom_scm_dt_match[] = {
{ .compatible = "qcom,scm" },
{}
};
-MODULE_DEVICE_TABLE(of, qcom_scm_dt_match);
static struct platform_driver qcom_scm_driver = {
.driver = {
@@ -1296,6 +1295,3 @@ static int __init qcom_scm_init(void)
return platform_driver_register(_scm_driver);
}
subsys_initcall(qcom_scm_init);
-
-MODULE_DESCRIPTION("Qualcomm Technologies, Inc. SCM driver");
-MODULE_LICENSE("GPL v2");
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index c64d7a2b6513..04878caf6da4 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -248,7 +248,6 @@ config SPAPR_TCE_IOMMU
config ARM_SMMU
tristate "ARM Ltd. System MMU (SMMU) Support"
depends on ARM64 || ARM || (COMPILE_TEST && !GENERIC_ATOMIC64)
- depends on QCOM_SCM || !QCOM_SCM #if QCOM_SCM=m this can't be =y
select IOMMU_API
select IOMMU_IO_PGTABLE_LPAE
select ARM_DMA_USE_IOMMU if ARM
@@ -376,7 +375,6 @@ config QCOM_IOMMU
# Note: iommu drivers cannot (yet?) be built as modules
bool "Qualcomm IOMMU Support"
depends on ARCH_QCOM || (COMPILE_TEST && !GENERIC_ATOMIC64)
- depends on QCOM_SCM=y
select IOMMU_API
select IOMMU_IO_PGTABLE_LPAE
select ARM_DMA_USE_IOMMU
diff --git a/drivers/net/wireless/ath/ath10k/Kconfig
b/drivers/net/wireless/ath/ath10k/Kconfig
index 741289e385d5..40f91bc8514d 100644
--- a/drivers/net/wireless/ath/ath10k/Kconfig
+++ b/drivers/net/wireless/ath/ath10k/Kconfig
@@ -44,7 +44,6 @@ config ATH10K_SNOC
tristate "Qualcomm ath10k SNOC support"
depends on ATH10K
depends on ARCH_QCOM || COMPILE_TEST
- depends on QCOM_SCM || !QCOM_SCM #if QCOM_SCM=m this can't be =y
select QCOM_QMI_HELPERS
help
This module adds support for integrated WCN3990 chip connected
--
2.29.2
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 1/7] arm64: mm: Move reserve_crashkernel() into mem_init()
On Thu, 2020-11-19 at 17:10 +, Catalin Marinas wrote:
> On Thu, Nov 19, 2020 at 03:09:58PM +0100, Nicolas Saenz Julienne wrote:
> > On Fri, 2020-11-13 at 11:29 +, Catalin Marinas wrote:
> > [...]
> > > > > > Let me stress that knowing the DMA constraints in the system before
> > > > > > reserving
> > > > > > crashkernel's regions is necessary if we ever want it to work
> > > > > > seamlessly on all
> > > > > > platforms. Be it small stuff like the Raspberry Pi or huge servers
> > > > > > with TB of
> > > > > > memory.
> > > > >
> > > > > Indeed. So we have 3 options (so far):
> > > > >
> > > > > 1. Allow the crashkernel reservation to go into the linear map but set
> > > > >it to invalid once allocated.
> > > > >
> > > > > 2. Parse the flattened DT (not sure what we do with ACPI) before
> > > > >creating the linear map. We may have to rely on some SoC ID here
> > > > >instead of actual DMA ranges.
> > > > >
> > > > > 3. Assume the smallest ZONE_DMA possible on arm64 (1GB) for
> > > > > crashkernel
> > > > >reservations and not rely on arm64_dma_phys_limit in
> > > > >reserve_crashkernel().
> > > > >
> > > > > I think (2) we tried hard to avoid. Option (3) brings us back to the
> > > > > issues we had on large crashkernel reservations regressing on some
> > > > > platforms (though it's been a while since, they mostly went quiet ;)).
> > > > > However, with Chen's crashkernel patches we end up with two
> > > > > reservations, one in the low DMA zone and one higher, potentially
> > > > > above
> > > > > 4GB. Having a fixed 1GB limit wouldn't be any worse for crashkernel
> > > > > reservations than what we have now.
> > > > >
> > > > > If (1) works, I'd go for it (James knows this part better than me),
> > > > > otherwise we can go for (3).
> > > >
> > > > Overall, I'd prefer (1) as well, and I'd be happy to have a got at it.
> > > > If not
> > > > I'll append (3) in this series.
> > >
> > > I think for 1 we could also remove the additional KEXEC_CORE checks,
> > > something like below, untested:
> > >
> > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> > > index 3e5a6913acc8..27ab609c1c0c 100644
> > > --- a/arch/arm64/mm/mmu.c
> > > +++ b/arch/arm64/mm/mmu.c
> > > @@ -477,7 +477,8 @@ static void __init map_mem(pgd_t *pgdp)
> > > int flags = 0;
> > > u64 i;
> > >
> > > - if (rodata_full || debug_pagealloc_enabled())
> > > + if (rodata_full || debug_pagealloc_enabled() ||
> > > + IS_ENABLED(CONFIG_KEXEC_CORE))
> > > flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
> > >
> > > /*
> > > @@ -487,11 +488,6 @@ static void __init map_mem(pgd_t *pgdp)
> > >* the following for-loop
> > >*/
> > > memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
> > > -#ifdef CONFIG_KEXEC_CORE
> > > - if (crashk_res.end)
> > > - memblock_mark_nomap(crashk_res.start,
> > > - resource_size(_res));
> > > -#endif
> > >
> > > /* map all the memory banks */
> > > for_each_mem_range(i, , ) {
> > > @@ -518,21 +514,6 @@ static void __init map_mem(pgd_t *pgdp)
> > > __map_memblock(pgdp, kernel_start, kernel_end,
> > > PAGE_KERNEL, NO_CONT_MAPPINGS);
> > > memblock_clear_nomap(kernel_start, kernel_end - kernel_start);
> > > -
> > > -#ifdef CONFIG_KEXEC_CORE
> > > - /*
> > > - * Use page-level mappings here so that we can shrink the region
> > > - * in page granularity and put back unused memory to buddy system
> > > - * through /sys/kernel/kexec_crash_size interface.
> > > - */
> > > - if (crashk_res.end) {
> > > - __map_memblock(pgdp, crashk_res.start, crashk_res.end + 1,
> > > -PAGE_KERNEL,
> > > -NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS);
> > > - memblock_clear_nomap(crashk_res.start,
> > > - resource_size(_res));
> > > - }
> > > -#endif
> > > }
> > >
> > > void mark_rodata_ro(void)
> >
> > So as far as I'm concerned this is good enough for me. I took the time to
> > properly test crashkernel on RPi4 using the series, this patch, and another
> > small fix to properly update /proc/iomem.
> >
> > I'll send v7 soon, but before, James (or anyone for that matter) any obvious
> > push-back to Catalin's solution?
>
> I talked to James earlier and he was suggesting that we check the
> command line for any crashkernel reservations and only disable block
> mappings in that case, see the diff below on top of the one I already
> sent (still testing it).
That's even better :)
> If you don't have any other changes for v7, I'm happy to pick v6 up on
> top of the no-block-mapping fix.
Yes I've got a small change in patch #1, the crashkernel reservation has to be
performed before request_standart_resouces() is called, which is OK, since
we're all setup by then, I moved the crashkernel reservation at the end of
bootmem_init(). I attached the patch. If it's easier for you I'll send v7.
Regards,
Nicolas
Re: [PATCH v6 1/7] arm64: mm: Move reserve_crashkernel() into mem_init()
On Thu, Nov 19, 2020 at 05:10:49PM +, Catalin Marinas wrote:
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index ed71b1c305d7..acdec0c67d3b 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -469,6 +469,21 @@ void __init mark_linear_text_alias_ro(void)
> PAGE_KERNEL_RO);
> }
>
> +static bool crash_mem_map __initdata;
> +
> +static int __init enable_crash_mem_map(char *arg)
> +{
> + /*
> + * Proper parameter parsing is done by reserve_crashkernel(). We only
> + * need to know if the linear map has to avoid block mappings so that
> + * the crashkernel reservations can be unmapped later.
> + */
> + crash_mem_map = false;
It should be set to true.
--
Catalin
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 1/7] arm64: mm: Move reserve_crashkernel() into mem_init()
On Thu, Nov 19, 2020 at 03:09:58PM +0100, Nicolas Saenz Julienne wrote:
> On Fri, 2020-11-13 at 11:29 +, Catalin Marinas wrote:
> [...]
> > > > > Let me stress that knowing the DMA constraints in the system before
> > > > > reserving
> > > > > crashkernel's regions is necessary if we ever want it to work
> > > > > seamlessly on all
> > > > > platforms. Be it small stuff like the Raspberry Pi or huge servers
> > > > > with TB of
> > > > > memory.
> > > >
> > > > Indeed. So we have 3 options (so far):
> > > >
> > > > 1. Allow the crashkernel reservation to go into the linear map but set
> > > >it to invalid once allocated.
> > > >
> > > > 2. Parse the flattened DT (not sure what we do with ACPI) before
> > > >creating the linear map. We may have to rely on some SoC ID here
> > > >instead of actual DMA ranges.
> > > >
> > > > 3. Assume the smallest ZONE_DMA possible on arm64 (1GB) for crashkernel
> > > >reservations and not rely on arm64_dma_phys_limit in
> > > >reserve_crashkernel().
> > > >
> > > > I think (2) we tried hard to avoid. Option (3) brings us back to the
> > > > issues we had on large crashkernel reservations regressing on some
> > > > platforms (though it's been a while since, they mostly went quiet ;)).
> > > > However, with Chen's crashkernel patches we end up with two
> > > > reservations, one in the low DMA zone and one higher, potentially above
> > > > 4GB. Having a fixed 1GB limit wouldn't be any worse for crashkernel
> > > > reservations than what we have now.
> > > >
> > > > If (1) works, I'd go for it (James knows this part better than me),
> > > > otherwise we can go for (3).
> > >
> > > Overall, I'd prefer (1) as well, and I'd be happy to have a got at it. If
> > > not
> > > I'll append (3) in this series.
> >
> > I think for 1 we could also remove the additional KEXEC_CORE checks,
> > something like below, untested:
> >
> > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> > index 3e5a6913acc8..27ab609c1c0c 100644
> > --- a/arch/arm64/mm/mmu.c
> > +++ b/arch/arm64/mm/mmu.c
> > @@ -477,7 +477,8 @@ static void __init map_mem(pgd_t *pgdp)
> > int flags = 0;
> > u64 i;
> >
> > - if (rodata_full || debug_pagealloc_enabled())
> > + if (rodata_full || debug_pagealloc_enabled() ||
> > + IS_ENABLED(CONFIG_KEXEC_CORE))
> > flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
> >
> > /*
> > @@ -487,11 +488,6 @@ static void __init map_mem(pgd_t *pgdp)
> > * the following for-loop
> > */
> > memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
> > -#ifdef CONFIG_KEXEC_CORE
> > - if (crashk_res.end)
> > - memblock_mark_nomap(crashk_res.start,
> > - resource_size(_res));
> > -#endif
> >
> > /* map all the memory banks */
> > for_each_mem_range(i, , ) {
> > @@ -518,21 +514,6 @@ static void __init map_mem(pgd_t *pgdp)
> > __map_memblock(pgdp, kernel_start, kernel_end,
> >PAGE_KERNEL, NO_CONT_MAPPINGS);
> > memblock_clear_nomap(kernel_start, kernel_end - kernel_start);
> > -
> > -#ifdef CONFIG_KEXEC_CORE
> > - /*
> > -* Use page-level mappings here so that we can shrink the region
> > -* in page granularity and put back unused memory to buddy system
> > -* through /sys/kernel/kexec_crash_size interface.
> > -*/
> > - if (crashk_res.end) {
> > - __map_memblock(pgdp, crashk_res.start, crashk_res.end + 1,
> > - PAGE_KERNEL,
> > - NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS);
> > - memblock_clear_nomap(crashk_res.start,
> > -resource_size(_res));
> > - }
> > -#endif
> > }
> >
> > void mark_rodata_ro(void)
>
> So as far as I'm concerned this is good enough for me. I took the time to
> properly test crashkernel on RPi4 using the series, this patch, and another
> small fix to properly update /proc/iomem.
>
> I'll send v7 soon, but before, James (or anyone for that matter) any obvious
> push-back to Catalin's solution?
I talked to James earlier and he was suggesting that we check the
command line for any crashkernel reservations and only disable block
mappings in that case, see the diff below on top of the one I already
sent (still testing it).
If you don't have any other changes for v7, I'm happy to pick v6 up on
top of the no-block-mapping fix.
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index ed71b1c305d7..acdec0c67d3b 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -469,6 +469,21 @@ void __init mark_linear_text_alias_ro(void)
PAGE_KERNEL_RO);
}
+static bool crash_mem_map __initdata;
+
+static int __init enable_crash_mem_map(char *arg)
+{
+ /*
+* Proper parameter parsing is done by reserve_crashkernel(). We only
+* need to know if the linear map has to avoid block mappings so that
+* the crashkernel reservations can be
Re: [PATCH v13 01/15] iommu: Introduce attach/detach_pasid_table API
Hi Jacob,
On 11/18/20 5:19 PM, Jacob Pan wrote:
> Hi Eric,
>
> On Wed, 18 Nov 2020 12:21:37 +0100, Eric Auger
> wrote:
>
>> In virtualization use case, when a guest is assigned
>> a PCI host device, protected by a virtual IOMMU on the guest,
>> the physical IOMMU must be programmed to be consistent with
>> the guest mappings. If the physical IOMMU supports two
>> translation stages it makes sense to program guest mappings
>> onto the first stage/level (ARM/Intel terminology) while the host
>> owns the stage/level 2.
>>
>> In that case, it is mandated to trap on guest configuration
>> settings and pass those to the physical iommu driver.
>>
>> This patch adds a new API to the iommu subsystem that allows
>> to set/unset the pasid table information.
>>
>> A generic iommu_pasid_table_config struct is introduced in
>> a new iommu.h uapi header. This is going to be used by the VFIO
>> user API.
>>
>> Signed-off-by: Jean-Philippe Brucker
>> Signed-off-by: Liu, Yi L
>> Signed-off-by: Ashok Raj
>> Signed-off-by: Jacob Pan
>> Signed-off-by: Eric Auger
>>
>> ---
>>
>> v12 -> v13:
>> - Fix config check
>>
>> v11 -> v12:
>> - add argsz, name the union
>> ---
>> drivers/iommu/iommu.c | 68 ++
>> include/linux/iommu.h | 21
>> include/uapi/linux/iommu.h | 54 ++
>> 3 files changed, 143 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index b53446bb8c6b..978fe34378fb 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -2171,6 +2171,74 @@ int iommu_uapi_sva_unbind_gpasid(struct
>> iommu_domain *domain, struct device *dev }
>> EXPORT_SYMBOL_GPL(iommu_uapi_sva_unbind_gpasid);
>>
>> +int iommu_attach_pasid_table(struct iommu_domain *domain,
>> + struct iommu_pasid_table_config *cfg)
>> +{
>> +if (unlikely(!domain->ops->attach_pasid_table))
>> +return -ENODEV;
>> +
>> +return domain->ops->attach_pasid_table(domain, cfg);
>> +}
>> +
>> +int iommu_uapi_attach_pasid_table(struct iommu_domain *domain,
>> + void __user *uinfo)
>> +{
>> +struct iommu_pasid_table_config pasid_table_data = { 0 };
>> +u32 minsz;
>> +
>> +if (unlikely(!domain->ops->attach_pasid_table))
>> +return -ENODEV;
>> +
>> +/*
>> + * No new spaces can be added before the variable sized union,
>> the
>> + * minimum size is the offset to the union.
>> + */
>> +minsz = offsetof(struct iommu_pasid_table_config, vendor_data);
>> +
>> +/* Copy minsz from user to get flags and argsz */
>> +if (copy_from_user(_table_data, uinfo, minsz))
>> +return -EFAULT;
>> +
>> +/* Fields before the variable size union are mandatory */
>> +if (pasid_table_data.argsz < minsz)
>> +return -EINVAL;
>> +
>> +/* PASID and address granu require additional info beyond minsz
>> */
>> +if (pasid_table_data.version != PASID_TABLE_CFG_VERSION_1)
>> +return -EINVAL;
>> +if (pasid_table_data.format == IOMMU_PASID_FORMAT_SMMUV3 &&
>> +pasid_table_data.argsz <
>> +offsetofend(struct iommu_pasid_table_config,
>> vendor_data.smmuv3))
>> +return -EINVAL;
>> +
>> +/*
>> + * User might be using a newer UAPI header which has a larger
>> data
>> + * size, we shall support the existing flags within the current
>> + * size. Copy the remaining user data _after_ minsz but not more
>> + * than the current kernel supported size.
>> + */
>> +if (copy_from_user((void *)_table_data + minsz, uinfo +
>> minsz,
>> + min_t(u32, pasid_table_data.argsz,
>> sizeof(pasid_table_data)) - minsz))
>> +return -EFAULT;
>> +
>> +/* Now the argsz is validated, check the content */
>> +if (pasid_table_data.config < IOMMU_PASID_CONFIG_TRANSLATE ||
>> +pasid_table_data.config > IOMMU_PASID_CONFIG_ABORT)
>> +return -EINVAL;
>> +
>> +return domain->ops->attach_pasid_table(domain,
>> _table_data); +}
>> +EXPORT_SYMBOL_GPL(iommu_uapi_attach_pasid_table);
>> +
>> +void iommu_detach_pasid_table(struct iommu_domain *domain)
>> +{
>> +if (unlikely(!domain->ops->detach_pasid_table))
>> +return;
>> +
>> +domain->ops->detach_pasid_table(domain);
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_detach_pasid_table);
>> +
>> static void __iommu_detach_device(struct iommu_domain *domain,
>>struct device *dev)
>> {
>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
>> index b95a6f8db6ff..464fcbecf841 100644
>> --- a/include/linux/iommu.h
>> +++ b/include/linux/iommu.h
>> @@ -223,6 +223,8 @@ struct iommu_iotlb_gather {
>> * @cache_invalidate: invalidate translation caches
>> * @sva_bind_gpasid: bind guest pasid and mm
>> * @sva_unbind_gpasid: unbind guest pasid and mm
>> + * @attach_pasid_table: attach a pasid table
>> + *
[PATCH] iommu: Check return of __iommu_attach_device()
Currently iommu_create_device_direct_mappings() is called
without checking the return of __iommu_attach_device(). This
may result in failures in iommu driver if dev attach returns
error.
Fixes: ce574c27ae27("iommu: Move iommu_group_create_direct_mappings() out of
iommu_group_add_device()")
Signed-off-by: Shameer Kolothum
---
Crash log:
[ 31.353605] hns3 :7d:00.3: Adding to iommu group 10
[ 31.358822] Unable to handle kernel NULL pointer dereference at virtual
address 0018
[ 31.367567] Mem abort info:
[ 31.370350] ESR = 0x9604
[ 31.373391] EC = 0x25: DABT (current EL), IL = 32 bits
[ 31.378680] SET = 0, FnV = 0
[ 31.381720] EA = 0, S1PTW = 0
[ 31.384847] Data abort info:
[ 31.387716] ISV = 0, ISS = 0x0004
[ 31.391535] CM = 0, WnR = 0
[ 31.394491] [0018] user address but active_mm is swapper
[ 31.400818] Internal error: Oops: 9604 [#1] PREEMPT SMP
[ 31.406365] Modules linked in:
[ 31.409409] CPU: 21 PID: 1 Comm: swapper/0 Not tainted
5.10.0-rc4-8-gdd5aba9d719-dirty #79
[ 31.417980] Hardware name: Huawei TaiShan 200 (Model 2280)/BC82AMDD, BIOS
2280-V2 CS V3.B220.01 03/19/2020
[ 31.427588] pstate: 00c9 (nzcv daif +PAN +UAO -TCO BTYPE=--)
[ 31.433566] pc : arm_smmu_tlb_inv_range+0x178/0x1f0
[ 31.438422] lr : arm_smmu_tlb_inv_range+0x5c/0x1f0
[ 31.443190] sp : 80001043b4e0
...
[ 31.531175] Call trace:
[ 31.533613] arm_smmu_tlb_inv_range+0x178/0x1f0
[ 31.538122] arm_smmu_iotlb_sync+0x2c/0x38
[ 31.542200] iommu_unmap+0x60/0x90
[ 31.545585] __iommu_map+0x110/0x1f0
[ 31.549144] iommu_create_device_direct_mappings.isra.34+0x1ac/0x250
[ 31.555468] iommu_probe_device+0x6c/0x110
[ 31.559551] iort_iommu_configure_id+0x114/0x218
[ 31.564148] acpi_dma_configure_id+0x94/0xe0
[ 31.568402] pci_dma_configure+0xc8/0xf0
[ 31.572310] really_probe+0xd4/0x3e0
[ 31.575871] driver_probe_device+0x5c/0xc0
---
drivers/iommu/iommu.c | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index b53446bb8c6b..0f4dc25d46c9 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -264,16 +264,18 @@ int iommu_probe_device(struct device *dev)
*/
iommu_alloc_default_domain(group, dev);
- if (group->default_domain)
+ if (group->default_domain) {
ret = __iommu_attach_device(group->default_domain, dev);
+ if (ret) {
+ iommu_group_put(group);
+ goto err_release;
+ }
+ }
iommu_create_device_direct_mappings(group, dev);
iommu_group_put(group);
- if (ret)
- goto err_release;
-
if (ops->probe_finalize)
ops->probe_finalize(dev);
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 1/7] arm64: mm: Move reserve_crashkernel() into mem_init()
Hi Catalin, James,
sorry for the late reply but I got sidetracked.
On Fri, 2020-11-13 at 11:29 +, Catalin Marinas wrote:
[...]
> > > > Let me stress that knowing the DMA constraints in the system before
> > > > reserving
> > > > crashkernel's regions is necessary if we ever want it to work
> > > > seamlessly on all
> > > > platforms. Be it small stuff like the Raspberry Pi or huge servers with
> > > > TB of
> > > > memory.
> > >
> > > Indeed. So we have 3 options (so far):
> > >
> > > 1. Allow the crashkernel reservation to go into the linear map but set
> > >it to invalid once allocated.
> > >
> > > 2. Parse the flattened DT (not sure what we do with ACPI) before
> > >creating the linear map. We may have to rely on some SoC ID here
> > >instead of actual DMA ranges.
> > >
> > > 3. Assume the smallest ZONE_DMA possible on arm64 (1GB) for crashkernel
> > >reservations and not rely on arm64_dma_phys_limit in
> > >reserve_crashkernel().
> > >
> > > I think (2) we tried hard to avoid. Option (3) brings us back to the
> > > issues we had on large crashkernel reservations regressing on some
> > > platforms (though it's been a while since, they mostly went quiet ;)).
> > > However, with Chen's crashkernel patches we end up with two
> > > reservations, one in the low DMA zone and one higher, potentially above
> > > 4GB. Having a fixed 1GB limit wouldn't be any worse for crashkernel
> > > reservations than what we have now.
> > >
> > > If (1) works, I'd go for it (James knows this part better than me),
> > > otherwise we can go for (3).
> >
> > Overall, I'd prefer (1) as well, and I'd be happy to have a got at it. If
> > not
> > I'll append (3) in this series.
>
> I think for 1 we could also remove the additional KEXEC_CORE checks,
> something like below, untested:
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 3e5a6913acc8..27ab609c1c0c 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -477,7 +477,8 @@ static void __init map_mem(pgd_t *pgdp)
> int flags = 0;
> u64 i;
>
> - if (rodata_full || debug_pagealloc_enabled())
> + if (rodata_full || debug_pagealloc_enabled() ||
> + IS_ENABLED(CONFIG_KEXEC_CORE))
> flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
>
> /*
> @@ -487,11 +488,6 @@ static void __init map_mem(pgd_t *pgdp)
>* the following for-loop
>*/
> memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
> -#ifdef CONFIG_KEXEC_CORE
> - if (crashk_res.end)
> - memblock_mark_nomap(crashk_res.start,
> - resource_size(_res));
> -#endif
>
> /* map all the memory banks */
> for_each_mem_range(i, , ) {
> @@ -518,21 +514,6 @@ static void __init map_mem(pgd_t *pgdp)
> __map_memblock(pgdp, kernel_start, kernel_end,
> PAGE_KERNEL, NO_CONT_MAPPINGS);
> memblock_clear_nomap(kernel_start, kernel_end - kernel_start);
> -
> -#ifdef CONFIG_KEXEC_CORE
> - /*
> - * Use page-level mappings here so that we can shrink the region
> - * in page granularity and put back unused memory to buddy system
> - * through /sys/kernel/kexec_crash_size interface.
> - */
> - if (crashk_res.end) {
> - __map_memblock(pgdp, crashk_res.start, crashk_res.end + 1,
> -PAGE_KERNEL,
> -NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS);
> - memblock_clear_nomap(crashk_res.start,
> - resource_size(_res));
> - }
> -#endif
> }
>
> void mark_rodata_ro(void)
So as far as I'm concerned this is good enough for me. I took the time to
properly test crashkernel on RPi4 using the series, this patch, and another
small fix to properly update /proc/iomem.
I'll send v7 soon, but before, James (or anyone for that matter) any obvious
push-back to Catalin's solution?
Regards,
Nicolas
signature.asc
Description: This is a digitally signed message part
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v2 8/8] iommu/arm-smmu-v3: Reserve any RMR regions associated with a dev
Get RMR regions associated with a dev reserved so that there is
a unity mapping for them in SMMU.
Signed-off-by: Shameer Kolothum
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 38 +
1 file changed, 38 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 97df1df001c9..174a9bcfd627 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2492,6 +2492,43 @@ static int arm_smmu_of_xlate(struct device *dev, struct
of_phandle_args *args)
return iommu_fwspec_add_ids(dev, args->args, 1);
}
+static bool arm_smmu_dev_has_rmr(struct arm_smmu_master *master,
+struct iommu_rmr *e)
+{
+ int i, j;
+
+ for (i = 0; i < master->num_sids; i++) {
+ for (j = 0; j < e->num_ids; j++) {
+ if (e->ids[j] == master->sids[i])
+ return true;
+ }
+ }
+
+ return false;
+}
+
+static void arm_smmu_rmr_get_resv_regions(struct device *dev,
+ struct list_head *head)
+{
+ struct arm_smmu_master *master = dev_iommu_priv_get(dev);
+ struct arm_smmu_device *smmu = master->smmu;
+ struct iommu_rmr *rmr;
+
+ list_for_each_entry(rmr, >rmr_list, list) {
+ struct iommu_resv_region *region;
+ int prot = IOMMU_READ | IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+
+ if (!arm_smmu_dev_has_rmr(master, rmr))
+ continue;
+ region = iommu_alloc_resv_region(rmr->base_address,
+rmr->length, prot,
+IOMMU_RESV_DIRECT);
+ if (!region)
+ return;
+
+ list_add_tail(>list, head);
+ }
+}
static void arm_smmu_get_resv_regions(struct device *dev,
struct list_head *head)
{
@@ -2506,6 +2543,7 @@ static void arm_smmu_get_resv_regions(struct device *dev,
list_add_tail(>list, head);
iommu_dma_get_resv_regions(dev, head);
+ arm_smmu_rmr_get_resv_regions(dev, head);
}
static bool arm_smmu_dev_has_feature(struct device *dev,
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v2 6/8] iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()
By default, disable_bypass is set and any dev without an iommu domain
installs STE with CFG_ABORT during arm_smmu_init_bypass_stes(). Introduce
a "bypass" flag to arm_smmu_write_strtab_ent() so that we can force it to
install CFG_BYPASS STE for specific SIDs. This will be useful for RMR
related SIDs.
Signed-off-by: Shameer Kolothum
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 1953b317d814..5f366d5a9ebf 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1174,7 +1174,7 @@ static void arm_smmu_sync_ste_for_sid(struct
arm_smmu_device *smmu, u32 sid)
}
static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
- __le64 *dst)
+ __le64 *dst, bool bypass)
{
/*
* This is hideously complicated, but we only really care about
@@ -1245,7 +1245,7 @@ static void arm_smmu_write_strtab_ent(struct
arm_smmu_master *master, u32 sid,
/* Bypass/fault */
if (!smmu_domain || !(s1_cfg || s2_cfg)) {
- if (!smmu_domain && disable_bypass)
+ if (!smmu_domain && disable_bypass && !bypass)
val |= FIELD_PREP(STRTAB_STE_0_CFG,
STRTAB_STE_0_CFG_ABORT);
else
val |= FIELD_PREP(STRTAB_STE_0_CFG,
STRTAB_STE_0_CFG_BYPASS);
@@ -1317,7 +1317,7 @@ static void arm_smmu_init_bypass_stes(__le64 *strtab,
unsigned int nent)
unsigned int i;
for (i = 0; i < nent; ++i) {
- arm_smmu_write_strtab_ent(NULL, -1, strtab);
+ arm_smmu_write_strtab_ent(NULL, -1, strtab, false);
strtab += STRTAB_STE_DWORDS;
}
}
@@ -2038,7 +2038,7 @@ static void arm_smmu_install_ste_for_dev(struct
arm_smmu_master *master)
if (j < i)
continue;
- arm_smmu_write_strtab_ent(master, sid, step);
+ arm_smmu_write_strtab_ent(master, sid, step, false);
}
}
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v2 7/8] iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
Check if there is any RMR info associated with the devices behind
the SMMUv3 and if any, install bypass STEs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMUv3 during probe().
Signed-off-by: Shameer Kolothum
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 40 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++
2 files changed, 42 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 5f366d5a9ebf..97df1df001c9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3486,6 +3486,42 @@ static void __iomem *arm_smmu_ioremap(struct device
*dev, resource_size_t start,
return devm_ioremap_resource(dev, );
}
+static void arm_smmu_rmr_install_bypass_ste(struct arm_smmu_device *smmu)
+{
+ struct iommu_rmr *e;
+ int i, ret;
+
+ /*
+* Since, we don't have a mechanism to differentiate the RMR
+* SIDs that has an ongoing live stream, install bypass STEs
+* for all the reported ones.
+* FixMe: Avoid duplicate SIDs in the list as one sid may
+*associate with multiple RMRs.
+*/
+ list_for_each_entry(e, >rmr_list, list) {
+ for (i = 0; i < e->num_ids; i++) {
+ __le64 *step;
+ u32 sid = e->ids[i];
+
+ ret = arm_smmu_init_sid_strtab(smmu, sid);
+ if (ret) {
+ dev_err(smmu->dev, "RMR bypass(0x%x) failed\n",
+ sid);
+ continue;
+ }
+
+ step = arm_smmu_get_step_for_sid(smmu, sid);
+ arm_smmu_write_strtab_ent(NULL, sid, step, true);
+ }
+ }
+}
+
+static int arm_smmu_get_rmr(struct arm_smmu_device *smmu)
+{
+ INIT_LIST_HEAD(>rmr_list);
+ return iommu_dma_get_rmrs(dev_fwnode(smmu->dev), >rmr_list);
+}
+
static int arm_smmu_device_probe(struct platform_device *pdev)
{
int irq, ret;
@@ -3569,6 +3605,10 @@ static int arm_smmu_device_probe(struct platform_device
*pdev)
/* Record our private device structure */
platform_set_drvdata(pdev, smmu);
+ /* Check for RMRs and install bypass STEs if any */
+ if (!arm_smmu_get_rmr(smmu))
+ arm_smmu_rmr_install_bypass_ste(smmu);
+
/* Reset the device */
ret = arm_smmu_device_reset(smmu, bypass);
if (ret)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index d4b7f40ccb02..17b517ddecee 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -636,6 +636,8 @@ struct arm_smmu_device {
/* IOMMU core code handle */
struct iommu_device iommu;
+
+ struct list_headrmr_list;
};
/* SMMU private data for each master */
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v2 5/8] iommu/arm-smmu-v3: Introduce strtab init helper
Introduce a helper to check the sid range and to init the l2 strtab
entries(bypass). This will be useful when we have to initialize the
l2 strtab for RMR SIDs.
Signed-off-by: Shameer Kolothum
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 26 -
1 file changed, 15 insertions(+), 11 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index e634bbe60573..1953b317d814 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2308,6 +2308,19 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device
*smmu, u32 sid)
static struct iommu_ops arm_smmu_ops;
+static int arm_smmu_init_sid_strtab(struct arm_smmu_device *smmu, u32 sid)
+{
+ /* Check the SIDs are in range of the SMMU and our stream table */
+ if (!arm_smmu_sid_in_range(smmu, sid))
+ return -ERANGE;
+
+ /* Ensure l2 strtab is initialised */
+ if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB)
+ return arm_smmu_init_l2_strtab(smmu, sid);
+
+ return 0;
+}
+
static struct iommu_device *arm_smmu_probe_device(struct device *dev)
{
int i, ret;
@@ -2336,21 +2349,12 @@ static struct iommu_device
*arm_smmu_probe_device(struct device *dev)
INIT_LIST_HEAD(>bonds);
dev_iommu_priv_set(dev, master);
- /* Check the SIDs are in range of the SMMU and our stream table */
for (i = 0; i < master->num_sids; i++) {
u32 sid = master->sids[i];
- if (!arm_smmu_sid_in_range(smmu, sid)) {
- ret = -ERANGE;
+ ret = arm_smmu_init_sid_strtab(smmu, sid);
+ if (ret)
goto err_free_master;
- }
-
- /* Ensure l2 strtab is initialised */
- if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
- ret = arm_smmu_init_l2_strtab(smmu, sid);
- if (ret)
- goto err_free_master;
- }
}
master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits);
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v2 3/8] iommu/dma: Introduce generic helper to retrieve RMR info
Reserved Memory Regions(RMR) associated with an IOMMU may be
described either through ACPI tables or DT in systems with
devices that require a unity mapping or bypass for those
regions in IOMMU drivers.
Introduce a generic interface so that IOMMU drivers can retrieve
and set up necessary mappings.
Signed-off-by: Shameer Kolothum
---
drivers/iommu/dma-iommu.c | 36
include/linux/dma-iommu.h | 7 +++
include/linux/iommu.h | 16
3 files changed, 59 insertions(+)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 0cbcd3fc3e7e..d73768ecdd1a 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -166,6 +166,42 @@ void iommu_dma_get_resv_regions(struct device *dev, struct
list_head *list)
}
EXPORT_SYMBOL(iommu_dma_get_resv_regions);
+/**
+ * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
+ * with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list to be populated
+ *
+ */
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
+ struct list_head *list)
+{
+ return 0;
+}
+EXPORT_SYMBOL(iommu_dma_get_rmrs);
+
+struct iommu_rmr *iommu_dma_alloc_rmr(u64 base, u64 length,
+ u32 *ids, int num_ids)
+{
+ struct iommu_rmr *rmr;
+ int i;
+
+ rmr = kzalloc(struct_size(rmr, ids, num_ids), GFP_KERNEL);
+ if (!rmr)
+ return NULL;
+
+ INIT_LIST_HEAD(>list);
+ rmr->base_address = base;
+ rmr->length = length;
+ rmr->num_ids = num_ids;
+
+ for (i = 0; i < num_ids; i++)
+ rmr->ids[i] = ids[i];
+
+ return rmr;
+}
+EXPORT_SYMBOL(iommu_dma_alloc_rmr);
+
static int cookie_init_hw_msi_region(struct iommu_dma_cookie *cookie,
phys_addr_t start, phys_addr_t end)
{
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index 2112f21f73d8..8900ccbc9e6a 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -37,6 +37,9 @@ void iommu_dma_compose_msi_msg(struct msi_desc *desc,
void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list);
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+struct iommu_rmr *iommu_dma_alloc_rmr(u64 base, u64 length,
+ u32 *ids, int num_ids);
#else /* CONFIG_IOMMU_DMA */
struct iommu_domain;
@@ -78,5 +81,9 @@ static inline void iommu_dma_get_resv_regions(struct device
*dev, struct list_he
{
}
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+{
+ return 0;
+}
#endif /* CONFIG_IOMMU_DMA */
#endif /* __DMA_IOMMU_H */
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index b95a6f8db6ff..e43c4e8084e7 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -592,6 +592,22 @@ struct iommu_sva {
struct device *dev;
};
+/**
+ * struct iommu_rmr - Reserved Memory Region details per IOMMU
+ * @list: Linked list pointers to hold RMR region info
+ * @base_address: base address of Reserved Memory Region
+ * @length: length of memory region
+ * @num_ids: number of associated device IDs
+ * @ids: associated device IDs
+ */
+struct iommu_rmr {
+ struct list_headlist;
+ phys_addr_t base_address;
+ u64 length;
+ unsigned intnum_ids;
+ u32 ids[];
+};
+
int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode,
const struct iommu_ops *ops);
void iommu_fwspec_free(struct device *dev);
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v2 2/8] ACPI/IORT: Add support for RMR node parsing
Add support for parsing RMR node information from ACPI.
Find associated stream ids and smmu node info from the
RMR node and populate a linked list with RMR memory
descriptors.
Signed-off-by: Shameer Kolothum
---
drivers/acpi/arm64/iort.c | 122 +-
1 file changed, 121 insertions(+), 1 deletion(-)
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 9929ff50c0c0..a9705aa35028 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -40,6 +40,25 @@ struct iort_fwnode {
static LIST_HEAD(iort_fwnode_list);
static DEFINE_SPINLOCK(iort_fwnode_lock);
+struct iort_rmr_id {
+ u32 sid;
+ struct acpi_iort_node *smmu;
+};
+
+/*
+ * One entry for IORT RMR.
+ */
+struct iort_rmr_entry {
+ struct list_head list;
+
+ unsigned int rmr_ids_num;
+ struct iort_rmr_id *rmr_ids;
+
+ struct acpi_iort_rmr_desc *rmr_desc;
+};
+
+static LIST_HEAD(iort_rmr_list); /* list of RMR regions from ACPI */
+
/**
* iort_set_fwnode() - Create iort_fwnode and use it to register
*iommu data in the iort_fwnode_list
@@ -393,7 +412,8 @@ static struct acpi_iort_node *iort_node_get_id(struct
acpi_iort_node *node,
if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
node->type == ACPI_IORT_NODE_SMMU_V3 ||
- node->type == ACPI_IORT_NODE_PMCG) {
+ node->type == ACPI_IORT_NODE_PMCG ||
+ node->type == ACPI_IORT_NODE_RMR) {
*id_out = map->output_base;
return parent;
}
@@ -1647,6 +1667,103 @@ static void __init iort_enable_acs(struct
acpi_iort_node *iort_node)
#else
static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
#endif
+static int iort_rmr_desc_valid(struct acpi_iort_rmr_desc *desc)
+{
+ struct iort_rmr_entry *e;
+ u64 end, start = desc->base_address, length = desc->length;
+
+ if (!IS_ALIGNED(start, SZ_64K) || !IS_ALIGNED(length, SZ_64K))
+ return -EINVAL;
+
+ end = start + length - 1;
+
+ /* Check for address overlap */
+ list_for_each_entry(e, _rmr_list, list) {
+ u64 e_start = e->rmr_desc->base_address;
+ u64 e_end = e_start + e->rmr_desc->length - 1;
+
+ if (start <= e_end && end >= e_start)
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int __init iort_parse_rmr(struct acpi_iort_node *iort_node)
+{
+ struct iort_rmr_id *rmr_ids, *ids;
+ struct iort_rmr_entry *e;
+ struct acpi_iort_rmr *rmr;
+ struct acpi_iort_rmr_desc *rmr_desc;
+ u32 map_count = iort_node->mapping_count;
+ int i, ret = 0, desc_count = 0;
+
+ if (iort_node->type != ACPI_IORT_NODE_RMR)
+ return 0;
+
+ if (!iort_node->mapping_offset || !map_count) {
+ pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
+ iort_node);
+ return -EINVAL;
+ }
+
+ rmr_ids = kmalloc(sizeof(*rmr_ids) * map_count, GFP_KERNEL);
+ if (!rmr_ids)
+ return -ENOMEM;
+
+ /* Retrieve associated smmu and stream id */
+ ids = rmr_ids;
+ for (i = 0; i < map_count; i++, ids++) {
+ ids->smmu = iort_node_get_id(iort_node, >sid, i);
+ if (!ids->smmu) {
+ pr_err(FW_BUG "Invalid SMMU reference, skipping RMR
node %p\n",
+ iort_node);
+ ret = -EINVAL;
+ goto out;
+ }
+ }
+
+ /* Retrieve RMR data */
+ rmr = (struct acpi_iort_rmr *)iort_node->node_data;
+ if (!rmr->rmr_offset || !rmr->rmr_count) {
+ pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node
%p\n",
+ iort_node);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
+ rmr->rmr_offset);
+
+ for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
+ ret = iort_rmr_desc_valid(rmr_desc);
+ if (ret) {
+ pr_err(FW_BUG "Invalid RMR descriptor[%d] for node %p,
skipping...\n",
+ i, iort_node);
+ goto out;
+ }
+
+ e = kmalloc(sizeof(*e), GFP_KERNEL);
+ if (!e) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ e->rmr_ids_num = map_count;
+ e->rmr_ids = rmr_ids;
+ e->rmr_desc = rmr_desc;
+
+ list_add_tail(>list, _rmr_list);
+ desc_count++;
+ }
+
+ return 0;
+
+out:
+ if (!desc_count)
+
[RFC PATCH v2 4/8] ACPI/IORT: Add RMR memory regions reservation helper
Add a helper function that retrieves RMR memory descriptors
associated with a given IOMMU. This will be used by IOMMU
drivers to setup necessary mappings.
Now that we have this, invoke this from the generic helper
interface.
Signed-off-by: Shameer Kolothum
---
drivers/acpi/arm64/iort.c | 60 +++
drivers/iommu/dma-iommu.c | 3 ++
include/linux/acpi_iort.h | 6
3 files changed, 69 insertions(+)
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index a9705aa35028..d1a2b09230ab 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -12,6 +12,7 @@
#include
#include
+#include
#include
#include
#include
@@ -842,6 +843,63 @@ static inline int iort_add_device_replay(struct device
*dev)
return err;
}
+/**
+ * iort_iommu_get_rmrs - Helper to retrieve RMR info associated with IOMMU
+ * @iommu: fwnode for the IOMMU
+ * @head: RMR list head to be populated
+ *
+ * Returns: 0 on success, <0 failure
+ */
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+ struct list_head *head)
+{
+ struct iort_rmr_entry *e;
+ struct acpi_iort_node *iommu;
+
+ iommu = iort_get_iort_node(iommu_fwnode);
+ if (!iommu)
+ return 0;
+
+ list_for_each_entry(e, _rmr_list, list) {
+ struct iort_rmr_id *rmr_ids = e->rmr_ids;
+ struct acpi_iort_rmr_desc *rmr_desc;
+ struct iommu_rmr *rmr;
+ u32 *ids, num_ids = 0;
+ int i, j = 0;
+
+ for (i = 0; i < e->rmr_ids_num; i++) {
+ if (rmr_ids[i].smmu == iommu)
+ num_ids++;
+ }
+
+ if (!num_ids)
+ continue;
+
+ ids = kmalloc_array(num_ids, sizeof(*ids), GFP_KERNEL);
+ if (!ids)
+ return -ENOMEM;
+
+ for (i = 0; i < e->rmr_ids_num; i++) {
+ if (rmr_ids[i].smmu == iommu)
+ ids[j++] = rmr_ids[i].sid;
+ }
+
+ rmr_desc = e->rmr_desc;
+ rmr = iommu_dma_alloc_rmr(rmr_desc->base_address,
+ rmr_desc->length,
+ ids, num_ids);
+ if (!rmr) {
+ kfree(ids);
+ return -ENOMEM;
+ }
+
+ list_add_tail(>list, head);
+ kfree(ids);
+ }
+
+ return 0;
+}
+
/**
* iort_iommu_msi_get_resv_regions - Reserved region driver helper
* @dev: Device from iommu_get_resv_regions()
@@ -1112,6 +1170,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev,
struct list_head *head)
const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
const u32 *input_id)
{ return NULL; }
+int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
+{ return 0; }
#endif
static int nc_dma_get_range(struct device *dev, u64 *size)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index d73768ecdd1a..aa8304e50786 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -176,6 +176,9 @@ EXPORT_SYMBOL(iommu_dma_get_resv_regions);
int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
struct list_head *list)
{
+ if (!is_of_node(iommu_fwnode))
+ return iort_iommu_get_rmrs(iommu_fwnode, list);
+
return 0;
}
EXPORT_SYMBOL(iommu_dma_get_rmrs);
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index 20a32120bb88..0b61b98a4941 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -38,6 +38,8 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64
*size);
const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
const u32 *id_in);
int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head
*head);
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+ struct list_head *list);
#else
static inline void acpi_iort_init(void) { }
static inline u32 iort_msi_map_id(struct device *dev, u32 id)
@@ -55,6 +57,10 @@ static inline const struct iommu_ops
*iort_iommu_configure_id(
static inline
int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
{ return 0; }
+static inline
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+ struct list_head *list)
+{ return 0; }
#endif
#endif /* __ACPI_IORT_H__ */
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
RFC v1 --> v2: - Added a generic interface for IOMMU drivers to retrieve all the RMR info associated with a given IOMMU. - SMMUv3 driver gets the RMR list during probe() and installs bypass STEs for all the SIDs in the RMR list. This is to keep the ongoing traffic alive(if any) during SMMUv3 reset. This is based on the suggestions received for v1 to take care of the EFI framebuffer use case. Only sanity tested for now. - During the probe/attach device, SMMUv3 driver reserves any RMR region associated with the device such that there is a unity mapping for them in SMMU. --- The series adds support to IORT RMR nodes specified in IORT Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory ranges that are used by endpoints and require a unity mapping in SMMU. We have faced issues with 3408iMR RAID controller cards which fail to boot when SMMU is enabled. This is because these controllers make use of host memory for various caching related purposes and when SMMU is enabled the iMR firmware fails to access these memory regions as there is no mapping for them. IORT RMR provides a way for UEFI to describe and report these memory regions so that the kernel can make a unity mapping for these in SMMU. RFC because, Patch #1 is to update the actbl2.h and should be done through acpica update. I have send out a pull request[1] for that. Tests: With a UEFI, that reports the RMR for the dev, [16F0h 5872 1] Type : 06 [16F1h 5873 2] Length : 007C [16F3h 5875 1] Revision : 00 [1038h 0056 2] Reserved : [1038h 0056 2] Identifier : [16F8h 5880 4]Mapping Count : 0001 [16FCh 5884 4] Mapping Offset : 0040 [1700h 5888 4]Number of RMR Descriptors : 0002 [1704h 5892 4]RMR Descriptor Offset : 0018 [1708h 5896 8] Base Address of RMR : E640 [1710h 5904 8]Length of RMR : 0010 [1718h 5912 4] Reserved : [171Ch 5916 8] Base Address of RMR : 27B0 [1724h 5924 8]Length of RMR : 00C0 [172Ch 5932 4] Reserved : [1730h 5936 4] Input base : [1734h 5940 4] ID Count : 0001 [1738h 5944 4] Output Base : 0003 [173Ch 5948 4] Output Reference : 0064 [1740h 5952 4]Flags (decoded below) : 0001 Single Mapping : 1 ... Without the series the RAID controller initialization fails as below, ... [ 12.631117] megaraid_sas :03:00.0: FW supports sync cache: Yes [ 12.637360] megaraid_sas :03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x4009 [ 18.776377] megaraid_sas :03:00.0: Init cmd return status FAILED for SCSI host 0 [ 23.019383] megaraid_sas :03:00.0: Waiting for FW to come to ready state [ 106.684281] megaraid_sas :03:00.0: FW in FAULT state, Fault code:0x1 subcode:0x0 func:megasas_transition_to_ready [ 106.695186] megaraid_sas :03:00.0: System Register set: [ 106.889787] megaraid_sas :03:00.0: Failed to transition controller to ready for scsi0. [ 106.910475] megaraid_sas :03:00.0: Failed from megasas_init_fw 6407 estuary:/$ With the series, now the kernel has direct mapping for the dev as below, estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions 0x0800 0x080f msi 0x27b0 0x286f direct 0xe640 0xe64f direct estuary:/$ [ 12.254318] megaraid_sas :03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x4009 [ 12.739089] megaraid_sas :03:00.0: FW provided supportMaxExtLDs: 0 max_lds: 32 [ 12.746628] megaraid_sas :03:00.0: controller type : iMR(0MB) [ 12.752694] megaraid_sas :03:00.0: Online Controller Reset(OCR) : Enabled [ 12.759798] megaraid_sas :03:00.0: Secure JBOD support : Yes [ 12.765778] megaraid_sas :03:00.0: NVMe passthru support : Yes [ 12.771931] megaraid_sas :03:00.0: FW provided TM TaskAbort/Reset timeou: 6 secs/60 secs
[RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
IORT revision E contains a few additions like,
-Added an identifier field in the node descriptors to aid table
cross-referencing.
-Introduced the Reserved Memory Range(RMR) node. This is used
to describe memory ranges that are used by endpoints and requires
a unity mapping in SMMU.
-Introduced a flag in the RC node to express support for PRI.
Signed-off-by: Shameer Kolothum
---
include/acpi/actbl2.h | 25 +++--
1 file changed, 19 insertions(+), 6 deletions(-)
diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h
index ec66779cb193..274fce7b5c01 100644
--- a/include/acpi/actbl2.h
+++ b/include/acpi/actbl2.h
@@ -68,7 +68,7 @@
* IORT - IO Remapping Table
*
* Conforms to "IO Remapping Table System Software on ARM Platforms",
- * Document number: ARM DEN 0049D, March 2018
+ * Document number: ARM DEN 0049E, June 2020
*
**/
@@ -86,7 +86,8 @@ struct acpi_iort_node {
u8 type;
u16 length;
u8 revision;
- u32 reserved;
+ u16 reserved;
+ u16 identifier;
u32 mapping_count;
u32 mapping_offset;
char node_data[1];
@@ -100,7 +101,8 @@ enum acpi_iort_node_type {
ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
ACPI_IORT_NODE_SMMU = 0x03,
ACPI_IORT_NODE_SMMU_V3 = 0x04,
- ACPI_IORT_NODE_PMCG = 0x05
+ ACPI_IORT_NODE_PMCG = 0x05,
+ ACPI_IORT_NODE_RMR = 0x06,
};
struct acpi_iort_id_mapping {
@@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
u8 reserved[3]; /* Reserved, must be zero */
};
-/* Values for ats_attribute field above */
+/* Masks for ats_attribute field above */
-#define ACPI_IORT_ATS_SUPPORTED 0x0001 /* The root complex
supports ATS */
-#define ACPI_IORT_ATS_UNSUPPORTED 0x /* The root complex
doesn't support ATS */
+#define ACPI_IORT_ATS_SUPPORTED (1)/* The root complex supports
ATS */
+#define ACPI_IORT_PRI_SUPPORTED (1<<1) /* The root complex supports
PRI */
struct acpi_iort_smmu {
u64 base_address; /* SMMU base address */
@@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
u64 page1_base_address;
};
+struct acpi_iort_rmr {
+ u32 rmr_count;
+ u32 rmr_offset;
+};
+
+struct acpi_iort_rmr_desc {
+ u64 base_address;
+ u64 length;
+ u32 reserved;
+};
+
/***
*
* IVRS - I/O Virtualization Reporting Structure
--
2.17.1
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [Devel] Re: [RFC PATCH 2/4] ACPI/IORT: Add support for RMR node parsing
> -Original Message-
> From: Sami Mujawar [mailto:sami.muja...@arm.com]
> Sent: 09 November 2020 12:30
> To: david.e@linux.intel.com; Shameerali Kolothum Thodi
> ;
> linux-arm-ker...@lists.infradead.org; linux-a...@vger.kernel.org;
> iommu@lists.linux-foundation.org; de...@acpica.org
> Cc: Linuxarm ; Lorenzo Pieralisi
> ; j...@8bytes.org; Robin Murphy
> ; wanghuiqiang ;
> Jonathan Cameron ; nd
> Subject: RE: [Devel] Re: [RFC PATCH 2/4] ACPI/IORT: Add support for RMR node
> parsing
>
> Hi,
>
> -Original Message-
> From: David E. Box
> Sent: 28 October 2020 06:44 PM
> To: Shameer Kolothum ;
> linux-arm-ker...@lists.infradead.org; linux-a...@vger.kernel.org;
> iommu@lists.linux-foundation.org; de...@acpica.org
> Cc: linux...@huawei.com; Lorenzo Pieralisi ;
> j...@8bytes.org; Robin Murphy ;
> wanghuiqi...@huawei.com; jonathan.came...@huawei.com
> Subject: [Devel] Re: [RFC PATCH 2/4] ACPI/IORT: Add support for RMR node
> parsing
>
> Hi,
>
> On Tue, 2020-10-27 at 11:26 +, Shameer Kolothum wrote:
>
> ...
>
> > @@ -1647,6 +1667,100 @@ static void __init iort_enable_acs(struct
> > acpi_iort_node *iort_node)
> > #else
> > static inline void iort_enable_acs(struct acpi_iort_node *iort_node)
> > { }
> > #endif
> > +static int iort_rmr_desc_valid(struct acpi_iort_rmr_desc *desc)
> > +{
> > + struct iort_rmr_entry *e;
> > + u64 end, start = desc->base_address, length = desc->length;
> > +
> > + if ((!IS_ALIGNED(start, SZ_64K)) || (length % SZ_64K != 0))
>
> You could just do:
>
> if ((!IS_ALIGNED(start, SZ_64K)) || (length % SZ_64K))
>
> [SAMI] In my opinion, the following may be better:
> if (!IS_ALIGNED(start, SZ_64K) || !IS_ALIGNED(length, SZ_64K))
> [/SAMI]
Thanks for your suggestions. I don't have a strong opinion on either
of those, but will change it with the latter one for now.
Thanks,
Shameer
> Regards,
>
> Sami Mujawar
>
> David
> ___
> Devel mailing list -- de...@acpica.org
> To unsubscribe send an email to devel-le...@acpica.org
> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] iommu/amd: Enforce 4k mapping for certain IOMMU data structures
Will,
I have already submitted v2 of this patch. Let me move the discussion there
instead ...
(https://lore.kernel.org/linux-iommu/20201105145832.3065-1-suravee.suthikulpa...@amd.com/)
Suravee
On 11/18/20 5:57 AM, Will Deacon wrote:
On Wed, Oct 28, 2020 at 11:18:24PM +, Suravee Suthikulpanit wrote:
AMD IOMMU requires 4k-aligned pages for the event log, the PPR log,
and the completion wait write-back regions. However, when allocating
the pages, they could be part of large mapping (e.g. 2M) page.
This causes #PF due to the SNP RMP hardware enforces the check based
on the page level for these data structures.
Please could you include an example backtrace here?
So, fix by calling set_memory_4k() on the allocated pages.
I think I'm missing something here. set_memory_4k() will break the kernel
linear mapping up into page granular mappings, but the IOMMU isn't using
that mapping, right? It's just using the physical address returned by
iommu_virt_to_phys(), so why does it matter?
Just be nice to capture some of this rationale in the log, especially as
I'm not familiar with this device.
Fixes: commit c69d89aff393 ("iommu/amd: Use 4K page for completion wait write-back
semaphore")
I couldn't figure out how that commit could cause this problem. Please can
you explain that to me?
Cheers,
Will
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 1/1] iommu/vt-d: Fix compile error with CONFIG_PCI_ATS not set
On Thu, 19 Nov 2020 13:51:19 +0800, Lu Baolu wrote: > Fix the compile error below (CONFIG_PCI_ATS not set): > > drivers/iommu/intel/dmar.c: In function ‘vf_inherit_msi_domain’: > drivers/iommu/intel/dmar.c:338:59: error: ‘struct pci_dev’ has no member > named ‘physfn’; did you mean ‘is_physfn’? > 338 | dev_set_msi_domain(>dev, > dev_get_msi_domain(>physfn->dev)); > | ^~ > | is_physfn Applied to arm64 (for-next/iommu/fixes), thanks! [1/1] iommu/vt-d: Fix compile error with CONFIG_PCI_ATS not set https://git.kernel.org/arm64/c/3645a34f5b96 Cheers, -- Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v6 3/3] firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module
On Tue, Nov 17, 2020 at 02:47:54PM +0100, Thierry Reding wrote: > On Mon, Nov 16, 2020 at 11:48:39AM -0800, John Stultz wrote: > > On Mon, Nov 16, 2020 at 8:36 AM Will Deacon wrote: > > > On Mon, Nov 16, 2020 at 04:59:36PM +0100, Thierry Reding wrote: > > > > On Fri, Nov 06, 2020 at 04:27:10AM +, John Stultz wrote: > > > > Unfortunately, the ARM SMMU module will eventually end up being loaded > > > > once the root filesystem has been mounted (for example via SDHCI or > > > > Ethernet, both with using just plain, non-IOMMU-backed DMA API) and then > > > > initialize, configuring as "fault by default", which then results from a > > > > slew of SMMU faults from all the devices that have previously configured > > > > themselves without IOMMU support. > > > > > > I wonder if fw_devlink=on would help here? > > > > > > But either way, I'd be more inclined to revert this change if it's causing > > > problems for !QCOM devices. > > > > > > Linus -- please can you drop this one (patch 3/3) for now, given that it's > > > causing problems? > > > > Agreed. Apologies again for the trouble. > > > > I do feel like the probe timeout to handle optional links is causing a > > lot of the trouble here. I expect fw_devlink would solve this, but it > > may be awhile before it can be always enabled. I may see about > > pushing the default probe timeout value to be a little further out > > than init (I backed away from my last attempt as I didn't want to > > cause long (30 second) delays for cases like NFS root, but maybe 2-5 > > seconds would be enough to make things work better for everyone). > > I think there are two problems here: 1) the deferred probe timeout can > cause a mismatch between what SMMU masters and the SMMU think is going > on and 2) a logistical problem of dealing with the SMMU driver being a > loadable module. > > The second problem can be dealt with by shipping the module in the > initial ramdisk. That's a bit annoying, but perhaps the right thing to > do. At least on Tegra we need this because all the devices that carry > the root filesystem (Ethernet for NFS and SDHCI/USB/SATA/PCI for disk > boot) are SMMU masters and will start to fault once the SMMU driver is > loaded. Realistically, if you're building an IOMMU driver as a module then it needs to be part of the initrd and fw_devlink needs to be enabled. Relying on timeouts and the phase of the moon is not going to be reliable. But back to the original issue, I think we should revert the Kconfig patch from Linus' tree. Please can you send a revert for that? Will ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Patch V8 3/3] iommu: Document usage of "/sys/kernel/iommu_groups//type" file
Hi Lu, On Thu, Nov 19, 2020 at 10:32:43AM +0800, Lu Baolu wrote: > On 11/18/20 9:51 PM, Will Deacon wrote: > > On Fri, Sep 25, 2020 at 12:06:20PM -0700, Ashok Raj wrote: > > I can't figure out from this description what string is returned to > > userspace in the case that the group is configured as blocked or unmanaged. > > This series only enables switching a default domain in use between DMA > and IDENTITY. Other cases will result in write failures. I understood that from the text, but what I couldn't figure out is what happens if you *read* the file when the default domain is not identity or DMA. I think that should be documented. Will ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Patch V8 1/3] iommu: Add support to change default domain of an iommu group
On Thu, Nov 19, 2020 at 10:18:05AM +0800, Lu Baolu wrote: > The original author of this patch series has left Intel. I am now the > backup. Ok, thanks for letting me know. > On 11/18/20 9:51 PM, Will Deacon wrote: > > On Fri, Sep 25, 2020 at 12:06:18PM -0700, Ashok Raj wrote: > > > From: Sai Praneeth Prakhya [...] > > > +free_new_domain: > > > + iommu_domain_free(group->default_domain); > > > + group->default_domain = prev_dom; > > > + group->domain = prev_dom;i > > > > Hmm. This seems to rely on all users of group->default_domain holding the > > group->mutex. Have you confirmed that this is the case? There's a funny > > use of iommu_group_get() in the exynos IOMMU driver at least. > > Emm. This change happens within the area with group->mutex held. Or I > am not getting your point? Yeah, sorry, I wasn't very clear. This code holds the group->mutex, and it relies on _anybody_ else who wants to inspect group->default_domain also holding that mutex, otherwise they could observe a transient domain pointer which we free on the failure path here. My question is whether or not there is code that inspects group->default_domain without group->mutex held? The exynos case doesn't obviously hold it, and I'd like to make sure that there aren't others that we need to worry about. Does that make more sense? Thanks, Will ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
