On Sat, Oct 17, 2026 at 11:31:28PM -0500, Sam Protsenko wrote:
> Since commit a3075db94d49 ("lmb: Reinstate access to memory above
> ram_top") secondary CPUs cannot be brought online on the E850-96 board
> in the Linux kernel:
>
> smp: Bringing up secondary CPUs ...
> CPU1: failed to come online
> CPU1: failed in unknown state : 0x0
> CPU2: failed to come online
> CPU2: failed in unknown state : 0x0
> ...
>
> This is happening because EL3_MON (the Secure Monitor firmware, which
> also implements PSCI) cannot handle 36-bit addresses in its CPU_ON PSCI
> function implementation. The E850-96 board has two 2 GiB RAM banks: the
> first one is mapped at 0x8000_0000, and the second one at 0x8_8000_0000.
> The commit mentioned above changed the LMB behavior (in U-Boot) in a way
> that the memory from the second RAM bank (which is mapped at 36-bit
> addresses) is not reserved anymore. The change in 'bdinfo' output:
>
> - reserved[4] [0x8_8000_0000 - 0x8_ffff_ffff], 0x8000_0000 bytes
> + reserved[3] [0x8_ffff_3000 - 0x8_ffff_ffff], 0xd000 bytes
>
> That in turn leads to the Linux kernel image being placed in the second
> RAM bank with 36-bit addresses. The corresponding change in /proc/iomem:
>
> - f480_0000 - f67e_ffff : Kernel code
> + 8_fa80_0000 - 8_fc7e_ffff : Kernel code
> - f6b6_0000 - f71a_ffff : Kernel data
> + 8_fcb6_0000 - 8_fd1a_ffff : Kernel data
>
> As evident from the Exynos850 device tree, it uses "psci" as a CPU
> enable method. That means the kernel relies on PSCI firmware (which is
> implemented as a part of EL3_MON firmware in case of Exynos850 SoC) to
> bring up secondary CPU cores. So when bringing up secondary CPUs the
> kernel makes an SMC call (for each secondary CPU) to execute CPU_ON PSCI
> function (see PSCI spec [1]). The kernel passes an "entry point address"
> to that SMC call, which is the address of the secondary_entry() function
> in the kernel, which should be executed when PSCI returns to the kernel
> from the SMC call:
>
> static int cpu_psci_cpu_boot(unsigned int cpu)
> {
> phys_addr_t pa_secondary_entry = __pa_symbol(secondary_entry);
> int err = psci_ops.cpu_on(cpu_logical_map(cpu),
> pa_secondary_entry);
> ...
>
> The pa_secondary_entry is a 64-bit variable, which is perfectly fine
> according to the PSCI spec. Unfortunately, in case when that address is
> actually bigger than 32-bit value, the EL3_MON firmware doesn't execute
> the function at that address -- that's the reason the secondary CPUs
> don't come up. And as discussed above, the kernel image is now located
> at a 36-bit address, which means the secondary_entry() function is at a
> 36-bit address too:
>
> - secondary_entry() address: f67d_f5a8 // 32-bit address
> + secondary_entry() address: 8_fc7d_f5a8 // 36-bit address
>
> It seems like a bug in the EL3_MON firmware, e.g. it might truncate the
> address to 32-bit by using a u32 variable (instead of u64) to store it
> internally. So the right way to fix this issue would be to fix the
> EL3_MON, but unfortunately it's closed source. So instead just enable
> LMB_LIMIT_DMA_BELOW_RAM_TOP for ARCH_EXYNOS9 to limit the available
> memory to the 32-bit boundary.
>
> Although this platform could benefit from having all memory available in
> U-Boot (e.g. for transferring big images over TFTP), and it looks like
> there are no hardware limitations on this platform too (DMA has 36-bit
> support, etc.), limiting the available memory in U-Boot is still the
> easiest way to workaround the EL3_MON problem, and 2 GiB is enough for
> all use cases in U-Boot currently. And of course all 4 GiB of RAM is
> available to use in Linux in any case, so it's not a bad tradeoff, and
> in fact it just brings back the previous behavior (before the "Reinstate
> access ..." patch).
>
> Enabling LMB_LIMIT_DMA_BELOW_RAM_TOP for the whole ARCH_EXYNOS9 family
> also seems justified, as it's likely that all modern ARM64 Exynos chips
> may share similar EL3_MON software, and thus possibly the same PSCI bug.
> For example, a similar issue was mentioned in exynos-mobile related
> patch [2], stating that after "Reinstate access ..." patch only CPU0 is
> able to come online on some variants of the Samsung Galaxy J6 (based on
> Exynos7870 SoC).
>
> This was tested on the E850-96 board:
>
> 1. Using 'bootefi' to boot the Linux kernel image
> - before this patch: the issue is reproducible
> - with this patch applied: the issue is not reproducible
> 2. Using 'booti' to boot the Linux kernel image from 0x8_8000_0000
> - before this patch: the issue is reproducible
> - with this patch applied: the image can't be loaded there anymore
> ("Reading file would overwrite reserved memory"), as the second
> RAM bank is reserved in LMB again
> 3. Using 'booti' to boot the Linux kernel image from 0x8000_0000
> - the issue is not reproducible
>
> [1] https://support.arm.com/documentation/den0022/fb/
> [2]
> https://lore.kernel.org/u-boot/[email protected]/
>
> Signed-off-by: Sam Protsenko <[email protected]>
> ---
> lib/Kconfig | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/lib/Kconfig b/lib/Kconfig
> index 021a82d47092..e0bfd10f2b7f 100644
> --- a/lib/Kconfig
> +++ b/lib/Kconfig
> @@ -1371,7 +1371,8 @@ config SPL_LMB_ARCH_MEM_MAP
> config LMB_LIMIT_DMA_BELOW_RAM_TOP
> bool
> depends on LMB
> - default y if ARCH_BCM283X || ARCH_MVEBU || ARCH_ROCKCHIP ||
> STARFIVE_JH7110
> + default y if ARCH_BCM283X || ARCH_EXYNOS9 || ARCH_MVEBU || \
> + ARCH_ROCKCHIP || STARFIVE_JH7110
> help
> Some architectures can not DMA above ram_top boundary,
> which is after 4 GiB or 32-bit boundary too. Limit theThis reminds me of my worry at the time, that we would keep hitting platforms where this is a functional problem (and there's also the imx8 patch to set this option). I know there's some valid use cases for this on Renesas hardware for example. Marek, how do you feel at this point about inverting the logic and making this be an opt-in rather than opt-out flag? -- Tom
signature.asc
Description: PGP signature
