On 2026-10-17 23:31 -05:00, Sam Protsenko wrote:
> Since commit a3075db94d49 ("lmb: Reinstate access to memory above
> ram_top") secondary CPUs cannot be brought online on the E850-96 board
> in the Linux kernel:
>
> smp: Bringing up secondary CPUs ...
> CPU1: failed to come online
> CPU1: failed in unknown state : 0x0
> CPU2: failed to come online
> CPU2: failed in unknown state : 0x0
> ...
>
> This is happening because EL3_MON (the Secure Monitor firmware, which
> also implements PSCI) cannot handle 36-bit addresses in its CPU_ON PSCI
> function implementation. The E850-96 board has two 2 GiB RAM banks: the
> first one is mapped at 0x8000_0000, and the second one at 0x8_8000_0000.
> The commit mentioned above changed the LMB behavior (in U-Boot) in a way
> that the memory from the second RAM bank (which is mapped at 36-bit
> addresses) is not reserved anymore. The change in 'bdinfo' output:
>
> - reserved[4] [0x8_8000_0000 - 0x8_ffff_ffff], 0x8000_0000 bytes
> + reserved[3] [0x8_ffff_3000 - 0x8_ffff_ffff], 0xd000 bytes
>
> That in turn leads to the Linux kernel image being placed in the second
> RAM bank with 36-bit addresses. The corresponding change in /proc/iomem:
>
> - f480_0000 - f67e_ffff : Kernel code
> + 8_fa80_0000 - 8_fc7e_ffff : Kernel code
> - f6b6_0000 - f71a_ffff : Kernel data
> + 8_fcb6_0000 - 8_fd1a_ffff : Kernel data
>
> As evident from the Exynos850 device tree, it uses "psci" as a CPU
> enable method. That means the kernel relies on PSCI firmware (which is
> implemented as a part of EL3_MON firmware in case of Exynos850 SoC) to
> bring up secondary CPU cores. So when bringing up secondary CPUs the
> kernel makes an SMC call (for each secondary CPU) to execute CPU_ON PSCI
> function (see PSCI spec [1]). The kernel passes an "entry point address"
> to that SMC call, which is the address of the secondary_entry() function
> in the kernel, which should be executed when PSCI returns to the kernel
> from the SMC call:
>
> static int cpu_psci_cpu_boot(unsigned int cpu)
> {
> phys_addr_t pa_secondary_entry = __pa_symbol(secondary_entry);
> int err = psci_ops.cpu_on(cpu_logical_map(cpu),
> pa_secondary_entry);
> ...
>
> The pa_secondary_entry is a 64-bit variable, which is perfectly fine
> according to the PSCI spec. Unfortunately, in case when that address is
> actually bigger than 32-bit value, the EL3_MON firmware doesn't execute
> the function at that address -- that's the reason the secondary CPUs
> don't come up. And as discussed above, the kernel image is now located
> at a 36-bit address, which means the secondary_entry() function is at a
> 36-bit address too:
>
> - secondary_entry() address: f67d_f5a8 // 32-bit address
> + secondary_entry() address: 8_fc7d_f5a8 // 36-bit address
>
> It seems like a bug in the EL3_MON firmware, e.g. it might truncate the
> address to 32-bit by using a u32 variable (instead of u64) to store it
> internally. So the right way to fix this issue would be to fix the
> EL3_MON, but unfortunately it's closed source. So instead just enable
> LMB_LIMIT_DMA_BELOW_RAM_TOP for ARCH_EXYNOS9 to limit the available
> memory to the 32-bit boundary.
>
> Although this platform could benefit from having all memory available in
> U-Boot (e.g. for transferring big images over TFTP), and it looks like
> there are no hardware limitations on this platform too (DMA has 36-bit
> support, etc.), limiting the available memory in U-Boot is still the
> easiest way to workaround the EL3_MON problem, and 2 GiB is enough for
> all use cases in U-Boot currently. And of course all 4 GiB of RAM is
> available to use in Linux in any case, so it's not a bad tradeoff, and
> in fact it just brings back the previous behavior (before the "Reinstate
> access ..." patch).
>
> Enabling LMB_LIMIT_DMA_BELOW_RAM_TOP for the whole ARCH_EXYNOS9 family
> also seems justified, as it's likely that all modern ARM64 Exynos chips
> may share similar EL3_MON software, and thus possibly the same PSCI bug.
> For example, a similar issue was mentioned in exynos-mobile related
> patch [2], stating that after "Reinstate access ..." patch only CPU0 is
> able to come online on some variants of the Samsung Galaxy J6 (based on
> Exynos7870 SoC).
>
> This was tested on the E850-96 board:
>
> 1. Using 'bootefi' to boot the Linux kernel image
> - before this patch: the issue is reproducible
> - with this patch applied: the issue is not reproducible
> 2. Using 'booti' to boot the Linux kernel image from 0x8_8000_0000
> - before this patch: the issue is reproducible
> - with this patch applied: the image can't be loaded there anymore
> ("Reading file would overwrite reserved memory"), as the second
> RAM bank is reserved in LMB again
> 3. Using 'booti' to boot the Linux kernel image from 0x8000_0000
> - the issue is not reproducible
>
> [1] https://support.arm.com/documentation/den0022/fb/
> [2]
> https://lore.kernel.org/u-boot/[email protected]/
>
> Signed-off-by: Sam Protsenko <[email protected]>
Acked-by: Kaustabh Chakraborty <[email protected]>
This is also an issue in Exynos 7870 platforms, so ARCH_EXYNOS7 also
needs to be added.
I did send a patch to fix this using some hacks, but this seems to be
the correct approach as various other platforms have also opted for it.
