Am 7. Juli 2021 05:18:20 MESZ schrieb Heinrich Schuchardt <[email protected]>: >Am 7. Juli 2021 03:44:35 MESZ schrieb AKASHI Takahiro ><[email protected]>: >>François, >> >>On Tue, Jul 06, 2021 at 08:10:08PM +0200, Heinrich Schuchardt wrote: >>> On 7/6/21 6:13 PM, François Ozog wrote: >>> > Hi Heinrich, U-Boot 2021-07rc5 does not take into account memory >>> > description when using Qemu 5.2 NUMA configuration to adapt memory >>map >>> > (kernel_addr_r...): >>> > >>> > -smp 4 \ >>> > -m 8G,slots=2,maxmem=16G \ >>> > -object memory-backend-ram,size=4G,id=m0 \ >>> > -object memory-backend-ram,size=4G,id=m1 \ >>> > -numa node,cpus=0-1,nodeid=0,memdev=m0 \ >>> > -numa node,cpus=2-3,nodeid=1,memdev=m1 >>> > >>> > kernel_addr_r is still 0x4040000 and thus you can't use it to >>bootefi. >>> > >>> > fdt addr 0x13ede6de0; fdt print >>> > >>> > Displays fdt while I think it should not. >>> > >>> > If I load the kernel at dram.start, the load works but not boot >>> > >>> > U-Boot 2021.07 (Jul 06 2021 - 13:26:43 +0000) >>> > >>> > >>> > DRAM:4 GiB >>> > >>> > Flash: 64 MiB >>> > >>> > Loading Environment from Flash... OK >>> > >>> > In:pl011@9000000 >>> > >>> > Out: pl011@9000000 >>> > >>> > Err: pl011@9000000 >>> > >>> > Net: eth0: virtio-net#32 >>> > >>> > Hit any key to stop autoboot:0 >>> > >>> > => >>> > >>> > => bdinfo >>> > >>> > boot_params = 0x0000000000000000 >>> > >>> > DRAM bank = 0x0000000000000000 >>> > >>> > -> start= 0x0000000140000000 >>> > >>> > -> size = 0x0000000100000000 >>> > >>> > flashstart= 0x0000000000000000 >>> > >>> > flashsize = 0x0000000004000000 >>> > >>> > flashoffset = 0x00000000000bc990 >>> > >>> > baudrate= 115200 bps >>> > >>> > relocaddr = 0x000000013ff27000 >>> > >>> > reloc off = 0x000000013ff27000 >>> > >>> > Build = 64-bit >>> > >>> > current eth = virtio-net#32 >>> > >>> > ethaddr = 52:52:52:52:52:52 >>> > >>> > IP addr = <NULL> >>> > >>> > fdt_blob= 0x000000013ede6de0 >>> > >>> > new_fdt = 0x000000013ede6de0 >>> > >>> > fdt_size= 0x0000000000100000 >>> > >>> > lmb_dump_all: >>> > >>> > memory.cnt= 0x1 >>> > >>> > memory.reg[0x0].base = 0x140000000 >>> > >>> > .size = 0x100000000 >>> > >>> > >>> > reserved.cnt= 0x0 >>> > >>> > arch_number = 0x0000000000000000 >>> > >>> > TLB addr= 0x000000013fff0000 >>> > >>> > irq_sp= 0x000000013ede6dd0 >>> > >>> > sp start= 0x000000013ede6dd0 >>> > >>> > Early malloc usage: 3a8 / 2000 >>> > >>> > => load virtio 0:1 0x140000000 /oskit.efi >>> > >>> > 853424 bytes read in 1 ms (813.9 MiB/s) >>> > >>> > => bootefi0x140000000 0x13ede6dd0 >>> > >>> > ERROR: Failed to register WaitForKey event >>> > >>> > Setting OsIndications failed >>> > >>> > Error: Cannot initialize UEFI sub-system, r = 9 >>> > >>> > >>> > I think there is a need to calculate memory map based on previous >>> > firmware (TFA, QEMU can be considered as previous frimware) >>information >>> > (DT or blob_list). >>> > >>> > What do you think ? >>> > >>> > Cheers >>> > >>> > FF >>> > >>> > -- >>> > >>> > François-Frédéric Ozog | /Director Business Development/ >>> > T: +33.67221.6485 >>> > [email protected] <mailto:[email protected]> >>| Skype: ffozog >>> > >>> > >>> >>> The kernel load address is hard coded here: >>> include/configs/qemu-arm.h:41: "kernel_addr_r=0x40400000\0" \ >>> >>> bdinfo shows: >>> DRAM start = 0x140000000 >>> DRAM size = 0x100000000 >>> >>> fdt addr $fdt_addr >>> fdt printf >>> >>> shows two memory areas. One at 40000000, one at 140000000. >> >>(This shows that U-Boot receives a correct memory map via dtb.) >> >>Is this a NUMA machine, isn't it? Why should we care of which >>memory region be used here? Please note that this is a virtual >machine, >>there is no practical difference between two regions. >> >>The root problem is that U-Boot did not recognize there were two >>memory regions. We can fix this issue in either way: >> >>1) >>diff --git a/configs/qemu_arm64_defconfig >>b/configs/qemu_arm64_defconfig >>index f6e586627a8e..b70ffae8bf6e 100644 >>--- a/configs/qemu_arm64_defconfig >>+++ b/configs/qemu_arm64_defconfig >>@@ -1,7 +1,7 @@ >> CONFIG_ARM=y >> CONFIG_POSITION_INDEPENDENT=y >> CONFIG_ARCH_QEMU=y >>-CONFIG_NR_DRAM_BANKS=1 >>+CONFIG_NR_DRAM_BANKS=2 >> CONFIG_ENV_SIZE=0x40000 >> CONFIG_ENV_SECT_SIZE=0x40000 >> CONFIG_AHCI=y >> >>2) >>diff --git a/lib/fdtdec.c b/lib/fdtdec.c >>index 4b097fb588ed..4067ea2dead6 100644 >>--- a/lib/fdtdec.c >>+++ b/lib/fdtdec.c >>@@ -1111,7 +1111,7 @@ int fdtdec_setup_memory_banksize(void) >> return -EINVAL; >> } >> >>- for (bank = 0; bank < CONFIG_NR_DRAM_BANKS; bank++) { >>+ for (bank = 0; ; bank++) { >> ret = ofnode_read_resource(mem, reg++, &res); >> if (ret < 0) { >> reg = 0; >> >> (fdtdec_setup_memory_banksize() is called in dram_init_banksize().) >> >> >>(2) seems much better, but I don't know why we had to use >>CONFIG_NR_DRAM_BANKS here. >> >>In this case, other occurrences of CONFIG_NR_DRAM_BANKS in this file >>should be replaced with a variable for it. >> >>> Your use case is well beyond the typical U-Boot usage. So I guess it >>> will be up to Linaro to provide the necessary patches: >>> >>> * determine the active CPU >>> * determine the RAM assigned to the active CPU according >>> to the numa-node-id in the device-tree >>> * make sure that U-Boot only uses the memory of the active CPU >>> internally >>> * make sure that the UEFI memory map contains a compliant >description >>> * possibly, dynamically set up the environment variables >>> >>> +CC Tuomas Tynkkynen (maintainer for qemu_arm64_defconfig) >> >>For (1), we'd better have a different config, or increase >>the value of CONFIG_NR_DRAM_BANKS to a bigger number? > >Is the system configured such that each CPU can access the others CPU's >RAM when entering U-Boot? > >Best regards > >Heinrich >
At least the comments for this patch sound as if on a physical system cross NUMA node memory access is only available after full SMP initialization: https://patchwork.kernel.org/project/linux-acpi/patch/[email protected]/ QEMU may be less restrictive. QEMU allows the node distance to be 255 indicating that cross node access is infeasible. Best regards Heinrich >> >>-Takahiro Akashi >> >> >>> Best regards >>> >>> Heinrich
