On Fri, Jun 16, 2023 at 4:52 PM Tim Harvey <[email protected]> wrote: > > On Thu, Jun 15, 2023 at 8:31 PM Peng Fan <[email protected]> wrote: > > > > > > > > On 6/16/2023 9:56 AM, Tim Harvey wrote: > > > Greetings, > > > > > > I've seen several IMX8M boards include a firmware/optee node in the > > > U-Boot dt (git grep optee arch/arm/dts/imx8m*.dtsi) yet but none that > > > I see configure binman to actually add the binary in the u-boot.dtsi, > > > do anything to keep U-Boot from accessing the OPTEE memory, or > > > document how to configure and build OPTEE for imx8m. I would like to > > > add such support but I find it odd that OPTEE needs to be built > > > differently depending on the dram size. > > > > > > Prior to switching to binman (v2021.10) > > > arch/arm/mach-imx/mkimage_fit_atf.sh [1] would include tee.bin in the > > > FIT image if it was found in the current directory and it was expected > > > that you provide a proper TEE_LOAD_ADDR via the env (commit > > > a9eed6e1b8d8 ("imx: imx8m: introduce script to generate fit image"). > > > > > > According to the IMX OPTEE documentation [2] the size and location of > > > OPTEE is hard coded (CFG_TZDRAM_START and CFG_TZDRAM_SIZE) but looking > > > at the imx-optee source [3] this is calculated based off of > > > CFG_DDR_SIZE (which you can provide via env at build time or just > > > provide CFG_TZDRAM_START and CFG_TZDRAM_SIZE via env directly). > > > > optee does not support PIE, it could only run at the address that built > > time defined. > > Hi Peng, > > I wasn't thinking about PIE but just building it at a location other > than the top of DRAM in order to come up with a generalized location > instead of having to customize u-boot.dtsi for different RAM sizes and > to workaround the 4GiB boundary issue. > > If my understanding is correct optee's load address needs to match between: > 1. atf: BL32_BASE at build time defines the address in secure memory > where BL2 loads the BL32 binary image (tee.bin) (must be aligned on a > page-size boundary) > 2. optee: CFG_TZDRAM_START at build time (calculated based off of > CFG_DDR_SIZE) defines the address it is built to run at. Note you must > still define CFG_DDR_SIZE properly as this is passed to > imx_tzc_auto_configure() > 3. uboot: tee.bin is contained in u-boot.itb FIT image and specifies > the address that U-Boot SPL loads tee.bin into > > I'm really trying to understand the imx8m 4GiB limit issue as that > seems to be a huge limiation.
Peng, I've made good progress and now have a 4GiB imx8mm board working with optee (with 1 hack): > > On an imx8mm board with 4GiB of DRAM configured with > atf:BL32_BASE=0x13e000000 > optee:CFG_DDR_SIZE=0x100000000,CFG_TZDRAM_START=0x13e000000 and > u-boot.dtsi tee.bin load/entry=<0x1 0x3e000000> we get this: > > U-Boot SPL 2023.04-00027-g578c653cbafa (Jun 16 2023 - 09:22:40 -0700) > GSCv3 : v58 0xf098 RST:VIN Thermal protection:disabled > RTC : 1970-01-01 0:05:40 UTC > Model : GW7201-01-EB > Serial : 852455 > MFGDate : 11-10-2020 > PMIC : MP5416 (IMX8MM) > DRAM : LPDDR4 4 GiB 3000MT/s 1500MHz > WDT: Started watchdog@30280000 with servicing every 1000ms (60s timeout) > Trying to boot from eMMC > DTB : imx8mm-venice-gw72xx-0x > Cannot use 64 bit addresses with SDMA > ^^^ This is the imx mmc driver warning that it can't load data into > dram across 32bit boundary > > Authenticate image from DDR location 0x401fcdc0... > NOTICE: Do not release JR0 to NS as it can be used by HAB > ERROR: mmap_add_region_check() failed. error -34 > ASSERT: lib/xlat_tables_v2/xlat_tables_core.c:793 > BACKTRACE: START: assert > 0: EL3: 0x9289cc > 1: EL3: 0x929efc > 2: EL3: 0x92469c > 3: EL3: 0x9248f4 > 4: EL3: 0x928850 > 5: EL3: 0x927594 > 6: EL3: 0x920128 > 7: EL3: 0x7eb19c > 8: EL3: 0x7f3de0 > BACKTRACE: END: assert > > This is where the atf jumps to optee so I can understand why the above > doesn't work as U-Boot can't load tee.bin from the FIT to 0x13e000000 > (not sure where it gets loaded... the 64bit address probably wraps). > I've also tried setting CFG_CORE_LARGE_PHYS_ADDR=y and > CFG_CORE_ARM64_PA_BITS=36 in optee as this is what the imx8mp-evk > config with 6GiB does and it ended with the same result but I do > notice that doing this changes the link address The issue above appears to be a result of not setting CFG_CORE_LARGE_PHYS_ADDR=y and CFG_CORE_ARM64_PA_BITS=36 in optee when CFG_DDR_SIZE exceeds 3GiB. > > The imx8mp-evk has 6GiB of dram on it. Does NXP not have a solution > for secure boot on boards with >3GiB of dram? Is there some support > in their downstream U-Boot that I'm maybe missing? Personally I can't > even get NXP's lf_v2022.04 u-boot branch to build flash.bin for > imx8mp_evk so I can try it out. The optee configuration for imx8mp-evk in core/arch/arm/plat-imx/conf.mk is misleading as it does not override the default for CFG_TZDRAM_START which will get defaulted to above a 32bit address boundary and won't work. Instead it should set it below the 32bit address boundary in order for uboot to load it from a FIT image, like the mx8mp_rsb3720_6g PLATFORM_FLAVOR does: @@ -379,6 +391,7 @@ endif ifneq (,$(filter $(PLATFORM_FLAVOR),mx8mpevk)) CFG_DDR_SIZE ?= UL(0x180000000) CFG_UART_BASE ?= UART2_BASE +CFG_TZDRAM_START ?= 0x56000000 $(call force,CFG_CORE_LARGE_PHYS_ADDR,y) $(call force,CFG_CORE_ARM64_PA_BITS,36) endif > > Also, if I look at the latest NXP Yocto BSP I don't even see where > they configure BL32_BASE when building the atf which leaves it to > default to something SOC-specific that does not depend on dram size > and does not match the dram sizes for imx8mn_evk and imx8mp_evk > (https://github.com/nxp-imx/meta-imx/blob/kirkstone-5.15.71-2.2.0/meta-bsp/recipes-bsp/imx-atf/imx-atf_2.6.bb). > > > > > And the TZASC needs following it to protect the region properly. > > > > > > > > This results in the OPTEE location of 0x7e000000 for 1GiB systems, > > > 0xbe000000 for 2GiB systems, and 0x13e000000 for 4GiB systems for > > > example. This dependence on dram size causes a couple of issues: > > > - OPTEE location depends on build-time configuration and several > > > boards have run-time identification of their memory size > > > - for 4GiB systems the address 0x13e000000 crosses the 32bit address > > > boundary and U-Boot hangs > > > > imx optee seems has this. But for overlapping with U-Boot, I doubt that. > > > > > > > > Is there any reason OPTEE can't be located wherever you want and not > > > depend on dram size? > > > > You could , just modify the imx *.mk file per my recall. > > Right. So, for example on the same imx8mm board with 4GiB I can > configure uboot/atf/optee to load tee at top of 3GiB with > atf:BL32_BASE=0xfe000000 > optee:CFG_DDR_SIZE=0xc0000000,CFG_TZDRAM_START=0xfe000000 and > u-boot.dtsi tee.bin load/entry=<0xfe000000>. > > I finally found that this boots only if I also adjust the size > configured by board_phys_sdram_size() down to 3GiB otherwise we hang > somewhere before dram_init_banksize(). I am guessing this still has > something to do with missing a uboot carevout for the memory tee is > loaded in (and I'm not clear how to add a carve-out yet). I still > would like to understand this hang as while this is somewhat of a > work-around but it does cripple the board's memory size that gets > passed along to Linux. This hang between init_dram() and dram_init_banksize() appears to be caused by my U-Boot dram bank configuration which was essentially: /* SDRAM configuration: 4GiB */ #define CFG_SYS_SDRAM_BASE 0x40000000 #define PHYS_SDRAM 0x40000000 #define PHYS_SDRAM_SIZE 0x100000000 /* 4 GB */ This appears to be incorrect as it allows U-Boot to go over the 32bit address boundary and should instead use a 2 bank configuration: /* SDRAM configuration: 4GiB */ #define CFG_SYS_SDRAM_BASE 0x40000000 #define PHYS_SDRAM 0x40000000 #define PHYS_SDRAM_SIZE 0x80000000 /* 2 GB */ #define PHYS_SDRAM_2 0xC0000000 #define PHYS_SDRAM_2_SIZE 0x80000000 /* 2 GB */ Even with the above 2-bank configuration, I still find I hang in get_ram_size() called from my board_phys_sdram_size() override if I pass get_ram_size() a full 4GiB size: get_ram_size((void *)PHYS_SDRAM, PHYS_SDRAM_SIZE + PHYS_SDRAM_2_SIZE); If I avoid calling get_ram_size() and hard-code the dram size U-Boot boots fine: '*size = SZ_4G;' I don't think the get_ram_size() hang is a U-Boot carve-out issue because if I configure optee's load address for something like 0x7e000000 (below 2GiB DRAM) and use get_ram_size(PHYS_SDRAM, SZ_2G) it does not hang (but a memtest from 40000000 to 80000000 will, which makes sense). Do you have any ideas what could be causing the hang in get_ram_size() when optee is enabled? Best Regards, Tim > > > > > > I've tried to set it to 0x7e000000 (for a 1GiB > > > board) on a 4GiB board and U-Boot hangs near the DRAM banner which is > > > likely due to get_ram_size() stepping on OPTEE's memory range but then > > > > You may give a try to disable TZASC settings in OP-TEE first. > > And you also need to carve out the memory of OP-TEE from U-Boot MMU > > to avoid prefetch. > > I had not thought about this. It looks like the TZASC provides > configurable protection over dram memory space. It looks like both the > atf and optee configure the TZASC. U-Boot SPL enable_tzc380() enables > TZASC. If I disable that it does not change the hang so perhaps the > atf/optee is re-enabling it? > > > > > > I'm not clear why that doesn't occur when OPTEE is located at the top > > > of dram on a 1GiB and 2GiB system. How would I go about adding a > > > carevout for this? > > > > Are u using upstream optee or NXP downstream optee release? > > I've been trying many combinations including upstream optee/atf as > well as NXP's latest branches (imx-optee-os-lf-6.1.1_1.0.0 and > imx-atx-lf_v2.6). > > Best Regards, > > Tim > > > > > > > Regards, > > Peng. > > > > > > > > Best Regards, > > > > > > Tim > > > [1] > > > https://elixir.bootlin.com/u-boot/v2021.10/source/arch/arm/mach-imx/mkimage_fit_atf.sh > > > [2] https://www.nxp.com/docs/en/user-guide/IMX_PORTING_GUIDE.pdf > > > [3] https://github.com/nxp-imx/imx-optee-os
