casaroli opened a new pull request, #21463: URL: https://github.com/apache/nuttx/pull/21463
## Summary qemu-armv7a:smp with 4 CPUs sometimes aborts at boot: prefetch abort at PC 0xdeadbeee or data abort at 0xdeadbeef, in the CPU0 IDLE task, with SP on the interrupt stack of CPU 0 and registers 0xdeadbeef. On master this happens in 41 of 300 boots. `arm_color_intstack()` writes the color 0xdeadbeef into the interrupt stacks of all CPUs. Since 79711737bcd `up_irqinitialize()` calls it, and qemu, imx6, goldfish, rtl8730e, cxd56 and rp2040 call `up_irqinitialize()` on each secondary CPU also. So a secondary CPU that boots writes over the register frame of a CPU that handles an interrupt at that time. This PR colors all the stacks only on CPU 0. CPU 0 does this in `nx_start()`, before `nx_smp_start()` and before its own interrupts are enabled. The first commit fixes an older nxstyle error in the same file. ## Impact Only SMP builds on arm with CONFIG_STACK_COLORATION change. A secondary CPU no longer writes any interrupt stack. `up_check_intstack()` still works for each CPU, because CPU 0 colors them all. This includes lc823450, where a secondary CPU does not call `up_irqinitialize()`. The code without SMP does not change. No documentation change: Documentation/ does not describe this behavior. ## Testing Boots under QEMU TCG, qemu-armv7a:smp, 4 CPUs, same .config, GNU Arm 13.2. Each batch boots master and this PR in turn, to an NSH prompt: | Batch | master | this PR | |---|---|---| | 200 boots each, other builds run on the host | 35 abort, 2 stop in the fatal dump, 163 ok | 200 ok | | 100 boots each, host less busy | 4 abort, 96 ok | 100 ok | The first batch ran one commit before the final state of this PR (only the nxstyle blank line differs). Proof of the cause, with debug changes that are not in this PR, on master. CPU 0 colors all stacks once as usual, and only the secondary CPUs change. The longer loop is 200 passes: | Secondary CPU does this | Boots | Result | |---|---|---| | writes all stacks, longer loop | 100 | 30 abort, 70 no prompt, 0 ok | | reads all stacks, same loop, no writes | 100 | 100 ok | | writes only its own stack, same loop | 100 | 100 ok | | writes all stacks once, and reports a write to the stack of CPU 0 while CPU 0 is in `arm_doirq()` | 300 | 3 abort with the report, 0 abort without it, 12 ok with it, 285 ok without it | Stack usage: a forced data abort on this PR (`mw f0000000=1`) prints the interrupt stack of each CPU in the dump: 208, 200, 200 and 200 bytes of 2048. ostest, this PR, exit status 0: qemu-armv7a:smp (4 CPUs), qemu-armv7a:nsh (no SMP), sabre-6quad:smp (4 CPUs, QEMU sabrelite). sabre-6quad:smp also boots 30 of 30 times. Master was not run on sabrelite. Builds with EXTRAFLAGS="-Wno-cpp -Werror": qemu-armv7a:smp (GNU Arm 13.2 and 15.3), qemu-armv7a:nsh, qemu-armv7a:knsh_smp (kernel build, not run), sabre-6quad:smp. With STACK_COLORATION enabled by kconfig-tweak: spresense:smp, raspberrypi-pico:smp, lc823450-xgevk:nsh, which run on no QEMU here. Not built: goldfish (no board in the tree) and rtl8730e_evb (the SDK fetch failed here). They call the same function, by reading. Clang on qemu-armv7a stops at math.h in libc on this host. The change uses only `this_cpu()`. No test added: no CI configuration can show a race with this rate as a pass or fail in one boot. The runs and builds are on f19dbe6f6d3. The branch is now rebased on ba92fb6bc82 (no change in the touched files). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
