| Issue |
241851
|
| Summary |
[ARM] CortexM4Model pre-RA scheduling spills heavily on unrolled 64-bit multiplies (cortex-m3/m4/m33/m35p)
|
| Labels |
new issue
|
| Assignees |
|
| Reporter |
danmcleran
|
*Disclosure: this issue was drafted with the help of an AI tool (Claude Code), per the [LLVM AI Tool Use Policy](https://llvm.org/docs/AIToolPolicy.html). The compiler outputs and the STM32F411 measurements were collected in that tool-assisted session, on my board. I have reviewed all of it and can answer questions about it.*
## Summary
On the CPUs that use `CortexM4Model` (cortex-m3, cortex-m4, cortex-m33, cortex-m35p), the pre-RA machine scheduler turns a fully unrolled sum of 64-bit products into code that spills heavily. With `-mcpu=generic`, or with `-mcpu=cortex-m4 -mllvm -enable-misched=false`, the same function needs no stack on clang trunk; with cortex-m4 it makes 47 stack accesses in a 40-byte frame (llc 18 on the IR below: 88 against 7). The IR is identical in every case; only the CPU differs.
The scheduler appears to start most of a row's `smull`s before it narrows any of them. Each 64-bit product holds a register pair until it is narrowed to `(lo >> 16) | (hi << 16)`, so with several products live at once they no longer fit in the core registers. In the clang trunk output most spills are the high halves (4-byte spills); llc 18 on the IR below spills whole products with `strd` (22 8-byte spills and 11 4-byte spills, reloaded by 55 4-byte loads, against 3 spills and 4 reloads for generic).
Scheduling top-down avoids it: with `-mllvm -misched-prera-direction=topdown`, clang trunk needs no stack for cortex-m4 either. Bottom-up scheduling spills as much as the default.
With clang, `-march=armv7e-m` alone already selects `-target-cpu cortex-m4` (checked with clang 18), so C code for any Cortex-M4 gets this by default.
## Reproducer
```c
#include <stdint.h>
/* Q16.16 multiply: 64-bit product, arithmetic shift, truncate to 32 bits. */
static inline int32_t qmul(int32_t a, int32_t b) {
return (int32_t)(((int64_t)a * b) >> 16);
}
/* A small fully connected layer: 8 inputs, 4 outputs. */
void dense(const int32_t w[4][8], const int32_t bias[4],
const int32_t x[8], int32_t y[4]) {
#pragma clang loop unroll(full)
for (int n = 0; n < 4; n++) {
int32_t s = 0;
for (int i = 0; i < 8; i++)
s += qmul(w[n][i], x[i]);
y[n] = bias[n] + s;
}
}
```
```
clang --target=arm-none-eabi -mthumb -mfloat-abi=soft -O2 -S repro.c -mcpu=cortex-m4
clang --target=arm-none-eabi -mthumb -mfloat-abi=soft -O2 -S repro.c -march=armv7e-m -mcpu=generic
clang --target=arm-none-eabi -mthumb -mfloat-abi=soft -O2 -S repro.c -mcpu=cortex-m4 -mllvm -enable-misched=false
clang --target=arm-none-eabi -mthumb -mfloat-abi=soft -O2 -S repro.c -mcpu=cortex-m4 -mllvm -misched-prera-direction=topdown
```
Clang trunk (Compiler Explorer, "armv7-a clang (trunk)", 2026-11-11 and 2026-11-12):
| `dense` | stack loads + stores | stack frame |
|---|---:|---:|
| `-mcpu=generic` | 0 | none |
| `-mcpu=cortex-m4` | 47 | 40 bytes |
| `-mcpu=cortex-m4 -mllvm -enable-misched=false` | 0 | none |
| `-mcpu=cortex-m4 -mllvm -misched-prera-direction=topdown` | 0 | none |
All four have the same 32 `smull`s. Only the scheduling differs. "Stack frame" here and below is the spill area (`sub sp, #N`); every build also pushes the callee-saved registers. clang 18.1.3 gives the same picture with a larger cortex-m4 frame: 60 stack accesses in 52 bytes, and 0 with `-mllvm -enable-misched=false` or `-mllvm -misched-topdown` (the option's name before `-misched-prera-direction`).
The `repro.ll` at the end of this issue is the same function as rustc emits it (rustc 1.99.1, LLVM 23.1.1). It contains no CPU attribute, so `llc -mcpu` selects the CPU:
```
llc -O3 -mtriple=thumbv7em-none-eabi -mcpu=generic repro.ll
llc -O3 -mtriple=thumbv7em-none-eabi -mcpu=cortex-m4 repro.ll
```
llc 18.1.3: generic 186 instructions, 7 stack accesses, 12-byte frame; cortex-m4 273 instructions, 88 stack accesses, 220-byte frame. rustc's LLVM 23.1.1 gives the same split. With `-C target-cpu=cortex-m4` the IR rustc produces is byte-identical except for the `"target-cpu"` attribute, so this happens in the backend.
## Which CPUs
llc 18.1.3 on `repro.ll`:
| `-mtriple` / `-mcpu` | scheduling model | stack accesses | frame |
|---|---|---:|---:|
| thumbv7m, thumbv7em, thumbv8m.main, thumbv8.1m.main / generic | CortexA8Model, no `use-misched` | 7 | 12 |
| thumbv7m / cortex-m3 | CortexM4Model | 88 | 220 |
| thumbv7em / cortex-m4 | CortexM4Model | 88 | 220 |
| thumbv7em / cortex-m7 | CortexM7Model | 15 | 20 |
| thumbv8m.main / cortex-m33 | CortexM4Model | 88 | 220 |
| thumbv8m.main / cortex-m35p | CortexM4Model | 88 | 220 |
| thumbv8.1m.main / cortex-m55 | CortexM55Model | 12 | 16 |
| thumbv8.1m.main / cortex-m85 | CortexM85Model | 19 | 28 |
| thumbv7em / cortex-m4, `-enable-misched=false` | CortexM4Model | 7 | 12 |
| thumbv7em / cortex-m4, `-misched-topdown` | CortexM4Model | 9 | 8 |
| thumbv7em / cortex-m4, `-misched-bottomup` | CortexM4Model | 88 | 220 |
So it follows `CortexM4Model` (in `ARMProcessors.td` it covers cortex-m3, m4, m33 and m35p, all with `FeatureUseMISched`; `generic` uses CortexA8Model without it, so the pre-RA machine scheduler does not run there). `-enable-misched=false` removes the spills; `-enable-post-misched=false` makes no difference. Forcing top-down removes nearly all of them, while bottom-up matches the default bidirectional result exactly, so the bottom-up side seems to be where the products get hoisted. rustc's LLVM 23.1.1 agrees: 92 stack accesses with cortex-m4 by default, 15 with `-misched-prera-direction=topdown`, 9 with `-enable-misched=false`, 11 for generic.
## Effect on hardware
I found this by measuring first: the slowdown showed up in cycle counts, and reading the assembly then traced it to the spills. On an STM32F411 (Cortex-M4F, 100 MHz), a training step of a small 8-4-8 Q16.16 autoencoder takes:
| build (rustc, opt-level 3) | cycles per training step | cycles per inference (forward pass and error) |
|---|---:|---:|
| generic CPU | 6,437 | 1,332 |
| `-C target-cpu=cortex-m4` | 8,811 (+37%) | 1,611 (+21%) |
| `-C target-cpu=cortex-m4 -C llvm-args=-enable-misched=false` | 6,415 | 1,350 |
The numbers come from [tinymind-rs](https://github.com/danmcleran/tinymind-rs), a `no_std` Rust library for neural networks on microcontrollers; the on-board benchmark is in its [`bench/`](https://github.com/danmcleran/tinymind-rs/tree/master/bench) directory (`anomaly` workload).
Turning the scheduler off is not a general workaround. In the same project it makes int8 code slower: an int8 MLP goes from 20,437 to 22,428 cycles (+9.7%) and an int8 CNN from 189,097 to 196,624 (+4.0%). The scheduler does help in general; its register pressure handling just does not seem to cope with this pattern under `CortexM4Model`.
## Expected
`-mcpu=cortex-m4` should not spill where the generic schedule needs no stack at all. The pattern is common in fixed-point DSP and ML code on Cortex-M: wide products narrowed and accumulated, small loops that fully unroll.
## Versions
- clang trunk (Compiler Explorer, 2026-11-11 and 2026-11-12)
- clang / llc 18.1.3 (Ubuntu)
- rustc 1.99.1 with LLVM 23.1.1
<details>
<summary><code>repro.ll</code></summary>
```llvm
; dense(): 8x4 Q16.16 dense layer, fully unrolled (from rustc 1.99.1, LLVM 23.1.1, opt-level 3)
target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
target triple = "thumbv7em-unknown-none-eabihf"
; Function Attrs: mustprogress nofree norecurse nosync nounwind willreturn memory(argmem: readwrite)
define dso_local void @dense(ptr noalias nofree noundef readonly align 4 dereferenceable(128) %w, ptr noalias nofree noundef readonly align 4 dereferenceable(16) %bias, ptr noalias nofree noundef readonly align 4 dereferenceable(32) %x, ptr noalias nofree noundef writeonly align 4 dereferenceable(16) %y) unnamed_addr #0 {
start:
%_12 = load i32, ptr %w, align 4
%_15 = load i32, ptr %x, align 4
%_29 = sext i32 %_12 to i64
%_30 = sext i32 %_15 to i64
%_28 = mul nsw i64 %_30, %_29
%_27 = lshr i64 %_28, 16
%_11 = trunc i64 %_27 to i32
%0 = getelementptr inbounds i8, ptr %w, i32 4
%_12.1 = load i32, ptr %0, align 4
%1 = getelementptr inbounds i8, ptr %x, i32 4
%_15.1 = load i32, ptr %1, align 4
%_29.1 = sext i32 %_12.1 to i64
%_30.1 = sext i32 %_15.1 to i64
%_28.1 = mul nsw i64 %_30.1, %_29.1
%_27.1 = lshr i64 %_28.1, 16
%_11.1 = trunc i64 %_27.1 to i32
%2 = add i32 %_11, %_11.1
%3 = getelementptr inbounds i8, ptr %w, i32 8
%_12.2 = load i32, ptr %3, align 4
%4 = getelementptr inbounds i8, ptr %x, i32 8
%_15.2 = load i32, ptr %4, align 4
%_29.2 = sext i32 %_12.2 to i64
%_30.2 = sext i32 %_15.2 to i64
%_28.2 = mul nsw i64 %_30.2, %_29.2
%_27.2 = lshr i64 %_28.2, 16
%_11.2 = trunc i64 %_27.2 to i32
%5 = add i32 %2, %_11.2
%6 = getelementptr inbounds i8, ptr %w, i32 12
%_12.3 = load i32, ptr %6, align 4
%7 = getelementptr inbounds i8, ptr %x, i32 12
%_15.3 = load i32, ptr %7, align 4
%_29.3 = sext i32 %_12.3 to i64
%_30.3 = sext i32 %_15.3 to i64
%_28.3 = mul nsw i64 %_30.3, %_29.3
%_27.3 = lshr i64 %_28.3, 16
%_11.3 = trunc i64 %_27.3 to i32
%8 = add i32 %5, %_11.3
%9 = getelementptr inbounds i8, ptr %w, i32 16
%_12.4 = load i32, ptr %9, align 4
%10 = getelementptr inbounds i8, ptr %x, i32 16
%_15.4 = load i32, ptr %10, align 4
%_29.4 = sext i32 %_12.4 to i64
%_30.4 = sext i32 %_15.4 to i64
%_28.4 = mul nsw i64 %_30.4, %_29.4
%_27.4 = lshr i64 %_28.4, 16
%_11.4 = trunc i64 %_27.4 to i32
%11 = add i32 %8, %_11.4
%12 = getelementptr inbounds i8, ptr %w, i32 20
%_12.5 = load i32, ptr %12, align 4
%13 = getelementptr inbounds i8, ptr %x, i32 20
%_15.5 = load i32, ptr %13, align 4
%_29.5 = sext i32 %_12.5 to i64
%_30.5 = sext i32 %_15.5 to i64
%_28.5 = mul nsw i64 %_30.5, %_29.5
%_27.5 = lshr i64 %_28.5, 16
%_11.5 = trunc i64 %_27.5 to i32
%14 = add i32 %11, %_11.5
%15 = getelementptr inbounds i8, ptr %w, i32 24
%_12.6 = load i32, ptr %15, align 4
%16 = getelementptr inbounds i8, ptr %x, i32 24
%_15.6 = load i32, ptr %16, align 4
%_29.6 = sext i32 %_12.6 to i64
%_30.6 = sext i32 %_15.6 to i64
%_28.6 = mul nsw i64 %_30.6, %_29.6
%_27.6 = lshr i64 %_28.6, 16
%_11.6 = trunc i64 %_27.6 to i32
%17 = add i32 %14, %_11.6
%18 = getelementptr inbounds i8, ptr %w, i32 28
%_12.7 = load i32, ptr %18, align 4
%19 = getelementptr inbounds i8, ptr %x, i32 28
%_15.7 = load i32, ptr %19, align 4
%_29.7 = sext i32 %_12.7 to i64
%_30.7 = sext i32 %_15.7 to i64
%_28.7 = mul nsw i64 %_30.7, %_29.7
%_27.7 = lshr i64 %_28.7, 16
%_11.7 = trunc i64 %_27.7 to i32
%20 = add i32 %17, %_11.7
%_17 = load i32, ptr %bias, align 4
%_16 = add i32 %_17, %20
store i32 %_16, ptr %y, align 4
%21 = getelementptr inbounds i8, ptr %w, i32 32
%_12.112 = load i32, ptr %21, align 4
%_29.114 = sext i32 %_12.112 to i64
%_28.116 = mul nsw i64 %_30, %_29.114
%_27.117 = lshr i64 %_28.116, 16
%_11.118 = trunc i64 %_27.117 to i32
%22 = getelementptr inbounds i8, ptr %w, i32 36
%_12.1.1 = load i32, ptr %22, align 4
%_29.1.1 = sext i32 %_12.1.1 to i64
%_28.1.1 = mul nsw i64 %_30.1, %_29.1.1
%_27.1.1 = lshr i64 %_28.1.1, 16
%_11.1.1 = trunc i64 %_27.1.1 to i32
%23 = add i32 %_11.118, %_11.1.1
%24 = getelementptr inbounds i8, ptr %w, i32 40
%_12.2.1 = load i32, ptr %24, align 4
%_29.2.1 = sext i32 %_12.2.1 to i64
%_28.2.1 = mul nsw i64 %_30.2, %_29.2.1
%_27.2.1 = lshr i64 %_28.2.1, 16
%_11.2.1 = trunc i64 %_27.2.1 to i32
%25 = add i32 %23, %_11.2.1
%26 = getelementptr inbounds i8, ptr %w, i32 44
%_12.3.1 = load i32, ptr %26, align 4
%_29.3.1 = sext i32 %_12.3.1 to i64
%_28.3.1 = mul nsw i64 %_30.3, %_29.3.1
%_27.3.1 = lshr i64 %_28.3.1, 16
%_11.3.1 = trunc i64 %_27.3.1 to i32
%27 = add i32 %25, %_11.3.1
%28 = getelementptr inbounds i8, ptr %w, i32 48
%_12.4.1 = load i32, ptr %28, align 4
%_29.4.1 = sext i32 %_12.4.1 to i64
%_28.4.1 = mul nsw i64 %_30.4, %_29.4.1
%_27.4.1 = lshr i64 %_28.4.1, 16
%_11.4.1 = trunc i64 %_27.4.1 to i32
%29 = add i32 %27, %_11.4.1
%30 = getelementptr inbounds i8, ptr %w, i32 52
%_12.5.1 = load i32, ptr %30, align 4
%_29.5.1 = sext i32 %_12.5.1 to i64
%_28.5.1 = mul nsw i64 %_30.5, %_29.5.1
%_27.5.1 = lshr i64 %_28.5.1, 16
%_11.5.1 = trunc i64 %_27.5.1 to i32
%31 = add i32 %29, %_11.5.1
%32 = getelementptr inbounds i8, ptr %w, i32 56
%_12.6.1 = load i32, ptr %32, align 4
%_29.6.1 = sext i32 %_12.6.1 to i64
%_28.6.1 = mul nsw i64 %_30.6, %_29.6.1
%_27.6.1 = lshr i64 %_28.6.1, 16
%_11.6.1 = trunc i64 %_27.6.1 to i32
%33 = add i32 %31, %_11.6.1
%34 = getelementptr inbounds i8, ptr %w, i32 60
%_12.7.1 = load i32, ptr %34, align 4
%_29.7.1 = sext i32 %_12.7.1 to i64
%_28.7.1 = mul nsw i64 %_30.7, %_29.7.1
%_27.7.1 = lshr i64 %_28.7.1, 16
%_11.7.1 = trunc i64 %_27.7.1 to i32
%35 = add i32 %33, %_11.7.1
%36 = getelementptr inbounds i8, ptr %bias, i32 4
%_17.1 = load i32, ptr %36, align 4
%_16.1 = add i32 %_17.1, %35
%37 = getelementptr inbounds i8, ptr %y, i32 4
store i32 %_16.1, ptr %37, align 4
%38 = getelementptr inbounds i8, ptr %w, i32 64
%_12.219 = load i32, ptr %38, align 4
%_15.220 = load i32, ptr %x, align 4
%_29.221 = sext i32 %_12.219 to i64
%_30.222 = sext i32 %_15.220 to i64
%_28.223 = mul nsw i64 %_30.222, %_29.221
%_27.224 = lshr i64 %_28.223, 16
%_11.225 = trunc i64 %_27.224 to i32
%39 = getelementptr inbounds i8, ptr %w, i32 68
%_12.1.2 = load i32, ptr %39, align 4
%_15.1.2 = load i32, ptr %1, align 4
%_29.1.2 = sext i32 %_12.1.2 to i64
%_30.1.2 = sext i32 %_15.1.2 to i64
%_28.1.2 = mul nsw i64 %_30.1.2, %_29.1.2
%_27.1.2 = lshr i64 %_28.1.2, 16
%_11.1.2 = trunc i64 %_27.1.2 to i32
%40 = add i32 %_11.225, %_11.1.2
%41 = getelementptr inbounds i8, ptr %w, i32 72
%_12.2.2 = load i32, ptr %41, align 4
%_15.2.2 = load i32, ptr %4, align 4
%_29.2.2 = sext i32 %_12.2.2 to i64
%_30.2.2 = sext i32 %_15.2.2 to i64
%_28.2.2 = mul nsw i64 %_30.2.2, %_29.2.2
%_27.2.2 = lshr i64 %_28.2.2, 16
%_11.2.2 = trunc i64 %_27.2.2 to i32
%42 = add i32 %40, %_11.2.2
%43 = getelementptr inbounds i8, ptr %w, i32 76
%_12.3.2 = load i32, ptr %43, align 4
%_15.3.2 = load i32, ptr %7, align 4
%_29.3.2 = sext i32 %_12.3.2 to i64
%_30.3.2 = sext i32 %_15.3.2 to i64
%_28.3.2 = mul nsw i64 %_30.3.2, %_29.3.2
%_27.3.2 = lshr i64 %_28.3.2, 16
%_11.3.2 = trunc i64 %_27.3.2 to i32
%44 = add i32 %42, %_11.3.2
%45 = getelementptr inbounds i8, ptr %w, i32 80
%_12.4.2 = load i32, ptr %45, align 4
%_15.4.2 = load i32, ptr %10, align 4
%_29.4.2 = sext i32 %_12.4.2 to i64
%_30.4.2 = sext i32 %_15.4.2 to i64
%_28.4.2 = mul nsw i64 %_30.4.2, %_29.4.2
%_27.4.2 = lshr i64 %_28.4.2, 16
%_11.4.2 = trunc i64 %_27.4.2 to i32
%46 = add i32 %44, %_11.4.2
%47 = getelementptr inbounds i8, ptr %w, i32 84
%_12.5.2 = load i32, ptr %47, align 4
%_15.5.2 = load i32, ptr %13, align 4
%_2<truncated>Please see the issue for the entire body.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs