This series teaches pass_loop_jam to recognize the GEMV / NNUE inference
shape and synthesize N parallel reduction streams across N adjacent
outer iterations, so the vectorizer can emit N independent dot-product /
FMA accumulators that hide the loop-carried op latency.
Patch breakdown:
1/2 unroll-and-jam: Synthesize parallel reduction streams for inner
reductions
The feature, the new option / params / target hook, the AArch64
opt-in, and five aarch64 testcases.
2/2 unroll-and-jam: Use compute_jam_dependence_factors in
tree_loop_unroll_and_jam
NFC cleanup retrofitting the pre-existing path onto the shared
data-dep helper introduced in patch 1.
~8% speedup on SPEC CPU 2026 706.stockfish_r on AArch64.
Bootstrapped and regression-tested on aarch64-unknown-linux-gnu and
x86_64-linux-gnu.
Konstantinos Eleftheriou (2):
unroll-and-jam: Synthesize parallel reduction streams for inner
reductions
unroll-and-jam: Use compute_jam_dependence_factors in
tree_loop_unroll_and_jam
gcc/common.opt | 5 +
gcc/config/aarch64/aarch64-protos.h | 8 +
gcc/config/aarch64/aarch64.cc | 29 +
gcc/config/aarch64/tuning_models/a64fx.h | 3 +-
gcc/config/aarch64/tuning_models/ampere1.h | 3 +-
gcc/config/aarch64/tuning_models/ampere1a.h | 3 +-
gcc/config/aarch64/tuning_models/ampere1b.h | 3 +-
gcc/config/aarch64/tuning_models/cortexa320.h | 3 +-
gcc/config/aarch64/tuning_models/cortexa35.h | 3 +-
gcc/config/aarch64/tuning_models/cortexa53.h | 3 +-
gcc/config/aarch64/tuning_models/cortexa57.h | 3 +-
gcc/config/aarch64/tuning_models/cortexa72.h | 3 +-
gcc/config/aarch64/tuning_models/cortexa73.h | 3 +-
gcc/config/aarch64/tuning_models/cortexx925.h | 3 +-
gcc/config/aarch64/tuning_models/emag.h | 3 +-
gcc/config/aarch64/tuning_models/exynosm1.h | 3 +-
.../aarch64/tuning_models/fujitsu_monaka.h | 3 +-
gcc/config/aarch64/tuning_models/generic.h | 3 +-
.../aarch64/tuning_models/generic_armv8_a.h | 3 +-
.../aarch64/tuning_models/generic_armv9_a.h | 3 +-
gcc/config/aarch64/tuning_models/hip12.h | 3 +-
.../aarch64/tuning_models/neoverse512tvb.h | 3 +-
gcc/config/aarch64/tuning_models/neoversen1.h | 3 +-
gcc/config/aarch64/tuning_models/neoversen2.h | 3 +-
gcc/config/aarch64/tuning_models/neoversen3.h | 3 +-
gcc/config/aarch64/tuning_models/neoversev1.h | 3 +-
gcc/config/aarch64/tuning_models/neoversev2.h | 3 +-
gcc/config/aarch64/tuning_models/neoversev3.h | 3 +-
.../aarch64/tuning_models/neoversev3ae.h | 3 +-
gcc/config/aarch64/tuning_models/olympus.h | 3 +-
gcc/config/aarch64/tuning_models/qdf24xx.h | 3 +-
gcc/config/aarch64/tuning_models/saphira.h | 3 +-
gcc/config/aarch64/tuning_models/thunderx.h | 3 +-
.../aarch64/tuning_models/thunderx2t99.h | 3 +-
.../aarch64/tuning_models/thunderx3t110.h | 3 +-
.../aarch64/tuning_models/thunderxt88.h | 3 +-
gcc/config/aarch64/tuning_models/tsv110.h | 3 +-
gcc/config/aarch64/tuning_models/xgene1.h | 3 +-
gcc/doc/invoke.texi | 17 +-
gcc/doc/tm.texi | 11 +
gcc/doc/tm.texi.in | 2 +
gcc/gimple-loop-jam.cc | 1020 +++++++++++++++--
gcc/hooks.cc | 7 +
gcc/hooks.h | 1 +
gcc/params.opt | 8 +
gcc/target.def | 13 +
.../aarch64/unroll-jam-reductions-1.c | 78 ++
.../aarch64/unroll-jam-reductions-2.c | 89 ++
.../aarch64/unroll-jam-reductions-3.c | 189 +++
.../aarch64/unroll-jam-reductions-4.c | 30 +
.../aarch64/unroll-jam-reductions-5.c | 72 ++
51 files changed, 1564 insertions(+), 120 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/aarch64/unroll-jam-reductions-1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/unroll-jam-reductions-2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/unroll-jam-reductions-3.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/unroll-jam-reductions-4.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/unroll-jam-reductions-5.c
--
2.55.0
