https://gcc.gnu.org/bugzilla/show_bug.cgi?id=126376
Bug ID: 126376
Summary: aarch64: cunrolli unrolls small constant-trip inner
loops and blocks dot-product vectorization
Product: gcc
Version: 17.0
Status: UNCONFIRMED
Keywords: missed-optimization
Severity: normal
Priority: P3
Component: tree-optimization
Assignee: unassigned at gcc dot gnu.org
Reporter: konstantinos.eleftheriou at vrull dot eu
Target Milestone: ---
Target: aarch64-*-*
Created attachment 65118
--> https://gcc.gnu.org/bugzilla/attachment.cgi?id=65118&action=edit
reduced_testcase
The inner loop of a widening multiply-accumulate nest gets fully
unrolled by cunrolli before the vectorizer runs. Once it is
straight-line code there is no loop left for vect_recog_dot_prod_pattern
to match, so the vectorizer falls back to element-wise SLP on the outer
body: widening multiplies and uzp1/uzp2 shuffles instead of a dot
product.
For the attached testcase with `-O3 -mcpu=ampere1a -S reduced_testcase.c`
we get 466 lines of assembly, 144 uzp1/uzp2 instructions and no usdot.
The inner loop is already gone by the time the vectorizer runs.
Using `#pragma GCC unroll 0` for the inner loop fixes this. A usdot is emitted,
no uzp instructions and the assembly goes down to 39 lines.
The testcase is extracted from SPEC2026's 706.stockfish_r benchmark.
Suggested fix:
A small pass that runs just before cunrolli, spots these dot-product-shaped
inner loops and marks them so cunrolli leaves them alone. The loop then
reaches the vectorizer intact and the existing DOT_PROD recognizer takes it
from there.
