The arm backend implemented neither of the register-allocator cost hooks
added by r15-7895-gb191e8bdecf, so IRA charged a full store and load for
every callee-saved register and spilled values across calls rather than
keep them in a callee-saved register. ARM fits the hooks as aarch64
does: one push/pop (STM/LDM) or vpush/vpop takes a register list of
arbitrary length, so once a save block exists a further register only
lengthens that list and costs the words transferred. Thumb-1 is the
exception -- only r0-r7 and LR may be named in a PUSH or POP, so a high
callee-save is copied through a low one -- and keeps the default form.
A spill slot instead needs its own sub sp, add sp pair. That is priced
as a core-to-core register move, as aarch64 prices the same pair with
regmove_cost->GP2GP, and at zero when optimizing for size, where
HONOR_REG_ALLOC_ORDER hands the choice of register to REG_ALLOC_ORDER
rather than to IRA's cost model. Neither number is a literal: both come
from arm_register_move_cost, the frame one move and a new push/pop block,
which moves SP as it transfers, half of one.
Measured on the Raspberry Pi Linux tree (github.com/raspberrypi/linux at
f5a99b953), vmlinux .text, each arm an arm-none-eabi cross built from
this base and driven through a bare xgcc wrapper that injects no flags,
with SOURCE_DATE_EPOCH and KBUILD_BUILD_TIMESTAMP/USER/HOST pinned:
bcmrpi_defconfig ARMv6 -O2 10880364 -> 10683796 -196568 B (-1.81%)
bcm2709_defconfig ARMv7 -O2 11397064 -> 11183780 -213284 B (-1.87%)
bcmrpi_defconfig ARMv6 -Os 8709752 -> 8709496 -256 B (-0.00%)
This is not a never-grow change at -O2: over the 1621
gcc.c-torture/compile objects, -marm -mcpu=arm1176jzf-s gives 81 smaller
and 9 larger. At -Os both hooks are inert and all 1621 are byte
identical to an unpatched compiler; the -Os row above moves only because
lib/lz4 is built -O3.
On gcc.c-torture/execute (1663 tests, exact dynamic instruction counts
under qemu) ARMv6 runs 0.96% fewer instructions in total, 9 tests more and
56 fewer; Cortex-M33 is flat, 19 more and 61 fewer. Almost every increase
is one instruction, one more register in an existing push and pop. On an
arm1176 the cycle counter agrees or better: strcmp-1 -0.6%, 981001-1
-8.0%, pr69691 -8.9%, and 20040709-1, the one test with more instructions,
-4.0%; the noise floor from a byte-identical control is 0.3%.
Bootstrapped on x86_64-pc-linux-gnu. Regression tested on arm-none-eabi
against a pristine compiler built from the same base: gcc.target/arm,
gcc.c-torture/compile.exp, and gcc.c-torture/execute.exp executed under
qemu in two configurations (-marm -march=armv6zk -mtune=arm1176jzf-s on
realview-eb-mpcore and -mthumb -mcpu=cortex-m4 on mps2-an386); no new
failures.
With the hooks in place gcc.target/arm/mve/dlstp-compile-asm-2.c test5
keeps d in r3 and n in r4 again, 74 -> 60 bytes, which is the body the
test expected before r15-1619; the relaxation from PR testsuite/116623
is reverted.
Assisted-by: Claude Opus 5 (Anthropic)
gcc/ChangeLog:
* config/arm/arm.cc (arm_block_save_exists): New function.
(arm_callee_save_cost): New function.
(arm_frame_allocation_cost): New function.
(TARGET_CALLEE_SAVE_COST): Define.
(TARGET_FRAME_ALLOCATION_COST): Define.
gcc/testsuite/ChangeLog:
* gcc.target/arm/callee-save-cost-1.c: New test.
* gcc.target/arm/callee-save-cost-mprofile-1.c: New test.
* gcc.target/arm/callee-save-cost-vfp-1.c: New test.
* gcc.target/arm/mve/dlstp-compile-asm-2.c (test5, test8): Expect the
register allocation from before r15-1619 again.
Signed-off-by: Dominic P <[email protected]>
---
v2: the Linaro CI FAIL in gcc.target/arm/mve/dlstp-compile-asm-2.c test5 is
mine. Reproduced on arm-none-eabi with the CI's body byte for byte;
mve.exp is otherwise unchanged. test5 now keeps d in r3 instead of copying
it to r6 and puts n in r4 rather than ip, so three loop subs shrink to
16-bit: 22 insns / 74 bytes -> 21 / 60, same push list, no spills either
way. test8 gets r4 back too, 48 -> 44 bytes. That is the codegen the test
expected before r15-1619, so this version reverts the relaxation from
PR testsuite/116623 rather than loosening it further (its optional
"mov (r[0-9]+), r3" line could never match without the mov). Say if you
would rather it accept both forms.
The arm.cc change is unchanged from v1.
gcc/config/arm/arm.cc | 80 +++++++++++++++++++
.../gcc.target/arm/callee-save-cost-1.c | 27 +++++++
.../arm/callee-save-cost-mprofile-1.c | 77 ++++++++++++++++++
.../gcc.target/arm/callee-save-cost-vfp-1.c | 45 +++++++++++
.../gcc.target/arm/mve/dlstp-compile-asm-2.c | 10 +--
5 files changed, 233 insertions(+), 6 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/arm/callee-save-cost-1.c
create mode 100644 gcc/testsuite/gcc.target/arm/callee-save-cost-mprofile-1.c
create mode 100644 gcc/testsuite/gcc.target/arm/callee-save-cost-vfp-1.c
diff --git a/gcc/config/arm/arm.cc b/gcc/config/arm/arm.cc
index 33b64995442..74f97b2cc2d 100644
--- a/gcc/config/arm/arm.cc
+++ b/gcc/config/arm/arm.cc
@@ -187,6 +187,9 @@ static int arm_insn_cost (rtx_insn *, bool);
static int arm_address_cost (rtx, machine_mode, addr_space_t, bool);
static int arm_register_move_cost (machine_mode, reg_class_t, reg_class_t);
static int arm_memory_move_cost (machine_mode, reg_class_t, bool);
+static int arm_callee_save_cost (spill_cost_type, unsigned int, machine_mode,
+ unsigned int, int, const HARD_REG_SET &, bool);
+static int arm_frame_allocation_cost (frame_cost_type, const HARD_REG_SET &);
static void emit_constant_insn (rtx cond, rtx pattern);
static rtx_insn *emit_set_insn (rtx, rtx);
static void arm_add_cfa_adjust_cfa_note (rtx, int, rtx, rtx);
@@ -474,6 +477,12 @@ static const scoped_attribute_specs *const
arm_attribute_table[] =
#undef TARGET_MEMORY_MOVE_COST
#define TARGET_MEMORY_MOVE_COST arm_memory_move_cost
+#undef TARGET_CALLEE_SAVE_COST
+#define TARGET_CALLEE_SAVE_COST arm_callee_save_cost
+
+#undef TARGET_FRAME_ALLOCATION_COST
+#define TARGET_FRAME_ALLOCATION_COST arm_frame_allocation_cost
+
#undef TARGET_ENCODE_SECTION_INFO
#define TARGET_ENCODE_SECTION_INFO arm_encode_section_info
@@ -12677,6 +12686,77 @@ arm_memory_move_cost (machine_mode mode, reg_class_t
rclass,
}
}
+/* Return true if SAVED, the set of callee-saved registers that the register
+ allocator has already committed to, contains a register that REGNO would
+ share a push/pop or vpush/vpop reglist with. */
+
+static bool
+arm_block_save_exists (const HARD_REG_SET &saved, unsigned int regno)
+{
+ unsigned int first, last;
+
+ if (regno < FIRST_VFP_REGNUM)
+ {
+ first = FIRST_LO_REGNUM;
+ last = TARGET_THUMB1 ? LAST_LO_REGNUM : LAST_HI_REGNUM;
+ }
+ else
+ {
+ first = D7_VFP_REGNUM + 1;
+ last = LAST_LO_VFP_REGNUM;
+ }
+
+ for (unsigned int r = first; r <= last; r++)
+ if (TEST_HARD_REG_BIT (saved, r))
+ return true;
+
+ return false;
+}
+
+/* Implement TARGET_CALLEE_SAVE_COST. */
+
+static int
+arm_callee_save_cost (spill_cost_type spill_type, unsigned int regno,
+ machine_mode mode, unsigned int nregs, int mem_cost,
+ const HARD_REG_SET &allocated_callee_regs,
+ bool existing_spills_p)
+{
+ bool core = regno < FIRST_VFP_REGNUM;
+ bool vfp = (TARGET_HARD_FLOAT
+ && IN_RANGE (regno, D7_VFP_REGNUM + 1, LAST_LO_VFP_REGNUM));
+
+ /* Only r0-r7 and LR may be named in a Thumb-1 PUSH or POP reglist. */
+ if (core && TARGET_THUMB1 && regno > LAST_LO_REGNUM)
+ core = false;
+
+ if (core || vfp)
+ {
+ if (arm_block_save_exists (allocated_callee_regs, regno)
+ || (core && (!crtl->is_leaf || df_regs_ever_live_p (LR_REGNUM))))
+ return nregs;
+
+ /* The push and the pop adjust SP as they transfer, so creating a block
+ costs less than the frame a spill slot needs. */
+ return mem_cost + arm_register_move_cost (word_mode, GENERAL_REGS,
+ GENERAL_REGS) / 2;
+ }
+
+ return default_callee_save_cost (spill_type, regno, mode, nregs, mem_cost,
+ allocated_callee_regs, existing_spills_p);
+}
+
+/* Implement TARGET_FRAME_ALLOCATION_COST. Price the sub sp, add sp that a
+ spill slot needs as a core register move. */
+
+static int
+arm_frame_allocation_cost (frame_cost_type, const HARD_REG_SET &)
+{
+ if (optimize_function_for_size_p (cfun))
+ return 0;
+
+ return arm_register_move_cost (word_mode, GENERAL_REGS, GENERAL_REGS);
+}
+
/* Vectorizer cost model implementation. */
/* Implement targetm.vectorize.builtin_vectorization_cost. */
diff --git a/gcc/testsuite/gcc.target/arm/callee-save-cost-1.c
b/gcc/testsuite/gcc.target/arm/callee-save-cost-1.c
new file mode 100644
index 00000000000..e4b3f60869f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/callee-save-cost-1.c
@@ -0,0 +1,27 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_arch_v7a_arm_ok } */
+/* { dg-options "-O2" } */
+/* { dg-add-options arm_arch_v7a_arm } */
+/* { dg-final { check-function-bodies "**" "" "" } } */
+
+extern int g (int);
+
+/* A and B stay in callee-saved registers across the call; nothing is spilled
+ and no frame is allocated. */
+/*
+** cross_call_live:
+** push\t{r4, r5, r6, lr}
+** mov\tr4, r0
+** add\tr0, r0, r1
+** mov\tr5, r1
+** bl\tg
+** add\tr0, r4, r0
+** sub\tr0, r0, r5
+** pop\t{r4, r5, r6, pc}
+*/
+int
+cross_call_live (int a, int b)
+{
+ int t = g (a + b);
+ return t + a - b;
+}
diff --git a/gcc/testsuite/gcc.target/arm/callee-save-cost-mprofile-1.c
b/gcc/testsuite/gcc.target/arm/callee-save-cost-mprofile-1.c
new file mode 100644
index 00000000000..f8866d8d2b4
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/callee-save-cost-mprofile-1.c
@@ -0,0 +1,77 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_arch_v7em_hard_ok } */
+/* { dg-options "-O2" } */
+/* { dg-add-options arm_arch_v7em_hard } */
+/* { dg-final { check-function-bodies "**" "" "" } } */
+
+extern float fwork (float);
+extern int iwork (int);
+
+/* A Thumb-2 PUSH and an M-profile VPUSH take an arbitrary reglist, so the
+ block model applies here too. */
+/*
+** cross_call_live_sf:
+** push\t{r3, lr}
+** vpush.64\t{d8, d9, d10}
+** vmov.f32\ts18, s1
+** vmov.f32\ts17, s2
+** vmov.f32\ts20, s3
+** vmov.f32\ts19, s4
+** vmov.f32\ts21, s0
+** bl\tfwork
+** vmov.f32\ts16, s0
+** vmov.f32\ts0, s18
+** bl\tfwork
+** vmov.f32\ts15, s0
+** vmov.f32\ts0, s17
+** vadd.f32\ts16, s16, s15
+** bl\tfwork
+** vadd.f32\ts0, s16, s0
+** vadd.f32\ts0, s0, s21
+** vadd.f32\ts0, s0, s18
+** vadd.f32\ts0, s0, s17
+** vadd.f32\ts0, s0, s20
+** vadd.f32\ts0, s0, s19
+** vldm\tsp!, {d8-d10}
+** pop\t{r3, pc}
+*/
+float
+cross_call_live_sf (float a, float b, float c, float d, float e)
+{
+ float r = fwork (a);
+ r += fwork (b);
+ r += fwork (c);
+ return r + a + b + c + d + e;
+}
+
+/*
+** cross_call_live_si:
+** push\t{r3, r4, r5, r6, r7, r8, r9, lr}
+** ldr\tr5, \[sp, #32\]
+** mov\tr8, r1
+** mov\tr6, r2
+** mov\tr7, r3
+** mov\tr9, r0
+** bl\tiwork
+** mov\tr4, r0
+** mov\tr0, r8
+** bl\tiwork
+** add\tr4, r4, r0
+** mov\tr0, r6
+** bl\tiwork
+** add\tr0, r0, r4
+** add\tr0, r0, r9
+** add\tr0, r0, r8
+** add\tr0, r0, r6
+** add\tr0, r0, r7
+** add\tr0, r0, r5
+** pop\t{r3, r4, r5, r6, r7, r8, r9, pc}
+*/
+int
+cross_call_live_si (int a, int b, int c, int d, int e)
+{
+ int r = iwork (a);
+ r += iwork (b);
+ r += iwork (c);
+ return r + a + b + c + d + e;
+}
diff --git a/gcc/testsuite/gcc.target/arm/callee-save-cost-vfp-1.c
b/gcc/testsuite/gcc.target/arm/callee-save-cost-vfp-1.c
new file mode 100644
index 00000000000..f57e18738a8
--- /dev/null
+++ b/gcc/testsuite/gcc.target/arm/callee-save-cost-vfp-1.c
@@ -0,0 +1,45 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target arm_arch_v7a_fp_hard_ok } */
+/* { dg-options "-O2 -marm" } */
+/* { dg-add-options arm_arch_v7a_fp_hard } */
+/* { dg-final { check-function-bodies "**" "" "" } } */
+
+extern double g (double);
+
+/* A DFmode callee-save is nregs == 2, but it joins the same vpush, so the
+ arguments live across the calls lengthen the reglist instead of being
+ spilled. */
+/*
+** cross_call_live_df:
+** push\t{r4, lr}
+** vpush.64\t{d8, d9, d10, d11, d12, d13}
+** vmov.f64\td10, d1
+** vmov.f64\td9, d2
+** vmov.f64\td13, d3
+** vmov.f64\td11, d4
+** vmov.f64\td12, d0
+** bl\tg
+** vmov.f64\td8, d0
+** vmov.f64\td0, d10
+** bl\tg
+** vmov.f64\td7, d0
+** vmov.f64\td0, d9
+** vadd.f64\td8, d8, d7
+** bl\tg
+** vadd.f64\td0, d8, d0
+** vadd.f64\td0, d0, d12
+** vadd.f64\td0, d0, d10
+** vadd.f64\td0, d0, d9
+** vadd.f64\td0, d0, d13
+** vadd.f64\td0, d0, d11
+** vldm\tsp!, {d8-d13}
+** pop\t{r4, pc}
+*/
+double
+cross_call_live_df (double a, double b, double c, double d, double e)
+{
+ double r = g (a);
+ r += g (b);
+ r += g (c);
+ return r + a + b + c + d + e;
+}
diff --git a/gcc/testsuite/gcc.target/arm/mve/dlstp-compile-asm-2.c
b/gcc/testsuite/gcc.target/arm/mve/dlstp-compile-asm-2.c
index 37c54f22abc..83a94a3290d 100644
--- a/gcc/testsuite/gcc.target/arm/mve/dlstp-compile-asm-2.c
+++ b/gcc/testsuite/gcc.target/arm/mve/dlstp-compile-asm-2.c
@@ -147,17 +147,15 @@ void test5 (uint8_t *a, uint8_t *b, uint8_t *c, uint8_t
*d, int n)
/*
** test5:
**...
-** (?:mov (r[0-9]+), r3)?
-**...
-** dlstp.8 lr, (?:r[0-9]+|ip)
+** dlstp.8 lr, r[0-9]+
**...
** vldrb.8 q[0-9]+, \[r1\]
** vldrb.8 q[0-9]+, \[r2\]
**...
** vadd.i8 (q[0-9]+), q[0-9]+, q[0-9]+
**...
-** vstrb.8 \2, \[r2\]
-** vstrb.8 \2, \[(r3|\1)\]
+** vstrb.8 \1, \[r2\]
+** vstrb.8 \1, \[r3\]
** letp lr, .*
**...
*/
@@ -254,7 +252,7 @@ void test8 (int32_t *a, int32_t *b, int32_t *c, int n, int
g)
**...
** dlstp.32 lr, r3
** vldrw.32 q[0-9]+, \[r0\], #16
-** vctp.32 (?:r4|ip)
+** vctp.32 r4
** vpst
** vldrwt.32 q[0-9]+, \[r1\], #16
**...
--
2.55.0
