On Tue, Jun 23, 2026 at 2:12 PM Uros Bizjak <[email protected]> wrote: > > On Tue, Jun 23, 2026 at 6:42 AM H.J. Lu <[email protected]> wrote: > > > > On Mon, Jun 22, 2026 at 9:32 PM H.J. Lu <[email protected]> wrote: > > > > > > On Mon, Jun 22, 2026 at 7:50 PM Uros Bizjak <[email protected]> wrote: > > > > > > > > On Mon, Jun 22, 2026 at 1:29 PM Uros Bizjak <[email protected]> wrote: > > > > > > > > > > On Mon, Jun 22, 2026 at 8:33 AM H.J. Lu <[email protected]> wrote: > > > > > > > > > > > > On Mon, Jun 22, 2026 at 1:59 PM Uros Bizjak <[email protected]> > > > > > > wrote: > > > > > > > > > > > > > > -ENOPATCH > > > > > > > > > > > > Oops. Here it is. > > > > > > > > > > > > > On Sun, Jun 21, 2026 at 11:44 PM H.J. Lu <[email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > Since LCP stall peepholes are added after register allocation, > > > > > > > > each > > > > > > > > peephole may use a different scratch register. For input: > > > > > > > > > > > > > > > > extern void bar (void); > > > > > > > > > > > > > > > > void > > > > > > > > foo (short *dst) > > > > > > > > { > > > > > > > > dst[0] = 3; > > > > > > > > asm volatile ("" : : : "memory"); > > > > > > > > dst[2] = 3; > > > > > > > > bar (); > > > > > > > > dst[1] = 3; > > > > > > > > asm volatile ("" : : : "memory"); > > > > > > > > dst[4] = 3; > > > > > > > > } > > > > > > > > > > > > > > > > with LCP stall peepholes, GCC generates: > > > > > > > > > > > > > > > > movl $3, %eax > > > > > > > > pushq %rbx > > > > > > > > movq %rdi, %rbx > > > > > > > > movw %ax, (%rdi) > > > > > > > > movl $3, %edx > > > > > > > > movw %dx, 4(%rdi) > > > > > > > > call bar > > > > > > > > movl $3, %ecx > > > > > > > > movw %cx, 2(%rbx) > > > > > > > > movl $3, %esi > > > > > > > > movw %si, 8(%rbx) > > > > > > > > popq %rbx > > > > > > > > > > > > > > > > using 4 different scratch registers vs without LCP stall > > > > > > > > peepholes: > > > > > > > > > > > > > > > > pushq %rbx > > > > > > > > movq %rdi, %rbx > > > > > > > > movw $3, (%rdi) > > > > > > > > movw $3, 4(%rdi) > > > > > > > > call bar > > > > > > > > movw $3, 2(%rbx) > > > > > > > > movw $3, 8(%rbx) > > > > > > > > popq %rbx > > > > > > > > > > > > > > > > Add ix86_output_lcp_stall_peephole to generate LCP stall > > > > > > > > peepholes with > > > > > > > > the previous scratch register: > > > > > > > > > > > > > > > > 1. Scan backward for the previous scratch register definition > > > > > > > > with > > > > > > > > the same immediate operand in the same basic block. > > > > > > > > 2. The previous scratch register is unusable if it is set > > > > > > > > between the > > > > > > > > previous scratch register definition and the current > > > > > > > > instruction. > > > > > > > > 3. If a usable previous scratch register is found, ignore the > > > > > > > > allocated > > > > > > > > scratch register and use the previous scratch register. > > > > > > > > Otherwise, use > > > > > > > > the allocated scratch register. > > > > > > > > > > > > > > > > so that the same scratch register can be reused if possible: > > > > > > > > > > > > > > > > movl $3, %eax > > > > > > > > pushq %rbx > > > > > > > > movq %rdi, %rbx > > > > > > > > movw %ax, (%rdi) > > > > > > > > movw %ax, 4(%rdi) > > > > > > > > call bar > > > > > > > > movl $3, %ecx > > > > > > > > movw %cx, 2(%rbx) > > > > > > > > movw %cx, 8(%rbx) > > > > > > > > popq %rbx > > > > > > > > > > > > > > > > I backported this patch to GCC 16: > > > > > > > > > > > > > > > > 1. When bootstrapping GCC 16 with only C and C++ enabled, this > > > > > > > > optimization > > > > > > > > triggers 54 times. No regressions. > > > > > > > > 2. When building glibc 2.44, this optimization triggers 33 > > > > > > > > times. No > > > > > > > > regressions. > > > > > > > > 3. When building Linux kernel 7.1.1, this optimization triggers > > > > > > > > 2099 times. > > > > > > > > Kernel boots correctly. > > > > > > > > > > > > > > > > gcc/ > > > > > > > > > > > > > > > > PR target/125893 > > > > > > > > * config/i386/i386-protos.h (ix86_output_lcp_stall_peephole): > > > > > > > > New. > > > > > > > > * config/i386/i386.cc (ix86_output_lcp_stall_peephole): > > > > > > > > Likewise. > > > > > > > > * config/i386/i386.md (TARGET_LCP_STALL peepholes): Call > > > > > > > > ix86_output_lcp_stall_peephole. > > > > > > > > > > > > > > > > gcc/testsuite/ > > > > > > > > > > > > > > > > PR target/125893 > > > > > > > > * gcc.target/i386/pr125893-1.c: New test. > > > > > > > > * gcc.target/i386/pr125893-2.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-3.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-4.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-5.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-6.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-7.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-8.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-9.c: Likewise. > > > > > > > > * gcc.target/i386/pr125893-10.c: Likewise. > > > > > > > > > > > > > > > +/* Output LCP stall or long immediate peephole for INSN. Use the > > > > > + previous scratch register if possible. */ > > > > > + > > > > > +void > > > > > +ix86_output_lcp_stall_peephole (rtx_insn *insn, rtx *operands) > > > > > +{ > > > > > > > > > > Please name this function ix86_expand_lcp_stall_peephole. We are > > > > > expanding RTXes here, not outputting ASM. > > > > > > Will fix > > > > Done in the v2 patch. > > > > > > > > > > - [(parallel [(set (match_dup 2) (const_int 0)) > > > > > - (clobber (reg:CC FLAGS_REG))]) > > > > > - (set (match_dup 0) (match_dup 1))] > > > > > - "operands[2] = gen_lowpart (SImode, operands[1]);") > > > > > + [(const_int 0)] > > > > > +{ > > > > > + operands[2] = const0_rtx; > > > > > + ix86_output_lcp_stall_peephole (curr_insn, operands); > > > > > +}) > > > > > > > > > > Missing DONE; > > > > > > Will fix > > > > Done in the v2 patch. > > > > > > > > > > - [(set (match_dup 2) (match_dup 1)) > > > > > - (set (match_dup 0) (match_dup 2))]) > > > > > + [(const_int 0)] > > > > > + "ix86_output_lcp_stall_peephole (curr_insn, operands);") > > > > > > > > > > Also here. > > > > > > Will fix > > > > Done in the v2 patch. > > > > > > > > > > On a related note, the second peephole also allows const0_rtx, which > > > > > results in XOR form, which clobbers flags. Please better use some > > > > > boolean argument (e.g. use_xor) to the ix86_output_lcp_stall_peephole > > > > > and handle it accordingly in both peephole2 patterns. > > > > > > Will fix > > > > Done in the v2 patch. > > > > > > Another observation: You can't reuse %esi/%edi/%ebp/%esp as QImode > > > > scratch in 32-bit mode. So, the value, available in one of these > > > > registers, can not be reused as QImode value. Please re-test the patch > > > > > > Will fix. > > > > Added > > > > if (!TARGET_64BIT && mode == QImode) > > { > > /* SI_REG/DI_REG/BP_REG/SP_REG has no Qimode in > > 32-bit mode. */ > > unsigned int regno = REGNO (dest); > > if (regno >= SI_REG && regno <= SP_REG) > > continue; > > } > > if (mode == QImode && !ANY_QI_REGNO_P (REGNO (dest))) > continue; > > > > > together with a testcase, pr125893-11.c, extracted from i686 glibc build. > > > > > > also for 32-bit mode. > > > > > > > > Also tested with Linux/x86-64 and Linux/i686 glibc builds. > > > > -- > > H.J. > > --- > > Since LCP stall peepholes are added after register allocation, each > > peephole may use a different scratch register. For input: > > > > extern void bar (void); > > > > void > > foo (short *dst) > > { > > dst[0] = 3; > > asm volatile ("" : : : "memory"); > > dst[2] = 3; > > bar (); > > dst[1] = 3; > > asm volatile ("" : : : "memory"); > > dst[4] = 3; > > } > > > > with LCP stall peepholes, GCC generates: > > > > movl $3, %eax > > pushq %rbx > > movq %rdi, %rbx > > movw %ax, (%rdi) > > movl $3, %edx > > movw %dx, 4(%rdi) > > call bar > > movl $3, %ecx > > movw %cx, 2(%rbx) > > movl $3, %esi > > movw %si, 8(%rbx) > > popq %rbx > > > > using 4 different scratch registers vs without LCP stall peepholes: > > > > pushq %rbx > > movq %rdi, %rbx > > movw $3, (%rdi) > > movw $3, 4(%rdi) > > call bar > > movw $3, 2(%rbx) > > movw $3, 8(%rbx) > > popq %rbx > > > > Add ix86_output_lcp_stall_peephole to generate LCP stall peepholes with > > the previous scratch register: > > > > 1. Scan backward for the previous scratch register definition with > > the same immediate operand in the same basic block. > > 2. The previous scratch register is unusable if it is set between the > > previous scratch register definition and the current instruction. > > 3. If a usable previous scratch register is found, ignore the allocated > > scratch register and use the previous scratch register. Otherwise, use > > the allocated scratch register. > > > > so that the same scratch register can be reused if possible: > > > > movl $3, %eax > > pushq %rbx > > movq %rdi, %rbx > > movw %ax, (%rdi) > > movw %ax, 4(%rdi) > > call bar > > movl $3, %ecx > > movw %cx, 2(%rbx) > > movw %cx, 8(%rbx) > > popq %rbx > > > > I backported this patch to GCC 16: > > > > 1. When bootstrapping GCC 16 with only C and C++ enabled, this optimization > > triggers 54 times. No regressions. > > 2. When building glibc 2.44, this optimization triggers 33 times. No > > regressions. > > 3. When building Linux kernel 7.1.1, this optimization triggers 2099 times. > > Kernel boots correctly. > > > > Tested on Linux/x86-64 and Linux/i686. > > > > gcc/ > > > > PR target/125893 > > * config/i386/i386-expand.cc (ix86_expand_lcp_stall_peephole): New. > > * config/i386/i386-protos.h (ix86_expand_lcp_stall_peephole): > > Likewise. > > * config/i386/i386.md (TARGET_LCP_STALL peepholes): Call > > ix86_expand_lcp_stall_peephole. > > > > gcc/testsuite/ > > > > PR target/125893 > > * gcc.target/i386/pr125893-1.c: New test. > > * gcc.target/i386/pr125893-2.c: Likewise. > > * gcc.target/i386/pr125893-3.c: Likewise. > > * gcc.target/i386/pr125893-4.c: Likewise. > > * gcc.target/i386/pr125893-5.c: Likewise. > > * gcc.target/i386/pr125893-6.c: Likewise. > > * gcc.target/i386/pr125893-7.c: Likewise. > > * gcc.target/i386/pr125893-8.c: Likewise. > > * gcc.target/i386/pr125893-9.c: Likewise. > > * gcc.target/i386/pr125893-10.c: Likewise. > > * gcc.target/i386/pr125893-11.c: Likewise. > > + rtx dest = SET_DEST (set); > + /* The previous scratch register mode must be at least > + MODE. */ > + if (!REG_P (dest) > + || (GET_MODE_SIZE (GET_MODE (dest)) > + < GET_MODE_SIZE (mode))) > + continue; > > With improvement to QImode test, both tests can collapse to: > > rtx dest = SET_DEST (set); > > /* Reject DEST if a register is not wide enough to supply MODE > or invalid for QImode. */ > if (!REG_P (dest) > || GET_MODE_SIZE (GET_MODE (dest)) < GET_MODE_SIZE (mode) > || (mode == QImode && !ANY_QI_REGNO_P (REGNO (dest)))) > continue; > > OK with the above change. >
Changed. Here is the patch I am checking in. Thanks. -- H.J.
From 9a445f5c856aaba8be8f3f50af25735a17181e3b Mon Sep 17 00:00:00 2001 From: "H.J. Lu" <[email protected]> Date: Sun, 21 Jun 2026 07:13:38 +0800 Subject: [PATCH] x86: Use previous scratch register in LCP stall peepholes Since LCP stall peepholes are added after register allocation, each peephole may use a different scratch register. For input: extern void bar (void); void foo (short *dst) { dst[0] = 3; asm volatile ("" : : : "memory"); dst[2] = 3; bar (); dst[1] = 3; asm volatile ("" : : : "memory"); dst[4] = 3; } with LCP stall peepholes, GCC generates: movl $3, %eax pushq %rbx movq %rdi, %rbx movw %ax, (%rdi) movl $3, %edx movw %dx, 4(%rdi) call bar movl $3, %ecx movw %cx, 2(%rbx) movl $3, %esi movw %si, 8(%rbx) popq %rbx using 4 different scratch registers vs without LCP stall peepholes: pushq %rbx movq %rdi, %rbx movw $3, (%rdi) movw $3, 4(%rdi) call bar movw $3, 2(%rbx) movw $3, 8(%rbx) popq %rbx Add ix86_output_lcp_stall_peephole to generate LCP stall peepholes with the previous scratch register: 1. Scan backward for the previous scratch register definition with the same immediate operand in the same basic block. 2. The previous scratch register is unusable if it is set between the previous scratch register definition and the current instruction. 3. If a usable previous scratch register is found, ignore the allocated scratch register and use the previous scratch register. Otherwise, use the allocated scratch register. so that the same scratch register can be reused if possible: movl $3, %eax pushq %rbx movq %rdi, %rbx movw %ax, (%rdi) movw %ax, 4(%rdi) call bar movl $3, %ecx movw %cx, 2(%rbx) movw %cx, 8(%rbx) popq %rbx I backported this patch to GCC 16: 1. When bootstrapping GCC 16 with only C and C++ enabled, this optimization triggers 54 times. No regressions. 2. When building glibc 2.44, this optimization triggers 33 times. No regressions. 3. When building Linux kernel 7.1.1, this optimization triggers 2099 times. Kernel boots correctly. Tested on Linux/x86-64 and Linux/i686. gcc/ PR target/125893 * config/i386/i386-expand.cc (ix86_expand_lcp_stall_peephole): New. * config/i386/i386-protos.h (ix86_expand_lcp_stall_peephole): Likewise. * config/i386/i386.md (TARGET_LCP_STALL peepholes): Call ix86_expand_lcp_stall_peephole. gcc/testsuite/ PR target/125893 * gcc.target/i386/pr125893-1.c: New test. * gcc.target/i386/pr125893-2.c: Likewise. * gcc.target/i386/pr125893-3.c: Likewise. * gcc.target/i386/pr125893-4.c: Likewise. * gcc.target/i386/pr125893-5.c: Likewise. * gcc.target/i386/pr125893-6.c: Likewise. * gcc.target/i386/pr125893-7.c: Likewise. * gcc.target/i386/pr125893-8.c: Likewise. * gcc.target/i386/pr125893-9.c: Likewise. * gcc.target/i386/pr125893-10.c: Likewise. * gcc.target/i386/pr125893-11.c: Likewise. Signed-off-by: H.J. Lu <[email protected]> --- gcc/config/i386/i386-expand.cc | 159 ++++++++++++++++++++ gcc/config/i386/i386-protos.h | 2 + gcc/config/i386/i386.md | 16 +- gcc/testsuite/gcc.target/i386/pr125893-1.c | 27 ++++ gcc/testsuite/gcc.target/i386/pr125893-10.c | 38 +++++ gcc/testsuite/gcc.target/i386/pr125893-11.c | 99 ++++++++++++ gcc/testsuite/gcc.target/i386/pr125893-2.c | 33 ++++ gcc/testsuite/gcc.target/i386/pr125893-3.c | 27 ++++ gcc/testsuite/gcc.target/i386/pr125893-4.c | 33 ++++ gcc/testsuite/gcc.target/i386/pr125893-5.c | 34 +++++ gcc/testsuite/gcc.target/i386/pr125893-6.c | 34 +++++ gcc/testsuite/gcc.target/i386/pr125893-7.c | 31 ++++ gcc/testsuite/gcc.target/i386/pr125893-8.c | 38 +++++ gcc/testsuite/gcc.target/i386/pr125893-9.c | 31 ++++ 14 files changed, 596 insertions(+), 6 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-1.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-10.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-11.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-2.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-3.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-4.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-5.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-6.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-7.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-8.c create mode 100644 gcc/testsuite/gcc.target/i386/pr125893-9.c diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc index f422f65fbc5..429f00f03e9 100644 --- a/gcc/config/i386/i386-expand.cc +++ b/gcc/config/i386/i386-expand.cc @@ -28264,4 +28264,163 @@ ix86_expand_gfni_bitreverse (rtx dest, rtx src) emit_insn (gen_rtx_SET (dest, gen_rtx_BSWAP (mode, target))); } +/* Expand LCP stall or long immediate peephole for INSN. Use the + previous scratch register if possible. If USE_XOR is true, + generate "*movsi_xor". */ + +void +ix86_expand_lcp_stall_peephole (rtx_insn *insn, rtx *operands, + bool use_xor) +{ + rtx imm, scratch; + machine_mode mode = GET_MODE (operands[0]); + + /* Get the immediate operand and the allocated scratch register. */ + if (use_xor) + { + imm = const0_rtx; + scratch = operands[1]; + } + else + { + imm = operands[1]; + scratch = operands[2]; + } + + df_ref def; + rtx_insn *prev, *prev_insn = nullptr; + rtx set, prev_scratch = nullptr; + + /* Scan backward for the previous scratch register def with IMM in + the same basic block. */ + basic_block bb = BLOCK_FOR_INSN (insn); + for (prev = PREV_INSN (insn); + prev != BB_HEAD (bb); + prev = PREV_INSN (prev)) + { + if (NONDEBUG_INSN_P (prev)) + FOR_EACH_INSN_DEF (def, prev) + if (!DF_REF_IS_ARTIFICIAL (def) + && !DF_REF_FLAGS_IS_SET (def, DF_REF_MAY_CLOBBER) + && !DF_REF_FLAGS_IS_SET (def, DF_REF_MUST_CLOBBER)) + { + rtx reg = DF_REF_REG (def); + if (HARD_REGISTER_P (reg) && REGNO (reg) != FLAGS_REG) + { + set = single_set (prev); + if (!set) + continue; + + rtx dest = SET_DEST (set); + + /* Reject DEST if a register is not wide enough to + supply MODE or invalid for QImode. */ + if (!REG_P (dest) + || (GET_MODE_SIZE (GET_MODE (dest)) + < GET_MODE_SIZE (mode)) + || (mode == QImode + && !ANY_QI_REGNO_P (REGNO (dest)))) + continue; + + rtx src = SET_SRC (set); + if (rtx_equal_p (src, imm)) + { + /* A previous scratch register is found. */ + prev_scratch = dest; + prev_insn = prev; + break; + } + } + } + + if (prev_scratch) + break; + } + + if (prev_scratch) + { + /* The previous scratch register is unusable if it is set between + PREV_INSN and INSN. */ + unsigned int regno = REGNO (prev_scratch); + + /* Scan backward for the previous scratch register def. */ + for (prev = PREV_INSN (insn); + prev != prev_insn; + prev = PREV_INSN (prev)) + { + if (NONDEBUG_INSN_P (prev)) + FOR_EACH_INSN_DEF (def, prev) + if (HARD_REGISTER_P (DF_REF_REAL_REG (def)) + && DF_REF_REGNO (def) == regno) + { + /* Since the previous scratch register is set, it is + unusable. */ + if (dump_file) + { + fprintf (dump_file, + "\nThe previous scratch register:\n\n"); + print_rtl_single (dump_file, prev_scratch); + fprintf (dump_file, "\nset in:\n\n"); + print_rtl_single (dump_file, prev_insn); + fprintf (dump_file, + "\nis unusable by:\n\n"); + print_rtl_single (dump_file, insn); + fprintf (dump_file, + "\nsince it is overridden by:\n\n"); + print_rtl_single (dump_file, prev); + fprintf (dump_file, "\n"); + } + prev_scratch = nullptr; + break; + } + + if (!prev_scratch) + break; + } + + if (prev_scratch) + { + /* Ignore the allocated scratch register and use the previous + scratch register. */ + if (dump_file) + { + fprintf (dump_file, + "\nIgnore the allocated scratch register:\n\n"); + print_rtl_single (dump_file, scratch); + fprintf (dump_file, + "\nand use the previous scratch register:\n\n"); + print_rtl_single (dump_file, prev_scratch); + fprintf (dump_file, "\nset in:\n\n"); + print_rtl_single (dump_file, prev_insn); + fprintf (dump_file, "\nfor:\n\n"); + print_rtl_single (dump_file, insn); + fprintf (dump_file, "\n"); + } + scratch = gen_lowpart (mode, prev_scratch); + set = gen_rtx_SET (operands[0], scratch); + emit_insn (set); + return; + } + } + + /* If there is no usable previous scratch register, use the allocated + scratch register. */ + if (use_xor) + { + /* Generate "*movsi_xor". */ + rtx x = gen_lowpart (SImode, scratch); + rtvec p = rtvec_alloc (2); + set = gen_rtx_SET (x, const0_rtx); + RTVEC_ELT (p, 0) = set; + rtx clobber = gen_rtx_REG (CCmode, FLAGS_REG); + RTVEC_ELT (p, 1) = gen_rtx_CLOBBER (VOIDmode, clobber); + set = gen_rtx_PARALLEL (VOIDmode, p); + } + else + set = gen_rtx_SET (scratch, imm); + emit_insn (set); + set = gen_rtx_SET (operands[0], scratch); + emit_insn (set); +} + #include "gt-i386-expand.h" diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h index 106a79dd41e..15d71debbf5 100644 --- a/gcc/config/i386/i386-protos.h +++ b/gcc/config/i386/i386-protos.h @@ -314,6 +314,8 @@ extern bool ix86_expand_vector_init_one_nonzero (bool, machine_mode, rtx, rtx, int); extern bool ix86_extract_perm_from_pool_constant (int*, rtx); +extern void ix86_expand_lcp_stall_peephole (rtx_insn *, rtx *, bool); + /* In i386-c.cc */ extern void ix86_target_macros (void); extern void ix86_register_pragmas (void); diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md index 8db5d3b0152..45c27218293 100644 --- a/gcc/config/i386/i386.md +++ b/gcc/config/i386/i386.md @@ -29004,10 +29004,11 @@ (define_peephole2 && TARGET_SPLIT_LONG_MOVES && get_attr_length (insn) >= ix86_cur_cost ()->large_insn)) && peep2_regno_dead_p (0, FLAGS_REG)" - [(parallel [(set (match_dup 2) (const_int 0)) - (clobber (reg:CC FLAGS_REG))]) - (set (match_dup 0) (match_dup 1))] - "operands[2] = gen_lowpart (SImode, operands[1]);") + [(const_int 0)] +{ + ix86_expand_lcp_stall_peephole (curr_insn, operands, true); + DONE; +}) (define_peephole2 [(match_scratch:SWI124 2 "<r>") @@ -29018,8 +29019,11 @@ (define_peephole2 && TARGET_LCP_STALL) || (TARGET_SPLIT_LONG_MOVES && get_attr_length (insn) >= ix86_cur_cost ()->large_insn))" - [(set (match_dup 2) (match_dup 1)) - (set (match_dup 0) (match_dup 2))]) + [(const_int 0)] +{ + ix86_expand_lcp_stall_peephole (curr_insn, operands, false); + DONE; +}) ;; Don't compare memory with zero, load and use a test instead. (define_peephole2 diff --git a/gcc/testsuite/gcc.target/i386/pr125893-1.c b/gcc/testsuite/gcc.target/i386/pr125893-1.c new file mode 100644 index 00000000000..22749363c6b --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-1.c @@ -0,0 +1,27 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mtune-ctrl=lcp_stall" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** movl \$3, %[a-z0-9]+ +** movw %[a-z0-9]+, \(%[a-z0-9]+\) +** movw %[a-z0-9]+, 2\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 4\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 6\(%[a-z0-9]+\) +**... +*/ + +void +foo (short *dst) +{ + dst[0] = 3; + asm volatile ("" : : : "memory"); + dst[1] = 3; + asm volatile ("" : : : "memory"); + dst[2] = 3; + asm volatile ("" : : : "memory"); + dst[3] = 3; +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-10.c b/gcc/testsuite/gcc.target/i386/pr125893-10.c new file mode 100644 index 00000000000..5ad38f14092 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-10.c @@ -0,0 +1,38 @@ +/* { dg-do compile { target ia32 } } */ +/* { dg-options "-O2 -fomit-frame-pointer -momit-leaf-frame-pointer -mtune=pentiumpro" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** xorl %eax, %eax +**... +** movl %eax, 24\(%esp\) +** movl %eax, 28\(%esp\) +** leal 24\(%esp\), %ebx +** movl %ebx, \(%esp\) +** call bar +** xorl %ecx, %ecx +** movl %ecx, 32\(%esp\) +** movl %ebx, \(%esp\) +** movl %ecx, 36\(%esp\) +** call bar +**... +*/ + +extern void bar (int *dst); + +void +foo (void) +{ + int dst[10]; + dst[0] = 0; + asm volatile ("" : : : "memory"); + dst[1] = 0; + bar (dst); + dst[2] = 0; + asm volatile ("" : : : "memory"); + dst[3] = 0; + bar (dst); +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-11.c b/gcc/testsuite/gcc.target/i386/pr125893-11.c new file mode 100644 index 00000000000..f3735857313 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-11.c @@ -0,0 +1,99 @@ +/* { dg-do compile { target { ia32 && fpic } } } */ +/* { dg-options "-O2 -fPIC -fomit-frame-pointer -momit-leaf-frame-pointer -mtune=pentiumpro" } */ + +typedef int Elf32_Addr; +typedef struct +{ + int p_type; + unsigned p_offset; + Elf32_Addr p_vaddr; + int p_filesz; + int p_memsz; + int p_flags; + int p_align; +} Elf32_Phdr; +enum +{ + LA_SER_RUNPATH +} _rtld_local_ro_0; +struct loadcmd +{ + Elf32_Addr allocend, mapalign; + unsigned mapoff; + int prot; +}; +char cached; +int fbp, _dl_map_new_object_mode, _dl_map_new_object_fb, + _dl_map_new_object_namelen, _dl_map_new_object_loader, + _dl_map_new_object_loader_1; +struct dl_pt_load_iterator +{ + unsigned phnum; + short buf_base; + Elf32_Addr p_align_max; + Elf32_Addr pagesize; + int nloadcmds; +}; +Elf32_Phdr *_dl_map_object_scan_phdrs___trans_tmp_2; +char _dl_map_new_object_name; +char *_dl_map_new_object_realname, *_dl_map_new_object_errstring; +short _dl_map_new_object_l_0; +void __pread64 (int, void *, int, long); +int pf_to_prot (); +Elf32_Addr _dl_map_segment_align (struct loadcmd *, Elf32_Addr); +static void +_dl_pt_load_iterator_init (struct dl_pt_load_iterator *it, + unsigned short phnum) +{ + it->phnum = phnum; + it->pagesize = _rtld_local_ro_0; +} +static char * +_dl_map_object_scan_phdrs (struct dl_pt_load_iterator *it) +{ + for (unsigned i = 0; i < it->phnum; i++) + { + struct dl_pt_load_iterator __trans_tmp_3 = *it; + if (__builtin_expect (cached, 1)) + _dl_map_object_scan_phdrs___trans_tmp_2 = (Elf32_Phdr *)fbp + i; + if (__trans_tmp_3.buf_base) + { + short batch = __trans_tmp_3.phnum; + int off = i * sizeof (Elf32_Phdr); + __pread64 (0, 0, batch, off); + it->buf_base = i; + } + Elf32_Phdr ph = *_dl_map_object_scan_phdrs___trans_tmp_2; + switch (ph.p_type) + { + case 1: + if (__builtin_expect (ph.p_offset & it->pagesize - 1, 0)) + return ""; + Elf32_Addr mapstart = ph.p_vaddr - it->pagesize, + mapoff = ph.p_offset - pf_to_prot (); + it->p_align_max = _dl_map_segment_align ( + &(struct loadcmd){ mapstart, ph.p_vaddr, mapoff }, + it->p_align_max); + it->nloadcmds++; + } + } +} +extern int nloadcmds; +void open_path (char *, int, int, int *, char **, int *, int *, int, + char *); +void +_dl_map_new_object_empty_dynamic (void) +{ + char found_other_class = 0; + open_path (&_dl_map_new_object_name, _dl_map_new_object_namelen, + _dl_map_new_object_mode, &_dl_map_new_object_loader_1, + &_dl_map_new_object_realname, &_dl_map_new_object_fb, + &_dl_map_new_object_loader, LA_SER_RUNPATH, + &found_other_class); +lose: + struct dl_pt_load_iterator it; + _dl_pt_load_iterator_init (&it, _dl_map_new_object_l_0); + _dl_map_new_object_errstring = _dl_map_object_scan_phdrs (&it); + if (it.nloadcmds == nloadcmds) + goto lose; +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-2.c b/gcc/testsuite/gcc.target/i386/pr125893-2.c new file mode 100644 index 00000000000..e241633a3d4 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-2.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mtune-ctrl=lcp_stall" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** movl \$3, %[a-z0-9]+ +**... +** movw %[a-z0-9]+, \(%[a-z0-9]+\) +** movw %[a-z0-9]+, 4\(%[a-z0-9]+\) +** call _?bar +** movl \$3, %[a-z0-9]+ +** movw %[a-z0-9]+, 2\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 8\(%[a-z0-9]+\) +**... +*/ + + +extern void bar (void); + +void +foo (short *dst) +{ + dst[0] = 3; + asm volatile ("" : : : "memory"); + dst[2] = 3; + bar (); + dst[1] = 3; + asm volatile ("" : : : "memory"); + dst[4] = 3; +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-3.c b/gcc/testsuite/gcc.target/i386/pr125893-3.c new file mode 100644 index 00000000000..7c01661e7d0 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-3.c @@ -0,0 +1,27 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mtune-ctrl=lcp_stall" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** xorl %[a-z0-9]+, %[a-z0-9]+ +** movw %[a-z0-9]+, \(%[a-z0-9]+\) +** movw %[a-z0-9]+, 2\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 4\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 6\(%[a-z0-9]+\) +**... +*/ + +void +foo (short *dst) +{ + dst[0] = 0; + asm volatile ("" : : : "memory"); + dst[1] = 0; + asm volatile ("" : : : "memory"); + dst[2] = 0; + asm volatile ("" : : : "memory"); + dst[3] = 0; +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-4.c b/gcc/testsuite/gcc.target/i386/pr125893-4.c new file mode 100644 index 00000000000..88488a5b6a5 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-4.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mtune-ctrl=lcp_stall" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** xorl %[a-z0-9]+, %[a-z0-9]+ +**... +** movw %[a-z0-9]+, \(%[a-z0-9]+\) +** movw %[a-z0-9]+, 4\(%[a-z0-9]+\) +** call _?bar +** xorl %[a-z0-9]+, %[a-z0-9]+ +** movw %[a-z0-9]+, 2\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 8\(%[a-z0-9]+\) +**... +*/ + + +extern void bar (void); + +void +foo (short *dst) +{ + dst[0] = 0; + asm volatile ("" : : : "memory"); + dst[2] = 0; + bar (); + dst[1] = 0; + asm volatile ("" : : : "memory"); + dst[4] = 0; +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-5.c b/gcc/testsuite/gcc.target/i386/pr125893-5.c new file mode 100644 index 00000000000..fa8858533f4 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-5.c @@ -0,0 +1,34 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mtune-ctrl=lcp_stall" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** movl \$3, %[a-z0-9]+ +**... +** movw %[a-z0-9]+, \(%[a-z0-9]+\) +** movw %[a-z0-9]+, 4\(%[a-z0-9]+\) +** call _?bar +** movl \$3, %[a-z0-9]+ +** movw %[a-z0-9]+, 2\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 8\(%[a-z0-9]+\) +**... +*/ + + +extern int bar (void); + +int +foo (short *dst) +{ + dst[0] = 3; + asm volatile ("" : : : "memory"); + dst[2] = 3; + int ret = bar (); + dst[1] = 3; + asm volatile ("" : : : "memory"); + dst[4] = 3; + return ret; +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-6.c b/gcc/testsuite/gcc.target/i386/pr125893-6.c new file mode 100644 index 00000000000..62584cf4d77 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-6.c @@ -0,0 +1,34 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mtune-ctrl=lcp_stall" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** xorl %[a-z0-9]+, %[a-z0-9]+ +**... +** movw %[a-z0-9]+, \(%[a-z0-9]+\) +** movw %[a-z0-9]+, 4\(%[a-z0-9]+\) +** call _?bar +** xorl %[a-z0-9]+, %[a-z0-9]+ +** movw %[a-z0-9]+, 2\(%[a-z0-9]+\) +** movw %[a-z0-9]+, 8\(%[a-z0-9]+\) +**... +*/ + + +extern int bar (void); + +int +foo (short *dst) +{ + dst[0] = 0; + asm volatile ("" : : : "memory"); + dst[2] = 0; + int ret = bar (); + dst[1] = 0; + asm volatile ("" : : : "memory"); + dst[4] = 0; + return ret; +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-7.c b/gcc/testsuite/gcc.target/i386/pr125893-7.c new file mode 100644 index 00000000000..73292fcfb76 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-7.c @@ -0,0 +1,31 @@ +/* { dg-do compile { target ia32 } } */ +/* { dg-options "-O2 -fomit-frame-pointer -momit-leaf-frame-pointer -mtune=pentiumpro" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** movl \$3, %eax +** movl %eax, 16\(%esp\) +** movl %eax, 20\(%esp\) +** movl %eax, 24\(%esp\) +** movl %eax, 28\(%esp\) +**... +*/ + +extern void bar (int *dst); + +void +foo (void) +{ + int dst[16]; + dst[0] = 3; + asm volatile ("" : : : "memory"); + dst[1] = 3; + asm volatile ("" : : : "memory"); + dst[2] = 3; + asm volatile ("" : : : "memory"); + dst[3] = 3; + bar (dst); +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-8.c b/gcc/testsuite/gcc.target/i386/pr125893-8.c new file mode 100644 index 00000000000..2c60dd4271f --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-8.c @@ -0,0 +1,38 @@ +/* { dg-do compile { target ia32 } } */ +/* { dg-options "-O2 -fomit-frame-pointer -momit-leaf-frame-pointer -mtune=pentiumpro" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** movl \$3, %eax +**... +** movl %eax, 24\(%esp\) +** movl %eax, 28\(%esp\) +** leal 24\(%esp\), %ebx +** movl %ebx, \(%esp\) +** call bar +** movl \$3, %ecx +** movl %ecx, 32\(%esp\) +** movl %ebx, \(%esp\) +** movl %ecx, 36\(%esp\) +** call bar +**... +*/ + +extern void bar (int *dst); + +void +foo (void) +{ + int dst[10]; + dst[0] = 3; + asm volatile ("" : : : "memory"); + dst[1] = 3; + bar (dst); + dst[2] = 3; + asm volatile ("" : : : "memory"); + dst[3] = 3; + bar (dst); +} diff --git a/gcc/testsuite/gcc.target/i386/pr125893-9.c b/gcc/testsuite/gcc.target/i386/pr125893-9.c new file mode 100644 index 00000000000..c33b4913fb7 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr125893-9.c @@ -0,0 +1,31 @@ +/* { dg-do compile { target ia32 } } */ +/* { dg-options "-O2 -fomit-frame-pointer -momit-leaf-frame-pointer -mtune=pentiumpro" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {^\t?\.} } } */ + +/* +**foo: +**... +** xorl %eax, %eax +** movl %eax, 16\(%esp\) +** movl %eax, 20\(%esp\) +** movl %eax, 24\(%esp\) +** movl %eax, 28\(%esp\) +**... +*/ + +extern void bar (int *dst); + +void +foo (void) +{ + int dst[16]; + dst[0] = 0; + asm volatile ("" : : : "memory"); + dst[1] = 0; + asm volatile ("" : : : "memory"); + dst[2] = 0; + asm volatile ("" : : : "memory"); + dst[3] = 0; + bar (dst); +} -- 2.54.0
