> -----Original Message----- > From: H.J. Lu <[email protected]> > Sent: Friday, June 26, 2026 3:04 PM > To: Hongtao Liu <[email protected]> > Cc: Liu, Hongtao <[email protected]>; GCC Patches <gcc- > [email protected]>; Uros Bizjak <[email protected]> > Subject: [PATCH v3] x86-64: Expand bounded memset and memcpy like > memmove > > On Thu, Jun 25, 2026 at 5:48 PM Hongtao Liu <[email protected]> wrote: > > > > >>+/* Return a value rtx in MODE for memset from MEMSET_VALS. */ > > >>+ > > >>+static rtx > > >>+ix86_expand_memset_val (rtx *memset_vals, machine_mode mode) > > > > > >Where is "an odd blank line"? > > > > Nevermind. > > > > >+/* Return a value rtx in MODE for memset from MEMSET_VALS. */ > > >+ > > >+static rtx > > >+ix86_expand_memset_val (rtx *memset_vals, machine_mode mode) { > > >+ rtx val; > > >+ if (mode == QImode) > > >+ val = memset_vals[memset_val_byte]; > > >+ else if (mode == word_mode) > > >+ val = memset_vals[memset_val_word]; > > >+ else if (GET_MODE_CLASS (mode) == MODE_VECTOR_INT) > > >+ { > > >+ if (GET_MODE (memset_vals[memset_val_vector]) == mode) > > >+ val = memset_vals[memset_val_vector]; > > >+ else > > >+ { > > >+ machine_mode vmode = GET_MODE > (memset_vals[memset_val_vector]); > > >+ if (memset_vals[memset_val_vector] == CONST0_RTX (vmode)) > > >+ val = CONST0_RTX (mode); > > >+ else if (memset_vals[memset_val_vector] == CONSTM1_RTX > (vmode)) > > >+ val = CONSTM1_RTX (mode); > > >+ else > > ... > > >+ if (memset_vals[memset_val_word] == const0_rtx > > >+ || memset_vals[memset_val_word] == constm1_rtx) > > >+ val = memset_vals[memset_val_word]; > > > > const0_rtx or constm1_rtx in memset_vals[memset_val_word] or > > memset_vals[memset_val_vector] is propagated from > > memset_vals[memset_val_byte] in ix86_expand_movmem, so we can be > > simply rewrite ix86_expand_memset_val as below > > > > static rtx > > ix86_expand_memset_val (rtx *memset_vals, machine_mode mode) { > > rtx byte_val = memset_vals[memset_val_byte]; > > > > if (mode == QImode) > > return byte_val; > > if (mode == word_mode) > > return memset_vals[memset_val_word]; > > > > /* All-zero / all-ones is a property of the original byte value, so > > detect it once here rather than re-deriving it from each slot. */ > > if (byte_val == const0_rtx) > > return CONST0_RTX (mode); > > if (byte_val == constm1_rtx) > > return CONSTM1_RTX (mode); > > > > if (GET_MODE_CLASS (mode) == MODE_VECTOR_INT) > > { > > if (GET_MODE (memset_vals[memset_val_vector]) == mode) > > return memset_vals[memset_val_vector]; > > return gen_rtx_SUBREG (mode, memset_vals[memset_val_vector], 0); > > } > > > > gcc_assert (mode == HImode || mode == SImode); > > return gen_rtx_SUBREG (mode, memset_vals[memset_val_word], 0); } > > Fixed. > > > >+ /* Expand memcpy and memset like memmove only for bounded size. > > >+ */ if ((iscpymem || issetmem) > > >+ && (count != 0 /* Fixed size. */ > > >+ || max_size == 0 /* Unbounded size. */ > > >+ || max_size == HOST_WIDE_INT_M1U)) /* Unbounded size. */ > > > > For x32, unbounded max_size could be 0xffff(size_t) instead of > > HOST_WIDE_INT_M1U So maybe > > if (iscpymem || issetmem) > > { > > unsigned HOST_WIDE_INT unbounded > > = GET_MODE_MASK (counter_mode (count_exp)); > > if (count != 0 /* Fixed size. */ > > || max_size == 0 /* Unbounded size. */ > > || max_size == unbounded) /* Unbounded size. */ > > return false; > > } > > Fixed with testcases. > > > > > >+ machine_mode vector_mode; > > > if (between_32_63_label) > > > { > > > emit_label (between_32_63_label); > > >- ix86_expand_n_overlapping_move_movmem (dst, src, destreg, srcreg, > > >- count_exp, OImode, 2); > > >+ vector_mode = memset_vals ? V32QImode : OImode; > > >+ ix86_expand_n_overlapping_move_set_or_movmem (dst, src, > > >+ memset_vals, > > >+ destreg, srcreg, > > >+ count_exp, > > >+ vector_mode, 2); > > > emit_jump_insn (gen_jump (done_label)); > > > emit_barrier (); > > > } > > >@@ -10428,8 +10548,12 @@ ix86_expand_less_move_movmem (rtx dst, > rtx src, rtx destreg, > > > if (between_16_31_label) > > > { > > > emit_label (between_16_31_label); > > >- ix86_expand_n_overlapping_move_movmem (dst, src, destreg, srcreg, > > >- count_exp, TImode, 2); > > >+ vector_mode = memset_vals ? V16QImode : TImode; > > >+ ix86_expand_n_overlapping_move_set_or_movmem (dst, src, > > >+ memset_vals, > > >+ destreg, srcreg, > > >+ count_exp, > > >+ vector_mode, 2); > > > > Can we refactor those if(***_label) branches into > > > > /* For each size band, memset uses a QI-vector mode above a word so it > > can broadcast the fill value, while memmove uses the same-size scalar > > integer mode; at and below a word both use the scalar integer mode. */ > > struct { > > rtx_code_label *label; > > machine_mode set_mode; > > machine_mode move_mode; > > } bands[] = { > > { between_32_63_label, V32QImode, OImode }, > > { between_16_31_label, V16QImode, TImode }, > > { between_8_15_label, DImode, DImode }, > > { between_4_7_label, SImode, SImode }, > > { between_2_3_label, HImode, HImode }, > > }; > > > > for (auto &band : bands) > > if (band.label) > > { > > emit_label (band.label); > > machine_mode bmode = memset_vals ? band.set_mode : > band.move_mode; > > ix86_expand_n_overlapping_move_set_or_movmem (dst, src, > memset_vals, > > destreg, srcreg, > > count_exp, bmode, 2); > > emit_jump_insn (gen_jump (done_label)); > > emit_barrier (); > > } > > Fixed. > > Here is the v3 patch.
Ok. > > > -- > H.J. > ---- > commit 401199377c50045ede560daf3f6e8b51749c2a87 > Author: H.J. Lu <[email protected]> > Date: Tue Jun 17 10:17:17 2025 +0800 > > x86: Improve vector_loop/unrolled_loop for memset/memcpy > > uses move_by_pieces and store_by_pieces for memcpy and memset epilogues > with the fixed epilogue size. Since move_by_pieces and store_by_pieces don't > use the maximum size info, they generate extra branches and moves for > bounded memcpy and memset. Commit > > commit b41f96465190751561f6909e858604ceab00595b > Author: H.J. Lu <[email protected]> > Date: Mon Oct 20 16:14:34 2025 +0800 > > x86-64: Inline memmove with overlapping unaligned loads and stores. > > inlines memmove with overlapping unaligned and stores which reduces the > numbers of branches as well as moves when the maximum size is known. > Rename ix86_expand_movmem to ix86_expand_set_or_movmem and extend > it to inline bounded memcpy and memset. Update > ix86_expand_set_or_cpymem to call ix86_expand_set_or_movmem for > bounded memset and memcpy as memmove if misaligned moves are > preferred. > > In addition to reduce 727.cppcheck_r O2 code size by ~9%, there're another > ~8 benchmarks whose code sizes are reduced >2% across spec2026 and > spec2017 with march=x86-64-v3 O2, no big code size impact for Ofast, > performance impact is negligible(slightly better, but all under noise > range) for both O2 and Ofast. > > gcc/ > > PR target/125856 > PR target/125865 > * config/i386/i386-expand.cc (ix86_expand_set_or_cpymem): > Call ix86_expand_set_or_movmem for > TARGET_MISALIGNED_MOVE_STRING_PRO_EPILOGUES. > (ix86_expand_memset_val): New. > (ix86_expand_n_move_movmem): Renamed to ... > (ix86_expand_n_move_set_or_movmem): This. Add a pointer to rtx > argument for memset. Also expand memset. > (ix86_expand_n_overlapping_move_movmem): Renamed to ... > (ix86_expand_n_overlapping_move_set_or_movmem): This. Add a pointer to > rtx argument for memset. Also expand memset. > (ix86_expand_less_move_movmem): Renamed to ... > (ix86_expand_less_move_set_or_movmem): This. Add a pointer to rtx > argument for memset. Also expand memset. > (ix86_expand_movmem): Renamed to ... > (ix86_expand_set_or_movmem): This. Add bool arguments for memcpy and > memset. Also expand memcpy and memset. > * config/i386/i386-protos.h (ix86_expand_movmem): Renamed to ... > (ix86_expand_set_or_movmem): This. Add bool arguments for memcpy and > memset. > * config/i386/i386.md (movmem<mode>): Replace ix86_expand_movmem > with ix86_expand_set_or_movmem. > > gcc/testsuite/ > > PR target/125856 > PR target/125865 > * gcc.target/i386/builtin-memcpy-1a.c: New test. > * gcc.target/i386/builtin-memcpy-1b.c: Likewise. > * gcc.target/i386/builtin-memcpy-2a.c: Likewise. > * gcc.target/i386/builtin-memcpy-2b.c: Likewise. > * gcc.target/i386/builtin-memcpy-2c.c: Likewise. > * gcc.target/i386/builtin-memcpy-3a.c: Likewise. > * gcc.target/i386/builtin-memcpy-3b.c: Likewise. > * gcc.target/i386/builtin-memcpy-3c.c: Likewise. > * gcc.target/i386/builtin-memcpy-4a.c: Likewise. > * gcc.target/i386/builtin-memcpy-4b.c: Likewise. > * gcc.target/i386/builtin-memcpy-4c.c: Likewise. > * gcc.target/i386/builtin-memcpy-5a.c: Likewise. > * gcc.target/i386/builtin-memcpy-5b.c: Likewise. > * gcc.target/i386/builtin-memcpy-5c.c: Likewise. > * gcc.target/i386/builtin-memcpy-6.c: Likewise. > * gcc.target/i386/builtin-memcpy-bounded-1a.c: Likewise. > * gcc.target/i386/builtin-memcpy-bounded-1b.c: Likewise. > * gcc.target/i386/builtin-memset-1a.c: Likewise. > * gcc.target/i386/builtin-memset-1b.c: Likewise. > * gcc.target/i386/builtin-memset-2a.c: Likewise. > * gcc.target/i386/builtin-memset-2b.c: Likewise. > * gcc.target/i386/builtin-memset-2c.c: Likewise. > * gcc.target/i386/builtin-memset-3a.c: Likewise. > * gcc.target/i386/builtin-memset-3b.c: Likewise. > * gcc.target/i386/builtin-memset-3c.c: Likewise. > * gcc.target/i386/builtin-memset-4a.c: Likewise. > * gcc.target/i386/builtin-memset-4b.c: Likewise. > * gcc.target/i386/builtin-memset-4c.c: Likewise. > * gcc.target/i386/builtin-memset-5a.c: Likewise. > * gcc.target/i386/builtin-memset-5b.c: Likewise. > * gcc.target/i386/builtin-memset-5c.c: Likewise. > * gcc.target/i386/builtin-memset-6a.c: Likewise. > * gcc.target/i386/builtin-memset-6b.c: Likewise. > * gcc.target/i386/builtin-memset-6c.c: Likewise. > * gcc.target/i386/builtin-memset-7a.c: Likewise. > * gcc.target/i386/builtin-memset-7b.c: Likewise. > * gcc.target/i386/builtin-memset-7c.c: Likewise. > * gcc.target/i386/builtin-memset-8.c: Likewise. > * gcc.target/i386/builtin-memset-bounded-1a.c: Likewise. > * gcc.target/i386/builtin-memset-bounded-1b.c: Likewise.
