On Tue, 25 Aug 2026, Andrea Pinski wrote:
> On Tue, Aug 25, 2026 at 10:59 PM <[email protected]> wrote:
> >
> > From: Kyrylo Tkachov <[email protected]>
> >
> > REDUC_MAX, REDUC_MIN, REDUC_AND and REDUC_IOR are idempotent. Reducing
> > a nonempty uniform vector with one of these operations gives the element
> > that the vector contains. Fold this form to that element:
> >
> > v = _Literal (v4si) { x, x, x, x };
> > res = .REDUC_MAX (v);
> > return res;
> >
> > to:
> >
> > return x;
> >
> > An XOR reduction of a uniform vector is zero when the number of lanes is
> > even. GCC permits variable-length vectors whose lane count has unknown
> > parity, such as [1, 1]. Fold REDUC_XOR only when multiple_p proves that
> > every possible lane count is even:
> >
> > v = _Literal (v4si) { x, x, x, x };
> > res = .REDUC_XOR (v);
> > return res;
> >
> > to:
> >
> > return 0;
> >
> > Use vec_same_elem_p to match the uniform vector and
> > ssa_uniform_vector_p to get its element. Keep a floating-point
> > reduction when the element can be a signaling NaN because the reduction
> > can quiet the NaN and raise an exception.
> >
> > These folds also let the constant reduction in the PR 127039 rule
> > simplify for scalable uniform constants. The scalar result cannot match
> > the inverse reduction-pair rule, so the nontermination cycle stays
> > blocked.
> >
> > The assembly comparison uses this function from the new test:
> >
> > typedef int v4si __attribute__ ((vector_size (16)));
> >
> > int __GIMPLE ()
> > reduc_max (int x)
> > {
> > v4si v;
> > int res;
> >
> > v = _Literal (v4si) { x, x, x, x };
> > res = .REDUC_MAX (v);
> > return res;
> > }
> >
> > With -O2 -fgimple on AArch64, the code changes from:
> >
> > reduc_max:
> > dup v31.4s, w0
> > smaxv s31, v31.4s
> > fmov w0, s31
> > ret
> >
> > to:
> >
> > reduc_max:
> > ret
> >
> > Bootstrapped and tested on aarch64-unknown-linux-gnu.
> > Ok for trunk?
> > Thanks,
> > Kyrill
> >
> > gcc/ChangeLog:
> >
> > PR tree-optimization/127039
> > * match.pd (reduction of a uniform vector): New simplifications to
> > the splatted element or zero.
> >
> > gcc/testsuite/ChangeLog:
> >
> > PR tree-optimization/127039
> > * gcc.dg/tree-ssa/vec-reduc-splat-1.c: New test.
> > * gcc.target/aarch64/sve/vec-reduc-uniform-1.c: New test.
> > * gcc.target/aarch64/vec-reduc-splat-float-1.c: New test.
> > * gcc.target/riscv/rvv/base/reduc-xor-uniform-1.c: New test.
> >
> > Signed-off-by: Kyrylo Tkachov <[email protected]>
> > ---
> > gcc/match.pd | 17 +++++
> > .../gcc.dg/tree-ssa/vec-reduc-splat-1.c | 69 ++++++++++++++++++
> > .../aarch64/sve/vec-reduc-uniform-1.c | 71 +++++++++++++++++++
> > .../aarch64/vec-reduc-splat-float-1.c | 31 ++++++++
> > .../riscv/rvv/base/reduc-xor-uniform-1.c | 47 ++++++++++++
> > 5 files changed, 235 insertions(+)
> > create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/vec-reduc-splat-1.c
> > create mode 100644
> > gcc/testsuite/gcc.target/aarch64/sve/vec-reduc-uniform-1.c
> > create mode 100644
> > gcc/testsuite/gcc.target/aarch64/vec-reduc-splat-float-1.c
> > create mode 100644
> > gcc/testsuite/gcc.target/riscv/rvv/base/reduc-xor-uniform-1.c
> >
> > diff --git a/gcc/match.pd b/gcc/match.pd
> > index c7841cab459..529e3d5ff60 100644
> > --- a/gcc/match.pd
> > +++ b/gcc/match.pd
> > @@ -12536,6 +12536,23 @@ and,
> > (if (elem)
> > { build_vector_from_val (type, elem); }))))
> >
> > +/* A reduction of a uniform vector with an idempotent operation is the
> > + element it is a splat of. */
> > +(for reduc (IFN_REDUC_MAX IFN_REDUC_MIN IFN_REDUC_AND IFN_REDUC_IOR)
> > + (simplify
> > + (reduc vec_same_elem_p@0)
> > + (with { tree elem = ssa_uniform_vector_p (@0); }
> > + (if (elem
> > + && types_match (type, TREE_TYPE (elem))
> > + && !tree_expr_maybe_signaling_nan_p (elem))
> > + { elem; }))))
>
> No.
> Please instead do it similar to IFN_VEC_EXTRACT and IFN_VEC_SHL_INSERT.
But those use just (vec_duplicate ..), they possibly should use
vec_same_elem_p, but then there is only an ugly way to get the
duplicated element for
(match vec_same_elem_p
@0
(if (uniform_vector_p (@0))))
like with
(match (vec_same_elem_p@0 @1)
@0
(if ((@1 = uniform_vector_p (@0)))))
which works, but is not documented to be allowed. It also shows
that 't' is not captured (the implicit @0 for output-less (match ..)
is a workaround), when doing it this way we get two outputs,
so it will likely not work as intended. I'd probably suggest
to implicitly have @t captured but not as output, or, ignore
the expression root when assigning outputs ...
The last example currently code-generates
bool
gimple_vec_same_elem_p (tree t, tree *res_ops, tree (*valueize)(tree)
ATTRIBUTE_UNUSED)
{
const tree type = TREE_TYPE (t);
const bool debug_dump = dump_file && (dump_flags & TDF_FOLDING);
{
tree captures[2] ATTRIBUTE_UNUSED = { t };
if ((captures[1] = uniform_vector_p (captures[0]))
)
{
{
res_ops[0] = captures[0];
res_ops[1] = captures[1];
if (UNLIKELY (debug_dump)) gimple_dump_logs ("test.pd", 0,
__FILE__, __LINE__, false);
return true;
Richard.
> > +
> > +/* Fold an XOR reduction of an even-length uniform vector to zero. */
> > +(simplify
> > + (IFN_REDUC_XOR vec_same_elem_p@0)
> > + (if (multiple_p (TYPE_VECTOR_SUBPARTS (TREE_TYPE (@0)), 2))
> > + { build_zero_cst (type); }))
> > +
> > /* Push VEC_PERM earlier if that may help FMA perception (PR101895). */
> > (simplify
> > (plus:c (vec_perm:s (mult:c@0 @1 vec_same_elem_p@2) @0 @3) @4)
> > diff --git a/gcc/testsuite/gcc.dg/tree-ssa/vec-reduc-splat-1.c
> > b/gcc/testsuite/gcc.dg/tree-ssa/vec-reduc-splat-1.c
> > new file mode 100644
> > index 00000000000..2ad23472a06
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.dg/tree-ssa/vec-reduc-splat-1.c
> > @@ -0,0 +1,69 @@
> > +/* { dg-do compile } */
> > +/* { dg-options "-O2 -fgimple -Wno-psabi -fdump-tree-forwprop1" } */
> > +
> > +/* Check reductions of a variable splat. */
> > +
> > +typedef int v4si __attribute__ ((vector_size (16)));
> > +
> > +int __GIMPLE ()
> > +reduc_max (int x)
> > +{
> > + v4si v;
> > + int res;
> > +
> > + v = _Literal (v4si) { x, x, x, x };
> > + res = .REDUC_MAX (v);
> > + return res;
> > +}
> > +
> > +int __GIMPLE ()
> > +reduc_min (int x)
> > +{
> > + v4si v;
> > + int res;
> > +
> > + v = _Literal (v4si) { x, x, x, x };
> > + res = .REDUC_MIN (v);
> > + return res;
> > +}
> > +
> > +int __GIMPLE ()
> > +reduc_and (int x)
> > +{
> > + v4si v;
> > + int res;
> > +
> > + v = _Literal (v4si) { x, x, x, x };
> > + res = .REDUC_AND (v);
> > + return res;
> > +}
> > +
> > +int __GIMPLE ()
> > +reduc_ior (int x)
> > +{
> > + v4si v;
> > + int res;
> > +
> > + v = _Literal (v4si) { x, x, x, x };
> > + res = .REDUC_IOR (v);
> > + return res;
> > +}
> > +
> > +int __GIMPLE ()
> > +reduc_xor_even (int x)
> > +{
> > + v4si v;
> > + int res;
> > +
> > + v = _Literal (v4si) { x, x, x, x };
> > + res = .REDUC_XOR (v);
> > + return res;
> > +}
> > +
> > +/* { dg-final { scan-tree-dump-not {\.REDUC_MAX} "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-not {\.REDUC_MIN} "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-not {\.REDUC_AND} "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-not {\.REDUC_IOR} "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-not {\.REDUC_XOR} "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-times {return x_} 4 "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-times {return 0;} 1 "forwprop1" } } */
> > diff --git a/gcc/testsuite/gcc.target/aarch64/sve/vec-reduc-uniform-1.c
> > b/gcc/testsuite/gcc.target/aarch64/sve/vec-reduc-uniform-1.c
> > new file mode 100644
> > index 00000000000..9830511b18d
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/sve/vec-reduc-uniform-1.c
> > @@ -0,0 +1,71 @@
> > +/* { dg-do compile } */
> > +/* { dg-options "-O3 -msve-vector-bits=scalable
> > -mautovec-preference=sve-only -fvect-cost-model=unlimited
> > --param=vect-partial-vector-usage=0 -fdump-tree-vect-details
> > -fdump-tree-forwprop4" } */
> > +
> > +/* Check reductions of scalable uniform vector constants. */
> > +
> > +char c1, c2, c3, c4, c5;
> > +short d1, d2, d3, d4, d5;
> > +int rmax, rmin, r_and, r_ior, r_xor;
> > +unsigned n;
> > +
> > +__attribute__ ((noipa)) void
> > +max_uniform (void)
> > +{
> > + for (short i = 0; i < n; i += 3)
> > + {
> > + c1 ^= d1;
> > + rmax = rmax > 17 ? rmax : 17;
> > + }
> > +}
> > +
> > +__attribute__ ((noipa)) void
> > +min_uniform (void)
> > +{
> > + for (short i = 0; i < n; i += 3)
> > + {
> > + c2 ^= d2;
> > + rmin = rmin < -19 ? rmin : -19;
> > + }
> > +}
> > +
> > +__attribute__ ((noipa)) void
> > +and_uniform (void)
> > +{
> > + for (short i = 0; i < n; i += 3)
> > + {
> > + c3 ^= d3;
> > + r_and &= 1431655765;
> > + }
> > +}
> > +
> > +__attribute__ ((noipa)) void
> > +ior_uniform (void)
> > +{
> > + for (short i = 0; i < n; i += 3)
> > + {
> > + c4 ^= d4;
> > + r_ior |= 572662306;
> > + }
> > +}
> > +
> > +__attribute__ ((noipa)) void
> > +xor_uniform (void)
> > +{
> > + for (short i = 0; i < n; i += 3)
> > + {
> > + c5 ^= d5;
> > + r_xor ^= 858993459;
> > + }
> > +}
> > +
> > +/* { dg-final { scan-tree-dump-times {LOOP VECTORIZED} 5 "vect" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_MAX} 1 "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_MIN} 1 "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_AND} 1 "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_IOR} 1 "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_MAX \(vect_rmax_lsm} 1
> > "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_MIN \(vect_rmin_lsm} 1
> > "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_AND \(vect_r_and_lsm} 1
> > "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_IOR \(vect_r_ior_lsm} 1
> > "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_XOR} 5 "forwprop4" } } */
> > +/* { dg-final { scan-tree-dump-not {\.REDUC_XOR \(\{ 0, \.\.\. \}\)}
> > "forwprop4" } } */
> > diff --git a/gcc/testsuite/gcc.target/aarch64/vec-reduc-splat-float-1.c
> > b/gcc/testsuite/gcc.target/aarch64/vec-reduc-splat-float-1.c
> > new file mode 100644
> > index 00000000000..ca0764997a0
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/vec-reduc-splat-float-1.c
> > @@ -0,0 +1,31 @@
> > +/* { dg-do compile } */
> > +/* { dg-options "-O2 -fgimple -fsignaling-nans -fdump-tree-forwprop1" } */
> > +
> > +/* Keep reductions of a splat that can contain a signaling NaN. */
> > +
> > +typedef float v4sf __attribute__ ((vector_size (16)));
> > +
> > +float __GIMPLE ()
> > +reduc_max (float x)
> > +{
> > + v4sf v;
> > + float res;
> > +
> > + v = _Literal (v4sf) { x, x, x, x };
> > + res = .REDUC_MAX (v);
> > + return res;
> > +}
> > +
> > +float __GIMPLE ()
> > +reduc_min (float x)
> > +{
> > + v4sf v;
> > + float res;
> > +
> > + v = _Literal (v4sf) { x, x, x, x };
> > + res = .REDUC_MIN (v);
> > + return res;
> > +}
> > +
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_MAX} 1 "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_MIN} 1 "forwprop1" } } */
> > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/base/reduc-xor-uniform-1.c
> > b/gcc/testsuite/gcc.target/riscv/rvv/base/reduc-xor-uniform-1.c
> > new file mode 100644
> > index 00000000000..54339490fd8
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/riscv/rvv/base/reduc-xor-uniform-1.c
> > @@ -0,0 +1,47 @@
> > +/* { dg-do compile } */
> > +/* { dg-options "-O2 -fgimple -march=rv64im_zve64f -mabi=lp64
> > -fdump-tree-forwprop1" } */
> > +/* { dg-final { check-function-bodies "**" "" } } */
> > +
> > +#include "riscv_vector.h"
> > +
> > +/*
> > +** reduc_xor_even:
> > +** li a0,0
> > +** ret
> > +*/
> > +long __GIMPLE ()
> > +reduc_xor_even (void)
> > +{
> > + vbool32_t mask;
> > + vint64m2_t ones;
> > + long res;
> > +
> > + mask = _Literal (vint64m2_t) 0 == _Literal (vint64m2_t) 0;
> > + ones = .VEC_CONVERT (mask);
> > + res = .REDUC_XOR (ones);
> > + return res;
> > +}
> > +
> > +/*
> > +** reduc_xor_unknown:
> > +** vsetvli [a-x0-9]+,zero,e64,m1,ta,ma
> > +** ...
> > +** vredxor\.vs v[0-9]+,v[0-9]+,v[0-9]+
> > +** ...
> > +** ret
> > +*/
> > +long __GIMPLE ()
> > +reduc_xor_unknown (void)
> > +{
> > + vbool64_t mask;
> > + vint64m1_t ones;
> > + long res;
> > +
> > + mask = _Literal (vint64m1_t) 0 == _Literal (vint64m1_t) 0;
> > + ones = .VEC_CONVERT (mask);
> > + res = .REDUC_XOR (ones);
> > + return res;
> > +}
> > +
> > +/* { dg-final { scan-tree-dump-times {\.REDUC_XOR} 1 "forwprop1" } } */
> > +/* { dg-final { scan-tree-dump-times {return 0;} 1 "forwprop1" } } */
> > --
> > 2.50.1 (Apple Git-155)
> >
>
--
Richard Biener <[email protected]>
SUSE Software Solutions Germany GmbH,
Frankenstrasse 146, 90461 Nuernberg, Germany;
GF: Jochen Jaser, Andrew McDonald, Abhinav Puri; (HRB 36809, AG Nuernberg)
