https://gcc.gnu.org/g:eaca0fdd031bbed1d2ff7dd86c3adfecef1c51cc
commit r17-3224-geaca0fdd031bbed1d2ff7dd86c3adfecef1c51cc Author: Kyrylo Tkachov <[email protected]> Date: Wed Aug 5 14:58:44 2026 +0200 doc: describe the lane freedom of reduc_widen_[us]sum and WIDEN_SUM_EXPR The documentation of the reduc_widen_ssum and reduc_widen_usum patterns says only that operand 1 is added to operand 2, which reads as though each element of operand 1 has to be accumulated into the element of operand 0 at the same position. Nothing says the patterns are only ever used for a reassociable reduction, so a port cannot tell whether it is allowed to regroup the input elements, for example by using a pairwise widening add. vect_recog_widen_sum_pattern only forms WIDEN_SUM_EXPR through vect_reassociating_reduction_p, which requires the statement to be a reduction, refuses a statement nested in the inner loop of an outer-loop vectorization because the order of the computation matters there, and refuses a type that needs a fold-left reduction. The comment on the pattern already describes the idiom as producing N/2 results by summing up pairs of intermediate results. The accumulator is only ever consumed by a horizontal sum in the epilogue, so the distribution of input elements over accumulator elements is not observable. Spell that out in md.texi, along with the constraint that comes with it: an implementation that adds elements together before they reach the element width of operand 0 has to widen them first so that no intermediate sum can overflow. Note that the widen_[us]sum names are left free for a lane preserving pattern. Give WIDEN_SUM_EXPR the same statement in tree.def. DOT_PROD_EXPR and SAD_EXPR are formed through vect_reassociating_reduction_p as well and read the same way, so add a sentence to each pointing at the WIDEN_SUM_EXPR rule. gcc/ChangeLog: * doc/md.texi (reduc_widen_ssum@var{n}@var{m}3) (reduc_widen_usum@var{n}@var{m}3): Document that the assignment of input elements to accumulator elements is unconstrained, that intermediate sums must not overflow, and that the widen_[us]sum names are reserved. Fix a typo. * tree.def (WIDEN_SUM_EXPR): Document that the assignment of elements of the first argument to elements of the second is unconstrained. (DOT_PROD_EXPR, SAD_EXPR): Note the same freedom. Signed-off-by: Kyrylo Tkachov <[email protected]> Diff: --- gcc/doc/md.texi | 23 +++++++++++++++++++++-- gcc/tree.def | 19 +++++++++++++++++-- 2 files changed, 38 insertions(+), 4 deletions(-) diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi index 4b3dc950aeb0..f86fa8038e8a 100644 --- a/gcc/doc/md.texi +++ b/gcc/doc/md.texi @@ -5097,10 +5097,29 @@ in operand 0, which is of the same mode as operand 3. @itemx @samp{reduc_widen_usum@var{n}@var{m}3} Operands 0 and 2 are of the same mode, which is wider than the mode of operand 1. Add operand 1 to operand 2 and place the widened result in -operand 0. (This is used express accumulation of elements into an accumulator -of a wider mode.) +operand 0. (This is used to express accumulation of elements into an +accumulator of a wider mode.) @var{m} is the mode of operand 1 and @var{n} is the mode of operand 0. +These patterns are only used for a reduction whose summation the vectorizer +has already established may be reassociated. Operand 0 is the accumulator +of that reduction and is itself reduced to a scalar by a horizontal sum once +the loop is done, so which element of the accumulator a given input element +ends up in is not observable. An implementation is therefore free to choose +which elements of operand 1 it accumulates into which element of operand 0, +provided that every element of operand 1 is accumulated exactly once. In +particular it may add adjacent elements of operand 1 to each other before +accumulating them, which is what a pairwise widening add instruction does. + +The intermediate sums must not overflow. An implementation that combines +elements of operand 1 with each other before they reach the element width of +operand 0 therefore has to widen them first. + +The names @samp{widen_ssum@var{n}@var{m}3} and +@samp{widen_usum@var{n}@var{m}3} are deliberately left free for a future +lane preserving pattern, which would have to add element @var{i} of +operand 1 to element @var{i} of operand 2 for every @var{i}. + @mdindex smulhs@var{m}3 @mdindex umulhs@var{m}3 @item @samp{smulhs@var{m}3} diff --git a/gcc/tree.def b/gcc/tree.def index ad8e676e9421..2e767dbe0453 100644 --- a/gcc/tree.def +++ b/gcc/tree.def @@ -1465,7 +1465,10 @@ DEFTREECODE (TRANSACTION_EXPR, "transaction_expr", tcc_expression, 1) arg3 = PLUS_EXPR (tmp, arg3); or: tmp = WIDEN_MULT_EXPR(arg1, arg2); - arg3 = WIDEN_SUM_EXPR (tmp, arg3); */ + arg3 = WIDEN_SUM_EXPR (tmp, arg3); + Like WIDEN_SUM_EXPR, this code is only created for a reduction whose + summation may be reassociated, so which products are added into which + element of arg3 is not observable. */ DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr", tcc_expression, 3) /* Widening summation. @@ -1474,7 +1477,16 @@ DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr", tcc_expression, 3) the size of t1. The type of the entire expression is also t2. WIDEN_SUM_EXPR is equivalent to first widening (promoting) the first argument from type t1 to type t2, and then summing it - with the second argument. */ + with the second argument. + + This code is only created for a reduction whose summation the vectorizer + has established may be reassociated, and the result is reduced to a scalar + by a horizontal sum once the loop is done. Which element of the second + argument a given element of the first argument is added to is therefore + not observable, and an expander may choose that assignment freely, so long + as every element of the first argument is accumulated exactly once and no + intermediate sum overflows. See the reduc_widen_ssum and reduc_widen_usum + patterns in md.texi. */ DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr", tcc_binary, 2) /* Widening sad (sum of absolute differences). @@ -1490,6 +1502,9 @@ DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr", tcc_binary, 2) tmp = IFN_VEC_WIDEN_MINUS_EXPR (arg1, arg2) tmp2 = ABS_EXPR (tmp) arg3 = WIDEN_SUM_EXPR (tmp2, arg3) + Like WIDEN_SUM_EXPR, this code is only created for a reduction whose + summation may be reassociated, so which absolute differences are added + into which element of arg3 is not observable. */ DEFTREECODE (SAD_EXPR, "sad_expr", tcc_expression, 3)
