On Mon, 27 Jul 2026, Kyrylo Tkachov wrote:
>
>
> > On 27 Jul 2026, at 14:57, Tamar Christina <[email protected]> wrote:
> >
> >> -----Original Message-----
> >> From: [email protected] <[email protected]>
> >> Sent: 27 July 2026 12:58
> >> To: [email protected]
> >> Cc: Tamar Christina <[email protected]>; [email protected]; Kyrylo
> >> Tkachov <[email protected]>
> >> Subject: [PATCH] vect: do not scale the scalar iteration cost twice by the
> >> inner
> >> loop factor
> >>
> >> From: Kyrylo Tkachov <[email protected]>
> >>
> >> vect_compute_single_scalar_iteration_cost weights inner-loop statements
> >> by LOOP_VINFO_INNER_LOOP_COST_FACTOR through the count it passes to
> >> record_stmt_cost. It used to record them with a vect_prologue location
> >> so that vector_costs::adjust_cost_for_freq would not apply the same
> >> factor again, and the comment there still said so.
> >> r17-129-g7eb6d800c9cc ("Cost scalar into vect_body") switched the
> >> location to vect_body, so for outer-loop vectorisation the scalar
> >> iteration cost is now scaled by the factor squared while the vector body
> >> cost is scaled once.
> >>
> >> With the default factor of 50 that makes the scalar cost about 50x too
> >> large, so the profitability denominator stays positive no matter what
> >> the vector cost is and outer loops get vectorised that the cost model
> >> should reject. Only outer-loop vectorisation is affected, since
> >> stmt_in_inner_loop_p is false otherwise.
> >>
> >> Skip the weighting in adjust_cost_for_freq when costing the scalar loop.
> >> The vect_body location has to stay: scalar_costs->body_cost () must
> >> remain non-zero for vect_get_known_peeling_cost. The explicit factor
> >> has to stay too, because vect_estimate_min_profitable_iters re-uses
> >> these entries with vect_prologue and vect_epilogue locations, where
> >> adjust_cost_for_freq never applies.
> >>
> >> For gcc.target/aarch64/pr121290_1.c the reported scalar iteration cost
> >> goes from 12505 back to 255.
> >>
> >> Fixes gcc.target/aarch64/pr121290_1.c, pr121290_2.c and pr121290_3.c.
> >>
> >
> > Lol, so they broke again? In any case I tried to fix this before [1] and...
> >
> > [1] https://gcc.gnu.org/pipermail/gcc-patches/2025-August/692967.html
> >
> >> Bootstrapped and regtested on aarch64-none-linux-gnu.
> >> Ok for trunk?
> >> Thanks,
> >> Kyrill
> >>
> >> gcc/ChangeLog:
> >>
> >> * tree-vectorizer.cc (vector_costs::adjust_cost_for_freq): Do not
> >> apply the inner loop cost factor when costing the scalar loop.
> >> * tree-vect-loop.cc (vect_compute_single_scalar_iteration_cost):
> >> Update comment.
> >>
> >> Signed-off-by: Kyrylo Tkachov <[email protected]>
> >> ---
> >> gcc/tree-vect-loop.cc | 8 ++++++--
> >> gcc/tree-vectorizer.cc | 6 +++++-
> >> 2 files changed, 11 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
> >> index a9335ed68bb..d57508bd66c 100644
> >> --- a/gcc/tree-vect-loop.cc
> >> +++ b/gcc/tree-vect-loop.cc
> >> @@ -1412,8 +1412,12 @@ vect_compute_single_scalar_iteration_cost
> >> (loop_vec_info loop_vinfo)
> >> else
> >> kind = scalar_stmt;
> >>
> >> - /* We are using vect_prologue here to avoid scaling twice
> >> - by the inner loop factor. */
> >> + /* FACTOR already weights inner-loop statements by the inner
> >> + loop cost factor, so vector_costs::adjust_cost_for_freq must
> >> + not apply that weighting again for scalar costing. The cost
> >> + entries recorded here are also re-used by
> >> + vect_estimate_min_profitable_iters with a vect_prologue or
> >> + vect_epilogue location, where the weighting never applies. */
> >> record_stmt_cost (&LOOP_VINFO_SCALAR_ITERATION_COST
> >> (loop_vinfo),
> >> factor, kind, stmt_info, 0, vect_body);
> >> }
> >
> > The suggestion was to just delete the scaling code entirely here.
> > pass a fixed factor = 1 to the record statement.
> >
> >> diff --git a/gcc/tree-vectorizer.cc b/gcc/tree-vectorizer.cc
> >> index 8c824e4ee33..43d22b0930e 100644
> >> --- a/gcc/tree-vectorizer.cc
> >> +++ b/gcc/tree-vectorizer.cc
> >> @@ -1893,8 +1893,12 @@ vector_costs::adjust_cost_for_freq
> >> (stmt_vec_info stmt_info,
> >> {
> >> /* Statements in an inner loop relative to the loop being
> >> vectorized are weighted more heavily. The value here is
> >> - arbitrary and could potentially be improved with analysis. */
> >> + arbitrary and could potentially be improved with analysis.
> >> + When costing the scalar loop the weighting is already applied by
> >> + vect_compute_single_scalar_iteration_cost through the count it
> >> + passes to record_stmt_cost, so do not apply it a second time. */
> >> if (where == vect_body
> >> + && !m_costing_for_scalar
> >
> > and then you don't need this change here since it'll scale only once then.
> >
> > Can you try that instead?
> >
>
> Unfortunately it regresses gcc.target/aarch64/pr62178.c Looks like the
> scalar cost gets applied zero times. adjust_cost_for_freq fires only for
> where == vect_body, while vect_estimate_min_profitable_iters re-uses
> those same entries with vect_prologue/vect_epilogue.
Ah. So this is where I stopped the cleanup - I esp. wanted to
separate the scalar epilogue cost from the vect_epilogue cost at some
point to have a better way to do epiloge vect costing.
But then the 'factor' is only about outer loop vectorization for
which we never peel. So I wonder what "zero times" means here?
> Thanks,
> Kyrill
>
> > Thanks,
> > Tamar
> >
> >> && stmt_info
> >> && stmt_in_inner_loop_p (m_vinfo, stmt_info))
> >> {
> >> --
> >> 2.50.1 (Apple Git-155)
>
>
>
--
Richard Biener <[email protected]>
SUSE Software Solutions Germany GmbH,
Frankenstrasse 146, 90461 Nuernberg, Germany;
GF: Jochen Jaser, Andrew McDonald, Abhinav Puri; (HRB 36809, AG Nuernberg)
