Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
>> > The patch looks OK, but it would be nice to have some overall
>> > documentation on this length + bias thing somewhere. IMO this
>> > belongs to md.texi, but there's a PR about the patterns being
>> > an unordered mess without structure. Possibly amending the
>> > first paragraph is an option, short of subdividing the
>> > standard pattern name listing into logical parts.
>>
>> I've been trying to understand this myself, so lets see if my current
>> understanding is in the right direction. My understanding is that on some
>> targets len == 0 isn't a valid option, and so for those target a bias is
>> applied
>> such that the minimum len is biased so the operation is valid.
>>
>> At least that's my naïve understanding. How far off am I Robin?
>
> What adds to my confusion is that we pass down both 'len' and 'bias'
> but (IIRC for optimization reasons) apply the bias already to 'len'?
>
> So I thought vect_get_loop_len would always return biased len and
> to arrive at the actual "len" we have to un-apply the bias?
Apologies, I was out for most of the day so can catch up only now.
Given how later we are in the cycle the patch looks reasonable to me but I
would have preferred it to untangle things a bit.
The whole len handling is indeed confusing and very ripe for refactoring.
I'll take this on in stage 1.
Biasing/adjusting the length is an optimization. Subtracting 1 from the
length for every load and store was too costly when I benchmarked it.
Adding length and bias to the more advanced length-based load/store IFNs
(gather, lanes) was done for "symmetry". Neither s390 nor power support
those. As all other IFNs had a bias we also added it to vec_extract even
though that doesn't make a lot of sense. vec_extract as it's currently
defined can only reasonably operate on units/lanes and I hope we won't
ever see a vector ISA that cannot extract the first element but every other.
And even though we added the bias for comprehensiveness, only the code for
consecutive loads/stores (that s390 and power support) properly handles the
byte-based indexing at all, no other IFN does. So it's not like we have been
comprehensive but rather showed a bit of good will :)
IMHO what we need for legibility is
enum {
VECT_LEN_TYPE_BYTES,
VECT_LEN_TYPE_UNITS
} vect_len_type;
that we pass to record/get_loop_len instead of "factor" and "adjusted".
A vec_extract would ask for LEN_TYPE_UNITS, as would riscv. s390/power
would use LEN_TYPE_BYTES for their loads and stores.
We already have two "lens", the adjusted one and the regular one so all
that's left is gift-wrap the scaling from bytes to units and vice-versa
a bit nicer. What we could do as an optimization on s390 is keep the
regular len in units and the adjusted len in bytes.
--
Regards
Robin
Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
On Tue, 10 Feb 2026, Juergen Christ wrote: > > On Tue, 10 Feb 2026, Juergen Christ wrote: > > > > > > On Tue, 10 Feb 2026, Tamar Christina wrote: > > > > > > > > > > -Original Message- > > > > > > From: Richard Biener > > > > > > Sent: 10 February 2026 09:49 > > > > > > To: Juergen Christ > > > > > > Cc: [email protected]; Tamar Christina > > > > > > ; > > > > > > [email protected]; [email protected] > > > > > > Subject: Re: [PATCH] tree-optimization/122297 - fix load/store bias > > > > > > handling > > > > > > > > > > > > On Tue, 10 Feb 2026, Juergen Christ wrote: > > > > > > > > > > > > > When load/store with length is used and only QImode versions are > > > > > > > available, vectorizable_live_operation produces wrong results for > > > > > > > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if > > > > > > > bias-adjusted length should be used or not. > > > > > > > > > > > > > > PR tree-optimization/122297 > > > > > > > > > > > > > > gcc/ChangeLog: > > > > > > > > > > > > > > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust. > > > > > > > (vectorizable_induction): Adjust. > > > > > > > (vectorizable_live_operation_1): Adjust. > > > > > > > (vect_get_loop_len): Provide parameter to select bias-adjusted > > > > > > > length. > > > > > > > (vect_gen_loop_len_mask): Adjust. > > > > > > > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust. > > > > > > > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust. > > > > > > > (vectorizable_call): Adjust. > > > > > > > (vectorizable_operation): Adjust. > > > > > > > (vectorizable_store): Adjust. > > > > > > > (vectorizable_load): Adjust. > > > > > > > (vectorizable_condition): Adjust. > > > > > > > * tree-vectorizer.h (vect_get_loop_len): Add parameter. > > > > > > > > > > > > > > gcc/testsuite/ChangeLog: > > > > > > > > > > > > > > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo. > > > > > > > * gcc.dg/vect/nodump-extractlast-2.c: New test. > > > > > > > > > > > > > > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for > > > > > > > trunk? > > > > > > > > > > > > > > Signed-off-by: Juergen Christ > > > > > > > --- > > > > > > > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +- > > > > > > > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++ > > > > > > > gcc/tree-vect-loop.cc | 62 > > > > > > > +-- > > > > > > > gcc/tree-vect-stmts.cc| 24 +++ > > > > > > > gcc/tree-vectorizer.h | 2 +- > > > > > > > 5 files changed, 82 insertions(+), 31 deletions(-) > > > > > > > create mode 100644 > > > > > > > gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > > > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > > > index 980ac3e42188..83d8a38f13e3 100644 > > > > > > > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > > > @@ -1,4 +1,4 @@ > > > > > > > -/* Check for a bung in the treatment of > > > > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > > > > +/* Check for a bug in the treatment of > > > > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > > > > using VEC_EXTRACT. */ > > > > > > > /* { dg-require-effective-target vect_int } */ > > > > > > > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c >
Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
> On Tue, 10 Feb 2026, Juergen Christ wrote: > > > > On Tue, 10 Feb 2026, Tamar Christina wrote: > > > > > > > > -Original Message- > > > > > From: Richard Biener > > > > > Sent: 10 February 2026 09:49 > > > > > To: Juergen Christ > > > > > Cc: [email protected]; Tamar Christina > > > > > ; > > > > > [email protected]; [email protected] > > > > > Subject: Re: [PATCH] tree-optimization/122297 - fix load/store bias > > > > > handling > > > > > > > > > > On Tue, 10 Feb 2026, Juergen Christ wrote: > > > > > > > > > > > When load/store with length is used and only QImode versions are > > > > > > available, vectorizable_live_operation produces wrong results for > > > > > > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if > > > > > > bias-adjusted length should be used or not. > > > > > > > > > > > > PR tree-optimization/122297 > > > > > > > > > > > > gcc/ChangeLog: > > > > > > > > > > > > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust. > > > > > > (vectorizable_induction): Adjust. > > > > > > (vectorizable_live_operation_1): Adjust. > > > > > > (vect_get_loop_len): Provide parameter to select bias-adjusted > > > > > > length. > > > > > > (vect_gen_loop_len_mask): Adjust. > > > > > > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust. > > > > > > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust. > > > > > > (vectorizable_call): Adjust. > > > > > > (vectorizable_operation): Adjust. > > > > > > (vectorizable_store): Adjust. > > > > > > (vectorizable_load): Adjust. > > > > > > (vectorizable_condition): Adjust. > > > > > > * tree-vectorizer.h (vect_get_loop_len): Add parameter. > > > > > > > > > > > > gcc/testsuite/ChangeLog: > > > > > > > > > > > > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo. > > > > > > * gcc.dg/vect/nodump-extractlast-2.c: New test. > > > > > > > > > > > > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk? > > > > > > > > > > > > Signed-off-by: Juergen Christ > > > > > > --- > > > > > > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +- > > > > > > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++ > > > > > > gcc/tree-vect-loop.cc | 62 > > > > > > +-- > > > > > > gcc/tree-vect-stmts.cc| 24 +++ > > > > > > gcc/tree-vectorizer.h | 2 +- > > > > > > 5 files changed, 82 insertions(+), 31 deletions(-) > > > > > > create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > > index 980ac3e42188..83d8a38f13e3 100644 > > > > > > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > > @@ -1,4 +1,4 @@ > > > > > > -/* Check for a bung in the treatment of > > > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > > > +/* Check for a bug in the treatment of > > > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > > > using VEC_EXTRACT. */ > > > > > > /* { dg-require-effective-target vect_int } */ > > > > > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > > new file mode 100644 > > > > > > index ..9697687c1084 > > > > > > --- /dev/null > > > > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > > @@ -0,0 +1,23 @@ > > > > > > +/* Check for a bug in the treatment of > >
Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
On Tue, 10 Feb 2026, Juergen Christ wrote:
> > On Tue, 10 Feb 2026, Juergen Christ wrote:
> >
> > > When load/store with length is used and only QImode versions are
> > > available, vectorizable_live_operation produces wrong results for
> > > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if
> > > bias-adjusted length should be used or not.
> > >
> > > PR tree-optimization/122297
> > >
> > > gcc/ChangeLog:
> > >
> > > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust.
> > > (vectorizable_induction): Adjust.
> > > (vectorizable_live_operation_1): Adjust.
> > > (vect_get_loop_len): Provide parameter to select bias-adjusted
> > > length.
> > > (vect_gen_loop_len_mask): Adjust.
> > > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust.
> > > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust.
> > > (vectorizable_call): Adjust.
> > > (vectorizable_operation): Adjust.
> > > (vectorizable_store): Adjust.
> > > (vectorizable_load): Adjust.
> > > (vectorizable_condition): Adjust.
> > > * tree-vectorizer.h (vect_get_loop_len): Add parameter.
> > >
> > > gcc/testsuite/ChangeLog:
> > >
> > > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo.
> > > * gcc.dg/vect/nodump-extractlast-2.c: New test.
> > >
> > > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk?
> > >
> > > Signed-off-by: Juergen Christ
> > > ---
> > > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +-
> > > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++
> > > gcc/tree-vect-loop.cc | 62 +--
> > > gcc/tree-vect-stmts.cc| 24 +++
> > > gcc/tree-vectorizer.h | 2 +-
> > > 5 files changed, 82 insertions(+), 31 deletions(-)
> > > create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> > >
> > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > > index 980ac3e42188..83d8a38f13e3 100644
> > > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > > @@ -1,4 +1,4 @@
> > > -/* Check for a bung in the treatment of
> > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when
> > > +/* Check for a bug in the treatment of
> > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when
> > > using VEC_EXTRACT. */
> > > /* { dg-require-effective-target vect_int } */
> > >
> > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> > > new file mode 100644
> > > index ..9697687c1084
> > > --- /dev/null
> > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> > > @@ -0,0 +1,23 @@
> > > +/* Check for a bug in the treatment of
> > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when
> > > + using VEC_EXTRACT. This variant uses .LEN_LOAD which might use QImode
> > > + vectors during load, but SImode vectors for the extraction. */
> > > +int __attribute__ ((noinline, noclone))
> > > +test_int (int *x, int n, int value)
> > > +{
> > > + int last;
> > > + for (int j = 0; j < n; ++j)
> > > +{
> > > + last = x[j];
> > > + x[j] = last * value;
> > > +}
> > > + return last;
> > > +}
> > > +
> > > +int
> > > +main ()
> > > +{
> > > + int arr[] = {1,2,3,4,5,1};
> > > + if (test_int (arr, sizeof (arr) / sizeof (arr[0]), 42) != 1)
> > > +__builtin_abort();
> > > + return 0;
> > > +}
> > > diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
> > > index 8e60a433596f..8e40f0ceb93b 100644
> > > --- a/gcc/tree-vect-loop.cc
> > > +++ b/gcc/tree-vect-loop.cc
> > > @@ -6503,8 +6503,11 @@ vectorize_fold_left_reduction (loop_vec_info
> > > loop_vinfo,
> > > mask = vec_opmask[i];
> > >if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo))
> > > {
> > > + /* ??? Why do we use LOAD_STORE_BIAS here when we do not do a load or
> > > + a store? Shouldn't we instead just ensure that LEN represents the
> > > + number of elements in the vector? */
> >
> > Yes. I think the comment is unwarranted.
>
> Ups. Sorry, that comment should NOT be part of the patch. I will remove it.
>
> >
> > > len = vect_get_loop_len (loop_vinfo, gsi, lens, vec_num, vectype_in,
> > > -i, 1);
> > > +i, 1, false);
> > > signed char biasval = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo);
> > > bias = build_int_cst (intQI_type_node, biasval);
> > > if (!is_cond_op)
> > > @@ -9885,7 +9888,7 @@ vectorizable_induction (loop_vec_info loop_vinfo,
> > > _21 = vect_vec_iv_.6_22 + vect_cst__22; */
> > > vec_loop_lens *lens = &LOOP_VINFO_LENS (loop_vinfo);
> > > tree len = vect_get_loop_len (loop_vinfo, NULL, lens, 1,
> > > - vectype, 0, 0);
> > > + vectype, 0, 0, false);
> > >
Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
On Tue, 10 Feb 2026, Juergen Christ wrote: > > On Tue, 10 Feb 2026, Tamar Christina wrote: > > > > > > -Original Message- > > > > From: Richard Biener > > > > Sent: 10 February 2026 09:49 > > > > To: Juergen Christ > > > > Cc: [email protected]; Tamar Christina ; > > > > [email protected]; [email protected] > > > > Subject: Re: [PATCH] tree-optimization/122297 - fix load/store bias > > > > handling > > > > > > > > On Tue, 10 Feb 2026, Juergen Christ wrote: > > > > > > > > > When load/store with length is used and only QImode versions are > > > > > available, vectorizable_live_operation produces wrong results for > > > > > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if > > > > > bias-adjusted length should be used or not. > > > > > > > > > > PR tree-optimization/122297 > > > > > > > > > > gcc/ChangeLog: > > > > > > > > > > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust. > > > > > (vectorizable_induction): Adjust. > > > > > (vectorizable_live_operation_1): Adjust. > > > > > (vect_get_loop_len): Provide parameter to select bias-adjusted > > > > > length. > > > > > (vect_gen_loop_len_mask): Adjust. > > > > > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust. > > > > > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust. > > > > > (vectorizable_call): Adjust. > > > > > (vectorizable_operation): Adjust. > > > > > (vectorizable_store): Adjust. > > > > > (vectorizable_load): Adjust. > > > > > (vectorizable_condition): Adjust. > > > > > * tree-vectorizer.h (vect_get_loop_len): Add parameter. > > > > > > > > > > gcc/testsuite/ChangeLog: > > > > > > > > > > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo. > > > > > * gcc.dg/vect/nodump-extractlast-2.c: New test. > > > > > > > > > > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk? > > > > > > > > > > Signed-off-by: Juergen Christ > > > > > --- > > > > > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +- > > > > > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++ > > > > > gcc/tree-vect-loop.cc | 62 > > > > > +-- > > > > > gcc/tree-vect-stmts.cc| 24 +++ > > > > > gcc/tree-vectorizer.h | 2 +- > > > > > 5 files changed, 82 insertions(+), 31 deletions(-) > > > > > create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > index 980ac3e42188..83d8a38f13e3 100644 > > > > > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > > @@ -1,4 +1,4 @@ > > > > > -/* Check for a bung in the treatment of > > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > > +/* Check for a bug in the treatment of > > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > > using VEC_EXTRACT. */ > > > > > /* { dg-require-effective-target vect_int } */ > > > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > new file mode 100644 > > > > > index ..9697687c1084 > > > > > --- /dev/null > > > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > @@ -0,0 +1,23 @@ > > > > > +/* Check for a bug in the treatment of > > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > > + using VEC_EXTRACT. This variant uses .LEN_LOAD which might use > > > > QImode > > > > > + vectors during load, but SImode vectors for the extraction. */ > > > > > +int __attribute__ ((noinline, noclone)) > > > > > +test_int (int *x, int n, int
Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
> On Tue, 10 Feb 2026, Juergen Christ wrote:
>
> > When load/store with length is used and only QImode versions are
> > available, vectorizable_live_operation produces wrong results for
> > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if
> > bias-adjusted length should be used or not.
> >
> > PR tree-optimization/122297
> >
> > gcc/ChangeLog:
> >
> > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust.
> > (vectorizable_induction): Adjust.
> > (vectorizable_live_operation_1): Adjust.
> > (vect_get_loop_len): Provide parameter to select bias-adjusted
> > length.
> > (vect_gen_loop_len_mask): Adjust.
> > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust.
> > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust.
> > (vectorizable_call): Adjust.
> > (vectorizable_operation): Adjust.
> > (vectorizable_store): Adjust.
> > (vectorizable_load): Adjust.
> > (vectorizable_condition): Adjust.
> > * tree-vectorizer.h (vect_get_loop_len): Add parameter.
> >
> > gcc/testsuite/ChangeLog:
> >
> > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo.
> > * gcc.dg/vect/nodump-extractlast-2.c: New test.
> >
> > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk?
> >
> > Signed-off-by: Juergen Christ
> > ---
> > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +-
> > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++
> > gcc/tree-vect-loop.cc | 62 +--
> > gcc/tree-vect-stmts.cc| 24 +++
> > gcc/tree-vectorizer.h | 2 +-
> > 5 files changed, 82 insertions(+), 31 deletions(-)
> > create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> >
> > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > index 980ac3e42188..83d8a38f13e3 100644
> > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> > @@ -1,4 +1,4 @@
> > -/* Check for a bung in the treatment of LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS
> > when
> > +/* Check for a bug in the treatment of LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS
> > when
> > using VEC_EXTRACT. */
> > /* { dg-require-effective-target vect_int } */
> >
> > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> > new file mode 100644
> > index ..9697687c1084
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> > @@ -0,0 +1,23 @@
> > +/* Check for a bug in the treatment of LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS
> > when
> > + using VEC_EXTRACT. This variant uses .LEN_LOAD which might use QImode
> > + vectors during load, but SImode vectors for the extraction. */
> > +int __attribute__ ((noinline, noclone))
> > +test_int (int *x, int n, int value)
> > +{
> > + int last;
> > + for (int j = 0; j < n; ++j)
> > +{
> > + last = x[j];
> > + x[j] = last * value;
> > +}
> > + return last;
> > +}
> > +
> > +int
> > +main ()
> > +{
> > + int arr[] = {1,2,3,4,5,1};
> > + if (test_int (arr, sizeof (arr) / sizeof (arr[0]), 42) != 1)
> > +__builtin_abort();
> > + return 0;
> > +}
> > diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
> > index 8e60a433596f..8e40f0ceb93b 100644
> > --- a/gcc/tree-vect-loop.cc
> > +++ b/gcc/tree-vect-loop.cc
> > @@ -6503,8 +6503,11 @@ vectorize_fold_left_reduction (loop_vec_info
> > loop_vinfo,
> > mask = vec_opmask[i];
> >if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo))
> > {
> > + /* ??? Why do we use LOAD_STORE_BIAS here when we do not do a load or
> > +a store? Shouldn't we instead just ensure that LEN represents the
> > +number of elements in the vector? */
>
> Yes. I think the comment is unwarranted.
Ups. Sorry, that comment should NOT be part of the patch. I will remove it.
>
> > len = vect_get_loop_len (loop_vinfo, gsi, lens, vec_num, vectype_in,
> > - i, 1);
> > + i, 1, false);
> > signed char biasval = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo);
> > bias = build_int_cst (intQI_type_node, biasval);
> > if (!is_cond_op)
> > @@ -9885,7 +9888,7 @@ vectorizable_induction (loop_vec_info loop_vinfo,
> >_21 = vect_vec_iv_.6_22 + vect_cst__22; */
> > vec_loop_lens *lens = &LOOP_VINFO_LENS (loop_vinfo);
> > tree len = vect_get_loop_len (loop_vinfo, NULL, lens, 1,
> > - vectype, 0, 0);
> > + vectype, 0, 0, false);
> > if (SCALAR_FLOAT_TYPE_P (stept))
> > expr = gimple_build (&stmts, FLOAT_EXPR, stept, len);
> > else
> > @@ -10032,7 +10035,7 @@ vectorizable_live_operation_1 (loop_vec_info
> > loop_vin
Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
> On Tue, 10 Feb 2026, Tamar Christina wrote: > > > > -Original Message- > > > From: Richard Biener > > > Sent: 10 February 2026 09:49 > > > To: Juergen Christ > > > Cc: [email protected]; Tamar Christina ; > > > [email protected]; [email protected] > > > Subject: Re: [PATCH] tree-optimization/122297 - fix load/store bias > > > handling > > > > > > On Tue, 10 Feb 2026, Juergen Christ wrote: > > > > > > > When load/store with length is used and only QImode versions are > > > > available, vectorizable_live_operation produces wrong results for > > > > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if > > > > bias-adjusted length should be used or not. > > > > > > > > PR tree-optimization/122297 > > > > > > > > gcc/ChangeLog: > > > > > > > > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust. > > > > (vectorizable_induction): Adjust. > > > > (vectorizable_live_operation_1): Adjust. > > > > (vect_get_loop_len): Provide parameter to select bias-adjusted > > > > length. > > > > (vect_gen_loop_len_mask): Adjust. > > > > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust. > > > > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust. > > > > (vectorizable_call): Adjust. > > > > (vectorizable_operation): Adjust. > > > > (vectorizable_store): Adjust. > > > > (vectorizable_load): Adjust. > > > > (vectorizable_condition): Adjust. > > > > * tree-vectorizer.h (vect_get_loop_len): Add parameter. > > > > > > > > gcc/testsuite/ChangeLog: > > > > > > > > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo. > > > > * gcc.dg/vect/nodump-extractlast-2.c: New test. > > > > > > > > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk? > > > > > > > > Signed-off-by: Juergen Christ > > > > --- > > > > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +- > > > > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++ > > > > gcc/tree-vect-loop.cc | 62 +-- > > > > gcc/tree-vect-stmts.cc| 24 +++ > > > > gcc/tree-vectorizer.h | 2 +- > > > > 5 files changed, 82 insertions(+), 31 deletions(-) > > > > create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > index 980ac3e42188..83d8a38f13e3 100644 > > > > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > > @@ -1,4 +1,4 @@ > > > > -/* Check for a bung in the treatment of > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > +/* Check for a bug in the treatment of > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > using VEC_EXTRACT. */ > > > > /* { dg-require-effective-target vect_int } */ > > > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > new file mode 100644 > > > > index ..9697687c1084 > > > > --- /dev/null > > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > @@ -0,0 +1,23 @@ > > > > +/* Check for a bug in the treatment of > > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > > + using VEC_EXTRACT. This variant uses .LEN_LOAD which might use > > > QImode > > > > + vectors during load, but SImode vectors for the extraction. */ > > > > +int __attribute__ ((noinline, noclone)) > > > > +test_int (int *x, int n, int value) > > > > +{ > > > > + int last; > > > > + for (int j = 0; j < n; ++j) > > > > +{ > > > > + last = x[j]; > > > > + x[j] = last * value; > > > > +} > > > > + return last; > > > > +} > > > > + > > > > +int > > > > +main () > > > > +{ > > > >
RE: [PATCH] tree-optimization/122297 - fix load/store bias handling
On Tue, 10 Feb 2026, Tamar Christina wrote: > > -Original Message- > > From: Richard Biener > > Sent: 10 February 2026 09:49 > > To: Juergen Christ > > Cc: [email protected]; Tamar Christina ; > > [email protected]; [email protected] > > Subject: Re: [PATCH] tree-optimization/122297 - fix load/store bias handling > > > > On Tue, 10 Feb 2026, Juergen Christ wrote: > > > > > When load/store with length is used and only QImode versions are > > > available, vectorizable_live_operation produces wrong results for > > > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if > > > bias-adjusted length should be used or not. > > > > > > PR tree-optimization/122297 > > > > > > gcc/ChangeLog: > > > > > > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust. > > > (vectorizable_induction): Adjust. > > > (vectorizable_live_operation_1): Adjust. > > > (vect_get_loop_len): Provide parameter to select bias-adjusted > > > length. > > > (vect_gen_loop_len_mask): Adjust. > > > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust. > > > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust. > > > (vectorizable_call): Adjust. > > > (vectorizable_operation): Adjust. > > > (vectorizable_store): Adjust. > > > (vectorizable_load): Adjust. > > > (vectorizable_condition): Adjust. > > > * tree-vectorizer.h (vect_get_loop_len): Add parameter. > > > > > > gcc/testsuite/ChangeLog: > > > > > > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo. > > > * gcc.dg/vect/nodump-extractlast-2.c: New test. > > > > > > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk? > > > > > > Signed-off-by: Juergen Christ > > > --- > > > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +- > > > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++ > > > gcc/tree-vect-loop.cc | 62 +-- > > > gcc/tree-vect-stmts.cc| 24 +++ > > > gcc/tree-vectorizer.h | 2 +- > > > 5 files changed, 82 insertions(+), 31 deletions(-) > > > create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > index 980ac3e42188..83d8a38f13e3 100644 > > > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > > @@ -1,4 +1,4 @@ > > > -/* Check for a bung in the treatment of > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > +/* Check for a bug in the treatment of > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > using VEC_EXTRACT. */ > > > /* { dg-require-effective-target vect_int } */ > > > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > new file mode 100644 > > > index ..9697687c1084 > > > --- /dev/null > > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > @@ -0,0 +1,23 @@ > > > +/* Check for a bug in the treatment of > > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > > + using VEC_EXTRACT. This variant uses .LEN_LOAD which might use > > QImode > > > + vectors during load, but SImode vectors for the extraction. */ > > > +int __attribute__ ((noinline, noclone)) > > > +test_int (int *x, int n, int value) > > > +{ > > > + int last; > > > + for (int j = 0; j < n; ++j) > > > +{ > > > + last = x[j]; > > > + x[j] = last * value; > > > +} > > > + return last; > > > +} > > > + > > > +int > > > +main () > > > +{ > > > + int arr[] = {1,2,3,4,5,1}; > > > + if (test_int (arr, sizeof (arr) / sizeof (arr[0]), 42) != 1) > > > +__builtin_abort(); > > > + return 0; > > > +} > > > diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc > > > index 8e60a433596f..8e40f0ceb93b 100644 > > > --- a/gcc/tree-vect-loop.cc > > > +++ b/gcc/tree-vect-loop.cc > > > @@ -6503,8 +6503,11 @@ vectorize_fold_left_reduction (loop_vec_info > > loop_vinfo, > > > mask = vec_opmask[i]; > > >if (LOOP_VINFO_FU
RE: [PATCH] tree-optimization/122297 - fix load/store bias handling
> -Original Message- > From: Richard Biener > Sent: 10 February 2026 09:49 > To: Juergen Christ > Cc: [email protected]; Tamar Christina ; > [email protected]; [email protected] > Subject: Re: [PATCH] tree-optimization/122297 - fix load/store bias handling > > On Tue, 10 Feb 2026, Juergen Christ wrote: > > > When load/store with length is used and only QImode versions are > > available, vectorizable_live_operation produces wrong results for > > VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if > > bias-adjusted length should be used or not. > > > > PR tree-optimization/122297 > > > > gcc/ChangeLog: > > > > * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust. > > (vectorizable_induction): Adjust. > > (vectorizable_live_operation_1): Adjust. > > (vect_get_loop_len): Provide parameter to select bias-adjusted > > length. > > (vect_gen_loop_len_mask): Adjust. > > (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust. > > * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust. > > (vectorizable_call): Adjust. > > (vectorizable_operation): Adjust. > > (vectorizable_store): Adjust. > > (vectorizable_load): Adjust. > > (vectorizable_condition): Adjust. > > * tree-vectorizer.h (vect_get_loop_len): Add parameter. > > > > gcc/testsuite/ChangeLog: > > > > * gcc.dg/vect/nodump-extractlast-1.c: Fix typo. > > * gcc.dg/vect/nodump-extractlast-2.c: New test. > > > > Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk? > > > > Signed-off-by: Juergen Christ > > --- > > .../gcc.dg/vect/nodump-extractlast-1.c| 2 +- > > .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++ > > gcc/tree-vect-loop.cc | 62 +-- > > gcc/tree-vect-stmts.cc| 24 +++ > > gcc/tree-vectorizer.h | 2 +- > > 5 files changed, 82 insertions(+), 31 deletions(-) > > create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > index 980ac3e42188..83d8a38f13e3 100644 > > --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c > > @@ -1,4 +1,4 @@ > > -/* Check for a bung in the treatment of > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > +/* Check for a bug in the treatment of > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > using VEC_EXTRACT. */ > > /* { dg-require-effective-target vect_int } */ > > > > diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > new file mode 100644 > > index ..9697687c1084 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c > > @@ -0,0 +1,23 @@ > > +/* Check for a bug in the treatment of > LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when > > + using VEC_EXTRACT. This variant uses .LEN_LOAD which might use > QImode > > + vectors during load, but SImode vectors for the extraction. */ > > +int __attribute__ ((noinline, noclone)) > > +test_int (int *x, int n, int value) > > +{ > > + int last; > > + for (int j = 0; j < n; ++j) > > +{ > > + last = x[j]; > > + x[j] = last * value; > > +} > > + return last; > > +} > > + > > +int > > +main () > > +{ > > + int arr[] = {1,2,3,4,5,1}; > > + if (test_int (arr, sizeof (arr) / sizeof (arr[0]), 42) != 1) > > +__builtin_abort(); > > + return 0; > > +} > > diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc > > index 8e60a433596f..8e40f0ceb93b 100644 > > --- a/gcc/tree-vect-loop.cc > > +++ b/gcc/tree-vect-loop.cc > > @@ -6503,8 +6503,11 @@ vectorize_fold_left_reduction (loop_vec_info > loop_vinfo, > > mask = vec_opmask[i]; > >if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) > > { > > + /* ??? Why do we use LOAD_STORE_BIAS here when we do not do a > load or > > +a store? Shouldn't we instead just ensure that LEN represents the > > +number of elements in the vector? */ > > Yes. I think the comment is unwarranted. > > > len = vect_get_loop_len (loop_vinfo, gsi, lens, vec_num, vectype_in, > > - i, 1); > > +
Re: [PATCH] tree-optimization/122297 - fix load/store bias handling
On Tue, 10 Feb 2026, Juergen Christ wrote:
> When load/store with length is used and only QImode versions are
> available, vectorizable_live_operation produces wrong results for
> VEC_EXTRACT. Provide a flag to vect_get_loop_len to specify if
> bias-adjusted length should be used or not.
>
> PR tree-optimization/122297
>
> gcc/ChangeLog:
>
> * tree-vect-loop.cc (vectorize_fold_left_reduction): Adjust.
> (vectorizable_induction): Adjust.
> (vectorizable_live_operation_1): Adjust.
> (vect_get_loop_len): Provide parameter to select bias-adjusted
> length.
> (vect_gen_loop_len_mask): Adjust.
> (vect_update_ivs_after_vectorizer_for_early_breaks): Adjust.
> * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Adjust.
> (vectorizable_call): Adjust.
> (vectorizable_operation): Adjust.
> (vectorizable_store): Adjust.
> (vectorizable_load): Adjust.
> (vectorizable_condition): Adjust.
> * tree-vectorizer.h (vect_get_loop_len): Add parameter.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.dg/vect/nodump-extractlast-1.c: Fix typo.
> * gcc.dg/vect/nodump-extractlast-2.c: New test.
>
> Bootstrapped and regtested on s390, x86, and ppc64le. Ok for trunk?
>
> Signed-off-by: Juergen Christ
> ---
> .../gcc.dg/vect/nodump-extractlast-1.c| 2 +-
> .../gcc.dg/vect/nodump-extractlast-2.c| 23 +++
> gcc/tree-vect-loop.cc | 62 +--
> gcc/tree-vect-stmts.cc| 24 +++
> gcc/tree-vectorizer.h | 2 +-
> 5 files changed, 82 insertions(+), 31 deletions(-)
> create mode 100644 gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
>
> diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> index 980ac3e42188..83d8a38f13e3 100644
> --- a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-1.c
> @@ -1,4 +1,4 @@
> -/* Check for a bung in the treatment of LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS
> when
> +/* Check for a bug in the treatment of LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS
> when
> using VEC_EXTRACT. */
> /* { dg-require-effective-target vect_int } */
>
> diff --git a/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> new file mode 100644
> index ..9697687c1084
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/vect/nodump-extractlast-2.c
> @@ -0,0 +1,23 @@
> +/* Check for a bug in the treatment of LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS
> when
> + using VEC_EXTRACT. This variant uses .LEN_LOAD which might use QImode
> + vectors during load, but SImode vectors for the extraction. */
> +int __attribute__ ((noinline, noclone))
> +test_int (int *x, int n, int value)
> +{
> + int last;
> + for (int j = 0; j < n; ++j)
> +{
> + last = x[j];
> + x[j] = last * value;
> +}
> + return last;
> +}
> +
> +int
> +main ()
> +{
> + int arr[] = {1,2,3,4,5,1};
> + if (test_int (arr, sizeof (arr) / sizeof (arr[0]), 42) != 1)
> +__builtin_abort();
> + return 0;
> +}
> diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
> index 8e60a433596f..8e40f0ceb93b 100644
> --- a/gcc/tree-vect-loop.cc
> +++ b/gcc/tree-vect-loop.cc
> @@ -6503,8 +6503,11 @@ vectorize_fold_left_reduction (loop_vec_info
> loop_vinfo,
> mask = vec_opmask[i];
>if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo))
> {
> + /* ??? Why do we use LOAD_STORE_BIAS here when we do not do a load or
> + a store? Shouldn't we instead just ensure that LEN represents the
> + number of elements in the vector? */
Yes. I think the comment is unwarranted.
> len = vect_get_loop_len (loop_vinfo, gsi, lens, vec_num, vectype_in,
> -i, 1);
> +i, 1, false);
> signed char biasval = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo);
> bias = build_int_cst (intQI_type_node, biasval);
> if (!is_cond_op)
> @@ -9885,7 +9888,7 @@ vectorizable_induction (loop_vec_info loop_vinfo,
> _21 = vect_vec_iv_.6_22 + vect_cst__22; */
> vec_loop_lens *lens = &LOOP_VINFO_LENS (loop_vinfo);
> tree len = vect_get_loop_len (loop_vinfo, NULL, lens, 1,
> - vectype, 0, 0);
> + vectype, 0, 0, false);
> if (SCALAR_FLOAT_TYPE_P (stept))
> expr = gimple_build (&stmts, FLOAT_EXPR, stept, len);
> else
> @@ -10032,7 +10035,7 @@ vectorizable_live_operation_1 (loop_vec_info
> loop_vinfo, basic_block exit_bb,
> {
>/* Emit:
>
> - SCALAR_RES = VEC_EXTRACT
> + SCALAR_RES = VEC_EXTRACT
>
>where VEC_LHS is the vectorized live-out result, LEN is the length of
>the vector, BIAS is the load
