https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128451
Bug ID: 128451
Summary: wrong code at -O0: vector lowering of a variable-mask
VEC_PERM_EXPR with 256 signed char elements does not
reduce the index (mask computed in the 8-bit element
type)
Product: gcc
Version: 17.0
Status: UNCONFIRMED
Keywords: wrong-code
Severity: normal
Priority: P3
Component: middle-end
Assignee: unassigned at gcc dot gnu.org
Reporter: lzy20000419 at outlook dot com
Target Milestone: ---
Split out from PR128080 (comment 5) at the maintainer's request. r17-5542 fixed
the match.pd VEC_PERM_EXPR merge there, but the same problem (an index range
handled in the 8-bit mask element type) is also in vector lowering. It needs
only -O0:
```c
typedef signed char V __attribute__((vector_size (256)));
V x, r, m;
__attribute__((noipa)) void f (void) { r = __builtin_shuffle (x, m); }
int main ()
{
for (int i = 0; i < 256; i++) { x[i] = i & 0x7f; m[i] = i; }
m[0] = -1; m[1] = -128;
f ();
if (r[0] != x[255] || r[1] != x[128] || r[2] != x[2])
__builtin_abort ();
return 0;
}
```
```
$ gcc -O0 t.c && ./a.out
Aborted
```
Expected: no abort. The mask is taken modulo N (256 here), so -1 selects x[255]
and -128 selects x[128]. -O2 aborts as well.
Tested on trunk 17.0.0 20261105 (2175b27369e5, includes r17-5542), 16.2.1 and
14.2.0. On Compiler Explorer it aborts with every x86-64 gcc I tried, 6.5.0
through trunk (7.5.0, 8.5.0, 9.5.0, 10.5.0, 11.4.0, 12.4.0, 13.4.0, 14.2.0,
15.3.0, 16.2.0), so this is not a regression. Godbolt (-O0 with execution on
trunk, 16.2, 14.2 and 6.5): https://godbolt.org/z/YTsEdoEKs
In lower_vec_perm (tree-vect-generic.cc:1652 on trunk), variable-mask path, the
index is reduced with `build_int_cst (mask_elt_type, elements - 1)` (:1741) and
the two-operand select uses `build_int_cst (mask_elt_type, elements)` (:1735).
With 256 elements and a signed char mask element these are -1 and 0, so the AND
does not reduce the index and the negative index reads outside the spilled
vector.
A possible one-line fix (not tested) is to convert the mask element to an
unsigned type of at least sizetype precision before the BIT_AND_EXPR and the
select comparison (as r17-5542 does with ssizetype), so that elements and
elements - 1 are representable.
Disclaimer: This test case was found with LLM-assisted targeted testing, and
the report was written with LLM assistance.
