On Tue, 28 Jul 2026, Jakub Jelinek wrote:
> Hi!
>
> The following testcase ICEs on x86_64.
> The isel pass has a check for out of bounds constant index before
> optimizing into .VEC_SET, but it does it using
> // if index is a constant, then check the bounds
> poly_uint64 idx_poly;
> if (poly_int_tree_p (idx, &idx_poly))
> {
> poly_uint64 nelts = TYPE_VECTOR_SUBPARTS (TREE_TYPE (view_op0));
> if (known_gt (idx_poly, nelts))
> return false;
> }
> In the testcase below, idx is INTEGER_CST with long long type and
> negative value, that doesn't fit into poly_uint64, so we happily convert
> it into .VEC_SET.
>
> And another problem is that the x86 backend isn't trying to be careful
> and handle out of bounds elt gracefully (I think it could still in theory
> happen, if GIMPLE lets it through but e.g. something during expansion
> figures out the index is constant or whatever).
>
> The following patch does 2 things (both are enough to avoid the ICE).
> One is to also punt if idx is clearly negative (doesn't fit into
> poly_uint64 but fits into poly_int64).
> And the second fix is in the backend to avoid triggering UB at compile
> time by doing HOST_WIDE_INT_1U << elt etc. when elt is negative or too
> large. In order to avoid ICE, we need to emit something, so I emit
> a no-op move, out of bounds vector set shouldn't change anything in
> the target.
>
> Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?
>
> Now that I look at the patch, I think the if (known_gt (idx_poly, nelts))
> above is wrong as well, I think it should be
> if (known_ge (idx_poly, nelts))
> because vector set is out of bounds even when accessing nelts index.
> If you agree, I can post an incremental patch.
I agree to that.
> 2026-07-28 Jakub Jelinek <[email protected]>
>
> PR target/126446
> * gimple-isel.cc (gimple_expand_vec_set_extract_expr): Punt if
> idx doesn't fit into poly_uint64 but fits into poly_int64.
> * config/i386/i386-expand.cc (ix86_expand_vector_set): If elt is
> out of bounds, emit a no-op move.
>
> * gcc.target/i386/avx2-pr126446.c: New test.
>
> --- gcc/gimple-isel.cc.jj 2026-07-22 09:42:52.010559350 +0200
> +++ gcc/gimple-isel.cc 2026-07-28 13:07:46.663422172 +0200
> @@ -96,10 +96,11 @@ gimple_expand_vec_set_extract_expr (stru
> return false;
>
> tree op0 = TREE_OPERAND (ref, 0);
> - if (TREE_CODE (op0) == VIEW_CONVERT_EXPR && DECL_P (TREE_OPERAND (op0, 0))
> + if (TREE_CODE (op0) == VIEW_CONVERT_EXPR
> + && DECL_P (TREE_OPERAND (op0, 0))
> && VECTOR_TYPE_P (TREE_TYPE (TREE_OPERAND (op0, 0)))
> - && TYPE_MODE (TREE_TYPE (ref))
> - == TYPE_MODE (TREE_TYPE (TREE_TYPE (TREE_OPERAND (op0, 0)))))
> + && (TYPE_MODE (TREE_TYPE (ref))
> + == TYPE_MODE (TREE_TYPE (TREE_TYPE (TREE_OPERAND (op0, 0))))))
> {
> tree pos = TREE_OPERAND (ref, 1);
>
> @@ -114,6 +115,10 @@ gimple_expand_vec_set_extract_expr (stru
> if (known_gt (idx_poly, nelts))
> return false;
> }
> + else if (tree_fits_poly_int64_p (idx))
> + // if idx doesn't fit into poly_uint64, but fits into poly_int64,
> + // it must be negative
> + return false;
I'd say that iff poly_int_tree_p (idx, &idx_poly) checks it fits
poly_uint64 then if it is poly-int (like large uint128), it's
too large, so ...
else if (poly_int_tree_p (idx))
// if idx doesn't fit into poly_uint64 but is constant, it must
// be out of bounds
return false;
?
> machine_mode outermode = TYPE_MODE (TREE_TYPE (view_op0));
> machine_mode extract_mode = TYPE_MODE (TREE_TYPE (ref));
>
> --- gcc/config/i386/i386-expand.cc.jj 2026-07-24 11:52:32.590640836 +0200
> +++ gcc/config/i386/i386-expand.cc 2026-07-28 13:04:52.711678590 +0200
> @@ -19157,6 +19157,11 @@ ix86_expand_vector_set (bool mmx_ok, rtx
> machine_mode mmode = VOIDmode;
> rtx (*gen_blendm) (rtx, rtx, rtx, rtx);
>
> + if (elt < 0 || elt >= GET_MODE_NUNITS (mode))
> + {
> + emit_move_insn (target, target);
> + return;
> + }
> if (TARGET_SSE4_1 && mode == V4SImode && val == const0_rtx)
> {
> emit_insn (gen_sse4_1_insertps_v4si_zero (target, target,
> --- gcc/testsuite/gcc.target/i386/avx2-pr126446.c.jj 2026-07-28
> 13:15:10.373701905 +0200
> +++ gcc/testsuite/gcc.target/i386/avx2-pr126446.c 2026-07-28
> 13:14:54.663904357 +0200
> @@ -0,0 +1,14 @@
> +/* PR target/126446 */
> +/* { dg-do compile } */
> +/* { dg-options "-O1 -mavx2" } */
> +
> +typedef signed char V __attribute__((vector_size (16)));
> +
> +signed char
> +foo ()
> +{
> + V b = {};
> + long long c = ~2878966870562407444LL;
> + b[c] = 1;
> + return b[0];
> +}
>
> Jakub
>
>
--
Richard Biener <[email protected]>
SUSE Software Solutions Germany GmbH,
Frankenstrasse 146, 90461 Nuernberg, Germany;
GF: Jochen Jaser, Andrew McDonald, Abhinav Puri; (HRB 36809, AG Nuernberg)
