[x86 SSE] Avoid calling gen_lowpart for V2DImode on DDmode register.

[Roger Sayle]

Hongtao did warn me there might be corner cases in STV when
converting DImode SUBREGs to use V2DImode vector registers.
https://gcc.gnu.org/pipermail/gcc-patches/2026-May/718615.html
is exactly such a case, where with -m32 -O2 -march=cascadelake
we attempt to convert a comparison containing (subreg:DI (reg:DD) 0)
where the resulting call to gen_lowpart (V2DImode, ...) triggers
an ICE.  It's the lack of transitivity: DDmode may be SUBREGed to
DImode, and DImode may be SUBREGed to V2DImode, but DDmode can't
(directly) be SUBREGed to V2DImode.

Alas my attempts to reduce a testcase (simpler than the one
already in the testsuite) haven't be successful... The example
below contains the problematic SUBREG in a comparison, but in
this case, STV doesn't consider converting the chain profitable.

typedef float __decfloat64 __attribute__((mode(DD)));

__decfloat64 ext();
__decfloat64 x,y,z;

void foo()
{
  __decfloat64 t = ext();
  if (__builtin_memcmp((void*)&t,(void*)&x,sizeof(__decfloat64)) != 0
      && __builtin_memcmp((void*)&t,(void*)&y,sizeof(__decfloat64)) != 0
      && __builtin_memcmp((void*)&t,(void*)&z,sizeof(__decfloat64)) != 0) {
    x = t;
    y = t;
    z = t;
  }
}

Fortunately, the proposed fix is relatively straight forward, using
modes_tieable_p to check that the source pseudo is representable in
an SSE register.  When originally written, this code was expecting
V1TImode, V2DImode or V4SImode.  If this isn't a simple SUBREG, STV
can emit a move instruction (to preserve it), then operate on the
resulting pseudo (as usual).

This patch has been tested on x86_64-pc-linux-gnu with make bootstrap
and make -k check, both with and without --target_board=unix{-m32}
with no new failures.  Ok for mainline?


2026-06-14  Roger Sayle  <ro...@nextmovesoftware.com>

gcc/ChangeLog
        * config/i386/i386-features.cc (scalar_chain::convert_op): Check
        if the (DImode) SUBREG being converted by STV has an original mode
        that's tieable to the vector mode (V2DImode), if not (e.g. DDmode)
        emit this "conversion" as a separate move for reload to handle.


Thanks in advance (and apologies for any inconvenience),
Roger
--


diff --git a/gcc/config/i386/i386-features.cc b/gcc/config/i386/i386-features.cc
index 0694811e9da..1ad705cb356 100644
--- a/gcc/config/i386/i386-features.cc
+++ b/gcc/config/i386/i386-features.cc
@@ -1169,7 +1169,16 @@ scalar_chain::convert_op (rtx *op, rtx_insn *insn)
     {
       gcc_assert (SUBREG_P (*op));
       if (GET_MODE (*op) != vmode)
-       *op = gen_lowpart (vmode, *op);
+       {
+         if (!targetm.modes_tieable_p (vmode, GET_MODE (SUBREG_REG (*op))))
+           {
+             tmp = gen_reg_rtx (GET_MODE (*op));
+             emit_insn_before (gen_rtx_SET (tmp, *op), insn);
+             *op = gen_rtx_SUBREG (vmode, tmp, 0);
+           }
+         else
+           *op = gen_lowpart (vmode, *op);
+       }
     }
 }