Issue 240874
Summary [X86][RegisterCoalescer] `<2 x double>` loses its upper half across a call after the coalescer routes it through an `FR64` vreg
Labels new issue
Assignees
Reporter gbaraldi
    `llc` miscompiles the function below on x86-64. A `<2 x double>` that is live
across a call is spilled with an 8-byte `movsd` and reloaded with an 8-byte
`movsd`, so its upper element comes back as `0.0`.

It reproduces with plain `llc -O2` and no target features on 16.0.6,
18.1.3, 19.1.1, 20.1.2, 21.1.8, 22.1.8 and main (cdda042f213c).
`-join-liveintervals=false` and `-O0` give correct code.

## Reproducer

`repro.ll`:

```llvm
target triple = "x86_64-unknown-linux-gnu"

declare void @clobber()

define <2 x double> @f(<2 x double> %x, double %t0, double %s0, i1 %c1, i1 %c2, i1 %c3, ptr %p, ptr %q) {
entry:
  %v0 = insertelement <2 x double> poison, double %s0, i64 0
  store <2 x double> %v0, ptr %q
  br i1 %c1, label %lo, label %exit

call:

  br i1 %c3, label %exit, label %ret2

lo:
  %x0 = extractelement <2 x double> %x, i64 0
  br i1 %c2, label %call, label %ret1

ret1:
  store double %x0, ptr %p
  ret <2 x double> zeroinitializer

ret2:
  ret <2 x double> zeroinitializer

exit:
  %t = phi double [ %t0, %entry ], [ %x0, %call ]
  %s = phi double [ %s0, %entry ], [ %x0, %call ]
  %v = phi <2 x double> [ %v0, %entry ], [ %x, %call ]
 store double %t, ptr %p
  %p1 = getelementptr double, ptr %p, i64 1
  store double %s, ptr %p1
  ret <2 x double> %v
}
```

On the path `entry -> lo -> call -> exit` the funct
unchanged.

`main.c`:

```c
#include <emmintrin.h>
#include <stdio.h>

__m128d f(__m128d x, double t0, double s0, _Bool c1, _Bool c2, _Bool c3,
          double *p, __m128d *q);
void clobber(void) {}

int main(void) {
  double p[2];
  __m128d q;
  /* entry -> lo -> call -> exit: the result must b
  __m128d r = f(_mm_set_pd(2.0, 1.0), 3.0, 4.0, 1, 1, 1, p, &q);
  printf("f returned <%g, %g>, expected <1, 2>\n", r[0], r[1]);
  return !(r[0] == 1.0 && r[1] == 2.0);
}
```

```
$ llc -O2 repro.ll -filetype=obj -o repro.o && clang main.c repro.o && ./a.out
f returned <1, 0>, expected <1, 2>
$ llc -O2 -join-liveintervals=false repro.ll -filetype=obj -o repro.o && clang main.c repro.o && ./a.out
f returned <1, 2>, expected <1, 2>
```

`llc -O2 repro.ll` on main, with `%x` in `xmm0`:

```asm
# %bb.4: # %call
        pushq   %rbx
        subq    $16, %rsp
 movq    %rcx, 8(%rsp)                   # 8
        movl    %edx, %ebx
 movsd   %xmm0, (%rsp)                   # 8-byte Spill    <- only x[0] is saved
        callq   clobber@PLT
        movsd   (%rsp), %xmm1 # 8
                                        # xmm1 = mem[0],zero
 movq    8(%rsp), %rcx                   # 8-byte Reload
        testb $1, %bl
        movaps  %xmm1, %xmm2
        leaq    16(%rsp), %rsp
 popq    %rbx
        je      .LBB0_3
.LBB0_5: # %exit
        movsd   %xmm1, (%rcx)
        movsd   %xmm2, 8(%rcx)
 movaps  %xmm2, %xmm0                    # <- returned as the vector: <x[0], 0.0>
        retq
```


Some not very sloppy AI output ahead, but sounds reasonable to my non backend eyes.
## What happens

The MIR going into the register coalescer is fine (xmm copies only):

```
bb.0.entry:
 %7:fr64   = COPY $xmm2        ; s0
  %6:fr64   = COPY $xmm1        ; t0
 %5:vr128  = COPY $xmm0        ; x
  %0:vr128  = COPY %7           ; v0 = scalar_to_vector(s0)
  ...
  %20:fr64  = COPY %6           ; t
  %21:fr64 = COPY %7           ; s
  %22:vr128 = COPY %0           ; v
bb.1.call:
 CALL64pcrel32 @clobber
  %20:fr64  = COPY %1           ; t = x[0]
 %21:fr64  = COPY %1           ; s = x[0]
  %22:vr128 = COPY %5           ; v = x, all 128 bi
bb.2.lo:
  %1:fr64   = COPY %5           ; x0 = x[0]
```

The coalescer then does the following:

1. It joins `%1` into `%20`. `%20:fr64` now has two
   `entry` and `COPY %5` in `lo`. Joining `%20` wit
   with interference, so `%20` stays a separate `fr64` register.
2. It joins `%0`, `%7` and `%21` into one `vr128` r
   in `entry`.
3. It joins that register with `%22` through `%22:v
   In `call`, both registers are defined while the other is live:

   ```
   %21:vr128 = COPY %20:fr64    ; where %20:fr64 =
   %22:vr128 = COPY %5:vr128
   ```

 `JoinVals::analyzeValue` asks `valuesIdentical` about the two.
 `followCopyChain` walks `%21 <- %20 <- %5` and `%22 <- %5`, reaches the same
   def of `%5` on both sides, and reports the value
   coalescer resolves the conflict as `CR_Erase` and deletes
   `%22:vr128 = COPY %5:vr128`.

```
228B    %22:vr128 = COPY %21:vr128
        Considering merging to VR128 with %21 in %2
                RHS = %21 [96r,256B:2)[348r,384B:1)[624B,676r:0) 0@624B-phi 1@348r 2@96r
 LHS = %22 [228r,256B:2)[356r,384B:156r 2@228r
                merge %22:1@356r into %21:1@348r --> @348r
                merge %22:2@228r into %21:2@96r --> @96r
                merge %21:0@624B into %22:0@624B --> @624B
                erased: 356r    %22:vr128 = COPY %5
 erased: 228r    %22:vr128 = COPY %21:vr128
```

After the pass the vector reaches `%22` only throug
and live across the call:

```
bb.1.call:
 CALL64pcrel32 @clobber
  %22:vr128 = COPY %20          ; v = x, but throug
bb.2.lo:
  %20:fr64 = COPY %5            ; last use of %5
bb.5.exit:
  $xmm0 = COPY %22
```

`FR64` has a 64-bit spill size, so the allocator sa
`movsd`, and the upper half of `x` is gone.

`followCopyChain` looks through any full `COPY` between virtual registers
(`MI->isFullCopy()`) without looking at register cl
`FR64` and `VR128` contain the same physical registers, and isel uses plain
`COPY`s between them for `extractelement ..., 0` and `scalar_to_vector`. A
`vr128 -> fr64 -> vr128` chain therefore is not val
`fr64` link can be spilled, but `valuesIdentical` treats it as one.

One possible direction (not tested): have `followCopyChain` stop at a copy
whose source register class is narrower than the cl
being joined to (`CP.getNewRC()`), so that a value is never proven identical
through an intermediate register that cannot hold all of it.

## MIR-only reproducer

The same join, reduced by hand to the coalescer inp

`coalescer.mir`:

```
# RUN: llc -mtriple=x86_64-unknown-linux-gnu -run-pass=register-coalescer -verify-coalescing -o - %s
--- |
  target triple = "x86_64-unknown-linux-gnu"
  declare void @clobber()
  define <2 x double> @f(<2 x double> %x, double %t, double %s, i32 %c, ptr %p) {
    ret <2 x double> %x
  }
...
---
name:            f
tracksRegLiveness: true
frameInfo:
  adjustsStack:    true
  hasCalls:        true
body: |
  bb.0:
    successors: %bb.1, %bb.2
    liveins: $xmm0, $xmm1, $xmm2, $edi, $rsi

    %0:vr128 = COPY $xmm0   ; x, a full <2 x double
 %1:fr64 = COPY $xmm1    ; t, a scalar
    %2:fr64 = COPY $xmm2    ; s, a scalar
    %3:vr128 = COPY %2      ; v = scalar_to_vector(
    %4:gr32 = COPY $edi
    %5:gr64 = COPY $rsi
    TEST32rr %4, %4, implicit-def $eflags
    JCC_1 %bb.2, 4, implicit killed $eflags

  bb.1:
 successors: %bb.2

    %1:fr64 = COPY %0       ; t = x[0]
 ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $s, implicit-def dead $ssp, implicit $rsp, implicit $ssp
    CALL64pcrel32 @clobber, csr_64, implicit $rsp, rsp, implicit-def $ssp
    ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, plicit-def dead $ssp, implicit $rsp, implicit $ssp
    %2:fr64 = COPY %1       ; s = t      (= x[0])
    %3:vr128 = COPY %0      ; v = x      (all 128 b

  bb.2:
    MOVSDmr %5, 1, $noreg, 0, $noreg, %1
    MOVSDmr %5, 1, $noreg, 8, $noreg, %2
    $xmm0 = COPY %3
    RET 0, $xmm0
...
```

`llc -run-pass=register-coalescer coalescer.mir -o -` erases
`%3:vr128 = COPY %0` and leaves `bb.1` as:

```
  bb.1:
 %1:fr64 = COPY %0
    ADJCALLSTACKDOWN64 ...
    CALL64pcrel32 @clobber, csr_64, ...
    ADJCALLSTACKUP64 ...
    %3:vr128 = COPY %1
```

`llc -start-before=register-coalescer coalescer.mir

```asm
        movsd %xmm3, 8(%rsp)                  # 8
        callq   clobber
        movsd 8(%rsp), %xmm1                  # 8-byte Reload
 # xmm1 = me
        movaps  %xmm1, %xmm0
```

Linked against a `main` that calls
`f(_mm_set_pd(2.0, 1.0), 3.0, 4.0, 1, p)`, it returns `<1, 0>` on 18.1.3,
20.1.2, 22.1.8 and main, and `<1, 2>` with `-join-liveintervals=false`.

## Origin

Found through a Julia miscompile, https://github.com/JuliaLang/julia/issues/64827
(Julia 1.12 with LLVM 18.1.7 and Julia 1.13 with LL
that code the vector is a loop-carried `NTuple{2,Float64}` and the `fr64`
register is a loop phi holding its first element. T
only miscompiled by `llc` up to 20.x, because 21 an
different MIR for it before the coalescer. Feeding
MIR of that function to the 22.1.8 or main coalesce
code.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to