Issue 244529
Summary [fuzzer-found][X86] combineBasicSADPattern skips a sign extension of the vXi8 ABDU forms, so a sum of `(signed char)|a-b|` becomes PSADBW
Labels backend:X86, miscompilation, generated by fuzzer
Assignees
Reporter YuanchengJiang
    The following code:

```cpp
// x86-64: clang -O2 repro.c && ./a.out   (also -O1; -O3 fails even without the pragma; SSE2 is enough)
// The absolute differences are read back as *signed* chars, so 200 must count as -56.
#include <stdio.h>

__attribute__((noinline)) int sum_sabsdiff(const unsigned char *a, const unsigned char *b) {
  int s = 0;
  _Pragma("clang loop vectorize_width(16)")
  for (int i = 0; i < 16; i++) {
    unsigned char mx = a[i] > b[i] ? a[i] : b[i];
    unsigned char mn = a[i] < b[i] ? a[i] : b[i];
    s += (signed char)(unsigned char)(mx - mn);
  }
  return s;
}

int main(void) {
  unsigned char a[16] = {200}, b[16] = {0};
  int got = sum_sabsdiff(a, b);
  int want = (signed char)200; // -56; the other 15 lanes contribute 0
  printf("got %d, expected %d: %s\n", got, want, got == want ? "PASS" : "FAIL");
  return got != want;
}
```

Resulted in this output:
```
got 200, expected -56: FAIL
```

| build (x86-64) | result |
|---|---|
| clang -O0 | -56 (PASS) |
| gcc -O2 | -56 (PASS) |
| clang -O2 / -O3 (`-O3` even without the pragma) | 200 (FAIL) |

Wrong code at -O2 (the whole function):
```
 movdqu  (%rdi), %xmm0
        movdqu  (%rsi), %xmm1
        psadbw %xmm0, %xmm1
        pshufd  $238, %xmm1, %xmm0
        paddq   %xmm1, %xmm0
        movd    %xmm0, %eax
        retq
```

Trunk: https://godbolt.org/z/Kfrv97Gvz

Root cause: `combineBasicSADPattern` (X86ISelLowering.cpp) strips a SIGN_EXTEND/ZERO_EXTEND/ANY_EXTEND above the reduction operand. Its comment says the sign extension is "basically zero extend", because the extended value's sign bit is zero. That was true for the original `abs(sub(zext i8, zext i8))` form, which is computed in a wider type. PR #147570 also matches the vXi8 forms `abdu(a, b)` and `umax - umin`. There the value being extended is an i8 absolute difference, which can be 128..255. Its sign extension is negative, but PSADBW adds it as an unsigned value.

Suggested fix: when the stripped node is a SIGN_EXTEND and the matched operand has i8 elements, do not form PSADBW (or require its sign bit to be known zero). Validated locally: with the fix the program prints -56 at -O1/-O2/-O3, the AGT check no longer fires, and 453 llc RUN lines from the 103 X86 tests that mention psadbw/abdu/sad give identical output.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to