Issue 244156
Summary [SelectionDAG] expandCONVERT_TO_ARBITRARY_FP zero-extends vector setcc results that are already IntVT-wide, so 0/-1 booleans subtract instead of add: `<2 x double>` -> Float6E3M2FN round.upward gives 0x3f for 1.1 on x86-64
Labels miscompilation, llvm:SelectionDAG, generated by fuzzer
Assignees
Reporter YuanchengJiang
    The following code:

There is no C path (no front end emits `llvm.convert.to.arbitrary.fp` yet). Run with `clang -O2 repro-main.ll -o r && ./r` (the second function also with `-mavx2`):

```llvm
; <2 x double> -> Float6E3M2FN, round toward +inf: 1.1 -> 1.25 (0x0d), 0.01 -> 0.0625 (0x01).
define <2 x i8> @f(<2 x double> %x) noinline {
  %r = call <2 x i6> @llvm.convert.to.arbitrary.fp.v2i6.v2f64(<2 x double> %x, metadata !"Float6E3M2FN", metadata !"round.upward", i1 false)
  %z = zext <2 x i6> %r to <2 x i8>
  ret <2 x i8> %z
}

; <2 x half> -> Float8E5M2, round to nearest even: 125.3125 rounds up across a binade to 128.0 (0x58).
define <2 x i8> @g(<2 x half> %x) noinline {
  %r = call <2 x i8> @llvm.convert.to.arbitrary.fp.v2i8.v2f16(<2 x half> %x, metadata !"Float8E5M2", metadata !"round.tonearest", i1 false)
  ret <2 x i8> %r
}

@vin = global <2 x double> <double 1.1, double 0.01>
@hin = global <2 x half> <half 125.3125, half 1.5>
@f1 = private constant [43 x i8] c"f(1.1, 0.01) = %02x %02x (expected 0d 01)\0A\00"
@f2 = private constant [39 x i8] c"g(125.3125, 1.5) = %02x (expected 58)\0A\00"
declare i32 @printf(ptr, ...)

define i32 @main() {
  %vx = load volatile <2 x double>, ptr @vin
  %hx = load volatile <2 x half>, ptr @hin
  %a = call <2 x i8> @f(<2 x double> %vx)
  %b = call <2 x i8> @g(<2 x half> %hx)
  %a0 = extractelement <2 x i8> %a, i32 0
  %a1 = extractelement <2 x i8> %a, i32 1
  %b0 = extractelement <2 x i8> %b, i32 0
  %x0 = zext i8 %a0 to i32
  %x1 = zext i8 %a1 to i32
  %y0 = zext i8 %b0 to i32
  call i32 (ptr, ...) @printf(ptr @f1, i32 %x0, i32 %x1)
  call i32 (ptr, ...) @printf(ptr @f2, i32 %y0)
  %ok0 = icmp eq i32 %x0, 13
  %ok1 = icmp eq i32 %x1, 1
  %ok2 = icmp eq i32 %y0, 88
  %o01 = and i1 %ok0, %ok1
  %ok = and i1 %o01, %ok2
  %r = select i1 %ok, i32 0, i32 1
  ret i32 %r
}
```

Resulted in this output:
```
$ clang -O2 repro-main.ll -o r && ./r                (also -O0: the bug is in the expansion)
f(1.1, 0.01) = 3f 3f (expected 0d 01)                exit 1
g(125.3125, 1.5) = 58 (expected 58)
$ clang -O2 -mavx2 repro-main.ll -o r && ./r
f(1.1, 0.01) = 3f 3f (expected 0d 01)                exit 1
g(125.3125, 1.5) = 50 (expected 58)
```
0x3f is -28.0 (the round-up "increment" of -1 borrows through the sign bit). 0x50 is 32.0: the rounding carry from 125.3125 to 128.0 decremented the exponent instead of incrementing it. With AVX2, `<2 x half>` -> Float8E5M2 is wrong for every input whose rounding carries into the exponent, 15360 of the 65536 half bit patterns (checked exhaustively against an exact reference). AVX-512 and the scalar expansion are correct.

Trunk: https://godbolt.org/z/fzashf6n1 (LLVM IR, x86-64 clang trunk -O2, executed; `-O2 -mavx2` prints `50` on the second line; reproduces on 81d345fc0e, 2026-11-02)

Root cause: `TargetLowering::expandCONVERT_TO_ARBITRARY_FP` (llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp) turns five setcc results into 0/1 integers with `DAG.getNode(ISD::ZERO_EXTEND, dl, IntVT, Cond)`: the directed-rounding increments (`DoRound` for round.upward / round.downward), the normal-path sticky bit, the mantissa-overflow carry (`AdjExp = NewExp + zext(MantOverflow)`) and the denormal sticky bit. For vectors on x86 the setcc result type equals `IntVT` (v2i64 for `<2 x double>`, v16i16 for widened `<2 x half>` with AVX2), so `getNode` folds the zero-extend away as a no-op and the boolean keeps its 0/-1 vector contents. "Round up" then adds -1, and a mantissa carry subtracts 1 from the exponent. If the setcc type is wider than `IntVT`, the same `getNode` asserts "Invalid zext node, dst < src!" instead (AArch64 scalar f16). PR #221358 replaces these calls with `getZExtOrTrunc`. That fixes the assertion but not this bug, because `getZExtOrTrunc` is also a no-op for equal types. The in-tree test CodeGen/X86/float-to-arbitrary-fp.ll checks the wrong code for `to_f4e2m1fn_v4f32` (`paddd %xmm0, %xmm1` adds the compare mask).

Suggested fix: Convert the setcc results with a select, except for i1 setcc types where the zero-extend is already right:
```diff
   SDValue Zero = DAG.getConstant(0, dl, IntVT);
   SDValue _One_ = DAG.getConstant(1, dl, IntVT);
+  // A setcc result as 0/1 in IntVT. ZERO_EXTEND does not do this: the setcc
+  // type can equal IntVT (vector compares: the extend folds away and the
+  // boolean stays 0/-1) or be wider than it.
+  auto BoolToInt = [&](SDValue C) {
+    if (C.getValueType().getScalarSizeInBits() == 1)
+      return DAG.getNode(ISD::ZERO_EXTEND, dl, IntVT, C);
+ return DAG.getSelect(dl, IntVT, C, One, Zero);
+  };
...
-      return DAG.getNode(ISD::ZERO_EXTEND, dl, IntVT, DoRound);
+      return BoolToInt(DoRound);
...
-                  DAG.getNode(ISD::ZERO_EXTEND, dl, IntVT, MantOverflow));
+ BoolToInt(MantOverflow));
```
(and likewise for the other `DoRound` and the two sticky bits; the full patch is in fix.diff, which applies to 81d345fc0e). With the fix, the repro prints the expected values at -O0/-O2 with and without AVX2. Of the 9 in-tree convert-to-arbitrary-fp CodeGen tests, only X86/float-to-arbitrary-fp.ll changes, and only in that vector function.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to