Issue 241453
Summary [ValueTracking] fcmpImpliesClass ignores the input denormal mode for subnormal constants
Labels new issue
Assignees
Reporter moyix
    When `fcmp oeq float %x, C` is true and `C` is a positive subnormal constant, LLVM concludes that `%x` is a positive subnormal, even when the function declares that denormal inputs are flushed (`denormal_fpenv(preservesign)`, which is what `-fdenormal-fp-math=preserve-sign` emits). With flushed inputs the comparison is also true for negative subnormals and for both zeros.

```llvm
define i32 @sign_after_eq_subnormal(float %x) #0 {
entry:
  %c = fcmp oeq float %x, 0x3800000000000000   ; 0x00400000, a positive subnormal
  br i1 %c, label %t, label %f
t:
  %b = bitcast float %x to i32
  %s = lshr i32 %b, 31                          ; sign bit of %x
  ret i32 %s
f:
  ret i32 2
}

attributes #0 = { denormal_fpenv(preservesign) }
```

`opt -S -passes=instcombine` turns block `t` into `ret i32 0`: https://godbolt.org/z/qo9xs5rh6

That is wrong (https://alive2.llvm.org/ce/z/3cR9FR). The negative subnormal `0x80400000` satisfies the comparison, and its sign bit is 1. Without the attribute the fold is correct.

`fcmpImpliesClass()` gives up for a zero constant when inputs may be flushed:

https://github.com/llvm/llvm-project/blob/f9ad05d45704810856ef7a239a67abcc342d1a03/llvm/include/llvm/IR/GenericFloatingPointPredicateUtils.h#L131-L136

It has no such check for a subnormal constant, for any predicate. For `oeq` it returns the constant's class:

https://github.com/llvm/llvm-project/blob/f9ad05d45704810856ef7a239a67abcc342d1a03/llvm/include/llvm/IR/GenericFloatingPointPredicateUtils.h#L315-L316

Adding the same check for subnormal constants fixes it. With this patch the reproducer keeps `%s` and `check-llvm` passes:

```diff
diff --git a/llvm/include/llvm/IR/GenericFloatingPointPredicateUtils.h b/llvm/include/llvm/IR/GenericFloatingPointPredicateUtils.h
index 448a6e913..46eddc161 100644
--- a/llvm/include/llvm/IR/GenericFloatingPointPredicateUtils.h
+++ b/llvm/include/llvm/IR/GenericFloatingPointPredicateUtils.h
@@ -178,6 +178,13 @@ public:
     }
 
     const bool IsDenormalRHS = (OrigClass & fcSubnormal) == OrigClass;
+    if (IsDenormalRHS) {
+      // A subnormal constant compares like zero if input denormals are
+      // flushed, so none of the classes below hold.
+      DenormalMode Mode = queryDenormalMode(F, LHS);
+      if (Mode.Input != DenormalMode::IEEE)
+ return {Invalid, fcAllFlags, fcAllFlags};
+    }
 
     const bool IsInf = (OrigClass & fcInf) == OrigClass;
     if (IsInf) {
```

This started with #80941 (LLVM 19). 351a1bb0c525 (#181899) stopped InstCombine from turning an fcmp into an icmp when denormal inputs may be flushed, but doesn't cover this fold. #240229 and #237855 are about other folds that ignore the input denormal mode.

Found, reduced and patched with AI assistance (GPT-5.6 and Claude); I have checked the reproducers.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to