| Issue |
244208
|
| Summary |
[VectorCombine] shuffle(fop(x)) -> fop(shuffle(x)) duplicates the FP op when the mask repeats a lane or the op has other uses
|
| Labels |
new issue
|
| Assignees |
|
| Reporter |
he-weiwen
|
(The following analysis is heavily LLM assisted, I'm relatively confident that the phenomenon described is real, but less so on its severity and appropriate solution.)
Several VectorCombine folds rewrite `shuffle(fop(x))` into `fop(shuffle(x))`. In two situations the rewritten IR computes the same FP result more than once, where the original computed it once and copied it: when the shuffle mask uses the same lane of `fop` more than once (A below), and when `fop` has another user and is kept for it (B below). Per LangRef ([Behavior of Floating-Point NaN values](https://llvm.org/docs/LangRef.html#behavior-of-floating-point-nan-values)), each computation that returns a NaN may pick its sign and payload independently. So lanes (or uses) that were copies of one result, and therefore bitwise equal, may differ after the fold. That is not a refinement, and Alive2 reports a value mismatch. When the mask doesn't repeat a lane and `fop` has no other uses, the fold is fine.
@ParkHanbum found this in the review of #228472 ([Alive2 log](https://github.com/llvm/llvm-project/pull/228472#discussion_r4226574797)), where the new `MergePermutes` path in `foldShuffleOfBinops` does it. It already happens on main, in two ways:
**A. The shuffle mask repeats a lane.** Moving the shuffle above the operation computes the repeated lane once per output lane instead of once and copying it:
```llvm
define <4 x float> @a_shuffle_of_binops(<2 x float> %x, <2 x float> %z) {
%l = fadd <2 x float> %x, splat (float 1.0)
%r = fadd <2 x float> %z, splat (float 1.0)
%s = shufflevector <2 x float> %l, <2 x float> %r, <4 x i32> <i32 0, i32 0, i32 2, i32 2>
ret <4 x float> %s
}
; -passes=vector-combine:
; %1 = shufflevector <2 x float> %x, <2 x float> %z, <4 x i32> <i32 0, i32 0, i32 2, i32 2>
; %s = fadd <4 x float> %1, splat (float 1.000000e+00)
```
Alive2 (`<2 x half>` version): https://alive2.llvm.org/ce/z/6oKb3L
**B. The original operation has another use.** It is kept for that use, and the folded code computes the same lanes again, so the two uses can see different NaNs:
```llvm
define <4 x float> @b_shuffle_of_binops(<2 x float> %x, <2 x float> %z) {
%l = fadd <2 x float> %x, splat (float 1.0)
%r = fadd <2 x float> %z, splat (float 2.0)
call void @use2f(<2 x float> %l)
%s = shufflevector <2 x float> %l, <2 x float> %r, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
ret <4 x float> %s
}
; -passes=vector-combine -mtriple=x86_64--:
; %l = fadd <2 x float> %x, splat (float 1.000000e+00)
; %1 = shufflevector <2 x float> %x, <2 x float> %z, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
; %s = fadd <4 x float> %1, <float 1.000000e+00, float 1.000000e+00, float 2.000000e+00, float 2.000000e+00>
```
Affected folds on main (2517f566169a), with `opt -passes=vector-combine` on the repros below:
| Fold | A: repeated lane | B: original kept for another use |
|---|---|---|
| `foldShuffleOfBinops` | SSE2, AVX2, AArch64, no triple | SSE2, AVX2, AArch64 |
| `foldPermuteOfBinops` | SSE2, AVX2, AArch64, no triple | no (requires one use) |
| `foldShuffleOfIntrinsics` | SSE2, AVX2 (fma; minnum also with no triple) | SSE2, AVX2, AArch64, no triple (sqrt) |
| `foldPermuteOfIntrinsic` | SSE2, AVX2, AArch64, no triple (sqrt) | not seen: the kept intrinsic's cost is added, and my test did not fold |
| `foldShuffleOfCastops` | SSE2, AVX2, AArch64, no triple (fpext) | AVX2, no triple (fpext) |
| `foldShuffleToIdentity` | SSE2, AVX2, AArch64, no triple (when the operands are splats) | no (requires one user) |
<details><summary>All repros</summary>
```llvm
declare void @use2f(<2 x float>)
declare void @use2d(<2 x double>)
; A: foldPermuteOfBinops
define <4 x float> @a_permute_of_binops(<4 x float> %a, <4 x float> %b) {
%a0 = shufflevector <4 x float> %a, <4 x float> poison, <4 x i32> <i32 1, i32 0, i32 3, i32 2>
%b0 = shufflevector <4 x float> %b, <4 x float> poison, <4 x i32> <i32 1, i32 0, i32 3, i32 2>
%d = fdiv <4 x float> %a0, %b0
%s = shufflevector <4 x float> %d, <4 x float> poison, <4 x i32> <i32 3, i32 3, i32 2, i32 2>
ret <4 x float> %s
}
; A: foldShuffleOfIntrinsics
define <4 x float> @a_shuffle_of_intrinsics(<4 x float> %a, <4 x float> %b, <4 x float> %c, <4 x float> %d, <4 x float> %e, <4 x float> %f) {
%l = call <4 x float> @llvm.fma.v4f32(<4 x float> %a, <4 x float> %b, <4 x float> %c)
%r = call <4 x float> @llvm.fma.v4f32(<4 x float> %d, <4 x float> %e, <4 x float> %f)
%s = shufflevector <4 x float> %l, <4 x float> %r, <4 x i32> <i32 0, i32 0, i32 4, i32 4>
ret <4 x float> %s
}
; A: foldPermuteOfIntrinsic
define <4 x float> @a_permute_of_intrinsic(<4 x float> %a) {
%f = call <4 x float> @llvm.sqrt.v4f32(<4 x float> %a)
%s = shufflevector <4 x float> %f, <4 x float> poison, <4 x i32> <i32 1, i32 1, i32 0, i32 0>
ret <4 x float> %s
}
; A: foldShuffleOfCastops
define <4 x double> @a_shuffle_of_castops(<4 x float> %x) {
%e = fpext <4 x float> %x to <4 x double>
%s = shufflevector <4 x double> %e, <4 x double> poison, <4 x i32> <i32 1, i32 1, i32 0, i32 0>
ret <4 x double> %s
}
; A: foldShuffleToIdentity
define <4 x float> @a_shuffle_to_identity(<4 x float> %a) {
%xs = shufflevector <4 x float> %a, <4 x float> poison, <4 x i32> zeroinitializer
%m = fmul <4 x float> %xs, splat (float 3.0)
%d = fadd <4 x float> %m, splat (float 1.0)
%s = shufflevector <4 x float> %d, <4 x float> poison, <4 x i32> <i32 0, i32 0, i32 1, i32 1>
ret <4 x float> %s
}
; B: foldShuffleOfIntrinsics
define <4 x float> @b_shuffle_of_intrinsics(<2 x float> %x, <2 x float> %z) {
%l = call <2 x float> @llvm.sqrt.v2f32(<2 x float> %x)
%r = call <2 x float> @llvm.sqrt.v2f32(<2 x float> %z)
call void @use2f(<2 x float> %l)
%s = shufflevector <2 x float> %l, <2 x float> %r, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
ret <4 x float> %s
}
; B: foldShuffleOfCastops
define <4 x double> @b_shuffle_of_castops(<2 x float> %x, <2 x float> %y) {
%ex = fpext <2 x float> %x to <2 x double>
%ey = fpext <2 x float> %y to <2 x double>
call void @use2d(<2 x double> %ex)
%s = shufflevector <2 x double> %ex, <2 x double> %ey, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
ret <4 x double> %s
}
```
</details>
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs