Issue 245040
Summary [AMDGPU][GlobalISel] Flat 16-bit atomic min falls back during register bank legalization
Labels new issue
Assignees
Reporter jinmoza
    On gfx900 and gfx1100, GlobalISel falls back to SelectionDAG when compiling a 16-bit flat atomic min. The first failure is an AS0 `G_PTRMASK`. With the proposed AS0 rules applied, the standalone pointer mask finishes in GlobalISel, while the original atomic function reaches a second failure at `llvm.amdgcn.end.cf`.

## How we found it

We scanned LLVM's AMDGPU tests and ROCm Examples on gfx900, gfx1030, and gfx1100 using LLVM `4c22a9f755be`. The scan compiled existing tests and examples through SelectionDAG. Both reproducers below come from existing LLVM test IR or a reduction of it. For inputs that succeeded, it ran GlobalISel with fallback diagnostics enabled and grouped failures by function, pass, opcode, and result type. Of 7,041 cases that reached the GlobalISel comparison, 885 reported a fallback. These counts describe the selected corpus and configurations. [Published methodology and results](https://github.com/jinmoza/amdgpu-gisel-fallback-study/blob/cdb0005e546bb52203acfbe5bc166a1c30d89e2f/published-results/full-20261106-reviewed/README.md)

The `G_PTRMASK` / `p0` failure in AMDGPU RegBankLegalize accounted for 221 function/target pairs in the scan, across 74 functions in eight LLVM test files. We extracted those functions and replayed all 74 on each of the three targets at the same revision, filling in the one missing pair. All 222 pairs failed at AS0 `G_PTRMASK` in strict GlobalISel mode. With fallback enabled, all 222 reported a fallback and produced assembly. The SelectionDAG controls also produced assembly.

## Reproduce the original fallback

One of the affected functions is `flat_one_as_atomic_min_i16` from `atomics-system-scope.ll`. We reproduced it on gfx900 and gfx1100 at LLVM `5989210109e1`, the baseline used to validate the proposed AS0 rules. Save the function below as `atomic-min.ll`. [Upstream test](https://github.com/llvm/llvm-project/blob/5989210109e1f7f59ccfeb07dc76120eba5f48a5/llvm/test/CodeGen/AMDGPU/atomics-system-scope.ll)

```llvm
define i16 @flat_one_as_atomic_min_i16(ptr %ptr, i16 %val) {
  %result = atomicrmw min ptr %ptr, i16 %val syncscope("one-as") monotonic
  ret i16 %result
}
```

```sh
for arch in 9.00 11.00; do
  llc -mtriple="amdgpu${arch}-amd-amdhsa" -O2 -global-isel=1 \
    -global-isel-abort=2 -verify-machineinstrs -pass-remarks-missed='.*' \
    atomic-min.ll -o /dev/null
done
```

**Expected** — With GlobalISel enabled, the function compiles through GlobalISel without a fallback warning.

**Actual** — The baseline reports that no RegBankLegalize rule matches `G_PTRMASK` with a `p0` result. The entire function then falls back to SelectionDAG.

LLVM expands this 16-bit atomic update into a 32-bit compare-and-swap loop. It uses `llvm.ptrmask.p0.i64(ptr %p, i64 -4)` to clear the low two address bits and find the containing word. AMDGPU's register bank legalizer has `G_PTRMASK` mappings for AS1 and AS3, while the pointer produced here is a 64-bit flat pointer (AS0).

## Isolate the missing mapping

The same rule failure occurs with a standalone pointer mask using a runtime `i64` mask. Save the function below as `masked.ll` and substitute it for `atomic-min.ll` in the command above.

```llvm
define ptr @masked(ptr %p, i64 %mask) {
  %r = call ptr @llvm.ptrmask.p0.i64(ptr %p, i64 %mask)
  ret ptr %r
}
```

We checked instruction selection separately using MIR with register banks already assigned. It produced a single scalar 64-bit AND for a uniform result and two vector 32-bit ANDs for a divergent result. That narrows the first failure to the missing AS0 register bank mappings.

Adding uniform and divergent AS0 rules, following the AS1 mappings, makes the reduction finish in GlobalISel on gfx900 and gfx1100 with fallback enabled. The regression tests check both banks and track the pointer and mask operands through masking and the result store. Six focused lit tests passed. Six inputs per target also passed with fallback disabled and machine instruction verification enabled, including all four pointer/mask uniformity combinations and the original atomic function.

## Remaining fallback at `llvm.amdgcn.end.cf`

With the AS0 rules applied and fallback enabled, the original atomic min gets past `G_PTRMASK` and falls back later in the same pass. On gfx900, the MIR shows a single-input loop-exit PHI taking the SGPR result of `if.break` and supplying a VGPR to `end.cf`.

```text
%37:sgpr(i64) = G_INTRINSIC intrinsic(@llvm.amdgcn.if.break), ...
%40:vgpr(i64) = G_PHI %37(i64), %bb.2
G_INTRINSIC_W_SIDE_EFFECTS intrinsic(@llvm.amdgcn.end.cf), %40(i64)
```

The same intrinsic is the next failure on gfx1100. With `-global-isel-abort=1`, the tested gfx900 pipeline passes the SGPR value directly to `end.cf`, so the AS0-patched build can complete the function in strict mode. The fallback-enabled reproducer above is the end-to-end check for this issue.

This matches the loop-exit PHI pattern described in #206435. The open PR #209158 represents structured control-flow lane masks as `i1` and maps `end.cf` to the VCC bank. Its effect on this atomic reproducer remains to be verified with both changes applied.

## Code quality and compile time

On gfx900, a divergent pointer with a uniform mask compiles through GlobalISel, but the generated code has two extra `v_mov_b32` instructions that copy the low and high halves of the scalar mask from SGPRs to VGPRs. SelectionDAG reads the mask directly from SGPRs in the two vector ANDs. We also reproduced the copies with the existing AS1 mapping. We're investigating how to use the scalar mask directly in GlobalISel.

We measured `llc` wall time from startup through assembly emission on one host. The measurements used identical `-O2` options with fallback enabled and 32 paired blocks per configuration. The table reports median wall times.

| Input | gfx900 before → after | gfx1100 before → after |
| --- | ---: | ---: |
| Original atomic min | 7.05 → 7.10 ms | 7.17 → 7.21 ms |
| Standalone pointer mask | 6.09 → 5.88 ms | 6.14 → 5.90 ms |
| Full `atomics-system-scope.ll` | 90.24 → 90.36 ms | 92.33 → 92.26 ms |

The reduced case completed in GlobalISel and took about 3.6% less time on gfx900 and 4.0% less on gfx1100, based on the paired measurements. For the original function and the full 64-function test file, the timing intervals included zero. The original function still fell back and produced identical assembly before and after the AS0 rules. All measurements were compiler-side.

## Follow-up

1. Add the AS0 `G_PTRMASK` rules and regression tests. This covers the standalone pointer mask and removes the first fallback point in the atomic function.
2. Verify whether #209158 clears the remaining fallback with the AS0 rules applied. Verify on gfx900 and gfx1100 with the original function and fallback enabled.
3. Track the extra register copies as a related code quality improvement. This can be reviewed independently of the original function's fallback.

Assisted-by: Codex

_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to