| Issue |
185355
|
| Summary |
[MC][X86] Clang crashes when assembling x86-64 AVX `vcmpps`/`vcmppd` with symbol as comparison predicate
|
| Labels |
clang
|
| Assignees |
|
| Reporter |
venkyqz
|
## Summary
`llvm-mc` crashes with an assertion failure when assembling x86-64 AVX `vcmpps` or `vcmppd` instructions where the comparison-predicate operand is a symbol reference rather than a literal integer constant. The release build silently assembles the instruction with a wrong encoding (exit 0) — **crash-gap bug**.
This is the **first crash-gap bug documented in the x86/x86-64 backend**.
## Reproduction
**Godbolt Link**
+ https://godbolt.org/z/K6z3cjTT6
### AT&T Syntax (Primary Trigger)
**Test File (`poc.s`):**
```asm
.text
vcmpps $f0, %xmm0, %xmm1, %xmm2
```
Any symbol reference (e.g., `$f0`, `$xmm0`, `$rax`) as the comparison predicate will trigger the divergence.
**Commands:**
```bash
# Debug Build - Crashes with assertion
printf ".text\nvcmpps \$f0, %%xmm0, %%xmm1, %%xmm2\n" | llvm-mc - \
--triple=x86_64-linux-gnu -mattr=+avx --filetype=obj -o /dev/null
# Exit 134, Assertion failed
# Release Build - Silent miscompilation
printf ".text\nvcmpps \$f0, %%xmm0, %%xmm1, %%xmm2\n" | llvm-mc - \
--triple=x86_64-linux-gnu -mattr=+avx --filetype=obj -o /dev/null
# Exit 0, emits corrupted object code with wrong encoding
```
### Intel Syntax (OFFSET Modifier Bypass)
Intel syntax normally rejects bare symbols, but `OFFSET` modifier bypasses the protection:
```bash
# Debug Build - Crashes with assertion
echo "vcmpps xmm0, xmm1, xmm2, OFFSET f0" | llvm-mc - \
--triple=x86_64-linux-gnu -x86-asm-syntax=intel --filetype=obj -o /dev/null
# Exit 134
```
**Debug Build Output:**
```
llvm-mc: .../llvm/include/llvm/MC/MCInst.h:85:
int64_t llvm::MCOperand::getImm() const:
Assertion `isImm() && "This is not an immediate"' failed.
Stack dump:
#9 llvm::MCOperand::getImm() const MCInst.h:85
#10 llvm::X86::optimizeInstFromVEX3ToVEX2(...)
X86EncodingOptimization.cpp:58
#11 X86AsmParser::processInstruction(...) X86AsmParser.cpp:3855
...
```
---
## Root Cause
The vulnerability stems from an unguarded `getImm()` call in `optimizeInstFromVEX3ToVEX2`. For `VCMPPS{Y}rri` and `VCMPPD{Y}rri` opcodes, the function reads the comparison predicate to decide whether the instruction is commutative for VEX prefix optimization. When the predicate operand is a symbol reference (`MCExpr`) instead of an integer immediate (`MCImm`), `getImm()` unconditionally fires the assertion.
**Vulnerable Code (`X86EncodingOptimization.cpp:58`):**
```cpp
case X86::VCMPPSrri:
case X86::VCMPPDrri:
case X86::VCMPPSYrri:
case X86::VCMPPDYrri: {
switch (MI.getOperand(3).getImm() & 0x7) { // BUG: No isImm() check!
case 0x00: // EQ
case 0x03: // UNORD
...
```
The release build skips the assertion and returns garbage from `getImm()`, producing wrong binary encoding silently.
**Trigger Paths:**
| Syntax | Input | Parser Behavior | Result |
|--------|-------|-----------------|--------|
| AT&T | `vcmpps $f0, %xmm0, %xmm1, %xmm2` | Accepts `$f0` as symbol | Crash-gap |
| Intel | `vcmpps xmm0, xmm1, xmm2, f0` | Rejects bare symbol | Protected |
| Intel | `vcmpps xmm0, xmm1, xmm2, OFFSET f0` | Accepts `OFFSET` _expression_ | Crash-gap |
---
## Impact
| Build Type | Behavior | Security Risk |
|------------|----------|---------------|
| Debug | Assertion failure (Exit 134) | Detectable |
| Release | Silent miscompilation (Exit 0) | **Undetectable in CI/CD** |
**Affected Instructions:**
| Instruction | Debug Exit | Release Exit |
|-------------|------------|--------------|
| `vcmpps $f0, %xmm0, %xmm1, %xmm2` (AT&T) | 134 | 0 |
| `vcmppd $f0, %xmm0, %xmm1, %xmm2` (AT&T) | 134 | 0 |
| `vcmpps $f0, %ymm0, %ymm1, %ymm2` (AT&T) | 134 | 0 |
| `vcmppd $f0, %ymm0, %ymm1, %ymm2` (AT&T) | 134 | 0 |
| `vcmpps xmm0, xmm1, xmm2, OFFSET f0` (Intel) | 134 | 0 |
| `vcmppd xmm0, xmm1, xmm2, OFFSET f0` (Intel) | 134 | 0 |
**Not Affected:**
| Instruction | Reason |
|-------------|--------|
| `vcmpps $f0, (%rax), %xmm1, %xmm2` | Memory form — no VEX optimization |
| `vcmpps $f0, %xmm0, %xmm1, %k1` | AVX-512 EVEX — different encoding path |
| `vcmpss $f0, %xmm0, %xmm1, %xmm2` | Scalar — different fixup path |
| `cmpps $f0, %xmm0, %xmm1` | Legacy SSE — no VEX optimization |
This demonstrates a classic "Assertion Evaporation" vulnerability where security checks exist in debug builds but are stripped in release builds.
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
