| Issue |
202846
|
| Summary |
clang -O3 codegen mishandles a stack-top/guard pattern using __builtin_alloca(0) plus a later dynamic __builtin_alloca(bytes)
|
| Labels |
|
| Assignees |
|
| Reporter |
IshwaraK
|
Look at the below reproducer.
```
#include <stddef.h>
extern size_t guard_size(void);
extern int reuse_guard(void *old_top);
extern void install_guard(void *guard_start, size_t bytes, int reuse);
extern void use_live(void *live);
#define STACK_ALLOC(bytes) __builtin_alloca(bytes)
__attribute__((noinline))
int repro_tiny_no_touch(int seed)
{
char live[48];
size_t bytes = guard_size();
void *old_top = STACK_ALLOC(0);
int reuse;
live[0] = (char)seed;
live[47] = (char)(seed + 1);
reuse = reuse_guard(old_top);
if (!reuse)
{
void *tmp = STACK_ALLOC(bytes);
if (tmp)
old_top = (void *)((char *)old_top - bytes);
}
install_guard(old_top, bytes, reuse);
use_live(live);
return live[0] + live[47] + reuse;
}
__attribute__((noinline))
int repro_tiny_with_touch(int seed)
{
char live[48];
size_t bytes = guard_size();
void *old_top = STACK_ALLOC(0);
int reuse;
live[0] = (char)seed;
live[47] = (char)(seed + 1);
reuse = reuse_guard(old_top);
if (!reuse)
{
void *tmp = STACK_ALLOC(bytes);
if (tmp)
{
*(volatile char *)tmp = 0;
old_top = (void *)((char *)old_top - bytes);
}
}
install_guard(old_top, bytes, reuse);
use_live(live);
return live[0] + live[47] + reuse;
}
```
There are 2 experiments, 'with_touch' and 'no_touch' functions.
1. no_touch experiment: this case is useful but less decisive. Pairwise IR shows InstCombinePass removes the dynamic alloca.
Before InstCombine, %1 = alloca i8, i64 %call exists. After InstCombine, it is gone and only
%0 = alloca [0 x i8] remains. This may be debated because the allocated pointer is not touched.
2. with_touch experiment: this removes that ambiguity. Pairwise IR shows InstCombine keeps the dynamic alloca because of the volatile
store: %2 = alloca i8, i64 %call and store volatile i8 0, ptr %2 remain after InstCombine.
The first concrete bad frame layout appears during Prologue/Epilogue Insertion & Frame Finalization: before the pass,
frame objects are symbolic (fi#0, fi#1, fi#2) ; after the pass, fi#0 is [SP-40] and fi#1 is [SP-88] ,
producing old_top = rbp - 32 and live = rbp - 80..rbp - 33.
The no_touch shows a related optimization concern, but with_touch demonstrates a stronger codegen bug:
real dynamic stack movement exists, yet the captured old stack top is lowered to a fixed frame address that lets the guard overwrite a still-live local.
Required behavior:
The compiler must keep the guard range inside the dynamically allocated stack area, not inside any still-live fixed-frame local.
For with_touch, the required lowering is:
1. Place live[48] in the fixed frame.
2. Capture old_top so that it represents the real stack top below fixed locals.
3. Perform dynamic stack allocation:
rsp = rsp - align(bytes)
4. Compute guard_start from the safe old_top / dynamic allocation boundary:
guard_start = old_top - bytes
5. Call:
install_guard(guard_start, bytes, reuse)
6. Ensure:
[guard_start, guard_start + bytes) does not overlap live[48]
For the current bad -O3 output:
live[48] = [rbp - 80, rbp - 32)
old_top = rbp - 32
guard_start = rbp - 32 - bytes
For bytes = 48, that overlaps live[48]. Required behavior is that old_top must be below live[48], for example:
live[48] = [rbp - 80, rbp - 32)
old_top <= rbp - 80
guard_start = old_top - bytes
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
