https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128359
Bug ID: 128359
Summary: Misaligned vmovdqa64 store to ASan fake stack frame
(detect_stack_use_after_return) with
-march=sapphirerapids
Product: gcc
Version: 13.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: c
Assignee: unassigned at gcc dot gnu.org
Reporter: bingz at nvidia dot com
Target Milestone: ---
With -fsanitize=address on an AVX-512 target, GCC (-O1 and up) emits
64-byte aligned stores (vmovdqa64 %zmm) to locals in an ASan frame whose
base is only 32-byte aligned relative to those locals. On the real stack
it works, because the prologue realigns %rsp so the frame lines up. When
libasan puts the frame on the fake stack (detect_stack_use_after_return=1,
the default in libasan8 14.2), __asan_stack_malloc_7 returns a page-aligned
base. The store then lands 32 bytes off a 64-byte boundary, raises #GP,
and ASan reports "SEGV on unknown address ... high value address".
This is the same kind of problem as PR110027 and PR120201. The store here
is an aggregate zero-initializer, not a memcpy.
>>> Also reproduced with gcc <13.5 / 16.2 / trunk>, which has the PR120201
fix (r17-2387).
Versions / environment:
gcc 13.3.0 (Ubuntu 13.3.0-6ubuntu2~24.04) -O1/-O2/-O3 bad, -O0 ok
gcc 14.2.0 (Ubuntu 14.2.0-4ubuntu2~24.04.1) -O1/-O2/-O3 bad, -O0 ok
clang 20.1.2 with the same flags: ok
Ubuntu 24.04, x86_64-linux-gnu, Intel Xeon Platinum 8580 (Sapphire Rapids)
[paste `gcc -v` output here]
Reproducer (attached: mlx5_flow_dv.i.xz = preprocessed
drivers/net/mlx5/mlx5_flow_dv.c from DPDK 26.11-rc1; check_misalign.py):
xz -d mlx5_flow_dv.i.xz
gcc -w -O2 -march=sapphirerapids -mrtm -fPIC -fsanitize=address \
-c mlx5_flow_dv.i -o t.o
python3 check_misalign.py t.o # prints offsets, exit status 1 = bug
present
(The TU needs -mrtm. -march=x86-64-v4 doesn't compile it because it uses
AVX512IFMA intrinsics. check_misalign.py disassembles flow_dv_translate and
works out each aligned zmm access's offset mod 64 for both stacks.)
The affected function is flow_dv_translate(). The first faulting store at
run time is the zero-init of
void *sample_actions[8] = {0}; /* line 15106, 64 bytes, natural align
8 */
Other affected locals in the same function: line 14881
(struct mlx5_flow_dv_encap_decap_resource res = {...}, -O1/-O2) and
line 5155 (struct mlx5_dv_matcher_workspace wks = {...}, -O3).
gcc 13.3 -O3 code:
flow_dv_translate:
lea 0x8(%rsp),%r10
and $0xffffffffffffffc0,%rsp ; realign to 64
push -0x8(%r10)
push %rbp
mov %rsp,%rbp ; rbp == 0x30 (mod 64)
...
lea -0x18d0(%rbp),%rax ; real-stack frame base, == 0x20
(mod 64)
mov %rax,-0x1930(%rbp)
mov __asan_option_detect_stack_use_after_return,%rax
mov (%rax),%ebx
test %ebx,%ebx
jne <fake>
...
<fake>:
mov $0x1880,%edi
call __asan_stack_malloc_7 ; page-aligned base
test %rax,%rax
cmove -0x1930(%rbp),%rax
mov %rax,-0x1930(%rbp)
...
lea 0x18a0(%rax),%rbx ; rbx = base + 0x18a0
movq $0x41b58ab3,(%rax) ; ASan frame magic at base+0
...
vmovdqa64 %zmm0,-0xa40(%rbx) ; line 15106: base+0xe60 <--
faults
real stack: base == 0x20 (mod 64) -> base+0xe60 == 0 (mod 64) ok
fake stack: base == 0 (mod 64) -> base+0xe60 == 32 (mod 64) #GP
So the frame layout assumes base == 32 (mod 64), but the fake-stack base
is 0 (mod 64).
All aligned zmm accesses through the frame register (check_misalign.py),
each 32 (mod 64) on the fake stack and 0 (mod 64) on the real stack:
gcc-13 -O1: base+0xc60, +0xd20, +0xd20
gcc-13 -O2: base+0xe60, +0xf20
gcc-13 -O3: base+0xe60, +0xfa0, +0xf20, +0xf60
gcc-14 -O1: base+0xd20, +0xde0, +0xde0
gcc-14 -O2: base+0xe60, +0xf20
gcc-14 -O3: base+0xe60, +0xfa0, +0xf20, +0xf60
-O0 (both): no 64-byte realignment and no aligned zmm accesses
For comparison, other functions in the same DPDK build that have 64-byte
aligned locals (e.g. mlx5dr_send_wqe_fw, mlx5dr_rule_create_hws,
__mlx5_hws_cnt_svc) get a correct layout: the real frame base is 64-byte
aligned and every aligned zmm access is aligned on both stacks. My guess
is that in flow_dv_translate the 64-byte requirement comes from the
optimizer/expander (vectorized zero-init), not from the declared alignment
of the variables, so the PR110027 fix doesn't cover it.
Run-time failure (DPDK dpdk-testpmd with mlx5 NICs, gcc -O2):
AddressSanitizer:DEADLYSIGNAL
==155456==ERROR: AddressSanitizer: SEGV on unknown address (pc
0x5934bad309c4 bp 0x7ffeeeaa8830 sp 0x7ffeeeaa7000 T0)
==155456==The signal is caused by a READ memory access.
==155456==Hint: this fault was caused by a dereference of a high value
address
#0 flow_dv_translate ../drivers/net/mlx5/mlx5_flow_dv.c:15106
#1 flow_create_split_metadata ../drivers/net/mlx5/mlx5_flow.c:6651
...
SUMMARY: AddressSanitizer: SEGV ../drivers/net/mlx5/mlx5_flow_dv.c:15106
in flow_dv_translate
The same program runs cleanly when built with gcc -O0 or with clang -O3.
__attribute__((no_sanitize_address)) on flow_dv_translate also avoids it
(the zmm stores become vmovdqu8).
// The details are summarized and polished by the AI.
