https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128330
Bug ID: 128330
Summary: -fstack-protector does not protect functions with
plain char arrays under -flto
Product: gcc
Version: 16.2.0
Status: UNCONFIRMED
Keywords: lto, wrong-code
Severity: normal
Priority: P3
Component: lto
Assignee: unassigned at gcc dot gnu.org
Reporter: ianpike98cs at gmail dot com
Target Milestone: ---
Host: x86_64-linux-gnu
Target: x86_64-linux-gnu
Build: x86_64-linux-gnu
When building with -fstack-protector and -flto, a function whose only protected
local is a plain char array ends up without a stack guard. The same function
using a signed char or unsigned char array still gets one. From my testing
-fstack-protector-strong and -fstack-protector-all are not affected.
This looks to be a regression from 4.6, since 4.6.4 keeps the guard while every
release I tried from 4.7.4 through 16.2.0 drops it (4.7.4, 4.8.5, 5.4, 6.5,
7.5, 8.5, 9.5, 10.5, 11.5, 12.5, 13.4, 14.4, 15.3, 16.1, and 16.2.0). Trunk
(17.0.0 20261030) behaves the same way. I was also able to reproduce it on
aarch64-apple-darwin25 with Homebrew GCC 16.2.0. For the x86-64 runs I used
Compiler Explorer, which also adds -g -fno-verbose-asm to the command line.
The testcase (pr.c) has no includes:
__attribute__((noinline)) int f_char(int n)
{
char b[64];
__asm__ volatile("" : : "r"(b) : "memory");
b[n & 63] = 1;
__asm__ volatile("" : : "r"(b) : "memory");
return b[0];
}
__attribute__((noinline)) int f_schar(int n)
{
signed char b[64];
__asm__ volatile("" : : "r"(b) : "memory");
b[n & 63] = 1;
__asm__ volatile("" : : "r"(b) : "memory");
return b[0];
}
int main(int argc, char **argv)
{
(void) argv;
return f_char(argc) + f_schar(argc);
}
$ gcc -O2 -fstack-protector pr.c
$ gcc -O2 -fstack-protector -flto pr.c
Without -flto both functions get a guard. With -flto f_schar still gets one but
f_char does not. This is f_char from the -flto build (x86-64, 16.2.0):
f_char:
lea -0x48(%rsp),%rax
and $0x3f,%edi
movb $0x1,-0x48(%rsp,%rdi,1)
movsbl -0x48(%rsp),%eax
ret
To show the effect at runtime, smash.c overflows a 16 byte char buffer on
purpose:
__attribute__((noinline)) void g(int n)
{
char b[16];
__builtin_memset(b, 'A', n);
__asm__ volatile("" : : "r"(b) : "memory");
}
int main(int argc, char **argv)
{
(void) argv;
g(argc * 32);
return 0;
}
I built it the same way and ran it as ./a.out x y. Without -flto it aborts with
"stack smashing detected" (exit 134) but with -flto it just segfaults (exit
139).
I also went through the checks from the bug guidelines. -Wall -Wextra print
nothing and -fno-strict-aliasing -fwrapv -fno-aggressive-loop-optimizations do
not change the result. One thing to note is that with those flags you need
-fno-ipa-icf as well, otherwise ICF folds f_schar into f_char. pr.c also runs
clean under -fsanitize=address,undefined. As far as I can tell the option does
reach LTO, since -fstack-protector shows up on both the WPA and LTRANS lto1
command lines.
It still fails at -O0, with the array inside a struct, and with
-funsigned-char, -ffat-lto-objects, or -flto-partition=none. Marking the
functions noipa does not change anything either. For comparison Clang 23.1.0
guards both functions with -flto. The noipa version and the Clang comparison
are both on https://godbolt.org/z/7n5vsahME
>From my understanding the cause is in stack_protect_classify_type
(cfgexpand.cc), which checks for char arrays by comparing the element type by
pointer:
t = TYPE_MAIN_VARIANT (TREE_TYPE (type));
if (t == char_type_node
|| t == signed_char_type_node
|| t == unsigned_char_type_node)
The problem is that preload_common_nodes (tree-streamer.cc) skips itk_char and
lto1 builds its own char_type_node. So plain char gets streamed as a separate
node and fails the compare, while signed and unsigned char are preloaded and
still match. I also checked this in a debugger on 16.2.0. In lto1 the f_char
element type is none of integer_types[0..2] but in cc1 it is char_type_node.
The itk_char skip came in with r174519, which is in 4.7.0 but not in 4.6.4. So
I think that is where the regression started, though I have not bisected it to
confirm. Also PR 79062 looks like the same kind of compare failing in the
warning code.
If this gets confirmed I'd be happy to work on a patch for it. Also let me know
if you want me to test anything else.
gcc -v (x86-64):
Target: x86_64-linux-gnu
Configured with: ../gcc-16.2.0/configure
--prefix=/opt/compiler-explorer/gcc-build/staging --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=x86_64-linux-gnu --disable-bootstrap
--enable-multiarch --with-abi=m64 --with-multilib-list=m32,m64,mx32
--enable-multilib --enable-clocale=gnu
--enable-languages=c,c++,fortran,ada,objc,obj-c++,go,d,m2,rust,cobol,algol68
--enable-ld=yes --enable-gold=yes --enable-libstdcxx-time=yes
--enable-linker-build-id --enable-lto --enable-plugins --enable-threads=posix
--with-pkgversion=Compiler-Explorer-Build-gcc--binutils-2.44
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 16.2.0 (Compiler-Explorer-Build-gcc--binutils-2.44)
