https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118220
Bug ID: 118220
Summary: Optimization of malloc/free Produces Undefined
Behavior
Product: gcc
Version: 15.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: c
Assignee: unassigned at gcc dot gnu.org
Reporter: jonathan.gruber.jg at gmail dot com
Target Milestone: ---
Created attachment 59988
--> https://gcc.gnu.org/bugzilla/attachment.cgi?id=59988&action=edit
Preprocessed minimal test case.
Optimization of malloc and free calls can produce undefined behavior.
Consider this minimal test case (preprocessed version attached):
#include <stddef.h>
#include <stdint.h>
/* Declare these explicitly to avoid including <stdlib.h>. */
extern void *malloc(size_t size);
extern void free(void *ptr);
#define TOO_LARGE_SIZE ((size_t)PTRDIFF_MAX + 1) /* Request size greater than
PTRDIFF_MAX. */
ptrdiff_t test(void) {
char *const p = malloc(TOO_LARGE_SIZE);
if (!p) {
return 0;
}
char *const q = p + TOO_LARGE_SIZE;
const ptrdiff_t ret = q - p;
free(p);
return ret;
}
Critically, glibc's malloc unconditionally fails for requests larger than
PTRDIFF_MAX, so, with glibc, the above function test would always return 0.
However, on all nonzero optimization levels, GCC assumes that the call to
malloc will unconditionally succeed, effectively deleting the if-statement. As
a result, the function invokes undefined behavior by computing a pointer
subtraction whose result overflows (since TOO_LARGE_SIZE > PTRDIFF_MAX). If the
if-statement were not deleted, the body of the if-statement would always be
executed, thereby avoiding invoking undefined behavior.
On all nonzero optimization levels, GCC generates (x86_64, Intel syntax)
assembly similar to the following:
test:
movabs rax, -9223372036854775808
ret
Host system type: Arch Linux, x86_64 (with glibc, of course).
GCC information:
gcc version: 15.0.0 20241227 (experimental).
Configured with: ../gcc/configure
