Hi, This RFC is separate from the elf_newscn dirty-state fix for Sourceware Bug 34412 and from the section-header cache-extent hardening work.
Reported as Sourceware Bug 34420: https://sourceware.org/bugzilla/show_bug.cgi?id=34420 elf_update computes the output layout in the shared elf32_updatenull.c template before selecting the mmap or pwrite serializer. Several section, program-header, and section-header extents were added in the destination ELF field width. With ELF_F_LAYOUT, a caller-selected offset could wrap before the output file was allocated or mapped. I reproduced three public-API ASan failures on the unmodified base: 1. sh_offset + sh_size wraps before an out-of-mapping section-data write. 2. e_phoff + phdr_size wraps before a negative-size gap memset. 3. e_shoff + shdr_size wraps before an out-of-mapping section-header write. These require a caller to opt into ELF_F_LAYOUT and provide extreme values through public mutation APIs. This is proactive library hardening, not a malformed-file parser trigger or a remote-exploitation claim. libelf should return ELF_E_RANGE rather than derive an undersized output mapping and corrupt its caller’s process. The proposed invariant is that, before either serializer allocates, maps, truncates, writes, or performs pointer arithmetic, each output extent and alignment is computed without wrap, values fit their destination ELF fields, and the final size fits elf_update’s int64_t return type. Patch 1 implements this in the common ELF32/ELF64 layout builder using small file-local inline helpers for checked addition, alignment, and extent accumulation. INT64_MAX is the final bound because larger successful results cannot be represented by elf_update. ELF32 destination fields are checked before narrowing; valid ELF32 extents may still end just above UINT32_MAX. The automatic data path also checks its uint64_t offset before storing signed Elf_Data.d_off. Upstream’s update4 test establishes that SHT_NOBITS data may have d_buf == NULL and nonzero d_size. A two-descriptor ELF_C_NULL control can therefore exceed INT64_MAX without huge resources. Before the guard it stores INT64_MIN; afterward it returns ELF_E_RANGE without changing the second d_off. Consistent with existing non-transactional behavior, earlier fields may change before an error. This patch adds no rollback; it prevents a negative d_off. The checked alignment helper preserves existing behavior: d_align zero resets the descriptor offset, while one leaves it unchanged. The sh_addralign-zero path is normalized to one as before. Tests cover exact ELF32/ELF64 output. Patch 2 adds one focused regression program. It covers the three overflowing extent families, negative and overflowing data offsets, alignment overflow, exact valid ELF32 and INT64_MAX boundaries, the SHT_NOBITS signed-store case, and zero/one alignment compatibility. Rejection cases require -1 and ELF_E_RANGE. Written controls reopen through public libelf APIs and pass eu-elflint. Validation performed: - Native AArch64 Linux, GCC 13.3, ASan/UBSan: 265 total, 255 PASS, 10 SKIP, 0 FAIL; optimized: 265 total, 256 PASS, 9 SKIP, 0 FAIL. - x86_64 Linux VM under CPU emulation, GCC 13.3 and Clang 18.1.3: targeted ASan/UBSan passes. The suite passes with only UBSan alignment disabled. - All three baseline ASan failures reproduce 3/3; patched calls return -1 and ELF_E_RANGE 3/3. libelf.so.1 and all 120 exported symbols are unchanged. In 40 native AArch64 CPU-pinned pairs, median added cost was 1.9 ns for no-op ELF_C_NULL, 26 ns for 100 sections, 290 ns for 1,000 sections, and 161 ns for a synthetic growing loop. A final 20-pair matched-build check measured the new signed-store guard at +3.5 ns for the growing loop and +15.5 ns for 1,000 sections. Output size and peak RSS were unchanged. Questions for maintainers: 1. Is INT64_MAX the desired bound, matching elf_update’s public result type? 2. Is preserving the observed d_align-zero placement the desired behavior? The series does not modify elf_newscn.c, strip/unstrip, compression, public headers, or section-header cache handling. Signed-off-by: Karan Kurani [email protected] Karan Kurani (2): libelf: Check output layout arithmetic before serialization tests: Check elf_update layout arithmetic boundaries libelf/elf32_updatenull.c | 147 +++++++±- tests/Makefile.am | 6 ± tests/update-overflow.c | 604 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 730 insertions(+), 27 deletions(-) create mode 100644 tests/update-overflow.c – 2.44.0
