This series adds support for parsing DWARF Call Frame Information (CFI) from the .eh_frame_hdr and .eh_frame sections of user space ELF files.
The code is based on the deferred unwind user work originally done for SFrame by Josh, Steven, and myself: v4 : https://lore.kernel.org/all/[email protected]/ v10: https://lore.kernel.org/all/[email protected]/ v16: https://lore.kernel.org/all/[email protected]/ The goal is to make user space stack traces available in-kernel without requiring frame pointers and without copying large parts of the user stack for later processing. Today, reliable user stack traces from the kernel generally requires frame pointers. Otherwise, profilers such as perf have to copy large amounts of user space stack into the kernel ring buffer and process it later. Frame pointers are simple and robust, but enabling them for all executables and libraries has a performance cost. Another issue is that the frame layout can vary between compilers and architectures, and on architectures such as s390 there is no defined frame layout which allows reliable frame-pointer based stack tracing. The only way to perform user space profiling on there architectures is to copy the user space into the kernel buffer. The .eh_frame section is already emitted by most toolchains on most architectures unless explicitly disabled. It contains DWARF CFI describing how to recover the caller state at any point in a function. The .eh_frame_hdr section provides a binary search table for looking up the Frame Description Entry (FDE) for a given instruction pointer (IP). Because the .eh_frame_hdr and .eh_frame sections live in the ELF file, they need to be faulted in when used. This means that walking the user space stack requires being in a faultable context. As profilers like perf request a stack trace in interrupt or NMI context, the walking cannot be done when requested. This series reuses the deferred unwind user framework, that performed the actual user stack trace is later in a faultable context, before the task returns to user space. This series implements .eh_frame[_hdr] support for the deferred unwind user code and enables it for x86-64 and s390. It intentionally not implement a complete DWARF unwinder. It evaluates only the subset of DWARF CFI needed for stack tracing: - Call Frame Address (CFA): Using rule from DWARF CFI. - Stack pointer (SP): Using an implicit rule based on the CFA definition (SP = CFA for most architectures). - Frame pointer (FP): Using rule from DWARF CFI. - Return address (RA): Using rule from DWARF CFI. Unsupported CFI instructions, unsupported expressions, invalid data, or user memory faults stop the stack tracing safely and results in a partial stack trace. This series applies on top of v7.2 tag: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v7.2 The to be stack-traced user space executables and libraries need to contain .eh_frame_hdr and .eh_frame sections as well as a GNU_EH_FRAME PHDR. Namhyung Kim's related perf tools deferred callchain support can be used for testing, for example: perf record --call-graph fp,defer ... perf report perf script Why .eh_frame? This series is not meant to replace or undermine the SFrame work. SFrame remains the simpler and more purpose-built format for user stack tracing. The motivation for .eh_frame support is pragmatic: .eh_frame is already widely deployed today. - Availability and maturity: .eh_frame is already present in most ELF binaries for C++ exception handling. It has been used in production for decades for exception handling and debugger stack unwinding. - Toolchain support: .eh_frame is supported across all major compilers and architectures today, whereas .sframe adoption is still emerging. - Size: .sframe would be added in addition to existing .eh_frame[_hdr] rather than replacing it, increasing the ELF file size. [1] Addressing historical DWARF concerns: Using DWARF for kernel unwinding has a bad history. Previous attempts were complex, fragile, slow, and hard to maintain. Hand-written assembly and the complexity of the DWARF state machine were among the reasons the simpler ORC kernel unwind format was developed. [2,3,4] However, this implementation for user space stack tracing differs from those problematic kernel unwinding attempts: - It stack traces user space, not kernel. - It runs in a deferred, faultable context, not in NMI, interrupt, or oops context. - It may return partial stack traces. Bad CFI, unsupported operations, invalid user memory, or faults are allowed to terminate the unwind. - It implements only the CFI subset needed for stack tracing, not a general DWARF unwinder. - It does not include a general-purpose DWARF expression VM. Expression size is bounded. Only a small number of pattern-matched expressions is supported (e.g. DRAP and PLT expressions on x86). Unsupported expressions cause stack tracing to fail safely. - All user memory access uses [unsafe_]get_user() with proper bounds checking and fault handling. - Corruption detection with automatic section removal on invalid .eh_frame prevents further stack tracing attempts. Limitations and future work: - CIE version 1 support only and no DWARF64 support, as I have not run into either during my testing. - Signal frames are not handled yet. An architecture hook could support unwinding through FDEs whose CIE augmentation contains 'S' (signal frame), similar to Glibc's SFrame backtrace() support. See also my "[RFC PATCH v1 0/5] s390: Signal frame user space unwinding". [5] - x86-32, x86-x32, and 32-bit compat mode support not implemented yet. - CIE caching would be useful. Reading an FDE requires reading its referenced CIE first to obtain the FDE encoding. Most .eh_frame sections have only a very small number of CIEs, often one default CIE shared by most FDEs and possibly one signal frame CIE. Caching the last CIE per section, together with the initial CFA, FP, and RA rules, would avoid repeated CIE parsing and initial CFI instruction processing. [1]: https://lore.kernel.org/all/CAN30aBFVDxeoXApn_g_Hw0Ayhi4V=m7ccx8udo6zdti6xa-...@mail.gmail.com/ [2]: https://lwn.net/Articles/727553/ [3]: https://lkml.org/lkml/2012/2/10/356 [4]: https://lkml.org/lkml/2017/5/20/165 [5]: https://lore.kernel.org/all/[email protected]/ Patches 1-6 add base functionality to unwind user to support .eh_frame- based (or .sframe-based) unwinding. Patches originate from my latest .sframe patch series. Patches 7-10 add the basic infrastructure for reading .eh_frame_hdr and .eh_frame sections and storing them in a per-mm maple tree. Patches 11-14 wire up the eh_frame infrastructure to the unwind user framework and add error handling and debugging support. Patch 15 duplicates registered .eh_frame_hdr section data on clone/fork. Patch 16 adds an experimental linear .eh_frame search fallback, for the rare case, that .eh_frame_hdr does not contain a binary search table. Patch 17 improves .eh_frame DWARF CFI instruction processing. Patch 18 enables architectures to implement selected DWARF expressions in CFI instructions. Patches 19-22 enable .eh_frame unwinding on x86-64 with minimal DWARF expression support for DRAP and PLT expressions. Patches 23-24 enable .eh_frame unwinding on s390. Patch 25 adds a prctl() interface for (un)registering .eh_frame_hdr sections for shared libraries. I will send a related test-patch for Glibc separately. Regards, Jens Jens Remus (24): unwind_user: Add generic and arch-specific headers to MAINTAINERS unwind_user: Stop when reaching an outermost frame unwind_user: Enable archs that pass RA in a register unwind_user: Flexible FP/RA recovery rules unwind_user: Flexible CFA recovery rules unwind_user: Enable archs that define CFA = SP_callsite + offset unwind_user/eh_frame: Add support for reading .eh_frame_hdr section unwind_user/eh_frame: Store .eh_frame_hdr section data in per-mm maple tree unwind_user/eh_frame: Add support for reading .eh_frame section unwind_user/eh_frame: Detect .eh_frame_hdr sections in executables unwind_user/eh_frame: Wire up unwind_user to eh_frame unwind_user/eh_frame: Remove .eh_frame[_hdr] section on detected corruption unwind_user/eh_frame: Show file name in debug output unwind_user/eh_frame: Add .eh_frame[_hdr] validation option unwind_user/eh_frame: Duplicate registered .eh_frame[_hdr] section data on clone/fork unwind_user/eh_frame: Add linear .eh_frame search fallback unwind_user/eh_frame: Ignore DW_CFA_GNU_args_size unwind_user/eh_frame: Add support for DWARF expressions unwind_user/eh_frame/x86: Enable eh_frame unwinding on x86 unwind_user/eh_frame/x86: Handle PLT expressions unwind_user/eh_frame/x86: Handle DRAP expressions s390/ptrace: Provide frame_pointer() unwind_user/eh_frame/s390: Enable eh_frame unwinding on s390 unwind_user/eh_frame: Add prctl() interface for (un)registering .eh_frame_hdr sections Josh Poimboeuf (1): x86/uaccess: Add unsafe_copy_from_user() implementation MAINTAINERS | 5 + arch/Kconfig | 35 + arch/s390/Kconfig | 1 + arch/s390/include/asm/ptrace.h | 6 + arch/s390/include/asm/unwind_user.h | 71 + arch/s390/include/asm/unwind_user_eh_frame.h | 24 + arch/x86/Kconfig | 1 + arch/x86/include/asm/mmu.h | 2 +- arch/x86/include/asm/uaccess.h | 39 +- arch/x86/include/asm/unwind_user.h | 77 +- arch/x86/include/asm/unwind_user_eh_frame.h | 153 ++ fs/binfmt_elf.c | 49 +- include/asm-generic/Kbuild | 1 + include/asm-generic/unwind_user_eh_frame.h | 84 + include/linux/eh_frame.h | 104 + include/linux/mm_types.h | 3 + include/linux/unwind_user.h | 20 + include/linux/unwind_user_eh_frame_types.h | 41 + include/linux/unwind_user_types.h | 51 +- include/uapi/linux/eh_frame.h | 14 + include/uapi/linux/prctl.h | 4 + kernel/fork.c | 10 + kernel/sys.c | 11 + kernel/unwind/Makefile | 3 +- kernel/unwind/eh_frame.c | 1771 ++++++++++++++++++ kernel/unwind/eh_frame.h | 83 + kernel/unwind/eh_frame_debug.h | 71 + kernel/unwind/user.c | 142 +- mm/init-mm.c | 2 + mm/mmap.c | 5 + 30 files changed, 2842 insertions(+), 41 deletions(-) create mode 100644 arch/s390/include/asm/unwind_user.h create mode 100644 arch/s390/include/asm/unwind_user_eh_frame.h create mode 100644 arch/x86/include/asm/unwind_user_eh_frame.h create mode 100644 include/asm-generic/unwind_user_eh_frame.h create mode 100644 include/linux/eh_frame.h create mode 100644 include/linux/unwind_user_eh_frame_types.h create mode 100644 include/uapi/linux/eh_frame.h create mode 100644 kernel/unwind/eh_frame.c create mode 100644 kernel/unwind/eh_frame.h create mode 100644 kernel/unwind/eh_frame_debug.h base-commit: 8d3ae59288f1e7d58d76558a6ee96d533bc5019f -- 2.53.0
