Make CONFIG_DEBUG_INFO_BTF a tristate. With =m the vmlinux BTF is not part of the kernel image: it is carried by a new module, btf_vmlinux, and loaded the first time something needs it. Nothing that works with =y stops working; the 5.4 MiB of read-only data (distribution config) is simply not there on systems where nothing uses it.
The only way to save that memory today is CONFIG_DEBUG_INFO_BTF=n, which a distribution cannot ship: one binary goes to every user, and off takes BTF away from the users of CO-RE, fentry/fexit, kfuncs, struct_ops, sched_ext or bpf-lsm. Whether BTF is used is a property of the workload, not of the build, so let the first user decide. The BTF is generated as before, but with =m the .BTF section is linked as a non-loadable section (like .comment), so the kernel image does not load it, and the final step that makes vmlinux from vmlinux.unstripped strips it: every boot image made from vmlinux, whether a raw binary or an ELF copy, is without it. Module BTF is generated against vmlinux.unstripped, which keeps it. .BTF_ids stays loadable, the verifier needs it once the BTF is loaded. The object that carries .BTF also carries .BTF.meta, the size and SHA-256 of the BTF (struct btf_vmlinux_meta, checked by the module notifier); the first link, which the BTF is generated from, gets a zeroed .BTF.meta of the same size. kernel/bpf/btf_vmlinux.c is an empty carrier module; scripts/gen-btf.sh gives it the vmlinux .BTF as its own .BTF section instead of generating split BTF for it, so that one module depends on vmlinux with =m; with CONFIG_DEBUG_INFO_BTF_MODULES all of them do, as before. CONFIG_BPF_PRELOAD is not selectable with =m: its iterator programs attach through the vmlinux BTF, so every bpffs mount (systemd does one at boot) would load it and defeat the point. Module BTF is still kept when a module loads, as with =y; the saving is the vmlinux BTF only. Programs that need kernel types before the root file system is mounted need btf_vmlinux.ko in the initramfs; the Kconfig help says so. The module has no exit: once loaded the BTF stays, as with =y. The runtime side -- loading the module on first use, checking it against .BTF.meta, deferring kfunc and struct_ops registrations and module BTF until it arrives -- and the IS_ENABLED()/$(subst m,y,...) preparation of the existing checks are in the preceding patches; this one makes it selectable. Tested with 1 GiB of memory, same tree, =y vs =m, both with CONFIG_DEBUG_INFO_BTF_MODULES=y: - MemTotal is ~5.4 MB higher with =m while the BTF is unused: the size of the .BTF section. - stat() of /sys/kernel/btf/vmlinux reports the BTF size before it is loaded, as the btf_sysfs selftest expects. - With BTF in use, MemFree is the same within run-to-run noise. - Modules loaded before the trigger (ext4, nf_conntrack and its kfuncs, xfrm_interface) appear in /sys/kernel/btf immediately and get BTF ids once the BTF is loaded; a socket filter loads without loading the module; a kprobe program calling bpf_get_current_task_btf(), opening /sys/kernel/btf/vmlinux or BPF_BTF_GET_NEXT_ID each load it. - A carrier module with one byte of its .BTF changed is refused with "BTF does not match this kernel" and leaves no state behind. - After the load: fstat/read/mmap of /sys/kernel/btf/vmlinux, a struct_ops map for tcp_congestion_ops, a syscall program calling the bpf_task_from_pid()/bpf_task_release() kfuncs, and modules loaded afterwards (nf_nat) all work as with =y. - =m without DEBUG_INFO_BTF_MODULES, and =y, build and pass the same tests. Signed-off-by: Jay Wang <[email protected]> --- Documentation/bpf/btf.rst | 35 ++++++++++++ Makefile | 5 +- include/asm-generic/vmlinux.lds.h | 31 +++++++++- kernel/bpf/Makefile | 4 ++ kernel/bpf/btf_vmlinux.c | 23 ++++++++ kernel/bpf/preload/Kconfig | 4 ++ lib/Kconfig.debug | 22 ++++++- scripts/Makefile.modfinal | 26 +++++++-- scripts/Makefile.vmlinux | 5 ++ scripts/gen-btf.sh | 95 +++++++++++++++++++++++++++++-- scripts/link-vmlinux.sh | 25 ++++++-- 11 files changed, 257 insertions(+), 18 deletions(-) create mode 100644 kernel/bpf/btf_vmlinux.c diff --git a/Documentation/bpf/btf.rst b/Documentation/bpf/btf.rst index 29de1222c3e7..1bd0d35cdb05 100644 --- a/Documentation/bpf/btf.rst +++ b/Documentation/bpf/btf.rst @@ -1276,6 +1276,41 @@ format.:: .long 58 .long 8206 # Line 8 Col 14 +6.1 Kernel BTF +-------------- + +With CONFIG_DEBUG_INFO_BTF=y the BTF of the kernel is generated at link time +from its DWARF and placed in the .BTF section of vmlinux, which is read-only +data of the kernel image. It is available as /sys/kernel/btf/vmlinux and, if +CONFIG_DEBUG_INFO_BTF_MODULES is set, module BTF is generated as split BTF +against it and available as /sys/kernel/btf/<module>. + +With CONFIG_DEBUG_INFO_BTF=m the same BTF is generated, but it is not part of +the kernel image or of the vmlinux ELF file (vmlinux.unstripped in the build +tree keeps it, for module BTF generation). It is delivered by the +btf_vmlinux module, which the kernel loads on demand the first time the BTF is +needed: when /sys/kernel/btf/vmlinux is read or mmap()ed, when kernel BTF +objects are enumerated (BPF_BTF_GET_NEXT_ID), or when a BPF program needs +kernel type information (an attach_btf_id, a kfunc call, a ksym, a map pointer +or a helper that takes or returns a kernel BTF pointer). Until then no memory +is used for it, and afterwards nothing differs from =y. In particular: + + * /sys/kernel/btf/vmlinux exists from boot with its final size. + * Modules loaded before the vmlinux BTF are exposed in /sys/kernel/btf right + away, their BTF is parsed and gets a BTF id once the vmlinux BTF is + loaded, together with their kfunc and struct_ops registrations. + * kfunc, dtor kfunc and struct_ops registrations of the kernel itself are + applied before the BTF becomes visible. + * The kernel only accepts the BTF it was built with: the size and SHA-256 of + the BTF are linked into the kernel and checked against the module. + * Once loaded the BTF stays; the module cannot be unloaded. + +If the module is not available (not installed, or the root file system is not +mounted yet), the kernel behaves as one built without BTF and retries next +time. CONFIG_BPF_PRELOAD is not available with =m: its iterators attach through +the vmlinux BTF, so mounting bpffs would load it. bpf_snprintf_btf() and bpf_seq_printf_btf() only use the BTF if it has +already been parsed, as they run in program context. + 7. Testing ========== diff --git a/Makefile b/Makefile index f561516e1735..7ce5d478abd3 100644 --- a/Makefile +++ b/Makefile @@ -1745,8 +1745,9 @@ endif # # *.ko are usually independent of vmlinux, but CONFIG_DEBUG_INFO_BTF_MODULES -# is an exception. -ifdef CONFIG_DEBUG_INFO_BTF_MODULES +# is an exception, and so is the btf_vmlinux module with CONFIG_DEBUG_INFO_BTF=m, +# which carries the vmlinux BTF. +ifneq ($(CONFIG_DEBUG_INFO_BTF_MODULES)$(filter m,$(CONFIG_DEBUG_INFO_BTF)),) KBUILD_BUILTIN := y modules: vmlinux endif diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index b2988aa12f66..204a9ee171c6 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -674,8 +674,18 @@ /* * .BTF + * + * With CONFIG_DEBUG_INFO_BTF=y the vmlinux BTF is loaded as read-only data and + * bounded by __start_BTF/__stop_BTF. With CONFIG_DEBUG_INFO_BTF=m it is + * linked as a non-loadable section (see BTF_NOLOAD in ELF_DETAILS), so that + * module BTF generation can read it from vmlinux.unstripped; it is stripped + * from vmlinux (scripts/Makefile.vmlinux), and the btf_vmlinux module carries + * a copy and provides it on demand at runtime. + * What is loaded instead is .BTF.meta, the size and hash of that BTF (see + * scripts/gen-btf.sh), empty in the first link that the BTF is generated + * from. .BTF_ids is needed by the kernel in both cases. */ -#ifdef CONFIG_DEBUG_INFO_BTF +#if IS_BUILTIN(CONFIG_DEBUG_INFO_BTF) #define BTF \ . = ALIGN(PAGE_SIZE); \ .BTF : AT(ADDR(.BTF) - LOAD_OFFSET) { \ @@ -685,10 +695,28 @@ .BTF_ids : AT(ADDR(.BTF_ids) - LOAD_OFFSET) { \ *(.BTF_ids) \ } +#elif IS_MODULE(CONFIG_DEBUG_INFO_BTF) +#define BTF \ + . = ALIGN(8); \ + .BTF.meta : AT(ADDR(.BTF.meta) - LOAD_OFFSET) { \ + BOUNDED_SECTION_BY(.BTF.meta, _BTF_meta) \ + } \ + . = ALIGN(PAGE_SIZE); \ + .BTF_ids : AT(ADDR(.BTF_ids) - LOAD_OFFSET) { \ + *(.BTF_ids) \ + } #else #define BTF #endif +#if IS_MODULE(CONFIG_DEBUG_INFO_BTF) +/* quoted: BTF is a macro, an unquoted .BTF here would expand it */ +#define BTF_NOLOAD \ + ".BTF" 0 : { *(".BTF") } +#else +#define BTF_NOLOAD +#endif + /* * Init task */ @@ -849,6 +877,7 @@ /* Required sections not related to debugging. */ #define ELF_DETAILS \ .comment 0 : { *(.comment) } \ + BTF_NOLOAD \ .symtab 0 : { *(.symtab) } \ .strtab 0 : { *(.strtab) } \ .shstrtab 0 : { *(.shstrtab) } \ diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 0b7db88f1bed..8a7ad4303c0d 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -43,6 +43,10 @@ endif ifeq ($(CONFIG_SYSFS),y) obj-$(subst m,y,$(CONFIG_DEBUG_INFO_BTF)) += sysfs_btf.o endif +# With CONFIG_DEBUG_INFO_BTF=m the vmlinux BTF is carried by this module +ifeq ($(CONFIG_DEBUG_INFO_BTF),m) +obj-m += btf_vmlinux.o +endif ifeq ($(CONFIG_BPF_JIT),y) obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o obj-$(CONFIG_BPF_SYSCALL) += cpumask.o diff --git a/kernel/bpf/btf_vmlinux.c b/kernel/bpf/btf_vmlinux.c new file mode 100644 index 000000000000..8d89b4bb3c43 --- /dev/null +++ b/kernel/bpf/btf_vmlinux.c @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Carrier module for the vmlinux BTF when CONFIG_DEBUG_INFO_BTF=m. + * + * This module has no code of its own. Its .BTF section is a copy of the + * vmlinux BTF (see scripts/gen-btf.sh), which the BTF module notifier in + * kernel/bpf/btf.c recognizes by module name and installs as the vmlinux BTF. + * The kernel loads it on demand, the first time the vmlinux BTF is needed. + * + * There is deliberately no module_exit(): once the BTF is in use it cannot + * be taken away again, exactly as with CONFIG_DEBUG_INFO_BTF=y. + */ +#include <linux/init.h> +#include <linux/module.h> + +static int __init btf_vmlinux_init(void) +{ + return 0; +} +module_init(btf_vmlinux_init); + +MODULE_DESCRIPTION("BTF type information for vmlinux"); +MODULE_LICENSE("GPL"); diff --git a/kernel/bpf/preload/Kconfig b/kernel/bpf/preload/Kconfig index aef7b0bc96d6..b1600bdce7a0 100644 --- a/kernel/bpf/preload/Kconfig +++ b/kernel/bpf/preload/Kconfig @@ -6,6 +6,10 @@ menuconfig BPF_PRELOAD # The dependency on !COMPILE_TEST prevents it from being enabled # in allmodconfig or allyesconfig configurations depends on !COMPILE_TEST + # The preloaded iterators attach through the vmlinux BTF, so with + # CONFIG_DEBUG_INFO_BTF=m every bpffs mount would load the BTF, which + # defeats the point of =m on any system that mounts bpffs at boot. + depends on DEBUG_INFO_BTF!=m help This builds kernel module with several embedded BPF programs that are pinned into BPF FS mount point as human readable files that are diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 134b15a44625..418025c5e657 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -396,7 +396,7 @@ config DEBUG_INFO_SPLIT Incompatible with older versions of ccache. config DEBUG_INFO_BTF - bool "Generate BTF type information" + tristate "Generate BTF type information" depends on !DEBUG_INFO_SPLIT && !DEBUG_INFO_REDUCED depends on !GCC_PLUGIN_RANDSTRUCT || COMPILE_TEST depends on BPF_SYSCALL @@ -408,6 +408,26 @@ config DEBUG_INFO_BTF Turning this on requires pahole v1.22 or later, which will convert DWARF type info into equivalent deduplicated BTF type info. + If built as a module (=m), the vmlinux BTF is not part of the + kernel image. It is carried by the btf_vmlinux module, which is + loaded on demand the first time the BTF is needed: when a BPF + program requires kernel type information, or when + /sys/kernel/btf/vmlinux is opened. Until then, no memory is + spent on it. The vmlinux ELF file does not carry the BTF + either; module BTF is generated against vmlinux.unstripped, and + tools that read the BTF from a file can use that or + /sys/kernel/btf/vmlinux. + + Module BTF (DEBUG_INFO_BTF_MODULES) is kept when a module loads, + as with =y, and registered once the vmlinux BTF is available; the + saving is the vmlinux BTF only. + + If BPF programs that use kernel types run before the root file + system is mounted, put btf_vmlinux.ko into the initramfs: until + the module can be loaded, such programs fail as on a kernel + without BTF. Not compatible with BPF_PRELOAD, whose iterators + would load the BTF at every bpffs mount. + config PAHOLE_HAS_BTF_TAG def_bool PAHOLE_VERSION >= 123 depends on CC_IS_CLANG diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 01a37ec872b9..6a0958cf0e6f 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -38,20 +38,34 @@ quiet_cmd_ld_ko_o = LD [M] $@ $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) \ -T $(objtree)/scripts/module.lds -o $@ $(filter %.o, $^) +# The ELF file with the vmlinux BTF: with CONFIG_DEBUG_INFO_BTF=m the BTF is +# stripped from vmlinux (scripts/Makefile.vmlinux), vmlinux.unstripped keeps it. +btf-vmlinux := $(objtree)/vmlinux$(if $(filter m,$(CONFIG_DEBUG_INFO_BTF)),.unstripped) + quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ - if [ ! -f $(objtree)/vmlinux ]; then \ + if [ ! -f $(btf-vmlinux) ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ else \ - $(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ + $(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(btf-vmlinux) $@; \ fi; -# Re-generate module BTFs if either module's .ko or vmlinux changed -%.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE - +$(call if_changed,ld_ko_o) +# Modules that get a .BTF section: all of them with CONFIG_DEBUG_INFO_BTF_MODULES, +# otherwise only the vmlinux BTF carrier module with CONFIG_DEBUG_INFO_BTF=m. ifdef CONFIG_DEBUG_INFO_BTF_MODULES - +$(if $(newer-prereqs),$(call cmd,btf_ko)) +btf-modules := $(modules:%.o=%.ko) +else ifeq ($(CONFIG_DEBUG_INFO_BTF),m) +btf-modules := $(filter %/btf_vmlinux.ko,$(modules:%.o=%.ko)) +# Only the carrier depends on vmlinux, not every module +ifdef KBUILD_BUILTIN +$(btf-modules): $(btf-vmlinux) +endif endif + +# Re-generate module BTFs if either module's .ko or vmlinux changed +%.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(btf-vmlinux)) FORCE + +$(call if_changed,ld_ko_o) + +$(if $(and $(filter $@,$(btf-modules)),$(newer-prereqs)),$(call cmd,btf_ko)) +$(call cmd,check_tracepoint) targets += $(modules:%.o=%.ko) $(modules:%.o=%.mod.o) .module-common.o diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux index fcae1e432d9a..557db1ee1f3b 100644 --- a/scripts/Makefile.vmlinux +++ b/scripts/Makefile.vmlinux @@ -86,6 +86,11 @@ remove-section-$(CONFIG_ARCH_VMLINUX_NEEDS_RELOCS) += '.rel*' '!.rel*.dyn' # for compatibility with binutils < 2.32 # https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c12d9fa2afe7abcbe407a00e15719e1a1350c2a7 remove-section-$(CONFIG_ARCH_VMLINUX_NEEDS_RELOCS) += '.rel.*' +# With CONFIG_DEBUG_INFO_BTF=m the btf_vmlinux module carries the vmlinux BTF; +# only vmlinux.unstripped keeps it, for module BTF generation. +ifeq ($(CONFIG_DEBUG_INFO_BTF),m) +remove-section-y += .BTF +endif remove-symbols := -w --strip-unneeded-symbol='__mod_device_table__*' diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh index 8ca96eb10a69..d808d0006d32 100755 --- a/scripts/gen-btf.sh +++ b/scripts/gen-btf.sh @@ -22,16 +22,27 @@ # - ${1}.btf.o ready for linking into vmlinux # - ${1}.BTF_ids with .BTF_ids data blob # This output is consumed by scripts/link-vmlinux.sh +# +# With CONFIG_DEBUG_INFO_BTF=m the .BTF section in ${1}.btf.o is not +# allocatable, so the kernel image does not carry the BTF; vmlinux.unstripped +# does, for module BTF generation, and scripts/Makefile.vmlinux strips it from +# vmlinux. ${1}.btf.o then also carries .BTF.meta, the size and SHA-256 of +# the BTF for the kernel (struct btf_vmlinux_meta); "--placeholder ${1}" +# produces a ${1}.btf.o with a zeroed .BTF.meta and no .BTF for the first +# vmlinux link, which the BTF is generated from. The btf_vmlinux module gets +# no BTF of its own; its .BTF section is a copy of the vmlinux BTF, extracted +# from --btf_base. set -e usage() { - echo "Usage: $0 [--btf_base <file>] <target ELF file>" + echo "Usage: $0 [--btf_base <file>] [--placeholder] <target ELF file>" exit 1 } BTF_BASE="" +PLACEHOLDER="" while [ $# -gt 0 ]; do case "$1" in @@ -39,6 +50,10 @@ while [ $# -gt 0 ]; do BTF_BASE="$2" shift 2 ;; + --placeholder) + PLACEHOLDER=1 + shift + ;; -*) echo "Unknown option: $1" >&2 usage @@ -60,6 +75,10 @@ is_enabled() { grep -q "^$1=y" ${objtree}/include/config/auto.conf } +is_module() { + grep -q "^$1=m" ${objtree}/include/config/auto.conf +} + case "${KBUILD_VERBOSE}" in *1*) set -x @@ -79,6 +98,30 @@ gen_btf_data() --btf ${btf1} "${ELF_FILE}" } +# Write one byte with value $1 (0..255) +put_byte() +{ + printf "\\$(printf '%03o' "$1")" +} + +# CONFIG_DEBUG_INFO_BTF=m: write struct btf_vmlinux_meta { u32 size; u8 +# sha256[32]; } for the BTF in $1 to $2, in the target's byte order. +gen_btf_meta() +{ + size=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" "$1") + sha256=$(sha256sum < "$1" | cut -d' ' -f1) + { + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + for shift in 24 16 8 0; do put_byte $(( (size >> shift) & 255 )); done + else + for shift in 0 8 16 24; do put_byte $(( (size >> shift) & 255 )); done + fi + for byte in $(echo "${sha256}" | sed 's/../& /g'); do + put_byte $(( 0x${byte} )) + done + } > "$2" +} + gen_btf_o() { btf_data=${ELF_FILE}.btf.o @@ -88,9 +131,23 @@ gen_btf_o() # deletes all symbols including __start_BTF and __stop_BTF, which will # be redefined in the linker script. echo "" | ${CC} ${CLANG_FLAGS} ${KBUILD_CPPFLAGS} ${KBUILD_CFLAGS} -fno-lto -c -x c -o ${btf_data} - - ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF \ - --set-section-flags .BTF=alloc,readonly ${btf_data} - ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + if is_module CONFIG_DEBUG_INFO_BTF; then + # CONFIG_DEBUG_INFO_BTF=m: .BTF stays non-allocatable, kept in + # vmlinux.unstripped for module BTF but not loaded; the btf_vmlinux + # module provides it at runtime. What is loaded is .BTF.meta, + # its size and hash, so that /sys/kernel/btf/vmlinux has the right + # size from boot and only the matching BTF is accepted. + gen_btf_meta ${ELF_FILE}.BTF ${ELF_FILE}.BTF.meta + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF \ + --set-section-flags .BTF=readonly \ + --add-section .BTF.meta=${ELF_FILE}.BTF.meta \ + --set-section-flags .BTF.meta=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --only-section=.BTF.meta --strip-all ${btf_data} + else + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF \ + --set-section-flags .BTF=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + fi # Change e_type to ET_REL so that it can be used to link final vmlinux. # GNU ld 2.35+ and lld do not allow an ET_EXEC input. @@ -121,6 +178,7 @@ cleanup() { rm -f "${ELF_FILE}.BTF.1" rm -f "${ELF_FILE}.BTF" + rm -f "${ELF_FILE}.BTF.meta" if [ "${BTFGEN_MODE}" = "module" ]; then rm -f "${ELF_FILE}.BTF.base" rm -f "${ELF_FILE}.BTF_ids" @@ -133,6 +191,35 @@ if [ -n "${BTF_BASE}" ]; then BTFGEN_MODE="module" fi +if [ -n "${PLACEHOLDER}" ]; then + btf_data=${ELF_FILE}.btf.o + echo "" | ${CC} ${CLANG_FLAGS} ${KBUILD_CPPFLAGS} ${KBUILD_CFLAGS} -fno-lto -c -x c -o ${btf_data} - + dd if=/dev/zero of=${ELF_FILE}.BTF.meta bs=36 count=1 2>/dev/null + ${OBJCOPY} --add-section .BTF.meta=${ELF_FILE}.BTF.meta \ + --set-section-flags .BTF.meta=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF.meta --strip-all ${btf_data} + exit 0 +fi + +# CONFIG_DEBUG_INFO_BTF=m: the btf_vmlinux module carries the vmlinux BTF +# itself. Its own types are of no interest, so instead of generating split +# BTF for it, copy the (non-loadable) .BTF section of --btf_base +# (vmlinux.unstripped) into the module. +# The kernel recognizes the module by name and treats its .BTF as base BTF. +case "${BTFGEN_MODE}:${ELF_FILE}" in +module:*/btf_vmlinux.ko) + if is_module CONFIG_DEBUG_INFO_BTF; then + # -O binary only emits allocatable sections; make .BTF one for + # the extraction. ${BTF_BASE} itself is not modified. + ${OBJCOPY} -O binary --only-section=.BTF \ + --set-section-flags .BTF=alloc,load,readonly \ + "${BTF_BASE}" "${ELF_FILE}.BTF" + ${OBJCOPY} --add-section .BTF="${ELF_FILE}.BTF" "${ELF_FILE}" + exit 0 + fi + ;; +esac + gen_btf_data case "${BTFGEN_MODE}" in diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index ab0b8125c8cb..a9a066267ef6 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -37,6 +37,15 @@ is_enabled() { grep -q "^$1=y" include/config/auto.conf } +is_module() { + grep -q "^$1=m" include/config/auto.conf +} + +# =y or =m +is_set() { + grep -q "^$1=[ym]" include/config/auto.conf +} + # Nice output in kbuild format # Will be suppressed by "make -s" info() @@ -211,17 +220,25 @@ if is_enabled CONFIG_KALLSYMS; then kallsyms .tmp_vmlinux0.syms .tmp_vmlinux0.kallsyms fi -if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then +if is_module CONFIG_DEBUG_INFO_BTF; then + # The kernel refers to the size and hash of its BTF, which only the + # BTF generated from the first link can provide; link a placeholder + # of the same layout until then. + ${CONFIG_SHELL} ${srctree}/scripts/gen-btf.sh --placeholder .tmp_vmlinux0 + btf_vmlinux_bin_o=.tmp_vmlinux0.btf.o +fi + +if is_enabled CONFIG_KALLSYMS || is_set CONFIG_DEBUG_INFO_BTF; then # The kallsyms linking does not need debug symbols, but the BTF does. - if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + if ! is_set CONFIG_DEBUG_INFO_BTF; then strip_debug=1 fi vmlinux_link .tmp_vmlinux1 fi -if is_enabled CONFIG_DEBUG_INFO_BTF; then +if is_set CONFIG_DEBUG_INFO_BTF; then info BTF .tmp_vmlinux1 if ! ${CONFIG_SHELL} ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then echo >&2 "Failed to generate BTF for vmlinux" @@ -287,7 +304,7 @@ fi vmlinux_link "${VMLINUX}" -if is_enabled CONFIG_DEBUG_INFO_BTF; then +if is_set CONFIG_DEBUG_INFO_BTF; then info BTFIDS ${VMLINUX} ${RESOLVE_BTFIDS} --patch_btfids ${btfids_vmlinux} ${VMLINUX} fi -- 2.47.3
