Currently the BPF selftests can only be built by using the minimum kernel
configuration defined in tools/testing/selftests/bpf/config*. This poses a
problem in distribution kernels that may have some of the flags disabled or
set as module. For example, we have been running the tests regularly in
openSUSE Tumbleweed [1] [2] but to work around this fact we created a
special package [3] that build the tests against an auxiliary vmlinux with
the BPF Kconfig. We keep a list of known issues that may happen due to,
amongst other things, configuration mismatches [4] [5].
The maintenance of this package is far from ideal, especially for
enterprise kernels. The goal of this series is to enable the common usecase
of running the following in any system:
```sh
make -C tools/testing/selftests install \
SKIP_TARGETS= \
TARGETS=bpf \
BPF_STRICT_BUILD=0 \
O=/lib/modules/$(uname -r)/build
```
As an example, the following script targeting a minimal config can be used
for testing:
```sh
make defconfig
scripts/config --file .config \
--enable DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT \
--enable DEBUG_INFO_BTF \
--enable BPF_SYSCALL \
--enable BPF_JIT
make olddefconfig
make -j$(nproc)
make -j$(nproc) -C tools/testing/selftests install \
SKIP_TARGETS= \
TARGETS=bpf \
BPF_STRICT_BUILD=0
```
This produces a test_progs binary with 585 subtests, against the total of
717. Many of them will still fail or be skipped at runtime due to lack of
symbols, but at least there will be a clear way of building the tests.
[1]: https://openqa.opensuse.org/tests/5811715
[2]: https://openqa.opensuse.org/tests/5811730
[3]: https://src.opensuse.org/rmarliere/kselftests
[4]:
https://github.com/openSUSE/kernel-qe/blob/main/kselftests_known_issues.yaml
[5]:
https://openqa.opensuse.org/tests/5811730/logfile?filename=run_kselftests-config_mismatches.txt
---
Changes in v11:
- Gate the BTFIDS pretty-print on $(filter 1,$(V)) so V=0 and V=2 still
print the marker; only V=1 suppresses it (patch 6)
- Use asm volatile ("") in the uprobe_multi_func_{1,2,3} weak stubs to
match the strong definitions in prog_tests/uprobe_multi_test.c
(patch 10)
- Link to v10:
https://patch.msgid.link/[email protected]
Changes in v10:
- Drop $(wildcard $^) from the bench link recipe so cc fails cleanly on
the first missing input and the || fallback emits one SKIP-LINK line
instead of a wall of undefined-reference errors; also makes make -n
accurate (patch 9)
- Include <alloca.h> in testing_helpers.c for alloca() (patch 10)
- Link to v9:
https://patch.msgid.link/[email protected]
Changes in v9:
- Also pass KBUILD_OUTPUT=$(KMOD_O_VALID) when invoking kbuild so an
inherited command-line KBUILD_OUTPUT cannot disagree with O= (patch 2)
- Note BPFOBJ remains a normal prereq intentionally (patch 5)
- Sub-shell isolate cd/CC and use $@ for $(RM); gate the BTFIDS
pretty-print on $(V) so verbose mode does not double-print (patch 6)
- Restrict permissive compile-failure tolerance and partial-link to
test_progs%; runners with strong cross-object references (test_maps)
keep strict semantics (patches 6 and 8)
- Link to v8:
https://patch.msgid.link/[email protected]
Changes in v8:
- In permissive mode, keep source changes to already-built tests
triggering relinks, while using recipe-time $(wildcard ...) to pick
up fresh .test.o files without duplicate linker inputs (patch 8)
- Resolve relative O=/KBUILD_OUTPUT before recursing into test_kmods,
and only treat it as a kernel build dir when it contains
Module.symvers (patch 2)
- Note in the commit message that PERMISSIVE-mode bisecting here needs
the next two patches (patch 6)
- Clarify in the commit message that order-only skeleton prereqs do not
break the new-skel-via-local-header case (patch 5)
- Exclude not-built tests from the success count so summary and JSON
report them only as skipped (patch 7)
- Tighten commit messages for accuracy and clarity
- Link to v7:
https://patch.msgid.link/[email protected]
Changes in v7:
- Use $(abspath) for KMOD_O so relative O= paths resolve correctly
when make -C changes directory (patch 2)
- Guard make clean against missing KDIR unconditionally; there is
nothing to clean when kernel headers are absent (patch 2)
- Drop explicit $(TRUNNER_TEST_OBJS) from linker filter in strict mode;
$^ already contains them as normal prerequisites (patch 8)
- Link to v6:
https://patch.msgid.link/[email protected]
Changes in v6:
- Add --ignore-missing-args to -extras rsync so out-of-tree permissive
builds do not abort when .ko files are absent (patch 2)
- Use $(abspath) for KMOD_O so relative O= paths resolve correctly
when make -C changes directory (patch 2)
- Guard make clean against missing KDIR unconditionally so cleaning
does not abort when kernel headers are absent (patch 2)
- Remove stale skeleton headers in early-exit paths when .bpf.o is
missing on incremental builds (patch 3)
- Fix strict-mode skeleton rules: use && before temp file cleanup so
bpftool failures are not masked by rm -f exit code (patch 3)
- Track test filter selection separately from not_built so -t/-n flags
are respected for unbuilt tests (patch 7)
- Make TRUNNER_TEST_OBJS order-only only in permissive mode, preserving
incremental relinking in strict builds (patch 8)
- Drop explicit $(TRUNNER_TEST_OBJS) from linker filter in strict mode;
they are already in $^ as normal prerequisites (patch 8)
- Reorder: move skip-unbuilt-tests patch before partial-linking patch
for bisectability
- Link to v5:
https://patch.msgid.link/[email protected]
Changes in v5:
- Add BPF_STRICT_BUILD toggle as patch 1 so every subsequent patch
gates tolerance behind PERMISSIVE from the start, making the series
bisectable with strict-by-default at every point
- Fix O= commit message; make parent cp conditional (patch 2)
- Tolerate linked skeleton failures (patch 3)
- Skip feature detection for emit_tests (patch 4)
- Clarify bench is all-or-nothing in commit message (patch 8)
- Move stack_mprotect() to testing_helpers.c, drop weak stubs (patch 9)
- Report not-built tests as "SKIP (not built)" in output (patch 10)
- Drop overly broad 2>/dev/null || true from install rsync; rely solely
on --ignore-missing-args which already handles absent files (patch 11)
- Link to v4:
https://patch.msgid.link/[email protected]
Changes in v4:
- Drop the test_kmods kselftest module flow patch: lib.mk gen_mods_dir
invokes $(MAKE) -C $(TEST_GEN_MODS_DIR) without forwarding
RESOLVE_BTFIDS, breaking ASAN and GCC BPF CI builds (Makefile.modfinal
cannot find resolve_btfids in the kbuild output tree)
- Link to v3:
https://patch.msgid.link/[email protected]
Changes in v3:
- Split test_kmods patch into two: fix KDIR handling (O= passthrough,
EXTRA_CFLAGS/EXTRA_LDFLAGS clearing) and wire into lib.mk via
TEST_GEN_MODS_DIR
- Pass O= through to the kernel module build so artifacts land in the
output tree, not the source tree
- Clear EXTRA_CFLAGS and EXTRA_LDFLAGS when invoking the kernel build to
prevent host flags (e.g. -static) leaking into module compilation
- Replace the bespoke test_kmods pattern rule with lib.mk module
infrastructure (TEST_GEN_MODS_DIR); lib.mk now drives build and clean
lifecycle
- Make the .ko copy step resilient: emit SKIP instead of failing when a
module is absent
- Expand the uprobe weak stub comment in bpf_cookie.c to explain why
noinline is required
- Link to v2:
https://patch.msgid.link/[email protected]
Changes in v2:
- Skip test_kmods build/clean when KDIR directory does not exist
- Use `Module.symvers` instead of `.config` for in-tree detection
- Fix skeleton order-only prereqs commit message
- Guard BTFIDS step when .test.o is absent
- Add `__weak stack_mprotect()` stubs in `bpf_cookie.c` and `iters.c`
- Link to v1:
https://patch.msgid.link/[email protected]
To: Alexei Starovoitov <[email protected]>
To: Daniel Borkmann <[email protected]>
To: Andrii Nakryiko <[email protected]>
To: Martin KaFai Lau <[email protected]>
To: Eduard Zingerman <[email protected]>
To: Kumar Kartikeya Dwivedi <[email protected]>
To: Song Liu <[email protected]>
To: Yonghong Song <[email protected]>
To: Jiri Olsa <[email protected]>
To: Shuah Khan <[email protected]>
To: Nathan Chancellor <[email protected]>
To: Nick Desaulniers <[email protected]>
To: Bill Wendling <[email protected]>
To: Justin Stitt <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Assisted-by: {codex,claude}
Tested-by: Alan Maguire <[email protected]>
Signed-off-by: Ricardo B. Marliere <[email protected]>
---
Ricardo B. Marlière (11):
selftests/bpf: Add BPF_STRICT_BUILD toggle
selftests/bpf: Fix test_kmods KDIR to honor O= and distro kernels
selftests/bpf: Tolerate BPF and skeleton generation failures
selftests/bpf: Avoid rebuilds when running emit_tests
selftests/bpf: Make skeleton headers order-only prerequisites of .test.d
selftests/bpf: Tolerate test file compilation failures
selftests/bpf: Skip tests whose objects were not built
selftests/bpf: Allow test_progs to link with a partial object set
selftests/bpf: Tolerate benchmark build failures
selftests/bpf: Provide weak definitions for cross-test functions
selftests/bpf: Tolerate missing files during install
tools/testing/selftests/bpf/Makefile | 177 ++++++++++++++-------
.../testing/selftests/bpf/prog_tests/bpf_cookie.c | 17 +-
tools/testing/selftests/bpf/prog_tests/iters.c | 2 -
tools/testing/selftests/bpf/prog_tests/test_lsm.c | 22 ---
tools/testing/selftests/bpf/test_kmods/Makefile | 30 +++-
tools/testing/selftests/bpf/test_progs.c | 53 +++++-
tools/testing/selftests/bpf/test_progs.h | 1 +
tools/testing/selftests/bpf/testing_helpers.c | 18 +++
tools/testing/selftests/bpf/testing_helpers.h | 1 +
9 files changed, 226 insertions(+), 95 deletions(-)
---
base-commit: 2ca6723a5f7b68c739dba47b2639e3eaa7884b09
change-id: 20260401-selftests-bpf_misconfig-4c33ef5c56da
Best regards,
--
Ricardo B. Marlière <[email protected]>
