Hi Alan, On Fri Apr 10, 2026 at 7:06 AM -03, Alan Maguire wrote: > On 06/04/2026 19:22, Ricardo B. Marlière wrote: >> Currently the BPF selftests can only be built by using the minimum kernel >> configuration defined in tools/testing/selftests/bpf/config*. This poses a >> problem in distribution kernels that may have some of the flags disabled or >> set as module. For example, we have been running the tests regularly in >> openSUSE Tumbleweed [1] [2] but to work around this fact we created a >> special package [3] that build the tests against an auxiliary vmlinux with >> the BPF Kconfig. We keep a list of known issues that may happen due to, >> amongst other things, configuration mismatches [4] [5]. >> >> The maintenance of this package is far from ideal, especially for >> enterprise kernels. The goal of this series is to enable the common usecase >> of running the following in any system: >> >> ```sh >> make -C tools/testing/selftests install \ >> SKIP_TARGETS= \ >> TARGETS=bpf \ >> O=/lib/modules/$(uname -r)/build >> ``` >> >> As an example, the following script targeting a minimal config can be used >> for testing: >> >> ```sh >> make defconfig >> scripts/config --file .config \ >> --enable DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT \ >> --enable DEBUG_INFO_BTF \ >> --enable BPF_SYSCALL \ >> --enable BPF_JIT >> make olddefconfig >> make -j$(nproc) >> make -j$(nproc) -C tools/testing/selftests SKIP_TARGETS= TARGETS=bpf install >> ``` >> >> This produces a test_progs binary with 579 subtests, against the total of >> 708. Many of them will still fail or be skipped at runtime due to lack of >> symbols, but at least there will be a clear way of building the tests. >> > > hi Ricardo, I've been looking at this series and one concern I had was if > tolerating failures to build with incomplete config would create problems > in detecting failures when the config is complete, i.e. in BPF CI. So for > example if say a BPF skeleton didn't compile when it really should due to > an introduced bug, we wouldn't want to drive on in all cases.
Agreed. > > However, from what I can see, CI held up well with these changes in place [1]. > > That said it might be nice to preserve a "strict" mode which fails when > we hit compilation issues etc, failing as we do today; I couldn't find any > existing > general kselftest concept that fits with that but it would be valuable I think > if possible. Perhaps we could default to non-strict mode when a make -C is > initiated > or something like that, and continue to operate strictly when building > directly > in tools/testing/selftests/bpf? > > I ran the changes through the usual workflow I use - build bpf selftests at > tools/testing/selftests/bpf level and run ./test_progs ; no regressions were > observed, so feel free to add for the series > > Tested-by: Alan Maguire <[email protected]> Thanks for testing! > > There are some comments on sashiko.dev [2] that would be worth looking through > too, in particular the concern about O= and M= mentioned for patch 1. I will send v5 with those addressed and possibly a strict-mode switch. > > [1] https://github.com/kernel-patches/bpf/actions/runs/24168181680 > [2] > https://sashiko.dev/#/patchset/20260406-selftests-bpf_misconfig-v4-0-9914f50efdf7%40suse.com > > [1]: https://openqa.opensuse.org/tests/5811715 >> [2]: https://openqa.opensuse.org/tests/5811730 >> [3]: https://src.opensuse.org/rmarliere/kselftests >> [4]: >> https://github.com/openSUSE/kernel-qe/blob/main/kselftests_known_issues.yaml >> [5]: >> https://openqa.opensuse.org/tests/5811730/logfile?filename=run_kselftests-config_mismatches.txt >> >> Assisted-by: {codex,claude} >> Signed-off-by: Ricardo B. Marlière <[email protected]> >> --- >> Changes in v4: >> - Drop the test_kmods kselftest module flow patch: lib.mk gen_mods_dir >> invokes $(MAKE) -C $(TEST_GEN_MODS_DIR) without forwarding >> RESOLVE_BTFIDS, breaking ASAN and GCC BPF CI builds (Makefile.modfinal >> cannot find resolve_btfids in the kbuild output tree) >> - Link to v3: >> >> https://patch.msgid.link/[email protected] >> >> Changes in v3: >> - Split test_kmods patch into two: fix KDIR handling (O= passthrough, >> EXTRA_CFLAGS/EXTRA_LDFLAGS clearing) and wire into lib.mk via >> TEST_GEN_MODS_DIR >> - Pass O= through to the kernel module build so artifacts land in the >> output tree, not the source tree >> - Clear EXTRA_CFLAGS and EXTRA_LDFLAGS when invoking the kernel build to >> prevent host flags (e.g. -static) leaking into module compilation >> - Replace the bespoke test_kmods pattern rule with lib.mk module >> infrastructure (TEST_GEN_MODS_DIR); lib.mk now drives build and clean >> lifecycle >> - Make the .ko copy step resilient: emit SKIP instead of failing when a >> module is absent >> - Expand the uprobe weak stub comment in bpf_cookie.c to explain why >> noinline is required >> - Link to v2: >> >> https://patch.msgid.link/[email protected] >> >> Changes in v2: >> - Skip test_kmods build/clean when KDIR directory does not exist >> - Use `Module.symvers` instead of `.config` for in-tree detection >> - Fix skeleton order-only prereqs commit message >> - Guard BTFIDS step when .test.o is absent >> - Add `__weak stack_mprotect()` stubs in `bpf_cookie.c` and `iters.c` >> - Link to v1: >> >> https://patch.msgid.link/[email protected] >> >> --- >> Ricardo B. Marlière (10): >> selftests/bpf: Fix test_kmods KDIR to honor O= and distro kernels >> selftests/bpf: Tolerate BPF and skeleton generation failures >> selftests/bpf: Avoid rebuilds when running emit_tests >> selftests/bpf: Make skeleton headers order-only prerequisites of >> .test.d >> selftests/bpf: Tolerate test file compilation failures >> selftests/bpf: Allow test_progs to link with a partial object set >> selftests/bpf: Tolerate benchmark build failures >> selftests/bpf: Provide weak definitions for cross-test functions >> selftests/bpf: Skip tests whose objects were not built >> selftests/bpf: Tolerate missing files during install >> >> tools/testing/selftests/bpf/Makefile | 113 >> ++++++++++++++------- >> .../testing/selftests/bpf/prog_tests/bpf_cookie.c | 24 ++++- >> tools/testing/selftests/bpf/prog_tests/iters.c | 10 +- >> tools/testing/selftests/bpf/test_kmods/Makefile | 15 ++- >> tools/testing/selftests/bpf/test_progs.c | 7 +- >> 5 files changed, 120 insertions(+), 49 deletions(-) >> --- >> base-commit: a1aa9ef47c299c5bbc30594d3c2f0589edf908e6 >> change-id: 20260401-selftests-bpf_misconfig-4c33ef5c56da >> >> Best regards, >> -- >> Ricardo B. Marlière <[email protected]> >> >>
