> selftests/bpf: Benchmark tracing_multi bpf progs attachment
TL;DR:
- Benchmark may hit EMFILE with default fd limits (about 5000 fds needed) and
exit
- Reported 7.14x speedup compares unequal work: fentry timing includes load
and BTF resolution
> diff --git
> a/tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c
> b/tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c
> new file mode 100644
> index 000000000000..6cc6851eb488
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c
> @@ -0,0 +1,193 @@
[ ... ]
> +static void prepare(void)
> +{
>
> [ ... ]
>
> + bpf_program__set_autoload(target->progs.dummy_fentry, false);
> + bpf_program__set_autoload(target->progs.dummy_fentry_multi,
> false);
> + err = tracing_multi_attach_progs__load(target);
> + if (err) {
> + fprintf(stderr, "failed to load target program %d:
> %s\n",
> + i, strerror(-err));
> + exit(1);
> + }
Does this benchmark need more than the default RLIMIT_NOFILE?
Every loaded skeleton keeps two fds open until __destroy(), the object BTF
fd and the program fd. The BTF fd is not released after load:
bpf_object_load()
bpf_object_post_load_cleanup()
bpf_object_cleanup_btf()
only frees the module/vmlinux BTF, while obj->btf and its fd stay open until
bpf_object__close(). So prepare() alone holds about 2 * TARGET_CNT = 2000
fds for the 1000 target objects.
create_and_attach_fentry() then adds roughly 2000 more for the 1000 fentry
skeletons, plus 1000 entries in link_fds[], for a peak of about 5000 open
fds.
Nothing in bench.c, testing_helpers or libbpf raises RLIMIT_NOFILE (the only
RLIMIT_NOFILE user in selftests/bpf is map_tests/sk_storage_map.c). With a
1024 soft limit, './bench tracing-multi-attach-progs' looks like it would
hit EMFILE at around target 510 in prepare() and exit(1). Even a 4096 hard
limit is below the peak reached in create_and_attach_fentry().
Should setup() or prepare() raise RLIMIT_NOFILE, as sk_storage_map.c does, or
should the 'ulimit -n' requirement be documented in the commit message?
[ ... ]
> +static long create_and_attach_fentry(void)
> +{
> + int link_fds[TARGET_CNT];
> + long start_ns, end_ns;
> + int err, i, j;
> +
> + start_ns = get_time_ns();
> +
> + /* An ordinary fentry program is bound to one target when it is loaded.
> */
>
> [ ... ]
>
> + err =
> bpf_program__set_attach_target(fentry->progs.dummy_fentry, ctx.target_fds[i],
> + "target");
> + err = err ?: tracing_multi_attach_progs__load(fentry);
>
> [ ... ]
>
> + for (i = 0; i < TARGET_CNT; i++) {
> + link_fds[i] = bpf_link_create(ctx.fentry_prog_fds[i], 0,
> BPF_TRACE_FENTRY, NULL);
> + if (link_fds[i] < 0) {
>
> [ ... ]
>
> + end_ns = get_time_ns();
The commit message describes this as a comparison of "1000 individual
fentry link creations with one fentry.multi link creation", but do the two
timed regions measure the same work?
The fentry window starts at the get_time_ns() above, before the first loop,
so it covers:
1000 x tracing_multi_attach_progs__open()
1000 x bpf_program__set_attach_target()
1000 x tracing_multi_attach_progs__load()
1000 x bpf_link_create()
Each bpf_program__set_attach_target() call goes through
libbpf_find_prog_btf_id(), which does bpf_prog_get_info_by_fd() and loads
and parses the target BTF from the kernel. The load calls add 1000 object
BTF loads and 1000 verifier and JIT runs.
The fentry.multi window in create_and_attach_fentry_multi() covers one
open+load and one bpf_link_create(). Its 1000 target BTF IDs were already
resolved in prepare(), outside the timed region (ctx.target_btf_ids[i] =
btf_id).
So the reported "fentry.multi creation/attachment speedup is 7.14x" is not
a link-creation comparison. Part of it comes from BTF ID resolution being
timed for fentry but not for fentry.multi, and most of it is probably the
cost of loading 1000 programs versus one.
Would it make sense to time only the bpf_link_create() loop on its own and
resolve BTF IDs on the same side of the timer for both paths? Otherwise,
could the commit message say that the numbers cover program creation and BTF
ID resolution as well as attachment?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
