Since commit e66fe1bc6d25 ("bpf: arena: Reintroduce memcg accounting"),
arena pages are charged to the memcg of the process that created the arena.
That exposes a problem in the arena user page fault path: the fault-in
allocation runs under arena->spinlock, so it can only use the non-blocking
allocator, which never reclaims. Once memory.current is at memory.max the
allocation simply fails, even when the memcg is full of page cache that
could be dropped right away. Reaching memory.max is completely normal for a
healthy application - e.g. reading a large file fills memory.current with
page cache - and the process then gets SIGSEGV on a perfectly valid arena
address.
Preallocate the page outside the lock (patch 2), the way do_anonymous_page()
does, so the allocation can sleep and reclaim. It never invokes the OOM
killer: the page is charged to the map's memcg, which need not be the
faulting task's, so an OOM there could kill unrelated tasks in the map's
cgroup. On a genuine failure the fault returns VM_FAULT_SIGBUS. This needs a
sleepable allocator (patch 1), because bpf_map_alloc_pages() can use the
non-blocking allocator, which never reclaims. patch 3&4 adds a selftest that
fills a memcg with reclaimable page cache and faults an arena in under
memory.max: without the fix the child gets SIGSEGV on a valid address, with
it the fault-in reclaims and succeeds.
v4 -> v7:
- Simplify the implementation: do not aim for OOM anymore. An arena is
shared between processes and can be shared across cgroups, so the OOM
killer is the wrong tool here - it would act on the map's memcg, which
need not be the faulting task's. Only try to reclaim now, via
__GFP_RETRY_MAYFAIL. Everything else is kept as before.
- selftest: changed accordingly - fill the memcg with reclaimable page
cache and check that the arena fault-in succeeds by reclaiming it,
instead of relying on an OOM kill.
v3 -> v4:
- rebase bpf-next and fix conflict
- add Reviewed-by tag from Emil Tsalapatis
v2 -> v3:
- selftest: check the memcg OOM via memory.events "oom_kill" instead of
the exit signal; it only aims to pass on the fixed kernel, since the
unfixed SIGSEGV is racy.
v1 -> v2:
- Rebase on the separate deadlock fix (found by the Sashiko AI review),
now applied to bpf-next.
- Honor the map's NUMA node on fault-in.
- Return VM_FAULT_SIGBUS for the non-recoverable faults (lock, range-tree
and page-table failures); a scratch-page hole stays VM_FAULT_SIGSEGV
only under BPF_F_SEGV_ON_FAULT. (Kumar Kartikeya Dwivedi)
- Add read_cgroup_file() to cgroup_helpers instead of open-coding the
/mnt/... path in the test. (Emil Tsalapatis)
- Dump the cgroup memory stats on test failure to ease debugging.
v4:
https://lore.kernel.org/bpf/[email protected]/T/#t
v2:
https://lore.kernel.org/bpf/[email protected]/
v1:
https://lore.kernel.org/bpf/[email protected]/
Jiayuan Chen (4):
bpf: Add a sleepable page allocator for map memory
bpf: arena: allocate the fault-in page outside the lock
selftests/bpf: Add read_cgroup_file() to cgroup_helpers
selftests/bpf: Add a test for arena fault-in under memory.max
include/linux/bpf.h | 1 +
kernel/bpf/arena.c | 86 ++++++--
kernel/bpf/syscall.c | 28 ++-
tools/testing/selftests/bpf/cgroup_helpers.c | 67 ++++++
tools/testing/selftests/bpf/cgroup_helpers.h | 4 +
.../selftests/bpf/prog_tests/arena_memcg.c | 196 ++++++++++++++++++
.../testing/selftests/bpf/progs/arena_memcg.c | 23 ++
7 files changed, 381 insertions(+), 24 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_memcg.c
create mode 100644 tools/testing/selftests/bpf/progs/arena_memcg.c
--
2.43.0
