From: Robert Sanford <[email protected]> A thread walking the mempool list with rte_mempool_walk(), rte_mempool_lookup() or rte_mempool_list_dump() can crash while other threads create and free mempools. We hit this in production, in a process that walks the mempools while other threads create and free them.
There are two races: - rte_mempool_free() unlinks and frees the list entry without the mempool list lock that the walkers hold, so a walker can reach a freed entry or mempool. Patch 1 takes the lock in rte_mempool_free(), as rte_mempool_create_empty() already does. A rte_mempool_walk() callback can therefore no longer free a mempool; this is documented. - A mempool is on the list before its store exists, but rte_mempool_dump() queries the mempool driver, which reads the store. So rte_mempool_list_dump() and the telemetry /mempool/info handler crash on such a mempool. Patch 2 makes them report the mempool without querying the driver until RTE_MEMPOOL_F_POOL_CREATED is set, and documents that rte_mempool_walk() callbacks must check that flag before querying the driver. Patch 3 adds a test to mempool_autotest. On x86 with 4 or 8 lcores, it crashed in all 20 runs without patch 1, without patch 2, or without both, and passed in all 20 runs with both patches. v2: - Patch 2 reworked as suggested by Morten Brørup: rte_mempool_walk() no longer skips mempools that are not populated yet. Instead, rte_mempool_dump() and the telemetry callback report them without querying the mempool driver, and the rte_mempool_walk() documentation asks callbacks to check RTE_MEMPOOL_F_POOL_CREATED. The flag is now set and tested with release/acquire atomics instead of plain accesses with fences. New title. - Patch 3: the walk callback dumps every mempool, and checks the flag before querying the driver. The test waits 100 us instead of 10 us before populating, so that it still finds unpopulated mempools reliably. - Patch 1: unchanged, Reviewed-by added. Robert Sanford (3): mempool: fix race between list walk and free mempool: fix dump of unpopulated pool test/mempool: add list walk race test app/test/test_mempool.c | 146 ++++++++++++++++++++++++++++++++++++++ lib/mempool/rte_mempool.c | 44 ++++++++++-- lib/mempool/rte_mempool.h | 12 ++++ 3 files changed, 198 insertions(+), 4 deletions(-) -- 2.34.1
