The MANA driver can fail to load on systems with high memory
utilization because several allocations in the queue setup paths
require large physically contiguous blocks via kmalloc. Under memory
fragmentation these high-order allocations may fail, preventing the
driver from creating queues when opening the interface or when
reconfiguring channels, ring parameters or MTU at runtime.
Allocation sizes that are problematic:
mana_create_txq -> tx_qp flat array (sizeof(mana_tx_qp) = 35528):
16 queues (default): 35528 * 16 = ~555 KB contiguous
64 queues (max): 35528 * 64 = ~2220 KB contiguous
mana_create_rxq -> rxq struct with flex array
(sizeof(mana_rxq) = 35712, rx_oobs=296 per entry):
depth 1024 (default): 35712 + 296 * 1024 = ~331 KB per queue
depth 8192 (max): 35712 + 296 * 8192 = ~2403 KB per queue
mana_pre_alloc_rxbufs -> rxbufs_pre and das_pre arrays:
16 queues, depth 1024 (default): 16 * 1024 * 8 = 128 KB each
64 queues, depth 8192 (max): 64 * 8192 * 8 = 4096 KB each
This series addresses the issue by:
1. Converting the tx_qp flat array into an array of pointers with
per-queue kvzalloc (~35 KB each), replacing a single contiguous
allocation that can reach ~2.2 MB at 64 queues.
2. Switching rxbufs_pre, das_pre, and rxq allocations to
kvmalloc/kvzalloc so the allocator can fall back to vmalloc
when contiguous memory is unavailable.
Throughput testing confirms no regression. Since kvmalloc falls
back to vmalloc under memory fragmentation, all kvmalloc calls
were temporarily replaced with vmalloc to simulate the fallback
path (iperf3, GBits/sec):
Physically contiguous vmalloc region
Connections TX RX TX RX
--------------------------------------------------------------
1 47.2 46.9 46.8 46.6
16 181 181 181 181
32 181 181 181 181
64 181 181 181 181
---
Changes in v3:
- Rebased to latest net-next (net-next reopened)
Changes in v2:
- Rebased to latest net-next
Aditya Garg (2):
net: mana: Use per-queue allocation for tx_qp to reduce allocation
size
net: mana: Use kvmalloc for large RX queue and buffer allocations
.../net/ethernet/microsoft/mana/mana_bpf.c | 2 +-
drivers/net/ethernet/microsoft/mana/mana_en.c | 61 +++++++++++--------
.../ethernet/microsoft/mana/mana_ethtool.c | 2 +-
include/net/mana/mana.h | 2 +-
4 files changed, 39 insertions(+), 28 deletions(-)
--
2.43.0
