On Wed, 16 Apr 2025 09:13:23 +0200 Jaroslav Pulchart wrote: > By "traced" I mean using the kernel and checking memory situation on > numa nodes with and without production load. Numa nodes, with X810 > NIC, showing a quite less available memory with default queue length > (num of all cpus) and it needs to be lowered to 1-2 (for unused > interfaces) and up-to-count of numa node cores on used interfaces to > make the memory allocation reasonable and server avoiding "kswapd"... > > See "MemFree" on numa 0 + 1 on different/smaller but utilized (running > VMs + using network) host server with 8 numa nodes (32GB RAM each, 28G > in Hugepase for VMs and 4GB for host os):
FWIW you can also try the tools/net/ynl/samples/page-pool application, not sure if Intel NICs init page pools appropriately but this will show you exactly how much memory is sitting on Rx rings of the driver (and in net socket buffers). > 6.13.y vanilla (lot of kswapd0 in background): > NUMA nodes: 0 1 2 3 4 5 6 7 > HPTotalGiB: 28 28 28 28 28 28 28 28 > HPFreeGiB: 0 0 0 0 0 0 0 0 > MemTotal: 32220 32701 32701 32686 32701 32701 > 32701 32696 > MemFree: 274 254 1327 1928 1949 2683 2624 > 2769 > 6.13.y + Revert (no memory issues at all): > NUMA nodes: 0 1 2 3 4 5 6 7 > HPTotalGiB: 28 28 28 28 28 28 28 28 > HPFreeGiB: 0 0 0 0 0 0 0 0 > MemTotal: 32220 32701 32701 32686 32701 32701 32701 32696 > MemFree: 2213 2438 3402 3108 2846 2672 2592 3063 > > We need to lower the queue on all X810 interfaces from default (64 in > this case), to ensure we have memory available for host OS services. > ethtool -L em2 combined 1 > ethtool -L p3p2 combined 1 > ethtool -L em1 combined 6 > ethtool -L p3p1 combined 6 > This trick "does not work" without the revert. And you're reverting just and exactly 492a044508ad13 ? The memory for persistent config is allocated in alloc_netdev_mqs() unconditionally. I'm lost as to how this commit could make any difference :(
