From: Naman Jain <[email protected]> Sent: Sunday, August 9, 2026 11:22 PM
[snip] > @@ -510,6 +633,8 @@ struct cpumask *group_cpus_evenly(unsigned int numgrps, > unsigned int *nummasks) > if (!masks) > goto fail_node_to_cpumask; > > + spread_offset = (unsigned int)atomic_fetch_inc(&group_spread_cnt); > + > build_node_to_cpumask(node_to_cpumask); > One additional observation: In my testing, group_cpus_evenly() is often called with numgrps set to 1. This happens in the block "loop" devices (drivers/block/loop.c) and for the NVMe admin queue. In these cases, the spread_offset is never used, but group_spread_cnt gets incremented anyway. Incrementing for NVMe admin queues tends to dirty the spreading for multiple NVMe devices with the same configuration because it is usually interleaved with the spreading of the main NVMe I/O queues. To improve this, I changed the above code to this: + if (numgrps == 1) + spread_offset = 0; + else + spread_offset = (unsigned int)atomic_fetch_inc(&group_spread_cnt); With this change, my configuration #1 (Azure L48s v2 VM) is noticeably better. All CPUs in NUMA node 1 have either 3 or 4 IRQs assigned. NUMA node 0 ranges from 3 to 5 IRQs, but that's partly because the NUMA nodes themselves aren't balanced, as previously discussed. With your change to apply group_spread_cnt to the NUMA nodes, and my change above, my config #1 is likely to work out very near optimal. Of course, there's no guarantee that some other device won't increment group_spread_cnt and dirty things, but for the typical case it probably works very well. This change to skip incrementing group_spread_cnt when numgrps == 1 doesn't help my arm64 configs. I'm still thinking about ways to do better when there aren't any clusters. I have an idea that I'm experimenting with, but it may be a few more days before I reach any conclusions. Michael
