Hi,
Hi Jes,
The below patch seems to have broken PCI IRQ affinity assignments for mlx5.
I wouldn't call it breaking IRQ affinity assignments. It just makes them automatic.
Prior to this patch I could echo a value to /proc/irq/<x>/smp_affinity and it would get assigned. With this patch applied I get -EIO
Adding Christoph, Ideally the affinity assignments would be better left alone, but I wasn't aware that they are now immutable. Christoph?
The actual affinity assignments seem to have changed too, but I assume this is a result of the generic allocation?
The affinity assignments should be giving you perfect linear assignment of the rx/tx rings completion vectors to CPU cores: first -> 0 second -> 1 third -> 2 ... Before, mlx5 spread affinity starting from the local numa node as it relied on that when constructing RSS indirection table only to the local numa node (which as a side effect hurt applications running on the far node as the traffic was guaranteed to arrive rx rings located in the "wrong" node). Now the RSS indirection table is linear across both numa nodes just like the irq affinity settings. Another thing that was improved was the pre/post vectors which blacklisted any non rx/tx completion vectors from the affinity assignments (like port events completion vectors from example).
With this applied I get: [root@testbox /proc/irq]# cat 50/smp_affinity 000000,00000010 [root@textbox /proc/irq]# cat 100/smp_affinity 100000,00000000 Without I get: [root@testbox /proc/irq]# cat 50/smp_affinity 000000,00000200 [root@testbox /proc/irq]# cat 100/smp_affinity 000100,00000000 I am not wildly familiar with the affinity assignment code. Is there something obvious I am missing here?
The affinity assignment is indeed changed, should be better though. If you do not see a linear (diagonal) affinity assignment then its a bug that needs to be fixed (probably in msi.c code).
