Vladislav Bolkhovitin wrote:
Try the following variants:1. Affine IRQ 82, scsi_tgt0 to CPU0, fct0-worker to CPU2, IRQs 169 and 177 to CPU4, scsi_tgt1 to CPU1, fct1-worker to CPU3, scsi_tgt2 to CPU5, fct2-worker to CPU72. Affine IRQ 82 to CPU0, fct0-worker to CPU2, IRQs 169 and 177 to CPU4, fct1-worker to CPU3, fct2-worker to CPU7, no affinity for other processes.3. Affine IRQ 82 to CPU0, IRQs 169 and 177 to CPU4, fct1-worker's to all CPUs, except CPU0 and CPU4, no affinity for other processes.
These are tests 1, 2 and 3, respectively
I did two other tests (Tests 4,5), that has the mlx4_core (comp) IRQ (formerly known as IRQ 82) pinned to CPU0, the two ioDrive IRQs (169, 177) pinned to CPU 4, fct0 and scsi_tgt0 on CPUs 2&3, fct1 and scsi_tgt1 on CPUs 4&6 (test 4) OR fct1 and scsi_tgt1 on CPUs 5&6.Or other similar variants you'd like (even CPUs relate to physical CPU0, odd CPUs relate to physical CPU1). For instance, you can try to affine IRQs 169 and 177 to CPU1.
I ran with these anyway to look at differences among the tests. Having this thread enabled always results in better performance.No points to run for srptthread=1, for it just produce a baseline with no affinity at all.
I ran each test 3 times and took the averages. In order to get a quick look at performance per run, I added a column in the summary that sums the IOPs for each test with SRPT thread enabled and then not enabled. Test 4 seems to give the best results. Here's a brief summary of that summary with just SRPT thread=0:Please do each run several times and write down an average result between runs and approximate variation between them in %%. Otherwise we can't make any reliable conclusions.
Baseline: 356226.39 Test 1: 371217.6533 Test 2: 370553.78 Test 3: 373295.2033 Test 4: 399385.2233 Test 5: 393204.5833
SRP-affinity-tests.xls
Description: MS-Excel spreadsheet
_______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
