Cameron Harr, on 01/28/2009 10:58 PM wrote:
I've attached a spreadsheet with some of my findings. In the Summary
tab, I have a baseline with no affinity set. For other 5 tests, see below.
Vladislav Bolkhovitin wrote:
Try the following variants:
1. Affine IRQ 82, scsi_tgt0 to CPU0, fct0-worker to CPU2, IRQs 169 and
177 to CPU4, scsi_tgt1 to CPU1, fct1-worker to CPU3, scsi_tgt2 to
CPU5, fct2-worker to CPU7
2. Affine IRQ 82 to CPU0, fct0-worker to CPU2, IRQs 169 and 177 to
CPU4, fct1-worker to CPU3, fct2-worker to CPU7, no affinity for other
processes.
3. Affine IRQ 82 to CPU0, IRQs 169 and 177 to CPU4, fct1-worker's to
all CPUs, except CPU0 and CPU4, no affinity for other processes.
These are tests 1, 2 and 3, respectively
Or other similar variants you'd like (even CPUs relate to physical
CPU0, odd CPUs relate to physical CPU1). For instance, you can try to
affine IRQs 169 and 177 to CPU1.
I did two other tests (Tests 4,5), that has the mlx4_core (comp) IRQ
(formerly known as IRQ 82) pinned to CPU0, the two ioDrive IRQs (169,
177) pinned to CPU 4, fct0 and scsi_tgt0 on CPUs 2&3, fct1 and scsi_tgt1
on CPUs 4&6 (test 4) OR fct1 and scsi_tgt1 on CPUs 5&6.
No points to run for srptthread=1, for it just produce a baseline with
no affinity at all.
I ran with these anyway to look at differences among the tests. Having
this thread enabled always results in better performance.
Please do each run several times and write down an average result
between runs and approximate variation between them in %%. Otherwise
we can't make any reliable conclusions.
I ran each test 3 times and took the averages. In order to get a quick
look at performance per run, I added a column in the summary that sums
the IOPs for each test with SRPT thread enabled and then not enabled.
Test 4 seems to give the best results. Here's a brief summary of that
summary with just SRPT thread=0:
Baseline: 356226.39
Test 1: 371217.6533
Test 2: 370553.78
Test 3: 373295.2033
Test 4: 399385.2233
Test 5: 393204.5833
Linux CPU scheduler does really impressive job!
Interesting, will something change with:
1. The latest SVN. It has some changes, which might make a difference.
2. Pass-through dev handler instead of BLOCKIO, which you are using.
Thanks,
Vlad
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
