Alternatively, is there anything in the SCST layer I should tweak. I'm
still running rev 245 of that code (kinda old, but works with OFED 1.3.1
w/o hacks).
What is the mode (pass thru, blockio...)?
What is the scst_threads=<xx> parameters?
My target server (with DAS) contains 8 2.8 GHz CPU cores and can
sustain over 200K IOPs locally, but only around 73K IOPs over SRP.
Is this number from one initiator or multiple?
Looking at /proc/interrupts, I see that the mlx_core (comp) device is
pushing about 135K Int/s on 1 of 2 CPUs. All CPUs are enabled for
that PCI-E slot, but it only ever uses 2 of the CPUs, and only 1 at a
time. None of the other CPUs has an interrupt rate more than about
40-50K/s.
The number of interrupt can be cut down if there are more completions to
be processed by sw. ie. please test with multiple QPs between one
initiator vs. your target and multiple initiators vs. your target
Does anyone know of a trick to spread those interrupts out more
(which I realize might be bad due to context switching), or something
else that will reduce my interrupts on that cpu? The mlx4 is a MSI-X
interrupt. I've changed it to an APIC int, but it seems to give
slightly lower performance.
There userspace daemon, irqbalanced, that dynamically directs IRQs to
different CPUs. You can define which CPUs CAN handle an IRQ but you
cannot control how it is done. You can look at
Documentation/IRQ-affinity.txt for details how to configure it. In some
cases I found better performance-wise to shut the irqbalanced off and
assign the process to one (ore more) CPU and use a different CPU to
serve interrupts.
-vu
_______________________________________________
general mailing list
general@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
