On Jun 29, 2009, at 5:59 AM, Dave Love wrote:
Can you say something about any tuning you did to get decent results?
To get the lowest latency, turn off rx interrupt coalescence, either
with ethtool or module parameters, depending on the driver. Of
course,
you may not want to turn it off completely, depending on how much load
the extra interrupts cause.
When I test Open-MX, I turn interrupt coalescing off. I run
omx_pingpong to determine the lowest latency (LL). If the NIC's driver
allows one to specify the interrupt value, I set it to LL-1. If the
driver does not allow specifying the actual rate (i.e. it only has
predetermined values), then I leave it off.
The downside is lower throughput for large messages on 10G Ethernet. I
don't think it matters on gigabit.
Brice and Nathalie have a paper which implements an adaptive interrupt
coalescing so that you do not have to manually tune anything:
"Finding a Tradeoff between Host Interrupt Load and MPI Latency over
Ethernet" available at:
http://hal.inria.fr/inria-00397328
I do not know if it has been included in a release yet.
Scott
_______________________________________________
Beowulf mailing list, [email protected] sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
