Craig Plaisance wrote:
Hi - I have compiled vasp 4.6.34 using the Intel fortran compiler 11.1
with openmpi 1.3.3 on a cluster of 104 nodes running Rocks 5.2 with two
quad core opterons connected by a Gbit ethernet. Running in parallel on
Latency of gigabit is likely your issue. Lower quality gigabit switches
often have very high port to port latencies.
NICs on motherboards also tend to be less ... well ... good at low
latency operations.
Size of your problem set is important ... small problems don't scale well.
You might try MX, and building OpenMPI atop MX, and see if this helps,
assuming problem size isn't the issue.
I'd also recommend (though this is hard) upgrading your node kernels.
Rocks uses a Redhat kernel, built on a fairly (ancient) 2.6.18 base,
with lots of backports. We normally suggest alternatives. Modern
kernels tend to be faster. Unfortunately, installing new kernels on
Rocks compute nodes can be a challenge.
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: land...@scalableinformatics.com
web : http://scalableinformatics.com
http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
