Carsten Kutzner wrote:
himanshu khandelia wrote:
Hi Carsten,
Thank you very much for the prompt reply.
I know very little about network architecture, and therefore
understand your explanation only partly. Based on what you say,
however, would it be fair to conclude that on the quad core
woodcrests, it will not be possible to improve scaleup without
altering the network architecture hardware itself ? Do you think it
will be worthwhile to test an all-to-all optimization at all ?
If my guess is right and bandwidth is the problem here, the patch will
not improve the scaling. Are the benchmarks made with 1 or 2 NICs/node?
If they are for 1 NIC/node then there should be no network congestion
for the case of 8 CPUs (=2 nodes). You could try a back-to-back
connection between two nodes to be absolutely shure that the rest of the
network (switch etc.) does not play a role. I would try that and repeat
the benchmark for 8 CPUs. See if you get a different value.
and don't forget to use grompp -shuffle
Regards,
Carsten
Thank you,
-Himanshu
On 10/25/07, Carsten Kutzner <[EMAIL PROTECTED]> wrote:
Hi Himanshu,
maybe your problem is not even flow control, but the limited network
bandwidth which is shared among 4 CPUs in your case. I also have done
benchmarks on Woodcrests (2.33 GHz) and was not able to scale an 80000
atom system beyond 1 node with Gbit Ethernet. Looking in more detail,
the time gained by the additional 4 CPUs of a second node was exactly
balanced by the extra communication. I used only 1 network interface for
that benchmark, leaving effectively only 1/4 th of the bandwidth for
each CPU. Using two interfaces with OpenMPI did not double the network
performance on our cluster. In my tests nodes with 2 CPUs sharing one
NIC were faster than nodes with 4 CPUs sharing two NICs. Could be
on-node contention, since both interfaces probably end up on the same
bus internally.
Regards,
Carsten
himanshu khandelia wrote:
Hi,
We tried turning on switch control on our local cluster
(www.dcsc.sdu.dk) but were unable to achieve any improvement in scale
up whatsoever. I was wondering if you folks could shed light upon how
we should go ahead with this. (We have not installed the all-to-all
patch yet)
The cluster architecture is as follows:
##########
* Computing nodes
160x Dell PowerEdge 1950 1U rackmountable servers with 2 2,66Ghz Intel
Woodcrest CPUs, 4 GB Ram, 2x160 GB HDD (7200rpm, 8 MB buffer,
SATA150), 2x Gigabit Ethernet
40x Dell PowerEdge 1950 1U rackmountable servers with 2 2,66Ghz Intel
Woodcrest CPUs, 8 GB Ram, 2x160 GB HDD (7200rpm, 8 MB buffer,
SATA150), 2x Gigabit Ethernet
##########
* Switches
9 D-link SR3324
2 D-link SRi3324
The switches are organised in two stacks, each connected to the
infrastracture switch with an 8 Gb/s LACP trunk.Firmware Build on the
switches
##########
* Firmware Build on the switches: 3.00-B16
There are newer firmware builds available, but according to the update
logs, there is not update on the IEEE flow control protocol in the new
firmware
##########
* Tests (were run using OPENMPI, not LAMMPI)
DPPC-bilayer system of ~ 40000 atoms, with PME and cutoffs, 1fs time
step. The scaleup data is as follows. We are also currently running
some tests with larger systems.
# Procs nanoseconds/day Scaleup
1 0.526 1
2 1.0 1.90
4 1.768 3.36
8 1.089 2.07
16 0.39 0.74
Any inputs will be very helpful, thank you
Best,
-himanshu
_______________________________________________
gmx-users mailing list [email protected]
http://www.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at http://www.gromacs.org/search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to [EMAIL PROTECTED]
Can't post? Read http://www.gromacs.org/mailing_lists/users.php
--
Dr. Carsten Kutzner
Max Planck Institute for Biophysical Chemistry
Theoretical and Computational Biophysics Department
Am Fassberg 11
37077 Goettingen, Germany
Tel. +49-551-2012313, Fax: +49-551-2012302
http://www.mpibpc.mpg.de/research/dep/grubmueller/
http://www.gwdg.de/~ckutzne
_______________________________________________
gmx-users mailing list [email protected]
http://www.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at http://www.gromacs.org/search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to [EMAIL PROTECTED]
Can't post? Read http://www.gromacs.org/mailing_lists/users.php
_______________________________________________
gmx-users mailing list [email protected]
http://www.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at http://www.gromacs.org/search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to [EMAIL PROTECTED]
Can't post? Read http://www.gromacs.org/mailing_lists/users.php
--
David.
________________________________________________________________________
David van der Spoel, PhD, Assoc. Prof., Molecular Biophysics group,
Dept. of Cell and Molecular Biology, Uppsala University.
Husargatan 3, Box 596, 75124 Uppsala, Sweden
phone: 46 18 471 4205 fax: 46 18 511 755
[EMAIL PROTECTED] [EMAIL PROTECTED] http://folding.bmc.uu.se
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
_______________________________________________
gmx-users mailing list [email protected]
http://www.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at http://www.gromacs.org/search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to [EMAIL PROTECTED]
Can't post? Read http://www.gromacs.org/mailing_lists/users.php
