----- Original Message ----- From: Berk Hess <g...@hotmail.com> Date: Wednesday, February 3, 2010 5:13 Subject: RE: [gmx-users] Replica Exchange MD on more than 64 processors To: Discussion list for GROMACS users <gmx-users@gromacs.org>
----------------------------------------------------------- | > Hi, > > One issue could be MPI memory usage. > I have noticed that many MPI implementations use an amount of memory > per process that is quadratic (!) in the number of processes involved. > This can quickly get out of hand. But 28 GB is a lot of memory. The OP was using MVAPICH 1.1, which is not the most current version. MVAPICH 1.2 claims to scale with near-constant memory usage. I suggest an upgrade. Mark > One thing that might help slightly is to not use double precision, > which is almost never required. This will also make your simulations > a factor 1.4 faster. > > Berk > > > Date: Tue, 2 Feb 2010 18:55:37 +0100 > > From: breue...@uni-koeln.de > > To: gmx-users@gromacs.org > > Subject: [gmx-users] Replica Exchange MD on more than 64 processors > > > > Dear list > > > > I recently came up with a problem concerning a replica exchange simulation. > > The simulation is run > > with gromacs-mpi in Version 4.0.7 compiled with following flags > > --enable-threads --enable-mpi --with-fft=mkl -enable-double, > > > > intel compiler version 11.0 > > mvapich version 1.1.0 > > mkl version 10.1 > > > > The program is working fine in this cluster evironment consisting of 32 > > nodes with 8 processors and > > 32GB each. I've already run several simulations using the MPI feature. > > > > It seems that I stuck in a similar problem that was already announced on > > this list by bharat v. > > adkar in december 2009 without an eventual solution: > > > > http://www.mail-archive.com/gmx-users@gromacs.org/msg27175.html > > > > I am doing a replica exchange simulation on a simulation box with 5000 > > molecules (81 atoms each) and > > 4 different temperatures. The simulation runs nicely with 64 processors (8 > > nodes) but stops with an > > error message on 128 processors (16 nodes). > > > > Taking the following four points into account: > > > > 1. every cluster node has at least 28GB memory in a usable way > > available > > 2. the system I am working with should only use > > 5000*81*900B=347.614MB (according to the FAQ) > > 3. even if every replica (4) is run on the same node the memory > > usage > > should be less than 2GB > > 4. the simulation works fine with 64 processors > > > > it seems to me the following error > > > > ------------------------------------------------------- > > Program mdrun, VERSION 4.0.7 > > Source code file: smalloc.c, line: 179 > > > > Fatal error: > > Not enough memory. Failed to realloc 790760 bytes for nlist->jjnr, > > nlist->jjnr=0xae70b7b0 > > (called from file ns.c, line 503) > > ------------------------------------------------------- > > > > has to be caused by another issue than missing memory. > > > > I am wondering if there is anyone else who is still facing the same problem > > or has already found a > > solution for this issue. > > > > Kind regards > > > > Sebastian > > > > -- > > _____________________________________________________________________________ > > > > Sebastian Breuers Tel: +49-221-470-4108 > > EMail: breue...@uni-koeln.de > > > > Universität zu Köln University of Cologne > > Department für Chemie Department Chemistry > > Organische Chemie university of Cologne > > > > Greinstr. 4 Greinstr. 4 > > D-50939 Köln D-50939 Cologne, Federal Rep. of > > Germany > > _____________________________________________________________________________ > > > > -- > > gmx-users mailing list gmx-users@gromacs.org > > http://lists.gromacs.org/mailman/listinfo/gmx-users > > Please search the archive at http://www.gromacs.org/search before posting! > > Please don't post (un)subscribe requests to the list. Use the > > www interface or send it to gmx-users-requ...@gromacs.org. > > Can't post? Read http://www.gromacs.org/mailing_lists/users.php > ----------------------------------------------------------- > New Windows 7: Simplify what you do everyday. Find the right PC for you. | ----------------------------------------------------------- > -- > gmx-users mailing list gmx-users@gromacs.org > http://lists.gromacs.org/mailman/listinfo/gmx-users > Please search the archive at http://www.gromacs.org/search > before posting! > Please don't post (un)subscribe requests to the list. Use the > www interface or send it to gmx-users-requ...@gromacs.org. > Can't post? Read http://www.gromacs.org/mailing_lists/users.php -- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/mailing_lists/users.php
