Ok, this will be a long one.
Currently I am simulating large systems (20-35 nm sides; rhombic
dodecahedron or rectangular boxes) in the latest CVS version of
Gromacs on 70-500 nodes using PME to calculate long range
electrostatic interactions. Now I am looking for some advice
regarding how to set up my system for good performance.
There is a correlation between the short range cutoff (rlist/rcoulomb/
rvdw), PME grid spacing (fourierspacing, fourier_nx/fourier_ny/
fourier_nz) and the PME interpolation order (pme_order). Grompp tells
me that for best performance there should be about 25-33% load on the
dedicated PME nodes compared to the regular particle-particle nodes.
I guess this value is due to the communication overhead of the PME
nodes.
Two of my setups are:
SYSTEM 1
Dimentions: 24.12032 24.12032 17.05213 0.00000 0.00000
0.00000 0.00000 12.06016 12.06016
Fourier grid spacing: 0.137 x 0.137 x 0.137
Cutoffs: 1.1
Interpolation order: 4
Estimated PME load: 0.28
SYSTEM 2
Dimentions: 23.53603 24.05188 35.23882
Fourier grid spacing: 0.134 0.137 0.133
Cutoffs: 1.1
Interpolation order: 4
Estimated PME load: 0.51
So I am interested to hear from you how far I can push the fourier
grid spacing before I loose accuracy in terms of force and energy.
How much must I compensate with increased interpolation and increased
cutoff? Cutoff will put more load on the PP nodes which is what I
want in this case, but would an increased interpolation order also be
required/advantageous? I use the fft optimization option.
In the original paper on PME (Darden et al. 1993) there is a
comparison of PME order 1-4 and grid size of 0.1-0.05 nm which shows
that a grid spacing of 0.075 nm and an interpolation order of 3 gives
an rms force error of 2x10^-4, which they say is reasonable. Does
anyone of you know of a more recent equivalent test on larger systems?
A somewhat unrelated question is about the heuristics of mdrun when
it decides how many PME nodes versus PP nodes it should use and when
it does the PP domain decomposition. It would be quite useful with a
tool that could suggest a desirable number of nodes within a given
interval. If I can understand in which order the decisions are made
and which constrains are imposed I will gladly write such a tool
myself. (More than once has my simulations failed after a long time
in the queue due to an incompatible number of nodes. A feature of
mdrun would be to choose not to use x number of nodes if it would
improve performance rather than just die.)
---
Daniel Larsson
Molecular Biophysics group
Department of Cell and Molecular Biology
Uppsala University
+46-18-471 4006 (phone)
+46-18-511 755 (fax)
http://xray.bmc.uu.se/~larsson
[EMAIL PROTECTED]
_______________________________________________
gmx-users mailing list [email protected]
http://www.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at http://www.gromacs.org/search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to [EMAIL PROTECTED]
Can't post? Read http://www.gromacs.org/mailing_lists/users.php
