Not (all) directly related, but a few comments/questions: - Have you tried 4.6.7 or 5.0.4? - Have you considered using the Verlet scheme instead of doing manual buffering? - lincs-order=8 is very large for 2fs production runs - typically 4 is used. - Your fourier spacing is a lot (~25%) finer than it needs to be.
- The PP/PME rank split of 100+28 is _very_ inconvenient and it is the main cause of the horrible PME performance together with the overly coarse grid. That's why you get such a huge cut-off after the PP-PME load balancing. Even if you want to stick to these parameters, you should tune the rank split (manually or with tune_pme). - The above contributes to the high neighbor search cost too. -- Szilárd On Tue, Jan 20, 2015 at 9:18 PM, Jiaqi Lin <jq...@mit.edu> wrote: > Hi Mark, > > Thanks for reply. I put the md.log files in the following link > > https://www.dropbox.com/sh/d1d2fbwreizr974/AABYhSRU03nmijbTIXKKr-rra?dl=0 > > There are four log files > 1.GMX 4.6.5 -tunepme (the coulombic cutoff is tuned to 3.253) > 2.GMX 4.6.5 -notunepme rcoulomb= 3.3 , fourierspace = 0.33 > 3.GMX 4.6.5 -notunepme rcoulomb= 3.3 , fourierspace = 0.14 > 4.GMX 4.6.5 -notunepme rcoulomb= 1.4 , fourierspace = 0.14 > > Note that the LR Coulombic energy in the first one is almost twice the value > of that in the second one, whereas the grid spacing in both cases are nealy > the same. > > Only the first one gives a strong electrostatic interaction of a > nanoparticle with a lipid bilayer under ionic imbalance. In other cases I do > not observe such a strong interaction. > > GMX 5.0.1 give the same results as GMX 4.6.5 using Group cutoff. Thanks > > > Regards > Jiaqi > > > > > On 1/19/2015 3:22 PM, Mark Abraham wrote: >> >> On Thu, Jan 15, 2015 at 3:21 AM, Jiaqi Lin <jq...@mit.edu> wrote: >> >>> Dear GMX developers, >>> >>> I've encounter a problem in GROMACS concerning the auto-tuning feature of >>> PME that bugged me for months. As stated in the title, the auto-tuning >>> feature of mdrun changed my coulomb cutoff from 1.4 nm to ~3.3 nm (stated >>> in md.log) when I set -npme to be 28 (128 total CPU cores), and this >>> giving >>> me interesting simulation results. When I use -notunepme, I found Coulomb >>> (SR) and recip. giving me same energy but the actual simulation result is >>> different. This i can understand: scaling between coulombic cut-off/grid >>> size theoretically give same accuracy to electrostatics (according to GMX >>> manual and PME papers), but there actually some numerical error due to >>> grid >>> mapping and even if the energy is the same that does not mean system >>> configuration has to be the same (NVE ensemble: constant energy, >>> different >>> configuration). >>> >> Total electrostatic energy should be approximately the same with different >> PME partitions. >> >> >>> However the thing i don't understand is the following. I am interested in >>> the result under large coulomb cut-off, so I try to manually set cut-off >>> and grid space with -notunepme, using the value tuned by mdrun >>> previously. >>> This give me complete different simulation result, and the energy is also >>> different. I've tried to set rlist, rlistlong, or both to equal rcoulomb >>> (~3.3) still does not give me the result produced by auto-tuning PME. >> >> >> In what sense is the result different? >> >> >>> In addition, simulation speed dramatically reduces when I set rcoulomb to >>> be ~3.3 (using -tunepme the speed remains nearly the same no matter how >>> large the cutoff is tuned to). I've tested this in both GMX 4.6.5 and >>> 5.0.1, same thing happens, so clearly it's not because of versions. Thus >>> the question is: what exactly happened to PME calcualtion using the >>> auto-tuning feature in mdrun, why it does give different results when I >>> manually set the coulomb cutoff and grid space to the value tuned by >>> mdrun >>> without the auto-tuning feature (using -notunepme)? Thank you for help. >>> >> For the group scheme, these should all lead to essentially the same result >> and (if tuned) performance. If you can share your various log files on a >> file-sharing service (rc 1.4, rc 3.3, various -tunepme settings, 4.6.5 and >> 5.0.1) then we can be in a position to comment further. >> >> Mark >> >> >>> additional info: I use Group cutoff-scheme , rvdw is 1.2. >>> >>> >>> md.log file: >>> DD step 9 load imb.: force 29.4% pme mesh/force 3.627 >>> >>> step 30: timed with pme grid 280 280 384, coulomb cutoff 1.400: 1026.4 >>> M-cycles >>> step 50: timed with pme grid 256 256 324, coulomb cutoff 1.464: 850.3 >>> M-cycles >>> step 70: timed with pme grid 224 224 300, coulomb cutoff 1.626: 603.6 >>> M-cycles >>> step 90: timed with pme grid 200 200 280, coulomb cutoff 1.822: 555.2 >>> M-cycles >>> step 110: timed with pme grid 160 160 208, coulomb cutoff 2.280: 397.0 >>> M-cycles >>> step 130: timed with pme grid 144 144 192, coulomb cutoff 2.530: 376.0 >>> M-cycles >>> step 150: timed with pme grid 128 128 160, coulomb cutoff 2.964: 343.7 >>> M-cycles >>> step 170: timed with pme grid 112 112 144, coulomb cutoff 3.294: 334.8 >>> M-cycles >>> Grid: 12 x 14 x 14 cells >>> step 190: timed with pme grid 84 84 108, coulomb cutoff 4.392: 346.2 >>> M-cycles >>> step 190: the PME grid restriction limits the PME load balancing to a >>> coulomb cut-off of 4.392 >>> step 210: timed with pme grid 128 128 192, coulomb cutoff 2.846: 360.6 >>> M-cycles >>> step 230: timed with pme grid 128 128 160, coulomb cutoff 2.964: 343.6 >>> M-cycles >>> step 250: timed with pme grid 120 120 160, coulomb cutoff 3.036: 340.4 >>> M-cycles >>> step 270: timed with pme grid 112 112 160, coulomb cutoff 3.253: 334.3 >>> M-cycles >>> step 290: timed with pme grid 112 112 144, coulomb cutoff 3.294: 334.7 >>> M-cycles >>> step 310: timed with pme grid 84 84 108, coulomb cutoff 4.392: 348.0 >>> M-cycles >>> optimal pme grid 112 112 160, coulomb cutoff 3.253 >>> DD step 999 load imb.: force 18.4% pme mesh/force 0.918 >>> >>> At step 1000 the performance loss due to force load imbalance is 6.3 % >>> >>> NOTE: Turning on dynamic load balancing >>> >>> Step Time Lambda >>> 1000 20.00000 0.00000 >>> >>> Energies (kJ/mol) >>> Bond G96Angle LJ (SR) Coulomb (SR) Coul. >>> recip. >>> 1.98359e+05 1.79181e+06 -1.08927e+07 -7.04736e+06 >>> -2.32682e+05 >>> Position Rest. Potential Kinetic En. Total Energy Temperature >>> 6.20627e+04 -1.61205e+07 4.34624e+06 -1.17743e+07 3.00659e+02 >>> Pressure (bar) Constr. rmsd >>> 2.13582e+00 1.74243e-04 >>> >>> >>> Best >>> Jiaqi >>> >>> >>> >>> -- >>> Jiaqi Lin >>> postdoc fellow >>> The Langer Lab >>> >>> -- >>> Gromacs Users mailing list >>> >>> * Please search the archive at http://www.gromacs.org/ >>> Support/Mailing_Lists/GMX-Users_List before posting! >>> >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >>> >>> * For (un)subscribe requests visit >>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or >>> send a mail to gmx-users-requ...@gromacs.org. >>> > > -- > Jiaqi Lin > postdoc fellow > The Langer Lab > > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a > mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.