Thank you very much to all of you. That should explain the difference in performance.
I'll also discuss it with a more gromacs-knowledgeable colleague of mine. Best Regards. 2014-09-06 8:58 GMT+02:00 Abhi Acharya <abhi117acha...@gmail.com>: > Thank you Mark and Szilard for your replies. It gave more clarity on how > the new gromacs works, > especially in greater support for streamed computing. > > I hope David's problem is sorted too. :) > > Thanks again, > > Regards, > Abhishek Acharya > > > On Fri, Sep 5, 2014 at 10:45 PM, Szilárd Páll <pall.szil...@gmail.com> > wrote: > > > On Fri, Sep 5, 2014 at 6:40 PM, Abhishek Acharya > > <abhi117acha...@gmail.com> wrote: > > > Dear Mark, > > > > > > Thank you for the insightful reply. > > > In the manual for gromacs 5.0 it was mentioned that verlet scheme is > > better for GPU systems. > > > > More correctly, only the Verlet scheme supports GPU acceleration. The > > algorithms used by the group scheme are not appropriate for GPUs or > > other wide-SIMD accelerators. > > > > > Does that mean that we should give up on the group scheme scheme, even > > though we get good performance compared to verlet? > > > > That's up to you to decide. The algorithms are different, the group > > scheme does not use a buffer by default, while the verlet scheme does > > and aims to control the drift (and keep it quite low by default). > > > > > Future plan of removing group cut-off scheme indicates that it must > have > > been associated with a high cost-benefit ratio. > > > > What makes you conclude that? The reasons are described here: > > http://www.gromacs.org/Documentation/Cut-off_schemes > > > > In very brief summary: i) the groups scheme is not suitable for > > accelerators and wide SIMD architectures ii) energy conservation = > > high performance penalty iii) inconvenient for high parallalelization > > as it increases load imbalance > > > > Cheers, > > -- > > Szilárd > > > > > Could you please shed little light on this ? > > > Thanks. > > > > > > Regards, > > > Abhishek > > > > > > -----Original Message----- > > > From: "Mark Abraham" <mark.j.abra...@gmail.com> > > > Sent: 9/5/2014 7:57 PM > > > To: "Discussion list for GROMACS users" <gmx-us...@gromacs.org> > > > Subject: Re: [gmx-users] Gromacs 5.0 compilation slower than 4.6.5. > What > > wentwrong ? > > > > > > This cutoff-scheme difference is probably caused by using an .mdp file > > that > > > does not specify the cutoff scheme, and the default changed in 5.0. > > grompp > > > issued a note about this, if you go and check it. The change in the > -npme > > > choice is a direct consequence of this; the heuristics underlying the > > > splitting choice approximately understand the relative performance > > > characteristics of the two implementations, and you can see that in > > > practice the reported PP/PME balance is decent in each case. > > > > > > There is indeed a large chunk of water (which you can see in > group-scheme > > > log files e.g. the line in the FLOP accounting that says NB VdW & Elec. > > > [W3-W3,F] dominates the cost), and David's neighbour list is > unbuffered. > > > This is indeed the regime where the group scheme might still > out-perform > > > the Verlet scheme (depending whether you value buffering in the > neighbour > > > list, which you generally should!). > > > > > > Mark > > > > > > > > > On Fri, Sep 5, 2014 at 4:06 PM, Abhi Acharya <abhi117acha...@gmail.com > > > > > wrote: > > > > > >> Hello, > > >> Is you system solvated with water molecules? > > >> > > >> The reason I ask is that, in case of the run with 4.6.5 you gromacs > has > > >> used a group cut-off scheme, whereas 5.0 has used verlet scheme. For > > system > > >> with water molecules, group scheme gives better performance than > verlet. > > >> > > >> For more check out: > > >> http://www.gromacs.org/Documentation/Cut-off_schemes > > >> > > >> Regards, > > >> Abhishek Acharya > > >> > > >> On Fri, Sep 5, 2014 at 7:28 PM, Carsten Kutzner <ckut...@gwdg.de> > > wrote: > > >> > > >> > Hi, > > >> > > > >> > you might want to use g_tune_pme to find out the optimal number > > >> > of PME nodes for 4.6 and 5.0. > > >> > > > >> > Carsten > > >> > > > >> > > > >> > > > >> > On 05 Sep 2014, at 15:39, David McGiven <davidmcgiv...@gmail.com> > > wrote: > > >> > > > >> > > What is even more strange is that I tried with 10 pme nodes (mdrun > > >> -ntmpi > > >> > > 48 -v -c TEST_md.gro -npme 16), got a 15,8% performance loss and > > ns/day > > >> > are > > >> > > very similar : 33 ns/day > > >> > > > > >> > > D. > > >> > > > > >> > > 2014-09-05 14:54 GMT+02:00 David McGiven <davidmcgiv...@gmail.com > >: > > >> > > > > >> > >> Hi Abhi, > > >> > >> > > >> > >> Yes I noticed that imbalance but I thought gromacs knew better > than > > >> the > > >> > >> user how to split PP/PME!! > > >> > >> > > >> > >> How is it possible that 4.6.5 guesses better than 5.0 ? > > >> > >> > > >> > >> Anyway, I tried : > > >> > >> mdrun -nt 48 -v -c test.out > > >> > >> > > >> > >> Exits with an error "You need to explicitly specify the number of > > MPI > > >> > >> threads (-ntmpi) when using separate PME ranks" > > >> > >> > > >> > >> Then: > > >> > >> mdrun -ntmpi 48 -v -c TEST_md.gro -npme 12 > > >> > >> > > >> > >> Then again 35 ns/day with the warning : > > >> > >> NOTE: 8.5 % performance was lost because the PME ranks > > >> > >> had less work to do than the PP ranks. > > >> > >> You might want to decrease the number of PME ranks > > >> > >> or decrease the cut-off and the grid spacing. > > >> > >> > > >> > >> > > >> > >> I don't know much about Gromacs so I am puzzled. > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> 2014-09-05 14:32 GMT+02:00 Abhi Acharya < > abhi117acha...@gmail.com > > >: > > >> > >> > > >> > >>> Hello, > > >> > >>> > > >> > >>> From the log files it is clear that out of 48 cores, the 5.0 run > > had > > >> 8 > > >> > >>> cores allocated to PME while the 4.6.5 run had 12 cores. This > > seems > > >> to > > >> > >>> have > > >> > >>> caused a greater load imbalance in case of the 5.0 run. > > >> > >>> > > >> > >>> If you notice the last table in both .mdp files, you will notice > > that > > >> > the > > >> > >>> PME spread/gather wall time values for 5.0 is more than double > the > > >> wall > > >> > >>> time value in case of the 4.6.5. > > >> > >>> > > >> > >>> Try running the simulation by explicitly setting the -npme flag > as > > >> 12. > > >> > >>> > > >> > >>> Regards, > > >> > >>> Abhishek Acharya > > >> > >>> > > >> > >>> > > >> > >>> On Fri, Sep 5, 2014 at 4:43 PM, David McGiven < > > >> davidmcgiv...@gmail.com > > >> > > > > >> > >>> wrote: > > >> > >>> > > >> > >>>> Thanks Szilard, here it goes! : > > >> > >>>> > > >> > >>>> 4.6.5 : http://pastebin.com/nqBn3FKs > > >> > >>>> 5.0 : http://pastebin.com/kR4ntHtK > > >> > >>>> > > >> > >>>> 2014-09-05 12:47 GMT+02:00 Szilárd Páll < > pall.szil...@gmail.com > > >: > > >> > >>>> > > >> > >>>>> mdrun writes a log file, named md.log by default, which > contains > > >> > among > > >> > >>>>> other things results of hardware detection and performance > > >> > >>>>> measurements. The list does not accept attachments, please > > upload > > >> it > > >> > >>>>> somewhere (dropbox, pastebin, etc.) and post a link. > > >> > >>>>> > > >> > >>>>> Cheers, > > >> > >>>>> -- > > >> > >>>>> Szilárd > > >> > >>>>> > > >> > >>>>> > > >> > >>>>> On Fri, Sep 5, 2014 at 12:37 PM, David McGiven < > > >> > >>> davidmcgiv...@gmail.com> > > >> > >>>>> wrote: > > >> > >>>>>> Command line in both cases is : > > >> > >>>>>> 1st : grompp -f grompp.mdp -c conf.gro -n index.ndx > > >> > >>>>>> 2nd : mdrun -nt 48 -v -c test.out > > >> > >>>>>> > > >> > >>>>>> Log file you mean the standard output/error ? Attached to the > > >> email > > >> > >>> ? > > >> > >>>>>> > > >> > >>>>>> Thanks > > >> > >>>>>> > > >> > >>>>>> 2014-09-05 12:30 GMT+02:00 Szilárd Páll < > > pall.szil...@gmail.com>: > > >> > >>>>>> > > >> > >>>>>>> Please post the command lines you used to invoke mdrun as > > well as > > >> > >>> the > > >> > >>>>>>> log files of the runs you are comparing. > > >> > >>>>>>> > > >> > >>>>>>> Cheers, > > >> > >>>>>>> -- > > >> > >>>>>>> Szilárd > > >> > >>>>>>> > > >> > >>>>>>> > > >> > >>>>>>> On Fri, Sep 5, 2014 at 12:10 PM, David McGiven < > > >> > >>>> davidmcgiv...@gmail.com > > >> > >>>>>> > > >> > >>>>>>> wrote: > > >> > >>>>>>>> Dear Gromacs users, > > >> > >>>>>>>> > > >> > >>>>>>>> I just compiled gromacs 5.0 with the same compiler (gcc > > 4.7.2), > > >> > >>> same > > >> > >>>>> OS > > >> > >>>>>>>> (RHEL 6) same configuration options and basically > everything > > >> > >>> than my > > >> > >>>>>>>> previous gromacs 4.6.5 compilation and when doing one of > our > > >> > >>> typical > > >> > >>>>>>>> simulations, I get worst performance. > > >> > >>>>>>>> > > >> > >>>>>>>> 4.6.5 does 45 ns/day > > >> > >>>>>>>> 5.0 does 35 ns/day > > >> > >>>>>>>> > > >> > >>>>>>>> Do you have any idea of what could be happening ? > > >> > >>>>>>>> > > >> > >>>>>>>> Thanks. > > >> > >>>>>>>> > > >> > >>>>>>>> Best Regards, > > >> > >>>>>>>> D. > > >> > >>>>>>>> -- > > >> > >>>>>>>> Gromacs Users mailing list > > >> > >>>>>>>> > > >> > >>>>>>>> * Please search the archive at > > >> > >>>>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List > > >> before > > >> > >>>>>>> posting! > > >> > >>>>>>>> > > >> > >>>>>>>> * Can't post? Read > > http://www.gromacs.org/Support/Mailing_Lists > > >> > >>>>>>>> > > >> > >>>>>>>> * For (un)subscribe requests visit > > >> > >>>>>>>> > > >> > >>> > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > > >> > >>>> or > > >> > >>>>>>> send a mail to gmx-users-requ...@gromacs.org. > > >> > >>>>>>> -- > > >> > >>>>>>> Gromacs Users mailing list > > >> > >>>>>>> > > >> > >>>>>>> * Please search the archive at > > >> > >>>>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List > > >> before > > >> > >>>>>>> posting! > > >> > >>>>>>> > > >> > >>>>>>> * Can't post? Read > > http://www.gromacs.org/Support/Mailing_Lists > > >> > >>>>>>> > > >> > >>>>>>> * For (un)subscribe requests visit > > >> > >>>>>>> > > >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > > >> > >>> or > > >> > >>>>>>> send a mail to gmx-users-requ...@gromacs.org. > > >> > >>>>>>> > > >> > >>>>>> -- > > >> > >>>>>> Gromacs Users mailing list > > >> > >>>>>> > > >> > >>>>>> * Please search the archive at > > >> > >>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List > > before > > >> > >>>>> posting! > > >> > >>>>>> > > >> > >>>>>> * Can't post? Read > > http://www.gromacs.org/Support/Mailing_Lists > > >> > >>>>>> > > >> > >>>>>> * For (un)subscribe requests visit > > >> > >>>>>> > > >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > > >> > >>> or > > >> > >>>>> send a mail to gmx-users-requ...@gromacs.org. > > >> > >>>>> -- > > >> > >>>>> Gromacs Users mailing list > > >> > >>>>> > > >> > >>>>> * Please search the archive at > > >> > >>>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List > > before > > >> > >>>>> posting! > > >> > >>>>> > > >> > >>>>> * Can't post? Read > http://www.gromacs.org/Support/Mailing_Lists > > >> > >>>>> > > >> > >>>>> * For (un)subscribe requests visit > > >> > >>>>> > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > > >> > or > > >> > >>>>> send a mail to gmx-users-requ...@gromacs.org. > > >> > >>>>> > > >> > >>>> -- > > >> > >>>> Gromacs Users mailing list > > >> > >>>> > > >> > >>>> * Please search the archive at > > >> > >>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List > > before > > >> > >>>> posting! > > >> > >>>> > > >> > >>>> * Can't post? Read > http://www.gromacs.org/Support/Mailing_Lists > > >> > >>>> > > >> > >>>> * For (un)subscribe requests visit > > >> > >>>> > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > > >> or > > >> > >>>> send a mail to gmx-users-requ...@gromacs.org. > > >> > >>>> > > >> > >>> -- > > >> > >>> Gromacs Users mailing list > > >> > >>> > > >> > >>> * Please search the archive at > > >> > >>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List > > before > > >> > >>> posting! > > >> > >>> > > >> > >>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > >> > >>> > > >> > >>> * For (un)subscribe requests visit > > >> > >>> > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > > >> or > > >> > >>> send a mail to gmx-users-requ...@gromacs.org. > > >> > >>> > > >> > >> > > >> > >> > > >> > > -- > > >> > > Gromacs Users mailing list > > >> > > > > >> > > * Please search the archive at > > >> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > >> > posting! > > >> > > > > >> > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > >> > > > > >> > > * For (un)subscribe requests visit > > >> > > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > > or > > >> > send a mail to gmx-users-requ...@gromacs.org. > > >> > > > >> > > > >> > -- > > >> > Dr. Carsten Kutzner > > >> > Max Planck Institute for Biophysical Chemistry > > >> > Theoretical and Computational Biophysics > > >> > Am Fassberg 11, 37077 Goettingen, Germany > > >> > Tel. +49-551-2012313, Fax: +49-551-2012302 > > >> > http://www.mpibpc.mpg.de/grubmueller/kutzner > > >> > http://www.mpibpc.mpg.de/grubmueller/sppexa > > >> > > > >> > -- > > >> > Gromacs Users mailing list > > >> > > > >> > * Please search the archive at > > >> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > >> > posting! > > >> > > > >> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > >> > > > >> > * For (un)subscribe requests visit > > >> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users > or > > >> > send a mail to gmx-users-requ...@gromacs.org. > > >> > > > >> -- > > >> Gromacs Users mailing list > > >> > > >> * Please search the archive at > > >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > >> posting! > > >> > > >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > >> > > >> * For (un)subscribe requests visit > > >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > >> send a mail to gmx-users-requ...@gromacs.org. > > >> > > > -- > > > Gromacs Users mailing list > > > > > > * Please search the archive at > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > posting! > > > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > > > * For (un)subscribe requests visit > > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > send a mail to gmx-users-requ...@gromacs.org. > > > -- > > > Gromacs Users mailing list > > > > > > * Please search the archive at > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > posting! > > > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > > > * For (un)subscribe requests visit > > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > send a mail to gmx-users-requ...@gromacs.org. > > -- > > Gromacs Users mailing list > > > > * Please search the archive at > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > posting! > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > * For (un)subscribe requests visit > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > send a mail to gmx-users-requ...@gromacs.org. > > > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. > -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.