Szilárd, If I may jump in on this conversation, I am having the reverse problem (which I assume others may encounter also) where I am attempting a large REMD run (84 replicas) and I have access to say 12 GPUs and 84 CPUs.
Basically I have less GPUs than simulations. Is there a logical approach to using gputasks and other new options in GROMACS 2018 for this setup? I read through the available documentation,but as you mentioned it seems to be targeted for a single-GPU runs. Thanks so much, Dan On Fri, Feb 9, 2018 at 10:27 AM, Szilárd Páll <pall.szil...@gmail.com> wrote: > On Fri, Feb 9, 2018 at 4:25 PM, Szilárd Páll <pall.szil...@gmail.com> > wrote: > > > Hi, > > > > First of all,have you read the docs (admittedly somewhat brief): > > http://manual.gromacs.org/documentation/2018/user-guide/ > > mdrun-performance.html#types-of-gpu-tasks > > > > The current PME GPU was optimized for single-GPU runs. Using multiple > GPUs > > with PME offloaded works, but this mode hasn't been an optimization > target > > and it will often not give very good performance. Using multiple GPUs > > requires a separate PME rank (as you have realized), only one can be used > > (as we don't support PME decomposition on GPUs) and it comes some > > inherent scaling drawbacks. For this reason, unless you _need_ your > single > > run to be as fast as possible, you'll be better off running multiple > > simulations side-by side. > > > > PS: You can of course also run on two GPUs and run two simulations > side-by-side (on half of the cores for each) to improve the overall > aggregate throughput you get out of the hardware. > > > > > > A few tips for tuning the performance of a multi-GPU run with PME > offload: > > * expect to get at best 1.5 scaling to 2 GPUs (rarely 3 if the tasks > allow) > > * generally it's best to use about the same decomposition that you'd use > > with nonbonded-only offload, e.g. in your case 6-8 ranks > > * map the GPU task alone or at most together with 1 PP rank to a GPU, > i.e. > > use the new -gputasks option > > e.g. for your case I'd expect the following to work ~best: > > gmx mdrun -v -deffnm md -pme gpu -nb gpu -ntmpi 8 -ntomp 6 -npme 1 > > -gputasks 00000001 > > or > > gmx mdrun -v -deffnm md -pme gpu -nb gpu -ntmpi 8 -ntomp 6 -npme 1 > > -gputasks 00000011 > > > > > > Let me know if that gave some improvement. > > > > Cheers, > > > > -- > > Szilárd > > > > On Fri, Feb 9, 2018 at 8:51 AM, Gmx QA <gmxquesti...@gmail.com> wrote: > > > >> Hi list, > >> > >> I am trying out the new gromacs 2018 (really nice so far), but have a > few > >> questions about what command line options I should specify, specifically > >> with the new gnu pme implementation. > >> > >> My computer has two CPUs (with 12 cores each, 24 with hyper threading) > and > >> two GPUs, and I currently (with 2018) start simulations like this: > >> > >> $ gmx mdrun -v -deffnm md -pme gpu -nb gpu -ntmpi 2 -npme 1 -ntomp 24 > >> -gpu_id 01 > >> > >> this works, but gromacs prints the message that 24 omp threads per mpi > >> rank > >> is likely inefficient. However, trying to reduce the number of omp > threads > >> I see a reduction in performance. Is this message no longer relevant > with > >> gpu pme or am I overlooking something? > >> > >> Thanks > >> /PK > >> -- > >> Gromacs Users mailing list > >> > >> * Please search the archive at http://www.gromacs.org/Support > >> /Mailing_Lists/GMX-Users_List before posting! > >> > >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > >> > >> * For (un)subscribe requests visit > >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > >> send a mail to gmx-users-requ...@gromacs.org. > >> > > > > > -- > Gromacs Users mailing list > > * Please search the archive at http://www.gromacs.org/ > Support/Mailing_Lists/GMX-Users_List before posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. > -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
