On Fri, Feb 9, 2018, 16:57 Daniel Kozuch <dan.koz...@gmail.com> wrote:
> Szilárd, > > If I may jump in on this conversation, I am having the reverse problem > (which I assume others may encounter also) where I am attempting a large > REMD run (84 replicas) and I have access to say 12 GPUs and 84 CPUs. > > Basically I have less GPUs than simulations. Is there a logical approach to > using gputasks and other new options in GROMACS 2018 for this setup? I read > through the available documentation,but as you mentioned it seems to be > targeted for a single-GPU runs. > Try the example at http://manual.gromacs.org/documentation/2018-latest/user-guide/mdrun-features.html#examples-running-multi-simulations Mark Thanks so much, > Dan > > > > On Fri, Feb 9, 2018 at 10:27 AM, Szilárd Páll <pall.szil...@gmail.com> > wrote: > > > On Fri, Feb 9, 2018 at 4:25 PM, Szilárd Páll <pall.szil...@gmail.com> > > wrote: > > > > > Hi, > > > > > > First of all,have you read the docs (admittedly somewhat brief): > > > http://manual.gromacs.org/documentation/2018/user-guide/ > > > mdrun-performance.html#types-of-gpu-tasks > > > > > > The current PME GPU was optimized for single-GPU runs. Using multiple > > GPUs > > > with PME offloaded works, but this mode hasn't been an optimization > > target > > > and it will often not give very good performance. Using multiple GPUs > > > requires a separate PME rank (as you have realized), only one can be > used > > > (as we don't support PME decomposition on GPUs) and it comes some > > > inherent scaling drawbacks. For this reason, unless you _need_ your > > single > > > run to be as fast as possible, you'll be better off running multiple > > > simulations side-by side. > > > > > > > PS: You can of course also run on two GPUs and run two simulations > > side-by-side (on half of the cores for each) to improve the overall > > aggregate throughput you get out of the hardware. > > > > > > > > > > A few tips for tuning the performance of a multi-GPU run with PME > > offload: > > > * expect to get at best 1.5 scaling to 2 GPUs (rarely 3 if the tasks > > allow) > > > * generally it's best to use about the same decomposition that you'd > use > > > with nonbonded-only offload, e.g. in your case 6-8 ranks > > > * map the GPU task alone or at most together with 1 PP rank to a GPU, > > i.e. > > > use the new -gputasks option > > > e.g. for your case I'd expect the following to work ~best: > > > gmx mdrun -v -deffnm md -pme gpu -nb gpu -ntmpi 8 -ntomp 6 -npme 1 > > > -gputasks 00000001 > > > or > > > gmx mdrun -v -deffnm md -pme gpu -nb gpu -ntmpi 8 -ntomp 6 -npme 1 > > > -gputasks 00000011 > > > > > > > > > Let me know if that gave some improvement. > > > > > > Cheers, > > > > > > -- > > > Szilárd > > > > > > On Fri, Feb 9, 2018 at 8:51 AM, Gmx QA <gmxquesti...@gmail.com> wrote: > > > > > >> Hi list, > > >> > > >> I am trying out the new gromacs 2018 (really nice so far), but have a > > few > > >> questions about what command line options I should specify, > specifically > > >> with the new gnu pme implementation. > > >> > > >> My computer has two CPUs (with 12 cores each, 24 with hyper threading) > > and > > >> two GPUs, and I currently (with 2018) start simulations like this: > > >> > > >> $ gmx mdrun -v -deffnm md -pme gpu -nb gpu -ntmpi 2 -npme 1 -ntomp 24 > > >> -gpu_id 01 > > >> > > >> this works, but gromacs prints the message that 24 omp threads per mpi > > >> rank > > >> is likely inefficient. However, trying to reduce the number of omp > > threads > > >> I see a reduction in performance. Is this message no longer relevant > > with > > >> gpu pme or am I overlooking something? > > >> > > >> Thanks > > >> /PK > > >> -- > > >> Gromacs Users mailing list > > >> > > >> * Please search the archive at http://www.gromacs.org/Support > > >> /Mailing_Lists/GMX-Users_List before posting! > > >> > > >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > >> > > >> * For (un)subscribe requests visit > > >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > >> send a mail to gmx-users-requ...@gromacs.org. > > >> > > > > > > > > -- > > Gromacs Users mailing list > > > > * Please search the archive at http://www.gromacs.org/ > > Support/Mailing_Lists/GMX-Users_List before posting! > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > * For (un)subscribe requests visit > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > send a mail to gmx-users-requ...@gromacs.org. > > > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.