On Fri, Apr 20, 2018 at 9:33 AM, Salman Zarrini <[email protected]> wrote: > Thanks Christoph. > > On Friday, April 20, 2018 at 8:41:41 AM UTC-4, Christoph Junghans wrote: >> >> On Fri, Apr 20, 2018 at 3:35 AM, Salman Zarrini >> <[email protected]> wrote: >> > Dear Votca developers, >> > >> > My Votca calculations are conducted in a cluster which has 82 machines >> > each with 16 slots. >> > While the Gromacs can parallel either inter-nodes or intranode, the >> > Votca is only multithreaded so that only 16 slots can be used maximally. >> > Consequently the Gromacs as the most time consuming part of a IBI-Votca job >> > would be limited to 16 slots as well. >> > So, I wonder if there is any possibility to use a higher number of slots >> > for at least the Gromacs part of a IBI calculation in such situation? >> Sure nothing prevents you from running VOTCA and gromacs on a >> different number of cores, e.g. using 4 nodes (4*16 cores) for the >> gromacs part (with mpi) and only 1 node (16 cores) for VOTCA's >> csg_stat. > > Probably doable for the csg_stat by using --nt 16, but how about the > csg_inverse in IBI; > As far as I know, the allocation for running a IBI job (csg_inverse > --options settings.xml) in cluster is addressed by two below commands in job > submission script and setting.xml, respectively. > > #$ -pe shm n ## Or other types of parallel environment. > > <mdrun> > <command> mpirun -np m gmx_mpi mdrun </command> ### (m =< n) > </mdrun> > > The n slots indicated in the job submission script would be assigned to the > both VOTCA and Gromacs commands. Then how can I order to use 4 nodes (4*16 > cores) for the gromacs and 1 node (16 cores) for VOTCA (probably the other > way round was much easier in general)? And How VOTCA knows that those 16 > cores are all belong to a single node? because if only 1 core from the 16 > cores is contributed by another node the VOTCA part crashes. The only compute-heavy part in csg_inverse besides mdrun is csg_stat, all the other things take basically no time. And csg_stat when called inside csg_inverse will automatically be given the max. number of threads with the --nt options (check your inverse.log, you can overwrite that with the cg.inverse.simulation.tasks option in settings.xml). On the other hand the behavior of mdrun can be changed by cg.inverse.gromacs.mdrun.command, as you said above.
So in your batch script you should set the number in "-pe" option to same number as you want to use for gromacs (and set in cg.inverse.gromacs.mdrun.command, too), but then in body of the script, just call csg_inverse without mpirun/mpiexec. Then csg_inverse will run serially, but call gromacs with mpi options and csg_stat will automatically make use of all threads on the node where csg_inverse got started. Christoph > > Thank you. > Regards, > Salman Zarrini >> >> >> Christoph >> >> > BTW, Votca launches Gromacs directly with "mpirun” now. >> > >> > Regards, >> > >> > Salman Zarrini >> > Department of Chemical Engineering >> > Drexel University >> > Philadelphia, USA >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "votca" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at https://groups.google.com/group/votca. >> > For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> Christoph Junghans >> Web: http://www.compphys.de > > -- > You received this message because you are subscribed to the Google Groups > "votca" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/votca. > For more options, visit https://groups.google.com/d/optout. -- Christoph Junghans Web: http://www.compphys.de -- You received this message because you are subscribed to the Google Groups "votca" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/votca. For more options, visit https://groups.google.com/d/optout.
