On 4/18/2011 6:44 PM, Larcombe, Lee wrote: > Ok. Solved it. Nothing wrong with the LSF/SLURM/MPI stuff. GMXRC.bash > hadn't executed properly to set the environment up (typo) and the shared > libraries weren’t being found. Seems this makes mdrun_mpi run like serial > mdrun!
OK, but failing to pick up shared libraries would cause a fatal error to that effect. I think there is another mdrun_mpi in the PATH that is not actually MPI-configured. Mark > Oops. > > > Lee > > On 16/04/2011 22:51, "Larcombe, Lee" <l.larco...@cranfield.ac.uk> wrote: > >> Thanks Mark, >> >> HP-MPI is configured correctly on the system - an HP XC 3000 >800 cores. >> It works for all the other users (none gromacs) and no I've tested it, I >> can launch an mpi job which runs fine on the login node (two quad core >> xeons) >> >> It seems to be an issue with the number of processors passed from the BSUB >> settings in the LSF script via srun to gromacs. I just wondered whether >> anyone else had 4.5.3 working in a similar setup? >> >> Most of the LSF/SLURM stuff I can google (not much) seems to be pre 4.5 >> where mdrun_mpi will take the depreciated command line switch to tell it >> the number of processors >> >> Lee >> >> ================================== >> Dr Lee Larcombe >> - Lecturer in Genetics & Computational Biology >> - London Technology Network Business Fellow >> - Course Director of MSc Applied Bioinformatics >> >> Course website: http://bit.ly/cM6SkT >> Group Research: http://bit.ly/bfqtyo >> >> Bioinformatics Group, Cranfield University >> Bedfordshire MK43 0AL. T: +44 (0)1234 758320 >> ================================== >> >> >> >> >> >> >> On 16/04/2011 22:34, "Mark Abraham" <mark.abra...@anu.edu.au> wrote: >> >>> On 16/04/2011 12:13 AM, Larcombe, Lee wrote: >>>> Hi gmx-users >>>> >>>> We have an HPC setup running HP_MPI and LSF/SLURM. Gromacs 4.5.3 has >>>> been compiled with mpi support >>>> The compute nodes on the system contain 2 x dual core Xeons which the >>>> system sees as 4 processors >>>> >>>> An LSF script called gromacs_run.lsf is as shown below >>>> >>>> #BSUB -N >>>> #BSUB -J "gromacsTest5" >>>> #BSUB -u l.larco...@cranfield.ac.uk >>>> #BSUB -n 4 >>>> #BSUB -q short >>>> #BSUB -o %J.log >>>> mpirun -srun mdrun_mpi -v -s xxx.tpr -o xxx.trr >>>> >>>> Queued with: >>>> >>>> Bsub< gromacs_run.lsf >>>> >>>> This is intended to run 1 mdrun on a single node using all four cores >>>> of the two xeons. The result is that although the job is only submitted >>>> to one compute node, 4 mdruns are launched on each of the 4 cores = 16 >>>> jobs. These are all the same as if mdrun has not been compiled with mpi >>>> support. >>> mdrun_mpi will run one process on each core that the MPI configuration >>> declares is available. Spawning four separate runs of four processes >>> indicates that MPI is not configured to reflect the hardware (since each >>> run thinks it can have four processes), or that the submission script is >>> inappropriate (since four runs get spawned), or both. We can't help >>> there. Trouble-shoot with a trivial MPI test program. >>> >>>> If I tell srun to start just one task with "mpirun -srun -n1 mdrun_mpi >>>> -v -s xxx.tpr o xxx.trr" it starts one job on each core instead of 4: >>>> >>>> NNODES=1, MYRANK=0, HOSTNAME=comp195 >>>> NNODES=1, MYRANK=0, HOSTNAME=comp195 >>>> NNODES=1, MYRANK=0, HOSTNAME=comp195 >>>> NNODES=1, MYRANK=0, HOSTNAME=comp195 >>>> >>>> Logs show 4 mdrun_mpi starts, 4 file read ins and I get 4 of all run >>>> files in CWD. I am sure >>> >>> >>>> that mdrun_mpi is indeed compiled with mpi support - although our >>>> sysadmin did that, not me. For example, if I try and execute "mdrun_mpi >>>> h" I get a message from HPMPI and have to execute "mpirun mdrun_mpi >>>> h" to see the help text. >>>> >>>> Does anyone have any experience of running with this setup - any >>>> ideas? >>>> >>>> Thanks >>>> Lee >>> -- >>> gmx-users mailing list gmx-users@gromacs.org >>> http://lists.gromacs.org/mailman/listinfo/gmx-users >>> Please search the archive at >>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting! >>> Please don't post (un)subscribe requests to the list. Use the >>> www interface or send it to gmx-users-requ...@gromacs.org. >>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >> -- >> gmx-users mailing list gmx-users@gromacs.org >> http://lists.gromacs.org/mailman/listinfo/gmx-users >> Please search the archive at >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting! >> Please don't post (un)subscribe requests to the list. Use the >> www interface or send it to gmx-users-requ...@gromacs.org. >> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
