Hi, How can I start an MPI job with 4 tasks each using 1 GPU yet make sure that each task uses a different GPU on my nodes with 4 GPUs each?
I tried something like: #SBATCH --ntasks=4 #SBATCH --cpus-per-task=1 #SBATCH --gres=gpu:4 srun --gres=gpu:1 --exclusive ./show_device.sh but then they all use the first device. I then tried: #SBATCH --ntasks=4 #SBATCH --cpus-per-task=1 #SBATCH --gres=gpu:4 srun --gres=gpu:1 -n1 --exclusive /home/filipe/src/mpihello/mpihello & srun --gres=gpu:1 -n1 --exclusive /home/filipe/src/mpihello/mpihello & srun --gres=gpu:1 -n1 --exclusive /home/filipe/src/mpihello/mpihello & srun --gres=gpu:1 -n1 --exclusive /home/filipe/src/mpihello/mpihello & wait but then they end up as 4 independent MPI processes with a comm size of 1 (they cannot communicate). What's the right way? Cheers, Filipe
