I am now running two slurmds on one node with 4 GPUs, to simulate two nodes with 3 GPUs. The configuration is as follows:
#slurm.conf ... NodeName=bnode1,Gres=gpu:4 NodeName=foo1,NodeHostName=bnode1,Gres=gpu:3 NodeName=foo2,NodeHostName=bnode1,Gres=gpu:3 #gres.conf NodeName=bnode1,Name=gpu,File=/dev/nvidia[0-3] NodeName=foo1, Name=gpu,File=/dev/nvidia[0-2] NodeName=foo1, Name=gpu,File=/dev/nvidia[1-3] I expect that when I run a 3-card program on foo1, another 3-card program should be queued for resources. #srun -wfoo1 --gres=gpu:3 ./myprog1 & srun -wfoo2 --gres=gpu:3 ./myprog2 However the two programs are actually sharing GPU card[1-2]. Does anyone have an idea on this? -- James Zhang At University of Science and Technology of China 15556980026 [email protected]
