Hello,
I noticed an odd behaviour of the sinfo command when following up the use of
MaxCPUsPerNode suggested in the slurm.conf documentation.
I wanted to set up two partitions for two nodes each with 12 CPU cores and 2
GPUs such that there is always one CPU core per GPU available and the remaining
CPU cores can be used for other jobs. I started with this in slurm.conf
SelectType=select/cons_res
SelectTypeParameters=CR_Core_Memory
...
NodeName=cn[501-502] Sockets=2 CoresPerSocket=6 ThreadsPerCore=1
RealMemory=48000
...
PartitionName=seq5.q nodes=cn[501-502] MaxCPUsPerNode=10
MaxTime=INFINITE State=UP
PartitionName=gpu5.q nodes=cn[501-502] MaxCPUsPerNode=2
MaxTime=INFINITE State=UP
At this point there are no GPU resources included yet but there is a general
problem with the reporting of sinfo for the allocated/idle/other/total
combination. Instead of reporting the allocated CPUS/cores per partition it
reports this per node which may overstate the number of allocated cores.
For example I submitted several jobs to gpu5.q only and checked the states of
both queues:
# squeue -p gpu5.q
JOBID PARTITION NAME USER ST TIME NODES
NODELIST(REASON)
44593 gpu5.q serial.s root PD 0:00 1 (Resources)
44594 gpu5.q serial.s root PD 0:00 1 (Priority)
44589 gpu5.q serial.s root R 0:18 1 cn501
44590 gpu5.q serial.s root R 0:13 1 cn501
44591 gpu5.q serial.s root R 0:10 1 cn502
44592 gpu5.q serial.s root R 0:10 1 cn502
# squeue -p seq5.q
JOBID PARTITION NAME USER ST TIME NODES
NODELIST(REASON)
which is correct. But sinfo produces an inconsistent allocation when reporting
to the user:
# sinfo -o %10R%15C -p gpu5.q
PARTITION CPUS(A/I/O/T)
gpu5.q 4/20/0/24
# sinfo -o %10R%15C -p seq5.q
PARTITION CPUS(A/I/O/T)
seq5.q 4/20/0/24
The last sinfo output is not correct. Is this a bug or is there another way to
get the correct number of allocated cores per partition?
The version of slurm is 14.03.7
Thanks
Henk
