All, I'm experiencing the following unexpected behavior with SLURM reservations. If I create a reservation on some nodes and forget to point it to a specific partition, and I update the reservation later to point at the correct partition, it doesn't remove any nodes reserved in the wrong partition and replace them with nodes from the partition specified.
Here's the details, first beginning with some info about the relevant defined partitions: PARTITION AVAIL TIMELIMIT NODES STATE NODELIST SB2.7* up 2-00:00:00 2 down* c3-[401,421] SB2.7* up 2-00:00:00 26 idle c3-[402-420,422-428] IB2.2 up 2-00:00:00 12 idle c3-[501-512] Create the reservation: -bash-4.2$ sudo scontrol create reservation StartTime=2014-06-06T08:00:00 Duration=1:00:00 NodeCnt=4 Users=bbarth Reservation created: bbarth_3 -bash-4.2$ scontrol show res ReservationName=bbarth_3 StartTime=2014-06-06T08:00:00 EndTime=2014-06-06T09:00:00 Duration=01:00:00 Nodes=c3-[402-405] NodeCnt=4 CoreCnt=64 Features=(null) PartitionName=SB2.7 Flags= Users=bbarth Accounts=(null) Licenses=(null) State=INACTIVE Observe that the nodes happen to come from the SB2.7 partition. If update the partition on the reservation to be IB2.2, we see that the nodes from SB2.7 are still the ones reserved: -bash-4.2$ sudo scontrol update ReservationName=bbarth_3 Partition=IB2.2 Reservation updated. -bash-4.2$ scontrol show res ReservationName=bbarth_3 StartTime=2014-06-06T08:00:00 EndTime=2014-06-06T09:00:00 Duration=01:00:00 Nodes=c3-[402-405] NodeCnt=4 CoreCnt=64 Features=(null) PartitionName=IB2.2 Flags= Users=bbarth Accounts=(null) Licenses=(null) State=INACTIVE Is this the expected behavior? I also notice that if I drain a node it doesn't get replaced in the reservation, and if I stop SLURM on the node (/etc/init.d/slurm stop) it doesn't get replaced either. I would have sworn up and down that at least the latter worked. Can anyone provide some feedback? Thanks, Bill. -- Bill Barth, Ph.D., Director, HPC [email protected] | Phone: (512) 232-7069 Office: ROC 1.435 | Fax: (512) 475-9445
