On 03/14/2014 11:08 PM, Bill Barth wrote:
All, We're trying to see if we can coax SLURM into to helping us institute a policy to enable certain partitions to only run at night. We'd like to have two partitions with overlapping nodes, and we'd like have one of the partitions active 24-7 and the other only active 8am-8pm (say). Is this possible with SLURM? I could turn off one partition with a cron job, but that would allow jobs to schedule at 7:59am and run into the closed window. Any other thoughts on how this might be accomplished? Thanks in advance, Bill. -- Bill Barth, Ph.D., Director, HPC [email protected] | Phone: (512) 232-7069 Office: ROC 1.435 | Fax: (512) 475-9445
Bill, We run a certain partition (actually two, on the same nodes) only on certain hours on weekdays and accomplish this with cron and short reservations (the WEEKLY flag keeps the reservation alive between weeks): # scontrol show res ReservationName=devel_prepare_monday StartTime=2014-03-24T08:59:00 EndTime=2014-03-24T09:04:00 Duration=00:05:00 Nodes=m[169-188] NodeCnt=20 CoreCnt=320 Features=(null) PartitionName=core Flags=OVERLAP,IGNORE_JOBS,WEEKLY,SPEC_NODES,STATIC Users=lka Accounts=(null) Licenses=(null) State=INACTIVE ReservationName=devel_prepare_tuesday StartTime=2014-03-18T08:59:00 EndTime=2014-03-18T09:04:00 Duration=00:05:00 Nodes=m[169-188] NodeCnt=20 CoreCnt=320 Features=(null) PartitionName=core Flags=OVERLAP,IGNORE_JOBS,WEEKLY,SPEC_NODES,STATIC Users=lka Accounts=(null) Licenses=(null) State=INACTIVE ReservationName=devel_prepare_wednesday StartTime=2014-03-19T08:59:00 EndTime=2014-03-19T09:04:00 Duration=00:05:00 Nodes=m[169-188] NodeCnt=20 CoreCnt=320 Features=(null) PartitionName=core Flags=OVERLAP,IGNORE_JOBS,WEEKLY,SPEC_NODES,STATIC Users=lka Accounts=(null) Licenses=(null) State=INACTIVE ReservationName=devel_prepare_thursday StartTime=2014-03-20T08:59:00 EndTime=2014-03-20T09:04:00 Duration=00:05:00 Nodes=m[169-188] NodeCnt=20 CoreCnt=320 Features=(null) PartitionName=core Flags=OVERLAP,IGNORE_JOBS,WEEKLY,SPEC_NODES,STATIC Users=lka Accounts=(null) Licenses=(null) State=INACTIVE ReservationName=devel_prepare_friday StartTime=2014-03-21T08:59:00 EndTime=2014-03-21T09:04:00 Duration=00:05:00 Nodes=m[169-188] NodeCnt=20 CoreCnt=320 Features=(null) PartitionName=core Flags=OVERLAP,IGNORE_JOBS,WEEKLY,SPEC_NODES,STATIC Users=lka Accounts=(null) Licenses=(null) State=INACTIVE # grep devel /etc/crontab # Tricks with the devel and devcore partitions 0 9-19 * * 1-5 root (/usr/bin/scontrol update part=devel state=up; /usr/bin/scontrol update part=devcore state=up; /usr/bin/scontrol update part=node nodes=m[189-208]; /usr/bin/scontrol update part=core nodes=m[189-208]) 0 0-8,20-23 * * 1-5 root (/usr/bin/scontrol update part=devel state=inactive; /usr/bin/scontrol update part=devcore state=inactive; /usr/bin/scontrol update part=node nodes=m[169-208]; /usr/bin/scontrol update part=core nodes=m[169-208]) 0 * * * 6-7 root (/usr/bin/scontrol update part=devel state=inactive; /usr/bin/scontrol update part=devcore state=inactive; /usr/bin/scontrol update part=node nodes=m[169-208]; /usr/bin/scontrol update part=core nodes=m[169-208]) It is admittedly very ugly, but it works. Best wishes, -- Lennart Karlsson, UPPMAX, Uppsala University, Sweden
