Quoting Yuri D'Elia <[email protected]>:
On 02/20/2014 07:21 PM, Moe Jette wrote:
Slurm uses what is known as a conservative backfill scheduling
algorithm. No job will be started that adversely impacts the expected
start time of _any_ higher priority job. The scheduling can also be
effected by a job's requirements for memory, generic resources,
licenses, and resource limits.
I'm curious whether this could be changed with a setting to disregard
the expected start time of higher priority jobs.
Given that giving/estimating completion times of jobs is akin to sorcery
in many cases, it would be beneficial in my case to always
under-estimate the time limit.
I'm wondering if anybody is running with a overly-conservative TimeLimit
for jobs, and abusing OverTimeLimit [very high value] to achieve this.
I know I would definitely use a EstimatedTimeLimit parameter for
improved backfilling and give an absolute ceiling with TimeLimit (if I
could).
I haven't had time to work on this, but one idea would be estimate a
job's run time based upon historic data and use that as a basis for
backfill scheduling. I suspect the results would be better
responsiveness and higher utilization than when basing scheduling
decisions upon the user's time limit.
Moe Jette
SchedMD
