Nice idea. I have a concern that in both cases, in an ideal implementation, pending job priorities might need an expensive update as each new job is started. It is possibly sufficient to start a whole set of jobs in one cycle without updating priorities but the undesired behaviour of a block of one users jobs dominating the top of the priority queue may remain.
My cron approach with raised priority for a few jobs has been sufficient for our needs so far. Looking back through old email, I already supplied the script to Jacob at schedmd with a permissive license. Gareth > -----Original Message----- > From: Paul Edmon [mailto:[email protected]] > Sent: Wednesday, 7 October 2015 1:24 PM > To: slurm-dev <[email protected]> > Subject: [slurm-dev] RE: slurm job priorities depending on the number of > each user's jobs > > > There was some discussion of an analogous feature at the Slurm User Group > meeting. > > http://bugs.schedmd.com/show_bug.cgi?id=1951 > > Basically jobs would have their fairshare penalized immediately once a job > launched for the full runtime, rather than actual time used. Which would > have a similar effect to what you are asking for. > > That said they aren't exactly analogous and I can see situations where one > would want to do this sort of thing so that no one person can monopolize the > queue. > > -Paul Edmon- > > On 10/6/2015 6:30 PM, Kumar, Amit wrote: > > Just wanted to jump the wagon, this feature would help us as well...!! > > > > Thank you, > > Amit > > > > > >> -----Original Message----- > >> From: [email protected] [mailto:[email protected]] > >> Sent: Tuesday, October 06, 2015 5:23 PM > >> To: slurm-dev > >> Subject: [slurm-dev] RE: slurm job priorities depending on the number > >> of each user's jobs > >> > >> Hi Markus, > >> > >> We would also like such a feature. We have a clunky but practical > >> alternative where a cron job repeatedly runs to reset the 'nice' > >> factor for the top few pending jobs for each user to spread priority, > >> encourage interleaving of jobs starting and help each user to have at > >> least one job running quickly. I can provide our script if you are > interested. It requires allowing negative nice. > >> > >> Gareth > >> > >>> -----Original Message----- > >>> From: Dr. Markus Stöhr [mailto:[email protected]] > >>> Sent: Tuesday, 6 October 2015 12:18 AM > >>> To: slurm-dev <[email protected]> > >>> Subject: [slurm-dev] slurm job priorities depending on the number of > >>> each user's jobs > >>> > >>> > >>> Dear all, > >>> > >>> we are currently looking at the details of the slurm priority > >>> multifactor > >> plugin. > >>> Basically it is doing what it should, but there are, at least from > >>> our point of view, some limitations. The five contributions to > >>> priority (fairshare, qos, partition, job size and job > >>> age) are fixed at a time. So if a user has submitted a huge number > >>> of equal sized jobs, all jobs have nearly the same priority. If such > >>> a bunch of jobs has highest priority, nearly all of them might start > >>> simultanously, not allowing the start of jobs of other users. > >>> The situtation described above looks like this: > >>> > >>> prio / user / job_nr_of_user > >>> 500 X 1 > >>> 500 X 2 > >>> 500 X 3 > >>> 500 X 4 > >>> 500 X 5 > >>> 500 X 6 > >>> > >>> 400 Z 1 > >>> 400 Z 2 > >>> 400 Z 3 > >>> > >>> 300 A 1 > >>> 300 A 2 > >>> 300 A 3 > >>> > >>> What we want to achieve is something like the following list, ie. > >>> only a few jobs of each user are getting the highest possible > >>> priority, all others are reduced in priority, relative to the other users' > jobs: > >>> > >>> prio / user / job_nr_of_user > >>> 500 X 1 > >>> 450 X 2 > >>> > >>> 400 Z 1 > >>> 300 A 1 > >>> > >>> 250 X 3 > >>> 250 X 4 > >>> 250 X 5 > >>> 250 X 6 > >>> > >>> 220 Z 2 > >>> 220 Z 3 > >>> > >>> 200 A 2 > >>> 200 A 3 > >>> > >>> Basically this would use the number of jobs per user and weight the > >>> job index (of every user) with a damping function. The questions are: > >>> > >>> 1) Can this be done with existing parameters (e.g. limits of > QOS/Partitions)? > >>> 2) If a own multifactor priority plugin is the method of choice, has > >>> anyone maybe programmed one by him/herself? > >>> > >>> best regards, > >>> Markus > >>> > >>> > >>> > >>> > >>> -- > >>> ===================================================== > >>> Dr. Markus Stöhr > >>> Zentraler Informatikdienst BOKU Wien / TU Wien Wiedner Hauptstraße > >>> 8-10 > >>> 1040 Wien > >>> > >>> Tel. +43-1-58801-420754 > >>> Fax +43-1-58801-9420754 > >>> > >>> Email: [email protected] > >>> =====================================================
