Hi Albert, have a look at the defer scheduler parameter from the high throughput guide. This should help in your case.
Best regards Dominik ________________________________________ Von: Albert Solernou [[email protected]] Gesendet: Freitag, 24. Januar 2014 10:58 An: slurm-dev Betreff: [slurm-dev] Re: unresponsive slurmctld Hi, It's not that we have too many jobs in the queue, but that some users submit many quickly through shell scripts. I'll have a look and see if this sorts the problem out. Thank you, Albert On 23/01/14 16:56, Moe Jette wrote: > > There is advise about high-throughput computing here: > http://slurm.schedmd.com/high_throughput.html > > While circumstances at each site vary, I would generally recommend a > default_queue_depth value lower than 100 for systems with high rates of > job submissions. > > Quoting Ulf Markwardt <[email protected]>: > >> Hello Albert, >> >> * what is the numer of jobs in the queue? >> * did you see some heavy submitting ? >> >> - >> >> Just yesterday, we had 15.000 Jobs in the queue and a user submitting >> ~15.000 more in a shell script. To cope with that (surmctld >> scheduling, but not responding) we had to modify our slurm.conf. We >> now have something like this: >> >> SchedulerType=sched/backfill >> SchedulerParameters=default_queue_depth=100,max_job_bf=600,bf_interval=30,bf_max_job_user=10,bf_window=1440,bf_continue >> >> >> This wave is over now, but I know our biologists are preparing their >> next high-throughput experiment... >> >> Regards, >> Ulf >> >> -- >> ___________________________________________________________________ >> Dr. Ulf Markwardt >> >> Dresden University of Technology >> Center for Information Services and High Performance Computing (ZIH) >> 01062 Dresden, Germany >> >> Phone: (+49) 351/463-33640 WWW: http://www.tu-dresden.de/zih >> >> > -- --------------------------------- Dr. Albert Solernou Research Associate Oxford Supercomputing Centre, University of Oxford Tel: +44 (0)1865 610631 --------------------------------- Bull GmbH Sitz Köln, Amtsgericht Köln, HR B 8173 Ust-Id-Nr.: DE 121965133, WEEE-Reg.-Nr. DE 64193985 Geschäftsführer: Gerd-Lothar Leonhart, Michael Heinrichs, Philippe Miltin Zentrale: 51149 Köln, Von-der-Wettern-Strasse 27 Telefon: +49 (0) 2203 305-0 Telefax: +49 (0) 2203 305-1699 http://www.bull.de Bull, Architect of an Open World TM ** Folgen Sie uns auf Twitter: http://twitter.com/bull_de ** Bull Firmenprofil bei XING: https://www.xing.com/companies/bullgmbh
