There is advise about high-throughput computing here:
http://slurm.schedmd.com/high_throughput.html
While circumstances at each site vary, I would generally recommend a
default_queue_depth value lower than 100 for systems with high rates
of job submissions.
Quoting Ulf Markwardt <[email protected]>:
Hello Albert,
* what is the numer of jobs in the queue?
* did you see some heavy submitting ?
-
Just yesterday, we had 15.000 Jobs in the queue and a user
submitting ~15.000 more in a shell script. To cope with that
(surmctld scheduling, but not responding) we had to modify our
slurm.conf. We now have something like this:
SchedulerType=sched/backfill
SchedulerParameters=default_queue_depth=100,max_job_bf=600,bf_interval=30,bf_max_job_user=10,bf_window=1440,bf_continue
This wave is over now, but I know our biologists are preparing their
next high-throughput experiment...
Regards,
Ulf
--
___________________________________________________________________
Dr. Ulf Markwardt
Dresden University of Technology
Center for Information Services and High Performance Computing (ZIH)
01062 Dresden, Germany
Phone: (+49) 351/463-33640 WWW: http://www.tu-dresden.de/zih
