On Fri, 16 May 2014 11:07:45 -0700 Ian Kaufman wrote: > > > >> You can try to use "-tc" in `qsub` to limit the number of instances > >> which can be executed at a time, or set "max_pending_tasks_per_job" > >> in the scheduler to a lower limit - it's still set to 50? > > > > I have the second as I don't want to limit the amount of running > > jobs but I want to limit the amount of pending jobs that the > > scheuller considers each schedd cycle. > > > > > How many nodes does your cluster have? How many cores? Can you run
about 80 nodes and 1300 slots. > each of the 500,000 array tasks simultaneously without time sharing on > a core/slot? If not, you may want to limit the number of jobs to the > number of cores/slots you have, otherwise you will run into a backlog > of jobs, and if an array job, this can even be more serious. Why this backlog? Once all nodes are running jobs, in each sched cycle it's able to start from 50/100 jobs, and with that frequency it's enough for having all the cluster nodes running jobs. * From my experience, pending task parameter does what I want : reduce the scheduling overhead by not considering all task but few. > Ian Cheers, Arnau _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
