> >> You can try to use "-tc" in `qsub` to limit the number of instances >> which can be executed at a time, or set "max_pending_tasks_per_job" >> in the scheduler to a lower limit - it's still set to 50? > > I have the second as I don't want to limit the amount of running jobs > but I want to limit the amount of pending jobs that the scheuller > considers each schedd cycle. >
How many nodes does your cluster have? How many cores? Can you run each of the 500,000 array tasks simultaneously without time sharing on a core/slot? If not, you may want to limit the number of jobs to the number of cores/slots you have, otherwise you will run into a backlog of jobs, and if an array job, this can even be more serious. Ian -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
