On Tue, 13 May 2014 21:19:35 +0200 Reuti Reuti wrote: [...] > > after holding 2 of thos big job arrays the schedd is not dying > > anymore. So, could it be that 500000 jobs is too much even if they > > are submitted as job arrays? > > You removed the limit in SGE's configuration "max_aj_instances" then. No, but the limit if quite higher:
max_aj_tasks 500000 I understood that the above limit was not real because the amount of task taken into account was really limited by max_pending_tasks_per_job (which is low: 50). But seems that this combo works till it reaches some high number like 750000 tasks (in my example). > You can try to use "-tc" in `qsub` to limit the number of instances > which can be executed at a time, or set "max_pending_tasks_per_job" > in the scheduler to a lower limit - it's still set to 50? I have the second as I don't want to limit the amount of running jobs but I want to limit the amount of pending jobs that the scheuller considers each schedd cycle. > > Anyone has experience running big job arrays? > > No. > > -- Reuti Cheers, Arnau _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
