I've used a routing queue to solve this problem. The queue that the
user is running on can only utilize 32 cpu's. The thousands of jobs
are 1 cpu each. So I have this for a routing queue:
create queue physics
set queue physics queue_type = Route
set queue physics acl_group_enable = True
set queue physics route_destinations += herc
set queue physics enabled = True
set queue physics started = True
So jobs that go into here are moved to the herc execution queue. This
queue has the following setting:
set queue herc max_queuable = 36
This way only 36 jobs at time can be queue'd from the routing queue.
This way maui doesn't even have to worry about considering each of all
the thousand's of jobs each iteration. It only has to worry about
scheduling the jobs for the resources it has to run on.
I also use MAXIJOB in maui:
CLASSCFG[herc] QLIST=md QDEF=md MAXIJOB=4
This way even if a user had lots of jobs in the queue only their top 4
idle jobs will get considered for scheduling. This way others will be
able to get their jobs to run without having to wait for maui to
process thousands of jobs that can't run yet anyhow.
I hope this helps.
-Steve
On Dec 10, 2008, at 3:38 PM, Nicholas Geraedts wrote:
Thanks Troy and Halvor. You were both correct about the MMAX_JOB
definition in the .h file. I've increased it and asked the user to
try to break the system again. I'll let you know how things go.
Cheers,
-Nick
_______________________________________________
mauiusers mailing list
[email protected]
http://www.supercluster.org/mailman/listinfo/mauiusers
_______________________________________________
mauiusers mailing list
[email protected]
http://www.supercluster.org/mailman/listinfo/mauiusers
