Queues are just a piece of the puzzle when it comes to handling resource
allocation on a multi user system, what (if any) scheduling policies
and resource quotas are you currently using?
That said you are using the queue methods in a good way. There are
certain things that can only be really done on a per-queue basis and top
of the list would be ACL protection and the ability to impose hard or
soft wallclock limits.
A fairshare-by-user policy with the queue structure you set up would be
a decent starting point from which you can gather more data and user
feedback.
Thoughts
- resource quota would perfectly handle the "only N jobs per user can
run in the long-job.q cluster queue ..."
- I've had little success putting wallclock limits on interactive
queues; there are legit business/scientific reasons in many cases for a
long running interactive session. You might want to poll the users or
collect data on this. In a few different environments I've had decent
success by leaving interactive queue slots unrestricted but putting a
resource quota around how many slots a single user can consume. It's
also pretty easy to set up tools that would allow you to dynamically
adjust the size/count of the interactive slot pool to account for
changing demand - it's particularly easy when used with SGE hostgroup
objects.
My $.02
Stephen Spencer <mailto:[email protected]>
January 16, 2015 at 2:50 PM
Good morning.
With the number of users on our clusters growing, it's becoming less
realistic to say "play fair 'cause you're not the only user of the
cluster."
I'm looking for suggestions on setting up queues, both the "why" and
"how," that will allow more of our users access to the cluster.
What I'm thinking of is a multi-queue approach:
* some limited number of "interactive" slots (and they'd be
time-limited)
* a queue for jobs with short time duration - the "express" queue
* a queue for jobs that will run longer... but only so many of these
per user
Any and all suggestions are welcome.
Thank you!
Best,
--
Stephen Spencer
[email protected] <mailto:[email protected]>
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
