On Mon, Aug 03, 2015 at 09:20:46AM -0700, Carl G. Riches wrote: > > The 12% comes from the users' goal of how many CPUs out of the total > available a single user can have when the queue(s) are full. Each user's > share should be measured at the time a job is dispatched (functional > share?) but with some fair-sharing over a time frame (share-tree?). That > is, if a user has submitted 10,000 jobs (or 1000 10-core parallel jobs) to > the queue at one time and the queue doesn't have that many available > slots/CPUs, the sharing policy should dispatch other users' jobs in > preference to the large-volume user (but still allowing that user to have > some slots available to process jobs). In the case of users that dump > huge numbers of jobs to the queues at a time, the sharing policy should > remember that high level of use for a short time (no more than a month).
Hi Carl, We have similar policies in place for our GE clusters, and find the functional ticket policy is much easier to manage than the fair-share one, simply because people care (and can understand) the instanteous nature of the functional policy a lot better than the historical fair-share one. For this reason, we weight the functional policy 100x higher than the fair share one. We set auto_user_fshare to 1000 (every user gets 1000 tickets), and give an urgency boost for complex requests of slot (1000x) or m_mem_free (500x) since bulkier jobs are harder to schedule. Even with those boosts a swarm of small jobs will still slow down scheduling of really bulky jobs. Users who submit jobs with different relative priorities can use the "-js" option to qsub to allocate tickets differently for the different classes. Hope that helps. GE scheduling definitely can feel like black magic sometimes. -- -- Skylar Thompson ([email protected]) -- Genome Sciences Department, System Administrator -- Foege Building S046, (206)-685-7354 -- University of Washington School of Medicine _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
