Hi, Just wanted to followup with how we actually implemented our solution. Gave the default queue a high nice (low priority) Doubled our exec level slots config (2 x cores). Created high priority attribute, forced and default off. Created a high priority queue, doubled np_load_avg in it, gave it a lower nice (high priority), assigned the pe high priority attribute. Created the high parallel environment.
We use h_vmem as consumable resource. Now qsub must request -l high to get in the queue with the higher precedence nice value. There are enough slots configured for a job from each queue to get on each core. The high pe is used for high.q while threaded pe is used in the default queue, that was the hard part. We needed separate high.q counter for threaded jobs so we could get a multiple thread default queue and multi thread high priority queue job on the same cores. We went this route because we didn't want to suspend jobs (as subordinate queues would) and the config seemed simpler, closer to our prior config. Now if there is memory available users can submit (threaded or not) jobs to high.q and they will go straight to exec hosts with higher exec priority than running jobs in the default queue. -George On Fri, Jun 1, 2012 at 5:00 PM, George Georgalis <[email protected]> wrote: > inline... > > On Fri, Jun 1, 2012 at 3:46 PM, Rayson Ho <[email protected]>wrote: > >> On Fri, Jun 1, 2012 at 6:12 PM, George Georgalis <[email protected]> >> wrote: >> > our understanding is that priority _only_ impacts which queue gets drawn >> > from when resources become available. That's correct, right? >> >> That's different than the definition in the manpage. I sent you the >> URL in my previous email - can you click on it and read it? It says: >> >> priority >> The priority parameter specifies the nice(2) value at which >> jobs in this queue will be run... > > > > Indeed, queue_conf(5) priority applies to nice values (-20<x<20). The > ambiguity is I was referring to qsub definition (-1023<x<1024), which > "Defines or redefines the priority of the job relative to other jobs." > > I can make use of both, but I do not see how to set a default qsub per > queue priority? Is it possible? > > > > Thanks! never looked at that. Can you clarify what "suspension" means? >> is it >> > SIGSTOP; followed by SIGCONT when resources are free? As mentioned we >> don't >> > want jobs to swap out >> >> If you can't afford preempted jobs to be swapped out, one way to do it >> is to use the cgroup memory controller to change the kernel virtual >> memory manager's behavior. >> >> http://www.kernel.org/doc/Documentation/cgroups/memory.txt >> >> >> http://www.oracle.com/technetwork/articles/servers-storage-admin/resource-controllers-linux-1506602.html >> >> >> http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch01.html >> >> Grid Engine's cgroup integration is coming in Grid Engine 2011.11 update >> 1: >> >> >> http://blogs.scalablelogic.com/2012/05/grid-engine-cgroups-integration.html >> >> (Note that you can use cgroups with older versions of Grid Engine, but >> you won't get the full benefit of the Grid Engine cgroups >> integration.) >> > > csgroups looks interesting, I even have kernel experience from 10 years > back.... unfortunately, due to many factors it's not something I can touch > in the foreseeable future. Getting away from ge-6.1 will be a challenge too. > > since our nodes are actually VM containers on physical hosts. Probably the > easiest way to prevent swapping will be to limit the sum of available ram > to all containers to less than physical ram. Not a panacea, but should work > most of the time. > > -George > > > >, but hadn't considered subordinate_list, since it is >> > not something we are familiar with. If my new understanding is correct, >> we >> > can forget about using injecting nice into qsub; and get the desired >> effect >> > with subordinate_list and queue priority alone? Is there any way (best >> way) >> > to prevent stopped jobs from swapping out? >> > >> > Regards, >> > -George >> > >> > >> > >> > -- >> > George Georgalis, (415) 894-2710, http://www.galis.org/ >> > > > > -- > George Georgalis, (415) 894-2710, http://www.galis.org/ > -- George Georgalis, (415) 894-2710, http://www.galis.org/
_______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
