On 1 March 2013 14:46, Raymond Wan <[email protected]> wrote: > Hi William, > > > On Fri, Mar 1, 2013 at 10:09 PM, William Hay <[email protected]> wrote: >> On 1 March 2013 11:55, Raymond Wan <[email protected]> wrote: >>> Basically, I have a problem with users over-subscribing. I'm asking >>> them that they can use only n cores and they are *purposely* using >>> more than that. The number of cores being used added to the number of >>> cores in use by other users at the time is more than the number of >>> cores the system has. >> >> That's a slightly different question. Limiting cores/vcpus used >> rather than threads. >> We create a small cpuset with the appropriate number of cores from the >> prolog(run as root) and have a starter_method call an suid binary that >> drops its parent into the cpuset created (after checking that it is >> currently in the root cpuset). The epilog then cleans up the cpuset >> (kills any processes grid-engine missed and removes it). You can do >> the same with cgroups in later versions of linux. > > > I see! I guess I meant threads; I was quite sloppy with the > terminology. Sorry! > > So, I can limit cores but not threads? That's not as good as limiting > threads, but that still gives me something, at least. > I think we may be using the word threads slightly differently. I was interpreting it in the sense of a thread created with pthread_create or similar. I suspect you are using it to mean the sort of logical processor created by SMT or hyperthreading. These are exposed as processors in linux and can be assigned individually using cpusets or the cpuset functionality within cgroups. If you do mean the hardware/SMT sort of thread and you are using Linux or IRIX then you can specify which hardware threads you jobs use with a bit of elbow grease.
Unfortunately this sort of thing is rather platform dependent although the hwloc (http://www.open-mpi.org/projects/hwloc/) project provides some portable wrappers. I believe some grid engine variants can be compiled to use hwloc but I don't recall which or how advanced it is. The only standards compliant way to assign cores/threads AFAICT is sched_setaffinity but if your users aren't co-operative they can just call sched_setaffinity themselves to grab all cores. CPUSETs on the other hand can be restricted with permissions to stop non-privileged users from moving to another cpuset but only work on Linux/IRIX. > Thank you! > > Ray > > _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
