Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Ondrej Valousek
rieb Ondrej Valousek > >>> : > >>> Nope, > >>> SoGE (as of 8.1.9) supports CGROUPS w/o any code changes, just add > >>> ???USE_CGROUPS=yes??? to the exec parameter list to make shepherd use > >>> CGroup saveset controller. > >>> My pat

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Skylar Thompson
gt;> > >>> Also a quick note: cgroups is the way to _enforce_ CPU affinity. > >>> For vast majority of the jobs, I would say just a simple taskset > >>> configuration (i.e. i.e. something like ???-l binding linear???) would do > >>> as well. >

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Ondrej Valousek
Message- From: users-boun...@gridengine.org On Behalf Of Mike Serkov Sent: Thursday, August 29, 2019 4:50 PM To: Skylar Thompson Cc: users@gridengine.org Subject: Re: [gridengine users] limit CPU/slot resource to the number of reserved slots Load average indeed. The thing is that if, we have

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Mike Serkov
something like ???-l binding linear???) would do >>> as well. >>> >>> >>> From: Dietmar Rieder >>> Sent: Thursday, August 29, 2019 9:37 AM >>> To: users@gridengine.org; Ondrej Valousek ; >>> users >>> Subject: Re: [griden

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Skylar Thompson
> > From: Dietmar Rieder > > Sent: Thursday, August 29, 2019 9:37 AM > > To: users@gridengine.org; Ondrej Valousek ; > > users > > Subject: Re: [gridengine users] limit CPU/slot resource to the number of > > reserved slots > > > > Great, thanks so much!

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Ondrej Valousek
:28 AM To: Ondrej Valousek Cc: Dietmar Rieder ; users@gridengine.org Subject: Re: [gridengine users] limit CPU/slot resource to the number of reserved slots Also, something to keep in mind - cgroups will not solve this issue completely. It is just affinity enforcement. If the job spawns multiple

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Mike Serkov
> Sent: Monday, August 26, 2019 10:12 PM > To: Dietmar Rieder ; Ondrej Valousek > ; users > Subject: Re: [gridengine users] limit CPU/slot resource to the number of > reserved slots > > I don't think it's supported in Son of GridEngine. Ondrej Valousek (cc'd) > desc

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Ondrej Valousek
; Ondrej Valousek ; users Subject: Re: [gridengine users] limit CPU/slot resource to the number of reserved slots Great, thanks so much! Dietmar Am 29. August 2019 09:05:35 MESZ schrieb Ondrej Valousek mailto:ondrej.valou...@adestotech.com>>: Nope, SoGE (as of 8.1.9) supports CGROUPS w

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Dietmar Rieder
th only extends it to supports system and hence possibility to >hard enforce memory/cpu limits, etc… >Hth, >Ondrej > >From: Daniel Povey >Sent: Monday, August 26, 2019 10:12 PM >To: Dietmar Rieder ; Ondrej Valousek >; users >Subject: Re: [gridengine users] limit CPU/slot resource t

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-29 Thread Ondrej Valousek
: Daniel Povey Sent: Monday, August 26, 2019 10:12 PM To: Dietmar Rieder ; Ondrej Valousek ; users Subject: Re: [gridengine users] limit CPU/slot resource to the number of reserved slots I don't think it's supported in Son of GridEngine. Ondrej Valousek (cc'd) described in the first thread here http

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-28 Thread Hay, William
On Mon, Aug 26, 2019 at 02:15:22PM +0200, Dietmar Rieder wrote: > Hi, > > may be this is a stupid question, but I'd like to limit the used/usable > number of cores to the number of slots that were reserved for a job. > > We often see that people reserve 1 slot, e.g. "qsub -pe smp 1 [...]" > but

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-27 Thread Dietmar Rieder
Hmmm changing the source code is not my intention. But I'll have a look at the discussion you are referring to and see if I change my mind. Thanks Dietmar On 8/26/19 10:11 PM, Daniel Povey wrote: > I don't think it's supported in Son of GridEngine.  Ondrej Valousek > (cc'd) described in the

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-27 Thread Dietmar Rieder
Hi, sure I can just kill those jobs, but this is exactly what I'd like to avoid. Thanks Dietmar On 8/27/19 10:48 AM, Reuti wrote: > Hi, > >> Am 26.08.2019 um 14:15 schrieb Dietmar Rieder : >> >> Hi, >> >> may be this is a stupid question, but I'd like to limit the used/usable >> number of

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-27 Thread Reuti
[sorry for the (possibly) double post, they changed the mail server at the university and now my mail application got confused and uses often a wrong smtp server – not the one it claims to use] Hi, > Am 26.08.2019 um 14:15 schrieb Dietmar Rieder : > > Hi, > > may be this is a stupid

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-26 Thread Daniel Povey
I don't think it's supported in Son of GridEngine. Ondrej Valousek (cc'd) described in the first thread here http://arc.liv.ac.uk/pipermail/sge-discuss/2019-August/thread.html how he was able to implement it, but it required code changes, i.e. you would need to figure out how to build and install

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-26 Thread Dietmar Rieder
Hi, thanks for your reply. This sounds promising. We are using Son of Grid Engine though. Can you point me to the right docs to get cgroup enabled in the exec host (CentOS 7). I must admit I have no experience with cgroups. Thanks again Dietmar On 8/26/19 4:03 PM, Skylar Thompson wrote: > At

Re: [gridengine users] limit CPU/slot resource to the number of reserved slots

2019-08-26 Thread Skylar Thompson
At least for UGE, you will want to use the CPU set integration, which will assign the job to a cgroup that has one CPU per requested slot. Once you have cgroups enabled in the exec host OS, you can then set these options in sge_conf: cgroup_path=/cgroup cpuset=1 You can use this mechanism to