[slurm-dev] Re: —exclusive on per-job basis - or node-level block based on license?

[Alexandre Strube]

Answering so if someone has the same problem: define one unit of license
per node as a consumable resource. Something like

slurm.conf:

GresTypes=rd_license
NodeName=slave1 CPUS=20 State=UNKNOWN Gres=rd_license:1

And on gres.conf
Name=rd_license Count=1

And then call this process with
sbatch --gres="rd_license:1" ./rd.slurm


2017-03-20 16:20 GMT+01:00 Alexandre Strube :

> Hello,
>
> we have a cluster where we run two some codes, and one has a per-node
> license (so we can run only one instance of such software per node).
>
> Each of our nodes has 20 cores, and this code only uses 1 of them.
>
> Meanwhile, we run other software that could use the excess resources.
>
> If I use CR_CPU, I will clash multiple instances of the code with per-node
> license.
> If I use sbatch —exclusive, I waste the 19 other cores’ time.
>
> Any ideas how I can have a —exclusive per-software, but while still
> allowing other jobs to run on the same node?
>
> --
> []
> Alexandre Strube
> su...@ubuntu.com
>



-- 
[]
Alexandre Strube
su...@ubuntu.com

[slurm-dev] —exclusive on per-job basis - or node-level block based on license?

[Alexandre Strube]

Hello,

we have a cluster where we run two some codes, and one has a per-node
license (so we can run only one instance of such software per node).

Each of our nodes has 20 cores, and this code only uses 1 of them.

Meanwhile, we run other software that could use the excess resources.

If I use CR_CPU, I will clash multiple instances of the code with per-node
license.
If I use sbatch —exclusive, I waste the 19 other cores’ time.

Any ideas how I can have a —exclusive per-software, but while still
allowing other jobs to run on the same node?

-- 
[]
Alexandre Strube
su...@ubuntu.com

[slurm-dev] Re: Fwd: Scheduling jobs according to the CPU load

[Markus Koeberl]

On Monday 20 March 2017 05:38:29 Christopher Samuel wrote:
> 
> On 19/03/17 23:25, kesim wrote:
> 
> > I have 11 nodes and declared 7 CPUs per node. My setup is such that all
> > desktop belongs to group members who are using them mainly as graphics
> > stations. Therefore from time to time an application is requesting high
> > CPU usage.
> 
> In this case I would suggest you carve off 3 cores via cgroups for
> interactive users and give Slurm the other 7 to parcel out to jobs by
> ensuring that Slurm starts within a cgroup dedicated to those 7 cores..
> 
> This is similar to the "boot CPU set" concept that SGI came up with (at
> least I've not come across people doing that before them).
> 
> To be fair this is not really Slurm's problem to solve, Linux gives you
> the tools to do this already, it's just that people don't realise that
> you can use cgroups to do this.
> 
> Your use case is valid, but it isn't really HPC, and you can't really
> blame Slurm for not catering to this.  It can use cgroups to partition
> cores to jobs precisely so it doesn't need to care what the load average
> is - it knows the kernel is ensuring the cores the jobs want are not
> being stomped on by other tasks.

You could additionally define a higher "Weight" value for a host if you know 
that the load is usually higher on it than on the others. 


regards
Markus Köberl
-- 
Markus Koeberl
Graz University of Technology
Signal Processing and Speech Communication Laboratory
E-mail: markus.koeb...@tugraz.at