Re: [slurm-users] one job at a time - how to set?

2020-04-29 Thread Rutger Vos
One job total! On Wed, Apr 29, 2020 at 10:37 PM Killian Murphy wrote: > Hi Rutger. > > Are you trying to have one job *per user* running in your partition? Or > just one job total? > > Killian > > On Wed, 29 Apr 2020 at 21:27, Doug Meyer wrote: > >> Change node definition in slurm.conf for

Re: [slurm-users] one job at a time - how to set?

2020-04-29 Thread Baer, Troy
I don’t think there’s a way to do that in Slurm using just the node declaration, other than the previously mentioned way of configuring it to show up as having only 1 core. However, you could put the node in a partition that has OverSubscribe=EXCLUSIVE set, and have that partition be the only

Re: [slurm-users] one job at a time - how to set?

2020-04-29 Thread Doug Meyer
Change node definition in slurm.conf for that one node to 1 CPU. Doug Meyer From: slurm-users On Behalf Of Rutger Vos Sent: Wednesday, April 29, 2020 1:20 PM To: Slurm User Community List Subject: [External] Re: [slurm-users] one job at a time - how to set? Hi Michael, thanks very much for

Re: [slurm-users] one job at a time - how to set?

2020-04-29 Thread Killian Murphy
Hi Rutger. Are you trying to have one job *per user* running in your partition? Or just one job total? Killian On Wed, 29 Apr 2020 at 21:27, Doug Meyer wrote: > Change node definition in slurm.conf for that one node to 1 CPU. > > > > *Doug Meyer* > > > > *From:* slurm-users *On Behalf Of >

Re: [slurm-users] one job at a time - how to set?

2020-04-29 Thread Rutger Vos
Hi Michael, thanks very much for your swift reply. So here we would have to convince the users they'd have to specify this when submitting, right? I.e. 'sbatch --exclusive myjob.sh', if I understand correctly. Would there be a way to simply enforce this, i.e. at the slurm.conf level or something?

Re: [slurm-users] one job at a time - how to set?

2020-04-29 Thread Renfro, Michael
That’s a *really* old version, but https://slurm.schedmd.com/archive/slurm-15.08.13/sbatch.html indicates there’s an exclusive flag you can set. On Apr 29, 2020, at 1:54 PM, Rutger Vos wrote: . Hi, for a smallish machine that has been having degraded performance we want to implement a

[slurm-users] TensorRT script runs with srun but not from a sbatch file

2020-04-29 Thread Robert Kudyba
I'm using this TensorRT tutorial with MPS on Slurm 20.02 on Bright Cluster 8.2 Here are the contents of my mpsmovietest sbatch file: #!/bin/bash #SBATCH --nodes=1 #SBATCH --job-name=MPSMovieTest #SBATCH

Re: [slurm-users] not allocating jobs even resources are free

2020-04-29 Thread Brian W. Johanson
Navin, Check out 'sprio', this will give show you how the job priority changes with the weight changes you are making. -b On 4/29/20 5:00 AM, navin srivastava wrote: Thanks Daniel. All jobs went into run state so unable to provide the details but definitely will reach out later if we see

[slurm-users] one job at a time - how to set?

2020-04-29 Thread Rutger Vos
Hi, for a smallish machine that has been having degraded performance we want to implement a policy where only one job (submitted with sbatch) is allowed to run and any others submitted after it are supposed to wait in line. I assumed this was straightforward but I can't seem to figure it out.

[slurm-users] Feature request: SBATCH_NTASKS as input environment variable

2020-04-29 Thread Jaume Zaragoza
Hi all, There are some sbatch parameters that can be passed as input environment variables like SBATCH_PARTITION or SBATCH_TIMELIMIT. But, why the number of tasks cannot be passed (SBATCH_NTASKS or SLURM_NTASKS)? I've read on the man that srun already reads SLURM_NTASKS as input. Thanks,

Re: [slurm-users] not allocating jobs even resources are free

2020-04-29 Thread navin srivastava
Thanks Daniel. All jobs went into run state so unable to provide the details but definitely will reach out later if we see similar issue. i am more interested to understand the FIFO with Fair Tree.it will be good if anybody provide some insight on this combination and also if we will enable the