Re: [slurm-users] Automatically setting OMP_NUM_THREADS=SLURM_CPUS_PER_TASK?

2018-03-07 Thread Bjørn-Helge Mevik
We have the following code in our TaskProlog: ## Set $OMP_NUM_THREADS unless it was set when calling sbatch or in the job script: if [[ -z $OMP_NUM_THREADS ]]; then if [[ -n $SLURM_CPUS_PER_TASK ]]; then echo export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK else echo export

Re: [slurm-users] Automatically setting OMP_NUM_THREADS=SLURM_CPUS_PER_TASK?

2018-03-06 Thread Bill Barth
We do the same at TACC in our base module (which happens to be called “TACC”), and then we document it. Best, Bill. -- Bill Barth, Ph.D., Director, HPC bba...@tacc.utexas.edu| Phone: (512) 232-7069 Office: ROC 1.435| Fax: (512) 475-9445 On 3/6/18, 5:13 PM,

Re: [slurm-users] Automatically setting OMP_NUM_THREADS=SLURM_CPUS_PER_TASK?

2018-03-06 Thread Ryan Novosielski
Thanks again! I’d seen the second one but not the first one. > On Mar 6, 2018, at 6:28 PM, Martin Cuma wrote: > > MKL is trying to be flexible as it has different potential levels of > parallelism inside. Having MKL_ and OMP_NUM_THREADS can be beneficial in > programs

Re: [slurm-users] Automatically setting OMP_NUM_THREADS=SLURM_CPUS_PER_TASK?

2018-03-06 Thread Martin Cuma
MKL is trying to be flexible as it has different potential levels of parallelism inside. Having MKL_ and OMP_NUM_THREADS can be beneficial in programs where you may want to use your own OpenMP but restrict MKL's or vice versa. A good article on the different options that MKL provides is here:

Re: [slurm-users] Automatically setting OMP_NUM_THREADS=SLURM_CPUS_PER_TASK?

2018-03-06 Thread Ryan Novosielski
Thanks, Martin — I almost mentioned Utah in my original e-mail as I turned up your support page in a search. It is good to know definitively that MKL honors that variable — would be preferable to having to know about various different ones. > On Mar 6, 2018, at 6:07 PM, Martin Cuma

Re: [slurm-users] Automatically setting OMP_NUM_THREADS=SLURM_CPUS_PER_TASK?

2018-03-06 Thread Martin Cuma
Ryan, we set OMP_NUM_THREADS=1 in the R and Python modules (MKL will honor that), and instruct those users that want to run multi-threaded to set OMP_NUM_THREADS themselves after loading the module - and make sure they don't oversubscribe the node. In our experience majority of R and Python