Re: [slurm-users] Cluster usage with Slurm

2020-02-17 Thread Paul Edmon
Also if you want tracking of fairshare and other stats in graphite, you can use these: https://github.com/fasrc/slurm-diamond-collector -Paul Edmon- On 2/17/2020 8:57 AM, Chris Samuel wrote: On 17/2/20 4:19 am, Parag Khuraswar wrote: Does Slurm  provide cluster usage reports like mentioned

Re: [slurm-users] Inconsistent cpu bindings with cpu-bind=none

2020-02-17 Thread Chris Samuel
On 17/2/20 12:48 am, Marcus Boden wrote: I am facing a bit of a weird issue with CPU bindings and mpirun: I think if you want Slurm to have any control over bindings you'll be wanting to use srun to launch your MPI program, not mpirun. All the best, Chris -- Chris Samuel :

Re: [slurm-users] Cluster usage with Slurm

2020-02-17 Thread Chris Samuel
On 17/2/20 4:19 am, Parag Khuraswar wrote: Does Slurm  provide cluster usage reports like mentioned below ? For the detailed info you're being asked for I'd probably suggest looking at the OpenXDMoD project. https://open.xdmod.org/ Its "shredder" data importer can import data from a bunch

Re: [slurm-users] Job limit in slurm.

2020-02-17 Thread Ole Holm Nielsen
Hi Navin, Since you are new to Slurm, you may perhaps find my Slurm Wiki pages useful: https://wiki.fysik.dtu.dk/niflheim/SLURM These pages assume CentOS 7 Linux, but much of the information should be valid for other Linux variants as well. /Ole On 2/17/20 1:55 PM, navin srivastava wrote:

Re: [slurm-users] Job limit in slurm.

2020-02-17 Thread navin srivastava
Hi ole, Thanks Ole. After setting the Enforce it worked. I am new to slurm to thanks for helping me. Regards Navin On Mon, Feb 17, 2020 at 5:36 PM Ole Holm Nielsen wrote: > Hi Navin, > > I wonder if you have configured the Slurm database and the slurmdbd > daemon? I think the limit

Re: [slurm-users] Cluster usage with Slurm

2020-02-17 Thread Ole Holm Nielsen
On 2/17/20 1:19 PM, Parag Khuraswar wrote: Hi Team, Does Slurm  provide cluster usage reports like mentioned below ? Detailed reports about cluster usage statistics. Reports of every user and jobs including their monthly usage, node usage, percentage of utilization, History tracking, number of

[slurm-users] Cluster usage with Slurm

2020-02-17 Thread Parag Khuraswar
Hi Team, Does Slurm provide cluster usage reports like mentioned below ? Detailed reports about cluster usage statistics. Reports of every user and jobs including their monthly usage, node usage, percentage of utilization, History tracking, number of completed, failed, queued and running

Re: [slurm-users] Job limit in slurm.

2020-02-17 Thread Ole Holm Nielsen
Hi Navin, I wonder if you have configured the Slurm database and the slurmdbd daemon? I think the limit enforcement requires the use of the database. What is the output of: $ scontrol show config | grep AccountingStorageEnforce See also

Re: [slurm-users] Job limit in slurm.

2020-02-17 Thread navin srivastava
Hi ole, i am submitting 100 of jobs are i see all jobs starting at the same time and all job is going into the run state. if Maxjobs limit is set it should allow only 3 jobs at any point of time. Regards Navin. On Mon, Feb 17, 2020 at 4:48 PM Ole Holm Nielsen wrote: > Hi Navin, > > Why do

Re: [slurm-users] Job limit in slurm.

2020-02-17 Thread Ole Holm Nielsen
Hi Navin, Why do you think the limit is not working? The MaxJobs limits the number of running jobs to 3, but you can still submit as many jobs as you like! See "man sacctmgr" for definitions of the limits MaxJobs as well as MaxSubmitJobs. /Ole On 2/17/20 12:04 PM, navin srivastava wrote:

Re: [slurm-users] Job limit in slurm.

2020-02-17 Thread navin srivastava
Hi, Thanks for your script. with this i am able to show the limit what i set. but this limt is not working. MaxJobs =3, current value = 0 Regards Navin. On Mon, Feb 17, 2020 at 4:13 PM Ole Holm Nielsen wrote: > On 2/17/20 11:16 AM, navin srivastava wrote: > > i have an issue with the

Re: [slurm-users] Job limit in slurm.

2020-02-17 Thread Ole Holm Nielsen
On 2/17/20 11:16 AM, navin srivastava wrote: i have an issue with the slurm job limit. i applied the Maxjobs limit on user using  sacctmgr modify user navin1 set maxjobs=3 but still i see this is not getting applied. i am still bale to submit more jobs. Slurm version is 17.11.x Let me

[slurm-users] Job limit in slurm.

2020-02-17 Thread navin srivastava
Hi Team, i have an issue with the slurm job limit. i applied the Maxjobs limit on user using sacctmgr modify user navin1 set maxjobs=3 but still i see this is not getting applied. i am still bale to submit more jobs. Slurm version is 17.11.x Let me know what setting is required to implement

[slurm-users] Inconsistent cpu bindings with cpu-bind=none

2020-02-17 Thread Marcus Boden
Hi everyone, I am facing a bit of a weird issue with CPU bindings and mpirun: My jobscript: #SBATCH -N 20 #SBATCH --tasks-per-node=40 #SBATCH -p medium40 #SBATCH -t 30 #SBATCH -o out/%J.out #SBATCH -e out/%J.err #SBATCH --reservation=root_98 module load impi/2019.4 2>&1 export I_MPI_DEBUG=6