Re: [slurm-users] Can job submit plugin detect "--exclusive" ?

2022-02-18 Thread Greg Wickham
Hi Chris, You mentioned “But trials using this do not seem to be fruitful so far.” . . why? In our job_submit.lua there is: if job_desc.shared == 0 then slurm.user_msg("exclusive access is not permitted with GPU jobs.") slurm.user_msg("Remove '--exclusive' from your job

[slurm-users] Can job submit plugin detect "--exclusive" ?

2022-02-18 Thread Christopher Benjamin Coffey
Hello! The job_submit plugin doesn't appear to have a way to detect whether a user requested "--exclusive". Can someone confirm this? Going through the code: src/plugins/job_submit/lua/job_submit_lua.c I don't see anything related. Potentially "shared" could be possible in some way. But trials

Re: [slurm-users] Suspend QOS help

2022-02-18 Thread Walls, Mitchell
Looks as if this is a bug or at least seems to be kind of weird and definitely not mentioned in the docs but separate partition QOS suspend needs PriorityTier set on the partitions found in this https://bugs.schedmd.com/show_bug.cgi?id=13410 PartitionName=testDefault=NO Nodes=node04

Re: [slurm-users] Suspend QOS help

2022-02-18 Thread Walls, Mitchell
Time slicing based suspend works for the QOSes submitted to the same partition so I at least think the configuration is close. It just doesn't seem to be working for separate partitions for the suspend and general QOS. I'd prefer not to timeslice in the separate partition configuration but it

Re: [slurm-users] Suspend QOS help

2022-02-18 Thread Walls, Mitchell
Whoops Nodes name was wrong correction here for partitions. # test partitions PartitionName=testDefault=NO Nodes=node04 OverSubscribe=FORCE:1 MaxTime=30-00:00:00 Qos=general AllowQos=general PartitionName=suspend Default=NO Nodes=node04 OverSubscribe=FORCE:1 MaxTime=30-00:00:00

Re: [slurm-users] Suspend QOS help

2022-02-18 Thread Walls, Mitchell
Both jobs would be using the whole node same as below but with two nodes. I've reduced the problem space to two isolated partitions on just node04. NodeName=node04 CPUs=32 Boards=1 SocketsPerBoard=2 CoresPerSocket=16 ThreadsPerCore=1 RealMemory=257476 Features=cpu # qoses have stayed the same.

Re: [slurm-users] Suspend QOS help

2022-02-18 Thread Brian Andrus
First look and I would guess that there are enough resources to satisfy the requests of both jobs, so no need to suspend. Having the node info and the job info to compare would be the next step. Brian Andrus On 2/18/2022 7:20 AM, Walls, Mitchell wrote: Hello, Hoping someone can shed some

[slurm-users] Suspend QOS help

2022-02-18 Thread Walls, Mitchell
Hello, Hoping someone can shed some light on what is causing jobs to run on same nodes simultaneously rather than being actually suspended for the lower priority job? I can provide more info if someone can think of something to help! # Relevant config. PreemptType=preempt/qos

Re: [slurm-users] [External] What is an easy way to prevent users run programs on the master/login node.

2022-02-18 Thread Stefan Staeglich
Hi Mike, thank you very much :) Stefan Am Montag, 7. Februar 2022, 16:50:54 CET schrieb Michael Robbert: > They moved Arbiter2 to Github. Here is the new official repo: > https://github.com/CHPC-UofU/arbiter2 > > Mike > > On 2/7/22, 06:51, "slurm-users" > wrote: Hi, > > I've just noticed

Re: [slurm-users] Increasing /dev/shm max size?

2022-02-18 Thread Stefan Staeglich
Hi Diego, do you any new insights regarding this issue? Best, Stefan Am Montag, 26. Oktober 2020, 14:48:17 CET schrieb Diego Zuccato: > Il 22/10/20 12:56, Diego Zuccato ha scritto: > > 2) Is the shared memory accounted as belonging to the process and > > enforced accordingly by cgroups? > >