[slurm-users] FairShare priority questions

Hi, I’m trying to set up multifactor priority on our cluster and am having some trouble getting it to behave the way I’d like. My main issues seem to revolve around FairShare. We have multiple projects on our cluster and multiple users in those projects (and some users are in multiple

[slurm-users] FairShare value is always 0.

Hi guys, We've just setup our new cluster and are facing some issues regading fairshare calculation. Our slurm directive regarding priority calculation are defines as follows: PriorityType=priority/multifactor PriorityFlags=MAX_TRES PriorityDecayHalfLife=14-0 PriorityFavorSmall=NO

Re: [slurm-users] Fairshare: Penalising unused memory rather than used memory?

On 10/11/23 07:27, Cristian Huza wrote: I recall there was a built in tool named seff (slurm efficiency), not sure if it is still maintained "seff" is in the Slurm sources in the contribs/seff directory, if you're building RPMs from them then it's in the "slurm-contribs" RPM. -- Chris

Re: [slurm-users] Fairshare: Penalising unused memory rather than used memory?

> On the automation part, it would be pretty easy to do regularly(daily?) stats > of jobs for that period of time and dump them into an sql database. > Then a select statement where cpu_efficiency is less than desired value and > get the list of not so nice users on which you can apply whatever

Re: [slurm-users] Fairshare: Penalising unused memory rather than used memory?

you already use that in a conflicting way. > >> Gareth > >> Get Outlook for Android <https://aka.ms/ghei36> > >> -------- > >> *From:* slurm-users on > >> behalf of Loris Bennett >

Re: [slurm-users] Fairshare: Penalising unused memory rather than used memory?

--- >> *From:* slurm-users on >> behalf of Loris Bennett >> *Sent:* Wednesday, October 11, 2023 5:26:38 PM >> *To:* Slurm Users Mailing List >> *Subject:* [slurm-users] Fairshare: Penalising unused memory rather >> than used memory? >> Hi,

Re: [slurm-users] Fairshare: Penalising unused memory rather than used memory?

sers Mailing List *Subject:* [slurm-users] Fairshare: Penalising unused memory rather than used memory? Hi, We have an increasing number of users who are unable (or unwilling) to estimate their memory requirements accurately.  We include memory in our calculation of fairshare usage and thus

Re: [slurm-users] Fairshare: Penalising unused memory rather than used memory?

that in a conflicting way. Gareth Get Outlook for Android<https://aka.ms/ghei36> From: slurm-users on behalf of Loris Bennett Sent: Wednesday, October 11, 2023 5:26:38 PM To: Slurm Users Mailing List Subject: [slurm-users] Fairshare: Penalising unused memory rathe

[slurm-users] Fairshare: Penalising unused memory rather than used memory?

Hi, We have an increasing number of users who are unable (or unwilling) to estimate their memory requirements accurately. We include memory in our calculation of fairshare usage and thus penalise everyone who requests large amounts of memory, whether it is needed or not. Therefore I would be

Re: [slurm-users] Fairshare within a single Account (Project)

on behalf of Tomislav Maric *Date: *Tuesday, February 1, 2022 at 1:40 PM *To: *slurm-users@lists.schedmd.com *Subject: *Re: [slurm-users] Fairshare within a single Account (Project) *External Email Warning* *This email originated from outside the university. Please use caution when opening

Re: [slurm-users] Fairshare within a single Account (Project)

adding and removing users has been a non-issue. From: slurm-users on behalf of Tomislav Maric Date: Tuesday, February 1, 2022 at 1:40 PM To: slurm-users@lists.schedmd.com Subject: Re: [slurm-users] Fairshare within a single Account (Project) External Email Warning This email originated from

Re: [slurm-users] Fairshare within a single Account (Project)

@lists.schedmd.com *Subject: *[slurm-users] Fairshare within a single Account (Project) *External Email Warning* *This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests

Re: [slurm-users] Fairshare within a single Account (Project)

: slurm-users on behalf of Tomislav Maric Date: Sunday, January 30, 2022 at 5:32 AM To: slurm-users@lists.schedmd.com Subject: [slurm-users] Fairshare within a single Account (Project) External Email Warning This email originated from outside the university. Please use caution when opening

[slurm-users] Fairshare within a single Account (Project)

Hello everyone, We are a small research group that shares an account on the cluster and we thought we would be able to use usage reports to balance the CPUh from different users: we were wrong. Is it possible to set up Fairshare within a single

Re: [slurm-users] Fairshare config change affect on running/queued jobs?

It shouldn't impact running jobs, all it should really do is impact pending jobs as it will order them by their relative priority scores. -Paul Edmon- On 4/30/2021 12:39 PM, Walsh, Kevin wrote: Hello everyone, We wish to deploy "fair share" scheduling configuration and would like to inquire 

[slurm-users] Fairshare config change affect on running/queued jobs?

Hello everyone, We wish to deploy "fair share" scheduling configuration and would like to inquire if we should be aware of effects this might have on jobs already running or already queued when the config is changed. The proposed changes are from the example at

Re: [slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights

want to make predominant. From: slurm-users On Behalf Of Yap, Mike Sent: Wednesday, 7 April 2021 9:57 AM To: Slurm User Community List Subject: Re: [slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights Thanks Luke.. Will go through the 2 commands (will try to digest them) Wondering

Re: [slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights

ril 2021 9:57 AM To: Slurm User Community List Subject: Re: [slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights Thanks Luke.. Will go through the 2 commands (will try to digest them) Wondering if you're able to advise on TRESBillingWeights="CPU=1.0,Mem=0.25G,GRES/gpu=2.0&quo

Re: [slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights

htAge=1000 PriorityWeightPartition=1 From: slurm-users On Behalf Of Luke Yeager Sent: Tuesday, 6 April 2021 3:03 AM To: Slurm User Community List Subject: Re: [slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights * Rawshare is only a representation of weight where higher value equal

Re: [slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights

cpu ? Take a look at the output of this command. In particular, check out the value for GrpTRESMins for your "Association Records": scontrol show assoc_mgr Hope that helps! Luke From: slurm-users On Behalf Of Yap, Mike Sent: Wednesday, March 31, 2021 4:50 PM To: slurm-us...@s

[slurm-users] Fairshare +FairTree Algorithm + TRESBillingWeights

Hi All Need some clarification on Fairshare (multifactor priority plugin) and FairTree Algorithm If I read correctly, the current default for slurm is FairTree algorithm in which 1. Priority can set on various level 2. No fairshare-actual usage is being consider 3. Job submitted will

Re: [slurm-users] Fairshare tree after SLURM upgrade

On 1/29/21 8:03 AM, Gestió Servidors wrote: I’m going to upgrade my SLURM version from 17.11.5 to 19.05.1. I know this is not the last version, but I manage another cluster that is running, also, this version. My question is: during the process, I need to upgrade “slurmdbd”. All the fairshare

[slurm-users] Fairshare tree after SLURM upgrade

Hello, I'm going to upgrade my SLURM version from 17.11.5 to 19.05.1. I know this is not the last version, but I manage another cluster that is running, also, this version. My question is: during the process, I need to upgrade "slurmdbd". All the fairshare tree (with rawusage, effectvusage,

Re: [slurm-users] FairShare

-users@lists.schedmd.com *Subject: *Re: [slurm-users] FairShare *External Email Warning* *This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to re

Re: [slurm-users] FairShare

, December 2, 2020 10:55 AM To: Ryan Cox ; Slurm User Community List ; Erik Bryer Subject: Re: [slurm-users] FairShare Yes, that concept of rank tripped me up. The "count of user associations that start at root" you mean? Do you mean all associations across all accounts or just the acc

Re: [slurm-users] FairShare

't that equal 1? I'm clearly not understanding something fundamental. From: Ryan Cox Sent: Wednesday, December 2, 2020 10:43 AM To: Slurm User Community List ; Erik Bryer ; Micheal Krombopulous Subject: Re: [slurm-users] FairShare >From https://slurm.sch

Re: [slurm-users] FairShare

share of the resource. *From: *slurm-users *Date: *Wednesday, December 2, 2020 at 11:23 AM *To: *slurm-users@lists.schedmd.com *Subject: *Re: [slurm-users] FairShare *External Email Warning* *This email originated from outside the university. Please use caution when opening attachments, cl

Re: [slurm-users] FairShare

. *From:* slurm-users on behalf of Ryan Cox *Sent:* Wednesday, December 2, 2020 10:31 AM *To:* Slurm User Community List ; Micheal Krombopulous *Subject:* Re: [slurm-users] FairShare It's really similar to a binary search

Re: [slurm-users] FairShare

s on behalf of Renfro, Michael Sent: Wednesday, December 2, 2020 10:32 AM To: Slurm User Community List Subject: Re: [slurm-users] FairShare Yesterday, I posted https://docs.rc.fas.harvard.edu/kb/fairshare/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.rc.fas.h

Re: [slurm-users] FairShare

; Micheal Krombopulous Subject: Re: [slurm-users] FairShare It's really similar to a binary search tree. Within each account, it is Shares / Usage to calculate the Level FS. See https://slurm.schedmd.com/SUG14/fair_tree.pdf has more details, starting at page 34 or so. It even has

Re: [slurm-users] FairShare

md.com Subject: Re: [slurm-users] FairShare External Email Warning This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests. I've read the manual and I re-read the other link. What the

Re: [slurm-users] FairShare

ernel can understand. *From:* slurm-users on behalf of Micheal Krombopulous *Sent:* Wednesday, December 2, 2020 9:32 AM *To:* slurm-users@lists.schedmd.com *Subject:* [slurm-users] FairShare Can someone tell me how to calculate fairshare

Re: [slurm-users] FairShare

understand. From: slurm-users on behalf of Micheal Krombopulous Sent: Wednesday, December 2, 2020 9:32 AM To: slurm-users@lists.schedmd.com Subject: [slurm-users] FairShare Can someone tell me how to calculate fairshare (under fairtree)? I can't figure it out. I would ha

Re: [slurm-users] FairShare

Micheal, Details are at https://slurm.schedmd.com/fair_tree.html . If they have the same shares and usage as each other, they will have the same fair share value.  One thing to keep in mind is that sshare rounds or truncates the values, so 0.00

[slurm-users] FairShare

Can someone tell me how to calculate fairshare (under fairtree)? I can't figure it out. I would have thought it would be the same score for all users in an account. E.g., here is one of my accounts: Account User  RawShares  NormShares    RawUsage   NormUsage  EffectvUsage    LevelFS  

Re: [slurm-users] Fairshare per-partition?

I think that’s correct. From notes I’ve got for how we want to handle our fairshare in the future: Setting up a funded account (which can be assigned a fairshare): sacctmgr add account member1 Description="Member1 Description" FairShare=N Adding/removing a user to/from the funded

[slurm-users] Fairshare per-partition?

Hello all. Is it possible to configure Slurm so that fairshare calc on a partition does not impact calc on a different one? We'd need to have different "priorities" on the "postprocessing" nodes than the ones on "parallel" nodes, so that even if an user already used up all his "quota" on

Re: [slurm-users] Fairshare - root user

I think If you increase the share of mygroup to something like 999 then the share that the root user gets will drop by a factor of 1000 pretty sure I've seen this before and that's how I fixed it Antony On Wed, 27 Feb 2019 at 13:47, Will Dennis wrote: > Looking at output of 'sshare", I see: >

Re: [slurm-users] Fairshare - root user

Hi Will, as long as you do not submit massive number of jobs as root, there should be no problem. This is only a priority thing, so root will have a fairly high priority, it does not mean, the users can only use half of your cluster. Best Marcus On 2/27/19 2:43 PM, Will Dennis wrote:

[slurm-users] Fairshare - root user

Looking at output of 'sshare", I see: root@myserver:~# sshare -l Account User RawShares NormSharesRawUsage NormUsage EffectvUsage FairShare -- -- --- --- --- - -- root

Re: [slurm-users] Fairshare factor not reflected in job priority

Sorry for spamming, I've found the answer just answer posting the question. The resolution for my issue was to add FAIR_TREE to PriorityFlags. Fairshare factor for the user who submitted the job was low because of his high utilization of resources and sshare -u login was showing 0.0 for him.