[slurm-dev] Re: set maximum CPU usage per user

[Benjamin Redling]

Hi,

On 10/21/2016 18:58, Steven Lo wrote:
> Is MaxTRESPerUser a better option to use?

if you only ever want to restrict every user alike, that seems reasonable.
I would choose whatever fits your needs right now and in the not so
distant future. That way you gain time to learn about the options slurm
provides.

Anyway, didn't you have any progress with your former setup?
Did you understand what happened?

Regards,
Benjamin
-- 
FSU Jena | JULIELab.de/Staff/Benjamin+Redling.html
vox: +49 3641 9 44323 | fax: +49 3641 9 44321

[slurm-dev] Re: set maximum CPU usage per user

[Steven Lo]

Is MaxTRESPerUser a better option to use?

Steven.


On 10/20/16 10:21 AM, Steven Lo wrote:



Hi Benjamin,

We have the following set in slurm.conf as you have suggested:

AccountingStorageEnforce=limits,qos
PriorityWeightQOS=1000

And we did

sacctmgr modify qos normal set Grpcpus=300

sacctmgr show qos format=GrpTRES
  GrpTRES
-
  cpu=200


I see that when I submit a job requesting over 200 CPUs, the job get 
blocked, which is good.
However, when I submit a job requesting just few CPUs, the job get 
blocked as well.


[slurm-testing]$ squeue
 JOBID PARTITION NAME USER ST   TIME NODES 
NODELIST(REASON)
  2064 debug hello_pa  slo PD   0:00 10 
(QOSGrpCpuLimit)
  2065  smallmem c_gth-dz jmcclain PD   0:00 6 
(QOSGrpCpuLimit)



Do you know why it thinks the job is over 200 CPU limit?  Is there 
other setting we need?



Thanks

Steven.


On 10/20/16 2:13 AM, Benjamin Redling wrote:

Hi Steven,

On 10/20/2016 00:22, Steven Lo wrote:

We have the attribute commented out:
#AccountingStorageEnforce=0

I think the best is to (re)visit "Accounting and Resource Limits":
http://slurm.schedmd.com/accounting.html

Right know I have no setup that needs accounting but as far as I
currently understand you'll need AccoutingStorageEnforce=limits,qos to
get your examples to work.
And just in case you already didn't set it:
for QOS (http://slurm.schedmd.com/qos.html)

PriorityWeightQOS" configuration parameter must be defined in the
slurm.conf file and assigned an integer value greater than zero.


What I am unsure -- esp. not knowing your config -- if there are any
other unmet dependencies.
Would be nice somebody with real experience with accounting could affirm
or give a pointer.

Regards,
Benjamin

[slurm-dev] Re: set maximum CPU usage per user

[Steven Lo]

Hi Benjamin,

We have the following set in slurm.conf as you have suggested:

AccountingStorageEnforce=limits,qos
PriorityWeightQOS=1000

And we did

sacctmgr modify qos normal set Grpcpus=300

sacctmgr show qos format=GrpTRES
  GrpTRES
-
  cpu=200


I see that when I submit a job requesting over 200 CPUs, the job get 
blocked, which is good.
However, when I submit a job requesting just few CPUs, the job get 
blocked as well.


[slurm-testing]$ squeue
 JOBID PARTITION NAME USER ST   TIME  NODES 
NODELIST(REASON)
  2064 debug hello_pa  slo PD   0:00 10 
(QOSGrpCpuLimit)
  2065  smallmem c_gth-dz jmcclain PD   0:00  6 
(QOSGrpCpuLimit)



Do you know why it thinks the job is over 200 CPU limit?  Is there other 
setting we need?



Thanks

Steven.


On 10/20/16 2:13 AM, Benjamin Redling wrote:

Hi Steven,

On 10/20/2016 00:22, Steven Lo wrote:

We have the attribute commented out:
#AccountingStorageEnforce=0

I think the best is to (re)visit "Accounting and Resource Limits":
http://slurm.schedmd.com/accounting.html

Right know I have no setup that needs accounting but as far as I
currently understand you'll need AccoutingStorageEnforce=limits,qos to
get your examples to work.
And just in case you already didn't set it:
for QOS (http://slurm.schedmd.com/qos.html)

PriorityWeightQOS" configuration parameter must be defined in the
slurm.conf file and assigned an integer value greater than zero.


What I am unsure -- esp. not knowing your config -- if there are any
other unmet dependencies.
Would be nice somebody with real experience with accounting could affirm
or give a pointer.

Regards,
Benjamin

[slurm-dev] Re: set maximum CPU usage per user

[Benjamin Redling]

Hi Steven,

On 10/20/2016 00:22, Steven Lo wrote:
> We have the attribute commented out:
> #AccountingStorageEnforce=0

I think the best is to (re)visit "Accounting and Resource Limits":
http://slurm.schedmd.com/accounting.html

Right know I have no setup that needs accounting but as far as I
currently understand you'll need AccoutingStorageEnforce=limits,qos to
get your examples to work.
And just in case you already didn't set it:
for QOS (http://slurm.schedmd.com/qos.html)

PriorityWeightQOS" configuration parameter must be defined in the
slurm.conf file and assigned an integer value greater than zero.


What I am unsure -- esp. not knowing your config -- if there are any
other unmet dependencies.
Would be nice somebody with real experience with accounting could affirm
or give a pointer.

Regards,
Benjamin
-- 
FSU Jena | JULIELab.de/Staff/Benjamin+Redling.html
vox: +49 3641 9 44323 | fax: +49 3641 9 44321

[slurm-dev] Re: set maximum CPU usage per user

[Steven Lo]

Hi Benjamin,

We have the attribute commented out:

#AccountingStorageEnforce=0


This is our first time try out this CPU limitation.  They just want to 
make sure no user will

allocate more than 300 CPU from the cluster.

Thanks

Steven.


On 10/19/16 2:31 PM, Benjamin Redling wrote:
Hi, what are your AccountingStorage settings? Esp. 
AccountingStorageEnforce. Did limits work before, or is this a first try?

Regards, Benjamin

Am 19. Oktober 2016 22:14:27 MESZ, schrieb Steven Lo 
:



By the way, we do have the following attribute set:

  PriorityType=priority/multifactor


Thanks

Steven.


On 10/19/2016 12:55 PM, Steven Lo wrote:

Hi Chris, When we try the command as suggested, it said that
nothing modified: [root@pauling ~]# sacctmgr modify account
normal set Grpcpus=300 Nothing modified Do you know if there
is other method? Thanks Steven. On 10/19/2016 07:25 AM,
Christopher Benjamin Coffey wrote:

Hi Steven, If you are trying to restrict the cpus for a
group, I believe you need to set the account value:
sacctmgr modify account normal set Grpcpus=300 Best, Chris
— Christopher Coffey High-Performance Computing Northern
Arizona University 928-523-1167 On 10/18/16, 4:04 PM,
"Steven Lo"  wrote: Hi, We are trying
to limit 300 CPU usage per user in our cluster. We have
tried: sacctmgr modify qos normal set Grpcpus=300 and
sacctmgr modify user username set GrpCPUs=300 Both seems
to allow job to run which asking for 308 CPUs. Is there
other way to implement this requirement? Thanks advance
for your suggestion. Steven.


--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail 
gesendet. 

[slurm-dev] Re: set maximum CPU usage per user

[Benjamin Redling]

Hi, what are your AccountingStorage settings? Esp. AccountingStorageEnforce. 
Did limits work before, or is this a first try?
Regards, Benjamin

Am 19. Oktober 2016 22:14:27 MESZ, schrieb Steven Lo :
>
>
>By the way, we do have the following attribute set:
>
> PriorityType=priority/multifactor
>
>
>Thanks
>
>Steven.
>
>
>On 10/19/2016 12:55 PM, Steven Lo wrote:
>>
>>
>> Hi Chris,
>>
>> When we try the command as suggested, it said that nothing modified:
>>
>> [root@pauling ~]# sacctmgr modify account normal set Grpcpus=300
>>  Nothing modified
>>
>> Do you know if there is other method?
>>
>> Thanks
>>
>> Steven.
>>
>>
>> On 10/19/2016 07:25 AM, Christopher Benjamin Coffey wrote:
>>> Hi Steven,
>>>
>>> If you are trying to restrict the cpus for a group, I believe you 
>>> need to set the account value:
>>>
>>> sacctmgr modify account normal set Grpcpus=300
>>>
>>> Best,
>>> Chris
>>>
>>> —
>>> Christopher Coffey
>>> High-Performance Computing
>>> Northern Arizona University
>>> 928-523-1167
>>>
>>> On 10/18/16, 4:04 PM, "Steven Lo"  wrote:
>>>
>>>Hi,
>>>   We are trying to limit 300 CPU usage per user in our
>cluster.
>>>   We have tried:
>>>   sacctmgr modify qos normal set Grpcpus=300
>>>   and
>>>   sacctmgr modify user username set GrpCPUs=300
>>>Both seems to allow job to run which asking for 308
>CPUs.
>>>Is there other way to implement this requirement?
>>>Thanks advance for your suggestion.
>>>Steven.
>>>
>>

-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

[slurm-dev] Re: set maximum CPU usage per user

[Steven Lo]

Hi Chris,

When we try the command as suggested, it said that nothing modified:

[root@pauling ~]# sacctmgr modify account normal set Grpcpus=300
 Nothing modified

Do you know if there is other method?

Thanks

Steven.


On 10/19/2016 07:25 AM, Christopher Benjamin Coffey wrote:

Hi Steven,

If you are trying to restrict the cpus for a group, I believe you need to set 
the account value:

sacctmgr modify account normal set Grpcpus=300

Best,
Chris

—
Christopher Coffey
High-Performance Computing
Northern Arizona University
928-523-1167

On 10/18/16, 4:04 PM, "Steven Lo"  wrote:

 
 
 Hi,
 
 We are trying to limit 300 CPU usage per user in our cluster.
 
 We have tried:
 
 sacctmgr modify qos normal set Grpcpus=300
 
 and
 
 sacctmgr modify user username set GrpCPUs=300
 
 
 Both seems to allow job to run which asking for 308 CPUs.
 
 
 Is there other way to implement this requirement?
 
 
 Thanks advance for your suggestion.
 
 
 Steven.
 

[slurm-dev] Re: set maximum CPU usage per user

[Christopher Benjamin Coffey]

Hi Steven,

If you are trying to restrict the cpus for a group, I believe you need to set 
the account value:

sacctmgr modify account normal set Grpcpus=300

Best,
Chris

—
Christopher Coffey
High-Performance Computing
Northern Arizona University
928-523-1167

On 10/18/16, 4:04 PM, "Steven Lo"  wrote:



Hi,

We are trying to limit 300 CPU usage per user in our cluster.

We have tried:

sacctmgr modify qos normal set Grpcpus=300

and

sacctmgr modify user username set GrpCPUs=300


Both seems to allow job to run which asking for 308 CPUs.


Is there other way to implement this requirement?


Thanks advance for your suggestion.


Steven.