[slurm-dev] Re: RebootProgram - who uses it?

[Lachlan Musicman]

Yep, thanks Chris. I went with regular reboot and have now successfully used

scontrol reboot ASAP 

Very handy!

L.

--
"The antidote to apocalypticism is *apocalyptic civics*. Apocalyptic civics
is the insistence that we cannot ignore the truth, nor should we panic
about it. It is a shared consciousness that our institutions have failed
and our ecosystem is collapsing, yet we are still here — and we are
creative agents who can shape our destinies. Apocalyptic civics is the
conviction that the only way out is through, and the only way through is
together. "

*Greg Bloom* @greggish
https://twitter.com/greggish/status/873177525903609857

On 9 August 2017 at 15:02, Christopher Samuel  wrote:

>
> On 07/08/17 17:57, Aaron Knister wrote:
>
> > Good grief. "reboot" is a legacy tool?!?! I've about had enough of
> systemd.
>
> FWIW reboot is provided by the init system implementation (for instance
> on RHEL6 it's from upstart), and /sbin/reboot is only optional in the
> FHS. Only /sbin/shutdown is required by the FHS.
>
> http://www.pathname.com/fhs/2.2/fhs-3.14.html
>
> On proprietary UNIX versions reboot (not guaranteed to be in /sbin, it
> was /etc/reboot on Ultrix 4, /usr/sbin/reboot on Solaris) may not run
> shutdown scripts either (eg Solaris), you'd want to use shutdown for that.
>
> cheers,
> Chris
> --
>  Christopher SamuelSenior Systems Administrator
>  Melbourne Bioinformatics - The University of Melbourne
>  Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
>

[slurm-dev] Re: RebootProgram - who uses it?

[Christopher Samuel]

On 07/08/17 17:57, Aaron Knister wrote:

> Good grief. "reboot" is a legacy tool?!?! I've about had enough of systemd.

FWIW reboot is provided by the init system implementation (for instance
on RHEL6 it's from upstart), and /sbin/reboot is only optional in the
FHS. Only /sbin/shutdown is required by the FHS.

http://www.pathname.com/fhs/2.2/fhs-3.14.html

On proprietary UNIX versions reboot (not guaranteed to be in /sbin, it
was /etc/reboot on Ultrix 4, /usr/sbin/reboot on Solaris) may not run
shutdown scripts either (eg Solaris), you'd want to use shutdown for that.

cheers,
Chris
-- 
 Christopher SamuelSenior Systems Administrator
 Melbourne Bioinformatics - The University of Melbourne
 Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545

[slurm-dev] Re: RebootProgram - who uses it?

[Christopher Samuel]

On 07/08/17 14:08, Lachlan Musicman wrote:

> In slurm.conf, there is a RebootProgram - does this need to be a direct
> link to a bin or can it be a command?

We have:

RebootProgram = /sbin/reboot

Works for us.

cheers,
Chris
-- 
 Christopher SamuelSenior Systems Administrator
 Melbourne Bioinformatics - The University of Melbourne
 Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545

[slurm-dev] Re: Account preferation

[Manuel Rodríguez Pascual]

yes, you can use backfill and multifactor together. Sorry if that was
not clear on my previous mail. Multifactor is in charge of sorting the
job queue, and backifll is in charge of executing them



2017-08-08 16:04 GMT+02:00 Dennis Tants :
>
> Hello Manuel,
>
> thank you very much for your insight.
>
> I have read about FairShare too, but I am not sure if/how I can use it
> for this specific example.
> But I will consider it!
> Or, can anyone confirm FairShare should work in this case?
>
>
> Just for clarification:
> Can't I use backfill and multifactor priority together?
> First goes backfill, depending on the job length and then it is sorted
> by priority?
> Atleast thats were my first thoughts about it :D
>
> Best regards,
> Dennis
>
> Am 08.08.2017 um 14:53 schrieb Manuel Rodríguez Pascual:
>> Hi Dennis,
>>
>> If I undersand you correctly, what you need is to use the Multifactor Plugin
>> https://slurm.schedmd.com/priority_multifactor.html
>>
>> In particular, I guess this is relevant for your installation:
>>
>> Note: Computing the fair-share factor requires the installation and
>> operation of the Slurm Accounting Database to provide the assigned
>> shares and the consumed, computing resources described below.
>>
>> and
>>
>> PriorityDecayHalfLife   This determines the contribution of historical
>> usage on the composite usage value. The larger the number, the longer
>> past usage affects fair-share. If set to 0 no decay will be applied.
>> This is helpful if you want to enforce hard time limits per
>> association. If set to 0 PriorityUsageResetPeriod must be set to some
>> interval. The unit is a time string (i.e. min, hr:min:00,
>> days-hr:min:00, or days-hr). The default value is 7-0 (7 days).
>>
>>
>> Also, note that backfill and multifactor are two different things:
>> Backfill tries to run small jobs if this does not affect to other ones
>> waiting on the queue;  Multifactor determines the order of the queue.
>> They are set with different parameters in slurm.conf
>>
>> SchedulerType=sched/backfill
>> PriorityType=priority/multifactor
>>
>>
>> I am not a slurm expert so I may probably be wrong on some
>> information, but I hope this can serve you as a small help on your
>> problem.
>>
>> Best,
>>
>>
>> Manuel
>>
>> 2017-08-08 14:37 GMT+02:00 Dennis Tants :
>>> Hey guys,
>>>
>>> I was asked to implement a walltime of 24 hours in our cluster (we only
>>> have 16 nodes, so no need until now).
>>> Furthermore, I need to prefer single accounts based on compute time. The
>>> example I was given is further below.
>>>
>>> Btw., I'm upgrading to SLURM 17.02.4.
>>>
>>> Example:
>>> If the partition is full, we should start to prefer people who haven't
>>> used much computing time the last month.
>>>
>>> I don't know where I should start with this request to be honest and if
>>> it is even possible (walltime is not the problem).
>>>
>>> First of, I would need to also implement accounting I guess (also, no
>>> need for until now^^)
>>> But how to continue? I wanted to implement backfill instead of FIFO, but
>>> would I also need multifactor priority?
>>>
>>> Here are my thoughts of being able to achieve this:
>>>
>>> 1. Could I do something like:
>>> A. People are getting points every month (which they can't actually
>>> empty within one month)
>>> B. Compare the points (last month) of all people when a queue appears.
>>>
>>> 2. Or something like (same but time based):
>>> A. Account the whole computing time for two months (and then start to
>>> delete old entries)
>>> B. While queue, compare computing time (last month) of all waiting users?
>>>
>>> I hope I made myself clear and that you can help me. Every hint is
>>> appreciated.
>>>
>>> Best regards,
>>> Dennis
>>>
>>> --
>>> Dennis Tants
>>> Auszubildender: Fachinformatiker für Systemintegration
>>>
>>> ZARM - Zentrum für angewandte Raumfahrttechnologie und Mikrogravitation
>>> ZARM - Center of Applied Space Technology and Microgravity
>>>
>>> Universität Bremen
>>> Am Fallturm
>>> 28359 Bremen, Germany
>>>
>>> Telefon: 0421 218 57940
>>> E-Mail: ta...@zarm.uni-bremen.de
>>>
>>> www.zarm.uni-bremen.de
>>>
>
> --
> Dennis Tants
> Auszubildender: Fachinformatiker für Systemintegration
>
> ZARM - Zentrum für angewandte Raumfahrttechnologie und Mikrogravitation
> ZARM - Center of Applied Space Technology and Microgravity
>
> Universität Bremen
> Am Fallturm
> 28359 Bremen, Germany
>
> Telefon: 0421 218 57940
> E-Mail: ta...@zarm.uni-bremen.de
>
> www.zarm.uni-bremen.de
>

[slurm-dev] Re: Account preferation

[Dennis Tants]

Hello Manuel,

thank you very much for your insight.

I have read about FairShare too, but I am not sure if/how I can use it
for this specific example.
But I will consider it!
Or, can anyone confirm FairShare should work in this case?


Just for clarification:
Can't I use backfill and multifactor priority together?
First goes backfill, depending on the job length and then it is sorted
by priority?
Atleast thats were my first thoughts about it :D

Best regards,
Dennis

Am 08.08.2017 um 14:53 schrieb Manuel Rodríguez Pascual:
> Hi Dennis,
>
> If I undersand you correctly, what you need is to use the Multifactor Plugin
> https://slurm.schedmd.com/priority_multifactor.html
>
> In particular, I guess this is relevant for your installation:
>
> Note: Computing the fair-share factor requires the installation and
> operation of the Slurm Accounting Database to provide the assigned
> shares and the consumed, computing resources described below.
>
> and
>
> PriorityDecayHalfLife   This determines the contribution of historical
> usage on the composite usage value. The larger the number, the longer
> past usage affects fair-share. If set to 0 no decay will be applied.
> This is helpful if you want to enforce hard time limits per
> association. If set to 0 PriorityUsageResetPeriod must be set to some
> interval. The unit is a time string (i.e. min, hr:min:00,
> days-hr:min:00, or days-hr). The default value is 7-0 (7 days).
>
>
> Also, note that backfill and multifactor are two different things:
> Backfill tries to run small jobs if this does not affect to other ones
> waiting on the queue;  Multifactor determines the order of the queue.
> They are set with different parameters in slurm.conf
>
> SchedulerType=sched/backfill
> PriorityType=priority/multifactor
>
>
> I am not a slurm expert so I may probably be wrong on some
> information, but I hope this can serve you as a small help on your
> problem.
>
> Best,
>
>
> Manuel
>
> 2017-08-08 14:37 GMT+02:00 Dennis Tants :
>> Hey guys,
>>
>> I was asked to implement a walltime of 24 hours in our cluster (we only
>> have 16 nodes, so no need until now).
>> Furthermore, I need to prefer single accounts based on compute time. The
>> example I was given is further below.
>>
>> Btw., I'm upgrading to SLURM 17.02.4.
>>
>> Example:
>> If the partition is full, we should start to prefer people who haven't
>> used much computing time the last month.
>>
>> I don't know where I should start with this request to be honest and if
>> it is even possible (walltime is not the problem).
>>
>> First of, I would need to also implement accounting I guess (also, no
>> need for until now^^)
>> But how to continue? I wanted to implement backfill instead of FIFO, but
>> would I also need multifactor priority?
>>
>> Here are my thoughts of being able to achieve this:
>>
>> 1. Could I do something like:
>> A. People are getting points every month (which they can't actually
>> empty within one month)
>> B. Compare the points (last month) of all people when a queue appears.
>>
>> 2. Or something like (same but time based):
>> A. Account the whole computing time for two months (and then start to
>> delete old entries)
>> B. While queue, compare computing time (last month) of all waiting users?
>>
>> I hope I made myself clear and that you can help me. Every hint is
>> appreciated.
>>
>> Best regards,
>> Dennis
>>
>> --
>> Dennis Tants
>> Auszubildender: Fachinformatiker für Systemintegration
>>
>> ZARM - Zentrum für angewandte Raumfahrttechnologie und Mikrogravitation
>> ZARM - Center of Applied Space Technology and Microgravity
>>
>> Universität Bremen
>> Am Fallturm
>> 28359 Bremen, Germany
>>
>> Telefon: 0421 218 57940
>> E-Mail: ta...@zarm.uni-bremen.de
>>
>> www.zarm.uni-bremen.de
>>

-- 
Dennis Tants
Auszubildender: Fachinformatiker für Systemintegration

ZARM - Zentrum für angewandte Raumfahrttechnologie und Mikrogravitation
ZARM - Center of Applied Space Technology and Microgravity

Universität Bremen
Am Fallturm
28359 Bremen, Germany

Telefon: 0421 218 57940
E-Mail: ta...@zarm.uni-bremen.de

www.zarm.uni-bremen.de

[slurm-dev] Re: Account preferation

[Manuel Rodríguez Pascual]

Hi Dennis,

If I undersand you correctly, what you need is to use the Multifactor Plugin
https://slurm.schedmd.com/priority_multifactor.html

In particular, I guess this is relevant for your installation:

Note: Computing the fair-share factor requires the installation and
operation of the Slurm Accounting Database to provide the assigned
shares and the consumed, computing resources described below.

and

PriorityDecayHalfLife   This determines the contribution of historical
usage on the composite usage value. The larger the number, the longer
past usage affects fair-share. If set to 0 no decay will be applied.
This is helpful if you want to enforce hard time limits per
association. If set to 0 PriorityUsageResetPeriod must be set to some
interval. The unit is a time string (i.e. min, hr:min:00,
days-hr:min:00, or days-hr). The default value is 7-0 (7 days).


Also, note that backfill and multifactor are two different things:
Backfill tries to run small jobs if this does not affect to other ones
waiting on the queue;  Multifactor determines the order of the queue.
They are set with different parameters in slurm.conf

SchedulerType=sched/backfill
PriorityType=priority/multifactor


I am not a slurm expert so I may probably be wrong on some
information, but I hope this can serve you as a small help on your
problem.

Best,


Manuel

2017-08-08 14:37 GMT+02:00 Dennis Tants :
>
> Hey guys,
>
> I was asked to implement a walltime of 24 hours in our cluster (we only
> have 16 nodes, so no need until now).
> Furthermore, I need to prefer single accounts based on compute time. The
> example I was given is further below.
>
> Btw., I'm upgrading to SLURM 17.02.4.
>
> Example:
> If the partition is full, we should start to prefer people who haven't
> used much computing time the last month.
>
> I don't know where I should start with this request to be honest and if
> it is even possible (walltime is not the problem).
>
> First of, I would need to also implement accounting I guess (also, no
> need for until now^^)
> But how to continue? I wanted to implement backfill instead of FIFO, but
> would I also need multifactor priority?
>
> Here are my thoughts of being able to achieve this:
>
> 1. Could I do something like:
> A. People are getting points every month (which they can't actually
> empty within one month)
> B. Compare the points (last month) of all people when a queue appears.
>
> 2. Or something like (same but time based):
> A. Account the whole computing time for two months (and then start to
> delete old entries)
> B. While queue, compare computing time (last month) of all waiting users?
>
> I hope I made myself clear and that you can help me. Every hint is
> appreciated.
>
> Best regards,
> Dennis
>
> --
> Dennis Tants
> Auszubildender: Fachinformatiker für Systemintegration
>
> ZARM - Zentrum für angewandte Raumfahrttechnologie und Mikrogravitation
> ZARM - Center of Applied Space Technology and Microgravity
>
> Universität Bremen
> Am Fallturm
> 28359 Bremen, Germany
>
> Telefon: 0421 218 57940
> E-Mail: ta...@zarm.uni-bremen.de
>
> www.zarm.uni-bremen.de
>