Re: [slurm-users] Create users

[Andy Georges]

Hi,

> On 3 Oct 2018, at 16:51, Andy Georges  wrote:
> 
> Hi all,
> 
>> On 15 Sep 2018, at 14:47, Chris Samuel  wrote:
>> 
>> On Thursday, 13 September 2018 3:10:19 AM AEST Paul Edmon wrote:
>> 
>>> Another way would be to make all your Linux users and then map that in to
>>> Slurm using sacctmgr.
>> 
>> At ${JOB} and ${JOB-1} we've wired user creation in Slurm into our online 
>> user 
>> management systems (both Django based & independently created), so when 
>> people 
>> are added/modified/deleted then it runs sacctmgr to keep everything in step.
> 
> We basically have something similar, which boils down to a few set 
> comparisons, then creating and executing the right actions for each set.
> 
> For code, see 
> https://github.com/hpcugent/vsc-administration/blob/master/bin/sync_slurm_acct.py
>  and 
> https://github.com/hpcugent/vsc-administration/blob/master/lib/vsc/administration/slurm/sync.py

I was pointed out this repo was still private, it is public now. Sorry for the 
inconvenience of sending you to a 404 page.

Kind regards,
— Andy 

Re: [slurm-users] Create users

[Andy Georges]

Hi all,

> On 15 Sep 2018, at 14:47, Chris Samuel  wrote:
> 
> On Thursday, 13 September 2018 3:10:19 AM AEST Paul Edmon wrote:
> 
>> Another way would be to make all your Linux users and then map that in to
>> Slurm using sacctmgr.
> 
> At ${JOB} and ${JOB-1} we've wired user creation in Slurm into our online 
> user 
> management systems (both Django based & independently created), so when 
> people 
> are added/modified/deleted then it runs sacctmgr to keep everything in step.

We basically have something similar, which boils down to a few set comparisons, 
then creating and executing the right actions for each set.

For code, see 
https://github.com/hpcugent/vsc-administration/blob/master/bin/sync_slurm_acct.py
 and 
https://github.com/hpcugent/vsc-administration/blob/master/lib/vsc/administration/slurm/sync.py

Kind regards,
— Andy

Re: [slurm-users] Create users

[John Hearns]

Loris said:

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the simplest
option being to use pam_script.

When in Denmark this year (hello Ole!) I looked at pam_mkhomedir quite closely.
The object was to automatically create home directories, not for Slurm
users. What I found was that pam_mkhomedir uses a precompiled
executable, rather than triggering a script which you would expect. So
it is difficult to customise.
there is an Oddjob based version   https://access.redhat.com/discussions/903523
However in the past I have found the Oddjob version unreliable on an
HPC cluster, so used the standard pam_mkhomedir

pam_script does look to be the answer here. And I do admit to not
having heard of it.



On Thu, 13 Sep 2018 at 09:11, Loris Bennett  wrote:
>
> Hi Paul,
>
> I'd be interested in seeing your Lua submit script, if you're willing to
> share.
>
> Until now I had thought that the most elegant way of setting up Slurm
> users would be via a PAM module analogous to pam_mkhomedir, the simplest
> option being to use pam_script.
>
> However, given that we do have users who somehow never get round to
> submitting a job before their HPC access expires, setting up the Slurm
> account when the first job is submitted seems quite appealing.
>
> Cheers,
>
> Loris
>
> Paul Edmon  writes:
>
> > So useradd is adding a Linux user, which sacctmgr creates a Slurm user.
> >
> > What we do is that we run AD for our Linux user managment. We then in our 
> > job submit lua script look to see if the user has an account in slurm and 
> > if they don't we create it.
> >
> > Another way would be to make all your Linux users and then map that in to 
> > Slurm using sacctmgr.
> >
> > It really depends on if your Slurm users are a subset of your regular users 
> > or not.
> >
> > -Paul Edmon-
> >
> > On 9/12/2018 12:21 PM, Andre Torres wrote:
> >
> >  Hi all,
> >
> >  I’m new to slurm and I’m confused regarding user creation. I have an 
> > installation with 1 login node and 5 compute nodes. If I create a user 
> > across all the nodes with the same uid and gid I can execute jobs but
> >  I can’t understand the difference between user creation with “useradd” 
> > command and the “sacctmgr” command
> >
> >  sacctmgr create account name=test
> >
> >  sacctmgr create user jdoe account=test
> >
> >  Also, is there anyway of creating a user at login node and replicate to 
> > the compute nodes ? What is the best practice for user creation ?
> >
> >  Thanks in advance
> >
> >
> --
> Dr. Loris Bennett (Mr.)
> ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
>

Re: [slurm-users] Create users

[Chris Samuel]

On Thursday, 13 September 2018 3:10:19 AM AEST Paul Edmon wrote:

> Another way would be to make all your Linux users and then map that in to
> Slurm using sacctmgr.

At ${JOB} and ${JOB-1} we've wired user creation in Slurm into our online user 
management systems (both Django based & independently created), so when people 
are added/modified/deleted then it runs sacctmgr to keep everything in step.

-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

Re: [slurm-users] Create users

[Paul Edmon]

Users can control that:

https://slurm.schedmd.com/sbatch.html

-Paul Edmon-


On 09/13/2018 11:10 AM, Ariel Balter wrote:


Does anyone know how to change email settings?


On 9/13/2018 7:59 AM, Damien François wrote:
Just to add my 2c to the discussion: at our site, we use a utility we 
wrote [1] that monitors our LDAP and triggers Ansible playbooks upon 
addition or modification in the list of users. We have playbooks to 
setup the different directories for the user in the many filesystems, 
along with the respective quotas, setup SSH keys, and register to 
Slurm. To register to Slurm, we use a Slurm user module for Ansible [2].


damien

[1] https://github.com/damienfrancois/slufl2
[2] https://github.com/dylex/ansible-hpc/blob/master/slurm.py



On 13 Sep 2018, at 16:41, Paul Edmon > wrote:


So the Lua script I posted only does it for people who submit to the 
cluster.  To do it for all users it should just be a simple bash 
script to do that, I don't have one put together though.


-Paul Edmon-


On 09/13/2018 10:29 AM, Eric F. Alemany wrote:

Hi Paul

You said

“Another way would be to make all your Linux users and then map 
that in to Slurm using sacctmgr.”


I am curious to know how you do that.

Thank you

Best
Eric

._

*
*Eric F.  Alemany*
*
/System Administrator for Research/

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969 No Texting
Fax:1-650-723-7382 

On Sep 13, 2018, at 01:09, Loris Bennett 
mailto:loris.benn...@fu-berlin.de>> wrote:



Hi Paul,

I'd be interested in seeing your Lua submit script, if you're 
willing to

share.

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the 
simplest

option being to use pam_script.

However, given that we do have users who somehow never get round to
submitting a job before their HPC access expires, setting up the Slurm
account when the first job is submitted seems quite appealing.

Cheers,

Loris

Paul Edmon > writes:


So useradd is adding a Linux user, which sacctmgr creates a Slurm 
user.


What we do is that we run AD for our Linux user managment. We 
then in our job submit lua script look to see if the user has an 
account in slurm and if they don't we create it.


Another way would be to make all your Linux users and then map 
that in to Slurm using sacctmgr.


It really depends on if your Slurm users are a subset of your 
regular users or not.


-Paul Edmon-

On 9/12/2018 12:21 PM, Andre Torres wrote:

Hi all,

I’m new to slurm and I’m confused regarding user creation. I have 
an installation with 1 login node and 5 compute nodes. If I 
create a user across all the nodes with the same uid and gid I 
can execute jobs but
I can’t understand the difference between user creation with 
“useradd” command and the “sacctmgr” command


sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and 
replicate to the compute nodes ? What is the best practice for 
user creation ?


Thanks in advance



--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email 
loris.benn...@fu-berlin.de 









Ariel Balter
ar...@arielbalter.com
ariel.bal...@gmail.com
509-713-0087
http://arielbalter.com

Re: [slurm-users] Create users

[Ariel Balter]

Does anyone know how to change email settings?


On 9/13/2018 7:59 AM, Damien François wrote:
Just to add my 2c to the discussion: at our site, we use a utility we 
wrote [1] that monitors our LDAP and triggers Ansible playbooks upon 
addition or modification in the list of users. We have playbooks to 
setup the different directories for the user in the many filesystems, 
along with the respective quotas, setup SSH keys, and register to 
Slurm. To register to Slurm, we use a Slurm user module for Ansible [2].


damien

[1] https://github.com/damienfrancois/slufl2
[2] https://github.com/dylex/ansible-hpc/blob/master/slurm.py



On 13 Sep 2018, at 16:41, Paul Edmon > wrote:


So the Lua script I posted only does it for people who submit to the 
cluster.  To do it for all users it should just be a simple bash 
script to do that, I don't have one put together though.


-Paul Edmon-


On 09/13/2018 10:29 AM, Eric F. Alemany wrote:

Hi Paul

You said

“Another way would be to make all your Linux users and then map that 
in to Slurm using sacctmgr.”


I am curious to know how you do that.

Thank you

Best
Eric

._

*
*Eric F.  Alemany*
*
/System Administrator for Research/

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969 No Texting
Fax:1-650-723-7382 

On Sep 13, 2018, at 01:09, Loris Bennett > wrote:



Hi Paul,

I'd be interested in seeing your Lua submit script, if you're 
willing to

share.

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the 
simplest

option being to use pam_script.

However, given that we do have users who somehow never get round to
submitting a job before their HPC access expires, setting up the Slurm
account when the first job is submitted seems quite appealing.

Cheers,

Loris

Paul Edmon mailto:ped...@cfa.harvard.edu>> 
writes:


So useradd is adding a Linux user, which sacctmgr creates a Slurm 
user.


What we do is that we run AD for our Linux user managment. We then 
in our job submit lua script look to see if the user has an 
account in slurm and if they don't we create it.


Another way would be to make all your Linux users and then map 
that in to Slurm using sacctmgr.


It really depends on if your Slurm users are a subset of your 
regular users or not.


-Paul Edmon-

On 9/12/2018 12:21 PM, Andre Torres wrote:

Hi all,

I’m new to slurm and I’m confused regarding user creation. I have 
an installation with 1 login node and 5 compute nodes. If I create 
a user across all the nodes with the same uid and gid I can 
execute jobs but
I can’t understand the difference between user creation with 
“useradd” command and the “sacctmgr” command


sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and 
replicate to the compute nodes ? What is the best practice for 
user creation ?


Thanks in advance



--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email 
loris.benn...@fu-berlin.de 









Ariel Balter
ar...@arielbalter.com
ariel.bal...@gmail.com
509-713-0087
http://arielbalter.com

Re: [slurm-users] Create users

[Paul Edmon]

This is true.  I've seen this happen before in our set up if sacctmgr 
happens to wedge.  It doesn't happen very frequently though and doesn't 
impact us that much.  We've been running in this fashion for years now 
with only very infrequent degradation. We do about 80,000 jobs per day.  
Plus we only add them if they don't already exist so the impact is only 
when new users appear.


-Paul Edmon-


On 09/13/2018 10:48 AM, Douglas Jacobsen wrote:
At one point in time we would also use the job_submit.lua to add 
users, however, I cannot recommend it in general since job_submit runs 
while locks are held within slurmcltd, which could have dramatic 
performance or even functionality impacts if there are delays in 
adding the user.


Doug Jacobsen, Ph.D.
NERSC Computer Systems Engineer
Acting Group Lead, Computational Systems Group
National Energy Research Scientific Computing Center 


dmjacob...@lbl.gov 

- __o
-- _ '\<,_
--(_)/  (_)__



On Thu, Sep 13, 2018 at 7:40 AM Eric F. Alemany > wrote:


Hi Paul

You said

“Another way would be to make all your Linux users and then map
that in to Slurm using sacctmgr.”

I am curious to know how you do that.

Thank you

Best
Eric


._

*
*Eric F. Alemany*
*
/System Administrator for Research/

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969   No Texting
Fax:1-650-723-7382 

On Sep 13, 2018, at 01:09, Loris Bennett
mailto:loris.benn...@fu-berlin.de>>
wrote:


Hi Paul,

I'd be interested in seeing your Lua submit script, if you're
willing to
share.

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the
simplest
option being to use pam_script.

However, given that we do have users who somehow never get round to
submitting a job before their HPC access expires, setting up the
Slurm
account when the first job is submitted seems quite appealing.

Cheers,

Loris

Paul Edmon mailto:ped...@cfa.harvard.edu>> writes:


So useradd is adding a Linux user, which sacctmgr creates a
Slurm user.

What we do is that we run AD for our Linux user managment. We
then in our job submit lua script look to see if the user has an
account in slurm and if they don't we create it.

Another way would be to make all your Linux users and then map
that in to Slurm using sacctmgr.

It really depends on if your Slurm users are a subset of your
regular users or not.

-Paul Edmon-

On 9/12/2018 12:21 PM, Andre Torres wrote:

Hi all,

I’m new to slurm and I’m confused regarding user creation. I
have an installation with 1 login node and 5 compute nodes. If I
create a user across all the nodes with the same uid and gid I
can execute jobs but
I can’t understand the difference between user creation with
“useradd” command and the “sacctmgr” command

sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and
replicate to the compute nodes ? What is the best practice for
user creation ?

Thanks in advance


-- 
Dr. Loris Bennett (Mr.)

ZEDAT, Freie Universität Berlin Email
loris.benn...@fu-berlin.de 

Re: [slurm-users] Create users

[Damien François]

Just to add my 2c to the discussion: at our site, we use a utility we wrote [1] 
that monitors our LDAP and triggers Ansible playbooks upon addition or 
modification in the list of users. We have playbooks to setup the different 
directories for the user in the many filesystems, along with the respective 
quotas, setup SSH keys, and register to Slurm. To register to Slurm, we use a 
Slurm user module for Ansible [2].

damien

[1] https://github.com/damienfrancois/slufl2
[2] https://github.com/dylex/ansible-hpc/blob/master/slurm.py



> On 13 Sep 2018, at 16:41, Paul Edmon  wrote:
> 
> So the Lua script I posted only does it for people who submit to the cluster. 
>  To do it for all users it should just be a simple bash script to do that, I 
> don't have one put together though.
> 
> -Paul Edmon-
> 
> On 09/13/2018 10:29 AM, Eric F. Alemany wrote:
>> Hi Paul
>> 
>> You said 
>> 
>> “Another way would be to make all your Linux users and then map that in to 
>> Slurm using sacctmgr.”
>> 
>> I am curious to know how you do that.
>> 
>> Thank you
>> 
>> Best
>> Eric 
>> 
>> ._
>> 
>> Eric F.  Alemany
>> System Administrator for Research
>> 
>> Division of Radiation & Cancer  Biology
>> Department of Radiation Oncology
>> 
>> Stanford University School of Medicine
>> Stanford, California 94305
>> 
>> Tel:1-650-498-7969   No Texting
>> Fax:1-650-723-7382 
>> 
>> On Sep 13, 2018, at 01:09, Loris Bennett > > wrote:
>> 
>>> Hi Paul,
>>> 
>>> I'd be interested in seeing your Lua submit script, if you're willing to
>>> share.
>>> 
>>> Until now I had thought that the most elegant way of setting up Slurm
>>> users would be via a PAM module analogous to pam_mkhomedir, the simplest
>>> option being to use pam_script.
>>> 
>>> However, given that we do have users who somehow never get round to
>>> submitting a job before their HPC access expires, setting up the Slurm
>>> account when the first job is submitted seems quite appealing.
>>> 
>>> Cheers,
>>> 
>>> Loris
>>> 
>>> Paul Edmon mailto:ped...@cfa.harvard.edu>> writes:
>>> 
 So useradd is adding a Linux user, which sacctmgr creates a Slurm user.
 
 What we do is that we run AD for our Linux user managment. We then in our 
 job submit lua script look to see if the user has an account in slurm and 
 if they don't we create it.
 
 Another way would be to make all your Linux users and then map that in to 
 Slurm using sacctmgr.
 
 It really depends on if your Slurm users are a subset of your regular 
 users or not.
 
 -Paul Edmon-
 
 On 9/12/2018 12:21 PM, Andre Torres wrote:
 
 Hi all,
 
 I’m new to slurm and I’m confused regarding user creation. I have an 
 installation with 1 login node and 5 compute nodes. If I create a user 
 across all the nodes with the same uid and gid I can execute jobs but
 I can’t understand the difference between user creation with “useradd” 
 command and the “sacctmgr” command 
 
 sacctmgr create account name=test
 
 sacctmgr create user jdoe account=test
 
 Also, is there anyway of creating a user at login node and replicate to 
 the compute nodes ? What is the best practice for user creation ?
 
 Thanks in advance
 
 
>>> -- 
>>> Dr. Loris Bennett (Mr.)
>>> ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de 
>>> 
>>> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: [slurm-users] Create users

[Douglas Jacobsen]

At one point in time we would also use the job_submit.lua to add users,
however, I cannot recommend it in general since job_submit runs while locks
are held within slurmcltd, which could have dramatic performance or even
functionality impacts if there are delays in adding the user.

Doug Jacobsen, Ph.D.
NERSC Computer Systems Engineer
Acting Group Lead, Computational Systems Group
National Energy Research Scientific Computing Center 
dmjacob...@lbl.gov

- __o
-- _ '\<,_
--(_)/  (_)__



On Thu, Sep 13, 2018 at 7:40 AM Eric F. Alemany 
wrote:

> Hi Paul
>
> You said
>
> “Another way would be to make all your Linux users and then map that in
> to Slurm using sacctmgr.”
>
> I am curious to know how you do that.
>
> Thank you
>
> Best
> Eric
>
>
> ._
>
> * Eric F.  Alemany *
> *System Administrator for Research*
>
> Division of Radiation & Cancer  Biology
> Department of Radiation Oncology
>
> Stanford University School of Medicine
> Stanford, California 94305
>
> Tel:1-650-498-7969  No Texting
> Fax:1-650-723-7382
>
> On Sep 13, 2018, at 01:09, Loris Bennett 
> wrote:
>
> Hi Paul,
>
> I'd be interested in seeing your Lua submit script, if you're willing to
> share.
>
> Until now I had thought that the most elegant way of setting up Slurm
> users would be via a PAM module analogous to pam_mkhomedir, the simplest
> option being to use pam_script.
>
> However, given that we do have users who somehow never get round to
> submitting a job before their HPC access expires, setting up the Slurm
> account when the first job is submitted seems quite appealing.
>
> Cheers,
>
> Loris
>
> Paul Edmon  writes:
>
> So useradd is adding a Linux user, which sacctmgr creates a Slurm user.
>
>
> What we do is that we run AD for our Linux user managment. We then in our
> job submit lua script look to see if the user has an account in slurm and
> if they don't we create it.
>
>
> Another way would be to make all your Linux users and then map that in to
> Slurm using sacctmgr.
>
>
> It really depends on if your Slurm users are a subset of your regular
> users or not.
>
>
> -Paul Edmon-
>
>
> On 9/12/2018 12:21 PM, Andre Torres wrote:
>
>
> Hi all,
>
>
> I’m new to slurm and I’m confused regarding user creation. I have an
> installation with 1 login node and 5 compute nodes. If I create a user
> across all the nodes with the same uid and gid I can execute jobs but
>
> I can’t understand the difference between user creation with “useradd”
> command and the “sacctmgr” command
>
>
> sacctmgr create account name=test
>
>
> sacctmgr create user jdoe account=test
>
>
> Also, is there anyway of creating a user at login node and replicate to
> the compute nodes ? What is the best practice for user creation ?
>
>
> Thanks in advance
>
>
>
> --
> Dr. Loris Bennett (Mr.)
> ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
>
>

Re: [slurm-users] Create users

[Eric F. Alemany]

Thank you Paul!

._

Eric F.  Alemany
System Administrator for Research

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969  No Texting
Fax:1-650-723-7382

On Sep 13, 2018, at 07:42, Paul Edmon 
mailto:ped...@cfa.harvard.edu>> wrote:


So the Lua script I posted only does it for people who submit to the cluster.  
To do it for all users it should just be a simple bash script to do that, I 
don't have one put together though.

-Paul Edmon-

On 09/13/2018 10:29 AM, Eric F. Alemany wrote:
Hi Paul

You said

“Another way would be to make all your Linux users and then map that in to 
Slurm using sacctmgr.”

I am curious to know how you do that.

Thank you

Best
Eric

._

Eric F.  Alemany
System Administrator for Research

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969  No Texting
Fax:1-650-723-7382

On Sep 13, 2018, at 01:09, Loris Bennett 
mailto:loris.benn...@fu-berlin.de>> wrote:

Hi Paul,

I'd be interested in seeing your Lua submit script, if you're willing to
share.

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the simplest
option being to use pam_script.

However, given that we do have users who somehow never get round to
submitting a job before their HPC access expires, setting up the Slurm
account when the first job is submitted seems quite appealing.

Cheers,

Loris

Paul Edmon mailto:ped...@cfa.harvard.edu>> writes:

So useradd is adding a Linux user, which sacctmgr creates a Slurm user.

What we do is that we run AD for our Linux user managment. We then in our job 
submit lua script look to see if the user has an account in slurm and if they 
don't we create it.

Another way would be to make all your Linux users and then map that in to Slurm 
using sacctmgr.

It really depends on if your Slurm users are a subset of your regular users or 
not.

-Paul Edmon-

On 9/12/2018 12:21 PM, Andre Torres wrote:

Hi all,

I’m new to slurm and I’m confused regarding user creation. I have an 
installation with 1 login node and 5 compute nodes. If I create a user across 
all the nodes with the same uid and gid I can execute jobs but
I can’t understand the difference between user creation with “useradd” command 
and the “sacctmgr” command

sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and replicate to the 
compute nodes ? What is the best practice for user creation ?

Thanks in advance


--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email 
loris.benn...@fu-berlin.de

Re: [slurm-users] Create users

[Paul Edmon]

So the Lua script I posted only does it for people who submit to the 
cluster.  To do it for all users it should just be a simple bash script 
to do that, I don't have one put together though.


-Paul Edmon-


On 09/13/2018 10:29 AM, Eric F. Alemany wrote:

Hi Paul

You said

“Another way would be to make all your Linux users and then map that 
in to Slurm using sacctmgr.”


I am curious to know how you do that.

Thank you

Best
Eric

._

*
*Eric F.  Alemany*
*
/System Administrator for Research/

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969   No Texting
Fax:1-650-723-7382 

On Sep 13, 2018, at 01:09, Loris Bennett > wrote:



Hi Paul,

I'd be interested in seeing your Lua submit script, if you're willing to
share.

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the simplest
option being to use pam_script.

However, given that we do have users who somehow never get round to
submitting a job before their HPC access expires, setting up the Slurm
account when the first job is submitted seems quite appealing.

Cheers,

Loris

Paul Edmon mailto:ped...@cfa.harvard.edu>> 
writes:



So useradd is adding a Linux user, which sacctmgr creates a Slurm user.

What we do is that we run AD for our Linux user managment. We then 
in our job submit lua script look to see if the user has an account 
in slurm and if they don't we create it.


Another way would be to make all your Linux users and then map that 
in to Slurm using sacctmgr.


It really depends on if your Slurm users are a subset of your 
regular users or not.


-Paul Edmon-

On 9/12/2018 12:21 PM, Andre Torres wrote:

Hi all,

I’m new to slurm and I’m confused regarding user creation. I have an 
installation with 1 login node and 5 compute nodes. If I create a 
user across all the nodes with the same uid and gid I can execute 
jobs but
I can’t understand the difference between user creation with 
“useradd” command and the “sacctmgr” command


sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and replicate 
to the compute nodes ? What is the best practice for user creation ?


Thanks in advance



--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email 
loris.benn...@fu-berlin.de 

Re: [slurm-users] Create users

[Eric F. Alemany]

Hi Paul

You said

“Another way would be to make all your Linux users and then map that in to 
Slurm using sacctmgr.”

I am curious to know how you do that.

Thank you

Best
Eric

._

Eric F.  Alemany
System Administrator for Research

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969  No Texting
Fax:1-650-723-7382

On Sep 13, 2018, at 01:09, Loris Bennett 
mailto:loris.benn...@fu-berlin.de>> wrote:

Hi Paul,

I'd be interested in seeing your Lua submit script, if you're willing to
share.

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the simplest
option being to use pam_script.

However, given that we do have users who somehow never get round to
submitting a job before their HPC access expires, setting up the Slurm
account when the first job is submitted seems quite appealing.

Cheers,

Loris

Paul Edmon mailto:ped...@cfa.harvard.edu>> writes:

So useradd is adding a Linux user, which sacctmgr creates a Slurm user.

What we do is that we run AD for our Linux user managment. We then in our job 
submit lua script look to see if the user has an account in slurm and if they 
don't we create it.

Another way would be to make all your Linux users and then map that in to Slurm 
using sacctmgr.

It really depends on if your Slurm users are a subset of your regular users or 
not.

-Paul Edmon-

On 9/12/2018 12:21 PM, Andre Torres wrote:

Hi all,

I’m new to slurm and I’m confused regarding user creation. I have an 
installation with 1 login node and 5 compute nodes. If I create a user across 
all the nodes with the same uid and gid I can execute jobs but
I can’t understand the difference between user creation with “useradd” command 
and the “sacctmgr” command

sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and replicate to the 
compute nodes ? What is the best practice for user creation ?

Thanks in advance


--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email 
loris.benn...@fu-berlin.de

Re: [slurm-users] Create users

[Paul Edmon]

Sure. Here is our lua script.

-Paul Edmon-


On 09/13/2018 07:28 AM, Andre Torres wrote:

That's interesting using AD to maintain uid consistency across all the nodes. 
Like Loris, I'm also interested in your Lua script.

-
André

On 13/09/2018, 11:42, "slurm-users on behalf of Loris Bennett" 
 wrote:

 Hi Paul,
 
 I'd be interested in seeing your Lua submit script, if you're willing to

 share.
 
 Until now I had thought that the most elegant way of setting up Slurm

 users would be via a PAM module analogous to pam_mkhomedir, the simplest
 option being to use pam_script.
 
 However, given that we do have users who somehow never get round to

 submitting a job before their HPC access expires, setting up the Slurm
 account when the first job is submitted seems quite appealing.
 
 Cheers,
 
 Loris
 
 Paul Edmon  writes:
 
 > So useradd is adding a Linux user, which sacctmgr creates a Slurm user.

 >
 > What we do is that we run AD for our Linux user managment. We then in 
our job submit lua script look to see if the user has an account in slurm and if 
they don't we create it.
 >
 > Another way would be to make all your Linux users and then map that in 
to Slurm using sacctmgr.
 >
 > It really depends on if your Slurm users are a subset of your regular 
users or not.
 >
 > -Paul Edmon-
 >
 > On 9/12/2018 12:21 PM, Andre Torres wrote:
 >
 >  Hi all,
 >
 >  I’m new to slurm and I’m confused regarding user creation. I have an 
installation with 1 login node and 5 compute nodes. If I create a user across all 
the nodes with the same uid and gid I can execute jobs but
 >  I can’t understand the difference between user creation with “useradd” 
command and the “sacctmgr” command
 >
 >  sacctmgr create account name=test
 >
 >  sacctmgr create user jdoe account=test
 >
 >  Also, is there anyway of creating a user at login node and replicate to 
the compute nodes ? What is the best practice for user creation ?
 >
 >  Thanks in advance
 >
 >
 --
 Dr. Loris Bennett (Mr.)
 ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
 
 






-- Cache association existance for this many seconds, with the caveat
-- that if you manually remove a user's association, it will take SLURM
-- up to this long to recognize that it's gone.
local ASSOC_CACHE_LIFETIME = 24 * 60 * 60

-- Cache the start time of a downtime (if any) for this many minutes, so we
-- don't have to call 'scontrol show reservations' for every job submission.
local DOWNTIME_RESERVATION_CACHE = 30
-- Demand that jobs end this many minutes before a scheduled downtime.
local SUGGESTED_JOB_END_BEFORE_DOWNTIME = 60

local posix = require "posix"

function in_group(user, group)
	local group = posix.getgroup(group)

	local i = 0
	while group[i] do
		if group[i] == user then
			return true
		end
		i = i + 1
	end

	return false
end

-- Compatibility: Lua-5.1
-- Got this from: http://lua-users.org/wiki/SplitJoin
function split(str, pat)
   local t = {}  -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pat
   local last_end = 1
   local s, e, cap = str:find(fpat, 1)
   while s do
  if s ~= 1 or cap ~= "" then
	 table.insert(t,cap)
  end
  last_end = e+1
  s, e, cap = str:find(fpat, last_end)
   end
   if last_end <= #str then
  cap = str:sub(last_end)
  table.insert(t, cap)
   end
   return t
end

local existing_assocs = {}
local existing_assocs_last_expired = os.time()
function assoc_exists(user, account)
	-- Expire the cache if it's old.
	if existing_assocs_last_expired + ASSOC_CACHE_LIFETIME < os.time() then
		existing_assocs = {}
		existing_assocs_last_expired = os.time()
	end

	if existing_assocs[user .. "\n" .. account] then
		return true
	end

	-- Unfortunately, filehandles returned by io.popen() don't have a
	-- way to return their exitstatuses until lua 5.2.  We should be
	-- reasonably safe here, since if we erroneously conclude the
	-- association doesn't exist, we'll just try to add it.
	--
	-- http://lua-users.org/lists/lua-l/2012-01/msg00364.html
	local fp = io.popen(string.format(
		"sacctmgr --parsable2 --noheader list associations " ..
			"format=account users='%s'",
		user
	))
	for line in fp:lines() do
		if line == account then
			existing_assocs[user .. "\n" .. account] = true
			return true
		end
	end

	return false
end

function ensure_assoc_exists(user, account)
	--See if user exists and is associated

	oldaccount = assoc_exists(user,account)

	if oldaccount then
		return true
	end

	local fp = io.popen(string.format(
		"sacctmgr --parsable2 --noheader list associations " ..
			"format=account users='%s'",
		user
	))

	oldaccount=0

	for line in fp:lines() do
		oldaccount=line
	end

	if oldaccount == 0 then
		--Create new group if needed
		ret = os.execute(string.format(
			"sacctmgr -i create account 

Re: [slurm-users] Create users

[Loris Bennett]

Hi Paul,

I'd be interested in seeing your Lua submit script, if you're willing to
share.

Until now I had thought that the most elegant way of setting up Slurm
users would be via a PAM module analogous to pam_mkhomedir, the simplest
option being to use pam_script.

However, given that we do have users who somehow never get round to
submitting a job before their HPC access expires, setting up the Slurm
account when the first job is submitted seems quite appealing.

Cheers,

Loris

Paul Edmon  writes:

> So useradd is adding a Linux user, which sacctmgr creates a Slurm user.
>
> What we do is that we run AD for our Linux user managment. We then in our job 
> submit lua script look to see if the user has an account in slurm and if they 
> don't we create it.
>
> Another way would be to make all your Linux users and then map that in to 
> Slurm using sacctmgr.
>
> It really depends on if your Slurm users are a subset of your regular users 
> or not.
>
> -Paul Edmon-
>
> On 9/12/2018 12:21 PM, Andre Torres wrote:
>
>  Hi all,
>
>  I’m new to slurm and I’m confused regarding user creation. I have an 
> installation with 1 login node and 5 compute nodes. If I create a user across 
> all the nodes with the same uid and gid I can execute jobs but
>  I can’t understand the difference between user creation with “useradd” 
> command and the “sacctmgr” command 
>
>  sacctmgr create account name=test
>
>  sacctmgr create user jdoe account=test
>
>  Also, is there anyway of creating a user at login node and replicate to the 
> compute nodes ? What is the best practice for user creation ?
>
>  Thanks in advance
>
>
-- 
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de

Re: [slurm-users] Create users

[Ole Holm Nielsen]

On 12-09-2018 18:21, Andre Torres wrote:
I’m new to slurm and I’m confused regarding user creation. I have an 
installation with 1 login node and 5 compute nodes. If I create a user 
across all the nodes with the same uid and gid I can execute jobs but I 
can’t understand the difference between user creation with “/useradd”/ 
command and the “/sacctmgr”/ command


sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and replicate to 
the compute nodes ? What is the best practice for user creation ?


Perhaps you may find some useful hints about accounts in my Wiki page
https://wiki.fysik.dtu.dk/niflheim/Slurm_accounting

We use some account management scripts which you can find here:
https://github.com/OleHolmNielsen/Slurm_tools/tree/master/slurmaccounts

/Ole

Re: [slurm-users] Create users

[Paul Edmon]

So useradd is adding a Linux user, which sacctmgr creates a Slurm user.

What we do is that we run AD for our Linux user managment.  We then in 
our job submit lua script look to see if the user has an account in 
slurm and if they don't we create it.


Another way would be to make all your Linux users and then map that in 
to Slurm using sacctmgr.


It really depends on if your Slurm users are a subset of your regular 
users or not.


-Paul Edmon-


On 9/12/2018 12:21 PM, Andre Torres wrote:


Hi all,

I’m new to slurm and I’m confused regarding user creation. I have an 
installation with 1 login node and 5 compute nodes. If I create a user 
across all the nodes with the same uid and gid I can execute jobs but 
I can’t understand the difference between user creation with 
“/useradd”/ command and the “/sacctmgr”/ command


sacctmgr create account name=test

sacctmgr create user jdoe account=test

Also, is there anyway of creating a user at login node and replicate 
to the compute nodes ? What is the best practice for user creation ?


Thanks in advance