Andy Wettstein <[email protected]> writes: > I use pam_exec module to blindly put an ssh session into the user's most > recent task cgroup. > > This does seem to work fine for users that just log in to a node to where > there > job is running, but multinode jobs have sever limitations. The cgroup isn't > created unless a slurm task has actually been started. For programs that > launch > with ssh, this doesn't work at all really. If multinode jobs worked correctly, > then it would obviously be better to somehow detect the slurm job id and use > that.
A fully functional 'pam_slurm' module would do just that (it has to check for an active allocation on the node anyway), and would create the uid cgroup as needed if one does not already exist. This is how the old pam_slurm_cpuset[1] operated and eventually the cgroups code was supposed to work similarly. (You also have to make sure resources are added and subtracted from the UID cgroups as jobs are created and destroyed) [1] https://code.google.com/p/slurm-spank-plugins/wiki/CPUSET mark > > > Here is what I currently use for pam_exec: > > #!/bin/sh > > [ "$PAM_USER" = "root" ] && exit 0 > [ "$PAM_TYPE" = "open_session" ] || exit 0 > > . /etc/sysconfig/slurm > > squeue=$BINDIR/squeue > > if [ ! -x $squeue ]; then > exit 0 > fi > > uidnumber=$(id -u $PAM_USER) > host=$(hostname -s) > > # last job the user started is where these tasks will go > jobid=$($squeue --noheader --format=%i --user=$PAM_USER --node=localhost | > tail -1) > > [ -z "$jobid" ] && exit 0 > > for system in freezer cpuset; do > cgdir=/cgroup/$system/slurm_$host > > # if the cgdir doesn't exist skip it > [ -d $cgdir ] || continue > # first job step is where we'll put these tasks > cgtasks=$(find $cgdir/uid_$uidnumber/job_$jobid -mindepth 2 -type f -name > tasks -print -quit) > [ -f $cgtasks ] && echo $PPID > $cgtasks > > done > > exit 0 > > On Wed, Jan 01, 2014 at 09:43:55PM -0800, Christopher Samuel wrote: >> >> At SC13 a few of us were talking about the issue about what to do when >> you have to allow users to SSH into nodes where there jobs are running. >> >> Currently SLURM doesn't put SSH sessions permitted by the current >> pam_slurm module into any control groups, it would be nice if it would >> at least put them into the top level /cgroup/cpuset/slurm/uid_$UID for >> the user so they can only affect their own jobs. >> >> The problem is that With SSH's privilege separation the current SLURM >> PAM module cannot do this as it will run as an unprivileged process >> (not the user) prior to authentication and so cannot have the >> permissions to move processes around. >> >> However, it appears that something like a pam_slurm *session* library >> would (I believe) run as the user in question - as long as it it can >> learn the PID of the shell being spawned. >> >> The problem then is how to allow that to securely put itself into the >> users top level cgroup. If slurmd just made the tasks file for that >> top level cgroup owned by the user then the process could do it, >> though there would be a small risk that the user could then move >> processes from lower, insulated containers into the top level, though >> they'd only be affecting their own stuff. >> >> Thoughts? >> >> All the best, >> Chris >> -- >> Christopher Samuel Senior Systems Administrator >> VLSCI - Victorian Life Sciences Computation Initiative >> Email: [email protected] Phone: +61 (0)3 903 55545 >> http://www.vlsci.org.au/ http://twitter.com/vlsci >> > > -- > andy wettstein > hpc system administrator > research computing center > university of chicago > 773.702.1104
