I use pam_exec module to blindly put an ssh session into the user's most
recent task cgroup.
This does seem to work fine for users that just log in to a node to where there
job is running, but multinode jobs have sever limitations. The cgroup isn't
created unless a slurm task has actually been started. For programs that launch
with ssh, this doesn't work at all really. If multinode jobs worked correctly,
then it would obviously be better to somehow detect the slurm job id and use
that.
Here is what I currently use for pam_exec:
#!/bin/sh
[ "$PAM_USER" = "root" ] && exit 0
[ "$PAM_TYPE" = "open_session" ] || exit 0
. /etc/sysconfig/slurm
squeue=$BINDIR/squeue
if [ ! -x $squeue ]; then
exit 0
fi
uidnumber=$(id -u $PAM_USER)
host=$(hostname -s)
# last job the user started is where these tasks will go
jobid=$($squeue --noheader --format=%i --user=$PAM_USER --node=localhost | tail
-1)
[ -z "$jobid" ] && exit 0
for system in freezer cpuset; do
cgdir=/cgroup/$system/slurm_$host
# if the cgdir doesn't exist skip it
[ -d $cgdir ] || continue
# first job step is where we'll put these tasks
cgtasks=$(find $cgdir/uid_$uidnumber/job_$jobid -mindepth 2 -type f -name
tasks -print -quit)
[ -f $cgtasks ] && echo $PPID > $cgtasks
done
exit 0
On Wed, Jan 01, 2014 at 09:43:55PM -0800, Christopher Samuel wrote:
>
> At SC13 a few of us were talking about the issue about what to do when
> you have to allow users to SSH into nodes where there jobs are running.
>
> Currently SLURM doesn't put SSH sessions permitted by the current
> pam_slurm module into any control groups, it would be nice if it would
> at least put them into the top level /cgroup/cpuset/slurm/uid_$UID for
> the user so they can only affect their own jobs.
>
> The problem is that With SSH's privilege separation the current SLURM
> PAM module cannot do this as it will run as an unprivileged process
> (not the user) prior to authentication and so cannot have the
> permissions to move processes around.
>
> However, it appears that something like a pam_slurm *session* library
> would (I believe) run as the user in question - as long as it it can
> learn the PID of the shell being spawned.
>
> The problem then is how to allow that to securely put itself into the
> users top level cgroup. If slurmd just made the tasks file for that
> top level cgroup owned by the user then the process could do it,
> though there would be a small risk that the user could then move
> processes from lower, insulated containers into the top level, though
> they'd only be affecting their own stuff.
>
> Thoughts?
>
> All the best,
> Chris
> --
> Christopher Samuel Senior Systems Administrator
> VLSCI - Victorian Life Sciences Computation Initiative
> Email: [email protected] Phone: +61 (0)3 903 55545
> http://www.vlsci.org.au/ http://twitter.com/vlsci
>
--
andy wettstein
hpc system administrator
research computing center
university of chicago
773.702.1104
