Stegfan, regarding the Prolog/Task Prolog option, David Lee Braun sent me a comprehensive reply on that one back in January. The answer is that you have to se the TMPDIR in a separate /etc/profile.d/slurm.sh Is. The Prolog creates the directory OK, but the TMPDIR variable is only set if a profile.d script is used (I am sure there are other ways)
See this thread: https://groups.google.com/forum/#!topic/slurm-devel/kqzPbN8NpkQ And thanks again to David From: Stefan Seritan [mailto:[email protected]] Sent: 13 March 2017 17:19 To: slurm-dev <[email protected]> Subject: [slurm-dev] Job-Specific Working Directory on Local Scratch Hey, I am trying to set up SLURM to use job-specific working directories on local scratch for sbatch jobs, with the form /u/$USER/$SLURM_JOB_ID. Here's what I have tried: - Manual: Users can put cd $SCRATCH, and copy the files they need. This works, but output files would not be copied back in case of things like jobs being killed due to exceeding resource limits. - Prolog/TaskProlog: I can create the scratch directory and export a $SCRATCH environment variable, but I cannot cd into it or set any time of working directory environment variable. - job_submit.lua: Have access to working directory variable and $USER, but don't have $SLURM_JOB_ID (job_desc.job_id and job_desc.job_id_str are not set) - SPANK plugin: I can see the scratch directory that was created by the prolog, but again I cannot cd into it or change any kind of working directory environment. I've tried init, user_init, task_init_privileged, and task_init, and none of them worked for me. Any suggestions on how to set this up would be great. -- Stefan Seritan Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Employees of XMA Ltd are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to company policy and outside the scope of the employment of the individual concerned. The company will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. XMA Limited is registered in England and Wales (registered no. 2051703). Registered Office: Wilford Industrial Estate, Ruddington Lane, Wilford, Nottingham, NG11 7EP
