Awesome. I'm trying the task prolog approach as it seems a little more straightforward for our environment- seems to work great.
The LUA plugin looks interesting if we have more "heavy lifting" to do. Seems a bit overkill for this trivial task. Thanks for all the advice. Best Michael On Wed, Aug 1, 2012 at 12:07 AM, Carles Fenoy <[email protected]> wrote: > > Hi, > > We are doing this in the job prolog. We basically use 2 prologs: > Prolog: > creates de directory ad chowns it to the job user as the tmp directory is not > user writable. > TaskProlog: > sets the TMPDIR environment variable > > this is part of our TaskProlog perl script: > my $SLURM_JOBID = $ENV{SLURM_JOBID}; > > if (!defined($SLURM_JOBID)) { > panic(2, "${Script}: this script should be executed from slurm"); > } > > print "export TMPDIR=/scratch/tmp/$SLURM_JOBID;"; > > > Regards, > Carles Fenoy > > > On Wed, Aug 1, 2012 at 4:05 AM, Mark A. Grondona <[email protected]> wrote: >> >> >> >> >> Michael Gutteridge <[email protected]> writes: >> >> >> >> > Hi all >> >> > >> >> > I have what I thought was a simple issue, but I can't seem to get it >> >> > sorted. We'd like to have job-specific working directories- basically have >> >> > TMPDIR=/some/path/${SLURM_JOB_ID} set for the user in batch or interactive >> >> > environments. Using use_env from the Spank collection works well, but >> >> > only for srun. >> >> > >> >> > I thought I might be able to modify the use_env source slightly to run it >> >> > for sbatch as well, but it seems the functions available in the allocator >> >> > context (init and init_post_opt) don't have the slurm job ID available. >> >> > Total novice to writing things here- still working that angle. Does >> >> > context matter for those functions? Would init_post_opt game the job id if >> >> > run in the remote or allocator contexts only? >> >> >> >> sbatch itself doesn't have access to a jobid until *after* it submits >> >> the job. I don't think there are any plugin callbacks at that point, >> >> but it is probably moot anyway because the job has been submitted and >> >> sbatch isn't going to do much at this point besides print the jobid >> >> and exit. >> >> >> >> You could possibly use the use-env plugin for all jobs if you place >> >> the TMPDIR setting in the "in task" blocks. These blocks are run >> >> in per-task context (so they run on the remote node instead of in >> >> srun), but unfortunately that means you are setting TMPDIR N times >> >> for a N task job, a waste, but probably negligible. For example, >> >> >> >> in task { >> >> TMPDIR = /some/path/$SLURM_JOB_ID >> >> } >> >> >> >> might work. (I didn't try it yet though) >> >> >> >> However, I think for your use case, the use-env plugin is probably overkill. >> >> There is a "tmpdir.c" example in the slurm-spank-plugins source, but >> >> it is pretty old and I'm not even sure it works anymore, but you might >> >> check it out just for instruction. You could also use the spank-lua >> >> interface and develop this functionality in a lua plugin, if you are >> >> willing to learn or already know lua. My guess is that you can do >> >> most of what you want in just a few dozen lines of lua. Setting the >> >> TMPDIR directly in a plugin will give you more functionality down >> >> the road (for example you could create the directory, remove old >> >> directories, bind the directory over /tmp in the current namespace, >> >> etc.) >> >> >> >> Let me know if you have any further questions, >> >> mark >> >> >> >> > While I realize that we can configure $TMPDIR this way in the job script, >> >> > we would prefer to make sure this step doesn't get "forgotten" by our user >> >> > community. I have this nagging feeling I'm missing a very simple solution. >> >> > >> >> > Thanks >> >> > Michael >> >> > >> >> > >> >> > >> >> > -- >> >> > Hey! Somebody punched the foley guy! >> >> > - Crow, MST3K ep. 508 > > > > > -- > -- > Carles Fenoy > -- Hey! Somebody punched the foley guy! - Crow, MST3K ep. 508
