Hi, I found a solution to my problem. So this post is just for public record, if someone runs into the same kind of problem or to comment on our solution.
Our solution is to combine both ideas: We are using prolog / epilog scripts to run our tasks, like creating and deleting a RamFs. To pass the information if or which action to take, we create environment vars accessible to the prolog / epilog scripts via really small SPANK plugins. These plugins serve no other purpose than allowing users to pass options via srun and pass them via spank_job_control_setenv to the Prolog / Epilog scripts. So a simple example would be: 1. User executes: srun --helloworld cat /helloworld.txt 2. Slurm triggers a helloworld SPANK plugin which sets the env var: SPANK_HELLOWORLD=true 3. The prolog script checks if SPANK_HELLOWORLD is set and creates a file /helloworld.txt 4. (job is run) in this case: cat /helloworld.txt 5. The epilog script removes /helloworld.txt again Thank you Malte 2016-06-27 15:57 GMT-07:00 Malte Husmann <malte.husm...@gmail.com>: > Hi, > > I'm fairly new to Slurm and have a question I couldn't solve yet by > googling or searching through older posts in this mailing list: > > We are already using Slurm for our cluster and now plan to add features, > which require root-access before and after a job allocation (like setting > special CPU flags, or initialize a RamFs). > Each user should be able to do something like 'srun > --someplugins=cpuflags,ramfs someprogram' which runs some bash scripts > cpuflags and initramfs before the start of someprogram. > > Currently I have two ideas to implement this: Prolog/Epilog scripts or a > SPANK plugin. But both concepts only work to a certain degree: > > 1. SPANK plugin > I implemented a plugin which does nothing more than reading parameters > which are passed by --someplugins and then executes corresponding bash > scripts before and after the job allocation. > So this already matches the desired behavior, except that we want to drain > nodes on failure. Unfortunately, when I return an error from a SPANK > plugin, the node still runs the job and epilog and returns into the idle > state. > > > 2. Prolog/Epilog > An even simpler concept would be to use a prolog/epilog script which reads > some environment vars set by the user, and starts the corresponding bash > scripts. The prolog already behaves in the desired way and stops / drains a > node on an error. But I don't know how to pass vars or parameters to the > prolog script (run by slurmd). > > So is there a way to pass environment variables to the slurm.conf prolog? > > I wondered how other people were solving similar problems, I'm certain > there are some creative solutions out there. > > Thank you > Malte >
