Hi Magnus,
I forgot that I had problems with that too and just patched our local slurm to
make sharing the submission options with the plugin a little easier. I'm sure
there is a much better, more elegant solution then what I did as this is a bit
hackish (so apologies in advance).
Below is an example patch for sbatch that just pops the options I'm interested
in into the environment so I can query them in the plugin. It also has the
possibility of receiving one particular option (namely the account string) as
we handle our accounting outside of slurm and use the submission filter to
validate that a user has a valid project and sufficient credits to run.
Regards,
-k
--- src/sbatch/sbatch.c 2012-12-06 14:29:26.000000000 -0600
+++ src/sbatch/sbatch.c 2013-01-17 15:37:14.000000000 -0600
@@ -87,6 +87,9 @@
int script_size = 0;
int retries = 0;
+
+ char *tacc_project_string;
+
log_init(xbasename(argv[0]), logopt, 0, NULL);
_set_exit_code();
@@ -126,11 +129,48 @@
exit(error_exit);
}
+ /* Allow access to submission queue */
+ if(opt.partition != NULL)
+ setenv("SLURM_QUEUE",opt.partition,1);
+
+ if(opt.nodes_set)
+ setenv("SLURM_TACC_NNODES_SET","1",1);
+ else
+ setenv("SLURM_TACC_NNODES_SET","0",1);
+
+ if(opt.ntasks_set)
+ setenv("SLURM_TACC_NCORES_SET","1",1);
+ else
+ setenv("SLURM_TACC_NCORES_SET","0",1);
+
+ char local_var[40];
+
+ sprintf(local_var,"%i",opt.ntasks);
+ setenv("SLURM_TACC_CORES",local_var,1);
+
+ sprintf(local_var,"%i",opt.min_nodes);
+ setenv("SLURM_TACC_NODES",local_var,1);
+
+ sprintf(local_var,"%i",opt.time_limit);
+ setenv("SLURM_TACC_RUNLIMIT_MINS",local_var,1);
+
+ if(opt.account != NULL)
+ setenv("SLURM_TACC_ACCOUNT",opt.account,1);
+
if (spank_init_post_opt() < 0) {
error("Plugin stack post-option processing failed");
exit(error_exit);
}
+ /* Read back new project if set */
+
+ tacc_project_string = getenv("TACC_OVERRIDE_PROJECT");
+ if(tacc_project_string != NULL)
+ {
+ xfree(opt.account);
+ opt.account = xstrdup(tacc_project_string);
+ }
+
if (opt.get_user_env_time < 0) {
/* Moab does not propage the user's resource limits, so
* slurmd determines the values at the same time that it
On Jan 30, 2013, at 10:04 AM, Magnus Jonsson <[email protected]> wrote:
> Hi!
>
> I has not succeeded in getting the parameters from the sbatch
> command/submitscript into my spank plugin.
>
> If you have some example that shows this it would make my life easier.
>
> Best regards,
> Magnus
>
> On 2013-01-30 15:34, Karl Schulz wrote:
>>
>> If you really do want to have control over providing arbitrary strings back
>> to the user, then a spank plugin might also be a possibility. We have used
>> the slurm_spank_init_post_opt() callback as a mechanism to create a custom
>> job submission filter for srun/sbatch.
>>
>> It's nothing fancy, but it gives us a way to do some quick sanity checking
>> and apply some local site requirements like: verifying user is not over any
>> of their disk quotas, verifying user provided a max runlimit, additional
>> ACLs for the queue's, maximum jobs per user, etc. In this approach, stdout
>> will be seen by the user and you can customize as desired.
>>
>> On Jan 29, 2013, at 11:31 AM, Moe Jette <[email protected]> wrote:
>>
>>>
>>> I would suggest an job_submit plugin:
>>>
>>> http://www.schedmd.com/slurmdocs/job_submit_plugins.html
>>>
>>> There is no mechanism to return a string to the user, only an exit
>>> code, but adding a few new exit codes would be simple (see
>>> slurm/slurm_errno.h and src/common/slurm_errno.c). We have also
>>> discussed adding a mechanism to return an arbitrary string to the
>>> user, but this is not possible today.
>>>
>>> Quoting Magnus Jonsson <[email protected]>:
>>>
>>>> Hi!
>>>>
>>>> I looking for a way to look at users submitted parameters and if
>>>> they are using it in a "bad" way inform them that this might not be
>>>> a good usage of the system and point them to documentation about how
>>>> slurm works and how to best use it in our system.
>>>>
>>>> I have tried different approaches but failed on every one..
>>>>
>>>> Any hints?
>>>>
>>>> Best regards,
>>>> Magnus
>>>>
>>>> --
>>>> Magnus Jonsson, Developer, HPC2N, Umeå Universitet
>>>>
>>>>
>>>
>
> --
> Magnus Jonsson, Developer, HPC2N, Umeå Universitet
>
