> -----Original Message----- > From: [email protected] [mailto:owner-slurm- > [email protected]] On Behalf Of Michael Di Domenico > Sent: Tuesday, April 19, 2011 9:38 AM > To: [email protected] > Subject: Re: [slurm-dev] JobComp scripts > > Danny, we are using the accounting/slurmdbd module to push the records > into a database (just starting with it), but it was/is not clear to me > that this provides a duplication of the jobcomp information. If > that's the case, then yes, i could change the jobcomp/mysql plugin to > jobcomp/script.
Yes, the jobcomp is, outside of the fields you mentioned a small subset of what the accounting plugin stores. There is not much sense using both to write to the slurmdbd. > > Since we're on the accounting/slurmdbd subject, for a moment, i > noticed in the accounting table, there is/are a field missing. in the > jobcomp/script output you're setting a "BATCH" environment variable to > tell whether a program was produced as a batch or single, that column > seems to be missing from the database table. it would also be nice if > this table logged the clustername and what nodes were done on a > NODE_FAIL exit status as additional columns. I am not sure which version of SLURM you are using, but all job tables in 2.2+ are prefixed with the name of the cluster, hence why it isn't a column in the table. We don't store anything special about the nodes in the job table outside of the nodes actually ran on, but you could easily figure out which nodes had issues by using sacctmgr to list the events that happened on those nodes at the end of the job. As for the batch flag, if a user uses sbatch or salloc there is a flag set called track_steps in the database. If that isn't set the job was started directly from srun. As for telling if it really was a batch script or not, the only way to really tell if that was the case it isn't fool proof, but if the user does specify the name of the job with salloc the name of the job will be "allocation", you could query against that to find batch jobs outside of interactive jobs, but if the user states a job name that doesn't work. On another note, I am not sure how you are getting this data to your other database, but it is probably a good idea to either use the C api, perl api, or sacct[mgr] to get the data since that interface is pretty static. If you are looking at the actual database tables they could change often and would probably be difficult to keep up to sync. Especially going from 2.1 to 2.2 where the entire database scheme changed dramatically. Danny > > having these extra columns would eliminate my need for a script at all > as i could sql pull the data into our site-wide tracking databases > > On Tue, Apr 19, 2011 at 11:59 AM, Danny Auble <[email protected]> wrote: > > Hey Michael, is there any reason you don't want to use the > > accounting_storage/slurmdbd plugin for accounting? It provides an > outrageous > > amount more information about each job and step than the basic > jobcomp > > plugin. It also allows the ability for fairshare scheduling and > limits if > > you want. Since you are already using the slurmdbd what would be > considered > > the hard part is already over. Also, if you just want straight job > records > > no real setup is required. > > > > That would leave the jobcomp/script open for your other database. > > > > Danny > > > >> Yes, i guess in my expedience I wasn't clear. We're using the > jobcomp > > > >> module with slurmdbd now, i want to continue using that, but also > run > > > >> a script. effectively having two JobCompType parameters > > > >> > > > >> On Tue, Apr 19, 2011 at 11:37 AM, Jette, Moe <[email protected]> > wrote: > > > >> > We do something like that today. Just configure > > > >> > > > > >> > JobCompType=jobcomp/script > > > >> > JobCompLoc=<pathname> > > > >> > > > > >> > Your script can easily write information into a database and do > anything > >> > else you want. > > > >> > > > > >> > ________________________________________ > > > >> > From: [email protected] [owner-slurm- > [email protected]] On > >> > Behalf Of Michael Di Domenico [[email protected]] > > > >> > Sent: Tuesday, April 19, 2011 8:21 AM > > > >> > To: slurm-dev > > > >> > Subject: [slurm-dev] JobComp scripts > > > >> > > > > >> > Is it possible to have the JobComp logic insert it's records into > a > > > >> > database, but also run a script for each job run? > > > >> > > > > >> > > > > >> > > > > > >
