I had a look at Bruno's plugin -- it looks like a great starting point for what we need.
I think generalising Bruno's PBS Java API to support other similar systems with qsub/qstat type commands should work well: it seems that a PBSSlaveComputer could be generalised into a BatchSystemSlave. Regarding the problem of Jenkins going offline, my feeling would be to mitigate this risk by: (1) Warning and doing a qdel on all Jenkins-submitted jobs when Jenkins shuts down cleanly. (2) Warning and doing a qdel on all lingering Jenkins jobs when Jenkins comes back up after an unclean shutdown. I don't think I understand the durable-task solution, as to me that sounds like one would not be able to move existing freestyle jobs back and forth between conventional nodes and batch system nodes... It seems to me that the right solution would be a new kind of slave node, rather than a new kind of job, though I'm happy to be corrected... On Thu, Jun 12, 2014 at 1:29 AM, 'Bruno P. Kinoshita' via Jenkins Developers <[email protected]> wrote: > Hi Jesse, > > I'm waiting for the workflow plugin but hadn't heard about the durable > tasks plugin. > > The current implementation of the pbs plugin [1] was a POC to submit jobs > via the qsub command to a PBS Torque server. In our tests a PBS job was > triggered from a Freestyle project to the server via a SSH jump box in a > university cluster. It is working and creating new jobs. > > However, indeed if Jenkins goes offline the build stops running, even > though the PBS job might still be running. I thought about re-using the > monitor-external-job plug-in, but in some places qsub might be the only > option to submit jobs. > > I will experiment with the durable task plugin. Any advice on how to add > items created in the cluster to the build queue in Jenkins? > > The current implementation creates a PBSSlaveComputer that represents a > PBS Server. A Widget is created to retrieve the list of queues and its jobs > from the server. > > Thanks! > Bruno > > [1] https://github.com/biouno/pbs-plugin > > ------------------------------ > *From:* Jesse Glick <[email protected]> > *To:* [email protected] > *Sent:* Wednesday, June 11, 2014 1:11 PM > *Subject:* Re: Jenkins plugin for HPC job systems > > On Wed, Jun 11, 2014 at 10:05 AM, James Hetherington <[email protected]> > wrote: > > Is anyone aware of a plugin which already does this? > > No but I have heard of someone interested in SGE support. My idea was > to implement the durable-task-plugin API, at which point any client of > that plugin can use the system (and Jenkins does not need to be > continuously running while the scheduling system runs your batch job). > Jenkins Enterprise by CloudBees has one such client, which looks and > feels like a freestyle project; the upcoming Workflow plugin suite has > another caller, which is used routinely for running forked commands > like shell scripts. > > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Jenkins Developers" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/jenkinsci-dev/nLOBkO8ttVM/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
