On Mon, Feb 13, 2017 at 11:11 AM, Prince Sibanda <[email protected]> wrote:
> However, once one of these two cases starts running, i want to be able to > issue interactively a command to stop feeding certain types of files from > the joblist. I also want to be able to prioritise the jobs in joblist so > that those are run first. I would also like to be able to insert new jobs > into the joblist with a certain priority level, so that if the inserted is a > high priority job for example, it is run next as soon as any of the > currently running jobs has finished. I would like to be able to say skip a > certain job, or repeat a certain job, take a certain job out of joblist, > etc. All this i want to be able to do when one of those two cases has > already started running. You are describing a job queue system. GNU Parallel was not built as a job queue system, but can be used as a very minimal queue. GNU Parallel is not designed for interactivity - it has very few interactive features. It is not designed for removing jobs from the queue, and it has no concept of a priority level. Extending GNU Parallel to a proper job queue system is outside the scope of GNU Parallel, and even if someone made a patch for this, I would probably be reluctant to include it - as it would have to re-write huge sections of GNU Parallel. Some GNU Parallel users use Slurm. I would imagine GNU Parallel is useful for generating and submitting jobs to Slurm, and I would be open to making a few changes to make GNU Parallel interface better with Slurm, if there are obvious improvement ideas. Slurm already has the concept of priority, and it is possible to remove jobs from the queue, so my guess is that it will be easier for you to extend Slurm to meet your needs, and I encourage you to see if Slurm or some of the alternatives meet your needs already. Other alternatives include Torque and Rocks. /Ole
