On Thu, Feb 25, 2016 at 05:09:41PM -0500, berg...@merctech.com wrote:
> So, the lockfile contains the jobID, or something similar?
>
> Some jobs use a file as a flag (ie. checking the existence or content), but
> we've largely avoided POSIX file locking (the mixture of NFS, GPFS, & CIFS
> here
It's definitely not a good practice to do qsub from within a job as it might
cause a deadlocks and also very hard to support in case of error.
I would say you should change the workflow and submit jobs in different way,
using hold_jid parameter.
It should look like this:
submit all 'days' jobs
In the message dated: Thu, 25 Feb 2016 13:38:28 -0800,
The pithy ruminations from Skylar Thompson on
were:
=> We have pipelines that are driven by a qsub at the end of a batch script.
We have several pipelines like that. It's not uncommon to have the initial job
submit 5~20 other jobs.
I don't
We have pipelines that are driven by a qsub at the end of a batch script.
Error tracking is an issue but sometimes it's easier to do that than to
engineer a raft of job dependencies. As you note, concurrency can be an
issue, but there are a number of ways to deal with that:
* Lock file in a
Where I work, we have jobs that submit jobs that submit jobs.. this could
potentially cause a deadlock but we're somehow (probably luck) manage to
live with it.. I'm wondering if that's a reasonable practice and if not if
you can suggest a better way to do what we do..
Example:
we have these 3
