On May 5, 2008, at 11:12 AM, Jan Ploski wrote:
I think it's not really a good case for metascheduling because it
doesn't
make sense to separate the (typically very short) setup stage from the
actual (long) computation - there are simply no advantages in doing
that.
If you use a metascheduler, there is risk that your setup job becomes
separated in the queue from the computation job by other users' jobs,
unnecessarily delaying the whole execution.
Hi Jan,
Even after reflection, I do not really agree. This is really just
one step along a workflow. Setup jobs can generally be done via
fork job manager submissions, or submissions to fast-turnaround
queues that in principle always complete faster than the scheduling
for the bulk-execution MPI or multiple slow-serial jobs. Workflow or
metascheduling tools can simply be used to ensure the order of
execution of these steps.
One way or the other, you are always pursuing a workflow of data
production, analysis and/or processing steps with such jobs, so this
is just a special-case variant.
Of course, everyone is welcome to their opinion (and preferred way of
doing it). With such a simple use case, you could just build jobs
with a lock file in common space that lets the first job to start do
the setup, and others ignore it if the book-keeping lock file made by
the first job is in place. Of course, that would be the very
definition of a race condition, so you would want to be careful --
and it would also depend on having access to shared-access common
space between jobs, not always guaranteed for job execution sandbox
space.
That's why I would prefer to think of this as a workflow step.
Cheers,
Alan
Alan Sill, Ph.D
TIGRE Senior Scientist, High Performance Computing Center
Adjunct Professor of Physics
TTU
====================================================================
: Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
: e-mail: [EMAIL PROTECTED] ph. 806-742-4350 fax 806-742-4358 :
====================================================================
