Jan,
I think that was well put.
GridWay would be another option here. Keeps a familiar batch queue
interaction for users, but uses the Globus tools under the covers:
MDS, GRAM, delegation service, RFT, GridFTP, ... It can be used as a
"global queue" for one or more users for submitting jobs on grids.
-Stu
On May 6, 2008, at May 6, 1:32 PM, Jan Ploski wrote:
Steve White wrote:
Jan,
I agree with your assessment that the need to adjust the memory use
per
process is a general one in cluster job submission, and that it is in
some way implemented by any underlying job management system, and
that
these extensions ought not to be PBS-specific.
I also looked at your "messy solution". (The code looks very
professional,
really.) It won't do for my purposes, because I need to present a
minimal,
easily understood solution.
Let me explain my situation:
None of the compute resources is under my control. I can point out
problems to admins, that is all.
I have been assigned two jobs.
I and our users are familiar with doing conventional cluster job
submission. One job was to bring them into the grid fold, showing
them the advantages
of globusrun-ws. If it can be shown to be really a cross-platform
solution, giving them the ability to (almost) effortlessly switch
between grid clusters, the effort will be a success.
My other job is to write a report on practical MPI job submission
over
the grid.
We have come a long way, but still have to deal with a couple of
practical
details. At this point, it looks like both of them will end up as
work-arounds to incomplete implementation of a job submission
interface
in Globus.
If with a future release of Globus, these issues can be dealt with,
grid
job submission will look very attractive to real researchers.
Hi,
Based on my experience with Globus, you might be following a wrong
route (the route to disenchantment). I view Globus more as a
middleware that has to be adapted (as in: "wrapped around" or
"slightly modified") according to your users' needs and which plays
an important role behind the scenes, but it probably should not be
exposed directly to users as a drop-in replacement for their
familiar job submission tools.
There is a reason for that more important than the limitations you
have discovered so far: Globus doesn't ship with command-line job
management commands on par with those of TORQUE/Maui, Condor or SGE.
If you let users submit jobs with globus-job-submit, the next thing
they are going to ask you is "how can I see what jobs I have
submitted", "how can I cancel the job or resubmit it elsewhere", "is
my job running or not", "why is my job not running", "when is my job
going to start", etc.
You need something in front of Globus to make your users' life
bearable. Some projects lean toward application-specific web portals
(I think that's AstroGrid's approach). In our project, we have
deployed a largely application-agnostic frontend based on Condor-G,
but even so there was some customization and some user training
required. The Condor-G approach might be relevant for you because it
covers the scenario of making a transparent transition from a local
batch system to a Grid - the Condor tools for submitting jobs and
status querying are pretty much the same regardless of whether your
job goes to a machine from a local pool (equivalent to an SGE or PBS-
managed cluster) or to a pool of Globus hosts. (In fact, Condor can
submit to GT2 [gLite], GT4, Unicore, and some more Grid middlewares.)
The disadvantage of Condor is that it is a rather huge software
product and trying to understand all of it can be daunting. Still, I
suppose you could get the Grid submission piece of it running in a
couple of hours if you wish to give it a try (by following our
tutorials and asking questions where necessary).
Regards,
Jan Ploski
