Couldn't you just create a fourth executable which monitors the progress of the
first three and then returns once they are done? If foo, bar, and baz
communicate amongst each other via text files they could just as easily tell
the monitor when they are done via a text file as well.
i.e. submit this script to the grid
$ foo &
$ bar &
$ baz &
$ monitor_foo_bar_baz
And monitor_foo_bar_baz could look something like this:
while (!foo_done_file){};
while (!bar_done_file){};
while (!baz_done_file){};
Justin
From: [email protected] [mailto:[email protected]] On
Behalf Of Metzler, Chris
Sent: Friday, October 26, 2012 10:37 AM
To: '[email protected]'
Subject: [gridengine users] Q: how to set up a job in which multiple
executables run simultaneously? Also: Open MP environment variables honored?
Hi. I'm new to submitting jobs through Grid Engine, and have skimmed users
guides, googled, etc., without figuring this out and could use some advice.
#1: I need to be able to submit jobs that will start up several distinct
executables that need to run at the same time. If I was doing my work in an
interactive shell, I'd just do something like this:
$ foo &
$ bar &
$ baz &
. . .and so they'd all start around the same time, run in the background, and
complete around the same time. The executables talk to each other through the
output files, and I don't need to do any more than that.
But from what I've read in users guides, I shouldn't invoke processes in the
background in jobs submitted to GE; and that makes sense to me, since then the
interpreter would reach the bottom of the script while the executables are
still running. But submitting a script with contents of:
foo
bar
baz
won't work, because those are run sequentially.
The only thing I can think of to do is to separate each executable out into its
own job submitted to the queue; but then I need to guarantee that they start
executing at the same time, rather than two starting while one sits in a
pending state.
#2: Normally, when running these interactively, I'd set them up for Open MP
use doing something like this:
$ setenv OMP_NUM_THREADS 1
$ foo &
$ setenv OMP_NUM_THREADS 2
$ bar &
$ baz &
etc.
Will instructions like that be honored within a script submitted to the
scheduler?
Any advice on these would be very helpful. Thanks much.
-c
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
