Alex,
You do not want to spawn mpirun.
Or if this is really what you want, then just use system("env -i ...")
I think what you need is spawn a shell that do the redirection and then invoke
your app.
This is something like
MPI_Comm_spawn("/bin/sh", "-c", "siesta < infile")
That being said, i strongly recommend you patch siesta so it can be invoked
like this
siesta -in infile
(plus the MPI_Comm_disconnect call explained by George)
That would make everything so much easier
Cheers,
Gilles
"Alex A. Schmidt" <a...@ufsm.br> wrote:
>Let me rephrase the previous message:
>
>Putting "/bin/sh" in command with info key "ompi_non_mpi" set to ".true."
>(if command is empty, mpi_comm_spawn tries to execute ' ') of
>mpi_comm_spawn and "-c" "mpirun -n 1 myapp" in args results in
>this message:
>
>**********************************************************
>
>Open MPI does not support recursive calls of mpirun
>
>**********************************************************
>
>Putting a single string in args as "-c mpirun -n 1 myapp" or "-c 'mpirun -n 1
>myapp' "
>
>returns
>
>/usr/bin/sh: - : invalid option
>
>Alex
>
>
>
>
>2014-12-17 21:47 GMT-02:00 Alex A. Schmidt <a...@ufsm.br>:
>
>Putting "/bin/sh" in command with info key "ompi_non_mpi" set to ".true."
>(if command is empty, mpi_comm_spawn tries to execute ' ') of
>mpi_comm_spawn and "-c" "mpirun -n 1 myapp" in args results in
>this message:
>
>/usr/bin/sh: -c: option requires an argument
>
>Putting a single string in args as "-c mpirun -n 1 myapp" or "-c 'mpirun -n 1
>myapp' "
>
>returns
>
>/usr/bin/sh: - : invalid option
>
>Alex
>
>
>2014-12-17 20:17 GMT-02:00 George Bosilca <bosi...@icl.utk.edu>:
>
>I don't think this has any chance of working. The redirection is something
>interpreted by the shell, and when Open MPI "fork-exec" a process it does not
>behave as the shell.
>
>
>Thus a potentially non-portable solution would be to instead of launching the
>mpirun directly to launch it through a shell. Maybe something like "/bin/sh",
>"-c", "mpirun -n 1 myapp".
>
>
> George.
>
>
>
>On Wed, Dec 17, 2014 at 5:02 PM, Alex A. Schmidt <a...@ufsm.br> wrote:
>
>Ralph,
>
>Sorry, "<" as an element of argv to mpi_comm_spawn is interpreted just the
>
>same, as another parameter by the spawnee process.
>
>But I am confused: wouldn't it be redundant to put "mpirun" "-n" "1" "myapp"
>as elements of argv, considering role of the other parameters of mpi_comm_spawn
>
>like the 1st and 3rd ?
>
>
>Imho, it seems to me that stdin and stdout should be keys to "info" and the
>
>respective filenames the values of those keys, if that is possible to
>implement...
>
>Alex
>
>
>
>
>2014-12-17 15:16 GMT-02:00 Ralph Castain <r...@open-mpi.org>:
>
>Have you tried putting the "<" as a separate parameter? In other words, since
>you are specifying the argv, you have to specify each of them separately. So
>it looks more like:
>
>
>"mpirun", "-n", "1", "myapp", "<", "stdinfile"
>
>
>Does that work?
>
>Ralph
>
>
>
>On Wed, Dec 17, 2014 at 8:07 AM, Alex A. Schmidt <a...@ufsm.br> wrote:
>
>Ralph,
>
>I am afraid I will have to insist on i/o redirection matter
>for the spawnee process.
>
>I have a "child" mpi code that do just 2 things: read the 3 parameters
>
>passed to it and print them, and then read data from stdin and show it.
>So, if "stdin_file" is a text file with two lines, say:
>
>
>10
>20
>
>executing "mpirun -n 1 child A B < stdin_file" wiil ouput two lines:
>
>[A] [B] []
>10 20
>
>On the other hand , calling "child" from MPI_Comm_spawn("child",args,...)
>
>where
>
>args(1) = "A"
>
>args(2) = "B"
>
>args(3) ="< stdin_file"
>
>args(4) = " "
>
>will make "child" outputs only 1 line
>
>[A] [B] [< stdin_file]
>
>and then fails because there is not stdin data to read from.
>
>Please, note that surprisingly the whole string "< stdin_file" is interpreted
>as a third parameter to "child" and not a stdin...
>
>Alex
>
>
>
>
>
>
>
>
>2014-12-15 17:26 GMT-02:00 Alex A. Schmidt <a...@ufsm.br>:
>
>Ralph,
>
>I guess you mean "call mpi_comm_spawn( 'siesta', '< infile' , 2 ,...)"
>
>to execute 'mpirun -n 2 siesta < infile' on the spawnee side. That was
>my first choice. Well, siesta behaves as if no stdin file was present...
>
>Alex
>
>
>2014-12-15 17:07 GMT-02:00 Ralph Castain <r...@open-mpi.org>:
>
>You should be able to just include that in your argv that you pass to the
>Comm_spawn API.
>
>
>
>On Mon, Dec 15, 2014 at 9:27 AM, Alex A. Schmidt <a...@ufsm.br> wrote:
>
>George,
>
>Thanks for the tip. In fact, calling mpi_comm_spawn right away with
>MPI_COMM_SELF
>
>has worked for me just as well -- no subgroups needed at all.
>
>I am testing this openmpi app named "siesta" in parallel. The source code is
>available,
>so making it "spawn ready" by adding the pair mpi_comm_get_parent +
>mpi_comm_disconnect
>into the main code can be done. If it works, maybe the siesta's developers
>can be convinced
>to add this feature in a future release.
>
>
>However, siesta is launched only by specifying input/output files with i/o
>redirection like
>
>mpirun -n <some number> siesta < infile > outfile
>
>
>So far, I could not find anything about how to set an stdin file for an
>spawnee process.
>Specifiyng it in a app context file doesn't seem to work. Can it be done?
>Maybe through
>an MCA parameter?
>
>
>Alex
>
>
>
>
>
>
>2014-12-15 2:43 GMT-02:00 George Bosilca <bosi...@icl.utk.edu>:
>
>Alex,
>
>
>The code looks good, and is 100% MPI standard accurate.
>
>
>I would change the way you create the subcoms in the parent. You do a lot of
>useless operations, as you can achieve exactly the same outcome (one
>communicator per node), either by duplicating MPI_COMM_SELF or doing an
>MPI_Comm_split with the color equal to your rank.
>
>
> George.
>
>
>
>On Sun, Dec 14, 2014 at 2:20 AM, Alex A. Schmidt <a...@ufsm.br> wrote:
>
>Hi,
>
>Sorry, guys. I don't think the newbie here can follow any discussion
>beyond basic mpi...
>
>Anyway, if I add the pair
>
>call MPI_COMM_GET_PARENT(mpi_comm_parent,ierror)
>call MPI_COMM_DISCONNECT(mpi_comm_parent,ierror)
>
>on the spawnee side I get the proper response in the spawning processes.
>
>Please, take a look at the attached toy codes parent.F and child.F
>I've been playing with. 'mpirun -n 2 parent' seems to work as expected.
>
>Alex
>
>
>2014-12-13 23:46 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@gmail.com>:
>
>Alex,
>
>Are you calling MPI_Comm_disconnect in the 3 "master" tasks and with the same
>remote communicator ?
>
>I also read the man page again, and MPI_Comm_disconnect does not ensure the
>remote processes have finished or called MPI_Comm_disconnect, so that might
>not be the thing you need.
>George, can you please comment on that ?
>
>Cheers,
>
>Gilles
>
>George Bosilca <bosi...@icl.utk.edu> wrote:
>
>MPI_Comm_disconnect should be a local operation, there is no reason for it to
>deadlock. I looked at the code and everything is local with the exception of a
>call to PMIX.FENCE. Can you attach to your deadlocked processes and confirm
>that they are stopped in the pmix.fence?
>
>
> George.
>
>
>
>On Sat, Dec 13, 2014 at 8:47 AM, Alex A. Schmidt <a...@ufsm.br> wrote:
>
>Hi
>
>Sorry, I was calling mpi_comm_disconnect on the group comm handler, not
>on the intercomm handler returned from the spawn call as it should be.
>
>Well, calling the disconnect on the intercomm handler does halt the spwaner
>side but the wait is never completed since, as George points out, there is no
>disconnect call being made on the spawnee side.... and that brings me back
>to the beginning of the problem since, being a third party app, that call would
>never be there. I guess an mpi wrapper to deal with that could be made for
>the app, but I fell the wrapper itself, at the end, would face the same problem
>we face right now.
>
>My application is a genetic algorithm code that search optimal configuration
>(minimum or maximum energy) of cluster of atoms. The work flow bottleneck
>is the calculation of the cluster energy. For the cases which an analytical
>potential is available the calculation can be made internally and the workload
>is distributed among slaves nodes from a master node. This is also done
>when an analytical potential is not available and the energy calculation must
>be done externally by a quantum chemistry code like dftb+, siesta and Gaussian.
>So far, we have been running these codes in serial mode. No need to say that
>we could do a lot better if they could be executed in parallel.
>
>I am not familiar with DMRAA but it seems to be the right choice to deal with
>job schedulers as it covers the ones I am interested in (pbs/torque and
>loadlever).
>
>Alex
>
>
>2014-12-13 7:49 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@gmail.com>:
>
>George is right about the semantic
>
>However i am surprised it returns immediatly...
>That should either work or hang imho
>
>The second point is no more mpi related, and is batch manager specific.
>
>You will likely find a submit parameter to make the command block until the
>job completes. Or you can write your own wrapper.
>Or you can retrieve the jobid and qstat periodically to get the job state.
>If an api is available, this is also an option.
>
>Cheers,
>
>Gilles
>
>George Bosilca <bosi...@icl.utk.edu> wrote:
>You have to call MPI_Comm_disconnect on both sides of the intercommunicator.
>On the spawner processes you should call it on the intercom, while on the
>spawnees you should call it on the MPI_Comm_get_parent.
>
>
> George.
>
>
>On Dec 12, 2014, at 20:43 , Alex A. Schmidt <a...@ufsm.br> wrote:
>
>
>Gilles,
>
>MPI_comm_disconnect seem to work but not quite.
>
>The call to it returns almost immediatly while
>
>the spawn processes keep piling up in the background
>
>until they are all done...
>
>I think system('env -i qsub...') to launch the third party apps
>
>would take the execution of every call back to the scheduler
>queue. How would I track each one for their completion?
>
>Alex
>
>
>2014-12-12 22:35 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@gmail.com>:
>
>Alex,
>
>You need MPI_Comm_disconnect at least.
>I am not sure if this is 100% correct nor working.
>
>If you are using third party apps, why dont you do something like
>system("env -i qsub ...")
>with the right options to make qsub blocking or you manually wait for the end
>of the job ?
>
>That looks like a much cleaner and simpler approach to me.
>
>Cheers,
>
>Gilles
>
>"Alex A. Schmidt" <a...@ufsm.br> wrote:
>
>Hello Gilles,
>
>Ok, I believe I have a simple toy app running as I think it should:
>'n' parent processes running under mpi_comm_world, each one
>
>spawning its own 'm' child processes (each child group work
>together nicely, returning the expected result for an mpi_allreduce call).
>
>Now, as I mentioned before, the apps I want to run in the spawned
>
>processes are third party mpi apps and I don't think it will be possible
>to exchange messages with them from my app. So, I do I tell
>when the spawned processes have finnished running? All I have to work
>
>with is the intercommunicator returned from the mpi_comm_spawn call...
>
>
>Alex
>
>
>
>
>
>2014-12-12 2:42 GMT-02:00 Alex A. Schmidt <a...@ufsm.br>:
>
>Gilles,
>
>Well, yes, I guess....
>
>I'll do tests with the real third party apps and let you know.
>
>These are huge quantum chemistry codes (dftb+, siesta and Gaussian)
>
>which greatly benefits from a parallel environment. My code is just
>a front end to use those, but since we have a lot of data to process
>
>it also benefits from a parallel environment.
>
>
>Alex
>
>
>
>
>2014-12-12 2:30 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@iferc.org>:
>
>Alex,
>
>just to make sure ...
>this is the behavior you expected, right ?
>
>Cheers,
>
>Gilles
>
>
>
>On 2014/12/12 13:27, Alex A. Schmidt wrote:
>
>Gilles, Ok, very nice! When I excute do rank=1,3 call
>MPI_Comm_spawn('hello_world','
>',5,MPI_INFO_NULL,rank,MPI_COMM_WORLD,my_intercomm,MPI_ERRCODES_IGNORE,status)
>enddo I do get 15 instances of the 'hello_world' app running: 5 for each
>parent rank 1, 2 and 3. Thanks a lot, Gilles. Best regargs, Alex 2014-12-12
>1:32 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@iferc.org
>
>: Alex, just ask MPI_Comm_spawn to start (up to) 5 tasks via the maxprocs
>parameter : int MPI_Comm_spawn(char *command, char *argv[], int maxprocs,
>MPI_Info info, int root, MPI_Comm comm, MPI_Comm *intercomm, int
>array_of_errcodes[]) INPUT PARAMETERS maxprocs - maximum number of processes
>to start (integer, significant only at root) Cheers, Gilles On 2014/12/12
>12:23, Alex A. Schmidt wrote: Hello Gilles, Thanks for your reply. The "env -i
>PATH=..." stuff seems to work!!! call system("sh -c 'env -i
>PATH=/usr/lib64/openmpi/bin:/bin mpirun -n 2 hello_world' ") did produce the
>expected result with a simple openmi "hello_world" code I wrote. I might be
>harder though with the real third party app I have in mind. And I realize
>getting passed over a job scheduler with this approach might not work at
>all... I have looked at the MPI_Comm_spawn call but I failed to understand how
>it could help here. For instance, can I use it to launch an mpi app with the
>option "-n 5" ? Alex 2014-12-12 0:36 GMT-02:00 Gilles Gouaillardet
><gilles.gouaillar...@iferc.org
>
>: Alex, can you try something like call system(sh -c 'env -i /.../mpirun -np 2
>/.../app_name') -i start with an empty environment that being said, you might
>need to set a few environment variables manually : env -i PATH=/bin ... and
>that being also said, this "trick" could be just a bad idea : you might be
>using a scheduler, and if you empty the environment, the scheduler will not be
>aware of the "inside" run. on top of that, invoking system might fail
>depending on the interconnect you use. Bottom line, i believe Ralph's reply is
>still valid, even if five years have passed : changing your workflow, or using
>MPI_Comm_spawn is a much better approach. Cheers, Gilles On 2014/12/12 11:22,
>Alex A. Schmidt wrote: Dear OpenMPI users, Regarding to this previous
>post<http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
><http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
><http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
><http://www.open-mpi.org/community/lists/users/2009/06/9560.php> from 2009, I
>wonder if the reply from Ralph Castain is still valid. My need is similar but
>quite simpler: to make a system call from an openmpi fortran application to
>run a third party openmpi application. I don't need to exchange mpi messages
>with the application. I just need to read the resulting output file generated
>by it. I have tried to do the following system call from my fortran openmpi
>code: call system("sh -c 'mpirun -n 2 app_name") but I get
>********************************************************** Open MPI does not
>support recursive calls of mpirun
>********************************************************** Is there a way to
>make this work? Best regards, Alex
>_______________________________________________ users mailing
>listus...@open-mpi.org Subscription:
>http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25966.php
>_______________________________________________ users mailing
>listus...@open-mpi.org Subscription:
>http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this
>post:http://www.open-mpi.org/community/lists/users/2014/12/25967.php
>_______________________________________________ users mailing
>listus...@open-mpi.org Subscription:
>http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25968.php
>_______________________________________________ users mailing list
>us...@open-mpi.org Subscription:
>http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25969.php
>
>
>
>_______________________________________________ users mailing list
>us...@open-mpi.org Subscription:
>http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25970.php
>
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25971.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25974.php
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25975.php
>
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25978.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25979.php
>
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25981.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25982.php
>
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/25991.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/26002.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/26003.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/26010.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/26011.php
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/26013.php
>
>
>
>_______________________________________________
>users mailing list
>us...@open-mpi.org
>Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>Link to this post:
>http://www.open-mpi.org/community/lists/users/2014/12/26014.php
>
