Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-29 Thread Jordi Guitart
Hi, I try to provide some insights about how this could be accomplished (see inline). Do they seem feasible? On 26/03/2017 18:18, r...@open-mpi.org wrote: There are a couple of things you’d need to resolve before worrying about code: * IIRC, there is a separate ORTE daemon in each Docker

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-25 Thread Jordi Guitart
Hi, I don't have previous expertise on the source code of OpenMPI, so I don't have a clear idea of the needed changes to implement this feature. This probably requires some preliminary brainstorming to decide the most appropriate way to inform OpenMPI that underlying nodes can share memory

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jeff Squyres (jsquyres)
On Mar 24, 2017, at 6:41 AM, Jordi Guitart wrote: > > Docker containers have different IP addresses, indeed, so now we know why it > does not work. I think that this could be a nice feature for OpenMPI, so I'll > probably issue a request for it ;-) Cool. I don't think

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jordi Guitart
Hello Jeff, Docker containers have different IP addresses, indeed, so now we know why it does not work. I think that this could be a nice feature for OpenMPI, so I'll probably issue a request for it ;-) Thanks for your help. On 24/03/2017 11:27, Jeff Squyres (jsquyres) wrote: If the Docker

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jeff Squyres (jsquyres)
If the Docker containers have different IP addresses, Open MPI will think that they are different "nodes" (or "hosts" or "servers" or whatever your favorite word is), and therefore will assume that they processes in these different containers are unable to share memory. Meaning: no work has

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jordi Guitart
Hello John, Yes, in fact, I'm comparing Docker with Singularity regarding the execution of MPI applications :-) I'd like to make the comparison fairer by allowing Docker containers to share memory. Thanks On 24/03/2017 10:00, John Hearns via users wrote: Jordi, this is not an answer to

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread John Hearns via users
Jordi, this is not an answer to your question. However have you looked at Singularity: http://singularity.lbl.gov/ On 24 March 2017 at 08:54, Jordi Guitart wrote: > Hello, > > Docker allows several containers running in the same host to share the > same IPC namespace,

[OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jordi Guitart
Hello, Docker allows several containers running in the same host to share the same IPC namespace, thus they can share memory (see example here: https://github.com/docker/docker/pull/8211#issuecomment-56873448). I assume this could be used by OpenMPI to communicate MPI processes running in