Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Adrian Reber via users
I had a look at it and not sure if it really makes sense. In btl_vader_{put,get}.c it would be easy to check for the user namespace ID of the other process, but the function would then just return OPAL_ERROR a bit earlier instead of as a result of process_vm_{read,write}v(). Nothing would really c

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Gilles Gouaillardet via users
Adrian, An option is to involve the modex. each task would OPAL_MODEX_SEND() its own namespace ID, and then OPAL_MODEX_RECV() the one from its peers and decide whether CMA support can be enabled. Cheers, Gilles On 7/22/2019 4:53 PM, Adrian Reber via users wrote: I had a look at it and

Re: [OMPI users] When is it save to free the buffer after MPI_Isend?

2019-07-22 Thread Jeff Squyres (jsquyres) via users
> On Jul 21, 2019, at 11:31 AM, carlos aguni via users > wrote: > > MPI_Isend() > ... some stuff.. > flag = 0; > MPI_Test(req, &flag, &status); > if (flag){ > free(buffer); > } > > After the free() i'm getting errors like: > [[58327,1],0][btl_tcp_frag.c:130:mca_btl_tcp_frag_send] > mca_btl

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Ralph Castain via users
If that works, then it might be possible to include the namespace ID in the job-info provided by PMIx at startup - would have to investigate, so please confirm that the modex option works first. > On Jul 22, 2019, at 1:22 AM, Gilles Gouaillardet via users > wrote: > > Adrian, > > > An optio

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Adrian Reber via users
I have most of the code ready, but I still have troubles doing OPAL_MODEX_RECV. I am using the following lines, based on the code from orte/test/mpi/pmix.c: OPAL_MODEX_SEND_VALUE(rc, OPAL_PMIX_LOCAL, "user_ns_id", &value, OPAL_INT); This sets rc to 0. For receiving: OPAL_MODEX_RECV_VALUE(rc, "us

Re: [OMPI users] How is the rank determined (Open MPI and Podman)

2019-07-22 Thread Nathan Hjelm via users
Just add it to the existing modex. -Nathan > On Jul 22, 2019, at 12:20 PM, Adrian Reber via users > wrote: > > I have most of the code ready, but I still have troubles doing > OPAL_MODEX_RECV. I am using the following lines, based on the code from > orte/test/mpi/pmix.c: > > OPAL_MODEX_SEND_V