Re: [OMPI users] When is it save to free the buffer after MPI_Isend?

2019-08-11 Thread Jeff Hammond via users
The snippets suggest you were storing a reference to an object on the
stack. Stack variables go out of scope when the function returns. Using a
reference to them out-of-scope is illegal but often fails
nondeterministically. Good compilers will issue a warning about this under
the right conditions (ie compiler flags).

Jeff

On Sat, Aug 10, 2019 at 10:59 AM carlos aguni via users <
users@lists.open-mpi.org> wrote:

> Hi all,
>
> Sorry no reply.
>
> I just figured out the solution.
>
> The problem was that I had a function that would MPI_Isend a message on
> every call to it. Then I'd store its request pointer to a list.
> My MPI_Isend snippet:
> MPI_Request req;
> MPI_Isend(blabla, )
> task_push();
>
> From time to time at the beginning of that function I'd call another
> function that would iterate over that list MPI_Testing if that message
> had completed to then free the buffers used.
> The problem was that even if the flag returned from MPI_Test(, ,
> ) previouly assigned to 0 returned 1 but my guess is that C had
> already deallocated it from the heap (idk much about C though..)
> Snippet of my clean function:
> 
> int flag = 0;
> MPI_Test(req, , );
> if (flag) { // then free..
> ...
>
> My solution that worked was previously malloc it before the MPI_Isend call
> like:
> MPI_Request *rr = (MPI_Request *)malloc(sizeof(MPI_Request));
> MPI_Isend(blabla, rr);
> task_push(rr);
>
> All I know is that it's working now..
>
> Thanks to all.
>
> Regards,
> C.
>
> On Sun, Jul 28, 2019 at 11:53 AM Jeff Squyres (jsquyres) via users <
> users@lists.open-mpi.org> wrote:
>
>> On Jul 27, 2019, at 10:43 PM, Gilles Gouaillardet via users <
>> users@lists.open-mpi.org> wrote:
>> >
>> > MPI_Isend() does not automatically frees the buffer after it sends the
>> message.
>> > (it simply cannot do it since the buffer might be pointing to a global
>> > variable or to the stack).
>>
>> Gilles is correct: MPI_Isend does not free the buffer.  I was wondering
>> if you had somehow used that same buffer -- or some subset of that buffer
>> -- in other non-blocking MPI API calls, and freeing it triggered Bad Things
>> because MPI was still using (some of) that buffer because of other pending
>> MPI requests.
>>
>> > Can you please extract a reproducer from your program ?
>>
>> Yes, please do this.
>>
>> > Out of curiosity, what if you insert an (useless) MPI_Wait() like this ?
>> >
>> > MPI_Test(req, , );
>> > if (flag){
>> >MPI_Wait(req, MPI_STATUS_IGNORE);
>> >free(buffer);
>> > }
>>
>> That should be a no-op, because "req" should have been turned into
>> MPI_REQUEST_NULL if flag==true.
>>
>> --
>> Jeff Squyres
>> jsquy...@cisco.com
>>
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://lists.open-mpi.org/mailman/listinfo/users
>>
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users

-- 
Jeff Hammond
jeff.scie...@gmail.com
http://jeffhammond.github.io/
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] When is it save to free the buffer after MPI_Isend?

2019-08-10 Thread carlos aguni via users
Hi all,

Sorry no reply.

I just figured out the solution.

The problem was that I had a function that would MPI_Isend a message on
every call to it. Then I'd store its request pointer to a list.
My MPI_Isend snippet:
MPI_Request req;
MPI_Isend(blabla, )
task_push();

>From time to time at the beginning of that function I'd call another
function that would iterate over that list MPI_Testing if that message
had completed to then free the buffers used.
The problem was that even if the flag returned from MPI_Test(, ,
) previouly assigned to 0 returned 1 but my guess is that C had
already deallocated it from the heap (idk much about C though..)
Snippet of my clean function:

int flag = 0;
MPI_Test(req, , );
if (flag) { // then free..
...

My solution that worked was previously malloc it before the MPI_Isend call
like:
MPI_Request *rr = (MPI_Request *)malloc(sizeof(MPI_Request));
MPI_Isend(blabla, rr);
task_push(rr);

All I know is that it's working now..

Thanks to all.

Regards,
C.

On Sun, Jul 28, 2019 at 11:53 AM Jeff Squyres (jsquyres) via users <
users@lists.open-mpi.org> wrote:

> On Jul 27, 2019, at 10:43 PM, Gilles Gouaillardet via users <
> users@lists.open-mpi.org> wrote:
> >
> > MPI_Isend() does not automatically frees the buffer after it sends the
> message.
> > (it simply cannot do it since the buffer might be pointing to a global
> > variable or to the stack).
>
> Gilles is correct: MPI_Isend does not free the buffer.  I was wondering if
> you had somehow used that same buffer -- or some subset of that buffer --
> in other non-blocking MPI API calls, and freeing it triggered Bad Things
> because MPI was still using (some of) that buffer because of other pending
> MPI requests.
>
> > Can you please extract a reproducer from your program ?
>
> Yes, please do this.
>
> > Out of curiosity, what if you insert an (useless) MPI_Wait() like this ?
> >
> > MPI_Test(req, , );
> > if (flag){
> >MPI_Wait(req, MPI_STATUS_IGNORE);
> >free(buffer);
> > }
>
> That should be a no-op, because "req" should have been turned into
> MPI_REQUEST_NULL if flag==true.
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
>
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] When is it save to free the buffer after MPI_Isend?

2019-07-28 Thread Jeff Squyres (jsquyres) via users
On Jul 27, 2019, at 10:43 PM, Gilles Gouaillardet via users 
 wrote:
> 
> MPI_Isend() does not automatically frees the buffer after it sends the 
> message.
> (it simply cannot do it since the buffer might be pointing to a global
> variable or to the stack).

Gilles is correct: MPI_Isend does not free the buffer.  I was wondering if you 
had somehow used that same buffer -- or some subset of that buffer -- in other 
non-blocking MPI API calls, and freeing it triggered Bad Things because MPI was 
still using (some of) that buffer because of other pending MPI requests.

> Can you please extract a reproducer from your program ?

Yes, please do this.

> Out of curiosity, what if you insert an (useless) MPI_Wait() like this ?
> 
> MPI_Test(req, , );
> if (flag){
>MPI_Wait(req, MPI_STATUS_IGNORE);
>free(buffer);
> }

That should be a no-op, because "req" should have been turned into 
MPI_REQUEST_NULL if flag==true.

-- 
Jeff Squyres
jsquy...@cisco.com

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users


Re: [OMPI users] When is it save to free the buffer after MPI_Isend?

2019-07-27 Thread Gilles Gouaillardet via users
Carlos,

MPI_Isend() does not automatically frees the buffer after it sends the message.
(it simply cannot do it since the buffer might be pointing to a global
variable or to the stack).

Can you please extract a reproducer from your program ?

Out of curiosity, what if you insert an (useless) MPI_Wait() like this ?

MPI_Test(req, , );
if (flag){
MPI_Wait(req, MPI_STATUS_IGNORE);
free(buffer);
}

Cheers,

Gilles

On Sun, Jul 28, 2019 at 5:45 AM carlos aguni via users
 wrote:
>
> Hi Jeff,
>
> Thank you for your reply.
>
> If i don't free the program completes but I'm not sure whether MPI_Isend 
> automatically frees the buffer after it sends the message. Does it?
>
> I put a long sleep at the end to check the memory used using pmap.
>
> The pmap command reported I'm using around 2GB which I'm guessing it isn't 
> freeing it.
>
> Is there anything I could try?
>
> Regards,
> C.
>
> On Mon, Jul 22, 2019 at 10:59 AM Jeff Squyres (jsquyres)  
> wrote:
>>
>> > On Jul 21, 2019, at 11:31 AM, carlos aguni via users 
>> >  wrote:
>> >
>> > MPI_Isend()
>> > ... some stuff..
>> > flag = 0;
>> > MPI_Test(req, , );
>> > if (flag){
>> > free(buffer);
>> > }
>> >
>> > After the free() i'm getting errors like:
>> > [[58327,1],0][btl_tcp_frag.c:130:mca_btl_tcp_frag_send] 
>> > mca_btl_tcp_frag_send: writev error (0x2b9daf474000, 12800)
>> > Bad address(1)
>> > [[58327,1],0][btl_tcp_frag.c:130:mca_btl_tcp_frag_send] 
>> > mca_btl_tcp_frag_send: writev error (0x2b9daf473ee8, 19608)
>> > Bad address(1)
>> > pml_ob1_sendreq.c:308 FATAL
>>
>> Do you get the same error if you don't free()?
>>
>> --
>> Jeff Squyres
>> jsquy...@cisco.com
>>
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users


Re: [OMPI users] When is it save to free the buffer after MPI_Isend?

2019-07-27 Thread carlos aguni via users
Hi Jeff,

Thank you for your reply.

If i don't free the program completes but I'm not sure whether MPI_Isend
automatically frees the buffer after it sends the message. Does it?

I put a long sleep at the end to check the memory used using pmap.

The pmap command reported I'm using around 2GB which I'm guessing it isn't
freeing it.

Is there anything I could try?

Regards,
C.

On Mon, Jul 22, 2019 at 10:59 AM Jeff Squyres (jsquyres) 
wrote:

> > On Jul 21, 2019, at 11:31 AM, carlos aguni via users <
> users@lists.open-mpi.org> wrote:
> >
> > MPI_Isend()
> > ... some stuff..
> > flag = 0;
> > MPI_Test(req, , );
> > if (flag){
> > free(buffer);
> > }
> >
> > After the free() i'm getting errors like:
> > [[58327,1],0][btl_tcp_frag.c:130:mca_btl_tcp_frag_send]
> mca_btl_tcp_frag_send: writev error (0x2b9daf474000, 12800)
> > Bad address(1)
> > [[58327,1],0][btl_tcp_frag.c:130:mca_btl_tcp_frag_send]
> mca_btl_tcp_frag_send: writev error (0x2b9daf473ee8, 19608)
> > Bad address(1)
> > pml_ob1_sendreq.c:308 FATAL
>
> Do you get the same error if you don't free()?
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
>
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] When is it save to free the buffer after MPI_Isend?

2019-07-22 Thread Jeff Squyres (jsquyres) via users
> On Jul 21, 2019, at 11:31 AM, carlos aguni via users 
>  wrote:
> 
> MPI_Isend()
> ... some stuff..
> flag = 0;
> MPI_Test(req, , );
> if (flag){
> free(buffer);
> }
> 
> After the free() i'm getting errors like:
> [[58327,1],0][btl_tcp_frag.c:130:mca_btl_tcp_frag_send] 
> mca_btl_tcp_frag_send: writev error (0x2b9daf474000, 12800)
> Bad address(1)
> [[58327,1],0][btl_tcp_frag.c:130:mca_btl_tcp_frag_send] 
> mca_btl_tcp_frag_send: writev error (0x2b9daf473ee8, 19608)
> Bad address(1)
> pml_ob1_sendreq.c:308 FATAL

Do you get the same error if you don't free()?

-- 
Jeff Squyres
jsquy...@cisco.com

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users