Re: [OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[George Bosilca]

Or use MPI_Type_match_size to find the right type.

  George.

> On Jan 8, 2015, at 19:05 , Gus Correa  wrote:
> 
> Hi Diego
> 
> *EITHER*
> declare your QQ and PR (?) structure components as DOUBLE PRECISION
> *OR*
> keep them REAL(dp) but *fix* your "dp" definition, as George Bosilca 
> suggested.
> 
> Gus Correa
> 
> On 01/08/2015 06:36 PM, Diego Avesani wrote:
>> Dear Gus, Dear All,
>> so are you suggesting to use DOUBLE PRECISION and not REAL(dp)?
>> Thanks again
>> 
>> Diego
>> 
>> 
>> On 9 January 2015 at 00:02, Gus Correa > > wrote:
>> 
>>On 01/08/2015 05:50 PM, Diego Avesani wrote:
>> 
>>Dear George, Dear all,
>>what are the other issues?
>> 
>>Why did you put in selected_real_kind(15, 307) the number 307
>> 
>> 
>>Hi Diego
>> 
>>That is the Fortran 90 (and later) syntax for selected_real_kind.
>>The first number is the number of digits in the mantissa,
>>the second is the exponent range.
>>For (simpler) alternatives, see here:
>> 
>>http://fortranwiki.org/__fortran/show/Real+precision
>>
>> 
>>A lot of grief (and probably the segfault)
>>could have been saved if you just used
>>"DOUBLE PRECISION", instead of REAL in your
>>structure components declaration, as I suggested a while back.
>> 
>>I hope this helps,
>>Gus Correa
>> 
>> 
>>Thanks again
>> 
>>Diego
>> 
>> 
>>On 8 January 2015 at 23:24, George Bosilca >
>>>> wrote:
>> 
>> Diego,
>> 
>> Please find below the corrected example. There were several
>>issues
>> but the most important one, which is certainly the cause of the
>> segfault, is that "real(dp)" (with dp =
>>selected_real_kind(p=16)) is
>> NOT equal to MPI_DOUBLE_RECISION. For double precision you
>>should
>> use 15 (and not 16).
>> 
>>George.
>> 
>> 
>> On Thu, Jan 8, 2015 at 6:08 AM, Jeff Squyres (jsquyres)
>> 
>>>> wrote:
>> 
>> There were still some minor errors left over in the
>>attached
>> program.
>> 
>> I strongly encourage you to use "use mpi" instead of
>>"include
>> 'mpif.h'" because you will get compile time errors when
>>you pass
>> incorrect / forget to pass parameters to MPI
>>subroutines.  When
>> I switched your program to "use mpi", here's what the
>>compiler
>> showed me:
>> 
>> 1. the name "MPI" is reserved
>> 2. need to pass displacements(1:nstruct+1) to
>>mpi_type_create_struct
>> 3. need to pass request(1) to mpi_isend
>> 4. need to pass request(1) to mpi_wait
>> 5. need to pass ierr argument to mpi_wait
>> 
>> 1-4 are technically not correct, but the program would
>>likely
>> (usually) compile/run "correctly" anyway.  5 is
>>probably what
>> caused your segv.
>> 
>> Attached is my copy of your program with fixes for the
>> above-mentioned issues.
>> 
>> BTW, I missed the beginning of this thread -- I assume
>>that this
>> is an artificial use of mpi_type_create_resized for the
>>purposes
>> of a small example.  The specific use of it in this program
>> appears to be superfluous.
>> 
>> 
>> 
>> 
>> 
>> On Jan 8, 2015, at 4:26 AM, Gilles Gouaillardet
>> >
>> >>> wrote:
>> 
>> > Diego,
>> >
>> > yes, it works for me (at least with the v1.8 head and
>>gnu compilers)
>> >
>> > Cheers,
>> >
>> > Gilles
>> >
>> > On 2015/01/08 17:51, Diego Avesani wrote:
>> >> Dear Gilles,
>> >> thanks again, however it does not work.
>> >>
>> >> the program says:  "SIGSEGV, segmentation fault
>>occurred"
>> >>
>> >> Does the program run in your case?
>> >>
>> >> Thanks again
>> >>
>> >>
>> >>
>> >> Diego
>> >>
>> 

Re: [OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[Gus Correa]

Hi Diego

*EITHER*
declare your QQ and PR (?) structure components as DOUBLE PRECISION
*OR*
keep them REAL(dp) but *fix* your "dp" definition, as George Bosilca 
suggested.


Gus Correa

On 01/08/2015 06:36 PM, Diego Avesani wrote:

Dear Gus, Dear All,
so are you suggesting to use DOUBLE PRECISION and not REAL(dp)?
Thanks again

Diego


On 9 January 2015 at 00:02, Gus Correa > wrote:

On 01/08/2015 05:50 PM, Diego Avesani wrote:

Dear George, Dear all,
what are the other issues?

Why did you put in selected_real_kind(15, 307) the number 307


Hi Diego

That is the Fortran 90 (and later) syntax for selected_real_kind.
The first number is the number of digits in the mantissa,
the second is the exponent range.
For (simpler) alternatives, see here:

http://fortranwiki.org/__fortran/show/Real+precision


A lot of grief (and probably the segfault)
could have been saved if you just used
"DOUBLE PRECISION", instead of REAL in your
structure components declaration, as I suggested a while back.

I hope this helps,
Gus Correa


Thanks again

Diego


On 8 January 2015 at 23:24, George Bosilca 
>> wrote:

 Diego,

 Please find below the corrected example. There were several
issues
 but the most important one, which is certainly the cause of the
 segfault, is that "real(dp)" (with dp =
selected_real_kind(p=16)) is
 NOT equal to MPI_DOUBLE_RECISION. For double precision you
should
 use 15 (and not 16).

George.


 On Thu, Jan 8, 2015 at 6:08 AM, Jeff Squyres (jsquyres)
 
>> wrote:

 There were still some minor errors left over in the
attached
 program.

 I strongly encourage you to use "use mpi" instead of
"include
 'mpif.h'" because you will get compile time errors when
you pass
 incorrect / forget to pass parameters to MPI
subroutines.  When
 I switched your program to "use mpi", here's what the
compiler
 showed me:

 1. the name "MPI" is reserved
 2. need to pass displacements(1:nstruct+1) to
mpi_type_create_struct
 3. need to pass request(1) to mpi_isend
 4. need to pass request(1) to mpi_wait
 5. need to pass ierr argument to mpi_wait

 1-4 are technically not correct, but the program would
likely
 (usually) compile/run "correctly" anyway.  5 is
probably what
 caused your segv.

 Attached is my copy of your program with fixes for the
 above-mentioned issues.

 BTW, I missed the beginning of this thread -- I assume
that this
 is an artificial use of mpi_type_create_resized for the
purposes
 of a small example.  The specific use of it in this program
 appears to be superfluous.





 On Jan 8, 2015, at 4:26 AM, Gilles Gouaillardet
 
 >> wrote:

 > Diego,
 >
 > yes, it works for me (at least with the v1.8 head and
gnu compilers)
 >
 > Cheers,
 >
 > Gilles
 >
 > On 2015/01/08 17:51, Diego Avesani wrote:
 >> Dear Gilles,
 >> thanks again, however it does not work.
 >>
 >> the program says:  "SIGSEGV, segmentation fault
occurred"
 >>
 >> Does the program run in your case?
 >>
 >> Thanks again
 >>
 >>
 >>
 >> Diego
 >>
 >>
 >> On 8 January 2015 at 03:02, Gilles Gouaillardet <
 >>
 >>gilles.gouaillardet@iferc.__org

>
 >> > wrote:
 >>
 >>
 >>>  Diego,
 >>>
 >>> my bad, i should have 

Re: [OMPI users] send and receive vectors + variable length

[George Bosilca]

I'm confused by this statement. The examples pointed to are handling
blocking sends and receives, while this example is purely based on
non-blocking communications. In this particular case I see no hard of
waiting on the requests in any random order as long as all of them are
posted before the first wait.

  George.


On Thu, Jan 8, 2015 at 5:24 PM, Jeff Squyres (jsquyres) 
wrote:

> Also, you are calling WAITALL on all your sends and then WAITALL on all
> your receives.  This is also incorrect and may deadlock.
>
> WAITALL on *all* your pending requests (sends and receives -- put them all
> in a single array).
>
> Look at examples 3.8 and 3.9 in the MPI-3.0 document.
>
>
>
> On Jan 8, 2015, at 5:15 PM, George Bosilca  wrote:
>
> > Diego,
> >
> > Non-blocking communications only indicate a communication will happen,
> it does not force them to happen. They will only complete on the
> corresponding MPI_Wait, which also marks the moment starting from where the
> data can be safely altered or accessed (in the case of the MPI_Irecv). Thus
> deallocating your buffer after the MPI_Isend and MPI_Irecv is incorrect.
> Also printing the supposedly received values (line 127) is incorrect as
> there is no reason to have the non-blocking receive completed at that
> moment.
> >
> >   George.
> >
> >
> > On Thu, Jan 8, 2015 at 5:06 PM, Diego Avesani 
> wrote:
> > Dear Tom, Dear Jeff, Dear all,
> > Thanks again
> >
> > for Tom:
> > you are right, I fixed it.
> >
> > for Jeff:
> > if I do not insert the CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
> > in the line 112, the program does not stop.
> >
> > Am I right?
> > Here the new version
> >
> >
> >
> > Diego
> >
> >
> > On 8 January 2015 at 21:12, Tom Rosmond  wrote:
> > With array bounds checking your program returns an out-of-bounds error
> > in the mpi_isend call at line 104.  Looks like 'send_request' should be
> > indexed with 'sendcount', not 'icount'.
> >
> > T. Rosmond
> >
> >
> >
> > On Thu, 2015-01-08 at 20:28 +0100, Diego Avesani wrote:
> > > the attachment
> > >
> > > Diego
> > >
> > >
> > >
> > > On 8 January 2015 at 19:44, Diego Avesani 
> > > wrote:
> > > Dear all,
> > > I found the error.
> > > There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
> > > In the attachment there is the correct version of the program.
> > >
> > >
> > > Only one thing, could do you check if the use of MPI_WAITALL
> > > and MPI_BARRIER is correct?
> > >
> > >
> > > Thanks again
> > >
> > >
> > >
> > >
> > >
> > > Diego
> > >
> > >
> > >
> > > On 8 January 2015 at 18:48, Diego Avesani
> > >  wrote:
> > > Dear all,
> > > thanks thank a lot, I am learning a lot.
> > >
> > >
> > >
> > > I have written a simple program that send vectors of
> > > integers from a CPU to another.
> > >
> > >
> > > The program is written (at least for now) for 4 CPU.
> > >
> > >
> > > The program is quite simple:
> > > Each CPU knows how many data has to send to the other
> > > CPUs. This info is than send to the other CPUS. In
> > > this way each CPU knows how may data has to receive
> > > from other CPUs.
> > >
> > >
> > > This part of the program works.
> > >
> > >
> > > The problem is in the second part.
> > >
> > >
> > > In the second part, each processor sends a vector of
> > > integer to the other processor. The size is given and
> > > each CPU knows the size of the incoming vector form
> > > the first part of the program.
> > >
> > >
> > > In this second part the program fails and I do not
> > > know why.
> > >
> > >
> > > In the attachment you can find the program. Could you
> > > please help me. Problably I didn't understand properly
> > > the ISEND and IRECV subroutine.
> > >
> > >
> > > Thanks again
> > >
> > >
> > >
> > > Diego
> > >
> > >
> > >
> > >
> > >
> > >
> > > ___
> > > users mailing list
> > > us...@open-mpi.org
> > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> > > Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26131.php
> >
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> > Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26132.php
> >
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > Subscription: 

Re: [OMPI users] send and receive vectors + variable length

[Diego Avesani]

Dear Jeff, Dear George, Dear all,

Is not send_request a vector?
Are you suggesting to use  CALL MPI_WAIT(REQUEST(:), MPI_STATUS_IGNORE,
MPIdata%iErr)

I will try tomorrow morning, and also to fix the sending and receiving
allocate deallocate, Problaly I will have to think again to the program.
I will send to you a copy of the program tomorrow.

Thanks again


Diego


On 8 January 2015 at 23:24, Jeff Squyres (jsquyres) 
wrote:

> Also, you are calling WAITALL on all your sends and then WAITALL on all
> your receives.  This is also incorrect and may deadlock.
>
> WAITALL on *all* your pending requests (sends and receives -- put them all
> in a single array).
>
> Look at examples 3.8 and 3.9 in the MPI-3.0 document.
>
>
>
> On Jan 8, 2015, at 5:15 PM, George Bosilca  wrote:
>
> > Diego,
> >
> > Non-blocking communications only indicate a communication will happen,
> it does not force them to happen. They will only complete on the
> corresponding MPI_Wait, which also marks the moment starting from where the
> data can be safely altered or accessed (in the case of the MPI_Irecv). Thus
> deallocating your buffer after the MPI_Isend and MPI_Irecv is incorrect.
> Also printing the supposedly received values (line 127) is incorrect as
> there is no reason to have the non-blocking receive completed at that
> moment.
> >
> >   George.
> >
> >
> > On Thu, Jan 8, 2015 at 5:06 PM, Diego Avesani 
> wrote:
> > Dear Tom, Dear Jeff, Dear all,
> > Thanks again
> >
> > for Tom:
> > you are right, I fixed it.
> >
> > for Jeff:
> > if I do not insert the CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
> > in the line 112, the program does not stop.
> >
> > Am I right?
> > Here the new version
> >
> >
> >
> > Diego
> >
> >
> > On 8 January 2015 at 21:12, Tom Rosmond  wrote:
> > With array bounds checking your program returns an out-of-bounds error
> > in the mpi_isend call at line 104.  Looks like 'send_request' should be
> > indexed with 'sendcount', not 'icount'.
> >
> > T. Rosmond
> >
> >
> >
> > On Thu, 2015-01-08 at 20:28 +0100, Diego Avesani wrote:
> > > the attachment
> > >
> > > Diego
> > >
> > >
> > >
> > > On 8 January 2015 at 19:44, Diego Avesani 
> > > wrote:
> > > Dear all,
> > > I found the error.
> > > There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
> > > In the attachment there is the correct version of the program.
> > >
> > >
> > > Only one thing, could do you check if the use of MPI_WAITALL
> > > and MPI_BARRIER is correct?
> > >
> > >
> > > Thanks again
> > >
> > >
> > >
> > >
> > >
> > > Diego
> > >
> > >
> > >
> > > On 8 January 2015 at 18:48, Diego Avesani
> > >  wrote:
> > > Dear all,
> > > thanks thank a lot, I am learning a lot.
> > >
> > >
> > >
> > > I have written a simple program that send vectors of
> > > integers from a CPU to another.
> > >
> > >
> > > The program is written (at least for now) for 4 CPU.
> > >
> > >
> > > The program is quite simple:
> > > Each CPU knows how many data has to send to the other
> > > CPUs. This info is than send to the other CPUS. In
> > > this way each CPU knows how may data has to receive
> > > from other CPUs.
> > >
> > >
> > > This part of the program works.
> > >
> > >
> > > The problem is in the second part.
> > >
> > >
> > > In the second part, each processor sends a vector of
> > > integer to the other processor. The size is given and
> > > each CPU knows the size of the incoming vector form
> > > the first part of the program.
> > >
> > >
> > > In this second part the program fails and I do not
> > > know why.
> > >
> > >
> > > In the attachment you can find the program. Could you
> > > please help me. Problably I didn't understand properly
> > > the ISEND and IRECV subroutine.
> > >
> > >
> > > Thanks again
> > >
> > >
> > >
> > > Diego
> > >
> > >
> > >
> > >
> > >
> > >
> > > ___
> > > users mailing list
> > > us...@open-mpi.org
> > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> > > Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26131.php
> >
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> > Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26132.php
> >
> >
> > ___
> > users mailing 

Re: [OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[Diego Avesani]

Dear Gus, Dear All,
so are you suggesting to use DOUBLE PRECISION and not REAL(dp)?
Thanks again

Diego


On 9 January 2015 at 00:02, Gus Correa  wrote:

> On 01/08/2015 05:50 PM, Diego Avesani wrote:
>
>> Dear George, Dear all,
>> what are the other issues?
>>
>> Why did you put in selected_real_kind(15, 307) the number 307
>>
>>
> Hi Diego
>
> That is the Fortran 90 (and later) syntax for selected_real_kind.
> The first number is the number of digits in the mantissa,
> the second is the exponent range.
> For (simpler) alternatives, see here:
>
> http://fortranwiki.org/fortran/show/Real+precision
>
> A lot of grief (and probably the segfault)
> could have been saved if you just used
> "DOUBLE PRECISION", instead of REAL in your
> structure components declaration, as I suggested a while back.
>
> I hope this helps,
> Gus Correa
>
>
>  Thanks again
>>
>> Diego
>>
>>
>> On 8 January 2015 at 23:24, George Bosilca > > wrote:
>>
>> Diego,
>>
>> Please find below the corrected example. There were several issues
>> but the most important one, which is certainly the cause of the
>> segfault, is that "real(dp)" (with dp = selected_real_kind(p=16)) is
>> NOT equal to MPI_DOUBLE_RECISION. For double precision you should
>> use 15 (and not 16).
>>
>>George.
>>
>>
>> On Thu, Jan 8, 2015 at 6:08 AM, Jeff Squyres (jsquyres)
>> > wrote:
>>
>> There were still some minor errors left over in the attached
>> program.
>>
>> I strongly encourage you to use "use mpi" instead of "include
>> 'mpif.h'" because you will get compile time errors when you pass
>> incorrect / forget to pass parameters to MPI subroutines.  When
>> I switched your program to "use mpi", here's what the compiler
>> showed me:
>>
>> 1. the name "MPI" is reserved
>> 2. need to pass displacements(1:nstruct+1) to
>> mpi_type_create_struct
>> 3. need to pass request(1) to mpi_isend
>> 4. need to pass request(1) to mpi_wait
>> 5. need to pass ierr argument to mpi_wait
>>
>> 1-4 are technically not correct, but the program would likely
>> (usually) compile/run "correctly" anyway.  5 is probably what
>> caused your segv.
>>
>> Attached is my copy of your program with fixes for the
>> above-mentioned issues.
>>
>> BTW, I missed the beginning of this thread -- I assume that this
>> is an artificial use of mpi_type_create_resized for the purposes
>> of a small example.  The specific use of it in this program
>> appears to be superfluous.
>>
>>
>>
>>
>>
>> On Jan 8, 2015, at 4:26 AM, Gilles Gouaillardet
>> > > wrote:
>>
>> > Diego,
>> >
>> > yes, it works for me (at least with the v1.8 head and gnu
>> compilers)
>> >
>> > Cheers,
>> >
>> > Gilles
>> >
>> > On 2015/01/08 17:51, Diego Avesani wrote:
>> >> Dear Gilles,
>> >> thanks again, however it does not work.
>> >>
>> >> the program says:  "SIGSEGV, segmentation fault occurred"
>> >>
>> >> Does the program run in your case?
>> >>
>> >> Thanks again
>> >>
>> >>
>> >>
>> >> Diego
>> >>
>> >>
>> >> On 8 January 2015 at 03:02, Gilles Gouaillardet <
>> >>
>> >>gilles.gouaillar...@iferc.org > iferc.org>
>> >> > wrote:
>> >>
>> >>
>> >>>  Diego,
>> >>>
>> >>> my bad, i should have passed displacements(1) to
>> MPI_Type_create_struct
>> >>>
>> >>> here is an updated version
>> >>>
>> >>> (note you have to use a REQUEST integer for MPI_Isend and
>> MPI_Irecv,
>> >>> and you also have to call MPI_Wait to ensure the requests
>> complete)
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Gilles
>> >>>
>> >>>
>> >>> On 2015/01/08 8:23, Diego Avesani wrote:
>> >>>
>> >>> Dear Gilles, Dear all,
>> >>>
>> >>> I'm sorry to bother you again, but I have some problem with
>> send and
>> >>> receive the struct_data.
>> >>>
>> >>> I tried to send a MPI_Type_Create_Struct but I get a
>> segmentation fault
>> >>> occurred and I do not know why. The program is very simple,
>> it is the old
>> >>> one with the isend and irecv subroutines
>> >>>
>> >>> (you can find it in the attachment)
>> >>>
>> >>> Thanks again
>> >>>
>> >>>
>> >>> Diego
>> >>>
>> >>>
>> >>> On 5 January 2015 at 15:54, Diego Avesani
>> >>> 

Re: [OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[Gus Correa]

On 01/08/2015 05:50 PM, Diego Avesani wrote:

Dear George, Dear all,
what are the other issues?

Why did you put in selected_real_kind(15, 307) the number 307



Hi Diego

That is the Fortran 90 (and later) syntax for selected_real_kind.
The first number is the number of digits in the mantissa,
the second is the exponent range.
For (simpler) alternatives, see here:

http://fortranwiki.org/fortran/show/Real+precision

A lot of grief (and probably the segfault)
could have been saved if you just used
"DOUBLE PRECISION", instead of REAL in your
structure components declaration, as I suggested a while back.

I hope this helps,
Gus Correa



Thanks again

Diego


On 8 January 2015 at 23:24, George Bosilca > wrote:

Diego,

Please find below the corrected example. There were several issues
but the most important one, which is certainly the cause of the
segfault, is that "real(dp)" (with dp = selected_real_kind(p=16)) is
NOT equal to MPI_DOUBLE_RECISION. For double precision you should
use 15 (and not 16).

   George.


On Thu, Jan 8, 2015 at 6:08 AM, Jeff Squyres (jsquyres)
> wrote:

There were still some minor errors left over in the attached
program.

I strongly encourage you to use "use mpi" instead of "include
'mpif.h'" because you will get compile time errors when you pass
incorrect / forget to pass parameters to MPI subroutines.  When
I switched your program to "use mpi", here's what the compiler
showed me:

1. the name "MPI" is reserved
2. need to pass displacements(1:nstruct+1) to mpi_type_create_struct
3. need to pass request(1) to mpi_isend
4. need to pass request(1) to mpi_wait
5. need to pass ierr argument to mpi_wait

1-4 are technically not correct, but the program would likely
(usually) compile/run "correctly" anyway.  5 is probably what
caused your segv.

Attached is my copy of your program with fixes for the
above-mentioned issues.

BTW, I missed the beginning of this thread -- I assume that this
is an artificial use of mpi_type_create_resized for the purposes
of a small example.  The specific use of it in this program
appears to be superfluous.





On Jan 8, 2015, at 4:26 AM, Gilles Gouaillardet
> wrote:

> Diego,
>
> yes, it works for me (at least with the v1.8 head and gnu compilers)
>
> Cheers,
>
> Gilles
>
> On 2015/01/08 17:51, Diego Avesani wrote:
>> Dear Gilles,
>> thanks again, however it does not work.
>>
>> the program says:  "SIGSEGV, segmentation fault occurred"
>>
>> Does the program run in your case?
>>
>> Thanks again
>>
>>
>>
>> Diego
>>
>>
>> On 8 January 2015 at 03:02, Gilles Gouaillardet <
>>
>>gilles.gouaillar...@iferc.org 
>> > wrote:
>>
>>
>>>  Diego,
>>>
>>> my bad, i should have passed displacements(1) to 
MPI_Type_create_struct
>>>
>>> here is an updated version
>>>
>>> (note you have to use a REQUEST integer for MPI_Isend and MPI_Irecv,
>>> and you also have to call MPI_Wait to ensure the requests complete)
>>>
>>> Cheers,
>>>
>>> Gilles
>>>
>>>
>>> On 2015/01/08 8:23, Diego Avesani wrote:
>>>
>>> Dear Gilles, Dear all,
>>>
>>> I'm sorry to bother you again, but I have some problem with send and
>>> receive the struct_data.
>>>
>>> I tried to send a MPI_Type_Create_Struct but I get a segmentation 
fault
>>> occurred and I do not know why. The program is very simple, it is 
the old
>>> one with the isend and irecv subroutines
>>>
>>> (you can find it in the attachment)
>>>
>>> Thanks again
>>>
>>>
>>> Diego
>>>
>>>
>>> On 5 January 2015 at 15:54, Diego Avesani
>>> >
>
>>>  wrote:
>>>
>>>
>>>  Dear Gilles,
>>>
>>> Thanks, Thanks a lot.
>>> Now is more clear.
>>>
>>> Again, thanks a lot
>>>
>>> Diego
>>>
>>>
>>>
>>>
>>>
>>> ___
>>> users mailing
>>>listus...@open-mpi.org 
>>>
>>> 

Re: [OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[Diego Avesani]

Dear George, Dear all,
what are the other issues?

Why did you put in selected_real_kind(15, 307) the number 307

Thanks again

Diego


On 8 January 2015 at 23:24, George Bosilca  wrote:

> Diego,
>
> Please find below the corrected example. There were several issues but the
> most important one, which is certainly the cause of the segfault, is that
> "real(dp)" (with dp = selected_real_kind(p=16)) is NOT equal to
> MPI_DOUBLE_RECISION. For double precision you should use 15 (and not 16).
>
>   George.
>
>
> On Thu, Jan 8, 2015 at 6:08 AM, Jeff Squyres (jsquyres) <
> jsquy...@cisco.com> wrote:
>
>>  There were still some minor errors left over in the attached program.
>>
>> I strongly encourage you to use "use mpi" instead of "include 'mpif.h'"
>> because you will get compile time errors when you pass incorrect / forget
>> to pass parameters to MPI subroutines.  When I switched your program to
>> "use mpi", here's what the compiler showed me:
>>
>> 1. the name "MPI" is reserved
>> 2. need to pass displacements(1:nstruct+1) to mpi_type_create_struct
>> 3. need to pass request(1) to mpi_isend
>> 4. need to pass request(1) to mpi_wait
>> 5. need to pass ierr argument to mpi_wait
>>
>> 1-4 are technically not correct, but the program would likely (usually)
>> compile/run "correctly" anyway.  5 is probably what caused your segv.
>>
>> Attached is my copy of your program with fixes for the above-mentioned
>> issues.
>>
>> BTW, I missed the beginning of this thread -- I assume that this is an
>> artificial use of mpi_type_create_resized for the purposes of a small
>> example.  The specific use of it in this program appears to be superfluous.
>>
>>
>>
>>
>>
>> On Jan 8, 2015, at 4:26 AM, Gilles Gouaillardet <
>> gilles.gouaillar...@iferc.org> wrote:
>>
>> > Diego,
>> >
>> > yes, it works for me (at least with the v1.8 head and gnu compilers)
>> >
>> > Cheers,
>> >
>> > Gilles
>> >
>> > On 2015/01/08 17:51, Diego Avesani wrote:
>> >> Dear Gilles,
>> >> thanks again, however it does not work.
>> >>
>> >> the program says:  "SIGSEGV, segmentation fault occurred"
>> >>
>> >> Does the program run in your case?
>> >>
>> >> Thanks again
>> >>
>> >>
>> >>
>> >> Diego
>> >>
>> >>
>> >> On 8 January 2015 at 03:02, Gilles Gouaillardet <
>> >>
>> >> gilles.gouaillar...@iferc.org
>> >> > wrote:
>> >>
>> >>
>> >>>  Diego,
>> >>>
>> >>> my bad, i should have passed displacements(1) to
>> MPI_Type_create_struct
>> >>>
>> >>> here is an updated version
>> >>>
>> >>> (note you have to use a REQUEST integer for MPI_Isend and MPI_Irecv,
>> >>> and you also have to call MPI_Wait to ensure the requests complete)
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Gilles
>> >>>
>> >>>
>> >>> On 2015/01/08 8:23, Diego Avesani wrote:
>> >>>
>> >>> Dear Gilles, Dear all,
>> >>>
>> >>> I'm sorry to bother you again, but I have some problem with send and
>> >>> receive the struct_data.
>> >>>
>> >>> I tried to send a MPI_Type_Create_Struct but I get a segmentation
>> fault
>> >>> occurred and I do not know why. The program is very simple, it is the
>> old
>> >>> one with the isend and irecv subroutines
>> >>>
>> >>> (you can find it in the attachment)
>> >>>
>> >>> Thanks again
>> >>>
>> >>>
>> >>> Diego
>> >>>
>> >>>
>> >>> On 5 January 2015 at 15:54, Diego Avesani
>> >>>  
>> >>>  wrote:
>> >>>
>> >>>
>> >>>  Dear Gilles,
>> >>>
>> >>> Thanks, Thanks a lot.
>> >>> Now is more clear.
>> >>>
>> >>> Again, thanks a lot
>> >>>
>> >>> Diego
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ___
>> >>> users mailing
>> >>> listus...@open-mpi.org
>> >>>
>> >>> Subscription:
>> >>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> >>>
>> >>> Link to this post:
>> >>> http://www.open-mpi.org/community/lists/users/2015/01/26116.php
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ___
>> >>> users mailing list
>> >>>
>> >>> us...@open-mpi.org
>> >>>
>> >>> Subscription:
>> >>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> >>>
>> >>> Link to this post:
>> >>>
>> >>> http://www.open-mpi.org/community/lists/users/2015/01/26118.php
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >> ___
>> >> users mailing list
>> >>
>> >> us...@open-mpi.org
>> >>
>> >> Subscription:
>> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> >>
>> >> Link to this post:
>> >> http://www.open-mpi.org/community/lists/users/2015/01/26124.php
>> >
>> > ___
>> > users mailing list
>> > us...@open-mpi.org
>> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> > Link to this post:
>> http://www.open-mpi.org/community/lists/users/2015/01/26126.php
>>
>>
>> --
>> Jeff Squyres
>> jsquy...@cisco.com
>> For corporate legal information go to:
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>>
>>
>> ___
>> 

Re: [OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[George Bosilca]

Diego,

Please find below the corrected example. There were several issues but the
most important one, which is certainly the cause of the segfault, is that
"real(dp)" (with dp = selected_real_kind(p=16)) is NOT equal to
MPI_DOUBLE_RECISION. For double precision you should use 15 (and not 16).

  George.


On Thu, Jan 8, 2015 at 6:08 AM, Jeff Squyres (jsquyres) 
wrote:

>  There were still some minor errors left over in the attached program.
>
> I strongly encourage you to use "use mpi" instead of "include 'mpif.h'"
> because you will get compile time errors when you pass incorrect / forget
> to pass parameters to MPI subroutines.  When I switched your program to
> "use mpi", here's what the compiler showed me:
>
> 1. the name "MPI" is reserved
> 2. need to pass displacements(1:nstruct+1) to mpi_type_create_struct
> 3. need to pass request(1) to mpi_isend
> 4. need to pass request(1) to mpi_wait
> 5. need to pass ierr argument to mpi_wait
>
> 1-4 are technically not correct, but the program would likely (usually)
> compile/run "correctly" anyway.  5 is probably what caused your segv.
>
> Attached is my copy of your program with fixes for the above-mentioned
> issues.
>
> BTW, I missed the beginning of this thread -- I assume that this is an
> artificial use of mpi_type_create_resized for the purposes of a small
> example.  The specific use of it in this program appears to be superfluous.
>
>
>
>
>
> On Jan 8, 2015, at 4:26 AM, Gilles Gouaillardet <
> gilles.gouaillar...@iferc.org> wrote:
>
> > Diego,
> >
> > yes, it works for me (at least with the v1.8 head and gnu compilers)
> >
> > Cheers,
> >
> > Gilles
> >
> > On 2015/01/08 17:51, Diego Avesani wrote:
> >> Dear Gilles,
> >> thanks again, however it does not work.
> >>
> >> the program says:  "SIGSEGV, segmentation fault occurred"
> >>
> >> Does the program run in your case?
> >>
> >> Thanks again
> >>
> >>
> >>
> >> Diego
> >>
> >>
> >> On 8 January 2015 at 03:02, Gilles Gouaillardet <
> >>
> >> gilles.gouaillar...@iferc.org
> >> > wrote:
> >>
> >>
> >>>  Diego,
> >>>
> >>> my bad, i should have passed displacements(1) to MPI_Type_create_struct
> >>>
> >>> here is an updated version
> >>>
> >>> (note you have to use a REQUEST integer for MPI_Isend and MPI_Irecv,
> >>> and you also have to call MPI_Wait to ensure the requests complete)
> >>>
> >>> Cheers,
> >>>
> >>> Gilles
> >>>
> >>>
> >>> On 2015/01/08 8:23, Diego Avesani wrote:
> >>>
> >>> Dear Gilles, Dear all,
> >>>
> >>> I'm sorry to bother you again, but I have some problem with send and
> >>> receive the struct_data.
> >>>
> >>> I tried to send a MPI_Type_Create_Struct but I get a segmentation fault
> >>> occurred and I do not know why. The program is very simple, it is the
> old
> >>> one with the isend and irecv subroutines
> >>>
> >>> (you can find it in the attachment)
> >>>
> >>> Thanks again
> >>>
> >>>
> >>> Diego
> >>>
> >>>
> >>> On 5 January 2015 at 15:54, Diego Avesani
> >>>  
> >>>  wrote:
> >>>
> >>>
> >>>  Dear Gilles,
> >>>
> >>> Thanks, Thanks a lot.
> >>> Now is more clear.
> >>>
> >>> Again, thanks a lot
> >>>
> >>> Diego
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ___
> >>> users mailing
> >>> listus...@open-mpi.org
> >>>
> >>> Subscription:
> >>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>
> >>> Link to this post:
> >>> http://www.open-mpi.org/community/lists/users/2015/01/26116.php
> >>>
> >>>
> >>>
> >>>
> >>> ___
> >>> users mailing list
> >>>
> >>> us...@open-mpi.org
> >>>
> >>> Subscription:
> >>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>
> >>> Link to this post:
> >>>
> >>> http://www.open-mpi.org/community/lists/users/2015/01/26118.php
> >>>
> >>>
> >>>
> >>
> >>
> >> ___
> >> users mailing list
> >>
> >> us...@open-mpi.org
> >>
> >> Subscription:
> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>
> >> Link to this post:
> >> http://www.open-mpi.org/community/lists/users/2015/01/26124.php
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> > Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26126.php
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26127.php
>


test_struct_test_isend.f90
Description: Binary data

Re: [OMPI users] send and receive vectors + variable length

[Jeff Squyres (jsquyres)]

Also, you are calling WAITALL on all your sends and then WAITALL on all your 
receives.  This is also incorrect and may deadlock.

WAITALL on *all* your pending requests (sends and receives -- put them all in a 
single array).

Look at examples 3.8 and 3.9 in the MPI-3.0 document.



On Jan 8, 2015, at 5:15 PM, George Bosilca  wrote:

> Diego,
> 
> Non-blocking communications only indicate a communication will happen, it 
> does not force them to happen. They will only complete on the corresponding 
> MPI_Wait, which also marks the moment starting from where the data can be 
> safely altered or accessed (in the case of the MPI_Irecv). Thus deallocating 
> your buffer after the MPI_Isend and MPI_Irecv is incorrect. Also printing the 
> supposedly received values (line 127) is incorrect as there is no reason to 
> have the non-blocking receive completed at that moment.
> 
>   George.
> 
> 
> On Thu, Jan 8, 2015 at 5:06 PM, Diego Avesani  wrote:
> Dear Tom, Dear Jeff, Dear all,
> Thanks again
> 
> for Tom:
> you are right, I fixed it.
> 
> for Jeff:
> if I do not insert the CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr) 
> in the line 112, the program does not stop.
> 
> Am I right?
> Here the new version
> 
> 
> 
> Diego
> 
> 
> On 8 January 2015 at 21:12, Tom Rosmond  wrote:
> With array bounds checking your program returns an out-of-bounds error
> in the mpi_isend call at line 104.  Looks like 'send_request' should be
> indexed with 'sendcount', not 'icount'.
> 
> T. Rosmond
> 
> 
> 
> On Thu, 2015-01-08 at 20:28 +0100, Diego Avesani wrote:
> > the attachment
> >
> > Diego
> >
> >
> >
> > On 8 January 2015 at 19:44, Diego Avesani 
> > wrote:
> > Dear all,
> > I found the error.
> > There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
> > In the attachment there is the correct version of the program.
> >
> >
> > Only one thing, could do you check if the use of MPI_WAITALL
> > and MPI_BARRIER is correct?
> >
> >
> > Thanks again
> >
> >
> >
> >
> >
> > Diego
> >
> >
> >
> > On 8 January 2015 at 18:48, Diego Avesani
> >  wrote:
> > Dear all,
> > thanks thank a lot, I am learning a lot.
> >
> >
> >
> > I have written a simple program that send vectors of
> > integers from a CPU to another.
> >
> >
> > The program is written (at least for now) for 4 CPU.
> >
> >
> > The program is quite simple:
> > Each CPU knows how many data has to send to the other
> > CPUs. This info is than send to the other CPUS. In
> > this way each CPU knows how may data has to receive
> > from other CPUs.
> >
> >
> > This part of the program works.
> >
> >
> > The problem is in the second part.
> >
> >
> > In the second part, each processor sends a vector of
> > integer to the other processor. The size is given and
> > each CPU knows the size of the incoming vector form
> > the first part of the program.
> >
> >
> > In this second part the program fails and I do not
> > know why.
> >
> >
> > In the attachment you can find the program. Could you
> > please help me. Problably I didn't understand properly
> > the ISEND and IRECV subroutine.
> >
> >
> > Thanks again
> >
> >
> >
> > Diego
> >
> >
> >
> >
> >
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> > Link to this post: 
> > http://www.open-mpi.org/community/lists/users/2015/01/26131.php
> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26132.php
> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26137.php
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26138.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] send and receive vectors + variable length

[George Bosilca]

Diego,

Non-blocking communications only indicate a communication will happen, it
does not force them to happen. They will only complete on the corresponding
MPI_Wait, which also marks the moment starting from where the data can be
safely altered or accessed (in the case of the MPI_Irecv). Thus
deallocating your buffer after the MPI_Isend and MPI_Irecv is incorrect.
Also printing the supposedly received values (line 127) is incorrect as
there is no reason to have the non-blocking receive completed at that
moment.

  George.


On Thu, Jan 8, 2015 at 5:06 PM, Diego Avesani 
wrote:

> Dear Tom, Dear Jeff, Dear all,
> Thanks again
>
> for Tom:
> you are right, I fixed it.
>
> for Jeff:
> if I do not insert the CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
> in the line 112, the program does not stop.
>
> Am I right?
> Here the new version
>
>
>
> Diego
>
>
> On 8 January 2015 at 21:12, Tom Rosmond  wrote:
>
>> With array bounds checking your program returns an out-of-bounds error
>> in the mpi_isend call at line 104.  Looks like 'send_request' should be
>> indexed with 'sendcount', not 'icount'.
>>
>> T. Rosmond
>>
>>
>>
>> On Thu, 2015-01-08 at 20:28 +0100, Diego Avesani wrote:
>> > the attachment
>> >
>> > Diego
>> >
>> >
>> >
>> > On 8 January 2015 at 19:44, Diego Avesani 
>> > wrote:
>> > Dear all,
>> > I found the error.
>> > There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
>> > In the attachment there is the correct version of the program.
>> >
>> >
>> > Only one thing, could do you check if the use of MPI_WAITALL
>> > and MPI_BARRIER is correct?
>> >
>> >
>> > Thanks again
>> >
>> >
>> >
>> >
>> >
>> > Diego
>> >
>> >
>> >
>> > On 8 January 2015 at 18:48, Diego Avesani
>> >  wrote:
>> > Dear all,
>> > thanks thank a lot, I am learning a lot.
>> >
>> >
>> >
>> > I have written a simple program that send vectors of
>> > integers from a CPU to another.
>> >
>> >
>> > The program is written (at least for now) for 4 CPU.
>> >
>> >
>> > The program is quite simple:
>> > Each CPU knows how many data has to send to the other
>> > CPUs. This info is than send to the other CPUS. In
>> > this way each CPU knows how may data has to receive
>> > from other CPUs.
>> >
>> >
>> > This part of the program works.
>> >
>> >
>> > The problem is in the second part.
>> >
>> >
>> > In the second part, each processor sends a vector of
>> > integer to the other processor. The size is given and
>> > each CPU knows the size of the incoming vector form
>> > the first part of the program.
>> >
>> >
>> > In this second part the program fails and I do not
>> > know why.
>> >
>> >
>> > In the attachment you can find the program. Could you
>> > please help me. Problably I didn't understand properly
>> > the ISEND and IRECV subroutine.
>> >
>> >
>> > Thanks again
>> >
>> >
>> >
>> > Diego
>> >
>> >
>> >
>> >
>> >
>> >
>> > ___
>> > users mailing list
>> > us...@open-mpi.org
>> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> > Link to this post:
>> http://www.open-mpi.org/community/lists/users/2015/01/26131.php
>>
>>
>> ___
>> users mailing list
>> us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post:
>> http://www.open-mpi.org/community/lists/users/2015/01/26132.php
>>
>
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26137.php
>

Re: [OMPI users] send and receive vectors + variable length

[Diego Avesani]

Dear Tom, Dear Jeff, Dear all,
Thanks again

for Tom:
you are right, I fixed it.

for Jeff:
if I do not insert the CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
in the line 112, the program does not stop.

Am I right?
Here the new version



Diego


On 8 January 2015 at 21:12, Tom Rosmond  wrote:

> With array bounds checking your program returns an out-of-bounds error
> in the mpi_isend call at line 104.  Looks like 'send_request' should be
> indexed with 'sendcount', not 'icount'.
>
> T. Rosmond
>
>
>
> On Thu, 2015-01-08 at 20:28 +0100, Diego Avesani wrote:
> > the attachment
> >
> > Diego
> >
> >
> >
> > On 8 January 2015 at 19:44, Diego Avesani 
> > wrote:
> > Dear all,
> > I found the error.
> > There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
> > In the attachment there is the correct version of the program.
> >
> >
> > Only one thing, could do you check if the use of MPI_WAITALL
> > and MPI_BARRIER is correct?
> >
> >
> > Thanks again
> >
> >
> >
> >
> >
> > Diego
> >
> >
> >
> > On 8 January 2015 at 18:48, Diego Avesani
> >  wrote:
> > Dear all,
> > thanks thank a lot, I am learning a lot.
> >
> >
> >
> > I have written a simple program that send vectors of
> > integers from a CPU to another.
> >
> >
> > The program is written (at least for now) for 4 CPU.
> >
> >
> > The program is quite simple:
> > Each CPU knows how many data has to send to the other
> > CPUs. This info is than send to the other CPUS. In
> > this way each CPU knows how may data has to receive
> > from other CPUs.
> >
> >
> > This part of the program works.
> >
> >
> > The problem is in the second part.
> >
> >
> > In the second part, each processor sends a vector of
> > integer to the other processor. The size is given and
> > each CPU knows the size of the incoming vector form
> > the first part of the program.
> >
> >
> > In this second part the program fails and I do not
> > know why.
> >
> >
> > In the attachment you can find the program. Could you
> > please help me. Problably I didn't understand properly
> > the ISEND and IRECV subroutine.
> >
> >
> > Thanks again
> >
> >
> >
> > Diego
> >
> >
> >
> >
> >
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> > Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26131.php
>
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26132.php
>
MODULE MOD_PRECISION
  integer, parameter :: dp = selected_real_kind(p=16)
ENDMODULE MOD_PRECISION


PROGRAM TEST
USE MOD_PRECISION
USE MPI
IMPLICIT NONE
  !
  INTEGER :: iCPU,icount,iTag,ip,sendcount,sendrecv,i
  !
  TYPE tMPI
 INTEGER  :: myrank, nCPU, iErr, status
  END TYPE tMPI
  !
  INTEGER, ALLOCATABLE,DIMENSION(:) :: Ndata2send
  INTEGER, ALLOCATABLE,DIMENSION(:) :: Ndata2recv
  
  INTEGER, ALLOCATABLE,DIMENSION(:) :: DATASEND
  INTEGER, ALLOCATABLE,DIMENSION(:) :: DATARECV
  !
  INTEGER :: send_request (3)
  INTEGER :: recv_request (3)
  INTEGER :: send_status_list (MPI_STATUS_SIZE,3) 
  INTEGER :: recv_status_list (MPI_STATUS_SIZE,3)  
  !
  TYPE(tMPI) :: MPIdata

  CALL MPI_INIT(MPIdata%iErr)
  CALL MPI_COMM_RANK(MPI_COMM_WORLD, MPIdata%myrank, MPIdata%iErr)
  CALL MPI_COMM_SIZE(MPI_COMM_WORLD, MPIdata%nCPU,   MPIdata%iErr)
  !
  ALLOCATE(Ndata2send(0:MPIdata%nCPU-1))
  ALLOCATE(Ndata2recv(0:MPIdata%nCPU-1))
  !
  Ndata2send=0
  Ndata2recv=0
  !
  IF(MPIdata%myrank==0)THEN
 Ndata2send(0)=0
 Ndata2send(1)=100
 Ndata2send(2)=100
 Ndata2send(3)=160
  ENDIF
IF(MPIdata%myrank==1)THEN
 Ndata2send(0)=130
 Ndata2send(1)=0
 Ndata2send(2)=160
 Ndata2send(3)=160
  ENDIF
IF(MPIdata%myrank==2)THEN
 Ndata2send(0)=130
 Ndata2send(1)=160
 Ndata2send(2)=0
 Ndata2send(3)=160
  ENDIF
IF(MPIdata%myrank==3)THEN
 Ndata2send(0)=160
 Ndata2send(1)=160
 Ndata2send(2)=160
 Ndata2send(3)=0
  ENDIF
  !
  CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
  
  
  icount=1
  DO iCPU=0,MPIdata%nCPU-1
 IF(iCPU.NE.MPIdata%myrank)THEN
iTag=iCPU
CALL MPI_ISEND(Ndata2send(iCPU),1,MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,send_request(icount),MPIdata%iErr)
icount=icount+1
 ENDIF
  

Re: [OMPI users] libpsm_infinipath issues?

[Gus Correa]

Hi Michael, Andrew, list

knem is doesn't work in OMPI 1.8.3.
See this thread:
http://www.open-mpi.org/community/lists/users/2014/10/25511.php

A fix was promised on OMPI 1.8.4:
http://www.open-mpi.org/software/ompi/v1.8/

Have you tried it?

I hope this helps,
Gus Correa

On 01/08/2015 04:36 PM, Friedley, Andrew wrote:

Hi Mike,

Have you contacted your admins, or the vendor that provided your True
Scale and/or PSM installation? E.g. Redhat, or Intel via
ibsupp...@intel.com ?  They are normally the
recommended path for True Scale support.

That said, here’s some things to look into:

I think it’s strange that you’re hitting PSM linker problems while
compiling an app with Open MPI.  Normally only the PSM MTL is linked
directly against PSM.  Try verifying that nothing else is linking in
PSM. Also, it’s possible your OMPI build is different from normal too.

PSM has optional KNEM support that can be compiled in, but it’s off by
default and I’ve never heard of it being used.  It’s possible that your
PSM was changed or recompiled and part of your cluster upgrade, and the
installation isn’t quite right.  It looks like your PSM was compiled
with KNEM support, but the KNEM library isn’t being linked in or is not
being found.

Andrew

*From:* users [mailto:users-boun...@open-mpi.org] *On Behalf Of
*VanEss.Michael
*Sent:* Thursday, January 8, 2015 1:07 PM
*To:* us...@open-mpi.org
*Subject:* [OMPI users] libpsm_infinipath issues?

Hello all,

Our clusters were just upgraded to both a new version of PGI (14.9) as
well as openmpi (1.8.3).  Previous versions were 12.1 and 1.6
respectively, and those compiled and linked just fine.  The newest
versions are not linking my mpi applications at all.  Here’s the problem:

/opt/scyld/openmpi/1.8.3/pgi/bin/mpif90 -C chemcode_mpi.o mod_mpichem.o
plume_mpi.o amb2D_mpi.o fex.o jex.o use_tuv.o run_lsode.o mod_amb.o
utmgeo.o lsode.o newamb.o plume.o amb2D.o solve.o mod_cdf.o calc_rates.o
mod_tuv.o flux.o amb1D.o amb_com.o newamb2D.o vs_ccode.o ck_errors.o
newamb1D.o doChem.o mod_lsode.o stode.o plume_com.o typeSizes.o netcdf.o
mod_parameters.o mod_chem.o runfile.o com_codes.o mod_SLAM.o mod_CPUFF.o
calc_za.o mod_releases.o mod_site.o mod_distance.o nuclear_dates.o
mod_luparms.o deposition.o diffusion.o getTJ.o mod_met.o met_data.o
mod_h5tuv.o tuv10.o mod_h5camx.o cmxamb.o \

 -L/ensco/apps/cm/CMSOL/linux/zlib-1.2.1/lib
-L/ensco/apps/cm/CMSOL/linux/szip-2.1/lib -o \

 chemcode_mpi  -L/opt/pgi/linux86-64/14.9/lib -lpgf90 -lpgf90rtl \

 -L/ensco/apps/cm/CMSOL/linux/hdf5-1.8.3/lib -lhdf5_fortran -l
hdf5 -lz -lm

/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_put'

/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_open_device'

/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_get'

/usr/lib64/libpsm_infinipath.so.1: undefined reference to
`knem_register_region'

I’ve searched the net for any information on this and nothing has seemed
to help.  I’m fairly confident that all my variables and paths to the
new software is correct.

Any ideas?

Regards,

Mike



The information contained in this email message is intended only for the
use of the individual(s) to whom it is addressed and may contain
information that is privileged and sensitive. If you are not the
intended recipient, or otherwise have received this communication in
error, please notify the sender immediately by email at the above
referenced address and note that any further dissemination, distribution
or copying of this communication is strictly prohibited.

The U.S. Export Control Laws regulate the export and re-export of
technology originating in the United States. This includes the
electronic transmission of information and software to foreign countries
and to certain foreign nationals. Recipient agrees to abide by these
laws and their regulations -- including the U.S. Department of Commerce
Export Administration Regulations and the U.S. Department of State
International Traffic in Arms Regulations -- and not to transfer, by
electronic transmission or otherwise, any content derived from this
email to either a foreign national or a foreign destination in violation
of such laws.



___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2015/01/26135.php

Re: [OMPI users] libpsm_infinipath issues?

[Friedley, Andrew]

Hi Mike,

Have you contacted your admins, or the vendor that provided your True Scale 
and/or PSM installation? E.g. Redhat, or Intel via 
ibsupp...@intel.com?  They are normally the 
recommended path for True Scale support.

That said, here's some things to look into:

I think it's strange that you're hitting PSM linker problems while compiling an 
app with Open MPI.  Normally only the PSM MTL is linked directly against PSM.  
Try verifying that nothing else is linking in PSM.  Also, it's possible your 
OMPI build is different from normal too.

PSM has optional KNEM support that can be compiled in, but it's off by default 
and I've never heard of it being used.  It's possible that your PSM was changed 
or recompiled and part of your cluster upgrade, and the installation isn't 
quite right.  It looks like your PSM was compiled with KNEM support, but the 
KNEM library isn't being linked in or is not being found.

Andrew

From: users [mailto:users-boun...@open-mpi.org] On Behalf Of VanEss.Michael
Sent: Thursday, January 8, 2015 1:07 PM
To: us...@open-mpi.org
Subject: [OMPI users] libpsm_infinipath issues?

Hello all,

Our clusters were just upgraded to both a new version of PGI (14.9) as well as 
openmpi (1.8.3).  Previous versions were 12.1 and 1.6 respectively, and those 
compiled and linked just fine.  The newest versions are not linking my mpi 
applications at all.  Here's the problem:

/opt/scyld/openmpi/1.8.3/pgi/bin/mpif90 -C chemcode_mpi.o mod_mpichem.o 
plume_mpi.o amb2D_mpi.o fex.o jex.o use_tuv.o run_lsode.o mod_amb.o utmgeo.o 
lsode.o newamb.o plume.o amb2D.o solve.o mod_cdf.o calc_rates.o mod_tuv.o 
flux.o amb1D.o amb_com.o newamb2D.o vs_ccode.o ck_errors.o newamb1D.o doChem.o 
mod_lsode.o stode.o plume_com.o typeSizes.o netcdf.o mod_parameters.o 
mod_chem.o runfile.o com_codes.o mod_SLAM.o mod_CPUFF.o calc_za.o 
mod_releases.o mod_site.o mod_distance.o nuclear_dates.o mod_luparms.o 
deposition.o diffusion.o getTJ.o mod_met.o met_data.o mod_h5tuv.o tuv10.o 
mod_h5camx.o cmxamb.o \
-L/ensco/apps/cm/CMSOL/linux/zlib-1.2.1/lib 
-L/ensco/apps/cm/CMSOL/linux/szip-2.1/lib -o \
chemcode_mpi  -L/opt/pgi/linux86-64/14.9/lib -lpgf90 -lpgf90rtl \
-L/ensco/apps/cm/CMSOL/linux/hdf5-1.8.3/lib -lhdf5_fortran -l hdf5 -lz 
-lm
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_put'
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_open_device'
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_get'
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_register_region'

I've searched the net for any information on this and nothing has seemed to 
help.  I'm fairly confident that all my variables and paths to the new software 
is correct.

Any ideas?

Regards,

Mike



The information contained in this email message is intended only for the use of 
the individual(s) to whom it is addressed and may contain information that is 
privileged and sensitive. If you are not the intended recipient, or otherwise 
have received this communication in error, please notify the sender immediately 
by email at the above referenced address and note that any further 
dissemination, distribution or copying of this communication is strictly 
prohibited.

The U.S. Export Control Laws regulate the export and re-export of technology 
originating in the United States. This includes the electronic transmission of 
information and software to foreign countries and to certain foreign nationals. 
Recipient agrees to abide by these laws and their regulations -- including the 
U.S. Department of Commerce Export Administration Regulations and the U.S. 
Department of State International Traffic in Arms Regulations -- and not to 
transfer, by electronic transmission or otherwise, any content derived from 
this email to either a foreign national or a foreign destination in violation 
of such laws.

[OMPI users] -fgnu89-inline needed to avoid "multiple definition of `lstat64'" error

[Jesse Ziser]

Hello,

When building OpenMPI 1.8.4 on Linux using gcc 4.8.2, the build fails for me 
with errors like:


romio/.libs/libromio_dist.a(delete.o): In function `lstat64':
delete.c:(.text+0x0): multiple definition of `lstat64'
romio/.libs/libromio_dist.a(close.o):close.c:(.text+0x0): first defined here
romio/.libs/libromio_dist.a(fsync.o): In function `lstat64':
fsync.c:(.text+0x0): multiple definition of `lstat64'
romio/.libs/libromio_dist.a(close.o):close.c:(.text+0x0): first defined here
romio/.libs/libromio_dist.a(get_amode.o): In function `lstat64':
get_amode.c:(.text+0x0): multiple definition of `lstat64'
romio/.libs/libromio_dist.a(close.o):close.c:(.text+0x0): first defined here

and so on for a bunch more files.  I think I (vaguely) understand the problem. 
 I'm not sure whether a "fix" is possible or appropriate, but I'm reporting it 
anyway since I've tracked it down.


It builds perfectly fine with gcc 4.2.0.  It also builds perfectly fine with 
gcc 4.8.2 if I supply CFLAGS=-fgnu89-inline on the ./configure command line 
(and that's our current workaround).  I hope this is safe to do.  The reason I 
tried using this flag was because of the following vaguely related-looking 
pages Google was kind enough to show me:


http://lifecs.likai.org/2009/06/multiple-definition-of-extern-inline.html
https://gcc.gnu.org/gcc-4.3/porting_to.html

These pages made it sound like the "multiple definition" errors are because of 
"extern inline" declarations being used somewhere and the fact that OpenMPI 
apparently uses -std=c99.  -fgnu89-inline simply requests the old pre-gcc-4.3 
GNU-specific interpretation of "extern inline", instead of the c99 
interpretation, so the lstat64() function doesn't wind up in your object files.


My best guess is that it has to do with the fact that we're using a modern 
compiler, but on a relatively old system (Red Hat Enterprise Linux 5.11) so 
lstat64() is being declared extern inline because the old system include files 
don't expect the new gcc behavior.  However, we haven't seen this before when 
building other packages, just OpenMPI.


Perhaps there is a way to detect this and include the flag automatically.  I 
noticed one other message in the archives from someone else having this 
problem, but it was a while back and there were no replies.  So perhaps this 
would be helpful to others.


Just to be thorough, I've attached the configure and make output for the case 
that triggers the issue.  I'm configuring with:


  ./configure --prefix=/opt/depot/openmpi/1.8.4 CC=gcc CXX=g++ FC=gfortran

Thank you,

Jesse Ziser
Applied Research Laboratories:
The University of Texas at Austin


ompi-output.tar.bz2
Description: Binary data


smime.p7s
Description: S/MIME Cryptographic Signature

[OMPI users] libpsm_infinipath issues?

[VanEss.Michael]

Hello all,

Our clusters were just upgraded to both a new version of PGI (14.9) as well as 
openmpi (1.8.3).  Previous versions were 12.1 and 1.6 respectively, and those 
compiled and linked just fine.  The newest versions are not linking my mpi 
applications at all.  Here's the problem:

/opt/scyld/openmpi/1.8.3/pgi/bin/mpif90 -C chemcode_mpi.o mod_mpichem.o 
plume_mpi.o amb2D_mpi.o fex.o jex.o use_tuv.o run_lsode.o mod_amb.o utmgeo.o 
lsode.o newamb.o plume.o amb2D.o solve.o mod_cdf.o calc_rates.o mod_tuv.o 
flux.o amb1D.o amb_com.o newamb2D.o vs_ccode.o ck_errors.o newamb1D.o doChem.o 
mod_lsode.o stode.o plume_com.o typeSizes.o netcdf.o mod_parameters.o 
mod_chem.o runfile.o com_codes.o mod_SLAM.o mod_CPUFF.o calc_za.o 
mod_releases.o mod_site.o mod_distance.o nuclear_dates.o mod_luparms.o 
deposition.o diffusion.o getTJ.o mod_met.o met_data.o mod_h5tuv.o tuv10.o 
mod_h5camx.o cmxamb.o \
-L/ensco/apps/cm/CMSOL/linux/zlib-1.2.1/lib 
-L/ensco/apps/cm/CMSOL/linux/szip-2.1/lib -o \
chemcode_mpi  -L/opt/pgi/linux86-64/14.9/lib -lpgf90 -lpgf90rtl \
-L/ensco/apps/cm/CMSOL/linux/hdf5-1.8.3/lib -lhdf5_fortran -l hdf5 -lz 
-lm
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_put'
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_open_device'
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_get'
/usr/lib64/libpsm_infinipath.so.1: undefined reference to `knem_register_region'

I've searched the net for any information on this and nothing has seemed to 
help.  I'm fairly confident that all my variables and paths to the new software 
is correct.

Any ideas?

Regards,

Mike



The information contained in this email message is intended only for the use of 
the individual(s) to whom it is addressed and may contain information that is 
privileged and sensitive. If you are not the intended recipient, or otherwise 
have received this communication in error, please notify the sender immediately 
by email at the above referenced address and note that any further 
dissemination, distribution or copying of this communication is strictly 
prohibited.

The U.S. Export Control Laws regulate the export and re-export of technology 
originating in the United States. This includes the electronic transmission of 
information and software to foreign countries and to certain foreign nationals. 
Recipient agrees to abide by these laws and their regulations -- including the 
U.S. Department of Commerce Export Administration Regulations and the U.S. 
Department of State International Traffic in Arms Regulations -- and not to 
transfer, by electronic transmission or otherwise, any content derived from 
this email to either a foreign national or a foreign destination in violation 
of such laws.

Re: [OMPI users] send and receive vectors + variable length

[Tom Rosmond]

With array bounds checking your program returns an out-of-bounds error
in the mpi_isend call at line 104.  Looks like 'send_request' should be
indexed with 'sendcount', not 'icount'.

T. Rosmond



On Thu, 2015-01-08 at 20:28 +0100, Diego Avesani wrote:
> the attachment
> 
> Diego
> 
> 
> 
> On 8 January 2015 at 19:44, Diego Avesani 
> wrote:
> Dear all,
> I found the error. 
> There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
> In the attachment there is the correct version of the program.
> 
> 
> Only one thing, could do you check if the use of MPI_WAITALL
> and MPI_BARRIER is correct?
> 
> 
> Thanks again
> 
> 
> 
> 
> 
> Diego
> 
> 
> 
> On 8 January 2015 at 18:48, Diego Avesani
>  wrote:
> Dear all,
> thanks thank a lot, I am learning a lot.
> 
> 
> 
> I have written a simple program that send vectors of
> integers from a CPU to another.
> 
> 
> The program is written (at least for now) for 4 CPU.
> 
> 
> The program is quite simple:
> Each CPU knows how many data has to send to the other
> CPUs. This info is than send to the other CPUS. In
> this way each CPU knows how may data has to receive
> from other CPUs.
> 
> 
> This part of the program works.
> 
> 
> The problem is in the second part.
> 
> 
> In the second part, each processor sends a vector of
> integer to the other processor. The size is given and
> each CPU knows the size of the incoming vector form
> the first part of the program.
> 
> 
> In this second part the program fails and I do not
> know why.
> 
> 
> In the attachment you can find the program. Could you
> please help me. Problably I didn't understand properly
> the ISEND and IRECV subroutine. 
> 
> 
> Thanks again
> 
> 
> 
> Diego
> 
> 
> 
> 
> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26131.php

Re: [OMPI users] send and receive vectors + variable length

[Diego Avesani]

the attachment

Diego


On 8 January 2015 at 19:44, Diego Avesani  wrote:

> Dear all,
> I found the error.
> There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
> In the attachment there is the correct version of the program.
>
> Only one thing, could do you check if the use of MPI_WAITALL
> and MPI_BARRIER is correct?
>
> Thanks again
>
>
>
> Diego
>
>
> On 8 January 2015 at 18:48, Diego Avesani  wrote:
>
>> Dear all,
>> thanks thank a lot, I am learning a lot.
>>
>> I have written a simple program that send vectors of integers from a CPU
>> to another.
>>
>> The program is written (at least for now) for 4 CPU.
>>
>> The program is quite simple:
>> Each CPU knows how many data has to send to the other CPUs. This info is
>> than send to the other CPUS. In this way each CPU knows how may data has to
>> receive from other CPUs.
>>
>> This part of the program works.
>>
>> The problem is in the second part.
>>
>> In the second part, each processor sends a vector of integer to the other
>> processor. The size is given and each CPU knows the size of the incoming
>> vector form the first part of the program.
>>
>> In this second part the program fails and I do not know why.
>>
>> In the attachment you can find the program. Could you please help me.
>> Problably I didn't understand properly the ISEND and IRECV subroutine.
>>
>> Thanks again
>>
>>
>> Diego
>>
>>
>
MODULE MOD_PRECISION
  integer, parameter :: dp = selected_real_kind(p=16)
ENDMODULE MOD_PRECISION


PROGRAM TEST
USE MOD_PRECISION
USE MPI
IMPLICIT NONE
  !
  INTEGER :: iCPU,icount,iTag,ip,sendcount,sendrecv,i
  !
  TYPE tMPI
 INTEGER  :: myrank, nCPU, iErr, status
  END TYPE tMPI
  !
  INTEGER, ALLOCATABLE,DIMENSION(:) :: Ndata2send
  INTEGER, ALLOCATABLE,DIMENSION(:) :: Ndata2recv
  
  INTEGER, ALLOCATABLE,DIMENSION(:) :: DATASEND
  INTEGER, ALLOCATABLE,DIMENSION(:) :: DATARECV
  !
  INTEGER :: send_request (3)
  INTEGER :: recv_request (3)
  INTEGER :: send_status_list (MPI_STATUS_SIZE,3) 
  INTEGER :: recv_status_list (MPI_STATUS_SIZE,3)  
  !
  TYPE(tMPI) :: MPIdata

  CALL MPI_INIT(MPIdata%iErr)
  CALL MPI_COMM_RANK(MPI_COMM_WORLD, MPIdata%myrank, MPIdata%iErr)
  CALL MPI_COMM_SIZE(MPI_COMM_WORLD, MPIdata%nCPU,   MPIdata%iErr)
  !
  ALLOCATE(Ndata2send(0:MPIdata%nCPU-1))
  ALLOCATE(Ndata2recv(0:MPIdata%nCPU-1))
  !
  Ndata2send=0
  Ndata2recv=0
  !
  IF(MPIdata%myrank==0)THEN
 Ndata2send(0)=0
 Ndata2send(1)=100
 Ndata2send(2)=100
 Ndata2send(3)=160
  ENDIF
IF(MPIdata%myrank==1)THEN
 Ndata2send(0)=130
 Ndata2send(1)=0
 Ndata2send(2)=160
 Ndata2send(3)=160
  ENDIF
IF(MPIdata%myrank==2)THEN
 Ndata2send(0)=130
 Ndata2send(1)=160
 Ndata2send(2)=0
 Ndata2send(3)=160
  ENDIF
IF(MPIdata%myrank==3)THEN
 Ndata2send(0)=160
 Ndata2send(1)=160
 Ndata2send(2)=160
 Ndata2send(3)=0
  ENDIF
  !
  CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
  
  
  icount=1
  DO iCPU=0,MPIdata%nCPU-1
 IF(iCPU.NE.MPIdata%myrank)THEN
iTag=iCPU
CALL MPI_ISEND(Ndata2send(iCPU),1,MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,send_request(icount),MPIdata%iErr)
icount=icount+1
 ENDIF
  ENDDO
  !
  icount=1
  DO iCPU=0,MPIdata%nCPU-1
 IF(iCPU.NE.MPIdata%myrank)THEN
iTag=MPIdata%myrank
CALL MPI_IRECV(Ndata2recv(iCPU),1,MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,recv_request(icount),MPIdata%iErr)
icount=icount+1
 ENDIF
  ENDDO
  !
  CALL MPI_WAITALL(MPIdata%nCPU-1,send_request,send_status_list,MPIdata%iErr)
  CALL MPI_WAITALL(MPIdata%nCPU-1,recv_request,recv_status_list,MPIdata%iErr)
  !
  !+
  !+
  !+
  !
  sendcount=1
  DO iCPU=0,MPIdata%nCPU-1 
 IF(MPIdata%myrank.NE.iCPU)THEN
!  
ALLOCATE(DATASEND(Ndata2send(iCPU)))
!
DO ip=1,Ndata2send(iCPU)
  DATASEND(ip) = ip
ENDDO
!
iTag=iCPU
CALL MPI_ISEND(DATASEND(:),Ndata2send(iCPU),MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,send_request(icount),MPIdata%iErr)
sendcount=sendcount+1 
DEALLOCATE(DATASEND)
!
 ENDIF
  ENDDO
  !
  !
  CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
  !
  !
  sendrecv=1
  DO iCPU=0,MPIdata%nCPU-1
 !
 ALLOCATE(DATARECV(Ndata2recv(iCPU)))
 !
 IF(MPIdata%myrank.NE.iCPU)THEN
iTag=MPIdata%myrank
CALL MPI_IRECV(DATARECV,Ndata2recv(iCPU),MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,recv_request(sendrecv),MPIdata%iErr)
sendrecv=sendrecv+1
 ENDIF
 
 DO i=1,Ndata2recv(iCPU)
  WRITE(*,*) DATARECV(i)
 ENDDO
 !
 DEALLOCATE(DATARECV)
 !
   ENDDO
  
  

Re: [OMPI users] send and receive vectors + variable length

[Jeff Squyres (jsquyres)]

What do you need the barriers for?

On Jan 8, 2015, at 1:44 PM, Diego Avesani  wrote:

> Dear all,
> I found the error. 
> There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
> In the attachment there is the correct version of the program.
> 
> Only one thing, could do you check if the use of MPI_WAITALL and MPI_BARRIER 
> is correct?
> 
> Thanks again
> 
> 
> 
> Diego
> 
> 
> On 8 January 2015 at 18:48, Diego Avesani  wrote:
> Dear all,
> thanks thank a lot, I am learning a lot.
> 
> I have written a simple program that send vectors of integers from a CPU to 
> another.
> 
> The program is written (at least for now) for 4 CPU.
> 
> The program is quite simple:
> Each CPU knows how many data has to send to the other CPUs. This info is than 
> send to the other CPUS. In this way each CPU knows how may data has to 
> receive from other CPUs.
> 
> This part of the program works.
> 
> The problem is in the second part.
> 
> In the second part, each processor sends a vector of integer to the other 
> processor. The size is given and each CPU knows the size of the incoming 
> vector form the first part of the program.
> 
> In this second part the program fails and I do not know why.
> 
> In the attachment you can find the program. Could you please help me. 
> Problably I didn't understand properly the ISEND and IRECV subroutine. 
> 
> Thanks again
> 
> 
> Diego
> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26129.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] send and receive vectors + variable length

[Diego Avesani]

Dear all,
I found the error.
There is a  Ndata2send(iCPU) instead of Ndata2recv(iCPU).
In the attachment there is the correct version of the program.

Only one thing, could do you check if the use of MPI_WAITALL
and MPI_BARRIER is correct?

Thanks again



Diego


On 8 January 2015 at 18:48, Diego Avesani  wrote:

> Dear all,
> thanks thank a lot, I am learning a lot.
>
> I have written a simple program that send vectors of integers from a CPU
> to another.
>
> The program is written (at least for now) for 4 CPU.
>
> The program is quite simple:
> Each CPU knows how many data has to send to the other CPUs. This info is
> than send to the other CPUS. In this way each CPU knows how may data has to
> receive from other CPUs.
>
> This part of the program works.
>
> The problem is in the second part.
>
> In the second part, each processor sends a vector of integer to the other
> processor. The size is given and each CPU knows the size of the incoming
> vector form the first part of the program.
>
> In this second part the program fails and I do not know why.
>
> In the attachment you can find the program. Could you please help me.
> Problably I didn't understand properly the ISEND and IRECV subroutine.
>
> Thanks again
>
>
> Diego
>
>

[OMPI users] send and receive vectors + variable length

[Diego Avesani]

Dear all,
thanks thank a lot, I am learning a lot.

I have written a simple program that send vectors of integers from a CPU to
another.

The program is written (at least for now) for 4 CPU.

The program is quite simple:
Each CPU knows how many data has to send to the other CPUs. This info is
than send to the other CPUS. In this way each CPU knows how may data has to
receive from other CPUs.

This part of the program works.

The problem is in the second part.

In the second part, each processor sends a vector of integer to the other
processor. The size is given and each CPU knows the size of the incoming
vector form the first part of the program.

In this second part the program fails and I do not know why.

In the attachment you can find the program. Could you please help me.
Problably I didn't understand properly the ISEND and IRECV subroutine.

Thanks again


Diego
MODULE MOD_PRECISION
  integer, parameter :: dp = selected_real_kind(p=16)
ENDMODULE MOD_PRECISION


PROGRAM TEST
USE MOD_PRECISION
USE MPI
IMPLICIT NONE
  !
  INTEGER :: iCPU,icount,iTag,ip,sendcount,sendrecv,i
  !
  TYPE tMPI
 INTEGER  :: myrank, nCPU, iErr, status
  END TYPE tMPI
  !
  INTEGER, ALLOCATABLE,DIMENSION(:) :: Ndata2send
  INTEGER, ALLOCATABLE,DIMENSION(:) :: Ndata2recv
  
  INTEGER, ALLOCATABLE,DIMENSION(:) :: DATASEND
  INTEGER, ALLOCATABLE,DIMENSION(:) :: DATARECV
  !
  INTEGER :: send_request (3)
  INTEGER :: recv_request (3)
  INTEGER :: send_status_list (MPI_STATUS_SIZE,3) 
  INTEGER :: recv_status_list (MPI_STATUS_SIZE,3)  
  !
  TYPE(tMPI) :: MPIdata

  CALL MPI_INIT(MPIdata%iErr)
  CALL MPI_COMM_RANK(MPI_COMM_WORLD, MPIdata%myrank, MPIdata%iErr)
  CALL MPI_COMM_SIZE(MPI_COMM_WORLD, MPIdata%nCPU,   MPIdata%iErr)
  !
  ALLOCATE(Ndata2send(0:MPIdata%nCPU-1))
  ALLOCATE(Ndata2recv(0:MPIdata%nCPU-1))
  !
  Ndata2send=0
  Ndata2recv=0
  !
  IF(MPIdata%myrank==0)THEN
 Ndata2send(0)=0
 Ndata2send(1)=10
 Ndata2send(2)=10
 Ndata2send(3)=16
  ENDIF
IF(MPIdata%myrank==1)THEN
 Ndata2send(0)=13
 Ndata2send(1)=0
 Ndata2send(2)=16
 Ndata2send(3)=16
  ENDIF
IF(MPIdata%myrank==2)THEN
 Ndata2send(0)=13
 Ndata2send(1)=16
 Ndata2send(2)=0
 Ndata2send(3)=16
  ENDIF
IF(MPIdata%myrank==3)THEN
 Ndata2send(0)=16
 Ndata2send(1)=16
 Ndata2send(2)=16
 Ndata2send(3)=0
  ENDIF
  !
  CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
  
  
  icount=1
  DO iCPU=0,MPIdata%nCPU-1
 IF(iCPU.NE.MPIdata%myrank)THEN
iTag=iCPU
CALL MPI_ISEND(Ndata2send(iCPU),1,MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,send_request(icount),MPIdata%iErr)
icount=icount+1
 ENDIF
  ENDDO
  !
  icount=1
  DO iCPU=0,MPIdata%nCPU-1
 IF(iCPU.NE.MPIdata%myrank)THEN
iTag=MPIdata%myrank
CALL MPI_IRECV(Ndata2recv(iCPU),1,MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,recv_request(icount),MPIdata%iErr)
icount=icount+1
 ENDIF
  ENDDO
  !
  CALL MPI_WAITALL(MPIdata%nCPU-1,send_request,send_status_list,MPIdata%iErr)
  CALL MPI_WAITALL(MPIdata%nCPU-1,recv_request,recv_status_list,MPIdata%iErr)
  !
  !+
  !+
  !+
  !
  sendcount=1
  DO iCPU=0,MPIdata%nCPU-1 
 IF(MPIdata%myrank.NE.iCPU)THEN
!  
ALLOCATE(DATASEND(Ndata2send(iCPU)))
!
DO ip=1,Ndata2send(iCPU)
  DATASEND(ip) = ip
ENDDO
!
iTag=iCPU
CALL MPI_ISEND(DATASEND(:),Ndata2send(iCPU),MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,send_request(icount),MPIdata%iErr)
sendcount=sendcount+1 
DEALLOCATE(DATASEND)
!
 ENDIF
  ENDDO
  !
  !
  CALL MPI_BARRIER(MPI_COMM_WORLD, MPIdata%iErr)
  !
  !
  sendrecv=1
  DO iCPU=0,MPIdata%nCPU-1
 !
 ALLOCATE(DATARECV(Ndata2send(iCPU)))
 !
 IF(MPIdata%myrank.NE.iCPU)THEN
iTag=MPIdata%myrank
CALL MPI_IRECV(DATARECV,Ndata2recv(iCPU),MPI_INTEGER,iCPU,iTag,MPI_COMM_WORLD,recv_request(sendrecv),MPIdata%iErr)
sendrecv=sendrecv+1
 ENDIF
 
 DO i=1,Ndata2recv(iCPU)
  WRITE(*,*) DATARECV(i)
 ENDDO
 !
 DEALLOCATE(DATARECV)
 !
   ENDDO
  
  CALL MPI_WAITALL(MPIdata%nCPU-1,send_request,send_status_list,MPIdata%iErr)
  CALL MPI_WAITALL(MPIdata%nCPU-1,recv_request,recv_status_list,MPIdata%iErr)
  
  CALL MPI_Finalize(MPIdata%iErr)
ENDPROGRAM

Re: [hwloc-users] PCI devices topology

[Brice Goglin]

Hello,

hwloc_topology_init();
hwloc_topology_set_flags(topology, HWLOC_TOPOLOGY_FLAG_IO_DEVICES);
hwloc_topology_load(topology);

Then you can use hwloc_get_next_pcidev() to iterate over the entire list
PCI devices. If you want to know whether it's connected to a specific
NUMA node, start from the PCI hwloc_obj_t and walk up the ->parent
pointer until you find a NUMA node object (not guaranteed, could be
connected to something else, even something that is not a child of a
NUMA node, for instance the entire machine object).

Otherwise, you can start at the NUMA node object, walk its children
until you find some object of type Bridge, then recursively walk
children to find all PCI objects (it's often organized as a imbalanced
tree, there can other bridges in the middle).

Brice




Le 08/01/2015 17:19, Pradeep Kiruvale a écrit :
> Hi All,
>
> I am looking for a sample application which can provide me the list of
> PCI devices connected
> to each NUMA node. Please let me know how can access the information
> programmatically on 
> a Linux system using hwloc APIs.
>
> Regards,
> Pradeep
>
>
>
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
> Link to this post: 
> http://www.open-mpi.org/community/lists/hwloc-users/2015/01/1145.php

Re: [OMPI users] OMPI users] OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[Gilles Gouaillardet]

Diego,

yes, it works for me (at least with the v1.8 head and gnu compilers)

Cheers,

Gilles

On 2015/01/08 17:51, Diego Avesani wrote:
> Dear Gilles,
> thanks again, however it does not work.
>
> the program says:  "SIGSEGV, segmentation fault occurred"
>
> Does the program run in your case?
>
> Thanks again
>
>
>
> Diego
>
>
> On 8 January 2015 at 03:02, Gilles Gouaillardet <
> gilles.gouaillar...@iferc.org> wrote:
>
>>  Diego,
>>
>> my bad, i should have passed displacements(1) to MPI_Type_create_struct
>>
>> here is an updated version
>>
>> (note you have to use a REQUEST integer for MPI_Isend and MPI_Irecv,
>> and you also have to call MPI_Wait to ensure the requests complete)
>>
>> Cheers,
>>
>> Gilles
>>
>>
>> On 2015/01/08 8:23, Diego Avesani wrote:
>>
>> Dear Gilles, Dear all,
>>
>> I'm sorry to bother you again, but I have some problem with send and
>> receive the struct_data.
>>
>> I tried to send a MPI_Type_Create_Struct but I get a segmentation fault
>> occurred and I do not know why. The program is very simple, it is the old
>> one with the isend and irecv subroutines
>>
>> (you can find it in the attachment)
>>
>> Thanks again
>>
>>
>> Diego
>>
>>
>> On 5 January 2015 at 15:54, Diego Avesani  
>>  wrote:
>>
>>
>>  Dear Gilles,
>>
>> Thanks, Thanks a lot.
>> Now is more clear.
>>
>> Again, thanks a lot
>>
>> Diego
>>
>>
>>
>>
>>
>> ___
>> users mailing listus...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post: 
>> http://www.open-mpi.org/community/lists/users/2015/01/26116.php
>>
>>
>>
>> ___
>> users mailing list
>> us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post:
>> http://www.open-mpi.org/community/lists/users/2015/01/26118.php
>>
>
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26124.php

Re: [OMPI users] difference of behaviour for MPI_Publish_name between openmpi-1.4.5 and openmpi-1.8.4

[Bernard Secher]

Thanks Gilles!

That works with this MPI_Info given to MPI_Publish_name.

Cheers,
Bernard

Le 08/01/2015 03:47, Gilles Gouaillardet a écrit :

Well, per the source code, this is not a bug but a feature :


from publish function from ompi/mca/pubsub/orte/pubsub_orte.c

 ompi_info_get_bool(info, "ompi_unique", , );
 if (0 == flag) {
 /* uniqueness not specified - overwrite by default */
 unique = false;
 }

fwiw, and at first glance, i would have expected the default behaviour
is to *not* overwrite (e.g. unique = true;).

anyway, in order to get the expected result, the user program can be
modified like this :

MPI_Info info;
MPI_Info_create();
MPI_Info_set(info, "ompi_unique", "true");

and then invoke MPI_Publish_name() with info instead of MPI_INFO_NULL

an updated version of the program

Cheers,

Gilles

On 2015/01/08 10:12, Ralph Castain wrote:

Hmmm…I confess this API gets little, if any, testing as it is so seldom used, 
so it is quite possible that a buglet has crept into it. I’ll take a look and 
try to have something in 1.8.5.

Thanks!
Ralph


On Jan 7, 2015, at 3:14 AM, Bernard Secher  wrote:

Hello,

With the version openmpi-1.4.5 I got an error  when I tried to publish the same 
name twice with the MPI_Publish_name routine
With the version openmpi-1.8.4 I got no error when I published the same name 
twice with the MPI_Publish_name routine

I used the attached script and source code to perform the test.

With this test, it works well with openmpi-1.4.5, but I get a deadlock with 
openmpi-1.8.4. I can suppress the deadlock with openmpi-1.8.4 if I modify the shell 
script and add a "sleep 1" command between the 2 mpirun commands.

Bernard
___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2015/01/26114.php

___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2015/01/26117.php



___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2015/01/26119.php



--
Logo CEA_
Bernard Sécher DEN/DM2S/STMF/LGLS
CEA Saclay, Bât 451, Pièce 32A
91191 Gif-sur-Yvette Cedex, France
bernard.sec...@cea.fr
Phone: 33 (0)1 69 08 73 78
Fax: 33 (0)1 69 08 10 87
_



Ce message électronique et tous les fichiers attachés qu'il contient 
sont confidentiels et destinés exclusivement à l'usage de la personne à 
laquelle ils sont adressés. Si vous avez reçu ce message par erreur, 
merci d'en avertir immédiatement son émetteur et de ne pas en conserver 
de copie.


This e-mail and any files transmitted with it are confidential and 
intended solely for the use of the individual to whom they are 
addressed. If you have received this e-mail in error please inform the 
sender immediately, without keeping any copy thereof.

Re: [OMPI users] OMPI users] OMPI users] MPI_Type_Create_Struct + MPI_TYPE_CREATE_RESIZED

[Diego Avesani]

Dear Gilles,
thanks again, however it does not work.

the program says:  "SIGSEGV, segmentation fault occurred"

Does the program run in your case?

Thanks again



Diego


On 8 January 2015 at 03:02, Gilles Gouaillardet <
gilles.gouaillar...@iferc.org> wrote:

>  Diego,
>
> my bad, i should have passed displacements(1) to MPI_Type_create_struct
>
> here is an updated version
>
> (note you have to use a REQUEST integer for MPI_Isend and MPI_Irecv,
> and you also have to call MPI_Wait to ensure the requests complete)
>
> Cheers,
>
> Gilles
>
>
> On 2015/01/08 8:23, Diego Avesani wrote:
>
> Dear Gilles, Dear all,
>
> I'm sorry to bother you again, but I have some problem with send and
> receive the struct_data.
>
> I tried to send a MPI_Type_Create_Struct but I get a segmentation fault
> occurred and I do not know why. The program is very simple, it is the old
> one with the isend and irecv subroutines
>
> (you can find it in the attachment)
>
> Thanks again
>
>
> Diego
>
>
> On 5 January 2015 at 15:54, Diego Avesani  
>  wrote:
>
>
>  Dear Gilles,
>
> Thanks, Thanks a lot.
> Now is more clear.
>
> Again, thanks a lot
>
> Diego
>
>
>
>
>
> ___
> users mailing listus...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26116.php
>
>
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26118.php
>

Re: [OMPI users] Icreasing OFED registerable memory

[Waleed Lotfy]

You are right.

I didn't know that SGE used limits other than '/etc/security/limits.conf', even 
though you explained it :/

The resolution is by adding 'H_MEMORYLOCKED=unlimited' in the execd_params.

Thank you all for your time and efforts and keep up the great work :)

Waleed Lotfy
Bibliotheca Alexandrina

From: users [users-boun...@open-mpi.org] on behalf of Gus Correa 
[g...@ldeo.columbia.edu]
Sent: Tuesday, January 06, 2015 9:11 PM
To: Open MPI Users
Subject: Re: [OMPI users] Icreasing OFED registerable memory

Hi Waleed

As Devendar said (and I tried to explain before),
you need to allow the locked memory limit to be unlimited for
user processes (in /etc/security/limits.conf),
*AND* somehow the daemon/job_script/whatever that launches the mpiexec
command must request "ulimit -l unlimited" (directly or indirectly).
The latter part depends on how your system's details.
I am not familiar to SGE (I use Torque), but presumably you can
add "ulimit -l unlimited" when you launch
the SGE daemons on the nodes.
Presumably this will make the processes launched by that daemon
(i.e. your mpiexec) inherit those limits,
and that is how I do it on Torque.
A more brute force way is just to include "ulimit -l unlimited"
in you job script before mpiexec.
Inserting a "ulimit -a" in your jobscript may help diagnose what you
actually have.
Please, see the OMPI FAQ that I sent you before for more details.

I hope this helps,
Gus Correa

On 01/06/2015 01:37 PM, Deva wrote:
> Hi Waleed,
>
> --
>Memlock limit: 65536
> --
>
> such a low limit should be due to per-user lock memory limit . Can you
> make sure it is  set to "unlimited" on all nodes ( "ulimit -l unlimited")?
>
> -Devendar
>
> On Tue, Jan 6, 2015 at 3:42 AM, Waleed Lotfy  > wrote:
>
> Hi guys,
>
> Sorry for getting back so late, but we ran into some problems during
> the installation process and as soon as the system came up I tested
> the new versions for the problem but it showed another memory
> related warning.
>
> --
> The OpenFabrics (openib) BTL failed to initialize while trying to
> allocate some locked memory.  This typically can indicate that the
> memlock limits are set too low.  For most HPC installations, the
> memlock limits should be set to "unlimited".  The failure occured
> here:
>
>Local host:comp003.local
>OMPI source:   btl_openib_component.c:1200
>Function:  ompi_free_list_init_ex_new()
>Device:mlx4_0
>Memlock limit: 65536
>
> You may need to consult with your system administrator to get this
> problem fixed.  This FAQ entry on the Open MPI web site may also be
> helpful:
>
> http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
> --
> --
> WARNING: There was an error initializing an OpenFabrics device.
>
>Local host:   comp003.local
>Local device: mlx4_0
> --
>
> <<>>
>
> My current running versions:
>
> OpenMPI: 1.6.4
> OFED-internal-2.3-2
>
> I checked /etc/security/limits.d/, the scheduler's configurations
> (grid engine) and tried adding the following line to
> /etc/modprobe.d/mlx4_core: 'options mlx4_core log_num_mtt=22
> log_mtts_per_seg=1' as suggested by Gus.
>
> I am running out of ideas here, so please any help is appreciated.
>
> P.S. I am not sure if I should open a new thread with this issue or
> continue with the current one, so please advice.
>
> Waleed Lotfy
> Bibliotheca Alexandrina
> ___
> users mailing list
> us...@open-mpi.org 
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26107.php
>
>
>
>
> --
>
>
> -Devendar
>
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2015/01/26109.php
>

___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2015/01/26111.php