subject:"\[OMPI users\] ERR_TRUNCATE with MPI

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

2018-02-15 Thread Thomas Jahns


On 02/14/2018 11:19 AM, Florian Lindner wrote:

Hello,

I have this example code:

#include 
#include 

int main(int argc, char *argv[])
{
   MPI_Init(, );
   {
 MPI_Request req1, req2;
 std::vector vec = {1, 2, 3};
 int packSize = sizeof(int) * vec.size();


Why don't you use MPI_Pack_size for sizing of packSendBuf here? The way you 
write it is not guaranteed to work.



 int position = 0;
 std::vector packSendBuf(packSize);
 int vecSize = vec.size();
 MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
, MPI_COMM_WORLD);

 int estimatedPackSize = 0;
 MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
 std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
estimatedPackSize << std::endl;

 MPI_Isend(, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, );
 MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
);
   }


At the above you asynchronously send data which then goes out of scope. That's 
pretty much guaranteed to not work, have you tried to run your program with 
valgrind?


[...]

Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. Background is 
that I want to send multiple differently


Which routine gives MPI_ERR_TRUNCATE?


sized objects, also with more complex types that to not map to MPI_*, for which 
I plan to use MPI_BYTES. I plan to pack
them into one stream and unpack them one after one.


Which elemental type does not map to an MPI datatype? You are aware that for all 
derived types except pointer components there are ways to build corresponding 
data types?



I suspect I got somthig with the sizes wrong. The lines

 int estimatedPackSize = 0;
 MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
 std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
estimatedPackSize << std::endl;

Return the same number, that is 12, the packSize from get_cont is also 12.


But see above: a pack size must be computed with MPI_Pack_size.

Regards, Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

2018-02-14 Thread Gilles Gouaillardet

Florian,

My bad, I overlooked that.

Per the man page, the second parameter of MPI_Unpack() is the input buffer size 
in *bytes*.
In your case, it should be packSize instead of vecSize.

Makes sense ?

FWIW, I never hesitate to run small tests with mpich (or its derivative).
If both implementations fail, the odds are the problem is in the code and not 
in the library.


Gilles

Florian Lindner  wrote:
>Hi Gilles,
>
>
>Am 14.02.2018 um 11:46 schrieb Gilles Gouaillardet:
>> Florian,
>> 
>> You send position=0 MPI_PACKED instead of estimatedPackSize, so it is very 
>> odd you see get_count = 12
>> 
>> Can you please double check that part ?
>
>https://gist.github.com/floli/310980790d5d76caac0b19a937e2a502
>
>You mean in line 22:
>
> MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
> );
>
>but position was incremented (to 12, see output below) by the preceding 
>MPI_Pack call.
>
>> 
>> Also, who returns MPI_ERR_TRUNCATE ? MPI_Recv ? MPI_Unpack ?
>
>Sorry, forgot to include that crucial part:
>
>% mpirun -n 1 ./a.out
>packSize = 12, estimatedPackSize = 12
>position after pack = 12
>packSize from get_count = 12
>[asaru:30337] *** An error occurred in MPI_Unpack
>[asaru:30337] *** reported by process [4237492225,0]
>[asaru:30337] *** on communicator MPI_COMM_WORLD
>[asaru:30337] *** MPI_ERR_TRUNCATE: message truncated
>[asaru:30337] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will 
>now abort,
>[asaru:30337] ***and potentially your MPI job)
>
>Best Thanks,
>Florian
>
>> 
>> 
>> Cheers,
>> 
>> Gilles
>> 
>> Florian Lindner  wrote:
>>> Hello,
>>>
>>> I have this example code:
>>>
>>> #include 
>>> #include 
>>>
>>> int main(int argc, char *argv[])
>>> {
>>>  MPI_Init(, );
>>>  {
>>>MPI_Request req1, req2;
>>>std::vector vec = {1, 2, 3};
>>>int packSize = sizeof(int) * vec.size();
>>>int position = 0;
>>>std::vector packSendBuf(packSize);
>>>int vecSize = vec.size();
>>>MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
>>> , MPI_COMM_WORLD);
>>>
>>>int estimatedPackSize = 0;
>>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
>>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>>> estimatedPackSize << std::endl;
>>>
>>>MPI_Isend(, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, );
>>>MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, 
>>> MPI_COMM_WORLD, );
>>>  }
>>>  {
>>>int vecSize, msgSize;
>>>int packSize = 0, position = 0;
>>>
>>>MPI_Recv(, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
>>> MPI_STATUS_IGNORE);
>>>
>>>MPI_Status status;
>>>MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, );
>>>MPI_Get_count(, MPI_PACKED, );
>>>char packBuffer[packSize];
>>>std::cout << "packSize from get_count = " << packSize << std::endl;
>>>
>>>std::vector vec(vecSize);
>>>MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, 
>>> MPI_COMM_WORLD, MPI_STATUS_IGNORE);
>>>MPI_Unpack(packBuffer, vecSize, , vec.data(), vecSize, MPI_INT, 
>>> MPI_COMM_WORLD);
>>>  }
>>>  MPI_Finalize();
>>> }
>>>
>>>
>>> Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. 
>>> Background is that I want to send multiple differently
>>> sized objects, also with more complex types that to not map to MPI_*, for 
>>> which I plan to use MPI_BYTES. I plan to pack
>>> them into one stream and unpack them one after one.
>>>
>>> I suspect I got somthig with the sizes wrong. The lines
>>>
>>>int estimatedPackSize = 0;
>>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
>>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>>> estimatedPackSize << std::endl;
>>>
>>> Return the same number, that is 12, the packSize from get_cont is also 12.
>>>
>>> Could you give a hint, what the is problem is here?
>>>
>>> OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04
>>>
>>> Thanks,
>>> Florian
>>>
>>>
>>> ___
>>> users mailing list
>>> users@lists.open-mpi.org
>>> https://lists.open-mpi.org/mailman/listinfo/users
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://lists.open-mpi.org/mailman/listinfo/users
>> 
>___
>users mailing list
>users@lists.open-mpi.org
>https://lists.open-mpi.org/mailman/listinfo/users
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

2018-02-14 Thread Florian Lindner

Hi Gilles,


Am 14.02.2018 um 11:46 schrieb Gilles Gouaillardet:
> Florian,
> 
> You send position=0 MPI_PACKED instead of estimatedPackSize, so it is very 
> odd you see get_count = 12
> 
> Can you please double check that part ?

https://gist.github.com/floli/310980790d5d76caac0b19a937e2a502

You mean in line 22:

 MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
);

but position was incremented (to 12, see output below) by the preceding 
MPI_Pack call.

> 
> Also, who returns MPI_ERR_TRUNCATE ? MPI_Recv ? MPI_Unpack ?

Sorry, forgot to include that crucial part:

% mpirun -n 1 ./a.out
packSize = 12, estimatedPackSize = 12
position after pack = 12
packSize from get_count = 12
[asaru:30337] *** An error occurred in MPI_Unpack
[asaru:30337] *** reported by process [4237492225,0]
[asaru:30337] *** on communicator MPI_COMM_WORLD
[asaru:30337] *** MPI_ERR_TRUNCATE: message truncated
[asaru:30337] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now 
abort,
[asaru:30337] ***and potentially your MPI job)

Best Thanks,
Florian

> 
> 
> Cheers,
> 
> Gilles
> 
> Florian Lindner  wrote:
>> Hello,
>>
>> I have this example code:
>>
>> #include 
>> #include 
>>
>> int main(int argc, char *argv[])
>> {
>>  MPI_Init(, );
>>  {
>>MPI_Request req1, req2;
>>std::vector vec = {1, 2, 3};
>>int packSize = sizeof(int) * vec.size();
>>int position = 0;
>>std::vector packSendBuf(packSize);
>>int vecSize = vec.size();
>>MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
>> , MPI_COMM_WORLD);
>>
>>int estimatedPackSize = 0;
>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>> estimatedPackSize << std::endl;
>>
>>MPI_Isend(, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, );
>>MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
>> );
>>  }
>>  {
>>int vecSize, msgSize;
>>int packSize = 0, position = 0;
>>
>>MPI_Recv(, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
>> MPI_STATUS_IGNORE);
>>
>>MPI_Status status;
>>MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, );
>>MPI_Get_count(, MPI_PACKED, );
>>char packBuffer[packSize];
>>std::cout << "packSize from get_count = " << packSize << std::endl;
>>
>>std::vector vec(vecSize);
>>MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, 
>> MPI_COMM_WORLD, MPI_STATUS_IGNORE);
>>MPI_Unpack(packBuffer, vecSize, , vec.data(), vecSize, MPI_INT, 
>> MPI_COMM_WORLD);
>>  }
>>  MPI_Finalize();
>> }
>>
>>
>> Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. Background 
>> is that I want to send multiple differently
>> sized objects, also with more complex types that to not map to MPI_*, for 
>> which I plan to use MPI_BYTES. I plan to pack
>> them into one stream and unpack them one after one.
>>
>> I suspect I got somthig with the sizes wrong. The lines
>>
>>int estimatedPackSize = 0;
>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>> estimatedPackSize << std::endl;
>>
>> Return the same number, that is 12, the packSize from get_cont is also 12.
>>
>> Could you give a hint, what the is problem is here?
>>
>> OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04
>>
>> Thanks,
>> Florian
>>
>>
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://lists.open-mpi.org/mailman/listinfo/users
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
> 
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

2018-02-14 Thread Gilles Gouaillardet

Florian,

You send position=0 MPI_PACKED instead of estimatedPackSize, so it is very odd 
you see get_count = 12

Can you please double check that part ?

Also, who returns MPI_ERR_TRUNCATE ? MPI_Recv ? MPI_Unpack ?


Cheers,

Gilles

Florian Lindner  wrote:
>Hello,
>
>I have this example code:
>
>#include 
>#include 
>
>int main(int argc, char *argv[])
>{
>  MPI_Init(, );
>  {
>MPI_Request req1, req2;
>std::vector vec = {1, 2, 3};
>int packSize = sizeof(int) * vec.size();
>int position = 0;
>std::vector packSendBuf(packSize);
>int vecSize = vec.size();
>MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
> , MPI_COMM_WORLD);
>
>int estimatedPackSize = 0;
>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
> estimatedPackSize << std::endl;
>
>MPI_Isend(, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, );
>MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
> );
>  }
>  {
>int vecSize, msgSize;
>int packSize = 0, position = 0;
>
>MPI_Recv(, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
> MPI_STATUS_IGNORE);
>
>MPI_Status status;
>MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, );
>MPI_Get_count(, MPI_PACKED, );
>char packBuffer[packSize];
>std::cout << "packSize from get_count = " << packSize << std::endl;
>
>std::vector vec(vecSize);
>MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
> MPI_STATUS_IGNORE);
>MPI_Unpack(packBuffer, vecSize, , vec.data(), vecSize, MPI_INT, 
> MPI_COMM_WORLD);
>  }
>  MPI_Finalize();
>}
>
>
>Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. Background 
>is that I want to send multiple differently
>sized objects, also with more complex types that to not map to MPI_*, for 
>which I plan to use MPI_BYTES. I plan to pack
>them into one stream and unpack them one after one.
>
>I suspect I got somthig with the sizes wrong. The lines
>
>int estimatedPackSize = 0;
>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
> estimatedPackSize << std::endl;
>
>Return the same number, that is 12, the packSize from get_cont is also 12.
>
>Could you give a hint, what the is problem is here?
>
>OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04
>
>Thanks,
>Florian
>
>
>___
>users mailing list
>users@lists.open-mpi.org
>https://lists.open-mpi.org/mailman/listinfo/users
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

[OMPI users] ERR_TRUNCATE with MPI_Pack

2018-02-14 Thread Florian Lindner

Hello,

I have this example code:

#include 
#include 

int main(int argc, char *argv[])
{
  MPI_Init(, );
  {
MPI_Request req1, req2;
std::vector vec = {1, 2, 3};
int packSize = sizeof(int) * vec.size();
int position = 0;
std::vector packSendBuf(packSize);
int vecSize = vec.size();
MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
, MPI_COMM_WORLD);

int estimatedPackSize = 0;
MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
estimatedPackSize << std::endl;

MPI_Isend(, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, );
MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
);
  }
  {
int vecSize, msgSize;
int packSize = 0, position = 0;

MPI_Recv(, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
MPI_STATUS_IGNORE);

MPI_Status status;
MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, );
MPI_Get_count(, MPI_PACKED, );
char packBuffer[packSize];
std::cout << "packSize from get_count = " << packSize << std::endl;

std::vector vec(vecSize);
MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
MPI_STATUS_IGNORE);
MPI_Unpack(packBuffer, vecSize, , vec.data(), vecSize, MPI_INT, 
MPI_COMM_WORLD);
  }
  MPI_Finalize();
}


Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. Background is 
that I want to send multiple differently
sized objects, also with more complex types that to not map to MPI_*, for which 
I plan to use MPI_BYTES. I plan to pack
them into one stream and unpack them one after one.

I suspect I got somthig with the sizes wrong. The lines

int estimatedPackSize = 0;
MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, );
std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
estimatedPackSize << std::endl;

Return the same number, that is 12, the packSize from get_cont is also 12.

Could you give a hint, what the is problem is here?

OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04

Thanks,
Florian


___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

[OMPI users] ERR_TRUNCATE with MPI_Pack

5 matches

Site Navigation

Mail list logo

Footer information