Re: [OMPI users] MacOS - Running MPI code with 256 procs - “[warn] select: Invalid argument” message

[Gilles Gouaillardet]

Christophe,

I have no reason to suspect the compiler.

By default, the embedded libevent is used.
FWIW. homebrew uses libevent 2.1.8, but once again, I have no reason
to suspect the embedded library is wrong.

My best bet is you first upgrade your operating system, since Maverick
is pretty old now.


Cheers,

Gilles

On Thu, Feb 15, 2018 at 4:01 PM, Christophe Petit
 wrote:
> Thanks for your message.
>
> Actually, if I test ulimit -n, I get :
>
> $ulimit -n
> 65536
>
> that confirms that modifications (from the tutorial) are taken into account.
>
> To test the OpenMPI "ring_c.c" code, I get :
>
> 1) without "--oversubscribe" flag :
>
> $mpirun -np 128 ./ring_c
> --
> There are not enough slots available in the system to satisfy the 128 slots
> that were requested by the application:
>   ./a.out
>
> Either request fewer slots for your application, or make more slots
> available
> for use.
> --
>
> 2) with "--oversubscribe" flag, everything seems to work fine :
>
> mpirun --oversubscribe -np 128 ./ring_c
>
> Process 0 sending 10 to 1, tag 201 (128 processes in ring)
> Process 0 sent to 1
> Process 0 decremented value: 9
> Process 0 decremented value: 8
> Process 0 decremented value: 7
> Process 0 decremented value: 6
> Process 0 decremented value: 5
> Process 0 decremented value: 4
> Process 0 decremented value: 3
> Process 0 decremented value: 2
> Process 0 decremented value: 1
> Process 0 decremented value: 0
> Process 0 exiting
> Process 1 exiting
> Process 2 exiting
> ...
>
> --
>
> Now If I take np = 256 , I get the same warning as my original issue :
>
> $mpirun --oversubscribe -np 256 ./ring_c
>
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> ...
>
> I have installed Open-MPI 3.0 with the GCC-4.9 from MacPorts ( gcc-mp-4.9)
> and GFORTRAN also (gfortran-mp-4.9).
>
> Do you advise me to use gcc CLANG for Open-MPI 3.0 instead or GCC-5 ?
>
> Where can I find a suitable libevent version ? I would like to build firstly
> libevent and after Open-MPI 3.0.
>
> Regards
>
>
> 2018-02-15 1:42 GMT+01:00 Gilles Gouaillardet
> :
>>
>> Christophe,
>>
>> I can only test this on OS X 10.13.3 High Sierra, and it could differ
>> from Maverick.
>>
>> by default
>>
>> KA15-002:~ gilles$ ulimit -n
>> 256
>>
>> KA15-002:~ gilles$ ulimit -Hn
>> unlimited
>>
>>
>> but surprisingly,
>>
>> KA15-002:~ gilles$ ulimit -n unlimited
>> -bash: ulimit: open files: cannot modify limit: Operation not permitted
>>
>> KA15-002:~ gilles$ ulimit -n 2
>> -bash: ulimit: open files: cannot modify limit: Invalid argument
>>
>> and finally, a lower value works just fine.
>>
>> KA15-002:~ gilles$ ulimit -n 1
>>
>>
>> as a consequence, opal_set_max_sys_limits fails in my environment.
>> --oversubscribe is mandatory (since there are no 256 cores on my
>> laptop), and then
>>
>> ulimit -n 1; mpirun --oversubscribe -np 256 ./ring_c
>>
>> works just fine (fwiw, this is an example from Open MPI sources
>> examples/ring_c.c)
>>
>>
>>
>> So first, I invite you to double check with ulimit -n that your system
>> changes are effective.
>>
>> How did you build/install Open MPI ?
>> The message seems to come from libevent, and Open MPI uses an embedded
>> version of libevent.
>> It is possible to use an external version at configure time.
>> If you are using an external libevent, you might want to try
>> rebuilding Open MPI with the embedded one.
>>
>>
>> Cheers,
>>
>> Gilles
>>
>> On Wed, Feb 14, 2018 at 4:57 PM, Christophe Petit
>>  wrote:
>> > Hello,
>> >
>> > Using Open-Mpi 3.0 and following the tutorial on this link, I try to run
>> > a
>> > MPI code under MacOS 10.9.5 (Mavericks) with a number of process equal
>> > to
>> > 256 : the MPI code allocates for each process a 512x512 2D array, so it
>> > requires 256*256kB = 64MB of total used memory.
>> >
>> > My MacOS has 16GB RAM and 8 cores, so it seems to be weird.
>> >
>> > For a number of process lower than 256 (I tried :
>> > np=2,4,8,16,32,64,128),
>> > there is no problem, execution is good and I get expected results.
>> >
>> > But for np = 256, I get the following message which repeats itself :
>> >
>> > $ mpirun -np 256 ./explicitPar
>> >
>> > [warn] select: Invalid argument
>> > [warn] select: Invalid argument
>> > [warn] select: Invalid argument
>> >  ...
>> >
>> > I tried also to use -mca option by doing :
>> >
>> > $ mpirun -mca opal_set_max_sys_limits 1 -np 256 ./explicitPar
>> >
>> > But I get the same warning m

Re: [OMPI users] MacOS - Running MPI code with 256 procs - “[warn] select: Invalid argument” message

[Christophe Petit]

Thanks for your message.

Actually, if I test ulimit -n, I get :

$ulimit -n
65536

that confirms that modifications (from the tutorial) are taken into account.

To test the OpenMPI "ring_c.c" code, I get :

1) without "--oversubscribe" flag :

$mpirun -np 128 ./ring_c
--
There are not enough slots available in the system to satisfy the 128 slots
that were requested by the application:
  ./a.out

Either request fewer slots for your application, or make more slots
available
for use.
--

2) with "--oversubscribe" flag, everything seems to work fine :

mpirun --oversubscribe -np 128 ./ring_c

Process 0 sending 10 to 1, tag 201 (128 processes in ring)
Process 0 sent to 1
Process 0 decremented value: 9
Process 0 decremented value: 8
Process 0 decremented value: 7
Process 0 decremented value: 6
Process 0 decremented value: 5
Process 0 decremented value: 4
Process 0 decremented value: 3
Process 0 decremented value: 2
Process 0 decremented value: 1
Process 0 decremented value: 0
Process 0 exiting
Process 1 exiting
Process 2 exiting
...

--

Now If I take np = 256 , I get the same warning as my original issue :

$mpirun --oversubscribe -np 256 ./ring_c

[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
...

I have installed Open-MPI 3.0 with the GCC-4.9 from MacPorts
 ( gcc-mp-4.9) and GFORTRAN also
(gfortran-mp-4.9).

Do you advise me to use gcc CLANG for Open-MPI 3.0 instead or GCC-5 ?

Where can I find a suitable libevent version ? I would like to build
firstly libevent and after Open-MPI 3.0.

Regards


2018-02-15 1:42 GMT+01:00 Gilles Gouaillardet :

> Christophe,
>
> I can only test this on OS X 10.13.3 High Sierra, and it could differ
> from Maverick.
>
> by default
>
> KA15-002:~ gilles$ ulimit -n
> 256
>
> KA15-002:~ gilles$ ulimit -Hn
> unlimited
>
>
> but surprisingly,
>
> KA15-002:~ gilles$ ulimit -n unlimited
> -bash: ulimit: open files: cannot modify limit: Operation not permitted
>
> KA15-002:~ gilles$ ulimit -n 2
> -bash: ulimit: open files: cannot modify limit: Invalid argument
>
> and finally, a lower value works just fine.
>
> KA15-002:~ gilles$ ulimit -n 1
>
>
> as a consequence, opal_set_max_sys_limits fails in my environment.
> --oversubscribe is mandatory (since there are no 256 cores on my
> laptop), and then
>
> ulimit -n 1; mpirun --oversubscribe -np 256 ./ring_c
>
> works just fine (fwiw, this is an example from Open MPI sources
> examples/ring_c.c)
>
>
>
> So first, I invite you to double check with ulimit -n that your system
> changes are effective.
>
> How did you build/install Open MPI ?
> The message seems to come from libevent, and Open MPI uses an embedded
> version of libevent.
> It is possible to use an external version at configure time.
> If you are using an external libevent, you might want to try
> rebuilding Open MPI with the embedded one.
>
>
> Cheers,
>
> Gilles
>
> On Wed, Feb 14, 2018 at 4:57 PM, Christophe Petit
>  wrote:
> > Hello,
> >
> > Using Open-Mpi 3.0 and following the tutorial on this link, I try to run
> a
> > MPI code under MacOS 10.9.5 (Mavericks) with a number of process equal to
> > 256 : the MPI code allocates for each process a 512x512 2D array, so it
> > requires 256*256kB = 64MB of total used memory.
> >
> > My MacOS has 16GB RAM and 8 cores, so it seems to be weird.
> >
> > For a number of process lower than 256 (I tried : np=2,4,8,16,32,64,128),
> > there is no problem, execution is good and I get expected results.
> >
> > But for np = 256, I get the following message which repeats itself :
> >
> > $ mpirun -np 256 ./explicitPar
> >
> > [warn] select: Invalid argument
> > [warn] select: Invalid argument
> > [warn] select: Invalid argument
> >  ...
> >
> > I tried also to use -mca option by doing :
> >
> > $ mpirun -mca opal_set_max_sys_limits 1 -np 256 ./explicitPar
> >
> > But I get the same warning message.
> >
> > From this link cited above, I did :
> >
> > $ launchctl limit maxfiles
> >
> >   maxfiles65536  20
> >
> > Then, in root user, I created /etc/launchd.conf file and put into :
> >
> > limit maxfiles 65536 20
> >
> > I restarted the system for the new limits to take effect and type as
> normal
> > user :
> >
> > $ launchctl limit maxfiles
> >
> > maxfiles65536  20
> >
> > But unfortunately, these modifications have no effects on the MPI
> function
> > "mpirun" with 256 processes and don't make disappear the warning above.

Re: [OMPI users] MacOS - Running MPI code with 256 procs - “[warn] select: Invalid argument” message

[Gilles Gouaillardet]

Christophe,

I can only test this on OS X 10.13.3 High Sierra, and it could differ
from Maverick.

by default

KA15-002:~ gilles$ ulimit -n
256

KA15-002:~ gilles$ ulimit -Hn
unlimited


but surprisingly,

KA15-002:~ gilles$ ulimit -n unlimited
-bash: ulimit: open files: cannot modify limit: Operation not permitted

KA15-002:~ gilles$ ulimit -n 2
-bash: ulimit: open files: cannot modify limit: Invalid argument

and finally, a lower value works just fine.

KA15-002:~ gilles$ ulimit -n 1


as a consequence, opal_set_max_sys_limits fails in my environment.
--oversubscribe is mandatory (since there are no 256 cores on my
laptop), and then

ulimit -n 1; mpirun --oversubscribe -np 256 ./ring_c

works just fine (fwiw, this is an example from Open MPI sources
examples/ring_c.c)



So first, I invite you to double check with ulimit -n that your system
changes are effective.

How did you build/install Open MPI ?
The message seems to come from libevent, and Open MPI uses an embedded
version of libevent.
It is possible to use an external version at configure time.
If you are using an external libevent, you might want to try
rebuilding Open MPI with the embedded one.


Cheers,

Gilles

On Wed, Feb 14, 2018 at 4:57 PM, Christophe Petit
 wrote:
> Hello,
>
> Using Open-Mpi 3.0 and following the tutorial on this link, I try to run a
> MPI code under MacOS 10.9.5 (Mavericks) with a number of process equal to
> 256 : the MPI code allocates for each process a 512x512 2D array, so it
> requires 256*256kB = 64MB of total used memory.
>
> My MacOS has 16GB RAM and 8 cores, so it seems to be weird.
>
> For a number of process lower than 256 (I tried : np=2,4,8,16,32,64,128),
> there is no problem, execution is good and I get expected results.
>
> But for np = 256, I get the following message which repeats itself :
>
> $ mpirun -np 256 ./explicitPar
>
> [warn] select: Invalid argument
> [warn] select: Invalid argument
> [warn] select: Invalid argument
>  ...
>
> I tried also to use -mca option by doing :
>
> $ mpirun -mca opal_set_max_sys_limits 1 -np 256 ./explicitPar
>
> But I get the same warning message.
>
> From this link cited above, I did :
>
> $ launchctl limit maxfiles
>
>   maxfiles65536  20
>
> Then, in root user, I created /etc/launchd.conf file and put into :
>
> limit maxfiles 65536 20
>
> I restarted the system for the new limits to take effect and type as normal
> user :
>
> $ launchctl limit maxfiles
>
> maxfiles65536  20
>
> But unfortunately, these modifications have no effects on the MPI function
> "mpirun" with 256 processes and don't make disappear the warning above.
>
> On Linux platform, I can launch my MPI code with np = 256, without problem,
> the issue is only happening on MacOS 10.9.5.
>
> I didn't get this issue with previous version of Open-MPI.
>
> Any idea ? Thanks
>
>
>
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

[Gilles Gouaillardet]

Florian,

My bad, I overlooked that.

Per the man page, the second parameter of MPI_Unpack() is the input buffer size 
in *bytes*.
In your case, it should be packSize instead of vecSize.

Makes sense ?

FWIW, I never hesitate to run small tests with mpich (or its derivative).
If both implementations fail, the odds are the problem is in the code and not 
in the library.


Gilles

Florian Lindner  wrote:
>Hi Gilles,
>
>
>Am 14.02.2018 um 11:46 schrieb Gilles Gouaillardet:
>> Florian,
>> 
>> You send position=0 MPI_PACKED instead of estimatedPackSize, so it is very 
>> odd you see get_count = 12
>> 
>> Can you please double check that part ?
>
>https://gist.github.com/floli/310980790d5d76caac0b19a937e2a502
>
>You mean in line 22:
>
> MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
> &req2);
>
>but position was incremented (to 12, see output below) by the preceding 
>MPI_Pack call.
>
>> 
>> Also, who returns MPI_ERR_TRUNCATE ? MPI_Recv ? MPI_Unpack ?
>
>Sorry, forgot to include that crucial part:
>
>% mpirun -n 1 ./a.out
>packSize = 12, estimatedPackSize = 12
>position after pack = 12
>packSize from get_count = 12
>[asaru:30337] *** An error occurred in MPI_Unpack
>[asaru:30337] *** reported by process [4237492225,0]
>[asaru:30337] *** on communicator MPI_COMM_WORLD
>[asaru:30337] *** MPI_ERR_TRUNCATE: message truncated
>[asaru:30337] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will 
>now abort,
>[asaru:30337] ***and potentially your MPI job)
>
>Best Thanks,
>Florian
>
>> 
>> 
>> Cheers,
>> 
>> Gilles
>> 
>> Florian Lindner  wrote:
>>> Hello,
>>>
>>> I have this example code:
>>>
>>> #include 
>>> #include 
>>>
>>> int main(int argc, char *argv[])
>>> {
>>>  MPI_Init(&argc, &argv);
>>>  {
>>>MPI_Request req1, req2;
>>>std::vector vec = {1, 2, 3};
>>>int packSize = sizeof(int) * vec.size();
>>>int position = 0;
>>>std::vector packSendBuf(packSize);
>>>int vecSize = vec.size();
>>>MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
>>> &position, MPI_COMM_WORLD);
>>>
>>>int estimatedPackSize = 0;
>>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
>>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>>> estimatedPackSize << std::endl;
>>>
>>>MPI_Isend(&vecSize, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, &req1);
>>>MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, 
>>> MPI_COMM_WORLD, &req2);
>>>  }
>>>  {
>>>int vecSize, msgSize;
>>>int packSize = 0, position = 0;
>>>
>>>MPI_Recv(&vecSize, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
>>> MPI_STATUS_IGNORE);
>>>
>>>MPI_Status status;
>>>MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, &status);
>>>MPI_Get_count(&status, MPI_PACKED, &packSize);
>>>char packBuffer[packSize];
>>>std::cout << "packSize from get_count = " << packSize << std::endl;
>>>
>>>std::vector vec(vecSize);
>>>MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, 
>>> MPI_COMM_WORLD, MPI_STATUS_IGNORE);
>>>MPI_Unpack(packBuffer, vecSize, &position, vec.data(), vecSize, MPI_INT, 
>>> MPI_COMM_WORLD);
>>>  }
>>>  MPI_Finalize();
>>> }
>>>
>>>
>>> Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. 
>>> Background is that I want to send multiple differently
>>> sized objects, also with more complex types that to not map to MPI_*, for 
>>> which I plan to use MPI_BYTES. I plan to pack
>>> them into one stream and unpack them one after one.
>>>
>>> I suspect I got somthig with the sizes wrong. The lines
>>>
>>>int estimatedPackSize = 0;
>>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
>>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>>> estimatedPackSize << std::endl;
>>>
>>> Return the same number, that is 12, the packSize from get_cont is also 12.
>>>
>>> Could you give a hint, what the is problem is here?
>>>
>>> OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04
>>>
>>> Thanks,
>>> Florian
>>>
>>>
>>> ___
>>> users mailing list
>>> users@lists.open-mpi.org
>>> https://lists.open-mpi.org/mailman/listinfo/users
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://lists.open-mpi.org/mailman/listinfo/users
>> 
>___
>users mailing list
>users@lists.open-mpi.org
>https://lists.open-mpi.org/mailman/listinfo/users
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

[Florian Lindner]

Hi Gilles,


Am 14.02.2018 um 11:46 schrieb Gilles Gouaillardet:
> Florian,
> 
> You send position=0 MPI_PACKED instead of estimatedPackSize, so it is very 
> odd you see get_count = 12
> 
> Can you please double check that part ?

https://gist.github.com/floli/310980790d5d76caac0b19a937e2a502

You mean in line 22:

 MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
&req2);

but position was incremented (to 12, see output below) by the preceding 
MPI_Pack call.

> 
> Also, who returns MPI_ERR_TRUNCATE ? MPI_Recv ? MPI_Unpack ?

Sorry, forgot to include that crucial part:

% mpirun -n 1 ./a.out
packSize = 12, estimatedPackSize = 12
position after pack = 12
packSize from get_count = 12
[asaru:30337] *** An error occurred in MPI_Unpack
[asaru:30337] *** reported by process [4237492225,0]
[asaru:30337] *** on communicator MPI_COMM_WORLD
[asaru:30337] *** MPI_ERR_TRUNCATE: message truncated
[asaru:30337] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now 
abort,
[asaru:30337] ***and potentially your MPI job)

Best Thanks,
Florian

> 
> 
> Cheers,
> 
> Gilles
> 
> Florian Lindner  wrote:
>> Hello,
>>
>> I have this example code:
>>
>> #include 
>> #include 
>>
>> int main(int argc, char *argv[])
>> {
>>  MPI_Init(&argc, &argv);
>>  {
>>MPI_Request req1, req2;
>>std::vector vec = {1, 2, 3};
>>int packSize = sizeof(int) * vec.size();
>>int position = 0;
>>std::vector packSendBuf(packSize);
>>int vecSize = vec.size();
>>MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
>> &position, MPI_COMM_WORLD);
>>
>>int estimatedPackSize = 0;
>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>> estimatedPackSize << std::endl;
>>
>>MPI_Isend(&vecSize, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, &req1);
>>MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
>> &req2);
>>  }
>>  {
>>int vecSize, msgSize;
>>int packSize = 0, position = 0;
>>
>>MPI_Recv(&vecSize, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
>> MPI_STATUS_IGNORE);
>>
>>MPI_Status status;
>>MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, &status);
>>MPI_Get_count(&status, MPI_PACKED, &packSize);
>>char packBuffer[packSize];
>>std::cout << "packSize from get_count = " << packSize << std::endl;
>>
>>std::vector vec(vecSize);
>>MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, 
>> MPI_COMM_WORLD, MPI_STATUS_IGNORE);
>>MPI_Unpack(packBuffer, vecSize, &position, vec.data(), vecSize, MPI_INT, 
>> MPI_COMM_WORLD);
>>  }
>>  MPI_Finalize();
>> }
>>
>>
>> Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. Background 
>> is that I want to send multiple differently
>> sized objects, also with more complex types that to not map to MPI_*, for 
>> which I plan to use MPI_BYTES. I plan to pack
>> them into one stream and unpack them one after one.
>>
>> I suspect I got somthig with the sizes wrong. The lines
>>
>>int estimatedPackSize = 0;
>>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
>>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
>> estimatedPackSize << std::endl;
>>
>> Return the same number, that is 12, the packSize from get_cont is also 12.
>>
>> Could you give a hint, what the is problem is here?
>>
>> OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04
>>
>> Thanks,
>> Florian
>>
>>
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://lists.open-mpi.org/mailman/listinfo/users
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users
> 
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] ERR_TRUNCATE with MPI_Pack

[Gilles Gouaillardet]

Florian,

You send position=0 MPI_PACKED instead of estimatedPackSize, so it is very odd 
you see get_count = 12

Can you please double check that part ?

Also, who returns MPI_ERR_TRUNCATE ? MPI_Recv ? MPI_Unpack ?


Cheers,

Gilles

Florian Lindner  wrote:
>Hello,
>
>I have this example code:
>
>#include 
>#include 
>
>int main(int argc, char *argv[])
>{
>  MPI_Init(&argc, &argv);
>  {
>MPI_Request req1, req2;
>std::vector vec = {1, 2, 3};
>int packSize = sizeof(int) * vec.size();
>int position = 0;
>std::vector packSendBuf(packSize);
>int vecSize = vec.size();
>MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
> &position, MPI_COMM_WORLD);
>
>int estimatedPackSize = 0;
>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
> estimatedPackSize << std::endl;
>
>MPI_Isend(&vecSize, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, &req1);
>MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
> &req2);
>  }
>  {
>int vecSize, msgSize;
>int packSize = 0, position = 0;
>
>MPI_Recv(&vecSize, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
> MPI_STATUS_IGNORE);
>
>MPI_Status status;
>MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, &status);
>MPI_Get_count(&status, MPI_PACKED, &packSize);
>char packBuffer[packSize];
>std::cout << "packSize from get_count = " << packSize << std::endl;
>
>std::vector vec(vecSize);
>MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
> MPI_STATUS_IGNORE);
>MPI_Unpack(packBuffer, vecSize, &position, vec.data(), vecSize, MPI_INT, 
> MPI_COMM_WORLD);
>  }
>  MPI_Finalize();
>}
>
>
>Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. Background 
>is that I want to send multiple differently
>sized objects, also with more complex types that to not map to MPI_*, for 
>which I plan to use MPI_BYTES. I plan to pack
>them into one stream and unpack them one after one.
>
>I suspect I got somthig with the sizes wrong. The lines
>
>int estimatedPackSize = 0;
>MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
>std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
> estimatedPackSize << std::endl;
>
>Return the same number, that is 12, the packSize from get_cont is also 12.
>
>Could you give a hint, what the is problem is here?
>
>OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04
>
>Thanks,
>Florian
>
>
>___
>users mailing list
>users@lists.open-mpi.org
>https://lists.open-mpi.org/mailman/listinfo/users
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

[OMPI users] ERR_TRUNCATE with MPI_Pack

[Florian Lindner]

Hello,

I have this example code:

#include 
#include 

int main(int argc, char *argv[])
{
  MPI_Init(&argc, &argv);
  {
MPI_Request req1, req2;
std::vector vec = {1, 2, 3};
int packSize = sizeof(int) * vec.size();
int position = 0;
std::vector packSendBuf(packSize);
int vecSize = vec.size();
MPI_Pack(vec.data(), vec.size(), MPI_INT, packSendBuf.data(), packSize, 
&position, MPI_COMM_WORLD);

int estimatedPackSize = 0;
MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
estimatedPackSize << std::endl;

MPI_Isend(&vecSize, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, &req1);
MPI_Isend(packSendBuf.data(), position, MPI_PACKED, 0, 0, MPI_COMM_WORLD, 
&req2);
  }
  {
int vecSize, msgSize;
int packSize = 0, position = 0;

MPI_Recv(&vecSize, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
MPI_STATUS_IGNORE);

MPI_Status status;
MPI_Probe(0, MPI_ANY_TAG, MPI_COMM_WORLD, &status);
MPI_Get_count(&status, MPI_PACKED, &packSize);
char packBuffer[packSize];
std::cout << "packSize from get_count = " << packSize << std::endl;

std::vector vec(vecSize);
MPI_Recv(packBuffer, packSize, MPI_PACKED, 0, MPI_ANY_TAG, MPI_COMM_WORLD, 
MPI_STATUS_IGNORE);
MPI_Unpack(packBuffer, vecSize, &position, vec.data(), vecSize, MPI_INT, 
MPI_COMM_WORLD);
  }
  MPI_Finalize();
}


Which gives an MPI_ERR_TRUNCATE even when running on 1 rank only. Background is 
that I want to send multiple differently
sized objects, also with more complex types that to not map to MPI_*, for which 
I plan to use MPI_BYTES. I plan to pack
them into one stream and unpack them one after one.

I suspect I got somthig with the sizes wrong. The lines

int estimatedPackSize = 0;
MPI_Pack_size(vec.size(), MPI_INT, MPI_COMM_WORLD, &estimatedPackSize);
std::cout << "packSize = " << packSize << ", estimatedPackSize = " << 
estimatedPackSize << std::endl;

Return the same number, that is 12, the packSize from get_cont is also 12.

Could you give a hint, what the is problem is here?

OpenMPI 3.0.0 @ Arch or OpenMPI 1.1.0.2 @ Ubuntu 16.04

Thanks,
Florian


___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

[OMPI users] MacOS - Running MPI code with 256 procs - “[warn] select: Invalid argument” message

[Christophe Petit]

Hello,

Using Open-Mpi 3.0 and following the tutorial on this link
,
I try to run a MPI code under MacOS 10.9.5 (Mavericks) with a number of
process equal to 256 : the MPI code allocates for each process a 512x512 2D
array, so it requires 256*256kB = 64MB of total used memory.

My MacOS has 16GB RAM and 8 cores, so it seems to be weird.

For a number of process lower than 256 (I tried : np=2,4,8,16,32,64,128),
there is no problem, execution is good and I get expected results.

But for np = 256, I get the following message which repeats itself :

$ mpirun -np 256 ./explicitPar

[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
 ...

I tried also to use -mca option by doing :

$ mpirun -mca opal_set_max_sys_limits 1 -np 256 ./explicitPar

But I get the same warning message.

>From this link

cited above, I did :

$ launchctl limit maxfiles

  maxfiles65536  20

Then, in root user, I created /etc/launchd.conf file and put into :

limit maxfiles 65536 20

I restarted the system for the new limits to take effect and type as normal
user :

$ launchctl limit maxfiles

maxfiles65536  20

But unfortunately, these modifications have no effects on the MPI function
"mpirun" with 256 processes and don't make disappear the warning above.

On Linux platform, I can launch my MPI code with np = 256, without problem,
the issue is only happening on MacOS 10.9.5.

I didn't get this issue with previous version of Open-MPI.

Any idea ? Thanks
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users