[OMPI users] OpenMPI how large its buffer size ?

[Jack Bryan]

Dear All:
How to find the buffer size of OpenMPI ? 
I need to transfer large data between nodes on a cluster with OpenMPI 1.3.4.
Many nodes need to send data to the same node . 
Workers use mpi_isend, the receiver node use  mpi_irecv. 
because they are non-blocking, the messages are stored in buffers of senders. 
And then, the receiver collect messages from its buffer. 
If the receiver's buffer is too small, there will be truncate error. 
Any help is appreciated. 
Jack
July 9  2010
  
_
Hotmail is redefining busy with tools for the New Busy. Get more from your 
inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_2

Re: [OMPI users] Install OpenMPI on Win 7 machine

[Shiqing Fan]

 Hi Alex,

Building Open MPI using CMake is the recommended procedure, you probably 
should stick with it, and it has been proved working well on different 
Windows platforms. The other building procedures are not well supported 
for some time now, and they are deprecated.


For the problems that you got when building with CMake, it might because 
some dependent headers/libraries are not installed. Could you please 
provide the detail CMake error output, so that I can help you fix it?



Regards,
Shiqing


On 2010-7-9 8:19 PM, Alexandru Blidaru wrote:

Hey guys,

This is my first time posting on an email list, so if I don't follow 
certain protocols just tell me. Anyway, I am working at a research lab 
as a High School Summer Intern, and part of my project involves 
installing OpenMPI on a Windows machine to run some applications in 
virtual clusters. I tried following the first two methods from the 
Windows ReadMe file, but none of them work. For the first one, there 
are a bunch of header files cMake can't find. For the second one, the 
cannot find the ompi-static.tgz tarball. The set of files that I 
downloaded from OpenMPI can be found here (tried both tars): 
http://www.open-mpi.org/software/ompi/v1.4/ .


I noticed however that the ompi-static tarball can be found in the 
Subversion that i obtained this 
way:http://www.open-mpi.org/svn/mercurial.php . Can anyone tell me 
what I might be doing wrong? Or maybe more detailed steps? it's the 
first time I work with OpenMPI or any similar technologies.


Thanks for your help,
Alex Blidaru


___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users



--
--
Shiqing Fan  http://www.hlrs.de/people/fan
High Performance Computing   Tel.: +49 711 685 87234
  Center Stuttgart (HLRS)Fax.: +49 711 685 65832
Address:Allmandring 30   email: f...@hlrs.de
70569 Stuttgart

Re: [OMPI users] OpenMPI Hangs, No Error

[Robert Walters]

Hello again,

I believe my administrator has opened the ports I requested. The problem I am 
having now is that OpenMPI is not listening to my defined port assignments in 
openmpi-mca-params.conf (looks like permission 644 on those files should it be 
755?)

When I perform netstat -ltnup I see that orted is listening 14 processes in tcp 
but scaterred in the 26000ish port range when I specified 60001-60016 in the 
mca-params file. Is there a parameter I am missing? In any case I am still 
hanging as mentioned originally even with the port forwarding enabled and 
specifications in mca-param enabled. 

Any other ideas on what might be causing the hang? Is there a more verbose mode 
I can employ to see more deeply into the issue? I have run --debug-daemons and 
--mca plm_base_verbose 99.

Thanks!
--- On Tue, 7/6/10, Robert Walters  wrote:

From: Robert Walters 
Subject: Re: [OMPI users] OpenMPI Hangs, No Error
To: "Open MPI Users" 
List-Post: users@lists.open-mpi.org
Date: Tuesday, July 6, 2010, 5:41 PM

Thanks for your expeditious responses, Ralph.

Just to confirm with you, I should change openmpi-mca-params.conf to include:

oob_tcp_port_min_v4 = (My minimum port in the range)
oob_tcp_port_range_v4 = (My port range)
btl_tcp_port_min_v4 = (My minimum port in the range)
btl_tcp_port_range_v4 = (My port range)

correct?

Also, for a cluster of around 32-64 processes (8 processors per node), how wide 
of a range will I require? I've noticed some entries in the mailing list 
suggesting you need a few to get started and then it opens as necessary. Will I 
be safe with 20 or should I go for 100? 

Thanks again for all of your help!

--- On Tue, 7/6/10, Ralph Castain  wrote:

From:
 Ralph Castain 
Subject: Re: [OMPI users] OpenMPI Hangs, No Error
To: "Open MPI Users" 
List-Post: users@lists.open-mpi.org
Date: Tuesday, July 6, 2010, 5:31 PM

Problem isn't with ssh - the problem is that the daemons need to open a TCP 
connection back to the machine where mpirun is running. If the firewall blocks 
that connection, then we can't run.
If you can get a range of ports opened, then you can specify the ports OMPI 
should use for this purpose. If the sysadmin won't allow even that, then you 
are pretty well hosed.

On Jul 6, 2010, at 2:23 PM, Robert Walters wrote:
Yes, there is a system firewall. I don't think the sysadmin will allow it to go 
disabled. Each Linux machine
 has the built-in RHEL firewall. SSH is enabled through the firewall though.

--- On Tue, 7/6/10, Ralph Castain  wrote:

From: Ralph Castain 
Subject: Re: [OMPI users] OpenMPI Hangs, No Error
To: "Open MPI Users" 
List-Post: users@lists.open-mpi.org
Date: Tuesday, July 6, 2010, 4:19 PM

It looks like the remote daemon is starting - is there a firewall in the way?
On Jul 6, 2010, at 2:04 PM, Robert Walters
 wrote:
Hello all,

I am using OpenMPI 1.4.2 on RHEL. I have a cluster of AMD Opteron's and right 
now I am just working on getting OpenMPI itself up and running. I have a 
successful configure and make all install. LD_LIBRARY_PATH and PATH variables 
were correctly edited. mpirun -np 8 hello_c successfully works on all machines. 
I have setup my two test machines with DSA key pairs that successfully work 
with each other.

The problem comes when I initiate my hostfile to attempt to communicate across 
machines. The hostfile is setup correctly with   . 
When running with all verbose options enabled "mpirun --mca plm_base_verbose 99 
--debug-daemons --mca btl_base_verbose 30 --mca oob_base_verbose 99 --mca
 pml_base_verbose 99 -hostfile hostfile -np 16 hello_c" I receive the following 
text
 output.

[machine1:03578] mca: base: components_open: Looking for plm components
[machine1:03578] mca: base: components_open: opening plm components
[machine1:03578] mca: base: components_open: found loaded component rsh
[machine1:03578] mca: base: components_open: component rsh has no register 
function
[machine1:03578] mca: base: components_open: component rsh open function 
successful
[machine1:03578] mca: base: components_open: found loaded component slurm
[machine1:03578] mca: base: components_open: component slurm has no register 
function
[machine1:03578] mca: base: components_open: component slurm open function 
successful
[machine1:03578] mca:base:select: Auto-selecting plm components
[machine1:03578] mca:base:select:(  plm) Querying component [rsh]
[machine1:03578] mca:base:select:(  plm) Query of component [rsh] set priority 
to 10
[machine1:03578] mca:base:select:(  plm) Querying component
 [slurm]
[machine1:03578] mca:base:select:(  plm) Skipping component [slurm]. Query 
failed to return a module
[machine1:03578] mca:base:select:(  plm) Selected component [rsh]
[machine1:03578] mca: base: close: component slurm closed
[machine1:03578] mca: base: close: unloading component slurm

Re: [OMPI users] OpenMPI Hangs, No Error

[Ralph Castain]

Are there multiple interfaces on your nodes? I'm wondering if we are using a 
different network than the one where you opened these ports.

You'll get quite a bit of output, but you can turn on debug output in the oob 
itself with -mca oob_tcp_verbose xx. The higher the number, the more you get.


On Jul 10, 2010, at 11:14 AM, Robert Walters wrote:

> Hello again,
> 
> I believe my administrator has opened the ports I requested. The problem I am 
> having now is that OpenMPI is not listening to my defined port assignments in 
> openmpi-mca-params.conf (looks like permission 644 on those files should it 
> be 755?)
> 
> When I perform netstat -ltnup I see that orted is listening 14 processes in 
> tcp but scaterred in the 26000ish port range when I specified 60001-60016 in 
> the mca-params file. Is there a parameter I am missing? In any case I am 
> still hanging as mentioned originally even with the port forwarding enabled 
> and specifications in mca-param enabled. 
> 
> Any other ideas on what might be causing the hang? Is there a more verbose 
> mode I can employ to see more deeply into the issue? I have run 
> --debug-daemons and --mca plm_base_verbose 99.
> 
> Thanks!
> --- On Tue, 7/6/10, Robert Walters  wrote:
> 
> From: Robert Walters 
> Subject: Re: [OMPI users] OpenMPI Hangs, No Error
> To: "Open MPI Users" 
> Date: Tuesday, July 6, 2010, 5:41 PM
> 
> Thanks for your expeditious responses, Ralph.
> 
> Just to confirm with you, I should change openmpi-mca-params.conf to include:
> 
> oob_tcp_port_min_v4 = (My minimum port in the range)
> oob_tcp_port_range_v4 = (My port range)
> btl_tcp_port_min_v4 = (My minimum port in the range)
> btl_tcp_port_range_v4 = (My port range)
> 
> correct?
> 
> Also, for a cluster of around 32-64 processes (8 processors per node), how 
> wide of a range will I require? I've noticed some entries in the mailing list 
> suggesting you need a few to get started and then it opens as necessary. Will 
> I be safe with 20 or should I go for 100? 
> 
> Thanks again for all of your help!
> 
> --- On Tue, 7/6/10, Ralph Castain  wrote:
> 
> From: Ralph Castain 
> Subject: Re: [OMPI users] OpenMPI Hangs, No Error
> To: "Open MPI Users" 
> Date: Tuesday, July 6, 2010, 5:31 PM
> 
> Problem isn't with ssh - the problem is that the daemons need to open a TCP 
> connection back to the machine where mpirun is running. If the firewall 
> blocks that connection, then we can't run.
> 
> If you can get a range of ports opened, then you can specify the ports OMPI 
> should use for this purpose. If the sysadmin won't allow even that, then you 
> are pretty well hosed.
> 
> 
> On Jul 6, 2010, at 2:23 PM, Robert Walters wrote:
> 
>> Yes, there is a system firewall. I don't think the sysadmin will allow it to 
>> go disabled. Each Linux machine has the built-in RHEL firewall. SSH is 
>> enabled through the firewall though.
>> 
>> --- On Tue, 7/6/10, Ralph Castain  wrote:
>> 
>> From: Ralph Castain 
>> Subject: Re: [OMPI users] OpenMPI Hangs, No Error
>> To: "Open MPI Users" 
>> Date: Tuesday, July 6, 2010, 4:19 PM
>> 
>> It looks like the remote daemon is starting - is there a firewall in the way?
>> 
>> On Jul 6, 2010, at 2:04 PM, Robert Walters wrote:
>> 
>>> Hello all,
>>> 
>>> I am using OpenMPI 1.4.2 on RHEL. I have a cluster of AMD Opteron's and 
>>> right now I am just working on getting OpenMPI itself up and running. I 
>>> have a successful configure and make all install. LD_LIBRARY_PATH and PATH 
>>> variables were correctly edited. mpirun -np 8 hello_c successfully works on 
>>> all machines. I have setup my two test machines with DSA key pairs that 
>>> successfully work with each other.
>>> 
>>> The problem comes when I initiate my hostfile to attempt to communicate 
>>> across machines. The hostfile is setup correctly with   
>>> . When running with all verbose options enabled "mpirun --mca 
>>> plm_base_verbose 99 --debug-daemons --mca btl_base_verbose 30 --mca 
>>> oob_base_verbose 99 --mca pml_base_verbose 99 -hostfile hostfile -np 16 
>>> hello_c" I receive the following text output.
>>> 
>>> [machine1:03578] mca: base: components_open: Looking for plm components
>>> [machine1:03578] mca: base: components_open: opening plm components
>>> [machine1:03578] mca: base: components_open: found loaded component rsh
>>> [machine1:03578] mca: base: components_open: component rsh has no register 
>>> function
>>> [machine1:03578] mca: base: components_open: component rsh open function 
>>> successful
>>> [machine1:03578] mca: base: components_open: found loaded component slurm
>>> [machine1:03578] mca: base: components_open: component slurm has no 
>>> register function
>>> [machine1:03578] mca: base: components_open: component slurm open function 
>>> successful
>>> [machine1:03578] mca:base:select: 

Re: [OMPI users] OpenMPI Hangs, No Error

[Robert Walters]

I ran oob_tcp_verbose 99 and I am getting something interesting I never got 
before.

[machine 2:22347] bind() failed: no port available in the range [60001-60016]
[machine 2:22347] mca_oob_tcp_init: unable to create IPv4 listen socket: Error

I never got that error before we messed with the iptables but now I get that 
error... Very interesting, I will have to talk to my sysadmin again and make 
sure he opened the right ports on my two test machines. It looks as though 
there are no open ports. Another interesting thing is I see that the Daemon is 
still report:

Daemon [[28845,0],1] checking in as pid 22347 on host machine 2
Daemon [[28845,0],1] not using static ports

Which, I may be misunderstanding, should have been taken care of when I 
specified what ports to use. I am telling it a static set of ports... Anyhow, I 
will get with my sysadmin again and see what he says. At least OpenMPI is 
correctly interpreting the range. 

Thanks for the help.

--- On Sat, 7/10/10, Ralph Castain  wrote:

From: Ralph Castain 
Subject: Re: [OMPI users] OpenMPI Hangs, No Error
To: "Open MPI Users" 
List-Post: users@lists.open-mpi.org
Date: Saturday, July 10, 2010, 3:21 PM

Are there multiple interfaces on your nodes? I'm wondering if we are using a 
different network than the one where you opened these ports.
You'll get quite a bit of output, but you can turn on debug output in the oob 
itself with -mca oob_tcp_verbose xx. The higher the number, the more you get.

On Jul 10, 2010, at 11:14 AM, Robert Walters wrote:
Hello again,

I believe my administrator has opened the ports I requested. The problem I am 
having now is that OpenMPI is not listening to my defined port assignments in 
openmpi-mca-params.conf (looks like permission 644 on those files should it be 
755?)

When I perform netstat -ltnup I see that orted is listening 14 processes in tcp 
but scaterred in the 26000ish port range when I specified 60001-60016 in the 
mca-params file. Is there a parameter I am missing? In any case I am still 
hanging as mentioned originally even with the port forwarding enabled and 
specifications in mca-param enabled. 

Any other ideas on what might be causing the hang? Is there a more verbose mode 
I can employ to see more deeply into the issue? I have run --debug-daemons and 
--mca plm_base_verbose 99.

Thanks!
--- On Tue, 7/6/10, Robert Walters
  wrote:

From: Robert Walters 
Subject: Re: [OMPI users] OpenMPI Hangs, No Error
To: "Open MPI Users" 
List-Post: users@lists.open-mpi.org
Date: Tuesday, July 6, 2010, 5:41 PM

Thanks for your expeditious responses, Ralph.

Just to confirm with you, I should change openmpi-mca-params.conf to include:

oob_tcp_port_min_v4 = (My minimum port in the range)
oob_tcp_port_range_v4 = (My port range)
btl_tcp_port_min_v4 = (My minimum port in the range)
btl_tcp_port_range_v4 = (My port range)

correct?

Also, for a cluster of around 32-64 processes (8 processors per node), how wide 
of a range will I require? I've noticed some entries in
 the mailing list suggesting you need a few to get started and then it opens as 
necessary. Will I be safe with 20 or should I go for 100? 

Thanks again for all of your help!

--- On Tue, 7/6/10, Ralph Castain  wrote:

From:
 Ralph Castain 
Subject: Re: [OMPI users] OpenMPI Hangs, No Error
To: "Open MPI Users" 
List-Post: users@lists.open-mpi.org
Date: Tuesday, July 6, 2010, 5:31 PM

Problem isn't with ssh - the problem is that the daemons need to open a TCP 
connection back to the machine where mpirun is running. If the firewall blocks 
that connection, then we can't run.
If you can get a range of ports opened, then you can specify the ports OMPI 
should use for this purpose. If the sysadmin won't allow even that, then you 
are pretty well hosed.

On Jul 6, 2010, at 2:23 PM, Robert Walters wrote:
Yes, there is a system firewall. I don't think the sysadmin will allow it to go 
disabled. Each Linux machine
 has the built-in RHEL firewall. SSH is enabled through the firewall though.

--- On Tue, 7/6/10, Ralph Castain  wrote:

From: Ralph Castain 
Subject: Re: [OMPI users] OpenMPI Hangs, No Error
To: "Open MPI Users" 
List-Post: users@lists.open-mpi.org
Date: Tuesday, July 6, 2010, 4:19 PM

It looks like the remote daemon is starting - is there a firewall in the way?
On Jul 6, 2010, at 2:04 PM, Robert Walters
 wrote:
Hello all,

I am using OpenMPI 1.4.2 on RHEL. I have a cluster of AMD Opteron's and right 
now I am just working on getting OpenMPI itself up and running. I have a 
successful configure and make all install. LD_LIBRARY_PATH and PATH variables 
were correctly edited. mpirun -np 8 hello_c successfully works on all machines. 
I have setup my two test machines with DSA key pairs that successfully work 
with each other.

Re: [OMPI users] OpenMPI Hangs, No Error

[Ralph Castain]

The "static ports" flag means something different - it is used when the daemon 
is given a fixed port to use. In some installations, we lock every daemon to 
the same port number so that each daemon can compute exactly how to contact its 
peers (i.e., no contact info exchange required for wireup).

You have a "fixed range", but not "static port", scenario. Hence the message.

Let us know how it goes - I agree it sounds like something to discuss with the 
sysadmin.


On Jul 10, 2010, at 1:47 PM, Robert Walters wrote:

> I ran oob_tcp_verbose 99 and I am getting something interesting I never got 
> before.
> 
> [machine 2:22347] bind() failed: no port available in the range [60001-60016]
> [machine 2:22347] mca_oob_tcp_init: unable to create IPv4 listen socket: Error
> 
> I never got that error before we messed with the iptables but now I get that 
> error... Very interesting, I will have to talk to my sysadmin again and make 
> sure he opened the right ports on my two test machines. It looks as though 
> there are no open ports. Another interesting thing is I see that the Daemon 
> is still report:
> 
> Daemon [[28845,0],1] checking in as pid 22347 on host machine 2
> Daemon [[28845,0],1] not using static ports
> 
> Which, I may be misunderstanding, should have been taken care of when I 
> specified what ports to use. I am telling it a static set of ports... Anyhow, 
> I will get with my sysadmin again and see what he says. At least OpenMPI is 
> correctly interpreting the range. 
> 
> Thanks for the help.
> 
> --- On Sat, 7/10/10, Ralph Castain  wrote:
> 
> From: Ralph Castain 
> Subject: Re: [OMPI users] OpenMPI Hangs, No Error
> To: "Open MPI Users" 
> Date: Saturday, July 10, 2010, 3:21 PM
> 
> Are there multiple interfaces on your nodes? I'm wondering if we are using a 
> different network than the one where you opened these ports.
> 
> You'll get quite a bit of output, but you can turn on debug output in the oob 
> itself with -mca oob_tcp_verbose xx. The higher the number, the more you get.
> 
> 
> On Jul 10, 2010, at 11:14 AM, Robert Walters wrote:
> 
>> Hello again,
>> 
>> I believe my administrator has opened the ports I requested. The problem I 
>> am having now is that OpenMPI is not listening to my defined port 
>> assignments in openmpi-mca-params.conf (looks like permission 644 on those 
>> files should it be 755?)
>> 
>> When I perform netstat -ltnup I see that orted is listening 14 processes in 
>> tcp but scaterred in the 26000ish port range when I specified 60001-60016 in 
>> the mca-params file. Is there a parameter I am missing? In any case I am 
>> still hanging as mentioned originally even with the port forwarding enabled 
>> and specifications in mca-param enabled. 
>> 
>> Any other ideas on what might be causing the hang? Is there a more verbose 
>> mode I can employ to see more deeply into the issue? I have run 
>> --debug-daemons and --mca plm_base_verbose 99.
>> 
>> Thanks!
>> --- On Tue, 7/6/10, Robert Walters  wrote:
>> 
>> From: Robert Walters 
>> Subject: Re: [OMPI users] OpenMPI Hangs, No Error
>> To: "Open MPI Users" 
>> Date: Tuesday, July 6, 2010, 5:41 PM
>> 
>> Thanks for your expeditious responses, Ralph.
>> 
>> Just to confirm with you, I should change openmpi-mca-params.conf to include:
>> 
>> oob_tcp_port_min_v4 = (My minimum port in the range)
>> oob_tcp_port_range_v4 = (My port range)
>> btl_tcp_port_min_v4 = (My minimum port in the range)
>> btl_tcp_port_range_v4 = (My port range)
>> 
>> correct?
>> 
>> Also, for a cluster of around 32-64 processes (8 processors per node), how 
>> wide of a range will I require? I've noticed some entries in the mailing 
>> list suggesting you need a few to get started and then it opens as 
>> necessary. Will I be safe with 20 or should I go for 100? 
>> 
>> Thanks again for all of your help!
>> 
>> --- On Tue, 7/6/10, Ralph Castain  wrote:
>> 
>> From: Ralph Castain 
>> Subject: Re: [OMPI users] OpenMPI Hangs, No Error
>> To: "Open MPI Users" 
>> Date: Tuesday, July 6, 2010, 5:31 PM
>> 
>> Problem isn't with ssh - the problem is that the daemons need to open a TCP 
>> connection back to the machine where mpirun is running. If the firewall 
>> blocks that connection, then we can't run.
>> 
>> If you can get a range of ports opened, then you can specify the ports OMPI 
>> should use for this purpose. If the sysadmin won't allow even that, then you 
>> are pretty well hosed.
>> 
>> 
>> On Jul 6, 2010, at 2:23 PM, Robert Walters wrote:
>> 
>>> Yes, there is a system firewall. I don't think the sysadmin will allow it 
>>> to go disabled. Each Linux machine has the built-in RHEL firewall. SSH is 
>>> enabled through the firewall though.
>>> 
>>> --- On Tue, 7/6/10, Ralph Castain  wrote:
>>> 
>>> From: Ralph Castain 
>>> 

Re: [OMPI users] OpenMPI how large its buffer size ?

[jody]

Perhaps i misunderstand your question...
Generally, it is the user's job to provide the buffers both to send and receive.
If you call MPI_Recv, you must pass a buffer that is large enough to
hold the data sent by the
corresponding MPI_Send. I.e., if you know your sender will send
messages of 100kB,
then you must provide a buffer of size 100kB to the receiver.
If the message size is unknown at compile time, you may have to send
two messages:
first an integer which tells the receiver how large a buffer it has to
allocate, and then
the actual message (which then nicely fits into the freshly allocated buffer)

#include 
#include 

#include 


#include "mpi.h"

#define SENDER 1
#define RECEIVER   0
#define TAG_LEN   77
#define TAG_DATA  78
#define MAX_MESSAGE 16

int main(int argc, char *argv[]) {

int num_procs;
int rank;
int *send_buf;
int *recv_buf;
int send_message_size;
int recv_message_size;
MPI_Status st;
int i;

/* initialize random numbers */
srand(time(NULL));
MPI_Init(, );
MPI_Comm_size(MPI_COMM_WORLD, _procs);
MPI_Comm_rank(MPI_COMM_WORLD, );

if (rank == RECEIVER) {
/* the receiver */
/* wait for message length */
MPI_Recv(_message_size, 1, MPI_INT, SENDER, TAG_LEN,
MPI_COMM_WORLD, );
/* create a buffer of the required size */
recv_buf = (int*) malloc(recv_message_size*sizeof(int));
/* get data */
MPI_Recv(recv_buf, recv_message_size, MPI_INT, SENDER,
TAG_DATA, MPI_COMM_WORLD, );

printf("Receiver got %d integers:", recv_message_size);
for (i = 0; i < recv_message_size; i++) {
printf(" %d", recv_buf[i]);
}
printf("\n");

/* clean up */
free(recv_buf);

} else if (rank == SENDER) {
/* the sender */
/* random message size */
send_message_size = (int)((1.0*MAX_MESSAGE*rand())/(1.0*RAND_MAX));
/* create a buffer of the required size */
send_buf = (int*) malloc(send_message_size*sizeof(int));
/* create random message */
for (i = 0; i < send_message_size; i++) {
send_buf[i] = rand();
}

printf("Sender has %d integers:", send_message_size);
for (i = 0; i < send_message_size; i++) {
printf(" %d", send_buf[i]);
}
printf("\n");

/* send message size to receiver */
MPI_Send(_message_size,  1, MPI_INT, RECEIVER, TAG_LEN,
MPI_COMM_WORLD);
/* now send messagge */
MPI_Send(send_buf, send_message_size, MPI_INT, RECEIVER,
TAG_DATA, MPI_COMM_WORLD);

/* clean up */
free(send_buf);

}

MPI_Finalize();
}

I hope this helps
  Jody


On Sat, Jul 10, 2010 at 7:12 AM, Jack Bryan  wrote:
> Dear All:
> How to find the buffer size of OpenMPI ?
> I need to transfer large data between nodes on a cluster with OpenMPI 1.3.4.
> Many nodes need to send data to the same node .
> Workers use mpi_isend, the receiver node use  mpi_irecv.
> because they are non-blocking, the messages are stored in buffers of
> senders.
> And then, the receiver collect messages from its buffer.
> If the receiver's buffer is too small, there will be truncate error.
> Any help is appreciated.
> Jack
> July 9  2010
>
> 
> Hotmail is redefining busy with tools for the New Busy. Get more from your
> inbox. See how.
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

Re: [OMPI users] OpenMPI how large its buffer size ?

[David Zhang]

I believe his question is regarding when under non-blocking send/recv, how
does MPI know how much memory to allocate to receive the message, since the
size is determined AFTER the irecv is posted.  So if the send post isend,
but the receiver hasn't post irecv, what would the MPI do with the message.

I believe MPI would automatically create a buffer in the background to store
the message.

On Sat, Jul 10, 2010 at 1:55 PM, jody  wrote:

> Perhaps i misunderstand your question...
> Generally, it is the user's job to provide the buffers both to send and
> receive.
> If you call MPI_Recv, you must pass a buffer that is large enough to
> hold the data sent by the
> corresponding MPI_Send. I.e., if you know your sender will send
> messages of 100kB,
> then you must provide a buffer of size 100kB to the receiver.
> If the message size is unknown at compile time, you may have to send
> two messages:
> first an integer which tells the receiver how large a buffer it has to
> allocate, and then
> the actual message (which then nicely fits into the freshly allocated
> buffer)
>
> #include 
> #include 
>
> #include 
>
>
> #include "mpi.h"
>
> #define SENDER 1
> #define RECEIVER   0
> #define TAG_LEN   77
> #define TAG_DATA  78
> #define MAX_MESSAGE 16
>
> int main(int argc, char *argv[]) {
>
>int num_procs;
>int rank;
>int *send_buf;
>int *recv_buf;
>int send_message_size;
>int recv_message_size;
>MPI_Status st;
>int i;
>
>/* initialize random numbers */
>srand(time(NULL));
>MPI_Init(, );
>MPI_Comm_size(MPI_COMM_WORLD, _procs);
>MPI_Comm_rank(MPI_COMM_WORLD, );
>
>if (rank == RECEIVER) {
>/* the receiver */
>/* wait for message length */
>MPI_Recv(_message_size, 1, MPI_INT, SENDER, TAG_LEN,
> MPI_COMM_WORLD, );
>/* create a buffer of the required size */
>recv_buf = (int*) malloc(recv_message_size*sizeof(int));
>/* get data */
>MPI_Recv(recv_buf, recv_message_size, MPI_INT, SENDER,
> TAG_DATA, MPI_COMM_WORLD, );
>
>printf("Receiver got %d integers:", recv_message_size);
>for (i = 0; i < recv_message_size; i++) {
>printf(" %d", recv_buf[i]);
>}
>printf("\n");
>
>/* clean up */
>free(recv_buf);
>
>} else if (rank == SENDER) {
>/* the sender */
>/* random message size */
>send_message_size = (int)((1.0*MAX_MESSAGE*rand())/(1.0*RAND_MAX));
>/* create a buffer of the required size */
>send_buf = (int*) malloc(send_message_size*sizeof(int));
>/* create random message */
>for (i = 0; i < send_message_size; i++) {
>send_buf[i] = rand();
>}
>
>printf("Sender has %d integers:", send_message_size);
>for (i = 0; i < send_message_size; i++) {
>printf(" %d", send_buf[i]);
>}
>printf("\n");
>
>/* send message size to receiver */
>MPI_Send(_message_size,  1, MPI_INT, RECEIVER, TAG_LEN,
> MPI_COMM_WORLD);
>/* now send messagge */
>MPI_Send(send_buf, send_message_size, MPI_INT, RECEIVER,
> TAG_DATA, MPI_COMM_WORLD);
>
>/* clean up */
>free(send_buf);
>
>}
>
>MPI_Finalize();
> }
>
> I hope this helps
>  Jody
>
>
> On Sat, Jul 10, 2010 at 7:12 AM, Jack Bryan 
> wrote:
> > Dear All:
> > How to find the buffer size of OpenMPI ?
> > I need to transfer large data between nodes on a cluster with OpenMPI
> 1.3.4.
> > Many nodes need to send data to the same node .
> > Workers use mpi_isend, the receiver node use  mpi_irecv.
> > because they are non-blocking, the messages are stored in buffers of
> > senders.
> > And then, the receiver collect messages from its buffer.
> > If the receiver's buffer is too small, there will be truncate error.
> > Any help is appreciated.
> > Jack
> > July 9  2010
> >
> > 
> > Hotmail is redefining busy with tools for the New Busy. Get more from
> your
> > inbox. See how.
> > ___
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>



-- 
David Zhang
University of California, San Diego

Re: [OMPI users] OpenMPI how large its buffer size ?

[Jack Bryan]

Hi, 
thanks for the program from Jody. 
David indicated the question that I want to ask. 
But, Jody's approach is ok when the MPI built-in buffer size is large enough to 
hold the message such as 100kB in the buffer. 
In asynchronous communication, when the sender posts a mpi_isend, the message 
is put in a buffer provided by the MPI. 
At this point, the receiver may still not post its corresponding mpi_irecv. So, 
the buffer size is important here. 
Without knowing the buffer size, I may get " truncate error " on Open MPI. 
How to know the size of the buffer automatically created by Open MPI in the 
background ?
Any help is appreciated. 
Jack,
July 10 2010
From: solarbik...@gmail.com
List-Post: users@lists.open-mpi.org
Date: Sat, 10 Jul 2010 16:46:12 -0700
To: us...@open-mpi.org
Subject: Re: [OMPI users] OpenMPI how large its buffer size ?

I believe his question is regarding when under non-blocking send/recv, how does 
MPI know how much memory to allocate to receive the message, since the size is 
determined AFTER the irecv is posted.  So if the send post isend, but the 
receiver hasn't post irecv, what would the MPI do with the message.



I believe MPI would automatically create a buffer in the background to store 
the message.

On Sat, Jul 10, 2010 at 1:55 PM, jody  wrote:


Perhaps i misunderstand your question...

Generally, it is the user's job to provide the buffers both to send and receive.

If you call MPI_Recv, you must pass a buffer that is large enough to

hold the data sent by the

corresponding MPI_Send. I.e., if you know your sender will send

messages of 100kB,

then you must provide a buffer of size 100kB to the receiver.

If the message size is unknown at compile time, you may have to send

two messages:

first an integer which tells the receiver how large a buffer it has to

allocate, and then

the actual message (which then nicely fits into the freshly allocated buffer)



#include 

#include 



#include 





#include "mpi.h"



#define SENDER 1

#define RECEIVER   0

#define TAG_LEN   77

#define TAG_DATA  78

#define MAX_MESSAGE 16



int main(int argc, char *argv[]) {



int num_procs;

int rank;

int *send_buf;

int *recv_buf;

int send_message_size;

int recv_message_size;

MPI_Status st;

int i;



/* initialize random numbers */

srand(time(NULL));

MPI_Init(, );

MPI_Comm_size(MPI_COMM_WORLD, _procs);

MPI_Comm_rank(MPI_COMM_WORLD, );



if (rank == RECEIVER) {

/* the receiver */

/* wait for message length */

MPI_Recv(_message_size, 1, MPI_INT, SENDER, TAG_LEN,

MPI_COMM_WORLD, );

/* create a buffer of the required size */

recv_buf = (int*) malloc(recv_message_size*sizeof(int));

/* get data */

MPI_Recv(recv_buf, recv_message_size, MPI_INT, SENDER,

TAG_DATA, MPI_COMM_WORLD, );



printf("Receiver got %d integers:", recv_message_size);

for (i = 0; i < recv_message_size; i++) {

printf(" %d", recv_buf[i]);

}

printf("\n");



/* clean up */

free(recv_buf);



} else if (rank == SENDER) {

/* the sender */

/* random message size */

send_message_size = (int)((1.0*MAX_MESSAGE*rand())/(1.0*RAND_MAX));

/* create a buffer of the required size */

send_buf = (int*) malloc(send_message_size*sizeof(int));

/* create random message */

for (i = 0; i < send_message_size; i++) {

send_buf[i] = rand();

}



printf("Sender has %d integers:", send_message_size);

for (i = 0; i < send_message_size; i++) {

printf(" %d", send_buf[i]);

}

printf("\n");



/* send message size to receiver */

MPI_Send(_message_size,  1, MPI_INT, RECEIVER, TAG_LEN,

MPI_COMM_WORLD);

/* now send messagge */

MPI_Send(send_buf, send_message_size, MPI_INT, RECEIVER,

TAG_DATA, MPI_COMM_WORLD);



/* clean up */

free(send_buf);



}



MPI_Finalize();

}



I hope this helps

  Jody





On Sat, Jul 10, 2010 at 7:12 AM, Jack Bryan  wrote:

> Dear All:

> How to find the buffer size of OpenMPI ?

> I need to transfer large data between nodes on a cluster with OpenMPI 1.3.4.

> Many nodes need to send data to the same node .

> Workers use mpi_isend, the receiver node use  mpi_irecv.

> because they are non-blocking, the messages are stored in buffers of

> senders.

> And then, the receiver collect messages from its buffer.

> If the receiver's buffer is too small, there will be truncate error.

> Any help is appreciated.

> Jack

> July 9  2010

>

> 

> Hotmail is redefining busy with tools for the New Busy. Get more from your

> inbox. See how.

> ___

> users mailing list

> us...@open-mpi.org

>