Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
Thanks. > From: jsquy...@cisco.com > To: us...@open-mpi.org > Date: Mon, 28 Sep 2009 11:49:36 -0400 > Subject: Re: [OMPI users] How to create multi-thread parallel program using > thread-safe send and recv? > > On Sep 28, 2009, at 11:48 AM, guosong wrote: > > > Oh, thanks. I found that mpich2/gnu supports MPI_THREAD_MULTIPLE by > > default on my server. So if it supports MPI_THREAD_MULTIPLE, does it > > mean that I can run the program without segmentation fault (if there > > is no other bugs ^_^) > > Hypothetically, yes. :-) > > -- > Jeff Squyres > jsquy...@cisco.com > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users _ Messenger安全保护中心，免费修复系统漏洞，保护Messenger安全！ http://im.live.cn/safe/
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
On Sep 28, 2009, at 11:48 AM, guosong wrote: Oh, thanks. I found that mpich2/gnu supports MPI_THREAD_MULTIPLE by default on my server. So if it supports MPI_THREAD_MULTIPLE, does it mean that I can run the program without segmentation fault (if there is no other bugs ^_^) Hypothetically, yes. :-) -- Jeff Squyres jsquy...@cisco.com
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
Oh, thanks. I found that mpich2/gnu supports MPI_THREAD_MULTIPLE by default on my server. So if it supports MPI_THREAD_MULTIPLE, does it mean that I can run the program without segmentation fault (if there is no other bugs ^_^) > From: jsquy...@cisco.com > To: us...@open-mpi.org > Date: Mon, 28 Sep 2009 11:28:31 -0400 > Subject: Re: [OMPI users] How to create multi-thread parallel program using > thread-safe send and recv? > > On Sep 27, 2009, at 1:45 PM, guosong wrote: > > > Hi Loh, > > I used MPI_Init_thread(,, MPI_THREAD_MULTIPLE, ); > > in my program and got provided = 0 which turns out to be the > > MPI_THREAD_SINGLE. Does this mean that I can not use > > MPI_THREAD_MULTIPLE model? > > Correct. > > To get Open MPI to support MPI_THREAD_MULTIPLE, you need to configure > and build it with the --enable-mpi-threads switch to OMPI's ./ > configure script. We don't build MPI_THREAD_MULTIPLE support by > default because it does add some performance overhead. > > -- > Jeff Squyres > jsquy...@cisco.com > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users _ Messenger安全保护中心，免费修复系统漏洞，保护Messenger安全！ http://im.live.cn/safe/
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
On Sep 27, 2009, at 1:45 PM, guosong wrote: Hi Loh, I used MPI_Init_thread(,, MPI_THREAD_MULTIPLE, ); in my program and got provided = 0 which turns out to be the MPI_THREAD_SINGLE. Does this mean that I can not use MPI_THREAD_MULTIPLE model? Correct. To get Open MPI to support MPI_THREAD_MULTIPLE, you need to configure and build it with the --enable-mpi-threads switch to OMPI's ./ configure script. We don't build MPI_THREAD_MULTIPLE support by default because it does add some performance overhead. -- Jeff Squyres jsquy...@cisco.com
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
The reason I asked the background thread is that I need to make MPI calls in this thread and this is also why I got errors in the little testing program. List-Post: users@lists.open-mpi.org Date: Sun, 27 Sep 2009 14:00:00 -0700 From: eugene@sun.com To: us...@open-mpi.org Subject: Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv? guosong wrote: I used MPI_Init_thread(,, MPI_THREAD_MULTIPLE, ); in my program and got provided = 0 which turns out to be the MPI_THREAD_SINGLE. Does this mean that I can not use MPI_THREAD_MULTIPLE model?Right. I've not done much multithreaded MPI work. Someone else on this list can advise you better what you need to do to get provided=MPI_THREAD_MULTIPLE. I write a little program to test the multithreading and it worked sometimes and failed sometimes. It also hang there sometimes. Does this only because the MPI_THREAD_MULTIPLE is not supported or there are some bugs in the program?I don't know if there are bugs in the program, but without the MPI threads support you can't really test it. BTW, if I want to create a background thread which is sort of like a deamon thread, how can I achieve that in MPI programs? Thanks. I'm not sure I understand the question. Creating a background thread isn't part of MPI. You would use something else, like POSIX threads or OpenMP. _ 约会说不清地方？来试试微软地图最新msn互动功能！ http://ditu.live.com/?form=TL=1
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
guosong wrote: I used MPI_Init_thread(,, MPI_THREAD_MULTIPLE, ); in my program and got provided = 0 which turns out to be the MPI_THREAD_SINGLE. Does this mean that I can not use MPI_THREAD_MULTIPLE model? Right. I've not done much multithreaded MPI work. Someone else on this list can advise you better what you need to do to get provided=MPI_THREAD_MULTIPLE. I write a little program to test the multithreading and it worked sometimes and failed sometimes. It also hang there sometimes. Does this only because the MPI_THREAD_MULTIPLE is not supported or there are some bugs in the program? I don't know if there are bugs in the program, but without the MPI threads support you can't really test it. BTW, if I want to create a background thread which is sort of like a deamon thread, how can I achieve that in MPI programs? Thanks. I'm not sure I understand the question. Creating a background thread isn't part of MPI. You would use something else, like POSIX threads or OpenMP.
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
lt;< endl;
count++;
if(count == myid)
break;
};
cout << "***MAIN SUCESS!" << endl;
}
else
{
pret1 = pthread_create(, NULL, backRecv, NULL);
if(pret1 != 0)
{
cout << myid << "backRecv Thread Create Failed." << endl;
exit(1);
}
MPI_query_msg sMSG, rMSG;
rMSG.flag = myid;
rMSG.x = myid;
rMSG.y = myid;
rMSG.ignited = myid;
sMSG.flag = myid;
sMSG.x = myid;
sMSG.y = myid;
sMSG.ignited = myid;
int left = (myid - 1 + nprocs - 1) % (nprocs - 1);
int right = (myid + 1) % (nprocs - 1);
while(true)
{
MPI_Send(, MSG_QUERY_SIZE, MPI_CHAR, nprocs-1, myid, MPI_COMM_WORLD);
//MPI_Isend(, MSG_QUERY_SIZE, MPI_CHAR, nprocs-1, myid, MPI_COMM_WORLD,
);
//MPI_Wait(, );
cout << "SLAVE: " << myid << " sends to "<< nprocs-1 << " sMSG.x = " <<
sMSG.x << endl;
MPI_Recv(, MSG_QUERY_SIZE, MPI_CHAR, nprocs-1, myid, MPI_COMM_WORLD,
);
//MPI_Irecv(, MSG_QUERY_SIZE, MPI_CHAR, nprocs-1, myid, MPI_COMM_WORLD,
);
//MPI_Wait(, );
cout << "SLAVE: " << myid << " recvs from "<< nprocs-1 << " rMSG.flag = " <<
rMSG.flag << endl;
MPI_Send(, MSG_QUERY_SIZE, MPI_CHAR, right, 222, MPI_COMM_WORLD);
//MPI_Isend(, MSG_QUERY_SIZE, MPI_CHAR, right, 222, MPI_COMM_WORLD,
);
//MPI_Wait(, );
if(rMSG.flag == -1)
{
//MPI_Send(, MSG_QUERY_SIZE, MPI_CHAR, right, 222, MPI_COMM_WORLD);
break;
}
for(j=0; j<(myid+1)*300; ++j)
{}
};
cout << "***SLAVE" << myid << " SUCESS!" << endl;
pthread_join(pt1, NULL);
}
MPI_Finalize();
}
BTW, if I want to create a background thread which is sort of like a deamon
thread, how can I achieve that in MPI programs? Thanks.
List-Post: users@lists.open-mpi.org
Date: Tue, 22 Sep 2009 10:32:50 -0700
From: eugene@sun.com
To: us...@open-mpi.org
Subject: Re: [OMPI users] How to create multi-thread parallel program using
thread-safe send and recv?
guosong wrote:
Thanks for responding. I used a linux cluster. I think I would like to create a
model that is multithreaded and each thread can make MPI calls. I attached test
code as follow. It has two pthreads and there are MPI calls in both of those
two threads. In the main function, there are also MPI calls. Should I use a
full multithreading?I guess so. It seems like the created threads are expected
to make independent/concurrent message-passing calls. Do read the link I sent.
You need to convert from MPI_Init to MPI_Init_thread(), asking for a
full-multithreaded model and checking that you got it. Also note in main()
that the MPI_Isend() calls should be matched with MPI_Wait() or similar calls.
I guess the parent thread will sit in such calls while the child threads do
their own message passing. Good luck.
_
约会说不清地方？来试试微软地图最新msn互动功能！
http://ditu.live.com/?form=TL=1
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
This is just a test example. The real project behind it needs to configure like that. > From: te...@chem.gu.se > To: us...@open-mpi.org > Date: Wed, 23 Sep 2009 09:39:22 +1000 > Subject: Re: [OMPI users] How to create multi-thread parallel program using > thread-safe send and recv? > > If you want all threads to communicate via MPI, and your initially > launching multiple parents, I don't really see the advantage of using > threads at all. Why not launch 12 MPI processes? > > On Tue, 2009-09-22 at 10:32 -0700, Eugene Loh wrote: > > guosong wrote: > > > Thanks for responding. I used a linux cluster. I think I would like > > > to create a model that is multithreaded and each thread can make MPI > > > calls. I attached test code as follow. It has two pthreads and there > > > are MPI calls in both of those two threads. In the main function, > > > there are also MPI calls. Should I use a full multithreading? > > I guess so. It seems like the created threads are expected to make > > independent/concurrent message-passing calls. Do read the link I > > sent. You need to convert from MPI_Init to MPI_Init_thread(), asking > > for a full-multithreaded model and checking that you got it. Also > > note in main() that the MPI_Isend() calls should be matched with > > MPI_Wait() or similar calls. I guess the parent thread will sit in > > such calls while the child threads do their own message passing. Good > > luck. > > ___ > > users mailing list > > us...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users _ Messenger10年嘉年华，礼品大奖等你拿！ http://10.msn.com.cn
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
If you want all threads to communicate via MPI, and your initially launching multiple parents, I don't really see the advantage of using threads at all. Why not launch 12 MPI processes? On Tue, 2009-09-22 at 10:32 -0700, Eugene Loh wrote: > guosong wrote: > > Thanks for responding. I used a linux cluster. I think I would like > > to create a model that is multithreaded and each thread can make MPI > > calls. I attached test code as follow. It has two pthreads and there > > are MPI calls in both of those two threads. In the main function, > > there are also MPI calls. Should I use a full multithreading? > I guess so. It seems like the created threads are expected to make > independent/concurrent message-passing calls. Do read the link I > sent. You need to convert from MPI_Init to MPI_Init_thread(), asking > for a full-multithreaded model and checking that you got it. Also > note in main() that the MPI_Isend() calls should be matched with > MPI_Wait() or similar calls. I guess the parent thread will sit in > such calls while the child threads do their own message passing. Good > luck. > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
d+1)%nprocs, 222, , 1, MPI_INT,
(myid+1)%nprocs, 333, MPI_COMM_WORLD, );
cout << "MAIN:" << myid << " sends to "<< (myid+1)%nprocs << " " << myid <<
endl;
fflush(stdout);
}
pret1 = pthread_create(, NULL, backRecv, NULL);
if(pret1 != 0)
{
cout << myid << "backRecv Thread Create Failed." << endl;
exit(1);
}
pret2 = pthread_create(, NULL, backID, NULL);
if(pret2 != 0)
{
cout << myid << "backID Thread Create Failed." << endl;
exit(1);
}
//for(i=0; i<10; ++i)
//{
// c += i;
// MPI_Send(, 1, MPI_CHAR, (myid+1)%nprocs, 111, MPI_COMM_WORLD);
// cout << myid << " send " << (char)c << " to " << (myid+1)%nprocs << endl;
//}
pthread_join(pt2, NULL);
cout << "***THREAD 2 SUCESS!" << endl;
pthread_join(pt1, NULL);
cout << "***THREAD 1 SUCESS!" << endl;
MPI_Finalize();
cout << "***MAIN SUCESS!" << endl;
}
This peice of code works fine sometimes. But it can fail too and segmentation
will occur. I will read the reference you point out. Thanks.
List-Post: users@lists.open-mpi.org
Date: Tue, 22 Sep 2009 08:43:11 -0700
From: eugene@sun.com
To: us...@open-mpi.org
Subject: Re: [OMPI users] How to create multi-thread parallel program using
thread-safe send and recv?
guosong wrote:
Hi all,
I would like to write a multi-thread parallel program. I used pthread. Basicly,
I want to create two background threads besides the main thread(process). For
example, if I use "-np 4", the program should have 4 main processes on four
processors and two background threads for each main process. So there should be
8 threads totally.Wouldn't there be 4 main threads and 8 "slave" threads for a
total of 12 threads? Anyhow, doesn't matter.
I'm not sure where you're starting, but you should at least have a basic
understanding of the different sorts of multithreaded programming models in
MPI. One is that each process is single threaded. Another is the processes
are multithreaded, but only the main thread makes MPI calls. Another is
multithreaded, but only one MPI call at a time. Finally, there can be full
multithreading. You have to decide which of these programming models you want
and which is supported by your MPI (or, if OMPI, how OMPI was built).
For more information, try the MPI_Init_thread() man page or
http://www.mpi-forum.org./docs/mpi21-report.pdf ... see Section 12.4 on "MPI
and Threads".
I wrote a test program and it worked unpredictable. Sometimes I got the result
I want, but sometimes the program got segmentation fault. I used MPI_Isend and
MPI_Irecv for sending and recving. I do not know why? I attached the error
message as follow:
[cheetah:29780] *** Process received signal ***
[cheetah:29780] Signal: Segmentation fault (11)
[cheetah:29780] Signal code: Address not mapped (1)
[cheetah:29780] Failing at address: 0x10
[cheetah:29779] *** Process received signal ***
[cheetah:29779] Signal: Segmentation fault (11)
[cheetah:29779] Signal code: Address not mapped (1)
[cheetah:29779] Failing at address: 0x10
[cheetah:29780] [ 0] /lib64/libpthread.so.0 [0x334b00de70]
[cheetah:29780] [ 1] /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so [0x2b90e1227940]
[cheetah:29780] [ 2] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so
[0x2b90e05d61ca]
[cheetah:29780] [ 3] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so
[0x2b90e05cac86]
[cheetah:29780] [ 4] /act/openmpi/gnu/lib/libmpi.so.0(PMPI_Send+0x13d)
[0x2b90dde7271d]
[cheetah:29780] [ 5] pt_muti(_Z6backIDPv+0x29b) [0x409929]
[cheetah:29780] [ 6] /lib64/libpthread.so.0 [0x334b0062f7]
[cheetah:29780] [ 7] /lib64/libc.so.6(clone+0x6d) [0x334! a4d1e3d]
[cheetah:29780] *** End of error message ***
[cheetah:29779] [ 0] /lib64/libpthread.so.0 [0x334b00de70]
[cheetah:29779] [ 1] /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so [0x2b39785c0940]
[cheetah:29779] [ 2] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so
[0x2b397796f1ca]
[cheetah:29779] [ 3] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so
[0x2b3977963c86]
[cheetah:29779] [ 4] /act/openmpi/gnu/lib/libmpi.so.0(PMPI_Send+0x13d)
[0x2b397520b71d]
[cheetah:29779] [ 5] pt_muti(_Z6backIDPv+0x29b) [0x409929]
[cheetah:29779] [ 6] /lib64/libpthread.so.0 [0x334b0062f7]
[cheetah:29779] [ 7] /lib64/libc.so.6(clone+0x6d) [0x334a4d1e3d]
[cheetah:29779] *** End of error message ***
I used gdb to "bt" the error and I got :
Program terminated with signal 11, Segmentation fault.
#0 0x2b90e1227940 in mca_btl_sm_alloc ()
from /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so
(gdb) bt
#0 0x2b90e1227940 in mca_btl_sm_alloc ()
from /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so
#1 0x2b90e05d61ca in mca_pml_ob1_send_request_start_copy ()
from /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so
#2 0x2b90e05cac86 in mca_pml_ob1_send ()
from /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so
#3 0x2b90dde7271d in PMPI_Send () from /act/openmpi/gnu/lib/libmpi.so.0
#4 0x00409929 in backID (arg=0x0) at pt_muti.cpp:50
#5 0x00334b0062f7 in start_thread () from /lib64/libpthread.so.0
#6 0x00334a4d1e3d in clone () from /lib64/libc.so.6
So can anyone give me some suggestions or advice. Thanks very much.
_
打工，挣钱，买房子，快来MClub一起”金屋藏娇”！
http://club.msn.cn/?from=10
Re: [OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
guosong wrote: Hi all, I would like to write a multi-thread parallel program. I used pthread. Basicly, I want to create two background threads besides the main thread(process). For example, if I use "-np 4", the program should have 4 main processes on four processors and two background threads for each main process. So there should be 8 threads totally. Wouldn't there be 4 main threads and 8 "slave" threads for a total of 12 threads? Anyhow, doesn't matter. I'm not sure where you're starting, but you should at least have a basic understanding of the different sorts of multithreaded programming models in MPI. One is that each process is single threaded. Another is the processes are multithreaded, but only the main thread makes MPI calls. Another is multithreaded, but only one MPI call at a time. Finally, there can be full multithreading. You have to decide which of these programming models you want and which is supported by your MPI (or, if OMPI, how OMPI was built). For more information, try the MPI_Init_thread() man page or http://www.mpi-forum.org./docs/mpi21-report.pdf ... see Section 12.4 on "MPI and Threads". I wrote a test program and it worked unpredictable. Sometimes I got the result I want, but sometimes the program got segmentation fault. I used MPI_Isend and MPI_Irecv for sending and recving. I do not know why? I attached the error message as follow: [cheetah:29780] *** Process received signal *** [cheetah:29780] Signal: Segmentation fault (11) [cheetah:29780] Signal code: Address not mapped (1) [cheetah:29780] Failing at address: 0x10 [cheetah:29779] *** Process received signal *** [cheetah:29779] Signal: Segmentation fault (11) [cheetah:29779] Signal code: Address not mapped (1) [cheetah:29779] Failing at address: 0x10 [cheetah:29780] [ 0] /lib64/libpthread.so.0 [0x334b00de70] [cheetah:29780] [ 1] /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so [0x2b90e1227940] [cheetah:29780] [ 2] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b90e05d61ca] [cheetah:29780] [ 3] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b90e05cac86] [cheetah:29780] [ 4] /act/openmpi/gnu/lib/libmpi.so.0(PMPI_Send+0x13d) [0x2b90dde7271d] [cheetah:29780] [ 5] pt_muti(_Z6backIDPv+0x29b) [0x409929] [cheetah:29780] [ 6] /lib64/libpthread.so.0 [0x334b0062f7] [cheetah:29780] [ 7] /lib64/libc.so.6(clone+0x6d) [0x334! a4d1e3d] [cheetah:29780] *** End of error message *** [cheetah:29779] [ 0] /lib64/libpthread.so.0 [0x334b00de70] [cheetah:29779] [ 1] /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so [0x2b39785c0940] [cheetah:29779] [ 2] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b397796f1ca] [cheetah:29779] [ 3] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b3977963c86] [cheetah:29779] [ 4] /act/openmpi/gnu/lib/libmpi.so.0(PMPI_Send+0x13d) [0x2b397520b71d] [cheetah:29779] [ 5] pt_muti(_Z6backIDPv+0x29b) [0x409929] [cheetah:29779] [ 6] /lib64/libpthread.so.0 [0x334b0062f7] [cheetah:29779] [ 7] /lib64/libc.so.6(clone+0x6d) [0x334a4d1e3d] [cheetah:29779] *** End of error message *** I used gdb to "bt" the error and I got : Program terminated with signal 11, Segmentation fault. #0 0x2b90e1227940 in mca_btl_sm_alloc () from /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so (gdb) bt #0 0x2b90e1227940 in mca_btl_sm_alloc () from /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so #1 0x2b90e05d61ca in mca_pml_ob1_send_request_start_copy () from /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so #2 0x2b90e05cac86 in mca_pml_ob1_send () from /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so #3 0x2b90dde7271d in PMPI_Send () from /act/openmpi/gnu/lib/libmpi.so.0 #4 0x00409929 in backID (arg=0x0) at pt_muti.cpp:50 #5 0x00334b0062f7 in start_thread () from /lib64/libpthread.so.0 #6 0x00334a4d1e3d in clone () from /lib64/libc.so.6 So can anyone give me some suggestions or advice. Thanks very much.
[OMPI users] How to create multi-thread parallel program using thread-safe send and recv?
Hi all, I would like to write a multi-thread parallel program. I used pthread. Basicly, I want to create two background threads besides the main thread(process). For example, if I use "-np 4", the program should have 4 main processes on four processors and two background threads for each main process. So there should be 8 threads totally. I wrote a test program and it worked unpredictable. Sometimes I got the result I want, but sometimes the program got segmentation fault. I used MPI_Isend and MPI_Irecv for sending and recving. I do not know why? I attached the error message as follow: [cheetah:29780] *** Process received signal *** [cheetah:29780] Signal: Segmentation fault (11) [cheetah:29780] Signal code: Address not mapped (1) [cheetah:29780] Failing at address: 0x10 [cheetah:29779] *** Process received signal *** [cheetah:29779] Signal: Segmentation fault (11) [cheetah:29779] Signal code: Address not mapped (1) [cheetah:29779] Failing at address: 0x10 [cheetah:29780] [ 0] /lib64/libpthread.so.0 [0x334b00de70] [cheetah:29780] [ 1] /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so [0x2b90e1227940] [cheetah:29780] [ 2] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b90e05d61ca] [cheetah:29780] [ 3] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b90e05cac86] [cheetah:29780] [ 4] /act/openmpi/gnu/lib/libmpi.so.0(PMPI_Send+0x13d) [0x2b90dde7271d] [cheetah:29780] [ 5] pt_muti(_Z6backIDPv+0x29b) [0x409929] [cheetah:29780] [ 6] /lib64/libpthread.so.0 [0x334b0062f7] [cheetah:29780] [ 7] /lib64/libc.so.6(clone+0x6d) [0x334a4d1e3d] [cheetah:29780] *** End of error message *** [cheetah:29779] [ 0] /lib64/libpthread.so.0 [0x334b00de70] [cheetah:29779] [ 1] /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so [0x2b39785c0940] [cheetah:29779] [ 2] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b397796f1ca] [cheetah:29779] [ 3] /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so [0x2b3977963c86] [cheetah:29779] [ 4] /act/openmpi/gnu/lib/libmpi.so.0(PMPI_Send+0x13d) [0x2b397520b71d] [cheetah:29779] [ 5] pt_muti(_Z6backIDPv+0x29b) [0x409929] [cheetah:29779] [ 6] /lib64/libpthread.so.0 [0x334b0062f7] [cheetah:29779] [ 7] /lib64/libc.so.6(clone+0x6d) [0x334a4d1e3d] [cheetah:29779] *** End of error message *** I used gdb to "bt" the error and I got : Program terminated with signal 11, Segmentation fault. #0 0x2b90e1227940 in mca_btl_sm_alloc () from /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so (gdb) bt #0 0x2b90e1227940 in mca_btl_sm_alloc () from /act/openmpi/gnu/lib/openmpi/mca_btl_sm.so #1 0x2b90e05d61ca in mca_pml_ob1_send_request_start_copy () from /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so #2 0x2b90e05cac86 in mca_pml_ob1_send () from /act/openmpi/gnu/lib/openmpi/mca_pml_ob1.so #3 0x2b90dde7271d in PMPI_Send () from /act/openmpi/gnu/lib/libmpi.so.0 #4 0x00409929 in backID (arg=0x0) at pt_muti.cpp:50 #5 0x00334b0062f7 in start_thread () from /lib64/libpthread.so.0 #6 0x00334a4d1e3d in clone () from /lib64/libc.so.6 So can anyone give me some suggestions or advice. Thanks very much. _ 上Windows Live 中国首页，下载最新版Messenger！ http://www.windowslive.cn
