[OMPI users] seg faults with IB and RH ibverbs-1.1.1-9

2008-08-25 Thread Andrew J Caird
Hello all, We recently applied the latest RedHat update (/etc/redhat-release says "Red Hat Enterprise Linux WS release 4 (Nahant Update 7)") to our cluster, and now codes that use IB seg fault. We have tried multiple versions of OpenMPI and PGI and GNU compilers. We have compiled with --memory-

Re: [OMPI users] RDMA over IB between heterogenous processors with different endianness

2008-08-25 Thread Brian W. Barrett
In the entire 1.2 series, RDMA is only allowed if the architecture of the two processes match. The 1.3 series added the ability to chose based on datatype. Brian On Mon, 25 Aug 2008, Mi Yan wrote: Brian, I'm using OpenMPI 1.2.6 (r17946). Could you plese check which version works ? Thanks

Re: [OMPI users] RDMA over IB between heterogenous processors with different endianness

2008-08-25 Thread Jeff Squyres
I believe that this is something Brad at IBM worked on, wasn't it? I *think* you may just need the development trunk (i.e., upcoming v1.3), but I won't swear to that. Regardless, you need to have OMPI compiled for heterogeneous support because control headers still need to be adjusted for

Re: [OMPI users] RDMA over IB between heterogenous processors with different endianness

2008-08-25 Thread Mi Yan
Brian, I'm using OpenMPI 1.2.6 (r17946). Could you plese check which version works ? Thanks a lot, Mi "Brian W. Barrett"

Re: [OMPI users] RDMA over IB between heterogenous processors with different endianness

2008-08-25 Thread Brian W. Barrett
On Mon, 25 Aug 2008, Mi Yan wrote: Does OpenMPI always use SEND/RECV protocol between heterogeneous processors with different endianness? I tried btl_openib_flags to be 2 , 4 and 6 respectively to allowe RDMA, but the bandwidth between the two heterogeneous nodes is slow, same as the bandwidth

[OMPI users] RDMA over IB between heterogenous processors with different endianness

2008-08-25 Thread Mi Yan
Does OpenMPI always use SEND/RECV protocol between heterogeneous processors with different endianness? I tried btl_openib_flags to be 2 , 4 and 6 respectively to allowe RDMA, but the bandwidth between the two heterogeneous nodes is slow, same as the bandwidth when btl_openib_flags to be 1.