Thanks Paul,
i just found an other bug ...
(and i should be blamed for it)
here is attached a patch.
basically, xrc was incorrectly disabled on "older" ofed stacks
Cheers,
Gilles
On 7/10/2015 10:06 AM, Paul Hargrove wrote:
Gilles,
A bzip2-compressed config.log is attached.
I am unsure how to determine the OFED version, because the admins have
prevented normal users from reading the RPM database.
Perhaps the following helps:
$ nm /usr/lib64/libibverbs.a | grep -i xrc
00000000000000e0 T ibv_cmd_close_xrc_domain
0000000000000230 T ibv_cmd_create_xrc_rcv_qp
00000000000003b0 T ibv_cmd_create_xrc_srq
0000000000000a40 T ibv_cmd_modify_xrc_rcv_qp
0000000000000150 T ibv_cmd_open_xrc_domain
0000000000001e30 T ibv_cmd_query_xrc_rcv_qp
0000000000000070 T ibv_cmd_reg_xrc_rcv_qp
0000000000000000 T ibv_cmd_unreg_xrc_rcv_qp
00000000000002b0 T ibv_close_xrc_domain
00000000000002d0 T ibv_create_xrc_rcv_qp
00000000000007a0 T ibv_create_xrc_srq
0000000000000310 T ibv_modify_xrc_rcv_qp
0000000000000280 T ibv_open_xrc_domain
0000000000000340 T ibv_query_xrc_rcv_qp
0000000000000370 T ibv_reg_xrc_rcv_qp
0000000000000390 T ibv_unreg_xrc_rcv_qp
$ grep XRC /usr/include/infiniband/verbs.h
IBV_DEVICE_XRC = 1 << 20
IBV_XRC_QP_EVENT_FLAG = 0x80000000,
IBV_QPT_XRC,
[matches in comments have been removed].
When tonight's master tarball is posted (perhaps 10 minutes from now)
I will test it and report what I find.
-Paul
On Thu, Jul 9, 2015 at 5:17 PM, Gilles Gouaillardet <[email protected]
<mailto:[email protected]>> wrote:
Paul,
can you please compress and post your config.log ?
what is the OFED version you are running ?
on master, that fix did the trick on mellanox test cluster (recent
OFED version) but did not
enable XRC on lanl test clusters (my best bet is an old OFED library)
Thanks
Gilles
On 7/10/2015 9:08 AM, Paul Hargrove wrote:
Preliminary report:
1) I find that "ConnectX XRC support" is still not detected as it
was in 1.8.4 and earlier:
$ grep 'ConnectX XRC support'
openmpi-1.*-icc-14/LOG/configure.log| sort -u
openmpi-1.8-linux-x86_64-icc-14/LOG/configure.log:checking if
ConnectX XRC support is enabled... yes
openmpi-1.8.1-linux-x86_64-icc-14/LOG/configure.log:checking
if ConnectX XRC support is enabled... yes
openmpi-1.8.2-linux-x86_64-icc-14/LOG/configure.log:checking
if ConnectX XRC support is enabled... yes
openmpi-1.8.3-linux-x86_64-icc-14/LOG/configure.log:checking
if ConnectX XRC support is enabled... yes
openmpi-1.8.4-linux-x86_64-icc-14/LOG/configure.log:checking
if ConnectX XRC support is enabled... yes
openmpi-1.8.5-linux-x86_64-icc-14/LOG/configure.log:checking
if ConnectX XRC support is enabled... no
openmpi-1.8.6-linux-x86_64-icc-14/LOG/configure.log:checking
if ConnectX XRC support is enabled... no
openmpi-1.8.7rc1-linux-x86_64-icc-14/LOG/configure.log:checking
if ConnectX XRC support is enabled... no
2) I noticed a cosmetic "glitch" in the configure output:
checking for working epoll library interface... checking if epoll can
build... yes
yes
This just means AC_MSG_{CHECKING,RESULT} macros are nested when
they shouldn't be.
There is nothing to suggest that the results of the configure
probes are incorrect.
-Paul
On Thu, Jul 9, 2015 at 1:03 PM, Ralph Castain <[email protected]
<mailto:[email protected]>> wrote:
In the usual place:
http://www.open-mpi.org/software/ompi/v1.8/
Please test and let me know of any issues that surface. My
intent is to release this next week.
Ralph
_______________________________________________
devel mailing list
[email protected] <mailto:[email protected]>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
Link to this post:
http://www.open-mpi.org/community/lists/devel/2015/07/17604.php
--
Paul H. Hargrove [email protected] <mailto:[email protected]>
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department Tel: +1-510-495-2352
<tel:%2B1-510-495-2352>
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
<tel:%2B1-510-486-6900>
_______________________________________________ devel mailing
list [email protected] <mailto:[email protected]> Subscription:
http://www.open-mpi.org/mailman/listinfo.cgi/devel
Link to this
post:http://www.open-mpi.org/community/lists/devel/2015/07/17606.php
_______________________________________________
devel mailing list
[email protected] <mailto:[email protected]>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
Link to this post:
http://www.open-mpi.org/community/lists/devel/2015/07/17607.php
--
Paul H. Hargrove [email protected] <mailto:[email protected]>
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
_______________________________________________
devel mailing list
[email protected]
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
Link to this post:
http://www.open-mpi.org/community/lists/devel/2015/07/17608.php
diff --git a/config/ompi_check_openfabrics.m4 b/config/ompi_check_openfabrics.m4
index 797c987..bb572ae 100644
--- a/config/ompi_check_openfabrics.m4
+++ b/config/ompi_check_openfabrics.m4
@@ -168,7 +168,7 @@ AC_DEFUN([OMPI_CHECK_OPENFABRICS],[
# ibv_cmd_open_xrcd (aka XRC Domains) was added in OFED 3.12
if test "$enable_connectx_xrc" = "yes"; then
$1_have_xrc=1
- AC_CHECK_FUNCS([ibv_create_xrc_rcv_qp ibv_cmd_open_xrcd],
+ AC_CHECK_FUNCS([ibv_create_xrc_rcv_qp],
[], [$1_have_xrc=0])
AC_CHECK_DECLS([IBV_SRQT_XRC],
[], [$1_have_xrc=0],
