Re: [OMPI devel] Fwd: [OMPI svn-full] svn:open-mpi r16959
Thanks. On Dec 16, 2007, at 5:52 AM, Pavel Shamis (Pasha) wrote: Tested. XRC works. Pasha Pavel Shamis (Pasha) wrote: I will try it. Jeff Squyres wrote: This commit does what we previously discussed: it only compiles the XOOB openib CPC if XRC support is actually present (vs. having a stub XOOB when XRC is not present). This is on the /tmp-public/openib- cpc branch. I have some hermon hca's, but due to dumb issues, I don't have XRC- capable OFED on those nodes yet. It'll probably take me a few more days before I have that ready. Could someone try the openib-cpc tmp branch and ensure I didn't break the case where XRC support is available? It is easy to tell if the XOOB CPC compiled in -- run this command: ompi_info --param btl openib --parsable | grep xoob If the output is empty, then XOOB was not compiled in. If you see output, then XOOB was compiled in. Thanks! Begin forwarded message: From: jsquy...@osl.iu.edu Date: December 14, 2007 12:10:24 PM EST To: svn-f...@open-mpi.org Subject: [OMPI svn-full] svn:open-mpi r16959 Reply-To: de...@open-mpi.org Author: jsquyres Date: 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) New Revision: 16959 URL: https://svn.open-mpi.org/trac/ompi/changeset/16959 Log: Only compile in the XOOB CPC if a) configure found that we have XRC support available and b) the user didn't disable connectx support. Text files modified: tmp-public/openib-cpc/config/ ompi_check_openib.m4 | 3 ++- tmp-public/openib-cpc/ompi/mca/btl/openib/ Makefile.am | 8 ++-- tmp-public/openib-cpc/ompi/mca/btl/openib/ configure.m4 | 8 tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c | 2 ++ tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_xoob.c |23 --- 5 files changed, 18 insertions(+), 26 deletions(-) Modified: tmp-public/openib-cpc/config/ompi_check_openib.m4 = = = = = = = = = = = === --- tmp-public/openib-cpc/config/ompi_check_openib.m4 (original) +++ tmp-public/openib-cpc/config/ompi_check_openib.m4 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -102,7 +102,8 @@ AS_IF([test "$ompi_check_openib_happy" = "yes"], [AC_CHECK_DECLS([IBV_EVENT_CLIENT_REREGISTER], [], [], [#include ]) - AC_CHECK_FUNCS([ibv_get_device_list ibv_resize_cq ibv_open_xrc_domain])]) + AC_CHECK_FUNCS([ibv_get_device_list ibv_resize_cq]) + AC_CHECK_FUNCS([ibv_open_xrc_domain], [$1_have_xrc=1])]) CPPFLAGS="$ompi_check_openib_$1_save_CPPFLAGS" LDFLAGS="$ompi_check_openib_$1_save_LDFLAGS" Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am = = = = = = = = = = = === --- tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am (original) +++ tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -55,14 +55,18 @@ connect/btl_openib_connect_base.c \ connect/btl_openib_connect_oob.c \ connect/btl_openib_connect_oob.h \ -connect/btl_openib_connect_xoob.c \ -connect/btl_openib_connect_xoob.h \ connect/btl_openib_connect_rdma_cm.c \ connect/btl_openib_connect_rdma_cm.h \ connect/btl_openib_connect_ibcm.c \ connect/btl_openib_connect_ibcm.h \ connect/connect.h +if MCA_btl_openib_have_xrc +sources += \ +connect/btl_openib_connect_xoob.c \ +connect/btl_openib_connect_xoob.h +endif + # Make the output library in this directory, and name it either # mca__.la (for DSO builds) or libmca__.la # (for static builds). Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 = = = = = = = = = = = === --- tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 (original) +++ tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -18,6 +18,14 @@ # $HEADER$ # +# MCA_btl_openib_POST_CONFIG([should_build]) +# -- +AC_DEFUN([MCA_btl_openib_POST_CONFIG], [ +AS_IF([test $1 -eq 0 -a "$enable_dist" = "yes"], + [AC_MSG_ERROR([BTL openib is disabled but --enable-dist specifed. This will result in a bad tarball. Aborting configure.])]) +AM_CONDITIONAL([MCA_btl_openib_have_xrc], [test $1 -eq 1 -a "x $btl_openib_have_xrc" = "x1" -a "x$ompi_want_connectx_xrc" = "x1"]) +]) + # MCA_btl_openib_CONFIG([action-if-can-compile], # [action-if-cant-compile]) Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c = = = = = = = = = = = === --- tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c (original) +++
Re: [OMPI devel] Fwd: [OMPI svn-full] svn:open-mpi r16959
Tested. XRC works. Pasha Pavel Shamis (Pasha) wrote: I will try it. Jeff Squyres wrote: This commit does what we previously discussed: it only compiles the XOOB openib CPC if XRC support is actually present (vs. having a stub XOOB when XRC is not present). This is on the /tmp-public/openib-cpc branch. I have some hermon hca's, but due to dumb issues, I don't have XRC- capable OFED on those nodes yet. It'll probably take me a few more days before I have that ready. Could someone try the openib-cpc tmp branch and ensure I didn't break the case where XRC support is available? It is easy to tell if the XOOB CPC compiled in -- run this command: ompi_info --param btl openib --parsable | grep xoob If the output is empty, then XOOB was not compiled in. If you see output, then XOOB was compiled in. Thanks! Begin forwarded message: From: jsquy...@osl.iu.edu Date: December 14, 2007 12:10:24 PM EST To: svn-f...@open-mpi.org Subject: [OMPI svn-full] svn:open-mpi r16959 Reply-To: de...@open-mpi.org Author: jsquyres Date: 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) New Revision: 16959 URL: https://svn.open-mpi.org/trac/ompi/changeset/16959 Log: Only compile in the XOOB CPC if a) configure found that we have XRC support available and b) the user didn't disable connectx support. Text files modified: tmp-public/openib-cpc/config/ ompi_check_openib.m4 | 3 ++- tmp-public/openib-cpc/ompi/mca/btl/openib/ Makefile.am | 8 ++-- tmp-public/openib-cpc/ompi/mca/btl/openib/ configure.m4 | 8 tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c | 2 ++ tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_xoob.c |23 --- 5 files changed, 18 insertions(+), 26 deletions(-) Modified: tmp-public/openib-cpc/config/ompi_check_openib.m4 = = = = = = = = == --- tmp-public/openib-cpc/config/ompi_check_openib.m4 (original) +++ tmp-public/openib-cpc/config/ompi_check_openib.m4 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -102,7 +102,8 @@ AS_IF([test "$ompi_check_openib_happy" = "yes"], [AC_CHECK_DECLS([IBV_EVENT_CLIENT_REREGISTER], [], [], [#include ]) - AC_CHECK_FUNCS([ibv_get_device_list ibv_resize_cq ibv_open_xrc_domain])]) + AC_CHECK_FUNCS([ibv_get_device_list ibv_resize_cq]) + AC_CHECK_FUNCS([ibv_open_xrc_domain], [$1_have_xrc=1])]) CPPFLAGS="$ompi_check_openib_$1_save_CPPFLAGS" LDFLAGS="$ompi_check_openib_$1_save_LDFLAGS" Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am = = = = = = = = == --- tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am (original) +++ tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -55,14 +55,18 @@ connect/btl_openib_connect_base.c \ connect/btl_openib_connect_oob.c \ connect/btl_openib_connect_oob.h \ -connect/btl_openib_connect_xoob.c \ -connect/btl_openib_connect_xoob.h \ connect/btl_openib_connect_rdma_cm.c \ connect/btl_openib_connect_rdma_cm.h \ connect/btl_openib_connect_ibcm.c \ connect/btl_openib_connect_ibcm.h \ connect/connect.h +if MCA_btl_openib_have_xrc +sources += \ +connect/btl_openib_connect_xoob.c \ +connect/btl_openib_connect_xoob.h +endif + # Make the output library in this directory, and name it either # mca__.la (for DSO builds) or libmca__.la # (for static builds). Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 = = = = = = = = == --- tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 (original) +++ tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -18,6 +18,14 @@ # $HEADER$ # +# MCA_btl_openib_POST_CONFIG([should_build]) +# -- +AC_DEFUN([MCA_btl_openib_POST_CONFIG], [ +AS_IF([test $1 -eq 0 -a "$enable_dist" = "yes"], + [AC_MSG_ERROR([BTL openib is disabled but --enable-dist specifed. This will result in a bad tarball. Aborting configure.])]) +AM_CONDITIONAL([MCA_btl_openib_have_xrc], [test $1 -eq 1 -a "x $btl_openib_have_xrc" = "x1" -a "x$ompi_want_connectx_xrc" = "x1"]) +]) + # MCA_btl_openib_CONFIG([action-if-can-compile], # [action-if-cant-compile]) Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c = = = = = = = = == --- tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c (original) +++
Re: [OMPI devel] Fwd: [OMPI svn-full] svn:open-mpi r16959
I will try it. Jeff Squyres wrote: This commit does what we previously discussed: it only compiles the XOOB openib CPC if XRC support is actually present (vs. having a stub XOOB when XRC is not present). This is on the /tmp-public/openib-cpc branch. I have some hermon hca's, but due to dumb issues, I don't have XRC- capable OFED on those nodes yet. It'll probably take me a few more days before I have that ready. Could someone try the openib-cpc tmp branch and ensure I didn't break the case where XRC support is available? It is easy to tell if the XOOB CPC compiled in -- run this command: ompi_info --param btl openib --parsable | grep xoob If the output is empty, then XOOB was not compiled in. If you see output, then XOOB was compiled in. Thanks! Begin forwarded message: From: jsquy...@osl.iu.edu Date: December 14, 2007 12:10:24 PM EST To: svn-f...@open-mpi.org Subject: [OMPI svn-full] svn:open-mpi r16959 Reply-To: de...@open-mpi.org Author: jsquyres Date: 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) New Revision: 16959 URL: https://svn.open-mpi.org/trac/ompi/changeset/16959 Log: Only compile in the XOOB CPC if a) configure found that we have XRC support available and b) the user didn't disable connectx support. Text files modified: tmp-public/openib-cpc/config/ ompi_check_openib.m4 | 3 ++- tmp-public/openib-cpc/ompi/mca/btl/openib/ Makefile.am | 8 ++-- tmp-public/openib-cpc/ompi/mca/btl/openib/ configure.m4 | 8 tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c | 2 ++ tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_xoob.c |23 --- 5 files changed, 18 insertions(+), 26 deletions(-) Modified: tmp-public/openib-cpc/config/ompi_check_openib.m4 = = = = = = = = == --- tmp-public/openib-cpc/config/ompi_check_openib.m4 (original) +++ tmp-public/openib-cpc/config/ompi_check_openib.m4 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -102,7 +102,8 @@ AS_IF([test "$ompi_check_openib_happy" = "yes"], [AC_CHECK_DECLS([IBV_EVENT_CLIENT_REREGISTER], [], [], [#include ]) - AC_CHECK_FUNCS([ibv_get_device_list ibv_resize_cq ibv_open_xrc_domain])]) + AC_CHECK_FUNCS([ibv_get_device_list ibv_resize_cq]) + AC_CHECK_FUNCS([ibv_open_xrc_domain], [$1_have_xrc=1])]) CPPFLAGS="$ompi_check_openib_$1_save_CPPFLAGS" LDFLAGS="$ompi_check_openib_$1_save_LDFLAGS" Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am = = = = = = = = == --- tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am (original) +++ tmp-public/openib-cpc/ompi/mca/btl/openib/Makefile.am 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -55,14 +55,18 @@ connect/btl_openib_connect_base.c \ connect/btl_openib_connect_oob.c \ connect/btl_openib_connect_oob.h \ -connect/btl_openib_connect_xoob.c \ -connect/btl_openib_connect_xoob.h \ connect/btl_openib_connect_rdma_cm.c \ connect/btl_openib_connect_rdma_cm.h \ connect/btl_openib_connect_ibcm.c \ connect/btl_openib_connect_ibcm.h \ connect/connect.h +if MCA_btl_openib_have_xrc +sources += \ +connect/btl_openib_connect_xoob.c \ +connect/btl_openib_connect_xoob.h +endif + # Make the output library in this directory, and name it either # mca__.la (for DSO builds) or libmca__.la # (for static builds). Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 = = = = = = = = == --- tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 (original) +++ tmp-public/openib-cpc/ompi/mca/btl/openib/configure.m4 2007-12-14 12:10:23 EST (Fri, 14 Dec 2007) @@ -18,6 +18,14 @@ # $HEADER$ # +# MCA_btl_openib_POST_CONFIG([should_build]) +# -- +AC_DEFUN([MCA_btl_openib_POST_CONFIG], [ +AS_IF([test $1 -eq 0 -a "$enable_dist" = "yes"], + [AC_MSG_ERROR([BTL openib is disabled but --enable-dist specifed. This will result in a bad tarball. Aborting configure.])]) +AM_CONDITIONAL([MCA_btl_openib_have_xrc], [test $1 -eq 1 -a "x $btl_openib_have_xrc" = "x1" -a "x$ompi_want_connectx_xrc" = "x1"]) +]) + # MCA_btl_openib_CONFIG([action-if-can-compile], # [action-if-cant-compile]) Modified: tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c = = = = = = = = == --- tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c (original) +++ tmp-public/openib-cpc/ompi/mca/btl/openib/connect/ btl_openib_connect_base.c 2007-12-14 12:10:23 EST
