Folks,
this is a question (and my reply) initially posted on the users ML
in v1.10 libmpi.so depends on libibverbs.so.1 because of
ompi/mca/common/verbs_usnic
i can understand this is not desired if OpenMPI install tree is shared
with nodes without infiniband libraries.
so far, the non-user friendly workaround is to
configure DISABLE_common_verbs_usnic=1
configure --without-usnic still builds ompi/mca/common/verbs_usnic, and
hence the dependency on libibverbs.so.1
i made the attached patch as a proof of concept :
note there are now two AC_ARG_WITH([usnic],...)
- ompi/mca/common/verbs_usnic/configure.m4
- ompi/mca/btl/usnic/configure.m4
though it seems to work as expected, i am not sure this is the right way
to do things.
any thoughts ?
Cheers,
Gilles
-------- Forwarded Message --------
Subject: Re: [OMPI users] New libmpi.so dependency on libibverbs.so?
List-Post: devel@lists.open-mpi.org
Date: Tue, 2 Feb 2016 10:26:53 +0900
From: Gilles Gouaillardet <gil...@rist.or.jp>
To: Open MPI Users <us...@open-mpi.org>
Simon,
this is an usnic requirement
(mca/common/verbs_usnic to be more specific)
as a workaround (and assuming you do not need usnic stuff on any of your
nodes) you can
configure DISABLE_common_verbs_usnic=1 ... && make install
note that if libibverbs.so.1 is missing on some nodes, you will get warnings
(such as cannot dlopen mca_btl_openib.so) on these nodes unless you specify
--mca btl ^openib
on these nodes
i think it would be much easier if libibverbs.so.1 were available on all
your nodes, including those with no infiniband hardware
Cheers,
Gilles
On 2/2/2016 2:52 AM, Number Cruncher wrote:
Having compiled various recent Open MPI sources with the same
"configure" options, I've noticed that the "libmpi.so" shared library
from 1.10.1 now depends itself directly on libibverbs.so.1.
Previously, 1.10.0 for example, only plugins such as mca_btl_openib.so
depended on it.
E.g.
readelf -d ../ompi_install4/lib/libmpi.so.12.0.0:
0x0000000000000001 (NEEDED) Shared library:
[libopen-rte.so.12]
0x0000000000000001 (NEEDED) Shared library:
[libopen-pal.so.13]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libutil.so.1]
0x0000000000000001 (NEEDED) Shared library:
[libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libmpi.so.12]
readelf -d ../ompi_install4/lib/libmpi.so.12.0.1
0x0000000000000001 (NEEDED) Shared library:
[libibverbs.so.1]
0x0000000000000001 (NEEDED) Shared library:
[libopen-rte.so.12]
0x0000000000000001 (NEEDED) Shared library:
[libopen-pal.so.13]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libutil.so.1]
0x0000000000000001 (NEEDED) Shared library:
[libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libmpi.so.12]
This could make it difficult to share an install between machines. Is
this intended?
Thanks,
Simon
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/02/28424.php
diff --git a/ompi/mca/common/verbs_usnic/configure.m4
b/ompi/mca/common/verbs_usnic/configure.m4
index 60aaa62..ad6eeb1 100644
--- a/ompi/mca/common/verbs_usnic/configure.m4
+++ b/ompi/mca/common/verbs_usnic/configure.m4
@@ -40,15 +40,25 @@ AC_DEFUN([MCA_ompi_common_verbs_usnic_CONFIG],[
AC_CONFIG_FILES([ompi/mca/common/verbs_usnic/Makefile])
common_verbs_usnic_happy="no"
- OMPI_CHECK_OPENFABRICS([common_verbs_usnic],
- [common_verbs_usnic_happy="yes"])
+ AC_ARG_WITH([usnic],
+ [AS_HELP_STRING([--with-usnic],
+ [If specified, cause an error if usNIC
+ support cannot be built])])
- AS_IF([test "$common_verbs_usnic_happy" = "yes"],
- [$1],
- [$2])
+ # If --without-usnic was specified, then gracefully exit.
+ # Otherwise, do the rest of the config.
+ AS_IF([test "x$with_usnic" = "xno"],
+ [AC_MSG_WARN([--without-usnic specified; skipping verbs_usnic])
+ $2],
+ [OMPI_CHECK_OPENFABRICS([common_verbs_usnic],
+ [common_verbs_usnic_happy="yes"])
- # substitute in the things needed to build openib
- AC_SUBST([common_verbs_usnic_CPPFLAGS])
- AC_SUBST([common_verbs_usnic_LDFLAGS])
- AC_SUBST([common_verbs_usnic_LIBS])
+ AS_IF([test "$common_verbs_usnic_happy" = "yes"],
+ [$1],
+ [$2])
+
+ # substitute in the things needed to build openib
+ AC_SUBST([common_verbs_usnic_CPPFLAGS])
+ AC_SUBST([common_verbs_usnic_LDFLAGS])
+ AC_SUBST([common_verbs_usnic_LIBS])])
])dnl
