Re: [hwloc-devel] hwloc embedding vs libltdl

[Brice Goglin]

Le 08/05/2013 13:09, Jeff Squyres (jsquyres) a écrit :
> 2.  My thoughts were preventing hwloc from going thru some of the pain that 
> OMPI went thru w embedding. Libibverbs has the same problem. If you have 
> middleware that uses plugins that, in turn, uses plugins, it's a bit 
> complicated to support fully static builds  properly (OMPI and hwloc do, but 
> libibverbs doesn't easily, but still, it may require a rebuild of hwloc with 
> enable-static). Also, the problem with opening DSOs in private namespaces 
> still exists; there's no good solution for that (when both the middleware and 
> hwloc use plugins).  

But this issue is only in the embedders (OMPI), not in the embeddee
(hwloc), right?

I can get plugins to work in tests/embedded by adding 2 lines to its
configure.ac (see the attached patch, which also removes your error
message and creates a shared lib containing libhwloc_embedded).

In short, I don't really see what risk we would be taking on the hwloc
side if we keep embedding+plugins possible (and still don't enable
plugins by default).

> 3. Open MPI also get flack for embedding lib ltdl and libevent. Although 
> libltdl now has the built in options for using an external libltdl (which is 
> what the distros use), why go down this road if we don't need to embed 
> libltdl?

OK

Brice

diff --git a/config/hwloc.m4 b/config/hwloc.m4
index 059b72e..818fcc6 100644
--- a/config/hwloc.m4
+++ b/config/hwloc.m4
@@ -1040,12 +1040,6 @@ EOF])
 AS_IF([test "x$enable_plugins" = "xyes" -a "x$hwloc_windows" = "xyes"],
   [AC_MSG_WARN([Plugins not supported on non-native Windows build, plugins support cannot be enabled.])
AC_MSG_ERROR([Cannot continue])])
-# plugins are not supported in embedded mode (indeed, all the LTDL
-# setup stuff is up in hwloc's private configure.ac -- not down
-# here in hwloc.m4)
-AS_IF([test "x$enable_plugins" = "xyes" -a "$hwloc_mode" = "embedded"],
-  [AC_MSG_WARN([Embedded mode not supported with plugins])
-   AC_MSG_ERROR([Cannot continue])])

 AC_ARG_WITH([hwloc-plugins-path],
 		AC_HELP_STRING([--with-hwloc-plugins-path=dir:...],
diff --git a/tests/embedded/Makefile.am b/tests/embedded/Makefile.am
index 9f658f0..eae3c8c 100644
--- a/tests/embedded/Makefile.am
+++ b/tests/embedded/Makefile.am
@@ -10,12 +10,13 @@ AM_CPPFLAGS = $(HWLOC_EMBEDDED_CPPFLAGS)
 TESTS = main

 noinst_PROGRAMS = main
-noinst_LTLIBRARIES = libdo_test.la
+lib_LTLIBRARIES = libdo_test.la

 libdo_test_la_SOURCES = do_test.c
+libdo_test_la_LIBADD = $(HWLOC_EMBEDDED_LDADD) $(HWLOC_EMBEDDED_LIBS)

 main_SOURCES = main.c
-main_LDADD = libdo_test.la $(HWLOC_EMBEDDED_LDADD) $(HWLOC_EMBEDDED_LIBS)
+main_LDADD = libdo_test.la
 main_DEPENDENCIES = libdo_test.la

 EXTRA_DIST = run-embedded-tests.sh
diff --git a/tests/embedded/configure.ac b/tests/embedded/configure.ac
index d697e58..8e57207 100644
--- a/tests/embedded/configure.ac
+++ b/tests/embedded/configure.ac
@@ -19,10 +19,12 @@ cat <

Re: [hwloc-devel] hwloc embedding vs libltdl

[Jeff Squyres (jsquyres)]

I don't think libltdl has a .pc. :(

Sent from my phone. No type good.

On May 8, 2013, at 2:26 AM, "Brice Goglin" 
> wrote:

Le 08/05/2013 02:47, Jeff Squyres (jsquyres) a écrit :

How's this patch?

The only question I have is: how do we figure out what libraries to put in the 
.pc file in the --disable-shared --enable-static case?

There should be a ltdl.pc for this. But I don't see any. Is there a standard 
way to extract this line from ltdl.la ?
dependency_libs=' -ldl'

How about we do not support plugins when --enable-static is given?

Brice







On May 7, 2013, at 8:24 PM, Samuel Thibault 
 wrote:



Jeff Squyres (jsquyres), le Wed 08 May 2013 02:21:02 +0200, a écrit :


On May 7, 2013, at 6:25 PM, Brice Goglin 
 wrote:



I don't have anything against this. What was the reason for not using
the default/system libltdl in OMPI? libtool was buggy when you started
using it?



I neglected to answer this.

Yes, plus libltdl grew new functionality that we needed (global/local symbol 
visibility).

We might be getting to the point soon where we can rely on the installed 
libltdl to be new enough everywhere, but we haven't had that conversation.


We could already check that the installed version is new enough for our
needs.

Samuel
___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel







___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel

___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel

Re: [hwloc-devel] hwloc embedding vs libltdl

[Jeff Squyres (jsquyres)]

1.  Ok. 
2.  My thoughts were preventing hwloc from going thru some of the pain that 
OMPI went thru w embedding. Libibverbs has the same problem. If you have 
middleware that uses plugins that, in turn, uses plugins, it's a bit 
complicated to support fully static builds  properly (OMPI and hwloc do, but 
libibverbs doesn't easily, but still, it may require a rebuild of hwloc with 
enable-static). Also, the problem with opening DSOs in private namespaces still 
exists; there's no good solution for that (when both the middleware and hwloc 
use plugins).  

>From hwlocs perspective, the middleware author can fix the static build 
>option, but I figured: why even go here?

3. Open MPI also get flack for embedding lib ltdl and libevent. Although 
libltdl now has the built in options for using an external libltdl (which is 
what the distros use), why go down this road if we don't need to embed libltdl?

Sent from my phone. No type good. 

On May 8, 2013, at 2:53 AM, "Brice Goglin"  wrote:

> Actually, are we going too far here?
> 
> 1) The original problem was that embedding couldn't build (it couldn't
> even autoreconf) because of the embedded ltdl. Not because of plugins
> being enabled. That's fixed with my patch and with yours.
> tests/embedded/ runs fine now.
> 
> 2) Now, we're disallowing plugins entirely in the embedded case too.
> That's kind of orthogonal to (1). I don't think we currently have a
> single line of code to support this. If people want plugins and
> embedded, thay will need to setup ltdl in their own configure. I don't
> see any reason to prevent this. We may have users wanting this one day,
> so I think we should remove the current error message.
> 
> 3) And we're disallowing included ltdl too. I am actually not against
> this one. We don't use advanced ltdl features, and I don't plan to
> change the plugin management code. So the installed ltdl should be fine.
> But now that (1) is fixed, there's no direct reason to do (3) immediately.
> 
> Brice
> 
> ___
> hwloc-devel mailing list
> hwloc-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel

Re: [hwloc-devel] hwloc embedding vs libltdl

[Brice Goglin]

Actually, are we going too far here?

1) The original problem was that embedding couldn't build (it couldn't
even autoreconf) because of the embedded ltdl. Not because of plugins
being enabled. That's fixed with my patch and with yours.
tests/embedded/ runs fine now.

2) Now, we're disallowing plugins entirely in the embedded case too.
That's kind of orthogonal to (1). I don't think we currently have a
single line of code to support this. If people want plugins and
embedded, thay will need to setup ltdl in their own configure. I don't
see any reason to prevent this. We may have users wanting this one day,
so I think we should remove the current error message.

3) And we're disallowing included ltdl too. I am actually not against
this one. We don't use advanced ltdl features, and I don't plan to
change the plugin management code. So the installed ltdl should be fine.
But now that (1) is fixed, there's no direct reason to do (3) immediately.

Brice

Re: [hwloc-devel] [hwloc-svn] svn:hwloc r5606 - trunk

[Brice Goglin]

We actually used C++ during make check (we test the C++ build of
doc/hwloc-hello.c)
(got a build failure report from https://ci.inria.fr/hwloc/)

Brice



Le 08/05/2013 02:27, svn-commit-mai...@open-mpi.org a écrit :
> Author: jsquyres (Jeff Squyres)
> Date: 2013-05-07 20:27:25 EDT (Tue, 07 May 2013)
> New Revision: 5606
> URL: https://svn.open-mpi.org/trac/hwloc/changeset/5606
>
> Log:
> Revert r5604 -- it's redundant with LT_LANG([C]).
>
> Text files modified: 
>trunk/configure.ac | 4 
>1 files changed, 0 insertions(+), 4 deletions(-)
>
> Modified: trunk/configure.ac
> ==
> --- trunk/configure.acTue May  7 20:18:05 2013(r5605)
> +++ trunk/configure.ac2013-05-07 20:27:25 EDT (Tue, 07 May 2013)  
> (r5606)
> @@ -166,10 +166,6 @@
>  # Compiler support -- we don't need that stuff.
>  AM_ENABLE_SHARED
>  AM_DISABLE_STATIC
> -# Tell libtool that we don't need Fortran or C++ support.
> -FC=no
> -F77=no
> -CXX=no
>  AM_PROG_LIBTOOL([dlopen win32-dll])
>  LT_LANG([C])
>  LT_CONFIG_LTDL_DIR([src/libltdl])
> ___
> hwloc-svn mailing list
> hwloc-...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-svn

Re: [hwloc-devel] hwloc embedding vs libltdl

[Brice Goglin]

Le 08/05/2013 02:47, Jeff Squyres (jsquyres) a écrit :
> How's this patch?
>
> The only question I have is: how do we figure out what libraries to put in 
> the .pc file in the --disable-shared --enable-static case?

There should be a ltdl.pc for this. But I don't see any. Is there a
standard way to extract this line from ltdl.la ?
dependency_libs=' -ldl'

How about we do not support plugins when --enable-static is given?

Brice




>
>
> On May 7, 2013, at 8:24 PM, Samuel Thibault  wrote:
>
>> Jeff Squyres (jsquyres), le Wed 08 May 2013 02:21:02 +0200, a écrit :
>>> On May 7, 2013, at 6:25 PM, Brice Goglin  wrote:
>>>
 I don't have anything against this. What was the reason for not using
 the default/system libltdl in OMPI? libtool was buggy when you started
 using it?
>>>
>>> I neglected to answer this.
>>>
>>> Yes, plus libltdl grew new functionality that we needed (global/local 
>>> symbol visibility).
>>>
>>> We might be getting to the point soon where we can rely on the installed 
>>> libltdl to be new enough everywhere, but we haven't had that conversation.
>> We could already check that the installed version is new enough for our
>> needs.
>>
>> Samuel
>> ___
>> hwloc-devel mailing list
>> hwloc-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel
>
>
>
> ___
> hwloc-devel mailing list
> hwloc-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel