On Mon, Sep 21, 2009 at 9:45 AM, Jeff Squyres <jsquy...@cisco.com> wrote:
> Ick; I appreciate Lisandro's quandry, but don't quite know what to do.
>
I'm just asking the library "libopen-pal.so" exposing ltdl calls
wrapped with an "opal_" prefix. This way, the original ltdl calls hare
hidden (no chance to collide with user code using an incompatible
libtool version), but Open MPI provides a portable way to dlopen()
shared libs/dynamic modules. In simple terms, I'm asking
"libopen-pal.so" to contain ltdl wrapper calls like this one:
OMPI_DECLSPEC lt_dlhandle opal_lt_dlopenadvise(const char *filename,
lt_dladvise advise) /* note opal_ prefix! */
{
return lt_dlopenadvise(filename,advise); /* original ltdl call*/
}
Then, third-party code (like mpi4py or any other dynamic MPI module
for any other dynamic language) can do this:
#include <mpi.h>
#if defined(OPEN_MPI)
typedef void *lt_dlhandle;
typedef void *lt_dladvise;
OMPI_DECLSPEC extern lt_dlhandle opal_lt_dlopenadvise(const char *, lt_dladvise)
#endif
...
#if defined(OPEN_MPI)
/* init advice, not shown ... */
opal_lt_dlopenadvise("mpi", advice);
/* destroy advice, not shown ... */
#endif
MPI_Init(0,0);
>
> How about keeping libltdl fvisibility=hidden inside mpi4py?
>
Not sure if I was clear enough in my comments above, but mpi4py does
not bundles/link libtool. Just abuses on libtool availability in
"libopen-pal.so" for the sake of portability.
>
> On Sep 17, 2009, at 11:16 AM, Josh Hursey wrote:
>
>> So I started down this road a couple months ago. I was using the
>> lt_dlopen() and friends in the OPAL CRS self module. The visibility
>> changes broke that functionality. The one solution that I started
>> implementing was precisely what you suggested, wrapping a subset the
>> libtool calls and prefixing them with opal_*. The email thread is below:
>> http://www.open-mpi.org/community/lists/devel/2009/07/6531.php
>>
>> The problem that I hit was that libtool's build system did not play
>> well with the visibility symbols. This caused dlopen to be disabled
>> incorrectly. The libtool folks have a patch and, I believe, they are
>> planning on incorporating in the next release. The email thread is
>> below:
>> http://thread.gmane.org/gmane.comp.gnu.libtool.patches/9446
>>
>> So we would (others can speak up if not) certainly consider such a
>> wrapper, but I think we need to wait for the next libtool release
>> (unless there is other magic we can do) before it would be usable.
>>
>> Do others have any other ideas on how we might get around this in the
>> mean time?
>>
>> -- Josh
>>
>>
>> On Sep 16, 2009, at 5:59 PM, Lisandro Dalcin wrote:
>>
>> > Hi all.. I have to contact you again about the issues related to
>> > dlopen()ing libmpi with RTLD_LOCAL, as many dynamic languages (Python
>> > in my case) do.
>> >
>> > So far, I've been able to manage the issues (despite the "do nothing"
>> > policy from Open MPI devs, which I understand) in a more or less
>> > portable manner by taking advantage of the availability of libtool
>> > ltdl symbols in the Open MPI libraries (specifically, in libopen-pal).
>> > For reference, all this hackery is here:
>> > http://code.google.com/p/mpi4py/source/browse/trunk/src/compat/openmpi.h
>> >
>> > However, I noticed that in current trunk (v1.4, IIUC) things have
>> > changed and libtool symbols are not externally available. Again, I
>> > understand the reason and acknowledge that such change is a really
>> > good thing. However, this change has broken all my hackery for
>> > dlopen()ing libmpi before the call to MPI_Init().
>> >
>> > Is there any chance that libopen-pal could provide some properly
>> > prefixed (let say, using "opal_" as a prefix) wrapper calls to a small
>> > subset of the libtool ltdl API? The following set of wrapper calls
>> > would is the minimum required to properly load libmpi in a portable
>> > manner and cleanup resources (let me abuse of my previous suggestion
>> > and add the opal_ prefix):
>> >
>> > opal_lt_dlinit()
>> > opal_lt_dlexit()
>> >
>> > opal_lt_dladvise_init(a)
>> > opal_lt_dladvise_destroy(a)
>> > opal_lt_dladvise_global(a)
>> > opal_lt_dladvise_ext(a)
>> >
>> > opal_lt_dlopenadvise(n,a)
>> > opal_lt_dlclose(h)
>> >
>> > Any chance this request could be considered? I would really like to
>> > have this before any Open MPI tarball get released without libtool
>> > symbols exposed...
>> >
>> >
>> > --
>> > Lisandro Dalcín
>> > ---------------
>> > Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
>> > Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
>> > Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
>> > PTLC - Güemes 3450, (3000) Santa Fe, Argentina
>> > Tel/Fax: +54-(0)342-451.1594
>> >
>> > _______________________________________________
>> > devel mailing list
>> > de...@open-mpi.org
>> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>
>>
>> _______________________________________________
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
>
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
--
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
