Re: Mac OS X .dylib not working
On 03/04/2011 01:00 PM, Peter O'Gorman wrote: On 03/04/2011 12:47 PM, Ralf Wildenhues wrote: +if test $shlibpath_var = PATH; then This looks wrong; shouldn't it be != here? Otherwise, ... Looking at the original test, the above is correct there - it wants to avoid messing with PATH on Windows. Pushed this. Thanks. Peter From e51daabd9975b141ed491f2f49bb340b797a6383 Mon Sep 17 00:00:00 2001 From: Peter O'Gorman pe...@pogma.com Date: Fri, 4 Mar 2011 14:35:14 -0600 Subject: [PATCH] On Mac OS X try .dylib as well as .so with lt_dlopenext * libltdl/m4/ltdl.m4: Define extra extension if module extension differs from shared lib extension. * libltdl/ltdl.c: Use it. * tests/darwin.at: Test it. * NEWS: Announce it. Reported by Hans Aberg, Michael Ellis, and others. --- ChangeLog | 10 +++ NEWS |3 + libltdl/ltdl.c | 21 +- libltdl/m4/ltdl.m4 |9 ++- tests/darwin.at| 216 5 files changed, 257 insertions(+), 2 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7f74eab..528e404 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,13 @@ +2011-03-04 Peter O'Gorman pe...@pogma.com + + On Mac OS X try .dylib as well as .so with lt_dlopenext + * libltdl/m4/ltdl.m4: Define extra extension if module extension + differs from shared lib extension. + * libltdl/ltdl.c: Use it. + * tests/darwin.at: Test it. + * NEWS: Announce it. + Reported by Hans Aberg, Michael Ellis, and others. + 2011-02-12 Peter O'Gorman pe...@pogma.com Install ltmain.sh without execute bit set. diff --git a/NEWS b/NEWS index dbad2ae..90d14b7 100644 --- a/NEWS +++ b/NEWS @@ -10,6 +10,9 @@ New in 2.4.2 2011-??-??: git version 2.4.1a, Libtool team: - Initial support for Go, using the gccgo compiler. + - On Mac OS X .dylib is now tried as well as .so with +lt_dlopenext(). + * Bug fixes: - The generic approximation of the command line length limit (when getconf is diff --git a/libltdl/ltdl.c b/libltdl/ltdl.c index be1e4c0..01853e0 100644 --- a/libltdl/ltdl.c +++ b/libltdl/ltdl.c @@ -1,7 +1,7 @@ /* ltdl.c -- system independent dlopen wrapper Copyright (C) 1998, 1999, 2000, 2004, 2005, 2006, - 2007, 2008 Free Software Foundation, Inc. + 2007, 2008, 2011 Free Software Foundation, Inc. Written by Thomas Tanner, 1998 NOTE: The canonical source of this file is maintained with the @@ -80,6 +80,11 @@ static const char libprefix[] = LT_LIBPREFIX; #if defined(LT_MODULE_EXT) static const char shlib_ext[] = LT_MODULE_EXT; #endif +/* If the loadable module suffix is not the same as the linkable + * shared library suffix, this will be defined. */ +#if defined(LT_SHARED_EXT) +static const char shared_ext[] = LT_SHARED_EXT; +#endif #if defined(LT_DLSEARCH_PATH) static const char sys_dlsearch_path[] = LT_DLSEARCH_PATH; #endif @@ -1537,6 +1542,9 @@ has_library_ext (const char *filename) #if defined(LT_MODULE_EXT) || (streq (ext, shlib_ext)) #endif +#if defined(LT_SHARED_EXT) + || (streq (ext, shared_ext)) +#endif )) { return 1; @@ -1682,6 +1690,17 @@ lt_dlopenadvise (const char *filename, lt_dladvise advise) if (handle || ((errors 0) !file_not_found ())) return handle; #endif + +#if defined(LT_SHARED_EXT) + /* Try appending SHARED_EXT. */ + LT__SETERRORSTR (saved_error); + errors = try_dlopen (handle, filename, shared_ext, advise); + + /* As before, if the file was found but loading failed, return now + with the current error message. */ + if (handle || ((errors 0) !file_not_found ())) + return handle; +#endif } /* Still here? Then we really did fail to locate any of the file diff --git a/libltdl/m4/ltdl.m4 b/libltdl/m4/ltdl.m4 index 42e07e9..ea76f4d 100644 --- a/libltdl/m4/ltdl.m4 +++ b/libltdl/m4/ltdl.m4 @@ -1,6 +1,6 @@ # ltdl.m4 - Configure ltdl for the target system. -*-Autoconf-*- # -# Copyright (C) 1999-2006, 2007, 2008 Free Software Foundation, Inc. +# Copyright (C) 1999-2006, 2007, 2008, 2011 Free Software Foundation, Inc. # Written by Thomas Tanner, 1999 # # This file is free software; the Free Software Foundation gives @@ -553,12 +553,19 @@ AC_CACHE_CHECK([which extension is used for runtime loadable modules], [ module=yes eval libltdl_cv_shlibext=$shrext_cmds +module=no +eval libltdl_cv_shrext=$shrext_cmds ]) if test -n $libltdl_cv_shlibext; then m4_pattern_allow([LT_MODULE_EXT])dnl AC_DEFINE_UNQUOTED([LT_MODULE_EXT], [$libltdl_cv_shlibext], [Define to the extension used for runtime loadable modules, say, .so.]) fi +if test $libltdl_cv_shrext != $libltdl_cv_shlibext; then + m4_pattern_allow([LT_SHARED_EXT])dnl + AC_DEFINE_UNQUOTED([LT_SHARED_EXT], [$libltdl_cv_shrext], +[Define to the shared library suffix, say, .dylib.]) +fi ])# LT_SYS_MODULE_EXT # Old name: diff --git a/tests/darwin.at b/tests/darwin.at index 4e5034e..b255abe 100644 --- a/tests/darwin.at +++
Re: Mac OS X .dylib not working
On 4 Mar 2011, at 03:59, Peter O'Gorman wrote: Mac OS X does not care about file name extensions; .dylib is just a convention for native dynamic libraries. The static linker when it sees a -l flag will look for files beginning with lib and ending in .dylib, .so (though this is recent and perhaps not documented) and .a. So, I think you could say that it cares about file name extensions. One is free to implement programs that do such things. So the wanted behavior is to first try opening a library without adding an extension, and then try out different endings. On Mac OS X, it would be best trying out .dylib first - I haven't seen any other ending in use. It does seem as though guile should sometimes be trying lt_dlopen() first - then libltdl will attempt to open whatever it's given. That might be an hack. -- the libtool archive extension .la -- the extension used for native dynamically loadable modules on the host platform, e.g., .so, .sl, etc. On Mac OS X, libtool's idea of the native dynamically loadable module extension is .so, and it's not going to change. However, I can see the point that libltdl should try .dylib as well as .so for lt_dlopenext on Mac OS X. I will come up with a patch for that. The important thing is to try .dylib - all libraries I have sen use it. It can of course try .so as well. Hans
Re: Mac OS X .dylib not working
On 4 Mar 2011, at 04:00, Bob Friesenhahn wrote: So guile-2.0.0 using libltdl.7.dylib of libtool-2.4, can on Mac OS X 10.6.6 only open a dynamic library if the name of what it actually opens ends in .so (say by making a soft link using 'ln -s'); if it ends in .dylib, it cannot open it, even if the full name is given. See http://lists.gnu.org/archive/html/bug-guile/2011-03/msg8.html http://lists.gnu.org/archive/html/guile-devel/2011-03/msg00021.html Are you sure that libltdl from libtool-2.4 is being used? I used that - I made a new clean Mac OS X 10.6.6 installation, everything recompiled. Hans
Re: Mac OS X .dylib not working
On 03/04/2011 03:44 AM, Hans Aberg wrote: On 4 Mar 2011, at 03:59, Peter O'Gorman wrote: The important thing is to try .dylib - all libraries I have sen use it. It can of course try .so as well. Patch. Fails new testcase before (well, new testcase as in copy pasted lt_dlopenext testcase and slightly modified it), passes after. .dylib is tried after .so so there is no potentially surprising change in behaviour for existing applications. Oh, I forgot a NEWS entry, will add: On Mac OS X .dylib is now tried as well as .so with lt_dlopenext there before pushing. Ok? Peter From 9321b9aa351a1532ca9511d2aa1e1e174b9ffbd6 Mon Sep 17 00:00:00 2001 From: Peter O'Gorman pe...@pogma.com Date: Fri, 4 Mar 2011 12:00:23 -0600 Subject: [PATCH] On Mac OS X try .dylib as well as .so with lt_dlopenext * libltdl/m4/ltdl.m4: Define extra extension if module extension differs from shared lib extension. * libltdl/ltdl.c: Use it. * tests/darwin.at: Test it. Reported by Hans Aberg, Michael Ellis, and others. --- ChangeLog |9 ++ libltdl/ltdl.c | 19 + libltdl/m4/ltdl.m4 |7 ++ tests/darwin.at| 221 4 files changed, 256 insertions(+), 0 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7f74eab..d8fa215 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,12 @@ +2011-03-04 Peter O'Gorman pe...@pogma.com + + On Mac OS X try .dylib as well as .so with lt_dlopenext + * libltdl/m4/ltdl.m4: Define extra extension if module extension + differs from shared lib extension. + * libltdl/ltdl.c: Use it. + * tests/darwin.at: Test it. + Reported by Hans Aberg, Michael Ellis, and others. + 2011-02-12 Peter O'Gorman pe...@pogma.com Install ltmain.sh without execute bit set. diff --git a/libltdl/ltdl.c b/libltdl/ltdl.c index be1e4c0..942ed91 100644 --- a/libltdl/ltdl.c +++ b/libltdl/ltdl.c @@ -80,6 +80,11 @@ static const char libprefix[] = LT_LIBPREFIX; #if defined(LT_MODULE_EXT) static const char shlib_ext[] = LT_MODULE_EXT; #endif +/* If the loadable module suffix is not the same as the linkable + * shared library suffix, this will be defined. */ +#if defined(LT_SHARED_EXT) +static const char shared_ext[] = LT_SHARED_EXT; +#endif #if defined(LT_DLSEARCH_PATH) static const char sys_dlsearch_path[] = LT_DLSEARCH_PATH; #endif @@ -1537,6 +1542,9 @@ has_library_ext (const char *filename) #if defined(LT_MODULE_EXT) || (streq (ext, shlib_ext)) #endif +#if defined(LT_SHARED_EXT) + || (streq (ext, shared_ext)) +#endif )) { return 1; @@ -1682,6 +1690,17 @@ lt_dlopenadvise (const char *filename, lt_dladvise advise) if (handle || ((errors 0) !file_not_found ())) return handle; #endif + +#if defined(LT_SHARED_EXT) + /* Try appending SHARED_EXT. */ + LT__SETERRORSTR (saved_error); + errors = try_dlopen (handle, filename, shared_ext, advise); + + /* As before, if the file was found but loading failed, return now + with the current error message. */ + if (handle || ((errors 0) !file_not_found ())) + return handle; +#endif } /* Still here? Then we really did fail to locate any of the file diff --git a/libltdl/m4/ltdl.m4 b/libltdl/m4/ltdl.m4 index 42e07e9..c256e08 100644 --- a/libltdl/m4/ltdl.m4 +++ b/libltdl/m4/ltdl.m4 @@ -553,12 +553,19 @@ AC_CACHE_CHECK([which extension is used for runtime loadable modules], [ module=yes eval libltdl_cv_shlibext=$shrext_cmds +module=no +eval libltdl_cv_shrext=$shrext_cmds ]) if test -n $libltdl_cv_shlibext; then m4_pattern_allow([LT_MODULE_EXT])dnl AC_DEFINE_UNQUOTED([LT_MODULE_EXT], [$libltdl_cv_shlibext], [Define to the extension used for runtime loadable modules, say, .so.]) fi +if test $libltdl_cv_shrext != $libltdl_cv_shlibext; then + m4_pattern_allow([LT_SHARED_EXT])dnl + AC_DEFINE_UNQUOTED([LT_SHARED_EXT], [$libltdl_cv_shrext], +[Define to the shared library suffix, say, .dylib.]) +fi ])# LT_SYS_MODULE_EXT # Old name: diff --git a/tests/darwin.at b/tests/darwin.at index 4e5034e..f022f50 100644 --- a/tests/darwin.at +++ b/tests/darwin.at @@ -228,3 +228,224 @@ mv stdout expout LT_AT_CONFIGURE([LDFLAGS=-L/there/is/no/dir/here]) AT_CHECK([./libtool --config],[ignore],[expout],[ignore]) AT_CLEANUP + +AT_SETUP(darwin can lt_dlopen .dylib and .so files) + +AT_KEYWORDS([libltdl dylib]) + +# This test requires shared library support. +AT_CHECK([$LIBTOOL --features | grep 'enable shared libraries' || exit 77], + [], [ignore]) + + +eval `$LIBTOOL --config | $EGREP '^(shlibpath_var|shrext_cmds)='` + +module=no +eval shared_ext=\$shrext_cmds\ +module=yes +eval module_ext=\$shrext_cmds\ + +# Only bother with this test if module extension is different from +# shared extension +AT_CHECK([test $shared_ext != $module_ext || exit 77], + [], [ignore]) + +# Skip this test when called from: +#make distcheck DISTCHECK_CONFIGURE_FLAGS=--disable-ltdl-install +AT_CHECK([case $LIBLTDL in #( +
Re: Mac OS X .dylib not working
[ dropping bug-libtool ] Hi Peter, * Peter O'Gorman wrote on Fri, Mar 04, 2011 at 07:07:30PM CET: Ok? A few copyright year bumps are missing. Some minor nits inline below. Thank you, Ralf Subject: [PATCH] On Mac OS X try .dylib as well as .so with lt_dlopenext * libltdl/m4/ltdl.m4: Define extra extension if module extension differs from shared lib extension. * libltdl/ltdl.c: Use it. * tests/darwin.at: Test it. Reported by Hans Aberg, Michael Ellis, and others. --- a/tests/darwin.at +++ b/tests/darwin.at @@ -228,3 +228,224 @@ mv stdout expout LT_AT_CONFIGURE([LDFLAGS=-L/there/is/no/dir/here]) AT_CHECK([./libtool --config],[ignore],[expout],[ignore]) AT_CLEANUP + +AT_SETUP(darwin can lt_dlopen .dylib and .so files) Missing m4 quotes (for style only) +AT_KEYWORDS([libltdl dylib]) + +# This test requires shared library support. +AT_CHECK([$LIBTOOL --features | grep 'enable shared libraries' || exit 77], + [], [ignore]) + + +eval `$LIBTOOL --config | $EGREP '^(shlibpath_var|shrext_cmds)='` + +module=no +eval shared_ext=\$shrext_cmds\ +module=yes +eval module_ext=\$shrext_cmds\ + +# Only bother with this test if module extension is different from +# shared extension +AT_CHECK([test $shared_ext != $module_ext || exit 77], + [], [ignore]) You can drop arguments two and three here. +# This code is copied from the Autobook: +# http://sources.redhat.com/autobook/autobook/autobook_169.html +# so if it needs changes, be sure to notify the Autobook authors +# about them. +int +main (int argc, const char *argv[]) +{ + char *errormsg = NULL; + lt_dlhandle module = NULL; + entrypoint *run = NULL; + int errors = 0; Isn't this lacking LTDL_SET_PRELOADED_SYMBOLS(); or was that needed only for static libs (which you've excluded earlier)? + if (argc != 3) +{ + fprintf (stderr, USAGE: main MODULENAME ARGUMENT\n); + exit (EXIT_FAILURE); +} + + /* Initialise libltdl. */ + errors = lt_dlinit (); + + /* Set the module search path. */ + if (!errors) +{ + const char *path = getenv (MODULE_PATH_ENV); + + if (path != NULL) +errors = lt_dlsetsearchpath (path); +} + + /* Load the module. */ + if (!errors) +module = lt_dlopenext (argv[1]); + + /* Find the entry point. */ + if (module) +{ + run = (entrypoint *) lt_dlsym (module, run); + + /* In principle, run might legitimately be NULL, so + I don't use run == NULL as an error indicator + in general. */ + errormsg = dlerrordup (errormsg); + if (errormsg != NULL) +{ + errors = lt_dlclose (module); + module = NULL; +} +} + else +errors = 1; + + /* Call the entry point function. */ + if (!errors) +{ + int result = (*run) (argv[2]); + if (result 0) +errormsg = strdup (module entry point execution failed); + else +printf (\t= %d\n, result); +} + + /* Unload the module, now that we are done with it. */ + if (!errors) +errors = lt_dlclose (module); + + if (errors) +{ + /* Diagnose the encountered error. */ + errormsg = dlerrordup (errormsg); + + if (!errormsg) +{ + fprintf (stderr, %s: dlerror() failed.\n, argv[0]); + return EXIT_FAILURE; +} +} + + /* Finished with ltdl now. */ + if (!errors) +if (lt_dlexit () != 0) + errormsg = dlerrordup (errormsg); I'm not particularly fond of this coding style, where ownership information essentially gets lots once an error occurs in any of the commands. Might be ok for a test like this, but not such a good example for users. lt_dlexit could be warranted even if some error occurred before. Anyway, I won't reject the patch for this. + if (errormsg) +{ + fprintf (stderr, %s: %s.\n, argv[0], errormsg); + free (errormsg); + exit (EXIT_FAILURE); +} + + return EXIT_SUCCESS; +} + +/* Be careful to save a copy of the error message, + since the next API call may overwrite the original. */ +static char * +dlerrordup (char *errormsg) +{ + char *error = (char *) lt_dlerror (); + if (error !errormsg) +errormsg = strdup (error); + return errormsg; +} +]]) +if test $shlibpath_var = PATH; then This looks wrong; shouldn't it be != here? Otherwise, ... + $unset shlibpath_var || shlibpath_var= +fi +rm $libdir/simple-module.la ... this has only a small chance of succeeding. +rm $libdir/libsimple-dylib.la + +for dir in inst/lib $libdir; do + LT_AT_EXEC_CHECK([./ltdl-loader], [], [stdout], [ignore], + [$dir/simple-module World]) + AT_CHECK([grep Hello, World stdout], [], [ignore]) + LT_AT_EXEC_CHECK([./ltdl-loader], [], [stdout], [ignore], + [$dir/libsimple-dylib World]) + AT_CHECK([grep Hello, World stdout], [], [ignore]) +done + +AT_CLEANUP
Re: Mac OS X .dylib not working
On 2 Feb 2010, at 17:52, Bob Friesenhahn wrote: Under OS-X (Leopard and later), the 'dtruss' program can be used to see what is really going on. While at it, I found another problem involving libltdl.7.dylib, guile-1.8.7 and lilypond 2.13.7: When upgrading guile using libtool-2.2.6b, lilypond broke, the one which is in an Application distribution: /Applications/LilyPond.app/Contents/Resources/bin/guile It has its own /Applications/LilyPond.app/Contents/Resources/lib/libltdl.7.dylib However, dtruss shows that segmentation fault is caused when calling /usr/local/lib/libltdl.7.dylib When I move this to another name, then lilypond works, but dtruss now shows that it calls /usr/lib/libltdl.7.dylib So it seems lilypond never calls its own libltdl.7.dylib. The problem here, though, is the combination of incompatible versions of libltdl.7.dylib in combination with a library search path. If they really are incompatible, there seems to be no point in having a searchpath. I think that gcc on Mac OS X makes hardcoded full paths for libraries. So libtool would then implement its own library search paths. Also, it seems having no problem open .dylib files here. Hans
Re: Mac OS X .dylib not working
On 4 Feb 2010, at 14:49, Peter O'Gorman wrote: What does otool -L /Applications/LilyPond.app/Contents/Resources/bin/guile say? Which libltdl.7.dylib does it list? See below. They differ: libltdl.7.dylib current versions 10.0.0 and 10.1.0. If you run lilypond with DYLD_PRINT_LIBRARIES=1 set in the environment does more than one copy of libltdl.7.dylib get loaded? It just tries to load the one in /usr/local/lib/, which causes segmentation fault. This sounds like a packaging bug to me though, ... Part if it, at least. There is no point searching for incompatible libraries. ...you can probably fix it with use of install_name_tool(1). I can pass it on - I'm not a LilyPond developer. As for your earlier questions about .so and .dylib - On Mac OS X 10.0 and earlier, there was no way to load MH_DYLIB type files (usually .dylib extensions) at runtime. API was introduced to allow this in 10.1, and dlopen() was added in 10.3, rewritten in 10.4 and dlclose() actually removes the image in 10.5, prior to that only files of MH_BUNDLE type could be unloaded. When libtool support for Mac OS X was added, there was no way to load .dylib files, not much software had any knowledge of Mac OS X, and quite a lot of things had hardcoded .so when loading files at runtime, so to accomodate this, .so was chosen as the extension when creating loadable modules (MH_BUNDLE) and .dylib when creating MH_DYLIB. Changing this now would cause far too many problems. Not really: 10.4 and earlier are obsolete, and 10.5 is becoming. On 10.5, just ordinary load is fine. So just add .dylib to the list of searches. So, long story short, ltld looks for .so because that is the extension used for loadable modules. Well, that is not a part of the UNIX standard, and therefore not POSIX, which is nowadays a subset. Guile wants to use its loadable modules as input to ld, this is not portable to ancient Mac OS X, nor ancient Net BSD, and possibly other systems, however it seems unlikly to be a major issue. I think that covers most of the thread, but I admit to now reading all of it. If it can't load .dylib files, which now is the normal use, that is a bit confusing. Hans # otool -L /Applications/LilyPond.app/Contents/Resources/bin/guile /Applications/LilyPond.app/Contents/Resources/bin/guile: @executable_path/../lib//libguile.17.dylib (compatibility version 21.0.0, current version 21.1.0) @executable_path/../lib//libintl.8.dylib (compatibility version 9.0.0, current version 9.0.0) /usr/lib/libiconv.2.dylib (compatibility version 5.0.0, current version 5.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.3) @executable_path/../lib//libgmp.3.dylib (compatibility version 8.0.0, current version 8.1.0) @executable_path/../lib//libltdl.7.dylib (compatibility version 10.0.0, current version 10.0.0) /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 47.1.0) # otool -L /usr/local/bin/guile /usr/local/bin/guile: /usr/local/lib/libguile.17.dylib (compatibility version 21.0.0, current version 21.1.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/local/lib/libintl.8.dylib (compatibility version 9.0.0, current version 9.2.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4) /usr/local/lib/libgmp.3.dylib (compatibility version 9.0.0, current version 9.0.0) /usr/local/lib/libltdl.7.dylib (compatibility version 10.0.0, current version 10.1.0) /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) $ lilypond empty.ly dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/lilypond dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libintl.8.dylib dyld: loaded: /usr/lib/libSystem.B.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libguile.17.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libgmp.3.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libpangoft2-1.0.0.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libpango-1.0.0.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libfreetype.6.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libfontconfig.1.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libgobject-2.0.0.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libgmodule-2.0.0.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libglib-2.0.0.dylib dyld: loaded: /usr/lib/libstdc++.6.dylib dyld: loaded: /usr/lib/libgcc_s.1.dylib dyld: loaded: /usr/lib/libmx.A.dylib dyld: loaded: /usr/lib/libiconv.2.dylib dyld: loaded: /System/Library/Frameworks/CoreFoundation.framework/ Versions/A/CoreFoundation
Re: Mac OS X .dylib not working
On Thu, Feb 04, 2010 at 01:40:07PM +0100, Hans Aberg wrote: On 2 Feb 2010, at 17:52, Bob Friesenhahn wrote: Under OS-X (Leopard and later), the 'dtruss' program can be used to see what is really going on. While at it, I found another problem involving libltdl.7.dylib, guile-1.8.7 and lilypond 2.13.7: When upgrading guile using libtool-2.2.6b, lilypond broke, the one which is in an Application distribution: /Applications/LilyPond.app/Contents/Resources/bin/guile It has its own /Applications/LilyPond.app/Contents/Resources/lib/libltdl.7.dylib However, dtruss shows that segmentation fault is caused when calling /usr/local/lib/libltdl.7.dylib When I move this to another name, then lilypond works, but dtruss now shows that it calls /usr/lib/libltdl.7.dylib Sorry, I missed most of this thread as I was travelling. What does otool -L /Applications/LilyPond.app/Contents/Resources/bin/guile say? Which libltdl.7.dylib does it list? If you run lilypond with DYLD_PRINT_LIBRARIES=1 set in the environment does more than one copy of libltdl.7.dylib get loaded? This sounds like a packaging bug to me though, you can probably fix it with use of install_name_tool(1). As for your earlier questions about .so and .dylib - On Mac OS X 10.0 and earlier, there was no way to load MH_DYLIB type files (usually .dylib extensions) at runtime. API was introduced to allow this in 10.1, and dlopen() was added in 10.3, rewritten in 10.4 and dlclose() actually removes the image in 10.5, prior to that only files of MH_BUNDLE type could be unloaded. When libtool support for Mac OS X was added, there was no way to load .dylib files, not much software had any knowledge of Mac OS X, and quite a lot of things had hardcoded .so when loading files at runtime, so to accomodate this, .so was chosen as the extension when creating loadable modules (MH_BUNDLE) and .dylib when creating MH_DYLIB. Changing this now would cause far too many problems. So, long story short, ltld looks for .so because that is the extension used for loadable modules. Guile wants to use its loadable modules as input to ld, this is not portable to ancient Mac OS X, nor ancient Net BSD, and possibly other systems, however it seems unlikly to be a major issue. I think that covers most of the thread, but I admit to now reading all of it. Peter -- Peter O'Gorman http://pogma.com
Re: Mac OS X .dylib not working
On Thu, Feb 04, 2010 at 04:21:27PM +0100, Hans Aberg wrote: When libtool support for Mac OS X was added, there was no way to load .dylib files, not much software had any knowledge of Mac OS X, and quite a lot of things had hardcoded .so when loading files at runtime, so to accomodate this, .so was chosen as the extension when creating loadable modules (MH_BUNDLE) and .dylib when creating MH_DYLIB. Changing this now would cause far too many problems. Not really: 10.4 and earlier are obsolete, and 10.5 is becoming. On 10.5, just ordinary load is fine. 10.4 and earlier are not obsolete, sorry. So, long story short, ltld looks for .so because that is the extension used for loadable modules. Well, that is not a part of the UNIX standard, and therefore not POSIX, which is nowadays a subset. Which part of POSIX standardizes library extensions? $ lilypond empty.ly dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libintl.8.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libguile.17.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libltdl.7.dylib GNU LilyPond 2.13.7 dyld: loaded: /usr/local/lib/libguile-srfi-srfi-1-v-3.3.dylib dyld: loaded: /usr/local/lib/libguile.17.dylib dyld: loaded: /usr/local/lib/libintl.8.dylib dyld: loaded: /usr/local/lib/libgmp.3.dylib dyld: loaded: /usr/local/lib/libltdl.7.dylib Segmentation fault So lilypond starts up fine, but guile's first dlopen() for libguile-srfi-srfi-1-v-3.3 causes the library in /usr/local/lib to be loaded (and its dependent libraries, including another libguile, libintl, and libltdl). Ensuring that the search path is correct would fix this problem, look at setting the LTDL_LIBRARY_PATH environment variable, perhaps? Anyway, I am convinced that this is a packaging bug. Peter -- Peter O'Gorman http://pogma.com
Re: Mac OS X .dylib not working
On 4 Feb 2010, at 16:34, Peter O'Gorman wrote: Not really: 10.4 and earlier are obsolete, and 10.5 is becoming. On 10.5, just ordinary load is fine. 10.4 and earlier are not obsolete, sorry. The only computers that can't run 10.4 are G3 and they are really to slow. At least the one I installed it on. !0.5 runs fine also on a G4 350 Ghz I tried it on. So, long story short, ltld looks for .so because that is the extension used for loadable modules. Well, that is not a part of the UNIX standard, and therefore not POSIX, which is nowadays a subset. Which part of POSIX standardizes library extensions? None. There is some mentioning about c99 and make about .c, .f and .o files, but really no requirements. You can use those for portability reasons. Or at least that is what they say one the UNIX standardization list. $ lilypond empty.ly dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libintl.8.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libguile.17.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libltdl.7.dylib GNU LilyPond 2.13.7 dyld: loaded: /usr/local/lib/libguile-srfi-srfi-1-v-3.3.dylib dyld: loaded: /usr/local/lib/libguile.17.dylib dyld: loaded: /usr/local/lib/libintl.8.dylib dyld: loaded: /usr/local/lib/libgmp.3.dylib dyld: loaded: /usr/local/lib/libltdl.7.dylib Segmentation fault So lilypond starts up fine, but guile's first dlopen() for libguile-srfi-srfi-1-v-3.3 causes the library in /usr/local/lib to be loaded (and its dependent libraries, including another libguile, libintl, and libltdl). Ensuring that the search path is correct would fix this problem, look at setting the LTDL_LIBRARY_PATH environment variable, perhaps? Does what flags does libtool use when compiling dynamic libraries? One can compile with name lookup table and direct jump table. The former, it seems would not break if version differ, if they contain the same function. Anyway, I am convinced that this is a packaging bug. They will probably have to do something to make sure their own libraries are called - that's their bug. Hans
Re: Mac OS X .dylib not working
On 4 Feb 2010, at 16:34, Peter O'Gorman wrote: $ lilypond empty.ly dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libintl.8.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libguile.17.dylib dyld: loaded: /Applications/LilyPond.app/Contents/Resources/bin/../ lib//libltdl.7.dylib GNU LilyPond 2.13.7 dyld: loaded: /usr/local/lib/libguile-srfi-srfi-1-v-3.3.dylib dyld: loaded: /usr/local/lib/libguile.17.dylib dyld: loaded: /usr/local/lib/libintl.8.dylib dyld: loaded: /usr/local/lib/libgmp.3.dylib dyld: loaded: /usr/local/lib/libltdl.7.dylib Segmentation fault So lilypond starts up fine, but guile's first dlopen() for libguile-srfi-srfi-1-v-3.3 causes the library in /usr/local/lib to be loaded (and its dependent libraries, including another libguile, libintl, and libltdl). Ensuring that the search path is correct would fix this problem, look at setting the LTDL_LIBRARY_PATH environment variable, perhaps? Sorry, I should have answered this one: the variable isn't set, and exporting it to say LTDL_LIBRARY_PATH=/Applications/LilyPond.app/Contents/Resources/lib before starting 'lilypond' does not change the behavior. Hans
Re: Mac OS X .dylib not working
Hi, Ken Raeburn raeb...@raeburn.org writes: [Is Hans on one of these lists now? His original message to bug-guile said not and asked to be cc'ed.] We might have lost him then. ;-) On Feb 2, 2010, at 13:01, Ludovic Courtès wrote: The Guile manually specifically tells that FNAME should not contain an extension. That could be unfortunate, since it means that unlike other Mac applications, a Guile application would not be able to customize its plugin names to use Foo.quuxplugin type names. Guile apps would be limited to a hardcoded set of suffixes then, right? Guile doesn’t modify FNAME, it just passes it on to ‘lt_dlopenext ()’. Surprisingly, I just noticed that Guile itself doesn’t use the ‘-module’ option of Libtool when creating its ‘libguile-srfi-srfi-1’ module (which is meant to be dlopened *or* directly linked against), although this has never caused any problems on OS X. If you search for that in [1], ‘libguile-srfi-srfi-1’ is actually created with ‘-dynamiclib’. Current versions of Mac OS X can load shared libraries (.dylib) as well as the bundle format that seems to have been the original plugin form (.so, .bundle, ...). So in practice, assuming you can dlopen and dlclose a shared library works pretty well, though I gather it might not have worked as well in earlier releases. OK. But we should also support the format(s) intended for plugin modules as well, and the naming conventions (which appear to be somewhat varied, and less consistent than on other OSes). Since libguile-srfi-srfi-1 is intended both to be dlopened and linked directly against, we’d need to link it twice, once with ‘-module’ and another one to create the shared library. I can’t imagine myself tweaking the build system in non-trivial ways to accommodate old versions of OS X, though... Thanks, Ludo’.
Re: Mac OS X .dylib not working
On 2 Feb 2010, at 17:52, Bob Friesenhahn wrote: Unless I am missing something, the question to be answered is if Guile requests opening modules using a name like module.so (assuming a particular naming scheme), module.la (using libltdl as originally intended), or bare module (using libltdl heuristics, which tries several incantations, such as looking for .la, and .so). On the Bug-Guile list, Ludovic Courtès said it was the last one , specifically the code in 'ibguile/dynl.c', which looks like: static void * sysdep_dynl_link (const char *fname, const char *subr) { lt_dlhandle handle; handle = lt_dlopenext (fname); if (NULL == handle) { SCM fn; SCM msg; fn = scm_from_locale_string (fname); msg = scm_from_locale_string (lt_dlerror ()); scm_misc_error (subr, file: ~S, message: ~S, scm_list_2 (fn, msg)); } return (void *) handle; } OS-X's module loading does not care about the file extension. So from what you say, the problem may simply be that .dylib isn't in the list. Hans
Re: Mac OS X .dylib not working
Hi, Hans Aberg hab...@math.su.se writes: On 2 Feb 2010, at 17:52, Bob Friesenhahn wrote: Unless I am missing something, the question to be answered is if Guile requests opening modules using a name like module.so (assuming a particular naming scheme), module.la (using libltdl as originally intended), or bare module (using libltdl heuristics, which tries several incantations, such as looking for .la, and .so). [...] handle = lt_dlopenext (fname); The Guile manually specifically tells that FNAME should not contain an extension. Hans: in de114180-13f6-498c-9e48-9407e5cfb...@math.su.se [0], you said you compiled the module with “gcc -dynamiclib -lguile -o libguile-bessel.so bessel.c”. Is it the right incantation for loadable modules (not shared libraries) on Darwin? Surprisingly, I just noticed that Guile itself doesn’t use the ‘-module’ option of Libtool when creating its ‘libguile-srfi-srfi-1’ module (which is meant to be dlopened *or* directly linked against), although this has never caused any problems on OS X. If you search for that in [1], ‘libguile-srfi-srfi-1’ is actually created with ‘-dynamiclib’. (Either way, you’d probably be better off using Libtool to create the module.) Thanks, Ludo’. [0] http://thread.gmane.org/gmane.lisp.guile.bugs/4476 [1] http://hydra.nixos.org/build/275625/log/raw
Re: Mac OS X .dylib not working
On Tue, 2 Feb 2010, Hans Aberg wrote: On 2 Feb 2010, at 15:20, Ken Raeburn wrote: On Mac OS X (trying it on 10.5.8 PPC G4), guile-1.8.7 cannot open dynamic library files with name extensions .dylib, but only if they are renamed using .so instead. On the Bug-Guile list they say it just calls libltdl, in the libtool package. I have installed latest of both, but the problem persists: libtool should produce modules named *.so on Darwin if you pass the -module flag at link time. Typically, -avoid-version is used for modules as well. But dlopen() on Mac OS X can only open files in the native format, which isn't ELF, and they are typically named with the .dylib file name extension. If it finds a .so file on ELF format, all it does is reporting it cannot be opened. .so doesn't mean ELF format, and on some systems including Mac OS X, dynamically linked shared library (e.g., a .dylib file) is not the same as dynamically loadable object. Did you not see my earlier email to you and the bug-guile list? The fact is that currently Guile, which relies on libtool, cannot open .dylib files, though it works perfectly if they are renamed .so. As for what filenames to use, dlopen() does not care - it is something imposed by libtool. Also, all new native DLLs (see below) on Mac OS X are named .dylib. Unless I am missing something, the question to be answered is if Guile requests opening modules using a name like module.so (assuming a particular naming scheme), module.la (using libltdl as originally intended), or bare module (using libltdl heuristics, which tries several incantations, such as looking for .la, and .so). OS-X's module loading does not care about the file extension. Under OS-X (Leopard and later), the 'dtruss' program can be used to see what is really going on. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: Mac OS X .dylib
Hello, Hans Aberg hab...@math.su.se writes: It seems guile-1.8.7 does not admit dynamic library file name extensions .dylib, but only .so, on Mac OS X (tried 10.5.8. PPC G4), despite the manual saying guile should adapt to local standards. The example in the manual sec. 4.2.1 works fine with gcc -dynamiclib -lguile -o libguile-bessel.so bessel.c but not if changed to libguile-bessel.dylib, giving the error within guile: standard input:2:1: file: libguile-bessel, message: file not found The abstraction over file name extensions is handled by Libtool’s libltdl, used in ‘libguile/dynl.c’. Which version of Libtool/ltdl are you using? Did you try adding the directory where the ‘.dylib’ is to $DYLD_LIBRARY_PATH? Thanks, Ludo.
Re: Mac OS X .dylib
On Jan 30, 2010, at 09:41, Hans Aberg wrote: [I'm not on this list, so please cc me.] It seems guile-1.8.7 does not admit dynamic library file name extensions .dylib, but only .so, on Mac OS X (tried 10.5.8. PPC G4), despite the manual saying guile should adapt to local standards. The example in the manual sec. 4.2.1 works fine with gcc -dynamiclib -lguile -o libguile-bessel.so bessel.c but not if changed to libguile-bessel.dylib, giving the error within guile: standard input:2:1: file: libguile-bessel, message: file not found The Mac OS X situation is a bit more complicated than on normal ELF- based UNIX systems; shared libraries and dynamically loadable objects are not the same thing. It's easy to assume they're equivalent when working mostly on ELF or Windows systems where .so or .dll files work for both, but it's not always true. GNU libtool even has options for creating loadable modules as distinct from regular shared libraries. See for example http://www.finkproject.org/doc/porting/porting.en.html#shared.lib-and-mod for one brief discussion of shared libraries versus bundles on the Mac, and how the .so suffix is often used for loadable objects by convention in ported UNIX software (not just in fink, but in the OS: / usr/lib/pam/*.so, /usr/lib/sasl2/*.so), though I see a lot of .bundle names on my system too. Further confusing the matter: * The term bundle seems to be used in Apple documentation both for a particular type of file generated by the linker for loadable objects -- the linker has different flags for them vs shared libraries -- and a directory structure used for distributing a collection of files as an application, framework, or plugin. * The modern dlopen implementation on the Mac (it wasn't there at all in 10.0) seems capable of loading both bundles (the file version I think) and regular Mac shared libraries. In 10.4, dlclose() couldn't unload a dynamic library but could unload a bundle; in 10.5, it can unload both. So it's unclear how relevant the distinction between loadable objects and shared libraries is any more, at least if you're targeting 10.5 or 10.6 as your minimum required OS version. One distinction I haven't (yet) seen mentioned as having gone away is that bundles can reference symbols from the main application, but dynamic libraries cannot. See also http://lists.apple.com/archives/darwin-dev/2008/Dec/msg00045.html ... http://developer.apple.com/Mac/library/documentation/DeveloperTools/Conceptual/MachORuntime/Reference/reference.html describes the Mach-O file types, and says .bundle is the conventional suffix for plugins. So, in short, one could argue that looking only for .so for dynamically loadable objects *is* conforming to (one set of) standards for the Mac, annoying though it may be. On Mac OS X, it should probably look for .dylib first, and perhaps .so for backwards compatibility. I think perhaps it should try without a suffix (in case the application code specifies it), then both of these and maybe also .bundle, though I don't have a strong sense which order they should be tried in. Ken
Re: Mac OS X .dylib
Hello Hans, On Sat 30 Jan 2010 15:41, Hans Aberg hab...@math.su.se writes: It seems guile-1.8.7 does not admit dynamic library file name extensions .dylib, but only .so, on Mac OS X (tried 10.5.8. PPC G4) I think your example shold work, but it's something that's totally handled by libltdl. Are you using the latest libltdl? Note also that libltdl's error messages often aren't the most appropriate, due to another bug: http://thread.gmane.org/gmane.comp.gnu.libtool.general/10621 This (often) makes all failures come out as file not found. So try using a newer libltdl, and if that fails, bug-libt...@gnu.org is probably the place to go. A -- http://wingolog.org/
Re: Mac OS X .dylib
On 30 Jan 2010, at 19:37, Ludovic Courtès wrote: The abstraction over file name extensions is handled by Libtool’s libltdl, used in ‘libguile/dynl.c’. Which version of Libtool/ltdl are you using? I took down the latest stable before trying to compile guile: libtool --version ltmain.sh (GNU libtool) 2.2 Did you try adding the directory where the ‘.dylib’ is to $DYLD_LIBRARY_PATH? No, all was installed with standard './configure make make install', without tweaks. If you want me try something, please let me know. Hans
Re: Mac OS X .dylib
On 30 Jan 2010, at 19:39, Ken Raeburn wrote: The Mac OS X situation is a bit more complicated than on normal ELF-based UNIX systems; shared libraries and dynamically loadable objects are not the same thing. It's easy to assume they're equivalent when working mostly on ELF or Windows systems where .so or .dll files work for both, but it's not always true. GNU libtool even has options for creating loadable modules as distinct from regular shared libraries. Sure, it would be best to only load .dylib, but I suspect that there might be .dylib libraries with the .so extension on Mac OS X, due to past practice. Then such programs will suddenly break. One program using Guile is LilyPond, and I have a vague memory of they mentioning the .so problem on Mac OS X, but I do not recall details. If .dylib fails and it tries .so which isn't on the .dylib format, then the function should give an error. So it is harmless. Hans
Re: Mac OS X .dylib
On 30 Jan 2010, at 20:30, Andy Wingo wrote: It seems guile-1.8.7 does not admit dynamic library file name extensions .dylib, but only .so, on Mac OS X (tried 10.5.8. PPC G4) I think your example shold work, but it's something that's totally handled by libltdl. Are you using the latest libltdl? I have now update to latest libtool --version ltmain.sh (GNU libtool) 2.2.6b and the README says that libltdl should be a part of it; however, I just made an install of the libtools package. I also re-unpacked guile-1.8.7, and './configure make make install'. There is no change: .dylib fails; .so works. Note also that libltdl's error messages often aren't the most appropriate, due to another bug: http://thread.gmane.org/gmane.comp.gnu.libtool.general/10621 This (often) makes all failures come out as file not found. Since it works if I change to .so, it seems to only be that file name problem. So try using a newer libltdl, and if that fails, bug-libt...@gnu.org is probably the place to go. Ah - list chasing. :-) Hans