ld scripts and libltdl
I'm using a program that uses libltdl (from libtool) to wrap calls to dlopen (e.g. lt_dlopenext) and dlsym and it is utterly unable to dlopen "standard" shared libraries. Gentoo (probably following upstream glibc) has been putting linker scripts where shared libraries were orignially expected. ``` $ cat /usr/lib64/libm.so /* GNU ld script */ OUTPUT_FORMAT(elf64-x86-64) GROUP ( /lib64/libm.so.6 AS_NEEDED ( /lib64/libmvec.so.1 ) ) ``` So a call to lt_dlopenext("libm"), with ltdl compiled with -DLT_DEBUG_LOADERS, produces output like this (but lt_dlerror only reports "File not found") ``` try_dlopen (libm, .la) tryall_dlopen (libm.a, lt_preopen) Calling lt_preopen->module_open (libm.a) Result: Failed Error message << file not found >> try_dlopen (libm, .so) tryall_dlopen (libm.a, lt_preopen) Calling lt_preopen->module_open (libm.a) Result: Failed Error message << file not found >> tryall_dlopen (/usr/lib64/libm.so, (ALL)) Calling lt_dlopen->module_open (/usr/lib64/libm.so) Result: Failed Error message << /usr/lib64/libm.so: invalid ELF header >> Calling lt_preopen->module_open (/usr/lib64/libm.so) Result: Failed Error message << file not found >> ``` Only a call to lt_dlopen with the full path "/lib64/libm.so.6" would work. But the library I'm using only exposes lt_dlopenext. Perhaps ltdl should parse the linker scripts and dlopen the real dll? --- Regards, Madhu
[PATCH 2/2] libltdl: unify get_vtable boilerplate
These files all define get_vtable the same way, so create a single header file to hold it. This would fix desync problems like we had with some defining the return value as const, and it'll make it a bit easier to workaround GNU indent issues related to the defines. * Makefile.am (pkgltdl_files): Add lt_vtable.h. * libltdl/ltdl.mk (libltdl_libltdl_la_SOURCES): Likewise. * libtoolize.in (all_pkgltdl_files): Likewise. * libltdl/loaders/dld_link.c: Replace get_vtable prototype with lt_vtable.h include. * libltdl/loaders/dlopen.c, libltdl/loaders/dyld.c, libltdl/loaders/load_add_on.c, libltdl/loaders/loadlibrary.c, libltdl/loaders/preopen.c, libltdl/loaders/shl_load.c, libltdl/ltdl.c: Likewise. * libltdl/lt_vtable.h: New file. --- Makefile.am | 1 + libltdl/loaders/dld_link.c| 5 +--- libltdl/loaders/dlopen.c | 5 +--- libltdl/loaders/dyld.c| 5 +--- libltdl/loaders/load_add_on.c | 5 +--- libltdl/loaders/loadlibrary.c | 5 +--- libltdl/loaders/preopen.c | 5 +--- libltdl/loaders/shl_load.c| 5 +--- libltdl/lt_vtable.h | 44 +++ libltdl/ltdl.c| 4 +--- libltdl/ltdl.mk | 1 + libtoolize.in | 2 +- 12 files changed, 55 insertions(+), 32 deletions(-) create mode 100644 libltdl/lt_vtable.h diff --git a/Makefile.am b/Makefile.am index 904f3d73fc89..34384c5acf26 100644 --- a/Makefile.am +++ b/Makefile.am @@ -479,6 +479,7 @@ pkgltdl_files = COPYING.LIB \ lt__strl.c \ lt_dlloader.c \ lt_error.c \ + lt_vtable.h \ ltdl.c \ ltdl.h \ ltdl.mk \ diff --git a/libltdl/loaders/dld_link.c b/libltdl/loaders/dld_link.c index 9dc220d96c07..7e7541c7338d 100644 --- a/libltdl/loaders/dld_link.c +++ b/libltdl/loaders/dld_link.c @@ -37,10 +37,7 @@ or obtained by writing to the Free Software Foundation, Inc., Use the "_LTX_" prefix so that the symbol addresses can be fetched from the preloaded symbol list by lt_dlsym(): */ #define get_vtable dld_link_LTX_get_vtable - -LT_BEGIN_C_DECLS -LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); -LT_END_C_DECLS +#include "lt_vtable.h" /* Boilerplate code to set up the vtable for hooking this loader into diff --git a/libltdl/loaders/dlopen.c b/libltdl/loaders/dlopen.c index 5ec46b7f126b..c8c03bf42e25 100644 --- a/libltdl/loaders/dlopen.c +++ b/libltdl/loaders/dlopen.c @@ -37,10 +37,7 @@ or obtained by writing to the Free Software Foundation, Inc., Use the "_LTX_" prefix so that the symbol addresses can be fetched from the preloaded symbol list by lt_dlsym(): */ #define get_vtable dlopen_LTX_get_vtable - -LT_BEGIN_C_DECLS -LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); -LT_END_C_DECLS +#include "lt_vtable.h" /* Boilerplate code to set up the vtable for hooking this loader into diff --git a/libltdl/loaders/dyld.c b/libltdl/loaders/dyld.c index 2c62c6fb0642..bfd35b24c590 100644 --- a/libltdl/loaders/dyld.c +++ b/libltdl/loaders/dyld.c @@ -37,10 +37,7 @@ or obtained by writing to the Free Software Foundation, Inc., Use the "_LTX_" prefix so that the symbol addresses can be fetched from the preloaded symbol list by lt_dlsym(): */ #define get_vtable dyld_LTX_get_vtable - -LT_BEGIN_C_DECLS -LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); -LT_END_C_DECLS +#include "lt_vtable.h" /* Boilerplate code to set up the vtable for hooking this loader into diff --git a/libltdl/loaders/load_add_on.c b/libltdl/loaders/load_add_on.c index 513daa4c147a..6b07cd521e22 100644 --- a/libltdl/loaders/load_add_on.c +++ b/libltdl/loaders/load_add_on.c @@ -37,10 +37,7 @@ or obtained by writing to the Free Software Foundation, Inc., Use the "_LTX_" prefix so that the symbol addresses can be fetched from the preloaded symbol list by lt_dlsym(): */ #define get_vtable load_add_on_LTX_get_vtable - -LT_BEGIN_C_DECLS -LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); -LT_END_C_DECLS +#include "lt_vtable.h" /* Boilerplate code to set up the vtable for hooking this loader into diff --git a/libltdl/loaders/loadlibrary.c b/libltdl/loaders/loadlibrary.c index 10e44490548a..d8995eb5f2bc 100644 --- a/libltdl/loaders/loadlibrary.c +++ b/libltdl/loaders/loadlibrary.c @@ -41,10 +41,7 @@ or obtained by writing to the Free Software Foundation, Inc., Use the "_LTX_" prefix so that the symbol addresses can be fetched from the preloaded symbol list by lt_dlsym(): */ #define get_vtable loadlibrary_LTX_get_vtable - -LT_BEGIN_C_DECLS -LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); -LT_END_C_DECLS +#include "lt_vtable.h" /* Boilerplate code to set up the vtable for hooking this loader into diff -
[PATCH 1/2] libltdl: constify get_vtable in loaders
The lt_dlloader.h API already says these have const return values, and the main ltdl.c module defines its get_vtable in that way, so update all of the loaders to match. * libltdl/loaders/dld_link.c (get_vtable): Add const. * libltdl/loaders/dlopen.c, libltdl/loaders/dyld.c, libltdl/loaders/load_add_on.c, libltdl/loaders/loadlibrary.c, libltdl/loaders/preopen.c, libltdl/loaders/shl_load.c: Likewise. --- libltdl/loaders/dld_link.c| 4 ++-- libltdl/loaders/dlopen.c | 4 ++-- libltdl/loaders/dyld.c| 4 ++-- libltdl/loaders/load_add_on.c | 4 ++-- libltdl/loaders/loadlibrary.c | 4 ++-- libltdl/loaders/preopen.c | 4 ++-- libltdl/loaders/shl_load.c| 4 ++-- 7 files changed, 14 insertions(+), 14 deletions(-) diff --git a/libltdl/loaders/dld_link.c b/libltdl/loaders/dld_link.c index 3a9f345e53ae..9dc220d96c07 100644 --- a/libltdl/loaders/dld_link.c +++ b/libltdl/loaders/dld_link.c @@ -39,7 +39,7 @@ or obtained by writing to the Free Software Foundation, Inc., #define get_vtable dld_link_LTX_get_vtable LT_BEGIN_C_DECLS -LT_SCOPE lt_dlvtable *get_vtable (lt_user_data loader_data); +LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); LT_END_C_DECLS @@ -57,7 +57,7 @@ static lt_dlvtable *vtable = 0; /* Return the vtable for this loader, only the name and sym_prefix attributes (plus the virtual function implementations, obviously) change between loaders. */ -lt_dlvtable * +const lt_dlvtable * get_vtable (lt_user_data loader_data) { if (!vtable) diff --git a/libltdl/loaders/dlopen.c b/libltdl/loaders/dlopen.c index 66a129c9a472..5ec46b7f126b 100644 --- a/libltdl/loaders/dlopen.c +++ b/libltdl/loaders/dlopen.c @@ -39,7 +39,7 @@ or obtained by writing to the Free Software Foundation, Inc., #define get_vtable dlopen_LTX_get_vtable LT_BEGIN_C_DECLS -LT_SCOPE lt_dlvtable *get_vtable (lt_user_data loader_data); +LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); LT_END_C_DECLS @@ -57,7 +57,7 @@ static lt_dlvtable *vtable = 0; /* Return the vtable for this loader, only the name and sym_prefix attributes (plus the virtual function implementations, obviously) change between loaders. */ -lt_dlvtable * +const lt_dlvtable * get_vtable (lt_user_data loader_data) { if (!vtable) diff --git a/libltdl/loaders/dyld.c b/libltdl/loaders/dyld.c index 3889b6aa57a0..2c62c6fb0642 100644 --- a/libltdl/loaders/dyld.c +++ b/libltdl/loaders/dyld.c @@ -39,7 +39,7 @@ or obtained by writing to the Free Software Foundation, Inc., #define get_vtable dyld_LTX_get_vtable LT_BEGIN_C_DECLS -LT_SCOPE lt_dlvtable *get_vtable (lt_user_data loader_data); +LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); LT_END_C_DECLS @@ -58,7 +58,7 @@ static lt_dlvtable *vtable = 0; /* Return the vtable for this loader, only the name and sym_prefix attributes (plus the virtual function implementations, obviously) change between loaders. */ -lt_dlvtable * +const lt_dlvtable * get_vtable (lt_user_data loader_data) { if (!vtable) diff --git a/libltdl/loaders/load_add_on.c b/libltdl/loaders/load_add_on.c index 97b4fcb689c0..513daa4c147a 100644 --- a/libltdl/loaders/load_add_on.c +++ b/libltdl/loaders/load_add_on.c @@ -39,7 +39,7 @@ or obtained by writing to the Free Software Foundation, Inc., #define get_vtable load_add_on_LTX_get_vtable LT_BEGIN_C_DECLS -LT_SCOPE lt_dlvtable *get_vtable (lt_user_data loader_data); +LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); LT_END_C_DECLS @@ -57,7 +57,7 @@ static lt_dlvtable *vtable = 0; /* Return the vtable for this loader, only the name and sym_prefix attributes (plus the virtual function implementations, obviously) change between loaders. */ -lt_dlvtable * +const lt_dlvtable * get_vtable (lt_user_data loader_data) { if (!vtable) diff --git a/libltdl/loaders/loadlibrary.c b/libltdl/loaders/loadlibrary.c index 13fd2fb9c72c..10e44490548a 100644 --- a/libltdl/loaders/loadlibrary.c +++ b/libltdl/loaders/loadlibrary.c @@ -43,7 +43,7 @@ or obtained by writing to the Free Software Foundation, Inc., #define get_vtable loadlibrary_LTX_get_vtable LT_BEGIN_C_DECLS -LT_SCOPE lt_dlvtable *get_vtable (lt_user_data loader_data); +LT_SCOPE const lt_dlvtable *get_vtable (lt_user_data loader_data); LT_END_C_DECLS @@ -62,7 +62,7 @@ static lt_dlvtable *vtable = 0; /* Return the vtable for this loader, only the name and sym_prefix attributes (plus the virtual function implementations, obviously) change between loaders. */ -lt_dlvtable * +const lt_dlvtable * get_vtable (lt_user_data loader_data) { if (!vtable) diff --git a/libltdl/loaders/preopen.c b/libltdl/loaders/preopen.c index f1ed0aa3e306..e5440d3e07ac 100644 --- a/libltdl/loaders/preopen.c +++ b/libltdl/loaders/preopen.c @@ -39,7 +39,7 @@ or obtained by writing to the Free Software Foundation, Inc., #define get_vtable preopen_LTX_get_vtable LT_BEGIN_C_DECLS
branch master updated: libltdl: delete use of ^L
This is an automated email from the git hooks/post-receive script. vapier pushed a commit to branch master in repository libtool. The following commit(s) were added to refs/heads/master by this push: new 1879a0e6 libltdl: delete use of ^L 1879a0e6 is described below commit 1879a0e6db8e19d0a980a4740606b4f102503623 Author: Mike Frysinger AuthorDate: Sun Jan 14 21:48:38 2024 -0500 libltdl: delete use of ^L GNU indent does not process this well, and nothing else in the tree uses this convention, so drop it. (GNU indent is enforced by gnulib when running `make distcheck`.) * libltdl/ltdl.c: Delete ^L. * libltdl/ltdl.h: Likewise. --- libltdl/ltdl.c | 1 - libltdl/ltdl.h | 2 -- 2 files changed, 3 deletions(-) diff --git a/libltdl/ltdl.c b/libltdl/ltdl.c index 2cbe9a9c..db4905de 100644 --- a/libltdl/ltdl.c +++ b/libltdl/ltdl.c @@ -91,7 +91,6 @@ staticconst char sys_dlsearch_path[] = LT_DLSEARCH_PATH; - /* --- DYNAMIC MODULE LOADING --- */ diff --git a/libltdl/ltdl.h b/libltdl/ltdl.h index 67c9af9a..ec0fd70c 100644 --- a/libltdl/ltdl.h +++ b/libltdl/ltdl.h @@ -43,7 +43,6 @@ LT_BEGIN_C_DECLS /* LT_STRLEN can be used safely on NULL pointers. */ #define LT_STRLEN(s) (((s) && (s)[0]) ? strlen (s) : 0) - /* --- DYNAMIC MODULE LOADING API --- */ @@ -110,7 +109,6 @@ extern LT_DLSYM_CONST lt_dlsymlist lt__PROGRAM__LTX_preloaded_symbols[]; - /* --- MODULE INFORMATION --- */
[PATCH] libltdl: delete use of ^L
GNU indent does not process this well, and nothing else in the tree uses this convention, so drop it. (GNU indent is enforced by gnulib when running `make distcheck`.) * libltdl/ltdl.c: Delete ^L. * libltdl/ltdl.h: Likewise. --- libltdl/ltdl.c | 1 - libltdl/ltdl.h | 2 -- 2 files changed, 3 deletions(-) diff --git a/libltdl/ltdl.c b/libltdl/ltdl.c index 2cbe9a9cb9e6..db4905de1563 100644 --- a/libltdl/ltdl.c +++ b/libltdl/ltdl.c @@ -91,7 +91,6 @@ staticconst char sys_dlsearch_path[] = LT_DLSEARCH_PATH; - /* --- DYNAMIC MODULE LOADING --- */ diff --git a/libltdl/ltdl.h b/libltdl/ltdl.h index 67c9af9a277a..ec0fd70c9d0b 100644 --- a/libltdl/ltdl.h +++ b/libltdl/ltdl.h @@ -43,7 +43,6 @@ LT_BEGIN_C_DECLS /* LT_STRLEN can be used safely on NULL pointers. */ #define LT_STRLEN(s) (((s) && (s)[0]) ? strlen (s) : 0) - /* --- DYNAMIC MODULE LOADING API --- */ @@ -110,7 +109,6 @@ extern LT_DLSYM_CONST lt_dlsymlist lt__PROGRAM__LTX_preloaded_symbols[]; - /* --- MODULE INFORMATION --- */ -- 2.43.0
[PATCH] libltdl: fix dyld loader build failure
Fix for undefined symbol strneq. The original working code in 97ade81 used !strcmp() to check for string equality before it was incorrectly replaced with strneq(). This loader is built on OS X before 10.4. --- libltdl/loaders/dyld.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libltdl/loaders/dyld.c b/libltdl/loaders/dyld.c index bfd5869..d666128 100644 --- a/libltdl/loaders/dyld.c +++ b/libltdl/loaders/dyld.c @@ -406,7 +406,7 @@ lt__nsmodule_get_header (NSModule module) while (i > 0) { --i; - if (strneq (_dyld_get_image_name (i), modname)) + if (STREQ (_dyld_get_image_name (i), modname)) { mh = _dyld_get_image_header (i); break; @@ -455,7 +455,7 @@ lt__match_loadedlib (const char *name) --i; id = lt__header_get_instnam (_dyld_get_image_header (i)); - if (id && strneq (id, name)) + if (id && STREQ (id, name)) { mh = _dyld_get_image_header (i); break; -- 2.9.0
Re: [RFC] Fix libltdl preloader issues with MSVC
Hi Gary, On 2012-10-20 07:26, Gary V. Vaughan wrote: Hi Peter, On Oct 20, 2012, at 4:06 AM, Peter Rosin p...@lysator.liu.se wrote: With the latest dumpbin work in place, libtool starts to add symbols from import libraries to the generated fooS.c files in func_generate_dlsyms, as needed by the preloader. And then the next problem naturally rears its head. This time it is the age-old data exports problem responsible for code such as this (in demo.at): else if (STREQ (nothing, name)) #ifndef _WIN32 /* In an ideal world we could do this... */ pnothing = (int*)s-address; #else /* !_WIN32 */ /* In an ideal world a shared lib would be able to export data */ pnothing = (int*)nothing; #endif I.e., demo.at isn't even trying to use the data symbol from the preloader, it just picks data up directly using the import library on Windows. Which is of course cheating and circumvents the API etc etc. However, MinGW has long supported pseudo-relocs and anto-import and the Ideal world case should work just fine with gcc even on _WIN32. But not with MSVC, which fails to link since data symbols from import libraries are not declared __declspec(dllimport). That's because demo stems from the original test code that Gord wrote over 15 years ago alongside the embryonic preloading implementation, all before the current API was available! :) Having said that, it's kinda nice to have this test of the low-level structure of preloaded library data, although calling it demo might seem like we're still encouraging folk to use preloaded libraries this way :( If you feel inclined to move the existing low-level tests aside and rename them to something less inviting, and then rewrite the demo tests to actually use the API that would be great for our test coverage too! Yes, I was kind of surprised that evidently the most involved preloader test was demo. I thought that libltdl basically worked with MSVC, as long as you took care to apply the correct #ifdef madness. But that proved very wrong when you moved demo/ to demo.at and hardened it ever so slightly... Example from demo.at: helldl.exeS.obj : error LNK2001: unresolved external symbol _nothing Fixing that, and making data symbols __declspec(dllimport) is however not the entire solution, as that yields: helldl.exeS.c(47) : error C2099: initializer is not a constant during compilation instead. The underlying problem is that MSVC does not have pseudo-relocs. More hoops to jump through at the Microsoft circus. Hurray! Ain't it fun? :-/ So, this patch adds an @INIT@ entry to the preloader symbol table, with the address of a function that is intended to do the relocations of data items from user-provided code instead of relying on the tools. This function is then filled with relocation code for MSVC, but left empty everywhere else where it is not needed. This is clearly not a finalized patch (it's actually pretty rough), it's just proof of concept. I have e.g. not optimized it, but it is clearly possibly to set the @INIT@ address to NULL and don't call @INIT@ at all in non-MSVC cases. It should also not be necessary to relocate data symbols from static libraries from user code, and it should be possible to add a done variable to the relocation function, so that it only does the work once. What I'm looking for is feedback. Is it acceptable to add a new virtual @INIT@ entry like this? I.e. is this approach workable or should I just drop it? I have no objections, except perhaps that it would be good to check that the ABI doesn't stop pre-patch libltdl from preloading libraries built with post-patch libtool. But that's likely overly pedantic of me, since I don't think anyone is likely to preload libraries built from a different package (potentially using different libtool versions)... but if it's easy to be compatible, we might as well do that. Building the actual library has not changed one bit, only the preloading glue, so as long as the new glue is used I don't expect any trouble. I didn't even stop to think that it could be a problem... I can't think of any other use for @INIT@ than beating MSVC into submission, but as long as you can do the final implementation cleanly enough that it doesn't force all the well behaved platforms to carry around dead weight for MSVC (eg not adding @INIT@ to the symbol list when it's not going to be used), and that we can keep everything nicely separated so that people who don't need to understand it to contribute in other areas can easily skip past... then, sure, go right ahead :) Good, I'll post an updated patch for review when I have applied some polish. But not this week, as I'm traveling... Cheers, Peter
[RFC] Fix libltdl preloader issues with MSVC
() { return lt__PROGRAM__LTX_preloaded_symbols; } #endif #ifdef __cplusplus } #endif --8 Cheers, Peter From 3e840b158f7a9949457273ec070f35ab03b04ade Mon Sep 17 00:00:00 2001 From: Peter Rosin p...@lysator.liu.se Date: Fri, 19 Oct 2012 22:49:56 +0200 Subject: [PATCH] libtool: add @INIT@ to the preloader, for data imports on Windows * build-aux/ltmain.in (func_generate_dlsyms) [MSVC]: Mark all data symbols __declspec(dllimport) and handle static libraries and import libraries equivalently. Generate a relocation function lt_syminit that fills in the addresses of data items and point it out with a new virtual @INIT@ entry in the symbol list. * m4/libtool.m4 (_LT_CMD_GLOBAL_SYMBOLS) [MSVC]: Adjust lt_cv_sys_global_symbol_to_c_name_address and lt_cv_sys_global_symbol_to_c_name_address_lib_prefix so that they fill in (void*) 0 for data symbols which need to be relocated at runtime by the above lt_syminit. * libltdl/loaders/preopen.c (add_symlist): Look up the virtual @INIT@ symbol (i.e. lt_syminit) and call it if non-zero. (vm_sym, lt_dlpreload_open): Step past the @INIT@ symbol. * tests/demo.at (dlmain.c): Call the @INIT@ symbol if non-zero. --- build-aux/ltmain.in | 26 +++--- libltdl/loaders/preopen.c | 22 -- m4/libtool.m4 |2 ++ tests/demo.at |2 ++ 4 files changed, 47 insertions(+), 5 deletions(-) diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in index ed4a4b1..a153df0 100644 --- a/build-aux/ltmain.in +++ b/build-aux/ltmain.in @@ -2659,6 +2659,11 @@ extern \C\ { #else # define LT_DLSYM_CONST const #endif +#if defined _WIN32 defined _MSC_VER +#define LT_DLSYM_DATA extern __declspec(dllimport) +#else +#define LT_DLSYM_DATA extern +#endif /* External symbol declarations for the compiler. */\ @@ -2793,7 +2798,7 @@ extern \C\ { fi if test -f $nlistS; then - eval $global_symbol_to_cdecl' $nlistS $output_objdir/$my_dlsyms' + eval $global_symbol_to_cdecl' $nlistS' | $SED 's/^extern char /LT_DLSYM_DATA char /' $output_objdir/$my_dlsyms else echo '/* NONE */' $output_objdir/$my_dlsyms fi @@ -2807,10 +2812,25 @@ typedef struct { } lt_dlsymlist; extern LT_DLSYM_CONST lt_dlsymlist lt_${my_prefix}_LTX_preloaded_symbols[]; +static void lt_syminit(void) +{ +#if defined _WIN32 defined _MSC_VER + LT_DLSYM_CONST lt_dlsymlist *symbol = lt_${my_prefix}_LTX_preloaded_symbols; + for (; symbol-name; ++symbol) + {\ + + if test -f $nlistS; then + eval $global_symbol_to_cdecl' $nlistS' | $SED -n 's/extern char \([^;]*\).*/if (!strcmp (symbol-name, \\1\)) symbol-address = (void *) \\1;/p' $output_objdir/$my_dlsyms + fi + + echo $output_objdir/$my_dlsyms \ + } +#endif +} LT_DLSYM_CONST lt_dlsymlist lt_${my_prefix}_LTX_preloaded_symbols[] = -{\ - { \$my_originator\, (void *) 0 }, +{ {\$my_originator\, (void *) 0}, + {\@INIT@\, (void *) lt_syminit}, case $need_lib_prefix in no) diff --git a/libltdl/loaders/preopen.c b/libltdl/loaders/preopen.c index 1670085..432f552 100644 --- a/libltdl/loaders/preopen.c +++ b/libltdl/loaders/preopen.c @@ -210,7 +210,7 @@ vm_sym (lt_user_data LT__UNUSED loader_data, lt_module module, const char *name) { lt_dlsymlist *symbol = (lt_dlsymlist*) module; - symbol +=2; /* Skip header (originator then libname). */ + symbol +=3; /* Skip header (originator, init then libname). */ while (symbol-name) { @@ -270,9 +270,25 @@ add_symlist (const lt_dlsymlist *symlist) if (tmp) { + const lt_dlsymlist *symbol; + tmp-symlist = symlist; tmp-next = preloaded_symlists; preloaded_symlists = tmp; + + for (symbol = symlist; symbol-name; ++symbol) + { + if (STREQ (symbol-name, @INIT@)) + { + if (symbol-address) + { + void (*init_symlist)(void); + *(void **)(init_symlist) = symbol-address; + init_symlist(); + } + break; + } + } } else { @@ -345,7 +361,9 @@ lt_dlpreload_open (const char *originator, lt_dlpreload_callback_func *func) ++found; /* ...load the symbols per source compilation unit: - (we preincrement the index to skip over the originator entry) */ + (we increment the index once to skip over the originator entry + and then preincrement to also skip the init entry) */ + ++idx; while ((symbol = list-symlist[++idx])-name != 0) { if ((symbol-address == 0) diff --git a/m4/libtool.m4 b/m4/libtool.m4 index c85a85f..a7c3f4f 100644 --- a/m4/libtool.m4 +++ b/m4/libtool.m4 @@ -3674,6 +3674,8 @@ for ac_symprfx in _; do # Write the raw and C identifiers. if test $lt_cv_nm_interface = MS dumpbin; then +lt_cv_sys_global_symbol_to_c_name_address=sed -n -e 's/^: \([[^ ]]*\)[[ ]]*$/ {1\\\, (void *) 0},/p' -e 's/^T \([[^ ]]*\) \([[^ ]]*\)$/ {\\2\, (void *) \\2},/p' -e 's/^D \([[^ ]]*\) \([[^ ]]*\)$/ {\\2\, (void *) 0},/p
Re: [RFC] Fix libltdl preloader issues with MSVC
Hi Peter, On Oct 20, 2012, at 4:06 AM, Peter Rosin p...@lysator.liu.se wrote: With the latest dumpbin work in place, libtool starts to add symbols from import libraries to the generated fooS.c files in func_generate_dlsyms, as needed by the preloader. And then the next problem naturally rears its head. This time it is the age-old data exports problem responsible for code such as this (in demo.at): else if (STREQ (nothing, name)) #ifndef _WIN32 /* In an ideal world we could do this... */ pnothing = (int*)s-address; #else /* !_WIN32 */ /* In an ideal world a shared lib would be able to export data */ pnothing = (int*)nothing; #endif I.e., demo.at isn't even trying to use the data symbol from the preloader, it just picks data up directly using the import library on Windows. Which is of course cheating and circumvents the API etc etc. However, MinGW has long supported pseudo-relocs and anto-import and the Ideal world case should work just fine with gcc even on _WIN32. But not with MSVC, which fails to link since data symbols from import libraries are not declared __declspec(dllimport). That's because demo stems from the original test code that Gord wrote over 15 years ago alongside the embryonic preloading implementation, all before the current API was available! :) Having said that, it's kinda nice to have this test of the low-level structure of preloaded library data, although calling it demo might seem like we're still encouraging folk to use preloaded libraries this way :( If you feel inclined to move the existing low-level tests aside and rename them to something less inviting, and then rewrite the demo tests to actually use the API that would be great for our test coverage too! Example from demo.at: helldl.exeS.obj : error LNK2001: unresolved external symbol _nothing Fixing that, and making data symbols __declspec(dllimport) is however not the entire solution, as that yields: helldl.exeS.c(47) : error C2099: initializer is not a constant during compilation instead. The underlying problem is that MSVC does not have pseudo-relocs. More hoops to jump through at the Microsoft circus. Hurray! So, this patch adds an @INIT@ entry to the preloader symbol table, with the address of a function that is intended to do the relocations of data items from user-provided code instead of relying on the tools. This function is then filled with relocation code for MSVC, but left empty everywhere else where it is not needed. This is clearly not a finalized patch (it's actually pretty rough), it's just proof of concept. I have e.g. not optimized it, but it is clearly possibly to set the @INIT@ address to NULL and don't call @INIT@ at all in non-MSVC cases. It should also not be necessary to relocate data symbols from static libraries from user code, and it should be possible to add a done variable to the relocation function, so that it only does the work once. What I'm looking for is feedback. Is it acceptable to add a new virtual @INIT@ entry like this? I.e. is this approach workable or should I just drop it? I have no objections, except perhaps that it would be good to check that the ABI doesn't stop pre-patch libltdl from preloading libraries built with post-patch libtool. But that's likely overly pedantic of me, since I don't think anyone is likely to preload libraries built from a different package (potentially using different libtool versions)... but if it's easy to be compatible, we might as well do that. I can't think of any other use for @INIT@ than beating MSVC into submission, but as long as you can do the final implementation cleanly enough that it doesn't force all the well behaved platforms to carry around dead weight for MSVC (eg not adding @INIT@ to the symbol list when it's not going to be used), and that we can keep everything nicely separated so that people who don't need to understand it to contribute in other areas can easily skip past... then, sure, go right ahead :) Cheers, -- Gary V. Vaughan (gary AT gnu DOT org)
Re: [PATCH 00/12] Improve AIX support: libltdl, filename-based versioning.
On 10/02/2012 05:34 PM, Michael Haubenwallner wrote: here's a set of patches to fix the two major problems with current AIX support in libtool, that is: *) libltdl: lt_dlopen() does work /with/ runtime linking (LDFLAGS=-brtl) only. *) libtool: /Without/ runtime linking, there is some versioning support, but still far from being ideal for package managers, as they have to deal with flags (F_LOADONLY) of Shared Objects, and merge them inside archive files, maybe conflicting with existing archives with the same name in /usr/lib/. But /with/ runtime linking, there is no versioning support in libtool at all, while - even worse - the symlinks do suggest the opposite. Anyone about (to consider) to look at these patches? Thank you! /haubi/
[PATCH 00/12] Improve AIX support: libltdl, filename-based versioning.
Hi, here's a set of patches to fix the two major problems with current AIX support in libtool, that is: *) libltdl: lt_dlopen() does work /with/ runtime linking (LDFLAGS=-brtl) only. *) libtool: /Without/ runtime linking, there is some versioning support, but still far from being ideal for package managers, as they have to deal with flags (F_LOADONLY) of Shared Objects, and merge them inside archive files, maybe conflicting with existing archives with the same name in /usr/lib/. But /with/ runtime linking, there is no versioning support in libtool at all, while - even worse - the symlinks do suggest the opposite. With Import Files, it is possible to have filename-based versioning support on AIX, using those symlinks known from Linux. However, Import Files prevent static linking, so these should be used with the .so filename extension only. But this requires to enable runtime linking via the -brtl (or -G) linker flag. While there is a bunch of variants available to use Import Files, it turns out to be easiest to put both a Shared Object named shr.o and an Import File named shr.imp into an archive file named libN.so.X, having the F_LOADONLY flag set for shr.o. This allows to have the symlink libN.so referring to libN.so.X work as expected (as known from Linux). I do call this the aix-soname right now, but more obvious names are welcome. Unfortunately, this is incompatible with the current implementation /with/ runtime linking, as current lib.so is a standalone Shared Object rather than an archive file. Replacing that lib.so by (a symlink to) an archive file will break existing binaries built against (or inside) an existing site which has enabled the current implementation of runtime linking. So I'd like to introduce the configure switch --enable-aix-soname, to explicitly allow package /managers/ to enable the aix-soname for their site. Additionally, package /maintainers/ can enable aix-soname with LT_INIT, usually along some version bump, while still allowing package /managers/ to --disable-aix-soname. The patches can be split into a few chunks with different goals. For each chunk there is a bootstrapped snapshot available. Also, there is a script to run the tests with different setups, to be used as: $ ./runtests libtool-2.4.2.1*.tar.gz http://dev.gentoo.org/~haubi/libtool/aix-soname-r1/runtests [00]: Current master. http://dev.gentoo.org/~haubi/libtool/aix-soname-r1/libtool-2.4.2.177-d674ea.tar.gz [01..04]: Support lt_dlopen(libN.la) with shared archive members, to fix the current situation /without/ runtime linking. Fixes these tests /without/ LDFLAGS=-brtl: 85: libdir of installed modules ok, was: FAILED (ltdl-libdir.at) 110: C++ exception handling ok, was: FAILED (exceptions.at) No changes in test results /with/ LDFLAGS=-brtl. http://dev.gentoo.org/~haubi/libtool/aix-soname-r1/libtool-2.4.2.181-caaf.tar.gz [05..06]: Fix -no-undefined flag for current LDFLAGS=-brtl implementation. There aren't any tests that expose a problem, but the -no-undefined flag doesn't work as expected. And the symlinks really are useless - the versioning test is skipped for a reason. These patches aren't a real prerequisite, but the linkline fixed here serves as template for the linkline used in aix-soname later. http://dev.gentoo.org/~haubi/libtool/aix-soname-r1/libtool-2.4.2.183-62c0.tar.gz [07..10]: Introduce optional filename-based shared library versioning, also called the aix-soname. Tweak tests to respect aix-soname for expected results. Using --enable-aix-soname also does set LDFLAGS=-brtl, so the test results need to be compared against those /with/ runtime linking: 72: versioning ok, was: skipped (versioning.at) 115: build tree relpaths expected failure, was: ok (deplibs-ident.at) This one is broken /without/ runtime linking already, but (kind of) works /with/. However, it also breaks with aix-soname enabled - see patch [11]: 108: template test with subdirs FAILED, was: ok (template.at) http://dev.gentoo.org/~haubi/libtool/aix-soname-r1/libtool-2.4.2.187-5fb50.tar.gz [11]: The template test expects undefined symbols in libB to trigger libA being kept when the executable not needing libA itself is linked against both. This test does succeed /with/ LDFLAGS=-brtl currently: Not because of the symbols, but standalone Shared Objects being referenced /unconditionally/. http://dev.gentoo.org/~haubi/libtool/aix-soname-r1/libtool-2.4.2.188-1d353.tar.gz [12]: Not meant to be pushed (right now), but to have the traditional TESTS also run with aix-soname enabled. I've been unable to find out how to pass --enable-aix-soname to subsequent configure runs in make check-TESTS. http://dev.gentoo.org/~haubi/libtool/aix-soname-r1/libtool-2.4.2.189-06d1.tar.gz While there's lots of inline commenting that should explain the ideas behind, please tell if/where/how more documentation is needed. Thank you for consideration! /haubi/
conditionally disabling libltdl subconfigure
Hello, I am working at embedding libltdl inside our project (hwloc, an Open MPI subprojet). Our plugins are not expected to be enabled in the most common cases. So I am trying to see if the libltdl sub-configure could be skipped entirely unless really needed (when plugins are enabled, and libltdl is indeed built). I got the usual failure when I tried to put LTDL_INIT inside a conditional block so I am looking for something else now. Modifying the subdirs variable before AC_OUTPUT seems to work. But this hack may not be safe enough. Is there a better way to do this? Thanks Brice ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: conditionally disabling libltdl subconfigure
On Wed, 22 Aug 2012, Brice Goglin wrote: Hello, I am working at embedding libltdl inside our project (hwloc, an Open MPI subprojet). Our plugins are not expected to be enabled in the most common cases. So I am trying to see if the libltdl sub-configure could be skipped entirely unless really needed (when plugins are enabled, and libltdl is indeed built). I got the usual failure when I tried to put LTDL_INIT inside a conditional block so I am looking for something else now. While it does not accomplish exactly what you are looking for, the GraphicsMagick project (as an example) successfully uses the nonrecursive option so that the configure bits needed to configure libltdl are included into the primary configure script. The libltdl configuration then vastly less expensive since it just adds a few more tests and not a whole configure script and whole new Makefile.in. This approach causes the combined configure script to include some libltld-specific configuration options. It is good that these options are offered by the top-level configure since they will then appear in configure --help and the user is not left to guess about options which might be useful for subordinate configures. This is in configure.ac: LTDL_INIT([convenience nonrecursive]) and this in Makefile.am include libltdl/Makefile.inc where Makefile.inc is a makefile fragment from libltdl. The project can then choose not to request to build libltdl even though it was configured. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: conditionally disabling libltdl subconfigure
On 2012-08-22 15:12 +0200, Brice Goglin wrote: I am working at embedding libltdl inside our project (hwloc, an Open MPI subprojet). Our plugins are not expected to be enabled in the most common cases. So I am trying to see if the libltdl sub-configure could be skipped entirely unless really needed (when plugins are enabled, and libltdl is indeed built). I got the usual failure when I tried to put LTDL_INIT inside a conditional block so I am looking for something else now. Modifying the subdirs variable before AC_OUTPUT seems to work. But this hack may not be safe enough. Is there a better way to do this? Use either the recursive or nonrecursive libltdl styles. Do not use subproject (the default). Both these styles integrate all the configure machinery directly into your main configure script, so there may still be some extra checks but they should be fairly low-cost in the grand scheme of things (especially compared to subproject!). You can then use Automake conditionals to control whether or not libltdl will be built. Cheers, -- Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/) ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: conditionally disabling libltdl subconfigure
Le 22/08/2012 16:32, Nick Bowler a écrit : On 2012-08-22 15:12 +0200, Brice Goglin wrote: I am working at embedding libltdl inside our project (hwloc, an Open MPI subprojet). Our plugins are not expected to be enabled in the most common cases. So I am trying to see if the libltdl sub-configure could be skipped entirely unless really needed (when plugins are enabled, and libltdl is indeed built). I got the usual failure when I tried to put LTDL_INIT inside a conditional block so I am looking for something else now. Modifying the subdirs variable before AC_OUTPUT seems to work. But this hack may not be safe enough. Is there a better way to do this? Use either the recursive or nonrecursive libltdl styles. Do not use subproject (the default). Both these styles integrate all the configure machinery directly into your main configure script, so there may still be some extra checks but they should be fairly low-cost in the grand scheme of things (especially compared to subproject!). You can then use Automake conditionals to control whether or not libltdl will be built. Cheers, Thanks a lot! It looks promising, indeed. One small problem with this is that it requires -I$(top_builddir) in CPPFLAGS since it wants to include LT_CONFIG_H=relative/path/config.h Not sure how this could be improved. Maybe by passing the absolute path in LT_CONFIG_H? Or by allowing to manually override it? Brice ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: conditionally disabling libltdl subconfigure
On 2012-08-22 20:26 +0200, Brice Goglin wrote: Le 22/08/2012 19:44, Nick Bowler a écrit : On 2012-08-22 18:32 +0200, Brice Goglin wrote: Le 22/08/2012 16:32, Nick Bowler a écrit : [...] Other than adding -I$(top_builddir) to CPPFLAGS, as a workaround you could probably add something like this just before AC_OUTPUT (untested): LT_CONFIG_H=`expr $LT_CONFIG_H : '.*/\(.*\)'` Your analysis looks correct but the workaround doesn't seem to work. It looks like LT_CONFIG_H is set inside AC_OUTPUT. Indeed, it looks like LT_CONFIG_H is set using AC_CONFIG_COMMANDS_PRE, so it does in fact get set inside AC_OUTPUT. I guess you could write AC_CONFIG_COMMANDS_PRE([LT_CONFIG_H=`expr $LT_CONFIG_H : '.*/\(.*\)'`]) and hope that it runs after libltdl's registration. Unfortunately, the autoconf manual does not appear to specify the order in which multiple AC_CONFIG_COMMANDS_PRE registrations are run, so this might not be the most robust approach. Perhaps someone on the autoconf list can clarify. It would be prudent to report the issue to the bug-libt...@gnu.org mailing list, ideally with a small test case. Obviously if you can wait for an upstream fix and avoid any hack workarounds for old versions that'd be ideal... Cheers, -- Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/) ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: conditionally disabling libltdl subconfigure
Le 22/08/2012 21:05, Nick Bowler a écrit : [...] Indeed, it looks like LT_CONFIG_H is set using AC_CONFIG_COMMANDS_PRE, so it does in fact get set inside AC_OUTPUT. I guess you could write AC_CONFIG_COMMANDS_PRE([LT_CONFIG_H=`expr $LT_CONFIG_H : '.*/\(.*\)'`]) and hope that it runs after libltdl's registration. Unfortunately, the autoconf manual does not appear to specify the order in which multiple AC_CONFIG_COMMANDS_PRE registrations are run, so this might not be the most robust approach. Perhaps someone on the autoconf list can clarify. It would be prudent to report the issue to the bug-libt...@gnu.org mailing list, ideally with a small test case. Obviously if you can wait for an upstream fix and avoid any hack workarounds for old versions that'd be ideal... Thanks a lot. This workaround works. I just reported the issue to bug-libtool with a small testcase. We likely have a couple months before the next major hwloc release starts using libltdl, so I hope we'll have a better fix by then. Regards Brice ___ https://lists.gnu.org/mailman/listinfo/libtool
libltdl leaks when unloading plugins depending on external lib
Hello, I am fixing leaks in one of my software projects that embeds libltdl 2.4.2 to load plugins. I noticed that when the plugin depends on the dynamic library that the main program doesn't depend on, valgrind reports some leaks. We created a small testcase reproducing the problem. It's available at http://www.open-mpi.org/~jsquyres/ltdl-valgrind-leaks/ Just download it, ./configure, make, make install and then run valgrind as shown below. If we remove the reference to pthread in the plugin, the leaks go away. I assume that libltdl doesn't unload these dependency libraries when the plugin is unloaded. But I'd like to know if these leaks are expected or not. The good news is that these leaks do not depend on the number of times this code is executed within the program, so the leaked size should be constant. Regards Brice Goglin $ valgrind --leak-check=full --show-reachable=yes ./runme ==1446== Memcheck, a memory error detector ==1446== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==1446== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==1446== Command: ./runme ==1446== ==1446== ==1446== HEAP SUMMARY: ==1446== in use at exit: 1,748 bytes in 6 blocks ==1446== total heap usage: 52 allocs, 46 frees, 7,393 bytes allocated ==1446== ==1446== 32 bytes in 1 blocks are still reachable in loss record 1 of 6 ==1446==at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==1446==by 0x4013197: _dl_close_worker (dl-close.c:374) ==1446==by 0x401371D: _dl_close (dl-close.c:754) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4E2D2EB: _dlerror_run (dlerror.c:164) ==1446==by 0x4E2D00E: dlclose (dlclose.c:48) ==1446==by 0x4C297BB: ??? (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0) ==1446==by 0x4C276EC: lt_dlclose (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0) ==1446==by 0x400C6F: main (runme.c:31) ==1446== ==1446== 38 bytes in 1 blocks are still reachable in loss record 2 of 6 ==1446==at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==1446==by 0x4004E21: local_strdup (dl-load.c:162) ==1446==by 0x4007929: _dl_map_object (dl-load.c:2373) ==1446==by 0x400CF61: openaux (dl-deps.c:65) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x400C02B: _dl_map_object_deps (dl-deps.c:247) ==1446==by 0x4011EB7: dl_open_worker (dl-open.c:263) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4011899: _dl_open (dl-open.c:633) ==1446==by 0x4E2CF65: dlopen_doit (dlopen.c:67) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4E2D2EB: _dlerror_run (dlerror.c:164) ==1446== ==1446== 38 bytes in 1 blocks are still reachable in loss record 3 of 6 ==1446==at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==1446==by 0x400AA85: _dl_new_object (dl-object.c:161) ==1446==by 0x4005FB5: _dl_map_object_from_fd (dl-load.c:957) ==1446==by 0x40076B7: _dl_map_object (dl-load.c:2468) ==1446==by 0x400CF61: openaux (dl-deps.c:65) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x400C02B: _dl_map_object_deps (dl-deps.c:247) ==1446==by 0x4011EB7: dl_open_worker (dl-open.c:263) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4011899: _dl_open (dl-open.c:633) ==1446==by 0x4E2CF65: dlopen_doit (dlopen.c:67) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446== ==1446== 56 bytes in 1 blocks are still reachable in loss record 4 of 6 ==1446==at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==1446==by 0x400C0DC: _dl_map_object_deps (dl-deps.c:470) ==1446==by 0x4011EB7: dl_open_worker (dl-open.c:263) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4011899: _dl_open (dl-open.c:633) ==1446==by 0x4E2CF65: dlopen_doit (dlopen.c:67) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4E2D2EB: _dlerror_run (dlerror.c:164) ==1446==by 0x4E2CEE0: dlopen@@GLIBC_2.2.5 (dlopen.c:88) ==1446==by 0x4C29806: ??? (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0) ==1446==by 0x4C26919: ??? (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0) ==1446==by 0x4C26B97: ??? (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0) ==1446== ==1446== 408 bytes in 1 blocks are still reachable in loss record 5 of 6 ==1446==at 0x4027034: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==1446==by 0x400F4D3: _dl_check_map_versions (dl-version.c:299) ==1446==by 0x401238E: dl_open_worker (dl-open.c:269) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4011899: _dl_open (dl-open.c:633) ==1446==by 0x4E2CF65: dlopen_doit (dlopen.c:67) ==1446==by 0x400D925: _dl_catch_error (dl-error.c:178) ==1446==by 0x4E2D2EB: _dlerror_run (dlerror.c:164) ==1446
Re: libltdl leaks when unloading plugins depending on external lib
On Tue, 3 Apr 2012, Brice Goglin wrote: I am fixing leaks in one of my software projects that embeds libltdl 2.4.2 to load plugins. I noticed that when the plugin depends on the dynamic library that the main program doesn't depend on, valgrind reports some leaks. If the main program is directly linked with -lpthread do some leaks go away (or more are added)? We created a small testcase reproducing the problem. It's available at http://www.open-mpi.org/~jsquyres/ltdl-valgrind-leaks/ Just download it, ./configure, make, make install and then run valgrind as shown below. If we remove the reference to pthread in the plugin, the leaks go away. I assume that libltdl doesn't unload these dependency libraries when the plugin is unloaded. But I'd like to know if these leaks are expected or not. The good news is that these leaks do not depend on the number of times this code is executed within the program, so the leaked size should be constant. These reported leaks seem to be due to the glibc implementation and don't look like libltdl's fault. It is interesting to me that these reported leaks are different than I have seen before. Usually I see leaks reported for thread-specific errno and thread specific data. It is common for glibc to secretly allocate some memory for its own use, and this is particularly true for multi-threaded programs. Often it has no way to safely release this memory. Once you are sure that the reported issues are due to glibc (or otherwise outside of your control), you can add valgrind surpressions so that valgrind will ignore them (see valgrind documentation). Valgrind comes with its own set of surpressions but perhaps the issues you are seeing are not covered by those. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: libltdl leaks when unloading plugins depending on external lib
Le 03/04/2012 23:13, Bob Friesenhahn a écrit : On Tue, 3 Apr 2012, Brice Goglin wrote: I am fixing leaks in one of my software projects that embeds libltdl 2.4.2 to load plugins. I noticed that when the plugin depends on the dynamic library that the main program doesn't depend on, valgrind reports some leaks. If the main program is directly linked with -lpthread do some leaks go away (or more are added)? Everything goes away, but a new one appears. Anyway, see below. We created a small testcase reproducing the problem. It's available at http://www.open-mpi.org/~jsquyres/ltdl-valgrind-leaks/ Just download it, ./configure, make, make install and then run valgrind as shown below. If we remove the reference to pthread in the plugin, the leaks go away. I assume that libltdl doesn't unload these dependency libraries when the plugin is unloaded. But I'd like to know if these leaks are expected or not. The good news is that these leaks do not depend on the number of times this code is executed within the program, so the leaked size should be constant. These reported leaks seem to be due to the glibc implementation and don't look like libltdl's fault. It is interesting to me that these reported leaks are different than I have seen before. Usually I see leaks reported for thread-specific errno and thread specific data. It is common for glibc to secretly allocate some memory for its own use, and this is particularly true for multi-threaded programs. Often it has no way to safely release this memory. I tried with another dependency (libpci instead of libpthread), the leaks go away, so indeed the problem is likely outside of libltdl. Thanks and sorry for the noise! Brice ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: libltdl leaks when unloading plugins depending on external lib
On Tue, 3 Apr 2012, Brice Goglin wrote: It is common for glibc to secretly allocate some memory for its own use, and this is particularly true for multi-threaded programs. Often it has no way to safely release this memory. I tried with another dependency (libpci instead of libpthread), the leaks go away, so indeed the problem is likely outside of libltdl. The important thing to test for is that if you load and unload this module 1000 times, do you see 1000X as much leak or the same amount as when loading/unloading one time? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: libltdl leaks when unloading plugins depending on external lib
Le 04/04/2012 00:04, Bob Friesenhahn a écrit : On Tue, 3 Apr 2012, Brice Goglin wrote: It is common for glibc to secretly allocate some memory for its own use, and this is particularly true for multi-threaded programs. Often it has no way to safely release this memory. I tried with another dependency (libpci instead of libpthread), the leaks go away, so indeed the problem is likely outside of libltdl. The important thing to test for is that if you load and unload this module 1000 times, do you see 1000X as much leak or the same amount as when loading/unloading one time? As I said in my first mail, the amount of leaks doesn't seem to depend on the number of loads/unloads. So it's not annoying, except the valgrind warning, which can be suppressed anyway. thanks Brice ___ https://lists.gnu.org/mailman/listinfo/libtool
[PATCH 2/6] libtoolize: rename `libltdl/m4' directory to standard `m4'.
Sorry I wasn't able to break this up into smaller pieces without leaving the tree with failing tests between. In addition to moving the autoconf macro directory into the place expected by GNU developers ($top_srcdir/m4), we also simplify libtoolize somewhat by having libltdl use whatever directory was specified by the parent project. This is much more flexible, allows libltdl client projects to also use a single canonical `$top_srcdir/m4' macro directory, and maintains backward compatibility with existing projects that wish to continue using $ltdl_dir/m4. * libltdl/m4: Moved to it's parent directory. * Makefile.am (ACLOCAL_AMFLAGS, BUILDCHECK_ENVIRONMENT) (libltdl/Makefile.am, libltdl/aclocal.m4): Adjust. * configure.ac (AC_CONFIG_MACRO_DIR): Adjust. * libltdl/configure.ac (AC_CONFIG_MACRO_DIR): Remove. The libtoolize script can pick up the project macro directory from ACLOCAL_AMFLAGS in libltdl/Makefile.am, saving us from needing to regenerate configure for the sake of a different macro directory declared in the libltdl using client project. * libtoolize.m4sh (func_install_pkgmacro_subproject): Remove. (func_install_pkgmacro_parent): Remove. (subproject_macro_dir): Remove. (require_libltdl_Makefile_am, require_ltdl_aclocal_amflags) (require_ltdl_am_macro_dir): New functions, generated at runtime by adding libltdl/Makefile.am to the generation loop. (func_install_pkgmacro_files): Considerably simplified now that we have only a single macro directory to worry about. (require_ltdl_relative_macro_dir): New function. Calculate the relative path from $ltdl_dir to the parent project macro_dir. (require_Makefile_am_filter): Depending on $ltdl_mode, edit Makefile.am ACLOCAL_AMFLAGS to use the parent project macro directory if necessary. (require_aclocal_m4_filter): Make sure m4_includes point to the correct macro directory by editing the file as it is copied into the parent project. (func_install_pkgltdl_files): Use the filters. (func_check_macros): Remove various advice that no longer applies with an always shared macro directory. (func_require_ltdl_dir): Don't default ltdl_dir to subproject when `--ltdl' was not given. (func_require_ltdl_mode): Leave ltdl_mode empty when ltdl_dir is not set. (func_require_macro_dir): Default to $ltdl_dir/m4 in subproject mode with no other directory implied by AC_CONFIG_MACRO_DIR or ACLOCAL_AMFLAGS, otherwise m4 as before. * tests/libtoolize.at: Match new creating file `Makefile.am' messages correctly. (subproject ltdl with non-shared directories): This test no longer makes sense, so repurposed... (subproject ltdl with non-canonical macro dir): ...this, to ensure subproject ltdl continues to automatically share the parent macro directory. * tests/cdemo/Makefile.am, tests/demo/Makefile.am, tests/depdemo/Makefile.am, tests/f77demo/Makefile.am, tests/fcdemo/Makefile.am, tests/mdemo/Makefile.am, tests/mdemo2/Makefile.am, tests/pdemo/Makefile.am, tests/tagdemo/Makefile.am (ACLOCAL_AMFLAGS): Use new location of macro_dir. * NEWS: Updated. Signed-off-by: Gary V. Vaughan g...@gnu.org --- Makefile.am |8 +- NEWS |7 + configure.ac |2 +- libltdl/configure.ac |1 - libtoolize.m4sh | 319 - {libltdl/m4 = m4}/.gitignore |0 {libltdl/m4 = m4}/argz.m4|0 {libltdl/m4 = m4}/autobuild.m4 |0 {libltdl/m4 = m4}/libtool.m4 |0 {libltdl/m4 = m4}/ltdl.m4|0 {libltdl/m4 = m4}/ltoptions.m4 |0 {libltdl/m4 = m4}/ltsugar.m4 |0 {libltdl/m4 = m4}/ltversion.in |0 {libltdl/m4 = m4}/lt~obsolete.m4 |0 tests/cdemo/Makefile.am |2 +- tests/demo/Makefile.am|2 +- tests/depdemo/Makefile.am |2 +- tests/f77demo/Makefile.am |2 +- tests/fcdemo/Makefile.am |2 +- tests/libtoolize.at | 75 - tests/mdemo/Makefile.am |2 +- tests/mdemo2/Makefile.am |2 +- tests/pdemo/Makefile.am |2 +- tests/tagdemo/Makefile.am |2 +- 24 files changed, 229 insertions(+), 201 deletions(-) rename {libltdl/m4 = m4}/.gitignore (100%) rename {libltdl/m4 = m4}/argz.m4 (100%) rename {libltdl/m4 = m4}/autobuild.m4 (100%) rename {libltdl/m4 = m4}/libtool.m4 (100%) rename {libltdl/m4 = m4}/ltdl.m4 (100%) rename {libltdl/m4 = m4}/ltoptions.m4 (100%) rename {libltdl/m4 = m4}/ltsugar.m4 (100%) rename {libltdl/m4 = m4}/ltversion.in (100%) rename {libltdl/m4 = m4}/lt~obsolete.m4 (100%) diff --git a/Makefile.am b/Makefile.am index 56f492a..d6b8bd7 100644 --- a/Makefile.am +++ b/Makefile.am @@ -23,7 +23,7 @@ ## 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. # -ACLOCAL_AMFLAGS= -I libltdl/m4 +ACLOCAL_AMFLAGS= -I m4 AM_CPPFLAGS= AM_LDFLAGS = @@ -195,7 +195,7 @@ $(lt_Makefile_am): $(ltdl_mk
[PATCH 6/6] libtoolize: rename `libltdl/config' directory to standard `build-aux'.
In addition to moving the autoconf auxiliary script directory into the place expected by GNU developers ($top_srcdir/build-aux) we also simplify libtoolize somewhat by having libltdl use whatever directory was specified by the parent project. This is much more flexible, allows libltdl client projects to also use a single canonical `$top_srcdir/build-aux' auxiliary script directory, and maintains backward compatibility with existing projects that wish to continue using $ltdl_dir/config. * libltdl/config: Moved whole directory from here... * bulid-aux: ...to here. * libtoolize.m4sh (func_install_pkgaux_subproject): Remove. (func_install_pkgaux_parent): Remove. (func_install_pkgaux_files): Considerably simplified now that we have only a single auxiliary directory to worry about. (subproject_aux_dir): Remove. * bootstrap.conf (libtool_readme_release_package_substitutions): No need to substitute our old unusual aux_dir location into README-release any more. * cfg.mk (_build-aux): Now that we use the standard location for this directory, no need to set this variable either. * configure.ac (AC_INIT): git-version-gen has moved. (AC_CONFIG_AUX_DIR): Adjust. * Makefile.am (stamp-mk, auxexefiles, auxfiles) (install-data-local): Use build-aux instead of config. (libltdl/stamp-mk): Ditto. * libltdl/configure.ac, tests/cdemo/configure.ac, tests/demo/configure.ac, tests/depdemo/configure.ac, tests/f77demo/configure.ac, tests/fcdemo/configure.ac, tests/mdemo/configure.ac, tests/mdemo2/configure.ac, tests/pdemo/configure.ac, tests/tagdemo/configure.ac (AC_CONFIG_AUX_DIR): Ditto. * tests/defs.m4sh (aux_dir): Ditto. * tests/configure-iface.at, tests/getopt-m4sh.at, tests/libtoolize.at, tests/subproject.at, tests/testsuite.at: Ditto. * Makefile.am (ltdl_ac_aux_dir): Extract at build-time... * libtoolize.m4sh (require_ltdl_ac_aux_dir): ...so that libtoolize can access it at runtime without requiring GNU M4 via the extract-trace script. (require_Makefile_am_filter): Use it to determine whether the aux_dir declared in the current (parent) project configure is compatible with the declarations in installed ltdl autotools source files. (require_configure_ac_filter): New function to perform similar checks and rewrite configure.ac during installation if necessary. * NEWS: Updated. Signed-off-by: Gary V. Vaughan g...@gnu.org --- Makefile.am | 18 +- NEWS|8 + bootstrap |6 +- bootstrap.conf |3 +- {libltdl/config = build-aux}/.gitignore|0 {libltdl/config = build-aux}/cvsu |0 {libltdl/config = build-aux}/edit-readme-alpha |0 {libltdl/config = build-aux}/extract-trace |0 {libltdl/config = build-aux}/general.m4sh |0 {libltdl/config = build-aux}/getopt.m4sh |0 {libltdl/config = build-aux}/ltmain.m4sh |0 {libltdl/config = build-aux}/options-parser|0 cfg.mk |2 - configure.ac|4 +- libltdl/configure.ac|2 +- libtoolize.m4sh | 224 --- tests/cdemo/configure.ac|2 +- tests/demo/configure.ac |2 +- tests/depdemo/configure.ac |2 +- tests/f77demo/configure.ac |2 +- tests/fcdemo/configure.ac |2 +- tests/getopt-m4sh.at|4 +- tests/libtoolize.at | 92 +- tests/mdemo/configure.ac|2 +- tests/mdemo2/configure.ac |2 +- tests/pdemo/configure.ac|2 +- tests/tagdemo/configure.ac |2 +- tests/testsuite.at |6 +- 28 files changed, 204 insertions(+), 183 deletions(-) rename {libltdl/config = build-aux}/.gitignore (100%) rename {libltdl/config = build-aux}/cvsu (100%) rename {libltdl/config = build-aux}/edit-readme-alpha (100%) rename {libltdl/config = build-aux}/extract-trace (100%) rename {libltdl/config = build-aux}/general.m4sh (100%) rename {libltdl/config = build-aux}/getopt.m4sh (100%) rename {libltdl/config = build-aux}/ltmain.m4sh (100%) rename {libltdl/config = build-aux}/options-parser (100%) diff --git a/Makefile.am b/Makefile.am index 7d4c1e8..d686ab9 100644 --- a/Makefile.am +++ b/Makefile.am @@ -272,6 +272,8 @@ all-local: $(LTDL_BOOTSTRAP_DEPS) ## Libtool scripts. ## ## ## +ltdl_ac_aux_dir = `$(extract_trace) AC_CONFIG_AUX_DIR $(srcdir)/libltdl/configure.ac` + configure_edit = $(SED) \ -e 's,@aclocaldir\@,$(aclocaldir),g' \ -e 's,@aux_dir\@,$(aux_dir),g' \ @@ -281,6 +283,7 @@ configure_edit = $(SED) \ -e 's,@GREP\@,$(GREP),g
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
Hi Stefano, On 2 Nov 2011, at 16:29, Stefano Lattarini wrote: On Wednesday 02 November 2011, Gary V wrote: I like to make sure the tree is capable of passing `make distcheck' after every patch, And this is a very good policy, with which I fully agree. What I meant was: wouldn't be better to introduce the $(auxdest), $(macrodest) and $(ltdldest) in a preparatory patch (with no semantic change), so that the edit to `Makefile.am' done in the present patch patch can be brought down to a minimal change like auxdest = config = auxdest = build-aux? and splitting this one into move directories around and fixup the aftermath would leave the tree broken in the middle... This is not what I proposing, not at all! What I was proposing is: 1. in a first, preparatory patch, factor out the names/locations of the directories in proper variables and/or makefile macros; 2. in the real patch, move the directories around, leveraging on the previous refactoring to ensure this moving can be done with smaller and more self-contained changes. Sorry if I failed to express myself correctly; I hope I've made my point clearer this time. Yes indeed, the misunderstanding was mine. And you raise a good point. I have always tended to break a series of changes into patches in such a way that each individual patch is doing *something*... otherwise it's easy to get to a point where everything is so fine grained that you lose all sense of direction... my patch queue is already 40 patches deep, and if I had to go through those and break them into ever smaller pieces, it would slow my progress dramatically. However, I'll post all but the most trivial patches to this list at least a few days before pushing... and I would be extremely happy for you to raise a flag if you think any particular patch is too broad in scope. Cheers, -- Gary V. Vaughan (gary AT gnu DOT org)
Re: [PATCH 2/3] maint: rename `libltdl/m4' directory to standard `m4'.
Hi Gary. On Wednesday 02 November 2011, Gary V wrote: package_revision=`$SHELL $ac_aux_dir/git-version-gen .tarball-version` diff --git a/libltdl/.gitignore b/libltdl/.gitignore index 2f39096..5795dbc 100644 --- a/libltdl/.gitignore +++ b/libltdl/.gitignore @@ -2,5 +2,4 @@ /Makefile.am /argz.h /build-aux +/m4 -/dummy.c -/gnulib.mk Shouldn't these last two edits be done in a separate patch? No, because I don't want to introduce broken revisions that cannot build into git history. I think there's a misunderstanding here. What I meant is: since (as far as I can see) dummy.c and gnulib.mk are not touched/moved by this patch, shouldn't any edit to `.gitignore' that involves them better be done in a separate patch? Or am I missing something? Also, shouldn't you report the changes to file `libltdl/.gitignore' in the ChangeLog entry? That's interesting actually. Historically, we have only reported changes to distributed files in ChangeLog, and have always omitted at least VCS control files from the log entries. I'm still leaning slightly towards not introducing ChangeLog noise to describe things that are only important when you have the full git checkout available, and hence access to git log and friends if you want to dig this sort of information out -- BUT I could easily be persuaded to change my mind if you have a good argument for the advantage of putting this stuff into the git log entry, which would then eventually be put into the generated ChangeLog file... I have no strong opinion on the matter (even if I personally prefer, when writing a ChangeLog entry, to mention any non-generated, version-controlled file that gets modified, whther distributed or not). I just wanted to make sure the lack of `.gitignore' mention in the ChangeLog wasn't the result of an overlloking. Regards, Stefano
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
On Wednesday 02 November 2011, Gary V wrote: Hi Stefano, On 1 Nov 2011, at 21:57, Stefano Lattarini wrote: Hi Gary, hope you won't mind few nits from an outsider ... Absolutely not, any and all feedback is always extremely welcome. Thank you. On Tuesday 01 November 2011, Gary V wrote: # The timestamps on these files must be preserved carefully so we install, # uninstall and set executable with custom rules here. -auxfiles = $(pkgaux_scripts) config/ltmain.sh +auxdest = build-aux +auxfiles = $(pkgaux_scripts) $(auxdest)/ltmain.sh # Everything that gets picked up by aclocal is automatically distributed, # this is the list of macro files we install on the user's system. -aclocalfiles = m4/argz.m4 m4/libtool.m4 m4/ltdl.m4 m4/ltoptions.m4 \ -m4/ltsugar.m4 m4/ltversion.m4 m4/lt~obsolete.m4 +macrodest = m4 +aclocalfiles = $(macrodest)/argz.m4 \ +$(macrodest)/libtool.m4 \ +$(macrodest)/ltdl.m4 \ +$(macrodest)/ltoptions.m4 \ +$(macrodest)/ltsugar.m4 \ +$(macrodest)/ltversion.m4 \ +$(macrodest)/lt~obsolete.m4 Shouldn't this better be done in either a preparatory or follow-up patch? Ditto for other similar changes. I like to make sure the tree is capable of passing `make distcheck' after every patch, And this is a very good policy, with which I fully agree. What I meant was: wouldn't be better to introduce the $(auxdest), $(macrodest) and $(ltdldest) in a preparatory patch (with no semantic change), so that the edit to `Makefile.am' done in the present patch patch can be brought down to a minimal change like auxdest = config = auxdest = build-aux? and splitting this one into move directories around and fixup the aftermath would leave the tree broken in the middle... This is not what I proposing, not at all! What I was proposing is: 1. in a first, preparatory patch, factor out the names/locations of the directories in proper variables and/or makefile macros; 2. in the real patch, move the directories around, leveraging on the previous refactoring to ensure this moving can be done with smaller and more self-contained changes. Sorry if I failed to express myself correctly; I hope I've made my point clearer this time. Regards, Stefano
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
On Wed, 2 Nov 2011, Gary V. Vaughan wrote: Did you consider this existing use while developing the plan to rename this often-shared directory? Yes I did, and it doesn't seem at all onerous to me if I have gone to all the trouble of upgrading libtool and re-bootstrapping my project with it to also amend one line in configure.ac. As it stands, libtoolize will issue an upgrade recommendation, and if you ignore that, nothing will actually break, you will just get duplicate copies of libltdl's build-aux files (compile, config.guess, config.sub, depcomp, install-sh, ltmain.sh and missing) alongside your parent project's files in ltdl/config. Gary, Due to libtool's spotted past, many projects check the 'libtoolized' files (including libltdl) into their project's source control system, including rather crippled systems like CVS. Libtool files then become much more carefully managed than autoconf or automake generated files because libtool was historically more fragile. A change to the libltdl footprint then requires adding/removing/renaming directories. Likewise, it has become common for various OS distributions to patch package's bundled libtool files (which include non-libtool files like config.guess) as part of the procedure to build a package for that OS distribution. Moving the files increases effort and churn since the patches need to be re-generated and re-distributed. If an OS port is completely re-autotooling the packages (as some OS distributions mandate), then more effort is required if non-autotool files in the package now need to be patched to work with the specific libtool version used. If a package does not check generated autotools files into its source repository then each person using the source repository needs to autotool the package, and changing the path makes the package libtool-version sensitive so that only newer (or older) versions will work. It is rather inconvenient for developers to maintain several installed libtool versions in order to successfully autotool packages. I am not saying that the effort is insurmountable but it is perhaps more effort to the world at large than your are imagining. However, what did you think of my plan to adopt a gnulib inspired cruft removal warning and tidy up process (the second paragraph in this post: http://lists.gnu.org/archive/html/libtool-patches/2011-11/msg2.html)? If you don't object, in that vein I'll add code to libtoolize (scheduled for removal in 2013) which copies $pkgdatadir/build-aux contents to the existing ltdl/config you've specified in the parent configure.ac AC_CONFIG_AUX_DIR declaration... My project (and the derived ImageMagick) are among the very few which would be impacted by this proposal since its non-recursive build is including the libltdl bits from Makefile.am like: include ltdl/Makefile.inc This is another case of changing a documented external interface. The impact is surely much smaller than moving the 'config' files. Google shows very few packages using this approach. The deprecation detection logic does not seem fully robust since there is no standard extension for Automake include files and file naming other than *.mk and Makefile.am might be used. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
On 11/01/2011 08:19 AM, Gary V. Vaughan wrote: This next series of changesets is the beginning of modelling the directory and filenaming conventions of Libtool to match what is done by other prominent GNU projects... this means that gnulib scripts will require less tweaking and configuration, so we can use them in the configuration in which they have been most widely tested and debugged. Also, it makes everything look more familiar to anyone coming from another GNU project who is hoping to generate some patches for us... and every little thing we can do to reduce friction in that process is a net win as far as I am concerned. I've attached the full patch which is ridiculously large for a simple directory move and fixing the fallout in the files that care. Next time, when sending a file rename patch, consider using 'git format-patch/send-email -M -C', which finds moves and renames and compresses them into a much smaller patch output that names just the old and new file name and any incidental differences between the two files (if I understand git correctly, the only reason the smaller patch is not default is because it takes a bit more processor time to determine file similarity and because it was not understood by patch(1) back in the days when git used patch rather than its own tools for parsing changesets; but I don't ever notice the difference, and definitely appreciate the smaller diffs). Or make those options permanent for your git setup, with 'git config --global diff.renames true'. At any rate, I'm certainly in favor of this series, in the name of easier maintenance, although I didn't review it closely. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org
Re: [PATCH 2/3] maint: rename `libltdl/m4' directory to standard `m4'.
On Tuesday 01 November 2011, Gary V wrote: Similarly to 1/3, full compressed patch attached. 72 hours etc etc. * bootstrap.conf (libtool_link_libltdl_subdirs): Add `m4'. * cfg.mk (exclude_file_name_regexp--sc_prohibit_test_minus_ao): Adjust path. * configure.ac (AC_CONFIG_MACRO_DIR): Adjust. * libltdl/m4: Moved to it's parent directory. s/it's/its/. * tests/cdemo/Makefile.am, tests/demo/Makefile.am, tests/depdemo/Makefile.am, tests/f77demo/Makefile.am, tests/fcdemo/Makefile.am, tests/mdemo/Makefile.am, tests/mdemo2/Makefile.am, tests/pdemo/Makefile.am, tests/tagdemo/Makefile.am (ACLOCAL_AMFLAGS): Use new location of macro_dir. * libtoolize.m4sh: Ditto. diff --git a/bootstrap.conf b/bootstrap.conf index 2909d30..ebdb293 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -353,6 +353,7 @@ libtool_link_libltdl_subdirs () $debug_cmd $require_build_aux +$require_macro_dir $require_pkgaux_scripts my_pkgaux_files=$pkgaux_scripts $build_aux/ltmain.sh @@ -372,6 +373,11 @@ libtool_link_libltdl_subdirs () for my_file in $my_pkgaux_files; do ln -s ../../$my_file libltdl/$my_file done + +# Macro directory is identical, so link the directory. +func_verbose creating libltdl/$macro_dir +rm -f libltdl/$macro_dir || exit 1 +ln -s ../$macro_dir libltdl/$macro_dir } Same as before: is `ln -s' truly portable to e.g. MinGW? Should we care if the boostrapping process does not work there? package_revision=`$SHELL $ac_aux_dir/git-version-gen .tarball-version` diff --git a/libltdl/.gitignore b/libltdl/.gitignore index 2f39096..5795dbc 100644 --- a/libltdl/.gitignore +++ b/libltdl/.gitignore @@ -2,5 +2,4 @@ /Makefile.am /argz.h /build-aux +/m4 -/dummy.c -/gnulib.mk Shouldn't these last two edits be done in a separate patch? Also, shouldn't you report the changes to file `libltdl/.gitignore' in the ChangeLog entry? diff --git a/libltdl/m4/.gitignore b/libltdl/m4/.gitignore deleted file mode 100644 index 1facea7..000 --- a/libltdl/m4/.gitignore +++ /dev/null @@ -1,6 +0,0 @@ -/00gnulib.m4 -/gnulib-cache.m4 -/gnulib-common.m4 -/gnulib-comp.m4 -/gnulib-tool.m4 -/ltversion.m4 Ditto for the ChangeLog entry. Regards, Stefano
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
On Tue, 1 Nov 2011, Gary V. Vaughan wrote: This next series of changesets is the beginning of modelling the directory and filenaming conventions of Libtool to match what is done by other prominent GNU projects... this means that gnulib scripts will require less tweaking and configuration, so we can use them in the configuration in which they have been most widely tested and debugged. Also, it makes everything look more familiar to anyone coming from another GNU project who is hoping to generate some patches for us... and every little thing we can do to reduce friction in that process is a net win as far as I am concerned. I should not have to remind that GNU libtool is usually embedded to become part of other packages and some have become more intimate with libtool (as documented) in order to improve build times and reduce redundant waste. For packages (such as mine), the libltdl 'config' directory is treated as a well defined public interface which is referenced by (and shared by) exisiting package configure scripts. For example, this is how the configure.ac refers to it in my package: # Directory where autotools helper scripts lives. AC_CONFIG_AUX_DIR([ltdl/config]) Changing from 'config' to 'build-aux' will require existing configure.ac files to change. Did you consider this existing use while developing the plan to rename this often-shared directory? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
Hi Bob, Thanks for the feedback. On 1 Nov 2011, at 23:50, Bob Friesenhahn wrote: On Tue, 1 Nov 2011, Gary V. Vaughan wrote: This next series of changesets is the beginning of modelling the directory and filenaming conventions of Libtool to match what is done by other prominent GNU projects... this means that gnulib scripts will require less tweaking and configuration, so we can use them in the configuration in which they have been most widely tested and debugged. Also, it makes everything look more familiar to anyone coming from another GNU project who is hoping to generate some patches for us... and every little thing we can do to reduce friction in that process is a net win as far as I am concerned. I should not have to remind that GNU libtool is usually embedded to become part of other packages and some have become more intimate with libtool (as documented) in order to improve build times and reduce redundant waste. For packages (such as mine), the libltdl 'config' directory is treated as a well defined public interface which is referenced by (and shared by) exisiting package configure scripts. For example, this is how the configure.ac refers to it in my package: # Directory where autotools helper scripts lives. AC_CONFIG_AUX_DIR([ltdl/config]) Changing from 'config' to 'build-aux' will require existing configure.ac files to change. Did you consider this existing use while developing the plan to rename this often-shared directory? Yes I did, and it doesn't seem at all onerous to me if I have gone to all the trouble of upgrading libtool and re-bootstrapping my project with it to also amend one line in configure.ac. As it stands, libtoolize will issue an upgrade recommendation, and if you ignore that, nothing will actually break, you will just get duplicate copies of libltdl's build-aux files (compile, config.guess, config.sub, depcomp, install-sh, ltmain.sh and missing) alongside your parent project's files in ltdl/config. However, what did you think of my plan to adopt a gnulib inspired cruft removal warning and tidy up process (the second paragraph in this post: http://lists.gnu.org/archive/html/libtool-patches/2011-11/msg2.html)? If you don't object, in that vein I'll add code to libtoolize (scheduled for removal in 2013) which copies $pkgdatadir/build-aux contents to the existing ltdl/config you've specified in the parent configure.ac AC_CONFIG_AUX_DIR declaration... Cheers, -- Gary V. Vaughan (gary AT gnu DOT org)
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
Hi Chuck, On 1 Nov 2011, at 22:15, Charles Wilson wrote: On 11/1/2011 10:57 AM, Stefano Lattarini wrote: Is `ln -s' portable to e.g., MinGW? Or is the bootstrapping process not meant to work there anyway, so we can just not care? No, bootstrapping is supposed to work (and, until quite recently, did work) under MinGW/MSYS. Current breakage is related to lack of a 'git' implementation with stock MinGW/MSYS installations -- and now 'git' is required to bootstrap thanks to git-format-ChangeLog and other git-isms IIUC. Git should not be required to bootstrap the current master... although I did inadvertently break that with a previous build. If you find that the bootstrap script is choking when you don't have git installed, then please send me a bug report and I will fix it. (e.g. you can force bootstrap to run in a dist tarball with `bootstrap --force --skip-git'). Git is required for `make dist', but I don't think you need to run make dist in order to bootstrap and install libtool on cygwin. In the past I could create a git checkout using cygwin-git, and then switch to MinGW/MSYS and do a full bootstrap; but (again, IIUC) that is not possible any longer. (I didn't typically do it this way, BTW; I usually bootstrapped under cygwin, and later copied the 'make dist' tarball to MinGW and just built it there.) That process might need some help to continue working without a working git package, but I think you should be able to work around it with a stub git binary in your path... assuming you don't plan on uploading full release quality builds of libtool that were bootstrapped and rolled up from a git checkout entirely on a machine with no git binary. (There is the msys-git distribution, used by the mingw64 folks -- but it is really a sortof mingw32 git with msys bits added, plus a custom forked version of the msys dll and other tools...) I'm (slowly) working on adding a true msys git implementation to the MinGW/MSYS distribution... Cool! Cheers, -- Gary V. Vaughan (gary AT gnu DOT org)
Re: [PATCH 1/3] maint: rename `libltdl/config' directory to standard `build-aux'.
Hi Eric, On 1 Nov 2011, at 22:03, Eric Blake wrote: On 11/01/2011 08:19 AM, Gary V. Vaughan wrote: This next series of changesets is the beginning of modelling the directory and filenaming conventions of Libtool to match what is done by other prominent GNU projects... this means that gnulib scripts will require less tweaking and configuration, so we can use them in the configuration in which they have been most widely tested and debugged. Also, it makes everything look more familiar to anyone coming from another GNU project who is hoping to generate some patches for us... and every little thing we can do to reduce friction in that process is a net win as far as I am concerned. I've attached the full patch which is ridiculously large for a simple directory move and fixing the fallout in the files that care. Next time, when sending a file rename patch, consider using 'git format-patch/send-email -M -C' Ahah! I suspected it must be possible, because tig displays patches in that much shorter format by default, but I didn't think to chase down the right flags. Thanks for the hint :) , which finds moves and renames and compresses them into a much smaller patch output that names just the old and new file name and any incidental differences between the two files (if I understand git correctly, the only reason the smaller patch is not default is because it takes a bit more processor time to determine file similarity and because it was not understood by patch(1) back in the days when git used patch rather than its own tools for parsing changesets; but I don't ever notice the difference, and definitely appreciate the smaller diffs). Or make those options permanent for your git setup, with 'git config --global diff.renames true'. Done. At any rate, I'm certainly in favor of this series, in the name of easier maintenance, although I didn't review it closely. Me too... I'm kinda surprised at the amount of cruft libtool is carrying around right now, and moving decisively towards a gnulib based build is the perfect opportunity to spring clean :) Cheers, -- Gary V. Vaughan (gary AT gnu DOT org)
Re: [PATCH 2/3] maint: rename `libltdl/m4' directory to standard `m4'.
Hi Stefano, Thanks for the feedback. On 1 Nov 2011, at 22:06, Stefano Lattarini wrote: On Tuesday 01 November 2011, Gary V wrote: Similarly to 1/3, full compressed patch attached. 72 hours etc etc. * bootstrap.conf (libtool_link_libltdl_subdirs): Add `m4'. * cfg.mk (exclude_file_name_regexp--sc_prohibit_test_minus_ao): Adjust path. * configure.ac (AC_CONFIG_MACRO_DIR): Adjust. * libltdl/m4: Moved to it's parent directory. s/it's/its/. Oops. Applied. * tests/cdemo/Makefile.am, tests/demo/Makefile.am, tests/depdemo/Makefile.am, tests/f77demo/Makefile.am, tests/fcdemo/Makefile.am, tests/mdemo/Makefile.am, tests/mdemo2/Makefile.am, tests/pdemo/Makefile.am, tests/tagdemo/Makefile.am (ACLOCAL_AMFLAGS): Use new location of macro_dir. * libtoolize.m4sh: Ditto. diff --git a/bootstrap.conf b/bootstrap.conf index 2909d30..ebdb293 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -353,6 +353,7 @@ libtool_link_libltdl_subdirs () $debug_cmd $require_build_aux +$require_macro_dir $require_pkgaux_scripts my_pkgaux_files=$pkgaux_scripts $build_aux/ltmain.sh @@ -372,6 +373,11 @@ libtool_link_libltdl_subdirs () for my_file in $my_pkgaux_files; do ln -s ../../$my_file libltdl/$my_file done + +# Macro directory is identical, so link the directory. +func_verbose creating libltdl/$macro_dir +rm -f libltdl/$macro_dir || exit 1 +ln -s ../$macro_dir libltdl/$macro_dir } Same as before: is `ln -s' truly portable to e.g. MinGW? Should we care if the boostrapping process does not work there? I'll resubmit this patch with a fallback to copy too. package_revision=`$SHELL $ac_aux_dir/git-version-gen .tarball-version` diff --git a/libltdl/.gitignore b/libltdl/.gitignore index 2f39096..5795dbc 100644 --- a/libltdl/.gitignore +++ b/libltdl/.gitignore @@ -2,5 +2,4 @@ /Makefile.am /argz.h /build-aux +/m4 -/dummy.c -/gnulib.mk Shouldn't these last two edits be done in a separate patch? No, because I don't want to introduce broken revisions that cannot build into git history. Also, shouldn't you report the changes to file `libltdl/.gitignore' in the ChangeLog entry? That's interesting actually. Historically, we have only reported changes to distributed files in ChangeLog, and have always omitted at least VCS control files from the log entries. I'm still leaning slightly towards not introducing ChangeLog noise to describe things that are only important when you have the full git checkout available, and hence access to git log and friends if you want to dig this sort of information out -- BUT I could easily be persuaded to change my mind if you have a good argument for the advantage of putting this stuff into the git log entry, which would then eventually be put into the generated ChangeLog file... Cheers, -- Gary V. Vaughan (gary AT gnu DOT org)
Re: [RFC] [PATCH] libltdl error reporting
On 06/14/2010 04:41 PM, Peter O'Gorman wrote: dlopen nonexisting file dlopen existing file check dlerror() Ok, systems that return an error in this case: Solaris, AIX, HPUX/IA64, FreeBSD systems that don't return an error: HP-UX/HPPA, Tru64 5.1, IRIX 6.5, linux/glibc, Mac OS X. Whee. Peter ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Compiler warning in libltdl/m4/libtool.m4 test program
On Tue, 2010-06-15 at 20:14 +0200, Ralf Wildenhues wrote: Please note that in general, it is impossible for configure tests to be -Werror clean. That is not only due to the sheer amount of possible erros from different GCC compiler versions (not to speak of others), different sets of warning flags, and system-specific issues, but also because inherently, some configure tests have to do things that are only half-decent C. Thus we strongly recommend to add -Werror only at 'make' run time and not at configure time already. Hmm... not what I had imagined, but point taken. This was the only failure of this kind in the particular configure script I was looking at, but I shall bear this in mind if I run into anything similar in future. Is there a trivial way to specify cumulative flags at make-time, as opposed to an override? Not that the latter is a major issue for me, just means I may have to change a few habits. :) This notice is inappropriate for a public mailing lists. Your mail has already been archived at several locations and communicated to an unknown and uncontrollable number of recipients. If you can't turn off the note, please consider writing from a non-business account. Also a good point - it's a typical company policy thing, rather than something I personally append. Nobody's pointed it out before on other lists, but you're right. I might have a word and see if I can get a less draconian signature for mailing list use. In the mean time I'm sure a few sig-less messages won't hurt anyone. Regards, Phil signature.asc Description: This is a digitally signed message part
Re: Compiler warning in libltdl/m4/libtool.m4 test program
On 06/16/2010 01:48 AM, Philip Allison wrote: On Tue, 2010-06-15 at 20:14 +0200, Ralf Wildenhues wrote: Please note that in general, it is impossible for configure tests to be -Werror clean. That is not only due to the sheer amount of possible erros from different GCC compiler versions (not to speak of others), different sets of warning flags, and system-specific issues, but also because inherently, some configure tests have to do things that are only half-decent C. Thus we strongly recommend to add -Werror only at 'make' run time and not at configure time already. Hmm... not what I had imagined, but point taken. This was the only failure of this kind in the particular configure script I was looking at, but I shall bear this in mind if I run into anything similar in future. Is there a trivial way to specify cumulative flags at make-time, as opposed to an override? Not that the latter is a major issue for me, just means I may have to change a few habits. :) Coreutils serves as an example of a project that probes whether the compiler supports -Werror, then adds a specific Makefile substitution to append -Werror (or the empty string) to the various CFLAGS throughout the project. You can check out coreutils' ./configure --enable-gcc-warnings option. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Compiler warning in libltdl/m4/libtool.m4 test program
The test program for whether a program can dlopen itself generates a compiler warning, which turns into a failure when building with -Wall -Werror in CFLAGS. The error is in the libltdl/m4/libtool.m4 file, and is due to the fnord function containing an unused variable. Please find attached a simple patch for this issue. 2010-06-15 Philip Allison philip.alli...@smoothwall.net * libltdl/m4/libtool.m4 (_LT_TRY_DLOPEN_SELF): Fix unused variable warning in test program. -- Philip Allison Senior Developer SmoothWall Ltd 1 John Charles Way Leeds LS12 6QA United Kingdom 1 800 959 3760 (USA, Canada and North America) 0870 1 999 500 (United Kingdom) +44 870 1 999 500 (All other countries) SmoothWall is registered in England: 4298247 This email and any attachments transmitted with it are confidential to the intended recipient(s) and may not be communicated to any other person or published by any means without the permission of SmoothWall Limited. Any opinions stated in this message are solely those of the author. See: http://smoothwall.net/company/email.php for the full text of this notice. diff --git a/libltdl/m4/libtool.m4 b/libltdl/m4/libtool.m4 index 22924a8..d35aab0 100644 --- a/libltdl/m4/libtool.m4 +++ b/libltdl/m4/libtool.m4 @@ -1658,10 +1658,10 @@ else /* When -fvisbility=hidden is used, assume the code has been annotated correspondingly for the symbols needed. */ #if defined(__GNUC__) (((__GNUC__ == 3) (__GNUC_MINOR__ = 3)) || (__GNUC__ 3)) -void fnord () __attribute__((visibility(default))); +int fnord () __attribute__((visibility(default))); #endif -void fnord () { int i=42; } +int fnord () { return 42; } int main () { void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); signature.asc Description: This is a digitally signed message part
Re: Compiler warning in libltdl/m4/libtool.m4 test program
Hello Philip, thanks for the report and patch. * Philip Allison wrote on Tue, Jun 15, 2010 at 01:15:14PM CEST: The test program for whether a program can dlopen itself generates a compiler warning, which turns into a failure when building with -Wall -Werror in CFLAGS. Please note that in general, it is impossible for configure tests to be -Werror clean. That is not only due to the sheer amount of possible erros from different GCC compiler versions (not to speak of others), different sets of warning flags, and system-specific issues, but also because inherently, some configure tests have to do things that are only half-decent C. Thus we strongly recommend to add -Werror only at 'make' run time and not at configure time already. That said, 2010-06-15 Philip Allison philip.alli...@smoothwall.net * libltdl/m4/libtool.m4 (_LT_TRY_DLOPEN_SELF): Fix unused variable warning in test program. this patch looks ok to me. I've applied it in your name, and added you to THANKS. Please note that you will need a copyright assignment in place for nontrivial contributions (more off-list if you are interested), but not for this one. This email and any attachments transmitted with it are confidential to the intended recipient(s) and may not be communicated to any other person or published by any means [...] This notice is inappropriate for a public mailing lists. Your mail has already been archived at several locations and communicated to an unknown and uncontrollable number of recipients. If you can't turn off the note, please consider writing from a non-business account. Thanks! Ralf
Re: [RFC] [PATCH] libltdl error reporting
On 06/10/2010 12:33 AM, Peter O'Gorman wrote: At least glibc and Mac OS X reset the dlerror() string to NULL every time a dl*() api is called (I will check what other systems do in the next few days). This is so programmers can do: Sigh, but FreeBSD doesn't. dlopen nonexisting file dlopen existing file check dlerror() Says 'Cannot open nonexisting file' on FreeBSD (null with glibc and Mac OS X). Oh well, will be interesting to see what other systems to. Peter ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: [RFC] [PATCH] libltdl error reporting
Hi Peter, On 10 Jun 2010, at 12:33, Peter O'Gorman wrote: As I am sure many are aware, libltdl's error reporting is pretty dumb, lt_dlerror() regularly reports things like file not found where the actual problem might be something completely different, and a reasonable error string may be readily available from dlerror(). Treason! When I looked at the manual, and read the description for lt_dlerror() I saw Return a human readable string describing the most recent error that occurred from any of libltdl's functions. Return NULL if no errors have occurred since initialization or since it was last called. and I thought Well, that's stupid - that's not what dlerror() does. So, I went to see what POSIX says - it's also wrong - http://www.opengroup.org/onlinepubs/9699919799/functions/dlerror.html, I will file a bug with them later. At least glibc and Mac OS X reset the dlerror() string to NULL every time a dl*() api is called (I will check what other systems do in the next few days). This is so programmers can do: . dlsym(handle, foo); if ((err = dlerror()) != NULL) printf(stderr,dlsym error: %s\n,err); ... without dlerror() returning some other error that occurred long before the call to dlsym(). Interesting. IIRC, Thomas chose the semantics and API of libltdl to closely match the Unix dlfcn.h calls so that it could be used as a direct replacement by a client with very little editing. If glibc and OS X (which to my mind probably covers 80% of the users of libltdl) is already doing something saner than the documentation might have lead us to believe, then your change looks like a net win to me! Aside: I'm leaning away from upholding the 'drop-in-with-minimum-edits' philosophy for my rewrite, since the dlfcn.h API seems like a pretty bad design to me. After all, all people really need to do is call functions with a known name and known signature which happen to be in another library. I'm seriously contemplating using a *much* smaller and cleaner API, which ends up with client code more along the lines of: #include setjmp.h static jmp_buf exception; void * my_get_dso_func (const char *dsopath, const char *symname) { if (! setjmp (exception)) { /* setjmp returns 0 on first invocation */ return lt_dlsym (dsopath, symname); } else { /* but if lt_dlsym throws an exception, we end up here */ fputs (myapp: my_get_dso_func failed, stderr); } } With that as a starting point, it's easy enough to maintain an error stack in an exception struct that wraps around setjmp/longjmp, and to pretty up the code above with some macros to make it look more like the common throw/catch syntax. At that point, with libltdl2 doing the book-keeping, the entire API is just: typedef void *lt_exception; void *lt_dlsym (const char *dsopath, const char *symname); void lt_dlclose (const char *dsopath); plus some glue to make catching an error thrown from libltdl2 and for unwinding the error stack inside it as easy as possible. WDYT? Unfortunately, it looks like libltdl attempts to save error state across calls to lt_*() functions, and generally does not do it well. This patch resets the error every time a public ltdl function is called, and when setting an error tests if it is already set and doesn't overwrite it if it is. The fist error wins. The patch is fairly simple, and undoubtedly still imperfect, but test results are good, one test required a small patch to stop segfaulting, another should have unexpectedly passed, but doesn't, I think it is another error in the test, but haven't looked too closely yet. If something like this does get in, I'd prefer to not change the libltld soname, I don't believe better error reporting will cause client applications to break. I could be wrong, of course :) Patch attached (or course, a final patch will have to change the manual as well) - thoughts? And add some test coverage. I like it. And it gives us something to tide us over until I've finished libltdl2 in 2012 ;) Cheers, -- Gary V. Vaughan (g...@gnu.org) ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: [RFC] [PATCH] libltdl error reporting
Aside: I'm leaning away from upholding the 'drop-in-with-minimum-edits' philosophy for my rewrite, since the dlfcn.h API seems like a pretty bad design to me. After all, all people really need to do is call functions with a known name and known signature which happen to be in another library. I'm seriously contemplating using a *much* smaller and cleaner API, which ends up with client code more along the lines of: With that as a starting point, it's easy enough to maintain an error stack in an exception struct that wraps around setjmp/longjmp, and to plus some glue to make catching an error thrown from libltdl2 and for unwinding the error stack inside it as easy as possible. WDYT? I think it would be better in c++. I am unsure that a rewrite is necessary though. Peter ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: [RFC] [PATCH] libltdl error reporting
Hi Peter, On 10 Jun 2010, at 21:15, Peter O'Gorman wrote: Aside: I'm leaning away from upholding the 'drop-in-with-minimum-edits' philosophy for my rewrite, since the dlfcn.h API seems like a pretty bad design to me. After all, all people really need to do is call functions with a known name and known signature which happen to be in another library. I'm seriously contemplating using a *much* smaller and cleaner API, which ends up with client code more along the lines of: [[...]] With that as a starting point, it's easy enough to maintain an error stack in an exception struct that wraps around setjmp/longjmp, and to plus some glue to make catching an error thrown from libltdl2 and for unwinding the error stack inside it as easy as possible. WDYT? I think it would be better in c++. No, that would mean you have to jump through hoops to use it from C. And it would make me cry myself to sleep at night. I avoid C++, Perl, McDonalds and suicide bomber recruiters as much as I possibly can. I'm still undecided as to which one is worst for your health... I am unsure that a rewrite is necessary though. For me to get any further than I have already, without making things even uglier than they already are, rearchitecting the whole thing seems like the best way to learn from the mistakes of the 12 year old incumbent implementation. Otherwise I end up staring at it, and spending whole weekends trying to understand why it handed back another useless file-not-found error. ;) Cheers, -- Gary V. Vaughan (g...@gnu.org) ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: rewrite of ltdl and c++ (was: Re: [RFC] [PATCH] libltdl error reporting)
On 06/10/2010 09:45 AM, Gary V. Vaughan wrote: I think it would be better in c++. No, that would mean you have to jump through hoops to use it from C. And it would make me cry myself to sleep at night. I avoid C++, Perl, McDonalds and suicide bomber recruiters as much as I possibly can. I'm still undecided as to which one is worst for your health... It's simple to write a library in C++ but make its public interface C. There are many projects that use what I would describe as sane C++ (unfortunately there are also many that use every possible feature). IMO, setjmp/longjmp is significantly uglier. But, let's end this debate, it's unlikely to lead to anything productive :) Peter ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: [RFC] [PATCH] libltdl error reporting
On 06/10/2010 02:28 PM, Bob Friesenhahn wrote: On Thu, 10 Jun 2010, Peter O'Gorman wrote: As I am sure many are aware, libltdl's error reporting is pretty dumb, lt_dlerror() regularly reports things like file not found where the actual problem might be something completely different, and a reasonable error string may be readily available from dlerror(). As we have known for quite a while now, libltdl's error reporting is not sane. For example, the preloader always issues a useless error (file not found) when preloading is not used for the module. Since the loaders were nicely modularized, each loader is tried in turn (issuing an error on failure) until a loader is successful, or there are no more loaders. This is perhaps worse on OS-X where there are two OS APIs for loading modules. I think that some improvements were made, but functionality is still lacking. The patch that I attached should solve most of those issues. On Mac OS X, the dyld loader is not built if dlopen is available, if I remember correctly. Peter ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: [RFC] [PATCH] libltdl error reporting
On Thu, 10 Jun 2010, Peter O'Gorman wrote: As I am sure many are aware, libltdl's error reporting is pretty dumb, lt_dlerror() regularly reports things like file not found where the actual problem might be something completely different, and a reasonable error string may be readily available from dlerror(). As we have known for quite a while now, libltdl's error reporting is not sane. For example, the preloader always issues a useless error (file not found) when preloading is not used for the module. Since the loaders were nicely modularized, each loader is tried in turn (issuing an error on failure) until a loader is successful, or there are no more loaders. This is perhaps worse on OS-X where there are two OS APIs for loading modules. I think that some improvements were made, but functionality is still lacking. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
[RFC] [PATCH] libltdl error reporting
Hi, As I am sure many are aware, libltdl's error reporting is pretty dumb, lt_dlerror() regularly reports things like file not found where the actual problem might be something completely different, and a reasonable error string may be readily available from dlerror(). When I looked at the manual, and read the description for lt_dlerror() I saw Return a human readable string describing the most recent error that occurred from any of libltdl's functions. Return NULL if no errors have occurred since initialization or since it was last called. and I thought Well, that's stupid - that's not what dlerror() does. So, I went to see what POSIX says - it's also wrong - http://www.opengroup.org/onlinepubs/9699919799/functions/dlerror.html, I will file a bug with them later. At least glibc and Mac OS X reset the dlerror() string to NULL every time a dl*() api is called (I will check what other systems do in the next few days). This is so programmers can do: . dlsym(handle, foo); if ((err = dlerror()) != NULL) printf(stderr,dlsym error: %s\n,err); ... without dlerror() returning some other error that occurred long before the call to dlsym(). Unfortunately, it looks like libltdl attempts to save error state across calls to lt_*() functions, and generally does not do it well. This patch resets the error every time a public ltdl function is called, and when setting an error tests if it is already set and doesn't overwrite it if it is. The fist error wins. The patch is fairly simple, and undoubtedly still imperfect, but test results are good, one test required a small patch to stop segfaulting, another should have unexpectedly passed, but doesn't, I think it is another error in the test, but haven't looked too closely yet. If something like this does get in, I'd prefer to not change the libltld soname, I don't believe better error reporting will cause client applications to break. I could be wrong, of course :) Patch attached (or course, a final patch will have to change the manual as well) - thoughts? Peter diff --git a/libltdl/libltdl/lt__private.h b/libltdl/libltdl/lt__private.h index f4c4a3d..84f5622 100644 --- a/libltdl/libltdl/lt__private.h +++ b/libltdl/libltdl/lt__private.h @@ -138,7 +138,10 @@ struct lt__advise { #define LT__GETERROR(lvalue) (lvalue) = lt__get_last_error() #define LT__SETERRORSTR(errormsg) lt__set_last_error(errormsg) -#define LT__SETERROR(errorcode) LT__SETERRORSTR(LT__STRERROR(errorcode)) +#define LT__SETERROR(errorcode) if (0 == lt__get_last_error()) \ + LT__SETERRORSTR(LT__STRERROR(errorcode)) +#define LT__RESETERROR(x) lt__set_last_error(0) +#define LT__FORCEERROR(errorstr) LT__SETERRORSTR(errorstr) LT_SCOPE const char *lt__error_string (int errorcode); LT_SCOPE const char *lt__get_last_error (void); diff --git a/libltdl/loaders/preopen.c b/libltdl/loaders/preopen.c index 7149287..58464ca 100644 --- a/libltdl/loaders/preopen.c +++ b/libltdl/loaders/preopen.c @@ -293,6 +293,7 @@ int lt_dlpreload_default (const lt_dlsymlist *preloaded) { default_preloaded_symbols = preloaded; + LT__RESETERROR (); return 0; } @@ -303,6 +304,7 @@ int lt_dlpreload (const lt_dlsymlist *preloaded) { int errors = 0; + LT__RESETERROR (); if (preloaded) { @@ -331,6 +333,7 @@ lt_dlpreload_open (const char *originator, lt_dlpreload_callback_func *func) symlist_chain *list; int errors = 0; int found = 0; + LT__RESETERROR (); /* For each symlist in the chain... */ for (list = preloaded_symlists; list; list = list-next) diff --git a/libltdl/ltdl.c b/libltdl/ltdl.c index e1b4e2f..098effa 100644 --- a/libltdl/ltdl.c +++ b/libltdl/ltdl.c @@ -216,6 +216,8 @@ int lt_dlinit (void) { int errors = 0; + const char* saved_error = 0; + LT__RESETERROR (); /* Initialize only at first call. */ if (++initialized == 1) @@ -232,6 +234,7 @@ lt_dlinit (void) /* Now open all the preloaded module loaders, so the application can use _them_ to lt_dlopen its own modules. */ #ifdef HAVE_LIBDLLOADER + LT__GETERROR(saved_error); if (!errors) { errors += lt_dlpreload (preloaded_symbols); @@ -241,6 +244,7 @@ lt_dlinit (void) { errors += lt_dlpreload_open (LT_STR(LTDLOPEN), loader_init_callback); } + LT__FORCEERROR(saved_error); #endif /* HAVE_LIBDLLOADER */ } @@ -258,6 +262,7 @@ lt_dlexit (void) lt_dlloader *loader = 0; lt_dlhandle handle = handles; int errors = 0; + LT__RESETERROR (); if (!initialized) { @@ -359,7 +364,6 @@ tryall_dlopen (lt_dlhandle *phandle, const char *filename, lt_dladvise advise, const lt_dlvtable *vtable) { lt_dlhandle handle = handles; - const char * saved_error = 0; int errors = 0; #ifdef LT_DEBUG_LOADERS @@ -368,7 +372,6 @@ tryall_dlopen (lt_dlhandle *phandle, const char *filename, vtable ? vtable-name : (ALL)); #endif - LT__GETERROR (saved_error
Re: [PATCH] There's no point in setting up the libltdl build when we're using an installed copy
Hello Ari, thanks for the report and patch. * Ari Entlich wrote on Mon, Mar 08, 2010 at 05:58:59AM CET: The subject says it all. Comments appreciated. A very minor nit: mails are easier to grok if the contents don't refer to the subject. Patches are a lot easier to understand with a corresponding ChangeLog entry. (Sorry, couldn't resist.) I see two problems with this patch, but haven't tested it so far. First, in some cases, _LTDL_MODE_DISPATCH may expand to the empty string (or whitespace only), which will then cause a shell syntax error. More fundamentally though, when we don't configure the subpackage at all, then a 'make dist' won't package up the included libltdl, which means the package tarball can't be reproduced. Not totally sure about this, but changing this would be an API change and require a big warning, so probably a new configure switch. Lastly, the change to the comment isn't fully correct (but this is really a nit, too), since the _LTDL_MODE_DISPATCH macro also has effect at m4/autoconf run time, that means it will have an effect even if only inside a shell conditional branch that is not used at configure time. Cheers, Ralf --- a/libltdl/m4/ltdl.m4 +++ b/libltdl/m4/ltdl.m4 @@ -384,8 +384,11 @@ dnl Don't require this, or it will be expanded earlier than the code dnl that sets the variables it relies on: _LT_ENABLE_INSTALL -dnl _LTDL_MODE specific code must be called at least once: -_LTDL_MODE_DISPATCH +dnl _LTDL_MODE specific code must be called at least once if +dnl we're using the included copy of libltdl: +if test x$with_included_ltdl = xyes; then + _LTDL_MODE_DISPATCH +fi
[PATCH] There's no point in setting up the libltdl build when we're using an installed copy
The subject says it all. Comments appreciated. Thanks! Ari -- diff --git a/libltdl/m4/ltdl.m4 b/libltdl/m4/ltdl.m4 index 111fac1..67317c0 100644 --- a/libltdl/m4/ltdl.m4 +++ b/libltdl/m4/ltdl.m4 @@ -384,8 +384,11 @@ dnl Don't require this, or it will be expanded earlier than the code dnl that sets the variables it relies on: _LT_ENABLE_INSTALL -dnl _LTDL_MODE specific code must be called at least once: -_LTDL_MODE_DISPATCH +dnl _LTDL_MODE specific code must be called at least once if +dnl we're using the included copy of libltdl: +if test x$with_included_ltdl = xyes; then + _LTDL_MODE_DISPATCH +fi # In order that ltdl.c can compile, find out the first AC_CONFIG_HEADERS # the user used. This is so that ltdl.h can pick up the parent projects
ltdl .la file parsing (was: Coverage for libltdl/slist.c and fallout)
Hi Bob, * Bob Friesenhahn wrote on Sun, Dec 06, 2009 at 07:33:28PM CET: There is no doubt that testing is important. It seems most important (to me) that externally-exposed interfaces should be validated before there is intense focus on test coverage of internal interfaces. For example, it is already known that the .la file parser is fragile and trivial edits to an .la file will cause the using program to core-dump. The .la file parsing is an external interface so it should get more priority. Good idea. Proposed patch. What other things do you know about that are mishandled? OK to commit? Thanks, Ralf Improve parsing of .la files in libltdl. * libltdl/ltdl.c (trim): Do not dump core upon missing quote in module .la file. * tests/lalib-syntax.at (syntax of .la files): New file, new test. * Makefile.am (TESTSUITE_AT): Add tests/lalib-syntax.at. Report by Bob Friesenhahn. diff --git a/Makefile.am b/Makefile.am index 2b09991..c26f05b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -484,6 +484,7 @@ TESTSUITE_AT= tests/testsuite.at \ tests/lt_dlopen_a.at \ tests/lt_dlopenext.at \ tests/ltdl-api.at \ + tests/lalib-syntax.at \ tests/slist.at \ tests/need_lib_prefix.at \ tests/standalone.at \ diff --git a/libltdl/ltdl.c b/libltdl/ltdl.c index acbc68f..dc1d209 100644 --- a/libltdl/ltdl.c +++ b/libltdl/ltdl.c @@ -1000,7 +1000,7 @@ trim (char **dest, const char *str) FREE (*dest); - if (!end) + if (!end || end == str) return 1; if (len 3 str[0] == '\'') diff --git a/tests/lalib-syntax.at b/tests/lalib-syntax.at new file mode 100644 index 000..65d6b55 --- /dev/null +++ b/tests/lalib-syntax.at @@ -0,0 +1,125 @@ +# lalib-syntax.at -- parsing .la files robustly -*- Autotest -*- +# +# Copyright (C) 2009 Free Software Foundation, Inc. +# +# This file is part of GNU Libtool. +# +# GNU Libtool is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of +# the License, or (at your option) any later version. +# +# GNU Libtool is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Libtool; see the file COPYING. If not, a copy +# can be downloaded from http://www.gnu.org/licenses/gpl.html, +# or obtained by writing to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + + +AT_SETUP([syntax of .la files]) +AT_KEYWORDS([libtool]) +AT_KEYWORDS([libltdl]) + +AT_DATA([main.c], +[[#include ltdl.h +#include stdio.h + +int +main (int argc, char* argv[]) +{ + int err = 0; + lt_dlhandle plugin_handle; + + if (argc 2) +{ + fprintf (stderr, usage: %s plugin\n, argv[0]); + return 1; +} + + lt_dlinit (); + plugin_handle = lt_dlopenext (argv[1]); + if (NULL != plugin_handle) +{ + printf (plugin opened successfully!\n); + lt_dlclose (plugin_handle); +} + else +{ + printf (plugin failed to open: %s\n, lt_dlerror()); + err = 1; +} + lt_dlexit (); + return err; +} +]]) + +AT_DATA([module.c], +[[int foo (void) { return 0; } +]]) + +: ${LTDLINCL=-I$abs_top_srcdir/libltdl} +: ${LIBLTDL=$abs_builddir/../libltdl/libltdlc.la} +CPPFLAGS=$CPPFLAGS $LTDLINCL + +AT_CHECK([$CC $CPPFLAGS $CFLAGS -c main.c], [], [ignore], [ignore]) +AT_CHECK([$LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c module.c], +[], [ignore], [ignore]) +AT_CHECK([$LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -o module.la module.lo ]dnl +[-module -avoid-version -rpath /somewhere], [], [ignore], [ignore]) +AT_CHECK([$LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -o main main.$OBJEXT $LIBLTDL], +[], [ignore], [ignore]) + + +# Several bogus test cases. + +AT_DATA([missing-closing-quote.la], +[[# missing-closing-quote.la - a libtool library file +# Generated by libtool +dlname='module.so +library_names='module.so module.so module.so' +old_library='module.a' +installed=no +shouldnotlink=yes +libdir='/somewhere' +]]) + +AT_DATA([wrong-quotes.la], +[[# wrong-quotes.la - a libtool library file +# Generated by libtool +dlname=module.so +library_names='module.so module.so module.so' +old_library='module.a' +installed=no +shouldnotlink=yes +libdir='/somewhere' +]]) + +AT_DATA([no-dlname.la], +[[# no-dlname.la - a libtool library file +# Generated by libtool +installed=no +shouldnotlink=yes +libdir='/somewhere' +]]) + +AT_DATA([nonexistent-dlname.la], +[[# nonexistent-dlname.la - a libtool library file +# Generated by libtool +dlname='does-not-exist.so' +installed=no +shouldnotlink
Re: ltdl .la file parsing (was: Coverage for libltdl/slist.c and fallout)
On Tue, 8 Dec 2009, Ralf Wildenhues wrote: interfaces. For example, it is already known that the .la file parser is fragile and trivial edits to an .la file will cause the using program to core-dump. The .la file parsing is an external interface so it should get more priority. Good idea. Proposed patch. What other things do you know about that are mishandled? OK to commit? The fix and the tests look quite good and useful to me. I don't currently know of other parsing issues. As long as the application can trust that libltdl will search for .la files in secure locations, then there should not be great concern about corrupted .la files. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: Coverage for libltdl/slist.c and fallout
Hallo Ralf, On 1 Dec 2009, at 06:39, Ralf Wildenhues wrote: * Gary V. Vaughan wrote on Tue, Dec 01, 2009 at 12:02:42AM CET: On 30 Nov 2009, at 16:01, Ralf Wildenhues wrote: * Gary V. Vaughan wrote on Mon, Nov 30, 2009 at 06:29:54PM CET: On 29 Nov 2009, at 16:27, Ralf Wildenhues wrote: - slist_remove should IMVHO return an SList *, because otherwise there is no way to avoid a memory leak. APIs that force memleaks are bad. [[...]] Anyway, if you are proposing that SListCallback functions passed to slist_remove should always return SList *, then that does seem like a worthy addition to me. No. I don't propose that. I only propose it for slist_remove, because that's where otherwise, a memleak is inevitable. That's just what my patch does, by letting slist_remove return an SList *. We are in violent agreement! But rather than patch it the way you propose (which just hides the memory leak you describe from the compiler by casting the non-SList * valued return of SListCallback away), I suggest we enforce it strictly by adding a new SListRemoveCallback that ensures the callback function passed to slist_remove returns an SList *, and use that function pointer typedef for just slist_remove (in addition to changing the prototype of slist_remove to also return that same SList *). But that forces slist users to write two callback functions with the same functionality, if they also want to use it in slist_foreach for example. And they don't gain anything in return for that. I respectfully disagree. As a user of the SList API, IFF you know that you want to use the same callback function in your calls to slist_remove and slist_find, then you can cast the function pointer in your call to slist_find (or vice versa). In the common case where you need slist_find to return a pointer from deep inside the SList struct, and where you also want to use slist_remove to remove an item from the list, then you continue to write two callback functions, where each has its own function pointer typedef, and where each can be typechecked at compile time. Having the API quietly cast away some of the type checking behind the client code's back is a worse solution IMHO. SList is designed to manage two types of lists: (i) things that were specifically designed to be chained, in which case the first field of the structures to be chained has to be 'next': struct notboxed { struct notboxed *next; whatever real data needs to be stored in the element }; [[...]] I don't believe you that it was really designed to do (i). If it were, then there were some code using it as such, which I would like to ask you to submit as testsuite addition, so we can see whether, and *how* it works at all, and the coverage may keep us from breaking it. I don't believe that it works. Just like the sorting functions did not work. It really was. Early versions of slist.c did not have the boxing concept yet, and you had to cast your structs (with next field first) into SList * to use the API. It's possible (though I would be a little surprised) that the casting mode has been broken by the relatively recent additions of boxing... So, there is two choices: remove the API, or add test coverage. Which alternative do you prefer? I don't plan to do either. But, of course, I'd much rather you didn't remove half of the functionality from slist.[ch] just because I don't have time to write unit tests. Cheers, Gary -- Gary V. Vaughan (g...@gnu.org)
Re: Coverage for libltdl/slist.c and fallout
Hi Gary, * Gary V. Vaughan wrote on Sun, Dec 06, 2009 at 05:55:25PM CET: On 1 Dec 2009, at 06:39, Ralf Wildenhues wrote: So, there is two choices: remove the API, or add test coverage. Which alternative do you prefer? I don't plan to do either. But, of course, I'd much rather you didn't remove half of the functionality from slist.[ch] just because I don't have time to write unit tests. I'm fine with whoever writes unit tests. I would even write them, but for that I'd need to understand how it's supposed to work. There is no documentation, no examples, no other code that uses slist this way that I can look at. BTW, so far I haven't removed anything that wouldn't be fixable with a cast or two. More generally, I am really convinced that libltdl quality is the way it is only because authors never really cared to ensure their code really does what it was supposed to do. If we continue to treat testing and coverage as an afterthought, there is little reason to believe that is going to change. So yes, I pretty much think that all code that isn't exercised by the testsuite but could be, does not belong in the tree. If writing tests is too hard for some reason, we need to make it easier to do so. That would then be probably be even more important than any new feature. Cheers, Ralf
Re: Coverage for libltdl/slist.c and fallout
* Gary V. Vaughan wrote on Tue, Dec 01, 2009 at 12:02:42AM CET: On 30 Nov 2009, at 16:01, Ralf Wildenhues wrote: If (i) does work, then likely my patch is broken after all; so I guess I will have to wait with applying. I think that subject to the feedback you have received, the patch improves slist by test coverage and bug fixing, so there's no reason to hold back on applying in my mind. OK, I have squashed in the fix to let the test be in the new testsuite, dropped the NULL patch, and pushed the remaining two. Cheers, Ralf
Re: Coverage for libltdl/slist.c and fallout
Den 2009-11-29 22:27 skrev Ralf Wildenhues: *snip* - In the end I grew really lazy and added the new test to the old testsuite: that seemed the easiest way to integrate and catch all the compilation and include flags from toplevel Makefile.am in order to build and use a separate slist.o object. If this is not ok with you, then please complain loudly. Whispering a complaint, see below... *snip* Test and fix slist.c. * libltdl/libltdl/slist.h: Include stddef.h, for size_t. (slist_remove): Return pointer to SList, not void. * libltdl/slist.c: Include stdlib.h, for malloc and free. (slist_remove): Adjust prototype as above. (slist_sort): Do not loop forever on one-item list. * tests/test-slist.c: New file. This will be the first .c file in the tests directory, is that really desirable at this point? All other .c files for the old testsuite are in subdirs. But adding yet another subdir is not what I want either. bootstrapping takes forever as it is and that's the reason I don't want another directory, I want fewer, and I want the old testsuite to go away ASAP. But that said, I'd rather have my MSVC patches reviewed of course (since you asked me to not fix MSVC issues in the old testsuite I'm not regularly running it anyway, instead usually bootstrapping with reconfdirs='. libltdl'...) * Makefile.am (COMMON_TESTS): Add tests/test-slist. (check_PROGRAMS, tests_test_slist_SOURCES): New variables. Shouldn't this new file end with .test as all other such files in the old testsuite? I also miss this file from the patch. Ahhh, is the testsuite running the produced executable? Does that work when doing a cross build? What about $EXEEXT? Cheers, Peter -- They are in the crowd with the answer before the question. Why do you dislike Jeopardy?
Re: Coverage for libltdl/slist.c and fallout
Hallo Ralf! On 29 Nov 2009, at 16:27, Ralf Wildenhues wrote: A first step at more libltdl coverage: better coverage for the slist API. Excellent, and thank you. Notes: - I did find bugs in slist.c, albeit serious ones only in code not used elsewhere in libltdl. Ick. I think that libltdl might be the last live project still using slist.c, so we can probably call this the master copy from here on in. - slist_foreach and slist_find are redundant, as they have the same semantics. Oh, how come I never noticed that? Maybe we can make slist_find into a macro then... I like that the name of the call makes it obvious what the client code is doing. slist_foreach is also safe for deletion of list items, whereas slist_find is not. - slist_remove should IMVHO return an SList *, because otherwise there is no way to avoid a memory leak. APIs that force memleaks are bad. I don't understand that assertion; where is the memory leak in the following code? /* Return the contents of a `boxed' ITEM, recycling the box itself. */ void * slist_unbox (SList *item) { void *userdata = 0; if (item) { /* Strip the const, because responsibility for this memory passes to the caller on return. */ userdata = (void *) item-userdata; free (item); } return userdata; } ... static SList *loaders = 0; static void * loader_callback (SList *item, void *userdata) { const lt_dlvtable *vtable = (const lt_dlvtable *) item-userdata; const char * name= (const char *) userdata; assert (vtable); return streq (vtable-name, name) ? (void *) item : NULL; } ... return (lt_dlvtable *) slist_unbox ((SList *) slist_remove (loaders, loader_callback, name)); ... I think the boxing/unboxing is a very elegant way to provide for clear separation of concerns where the SList implementation handles the memory for the list chaining wrapper structures (the boxes), where the client code handles just the memory for the data being chained (a list of dlloaders in this case), while also avoiding hand writing the glue code to add the next pointers into structures that will be fed to SList. Anyway, if you are proposing that SListCallback functions passed to slist_remove should always return SList *, then that does seem like a worthy addition to me. In that case, the following is cleaner, and goes some way towards preventing misuse of the API by having the find callback return something from the innards of an element (which is, by the way still a valid use-case for other SListCallback occurences): libltd/libltdl/slist.h: typedef SList * SListRemoveCallback (SList *item, void *userdata); ... LT_SCOPE void * slist_remove(SList **phead, SListRemoveCallback *find, void *matchdata); libltdl/slist.c: SList * slist_remove (SList **phead, SListRemoveCallback *find, void *matchdata) { SList *stale = 0; SList *result = 0; ...rest of slist_remove is untouched... } (Apart from that, I've never really understood the difference between boxed and unboxed stuff, basically you have to have a SList header in order to put something in a list, period. But I didn't want to mangle the code beyond bug fixing really.) SList is designed to manage two types of lists: (i) things that were specifically designed to be chained, in which case the first field of the structures to be chained has to be 'next': struct notboxed { struct notboxed *next; whatever real data needs to be stored in the element }; (ii) things that were not designed to be chained, but that we want to handle lists of in any case. In this case, the SList api allows wrapping each item in an chainable element (boxing the item) and getting the original back out again (unboxing it). SList *open_filehandles = 0; ... FILE *fh = fopen (/home/gary/input, r); open_filehandles = slist_cons (slist_box (fh), open_filehandles); ... fclose (slist_unbox ((SList *) slist_remove (open_filehandles, match_cb, matchdata))); otherwise, we'd need to build our own wrapper struct to contain the chain pointer and the FILE pointer. - In the end I grew really lazy and added the new test to the old testsuite: that seemed the easiest way to integrate and catch all the compilation and include flags from toplevel Makefile.am in order to build and use a separate slist.o object. If this is not ok with you, then please complain loudly. I'm with Peter on phasing out the old testsuite. But I accept that it does offer coverage that AutoTest can't, so maybe we need to find a way to make the old testsuite less painful to bootstrap. In the end, I'd rather have slist coverage in the old testsuite than not at all :) So I have three patches I would like to commit as part of this. The first one adds the test and fixes slist.c, I recommend adding a new callback
Re: Coverage for libltdl/slist.c and fallout
Hi Peter, * Peter Rosin wrote on Mon, Nov 30, 2009 at 09:01:48AM CET: Den 2009-11-29 22:27 skrev Ralf Wildenhues: - In the end I grew really lazy and added the new test to the old testsuite: that seemed the easiest way to integrate and catch all the compilation and include flags from toplevel Makefile.am in order to build and use a separate slist.o object. If this is not ok with you, then please complain loudly. Whispering a complaint, see below... Loud enough. While this: This will be the first .c file in the tests directory, is that really desirable at this point? I don't really care about, and this: All other .c files for the old testsuite are in subdirs. But adding yet another subdir is not what I want either. bootstrapping takes forever as it is and that's the reason I don't want another directory, is unrelated to bootstrap time (we could also use another subdir and still build the stuff from toplevel Makefile.am; the thing that takes lots of bootstrapping time is the multiple configure scripts, which I am not going to add in any case). However, this: I want fewer, and I want the old testsuite to go away ASAP. is laudable, and while this: Shouldn't this new file end with .test as all other such files in the old testsuite? I really don't care about (except that double-dots like foo.test.c are problematic on some systems), your killing point: I also miss this file from the patch. Ahhh, is the testsuite running the produced executable? Does that work when doing a cross build? What about $EXEEXT? is a very good reason to redo the testsuite part of the patch. Thanks, I forgot to think about the cross case. Incremental patch below, to be squashed in with the posted version. Thanks, Ralf * tests/slist.at (SList functionality): New file, new test. * Makefile.am (TESTSUITE_AT): Add tests/slist.at. diff --git a/Makefile.am b/Makefile.am index a1097c7..2b09991 100644 --- a/Makefile.am +++ b/Makefile.am @@ -484,6 +484,7 @@ TESTSUITE_AT= tests/testsuite.at \ tests/lt_dlopen_a.at \ tests/lt_dlopenext.at \ tests/ltdl-api.at \ + tests/slist.at \ tests/need_lib_prefix.at \ tests/standalone.at \ tests/subproject.at \ @@ -672,7 +673,6 @@ COMMON_TESTS = \ tests/sh.test \ tests/suffix.test \ tests/tagtrace.test \ - tests/test-slist \ tests/cdemo-static.test \ tests/cdemo-static-make.test \ tests/cdemo-static-exec.test \ @@ -760,11 +760,6 @@ COMMON_TESTS = \ tests/cdemo-undef-make.test \ tests/cdemo-undef-exec.test -check_PROGRAMS = tests/test-slist -tests_test_slist_SOURCES = \ - tests/test-slist.c \ - libltdl/slist.c - tests/cdemo-undef-exec.log:tests/cdemo-undef-make.log tests/cdemo-undef-make.log:tests/cdemo-undef.log tests/cdemo-undef.log: @ORDER@ tests/cdemo-shared-exec.log diff --git a/tests/test-slist.c b/tests/slist.at similarity index 72% rename from tests/test-slist.c rename to tests/slist.at index 40279a4..3c851c0 100644 --- a/tests/test-slist.c +++ b/tests/slist.at @@ -1,3 +1,30 @@ + +# slist.at -- test slist.c -*- Autotest -*- +# +# Copyright (C) 2009 Free Software Foundation, Inc. +# +# This file is part of GNU Libtool. +# +# GNU Libtool is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of +# the License, or (at your option) any later version. +# +# GNU Libtool is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Libtool; see the file COPYING. If not, a copy +# can be downloaded from http://www.gnu.org/licenses/gpl.html, +# or obtained by writing to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + + +AT_SETUP([SList functionality]) + +AT_DATA([test-slist.c], [[ #include config.h #include stdlib.h #include string.h @@ -128,3 +155,15 @@ int main () assert (slist_delete (list, boxed_delete) == NULL); return 0; } +]]) + +CPPFLAGS=-I$top_srcdir/libltdl -I$top_srcdir/libltdl/libltdl -I$abs_top_builddir +AT_CHECK([$CC $CPPFLAGS $CFLAGS -c test-slist.c], +[], [ignore], [ignore]) +AT_CHECK([$CC $CPPFLAGS $CFLAGS -c $top_srcdir/libltdl/slist.c], +[], [ignore], [ignore]) +AT_CHECK([$CC $CFLAGS $LDFLAGS -o test-slist test-slist.$OBJEXT slist.$OBJEXT], +[], [ignore], [ignore]) +LT_AT_EXEC_CHECK([./test-slist], [], [ignore], [ignore]) + +AT_CLEANUP
Re: Coverage for libltdl/slist.c and fallout
Hi Gary, * Gary V. Vaughan wrote on Mon, Nov 30, 2009 at 06:29:54PM CET: On 29 Nov 2009, at 16:27, Ralf Wildenhues wrote: - slist_remove should IMVHO return an SList *, because otherwise there is no way to avoid a memory leak. APIs that force memleaks are bad. I don't understand that assertion; where is the memory leak in the following code? There is none. In your example, the callback, thus also slist_remove, pass back an SList* already, which is good. But the API also allows the callback to return a pointer to the userdata only (at least that's how I understand it). That would still work alright with slist_find, but with slist_remove, it necessarily causes a memory leak of the SList structure pertaining to the data returned. That's the case where the user code forgets to use slist_unbox, namely because the item it gets is already unboxed, and the box already leaked. I think the boxing/unboxing is a very elegant way to provide for clear separation of concerns where the SList implementation handles the memory for the list chaining wrapper structures (the boxes), where the client code handles just the memory for the data being chained (a list of dlloaders in this case), while also avoiding hand writing the glue code to add the next pointers into structures that will be fed to SList. I'm not sure I agree that this is elegant, but maybe I just misunderstand the code. But all of this is so unimportant to the libltdl code quality that I prefer we just keep the code we have, and *not* start introducing more new code with more new bugs here. Anyway, if you are proposing that SListCallback functions passed to slist_remove should always return SList *, then that does seem like a worthy addition to me. No. I don't propose that. I only propose it for slist_remove, because that's where otherwise, a memleak is inevitable. That's just what my patch does, by letting slist_remove return an SList *. (Apart from that, I've never really understood the difference between boxed and unboxed stuff, basically you have to have a SList header in order to put something in a list, period. But I didn't want to mangle the code beyond bug fixing really.) SList is designed to manage two types of lists: (i) things that were specifically designed to be chained, in which case the first field of the structures to be chained has to be 'next': struct notboxed { struct notboxed *next; whatever real data needs to be stored in the element }; (ii) things that were not designed to be chained, but that we want to handle lists of in any case. In this case, the SList api allows wrapping each item in an chainable element (boxing the item) and getting the original back out again (unboxing it). SList *open_filehandles = 0; ... FILE *fh = fopen (/home/gary/input, r); open_filehandles = slist_cons (slist_box (fh), open_filehandles); ... fclose (slist_unbox ((SList *) slist_remove (open_filehandles, match_cb, matchdata))); otherwise, we'd need to build our own wrapper struct to contain the chain pointer and the FILE pointer. I don't believe you that it was really designed to do (i). If it were, then there were some code using it as such, which I would like to ask you to submit as testsuite addition, so we can see whether, and *how* it works at all, and the coverage may keep us from breaking it. I don't believe that it works. Just like the sorting functions did not work. If (i) does work, then likely my patch is broken after all; so I guess I will have to wait with applying. So I have three patches I would like to commit as part of this. The first one adds the test and fixes slist.c, I recommend adding a new callback with SList* return type as I said above. the second one is only stylistic and uses NULL instead of 0 throughout slist.c, IIRC, it was trying to build libltdl with some old C++ compiler (maybe Solaris 7 or so, but I don't recall clearly) which complained about the use of NULL which drove me to using '0' in slist.c. I googled for NULL vs 0, and it seems there it has become a religious issue. Oh well. I'm not attached to the NULL patch. I'm not entirely comfortable with this change while we tout the feature of being able to build libltdl with a C++ compiler. I'd like to see that C++ builds will still work correctly with explicit NULLs before we settle on this patch. But, if you're keen to commit but don't have time to check, then I'll probably trip over it at some point for as long as we have the C++ build requirement in HACKING. Well I built it with g++ 4.2.4. and the third one changes our public APIs lt_dlloader_remove and lt_dlloader_find to accept `const char *' arguments instead of `char *': that's what fits the purpose best, and what we document in the manual. I'm not quite
Re: Coverage for libltdl/slist.c and fallout
Hallo Ralf, On 30 Nov 2009, at 16:01, Ralf Wildenhues wrote: * Gary V. Vaughan wrote on Mon, Nov 30, 2009 at 06:29:54PM CET: On 29 Nov 2009, at 16:27, Ralf Wildenhues wrote: - slist_remove should IMVHO return an SList *, because otherwise there is no way to avoid a memory leak. APIs that force memleaks are bad. [[...]] Anyway, if you are proposing that SListCallback functions passed to slist_remove should always return SList *, then that does seem like a worthy addition to me. No. I don't propose that. I only propose it for slist_remove, because that's where otherwise, a memleak is inevitable. That's just what my patch does, by letting slist_remove return an SList *. We are in violent agreement! But rather than patch it the way you propose (which just hides the memory leak you describe from the compiler by casting the non-SList * valued return of SListCallback away), I suggest we enforce it strictly by adding a new SListRemoveCallback that ensures the callback function passed to slist_remove returns an SList *, and use that function pointer typedef for just slist_remove (in addition to changing the prototype of slist_remove to also return that same SList *). SList is designed to manage two types of lists: (i) things that were specifically designed to be chained, in which case the first field of the structures to be chained has to be 'next': struct notboxed { struct notboxed *next; whatever real data needs to be stored in the element }; [[...]] I don't believe you that it was really designed to do (i). If it were, then there were some code using it as such, which I would like to ask you to submit as testsuite addition, so we can see whether, and *how* it works at all, and the coverage may keep us from breaking it. I don't believe that it works. Just like the sorting functions did not work. It really was. Early versions of slist.c did not have the boxing concept yet, and you had to cast your structs (with next field first) into SList * to use the API. It's possible (though I would be a little surprised) that the casting mode has been broken by the relatively recent additions of boxing... If (i) does work, then likely my patch is broken after all; so I guess I will have to wait with applying. I think that subject to the feedback you have received, the patch improves slist by test coverage and bug fixing, so there's no reason to hold back on applying in my mind. Cheers, Gary -- Gary V. Vaughan (g...@gnu.org)
Re: Backport of libltdl changes to branch-1-5
Hi Peter, * Peter O'Gorman wrote on Mon, Nov 16, 2009 at 05:06:26PM CET: If you happen to be stuck using an older libltdl for some reason, the attached untested patch should give you the same changes in behavior as the badly numbered 2.2.6b release. If this passes the branch-1-5 testsuite for you, then feel free to commit it to that branch. Thanks, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Backport of libltdl changes to branch-1-5
If you happen to be stuck using an older libltdl for some reason, the attached untested patch should give you the same changes in behavior as the badly numbered 2.2.6b release. Peter -- Peter O'Gorman http://pogma.com diff -urN libtool-1.5.26.orig/libltdl/ltdl.c libtool-1.5.26/libltdl/ltdl.c --- libtool-1.5.26.orig/libltdl/ltdl.c 2007-11-15 13:36:41.0 -0600 +++ libtool-1.5.26/libltdl/ltdl.c 2009-11-15 21:13:37.0 -0600 @@ -2192,7 +2192,8 @@ static int try_dlopen LT_PARAMS((lt_dlhandle *handle, const char *filename)); static int tryall_dlopen LT_PARAMS((lt_dlhandle *handle, - const char *filename)); + const char *filename, + const char * useloader)); static int unload_deplibs LT_PARAMS((lt_dlhandle handle)); static int lt_argz_insert LT_PARAMS((char **pargz, size_t *pargz_len, @@ -2390,9 +2391,10 @@ } static int -tryall_dlopen (handle, filename) +tryall_dlopen (handle, filename, useloader) lt_dlhandle *handle; const char *filename; + const char *useloader; { lt_dlhandle cur; lt_dlloader *loader; @@ -2459,6 +2461,11 @@ while (loader) { + if (useloader strcmp(loader-loader_name, useloader)) + { + loader = loader-next; + continue; + } lt_user_data data = loader-dlloader_data; cur-module = loader-module_open (data, filename); @@ -2528,7 +2535,7 @@ error += tryall_dlopen_module (handle, (const char *) 0, prefix, filename); } - else if (tryall_dlopen (handle, filename) != 0) + else if (tryall_dlopen (handle, filename, NULL) != 0) { ++error; } @@ -2549,7 +2556,7 @@ /* Try to open the old library first; if it was dlpreopened, we want the preopened version of it, even if a dlopenable module is available. */ - if (old_name tryall_dlopen (handle, old_name) == 0) + if (old_name tryall_dlopen (handle, old_name, dlpreload) == 0) { return 0; } @@ -2813,7 +2820,7 @@ /* Try to dlopen the file, but do not continue searching in any case. */ - if (tryall_dlopen (handle, filename) != 0) + if (tryall_dlopen (handle, filename,NULL) != 0) *handle = 0; return 1; @@ -3103,7 +3110,7 @@ /* lt_dlclose()ing yourself is very bad! Disallow it. */ LT_DLSET_FLAG (*phandle, LT_DLRESIDENT_FLAG); - if (tryall_dlopen (newhandle, 0) != 0) + if (tryall_dlopen (newhandle, 0, NULL) != 0) { LT_DLFREE (*phandle); return 1; @@ -3225,7 +3232,7 @@ } #endif } - if (!file) + else { file = fopen (filename, LT_READTEXT_MODE); } @@ -3412,7 +3419,7 @@ #endif ))) { - if (tryall_dlopen (newhandle, filename) != 0) + if (tryall_dlopen (newhandle, filename, NULL) != 0) { newhandle = NULL; } ___ http://lists.gnu.org/mailman/listinfo/libtool
libltdl is inefficient and a security hazard
I posted a bug report with the same subject several days ago to the bug-libtool list (with data) but no discussion ensued so I will open the issue for broader discussion here. I was observing very poor execution performance of the modules-based build of my application under Solaris. It used to execute quite well under Solaris (better than a static build) when using loadable modules. Now using any loadable modules via libltdl makes the minimum program execution time 4X slower, and this is when loading only two modules. Looking at system calls showed massive amounts of 'stat' calls and reloading the same shared libraries many times. While there are several issues to fix, it seems that libltdl has a bug with both performance and security implications. When a libtool module is opened via its .la file, I expect that the .la file will be parsed. The file name given by 'dlname' will be appended to the path name given by 'libdir'. This full path should then be given to the system's dlopen(). Then I expect that the path specified by lt_dlsetsearchpath() (LTDL_LIBRARY_PATH) should be used, while always passing a fully qualified path to dlopen(). If these explicit loads fail, then an unqualified path can be used to load the module and the system can have its way with it. Instead what I see happening is for a module called module.la, first there is an attempt to dlopen module.a and then module.so and the system's shared library search path is used. Lastly, there is (finally) a request to load module.so via the correct full path. Since libltdl also attempts to explicitly load all of the dependency libraries, I see the same wonky things done for all of the libraries that the module depends on (e.g. libc.so). This adds many hundreds of additional slow system calls to program initialization. This is what the libtool documentation says regarding search paths: If libltdl cannot find the library and the file name filename does not have a directory component it will additionally look in the following search paths for the module (in the following order): 1. user-defined search path: This search path can be changed by the program using the functions lt_dlsetsearchpath, lt_dladdsearchdir and lt_dlinsertsearchdir. 2. libltdl's search path: This search path is the value of the environment variable LTDL_LIBRARY_PATH. 3. system library search path: The system dependent library search path (e.g. on Linux it is LD_LIBRARY_PATH). I tried using lt_dlsetsearchpath() to explicitly set the search path to the directory where the module is located, but it had no influence on the searches performed. Bb -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: libltdl is inefficient and a security hazard
Doing some investigation under Linux, I see that libltdl does the same wrong things under Linux except that Linux employs some algorithms which makes doing the wrong things much less expensive than under Solaris. Linux prunes directories in the search path which were found to not exist from later searches. Linux also entirely ignores requests to load libraries which were already loaded. However, Linux does still try to dlopen() a .a archive file. Here is a strace done under Linux showing only the system calls which pass file names (strace -e trace=file command). The modules loaded were null.la and xc.la: execve(/usr/local/bin/gm, [gm, convert, -size, 1x1, xc:white, null:], [/* 50 vars */]) = 0 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) access(/etc/ld.so.preload, R_OK) = -1 ENOENT (No such file or directory) open(/etc/ld.so.cache, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/local/lib/libGraphicsMagick.so.3, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/liblcms.so.1, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libfreetype.so.6, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libXext.so.6, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libSM.so.6, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libICE.so.6, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libX11.so.6, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/lib/libbz2.so.1.0, O_RDONLY)= 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libz.so.1, O_RDONLY)= 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/lib/libm.so.6, O_RDONLY)= 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libgomp.so.1, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/lib/libpthread.so.0, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/lib/libdl.so.2, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/lib/libc.so.6, O_RDONLY)= 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libXau.so.6, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libxcb-xlib.so.0, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libxcb.so.1, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/lib/librt.so.1, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/usr/lib/libXdmcp.so.6, O_RDONLY) = 3 open(/usr/lib/locale/locale-archive, O_RDONLY) = 3 open(/proc/meminfo, O_RDONLY) = 3 open(/proc/stat, O_RDONLY)= 3 open(/usr/lib/gconv/gconv-modules.cache, O_RDONLY) = 3 open(/usr/local/share/GraphicsMagick-1.4/config/modules.mgk, O_RDONLY) = 3 getcwd(/..., 2052)= 2 access(/usr/local/lib/GraphicsMagick-1.4/modules-Q16/coders/null.la, R_OK) = 0 open(/usr/local/lib/GraphicsMagick-1.4/modules-Q16/coders/null.la, O_RDONLY) = 3 open(/etc/ld.so.cache, O_RDONLY) = 3 access(/etc/ld.so.nohwcap, F_OK) = -1 ENOENT (No such file or directory) open(/lib/tls/x86_64/null.a, O_RDONLY) = -1 ENOENT (No such file or directory) stat(/lib/tls/x86_64, 0x7fffbac4e380) = -1 ENOENT (No such file or directory) open(/lib/tls/null.a, O_RDONLY) = -1 ENOENT (No such file or directory) stat(/lib/tls, 0x7fffbac4e380)= -1 ENOENT (No such file or directory) open(/lib/x86_64/null.a, O_RDONLY)= -1 ENOENT (No such file or directory) stat(/lib/x86_64, 0x7fffbac4e380) = -1 ENOENT (No such file or directory) open(/lib/null.a, O_RDONLY) = -1 ENOENT (No such file or directory) stat(/lib, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 open(/usr/lib/tls/x86_64/null.a, O_RDONLY) = -1 ENOENT (No such file or directory) stat(/usr/lib/tls/x86_64, 0x7fffbac4e380) = -1 ENOENT (No such file or directory) open(/usr/lib/tls/null.a, O_RDONLY) = -1 ENOENT (No such file or directory) stat(/usr/lib/tls, 0x7fffbac4e380)= -1 ENOENT (No such file or directory) open(/usr/lib/x86_64/null.a, O_RDONLY) = -1 ENOENT (No such file or directory) stat(/usr/lib/x86_64, 0x7fffbac4e380) = -1 ENOENT (No such file or directory) open(/usr/lib/null.a, O_RDONLY) = -1 ENOENT (No such file or directory) stat(/usr/lib, {st_mode
Re: libltdl is inefficient and a security hazard
Bob Friesenhahn wrote: Doing some investigation under Linux, I see that libltdl does the same wrong things under Linux except that Linux employs some algorithms which makes doing the wrong things much less expensive than under Solaris. Linux prunes directories in the search path which were found to not exist from later searches. Linux also entirely ignores requests to load libraries which were already loaded. However, Linux does still try to dlopen() a .a archive file. Yeah, I recall noticing all this in the past. That's why I usually delete .la files after build time and make sure they never get installed. On platforms where the dependent shared libraries are already listed inside a shared library's dynamic loader section, all of this .la stuff is just wasted effort (and time). So far I only see .la files being useful when statically linking, since the .a archives don't include their dependency info. It would have been smarter for libtool to just store whatever it wanted to record as a stub file inside the .a, IMO. Also, instead of all of that relinking cruft, should have just provided a libbfd-based tool that rewrites the linkage paths as needed. I've seen too many times where the originally built library works perfectly, but the relinked file that got installed is broken because the relink info didn't include various settings that were in the environment when the original library was built. Basically, relinking is doing a lousy job of second-guessing, and on SVR4/ELF-based platforms it's not even needed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: libltdl is inefficient and a security hazard
Here is what happens when an uninstalled module gets loaded under FreeBSD: access(/home/bfriesen/build/GraphicsMagick-16-modules/coders/xc.la,4) = 0 (0x0) open(/home/bfriesen/build/GraphicsMagick-16-modules/coders/xc.la,O_RDONLY,0666) = 3 (0x3) access(/home/bfriesen/build/GraphicsMagick-16-modules/magick/.libs/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/xc.a,0) ERR#2 'No such file or directory' access(/lib/xc.a,0)ERR#2 'No such file or directory' access(/usr/lib/xc.a,0)ERR#2 'No such file or directory' access(/usr/lib/compat/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/compat/pkg/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/compat/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/evolution/2.24/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/gcc-4.2.5/xc.a,0)ERR#2 'No such file or directory' access(/usr/local/lib/gegl-0.0/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/graphviz/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/kde3/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/mysql/xc.a,0)ERR#2 'No such file or directory' access(/usr/local/lib/nss/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/pth/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/qt4/xc.a,0) ERR#2 'No such file or directory' access(/usr/local/lib/zsh/xc.a,0) ERR#2 'No such file or directory' access(/lib/xc.a,0)ERR#2 'No such file or directory' access(/usr/lib/xc.a,0)ERR#2 'No such file or directory' open(/home/bfriesen/build/GraphicsMagick-16-modules/coders/.libs/xc.so,O_RDONLY,010) = 3 (0x3) Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Problem with libltdl on Solaris
Hello Will, * Will Mason wrote on Mon, May 11, 2009 at 06:47:58AM CEST: I've been using a project with autoconf, automake and libtool for some time. Today I added libltdl to my code. I ran libtoolize --ltdl --copy. My configure.ac has this: LT_CONFIG_LTDL_DIR([libltdl]) LT_INIT([dlopen]) LT_LANG([C++]) LTDL_INIT([recursive]) When I build on Mac OS X, everything is great. It all works perfectly. When I build on Solaris 10 with CC, my executables fail to link because of this: Undefined first referenced symbol in file lt__argz_insert ./.libs/libforxxq.so lt__argz_append ./.libs/libforxxq.so lt__argz_create_sep ./.libs/libforxxq.so lt__argz_stringify ./.libs/libforxxq.so lt__argz_next ./.libs/libforxxq.so I can see no way to convince libtool that it should actually compile the argz stuff. In the Makefile I see ltdl_LIBOBJS = argz.o, but I can't see how to tell the system to actually target that file. Has anyone seen this problem before? Can anyone help me sort this out? Which Autoconf and Automake versions do you use? Thanks, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Problem with libltdl on Solaris
On Mon, May 11, 2009 at 1:10 AM, Ralf Wildenhues ralf.wildenh...@gmx.dewrote: Hello Will, * Will Mason wrote on Mon, May 11, 2009 at 06:47:58AM CEST: I've been using a project with autoconf, automake and libtool for some time. Today I added libltdl to my code. I ran libtoolize --ltdl --copy. My configure.ac has this: LT_CONFIG_LTDL_DIR([libltdl]) LT_INIT([dlopen]) LT_LANG([C++]) LTDL_INIT([recursive]) When I build on Mac OS X, everything is great. It all works perfectly. When I build on Solaris 10 with CC, my executables fail to link because of this: Undefined first referenced symbol in file lt__argz_insert ./.libs/libforxxq.so lt__argz_append ./.libs/libforxxq.so lt__argz_create_sep ./.libs/libforxxq.so lt__argz_stringify ./.libs/libforxxq.so lt__argz_next ./.libs/libforxxq.so I can see no way to convince libtool that it should actually compile the argz stuff. In the Makefile I see ltdl_LIBOBJS = argz.o, but I can't see how to tell the system to actually target that file. Has anyone seen this problem before? Can anyone help me sort this out? Which Autoconf and Automake versions do you use? Thanks, Ralf Hi, Thanks for the reply. I'm using autoconf 2.63 and automake 1.10.1. Thanks, Will ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Problem with libltdl on Solaris
Hi Will, * Will Mason wrote on Mon, May 11, 2009 at 05:07:36PM CEST: On Mon, May 11, 2009 at 1:10 AM, Ralf Wildenhues ralf.wildenh...@gmx.dewrote: * Will Mason wrote on Mon, May 11, 2009 at 06:47:58AM CEST: I've been using a project with autoconf, automake and libtool for some time. Today I added libltdl to my code. I ran libtoolize --ltdl --copy. My configure.ac has this: LT_CONFIG_LTDL_DIR([libltdl]) LT_INIT([dlopen]) LT_LANG([C++]) LTDL_INIT([recursive]) When I build on Mac OS X, everything is great. It all works perfectly. When I build on Solaris 10 with CC, my executables fail to link because of this: Undefined first referenced symbol in file lt__argz_insert ./.libs/libforxxq.so I can see no way to convince libtool that it should actually compile the argz stuff. In the Makefile I see ltdl_LIBOBJS = argz.o, but I can't see how to tell the system to actually target that file. Has anyone seen this problem before? Can anyone help me sort this out? Which Autoconf and Automake versions do you use? Thanks for the reply. I'm using autoconf 2.63 and automake 1.10.1. Hmm. libltdl/Makefile should contain all of these lines: ltdl_LTLIBOBJS = argz.lo libltdl_la_LIBADD = $(ltdl_LTLIBOBJS) libltdl_la_DEPENDENCIES = $(LT_DLLOADERS) $(ltdl_LTLIBOBJS) libltdlc_la_LIBADD = $(libltdl_la_LIBADD) libltdlc_la_DEPENDENCIES = $(libltdl_la_DEPENDENCIES) libltdl.la: $(libltdl_la_OBJECTS) $(libltdl_la_DEPENDENCIES) $(libltdl_la_LINK) $(am_libltdl_la_rpath) $(libltdl_la_OBJECTS) $(libltdl_la_LIBADD) $(LIBS) libltdlc.la: $(libltdlc_la_OBJECTS) $(libltdlc_la_DEPENDENCIES) $(libltdlc_la_LINK) $(am_libltdlc_la_rpath) $(libltdlc_la_OBJECTS) $(libltdlc_la_LIBADD) $(LIBS) What's missing in your build tree? Also, I think you haven't stated yet which Libtool version you're using. Libtool-2.2.6 has this in the NEWS file: * Bug fixes: - Fix 2.2 regression that caused argz symbols to be exported from libltdl unrenamed on systems that do not have working argz. so if you're using something older, please update to 2.2.6a. Thanks, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Problem with libltdl on Solaris
On Mon, May 11, 2009 at 4:33 PM, Ralf Wildenhues ralf.wildenh...@gmx.dewrote: Hi Will, * Will Mason wrote on Mon, May 11, 2009 at 05:07:36PM CEST: On Mon, May 11, 2009 at 1:10 AM, Ralf Wildenhues ralf.wildenh...@gmx.de wrote: * Will Mason wrote on Mon, May 11, 2009 at 06:47:58AM CEST: I've been using a project with autoconf, automake and libtool for some time. Today I added libltdl to my code. I ran libtoolize --ltdl --copy. My configure.ac has this: LT_CONFIG_LTDL_DIR([libltdl]) LT_INIT([dlopen]) LT_LANG([C++]) LTDL_INIT([recursive]) When I build on Mac OS X, everything is great. It all works perfectly. When I build on Solaris 10 with CC, my executables fail to link because of this: Undefined first referenced symbol in file lt__argz_insert ./.libs/libforxxq.so I can see no way to convince libtool that it should actually compile the argz stuff. In the Makefile I see ltdl_LIBOBJS = argz.o, but I can't see how to tell the system to actually target that file. Has anyone seen this problem before? Can anyone help me sort this out? Which Autoconf and Automake versions do you use? Thanks for the reply. I'm using autoconf 2.63 and automake 1.10.1. Hmm. libltdl/Makefile should contain all of these lines: ltdl_LTLIBOBJS = argz.lo libltdl_la_LIBADD = $(ltdl_LTLIBOBJS) libltdl_la_DEPENDENCIES = $(LT_DLLOADERS) $(ltdl_LTLIBOBJS) libltdlc_la_LIBADD = $(libltdl_la_LIBADD) libltdlc_la_DEPENDENCIES = $(libltdl_la_DEPENDENCIES) libltdl.la: $(libltdl_la_OBJECTS) $(libltdl_la_DEPENDENCIES) $(libltdl_la_LINK) $(am_libltdl_la_rpath) $(libltdl_la_OBJECTS) $(libltdl_la_LIBADD) $(LIBS) libltdlc.la: $(libltdlc_la_OBJECTS) $(libltdlc_la_DEPENDENCIES) $(libltdlc_la_LINK) $(am_libltdlc_la_rpath) $(libltdlc_la_OBJECTS) $(libltdlc_la_LIBADD) $(LIBS) What's missing in your build tree? Also, I think you haven't stated yet which Libtool version you're using. Libtool-2.2.6 has this in the NEWS file: * Bug fixes: - Fix 2.2 regression that caused argz symbols to be exported from libltdl unrenamed on systems that do not have working argz. so if you're using something older, please update to 2.2.6a. Thanks, Ralf Thanks a ton for the help. You solved the problem. I was using libtool 2.2.4, and as soon as I upgraded to 2.2.6a, the problem went away. I really appreciate your taking the time to have a look at this issue, Will ___ http://lists.gnu.org/mailman/listinfo/libtool
Problem with libltdl on Solaris
Hello, I've been using a project with autoconf, automake and libtool for some time. Today I added libltdl to my code. I ran libtoolize --ltdl --copy. My configure.ac has this: LT_CONFIG_LTDL_DIR([libltdl]) LT_INIT([dlopen]) LT_LANG([C++]) LTDL_INIT([recursive]) And my Makefile.am has this: SUBDIRS = libltdl generator resources And this: libforxxq_la_LIBADD = $(LIBLTDL) And this: AM_CPPFLAGS = $(INCLTDL) When I build on Mac OS X, everything is great. It all works perfectly. When I build on Solaris 10 with CC, my executables fail to link because of this: Undefined first referenced symbol in file lt__argz_insert ./.libs/libforxxq.so lt__argz_append ./.libs/libforxxq.so lt__argz_create_sep ./.libs/libforxxq.so lt__argz_stringify ./.libs/libforxxq.so lt__argz_next ./.libs/libforxxq.so I can see no way to convince libtool that it should actually compile the argz stuff. In the Makefile I see ltdl_LIBOBJS = argz.o, but I can't see how to tell the system to actually target that file. Has anyone seen this problem before? Can anyone help me sort this out? Thanks a lot for any help, Will ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Building a Libtool 2 package with libltdl from 1.5
Hello, [EMAIL PROTECTED] (Ludovic Courtès) writes: The question now is: will people be able to build tarballs that come with Libtool 2.2 with ltdl.h from Libtool 1.5? The answer is actually no, precisely because of this additional line in ltdl.h (missing from 1.5) [*]: #define lt_preloaded_symbols lt__PROGRAM__LTX_preloaded_symbols Thanks, Ludo'. [*] As explained in http://lists.gnu.org/archive/html/guile-devel/2008-11/msg00073.html . ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Building a Libtool 2 package with libltdl from 1.5
Hello, Ralf Wildenhues [EMAIL PROTECTED] writes: * Ludovic Courtès wrote on Tue, Nov 25, 2008 at 09:38:53AM CET: The Guile project is considering moving from Libtool 1.5 to 2.2. We noticed subtle incompatibilities when people tried to compile Guile tarballs (which use Libtool 1.5 up to now) with ltdl.h from Libtool 2.2 [0]. The question now is: will people be able to build tarballs that come with Libtool 2.2 with ltdl.h from Libtool 1.5? [0] http://thread.gmane.org/gmane.lisp.guile.devel/7876 Please describe those limitations, this URL is not accessible here ATM. See this thread: http://lists.gnu.org/archive/html/guile-devel/2008-11/msg00067.html . Could be the same as those described here? http://lists.gnu.org/archive/html/bug-libtool/2008-11/msg7.html Like the second problem listed in this message, i.e., the fact that libtool 2.2 [renames] the symbol table from lt_preloaded_symbols to lt__PROGRAM__LTX_preloaded_symbols. Thanks, Ludo'. ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Building a Libtool 2 package with libltdl from 1.5
Something that may be of interest: http://bugs.gentoo.org/show_bug.cgi?id=212723 especially my comment #8. For non-Gentoo users: eautoreconf (if I were to simplify it) is something like this: - first call aclocal (with additional -I options, if needed) - then check for libtool use and call 'libtoolize -c -f -i', if required (of course, -i is just for libtool 2) - if libtoolize was called, call aclocal again - call autoconf (and autoheader, if needed) - finally, call 'automake -c -f' AC_CONFIG_MACRO_DIR conflict (comment #15) is a well known libtool 2 change. So, guile does build correctly with libtool 2 with only a trivial change (that AC_CONFIG_MACRO_DIR thing). ___ http://lists.gnu.org/mailman/listinfo/libtool
Building a Libtool 2 package with libltdl from 1.5
Hello, The Guile project is considering moving from Libtool 1.5 to 2.2. We noticed subtle incompatibilities when people tried to compile Guile tarballs (which use Libtool 1.5 up to now) with ltdl.h from Libtool 2.2 [0]. The question now is: will people be able to build tarballs that come with Libtool 2.2 with ltdl.h from Libtool 1.5? Thanks, Ludovic. [0] http://thread.gmane.org/gmane.lisp.guile.devel/7876 ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Building a Libtool 2 package with libltdl from 1.5
On Tue, 25 Nov 2008, Ludovic Courtès wrote: The question now is: will people be able to build tarballs that come with Libtool 2.2 with ltdl.h from Libtool 1.5? I don't think there was ever any intention for libtool 2.2.X to work with libltdl from libtool 1.5.X. If libltdl is not bundled with the package, then it should be possible to use an already installed libltdl from libtool 1.5.X. I am suspicious that the way libtool 2.2.X bundles libltdl may sometimes fail if a libltdl from 1.5.X is already installed and the user does not request to use the bundled libltdl. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Building a Libtool 2 package with libltdl from 1.5
On Tue, 25 Nov 2008, Ludovic Courtès wrote: I don't think there was ever any intention for libtool 2.2.X to work with libltdl from libtool 1.5.X. If libltdl is not bundled with the package, then it should be possible to use an already installed libltdl from libtool 1.5.X. What do you mean exactly by should? That it hasn't been tested, right? It does actually work because then libltdl is just a library like any other library. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Building a Libtool 2 package with libltdl from 1.5
Hello Ludovic, * Ludovic Courtès wrote on Tue, Nov 25, 2008 at 09:38:53AM CET: The Guile project is considering moving from Libtool 1.5 to 2.2. We noticed subtle incompatibilities when people tried to compile Guile tarballs (which use Libtool 1.5 up to now) with ltdl.h from Libtool 2.2 [0]. The question now is: will people be able to build tarballs that come with Libtool 2.2 with ltdl.h from Libtool 1.5? [0] http://thread.gmane.org/gmane.lisp.guile.devel/7876 Please describe those limitations, this URL is not accessible here ATM. Could be the same as those described here? http://lists.gnu.org/archive/html/bug-libtool/2008-11/msg7.html Cheers, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Two small libltdl patches.
Hi Peter, * Peter O'Gorman wrote on Sat, Aug 23, 2008 at 11:59:00PM CEST: On systems without argz we were exporting unmangled symbols argz_count and argz_add from libltdl. Not really a good idea. Rather than mangle the names, since we do not use either function, they were removed. Subject: [PATCH] Remove unnecessary global argz functions. * libltdl/argz.c (argz_add,argz_count): Remove. * Peter O'Gorman wrote on Mon, Aug 25, 2008 at 04:43:57AM CEST: I just went back and re-read the argz threads on bug-libtool. The two functions were added back in February, the June thread then decided that libtool could keep its own argz and only have the functions it needs, while gnulib's could evolve along glibc's. ACK. Now that I look at the patch again, I see that I forgot to remove the functions from argz_.h :) So, with the functions also removed from the header, ok? Yes, certainly. I think a NEWS entry would be good, too, as well as a test so we don't regress again. What do you think of this? You can squash it into your patch when you commit. The test doesn't work on many systems (only those that need argz.c, and only those without leading underscore in symbol names) and skips resp. doesn't expose a failure elsewhere, but I think that should be sufficient. Thanks, Ralf * NEWS: Update. * tests/ltdl-api.at: New file. * Makefile.am: Adjust. diff --git a/NEWS b/NEWS index e0d1062..c85f741 100644 --- a/NEWS +++ b/NEWS @@ -21,6 +21,9 @@ New in 2.2.??: 2008-08-??: git version 2.2.5a, Libtool team: - Several testsuite issues have been fixed, thanks to user feedback. + - Fix 2.2 regression that caused argz symbols to be exported from +libltdl unrenamed on systems that do not have working argz. + New in 2.2.4: 2008-05-04: git version 2.2.3a, Libtool team: * New features: diff --git a/Makefile.am b/Makefile.am index 0085c7f..1417e84 100644 --- a/Makefile.am +++ b/Makefile.am @@ -454,6 +454,7 @@ TESTSUITE_AT= tests/testsuite.at \ tests/am-subdir.at \ tests/lt_dlexit.at \ tests/lt_dladvise.at \ + tests/ltdl-api.at \ tests/need_lib_prefix.at \ tests/standalone.at \ tests/subproject.at \ diff --git a/tests/ltdl-api.at b/tests/ltdl-api.at new file mode 100644 index 000..b811ae3 --- /dev/null +++ b/tests/ltdl-api.at @@ -0,0 +1,36 @@ +# ltdl-api.at -- test libltdl API-*- Autotest -*- +# +# Copyright (C) 2008 Free Software Foundation, Inc. +# Written by Ralf Wildenhues, 2008. +# +# This file is part of GNU Libtool. +# +# GNU Libtool is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of +# the License, or (at your option) any later version. +# +# GNU Libtool is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Libtool; see the file COPYING. If not, a copy +# can be downloaded from http://www.gnu.org/licenses/gpl.html, +# or obtained by writing to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + + +AT_SETUP([ltdl API]) +AT_KEYWORDS([libltdl]) + +# Ensure that no symbols from argz are unrenamed. +eval `$LIBTOOL --config | $EGREP '^(NM|global_symbol_pipe|objext)='` +argz_o=$abs_top_builddir/libltdl/argz.$objext +AT_CHECK([test -f $argz_o || exit 77]) +AT_CHECK([eval $NM \\$argz_o\ | $global_symbol_pipe], +[], [stdout], [ignore]) +AT_CHECK([grep ^T argz_ stdout], [1]) + +AT_CLEANUP
Re: Two small libltdl patches.
Hi Peter, * Peter O'Gorman wrote on Sat, Aug 23, 2008 at 11:59:00PM CEST: Subject: [PATCH] Allow for extensions other than .a for preloaded modules. * libltdl/m4/ltdl.m4 (_LTDL_SETUP): Define LT_LIBEXT. * libltdl/ltdl.c (lt_dladvise_preload): Use it. Reported by Ralf Wildenhues. OK, thanks. Does this need a NEWS entry? I'm not sure. Cheers, Ralf
Re: Two small libltdl patches.
Ralf Wildenhues wrote: Yes, certainly. I think a NEWS entry would be good, too, as well as a test so we don't regress again. What do you think of this? You can squash it into your patch when you commit. Thanks. The test doesn't work on many systems (only those that need argz.c, and only those without leading underscore in symbol names) and skips resp. doesn't expose a failure elsewhere, but I think that should be sufficient. I'd like this to fail for me on darwin (more likely to notice it), so I will probably add: +AT_CHECK([eval $NM \\$argz_o\ | $global_symbol_pipe], + [], [stdout], [ignore]) +AT_CHECK([grep ^T argz_ stdout], [1]) +AT_CHECK([grep ^T _argz_ stdout], [1]) + +AT_CLEANUP This should be fine, because all global symbols exported from argz.o must be lt__ prefixed. Will commit with these changes later today. Peter -- Peter O'Gorman http://pogma.com
Re: Two small libltdl patches.
* Peter O'Gorman wrote on Tue, Aug 26, 2008 at 06:24:11PM CEST: Ralf Wildenhues wrote: I'd like this to fail for me on darwin (more likely to notice it), so I will probably add: +AT_CHECK([eval $NM \\$argz_o\ | $global_symbol_pipe], +[], [stdout], [ignore]) +AT_CHECK([grep ^T argz_ stdout], [1]) +AT_CHECK([grep ^T _argz_ stdout], [1]) Of course. Good idea. Thanks, Ralf
Re: Two small libltdl patches.
Peter O'Gorman wrote: On systems without argz we were exporting unmangled symbols argz_count and argz_add from libltdl. Not really a good idea. Rather than mangle the names, since we do not use either function, they were removed. But wasn't there a thread back in June where we explicitly added a bunch of argz functions, so that (a) we would be using the same file as gnulib, and (b) that file would have a complete argz implementation? Doesn't your patch fork our copy again? (Or have I missed some intervening decisions/development)? -- Chuck
Re: Two small libltdl patches.
On Sun, 24 Aug 2008, Charles Wilson wrote: But wasn't there a thread back in June where we explicitly added a bunch of argz functions, so that (a) we would be using the same file as gnulib, and (b) that file would have a complete argz implementation? I recall that it was decided that it was not necessary for the implementation to be identical to gnulib and that the code should not need to be updated unless it is found to be broken. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: Two small libltdl patches.
Peter O'Gorman wrote: So, with the functions also removed from the header, ok? Fine by me -- but I can't give official approval. g -- Chuck
Two small libltdl patches.
Hi, On systems without argz we were exporting unmangled symbols argz_count and argz_add from libltdl. Not really a good idea. Rather than mangle the names, since we do not use either function, they were removed. And as Ralf reported previously, preloaded modules can have the .lib extension as well as .a. Does not fail any unexpected tests on darwin or linux with these patches. I'm thinking of just pushing these as obvious Oh what the heck, not like I am going to have time to make a release in the next couple of days anyway. Ok to commit? Peter -- Peter O'Gorman http://pogma.com From 1fa70a57ef579ae717c504346a1aad13a03b30cc Mon Sep 17 00:00:00 2001 From: Peter O'Gorman [EMAIL PROTECTED] Date: Sat, 23 Aug 2008 16:28:25 -0500 Subject: [PATCH] Remove unnecessary global argz functions. * libltdl/argz.c (argz_add,argz_count): Remove. Reported by Ralf Wildenhues [EMAIL PROTECTED]. --- ChangeLog |6 ++ libltdl/argz.c | 28 2 files changed, 6 insertions(+), 28 deletions(-) diff --git a/ChangeLog b/ChangeLog index 13cc2af..567fbfd 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2008-08-23 Peter O'Gorman [EMAIL PROTECTED] + + Remove unnecessary global argz functions. + * libltdl/argz.c (argz_add,argz_count): Remove. + Reported by Ralf Wildenhues [EMAIL PROTECTED]. + 2008-08-22 Ralf Wildenhues [EMAIL PROTECTED] Initial support for Lahey Fortran on GNU/Linux. diff --git a/libltdl/argz.c b/libltdl/argz.c index f31ce17..8567723 100644 --- a/libltdl/argz.c +++ b/libltdl/argz.c @@ -76,14 +76,6 @@ argz_append (char **pargz, size_t *pargz_len, const char *buf, size_t buf_len) } -/* Add a string to the argz vector. */ -error_t -argz_add (char **pargz, size_t *pargz_len, const char *str) -{ - return argz_append (pargz, pargz_len, str, strlen (str) + 1); -} - - error_t argz_create_sep (const char *str, int delim, char **pargz, size_t *pargz_len) { @@ -232,23 +224,3 @@ argz_stringify (char *argz, size_t argz_len, int sep) } } - -/* Count number of elements (null bytes) in argz vector. */ - -size_t -argz_count (const char *argz, size_t argz_len) -{ - size_t count = 0; - - assert ((argz argz_len) || (!argz !argz_len)); - - while (argz_len 0) -{ - size_t part_len = strlen (argz); - argz += part_len + 1; - argz_len -= part_len + 1; - count++; -} - - return count; -} -- 1.5.4.1 From 38a2f1bcab8c64650b096830d1def5c6fb920c85 Mon Sep 17 00:00:00 2001 From: Peter O'Gorman [EMAIL PROTECTED] Date: Sat, 23 Aug 2008 16:31:17 -0500 Subject: [PATCH] Allow for extensions other than .a for preloaded modules. * libltdl/m4/ltdl.m4 (_LTDL_SETUP): Define LT_LIBEXT. * libltdl/ltdl.c (lt_dladvise_preload): Use it. Reported by Ralf Wildenhues. --- ChangeLog |5 + libltdl/ltdl.c | 10 -- libltdl/m4/ltdl.m4 |2 ++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/ChangeLog b/ChangeLog index 567fbfd..3dc46fc 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,10 @@ 2008-08-23 Peter O'Gorman [EMAIL PROTECTED] + Allow for extensions other than .a for preloaded modules. + * libltdl/m4/ltdl.m4 (_LTDL_SETUP): Define LT_LIBEXT. + * libltdl/ltdl.c (lt_dladvise_preload): Use it. + Reported by Ralf Wildenhues. + Remove unnecessary global argz functions. * libltdl/argz.c (argz_add,argz_count): Remove. Reported by Ralf Wildenhues [EMAIL PROTECTED]. diff --git a/libltdl/ltdl.c b/libltdl/ltdl.c index 5a26565..6e19599 100644 --- a/libltdl/ltdl.c +++ b/libltdl/ltdl.c @@ -50,6 +50,10 @@ or obtained by writing to the Free Software Foundation, Inc., # define LT_FILENAME_MAX 1024 #endif +#if !defined(LT_LIBEXT) +# define LT_LIBEXT a +#endif + /* This is the maximum symbol size that won't require malloc/free */ #undef LT_SYMBOL_LENGTH #define LT_SYMBOL_LENGTH 128 @@ -67,6 +71,7 @@ or obtained by writing to the Free Software Foundation, Inc., static const char objdir[] = LT_OBJDIR; static const char archive_ext[] = LT_ARCHIVE_EXT; +static const char libext[] = LT_LIBEXT; #if defined(LT_MODULE_EXT) static const char shlib_ext[] = LT_MODULE_EXT; #endif @@ -1258,7 +1263,8 @@ try_dlopen (lt_dlhandle *phandle, const char *filename, const char *ext, if (vtable) { - archive_name = MALLOC (char, LT_STRLEN (name) + 3); + /* name + . + libext + NULL */ + archive_name = MALLOC (char, LT_STRLEN (name) + LT_STRLEN (libext) + 2); *phandle = (lt_dlhandle) lt__zalloc (sizeof (struct lt__handle)); if ((*phandle == NULL) || (archive_name == NULL)) @@ -1270,7 +1276,7 @@ try_dlopen (lt_dlhandle *phandle, const char *filename, const char *ext, /* Preloaded modules are always named according to their old archive name. */ - sprintf (archive_name, %s.a, name); + sprintf (archive_name, %s.%s, name, libext); if (tryall_dlopen (newhandle, archive_name, advise, vtable) == 0) { diff --git a/libltdl/m4/ltdl.m4 b/libltdl/m4/ltdl.m4 index
Building libltdl on MSVC w/o Cygwin or Mingw32
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm working on a cross-platform game, and for plugin loading, I was hoping to use libltdl, since Win32 DLL loading is a pain to work with. However, it only builds on Windows with mingw or the cygwin system, and this is sort of a deal-breaker, since one of the libraries we require is currently broken on all Windows compilers besides MSVC. Is there some magic incantation that will make libltdl build with MSVC? Something I've overlooked? Please CC me, as I'm not subscribed to the list. ~ C. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkhzESUACgkQeCCY8PC5utCgsgCfTGEqlk3ExhcRjzEDXdFoiT8v lsEAn2xbpg+/XWLzgaE+bwz+QUe6gp4u =vh65 -END PGP SIGNATURE- ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Building libltdl on MSVC w/o Cygwin or Mingw32
Hello Corbin, * Corbin Simpson wrote on Tue, Jul 08, 2008 at 09:03:01AM CEST: Hi, I'm working on a cross-platform game, and for plugin loading, I was hoping to use libltdl, since Win32 DLL loading is a pain to work with. However, it only builds on Windows with mingw or the cygwin system, and this is sort of a deal-breaker, since one of the libraries we require is currently broken on all Windows compilers besides MSVC. Is there some magic incantation that will make libltdl build with MSVC? Something I've overlooked? You can look at two separate efforts to get Libtool working with MSVC, one by Peter Rosin aka. Ekberg, and one by Markus Duft. They are still in need of integration into the tree, cleanup and all... :-/ You can find them in the list archives of the libtool-patches list. Cheers, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
RE: Using libltdl with the Microsoft compiler
On Mon, 2008-06-23 at 09:02 +0200, Duft Markus wrote: [snip] Historically (i.e. prior to libtool 2.x), I've found that those shell environments have not been up to the task of building my project with the Microsoft compiler. So I've maintained a Visual C++ project file build for that compiler. I've just recently started using libltdl in my project, though; and I'd like to build it using the same facilities as the rest of the software. Hmm... doesn't sound too easy :) at our company we use MS compilers through a compiler frontend that mimics GCC/LD, so we can use the same build system as on all the other platforms (i.e. libtool, autotools, etc...) - see below. Thank you for the suggestions. But for now I've found it most expedient simply not to use libltdl on Windows. Hopefully I'll be able to revisit this sometime in the future. -- Braden McDaniel e-mail: [EMAIL PROTECTED] http://endoframe.comJabber: [EMAIL PROTECTED] ___ http://lists.gnu.org/mailman/listinfo/libtool
RE: Using libltdl with the Microsoft compiler
[snip] Historically (i.e. prior to libtool 2.x), I've found that those shell environments have not been up to the task of building my project with the Microsoft compiler. So I've maintained a Visual C++ project file build for that compiler. I've just recently started using libltdl in my project, though; and I'd like to build it using the same facilities as the rest of the software. Hmm... doesn't sound too easy :) at our company we use MS compilers through a compiler frontend that mimics GCC/LD, so we can use the same build system as on all the other platforms (i.e. libtool, autotools, etc...) - see below. Really, I'd be happy to jettison the Visual C++ project file build and build the software with the autotools on Windows. But libtool 1.5 certainly didn't play nicely with the Microsoft compiler; and while I haven't tried 2.2, the traffic I've seen on this list suggests that patches to improve that situation haven't made it in yet. Try using parity [1] as frontend fort he microsoft compiler. The distfile also includes patches for various libtool versions to make is play nice with parity/ms compiler/ms linker etc. there are some wikis on how to get started at [2] and [3]. You will want to use MS Interix (SFU/SUA) as environment. Depending on your windows version you will have to install SFU 3.5 (Windows 2000, Windows XP, Windows Server 2003), SUA 5.2 (Windows Server 2003 R2) or SUA 6.0 (Windows Vista, Windows Server 2008) Hope that helps :) [1] http://sourceforge.net/projects/parity [2] http://parity.wiki.sourceforge.net/Building+parity+from+Interix [3] http://parity.wiki.sourceforge.net/HowTo+Use+parity+and+libtool+together Cheers, Markus -- Braden McDaniel e-mail: [EMAIL PROTECTED] http://endoframe.com Jabber: [EMAIL PROTECTED] ___ http://lists.gnu.org/mailman/listinfo/libtool ___ http://lists.gnu.org/mailman/listinfo/libtool
RE: Using libltdl with the Microsoft compiler
* Braden McDaniel wrote on Fri, Jun 20, 2008 at 10:20:13PM CEST: Ralf Wildenhues wrote: * Braden McDaniel wrote on Fri, Jun 20, 2008 at 11:03:04AM CEST: [snip] Yes, unfortunately there is still a ways to go. Btw. @Ralf: how is integration of the windows patchery going? :) has there been something moving on? You said there will be branches for it when libtool has migrated to GIT. How ist he progress there? Cheers, Markus Cheers, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Using libltdl with the Microsoft compiler
* Braden McDaniel wrote on Fri, Jun 20, 2008 at 10:20:13PM CEST: Ralf Wildenhues wrote: * Braden McDaniel wrote on Fri, Jun 20, 2008 at 11:03:04AM CEST: In lt__glibc.h, there are symbols HAVE_ARGZ_H and HAVE_WORKING_ARGZ. Yet, argz.h appears to be included irrespective of them. Is this intentional? (It's a problem here because there is no argz.h.) Yes. Libtool provides a replacement argz.h, see libltdl/argz_.h. I did. But argz_.h is not getting included by lt__glibc.h; argz.h is. Does the configure build copy argz_.h to argz.h? Yes. The copying is done so that, if the system argz.h is good, we don't accidentally include our own one. How come you cannot use a shell environment like MinGW or SFU to build Libtool? You're going to have a pretty rough time emulating the rather involved build logic for it. It didn't seem so bad for libtool 1.5; I guess it's gotten quite a bit more complicated for 2.x? Yes. Really, I'd be happy to jettison the Visual C++ project file build and build the software with the autotools on Windows. But libtool 1.5 certainly didn't play nicely with the Microsoft compiler; and while I haven't tried 2.2, the traffic I've seen on this list suggests that patches to improve that situation haven't made it in yet. Yes, unfortunately there is still a ways to go. Cheers, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Using libltdl with the Microsoft compiler
I've built libltdl with the Microsoft compiler and I'm trying to use it to load a DLL. I am calling lt_dlforeachfile; the problem seems to be that the call to loader-module_open in tryall_dlopen is failing. That call resolves to presym_open. This call is failing because preloaded_symbols is set to 0. I'm hoping someone more familiar with this code could explain to me what is supposed to have happened at this point that would make preloaded_symbols nonzero (and thus give the module_open call some chance of success). It is conceivable (in fact, likely) that I have built libltdl incorrectly; but if that's the problem here, the nature of the error has yet to jump out at me. -- Braden McDaniel e-mail: [EMAIL PROTECTED] http://endoframe.comJabber: [EMAIL PROTECTED] ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Using libltdl with the Microsoft compiler
Hi Braden, * Braden McDaniel wrote on Fri, Jun 20, 2008 at 09:37:59AM CEST: I've built libltdl with the Microsoft compiler and I'm trying to use it to load a DLL. without having looked at this at all (and not sure whether I can): please give us rope so this can be reproduced: are you using MinGW, or Cygwin, or Interix, do you use a compiler wrapper like cccl, or one of the other ones, which libltdl version, and how did you configure it? Probably its config.log (please gzip) would be helpful, too. Thanks, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Using libltdl with the Microsoft compiler
On Fri, 2008-06-20 at 09:52 +0200, Ralf Wildenhues wrote: Hi Braden, * Braden McDaniel wrote on Fri, Jun 20, 2008 at 09:37:59AM CEST: I've built libltdl with the Microsoft compiler and I'm trying to use it to load a DLL. without having looked at this at all (and not sure whether I can): please give us rope so this can be reproduced: are you using MinGW, or Cygwin, or Interix, do you use a compiler wrapper like cccl, or one of the other ones, which libltdl version, and how did you configure it? Probably its config.log (please gzip) would be helpful, too. I'm building in the Visual C++ Express 9.0 IDE. I have configured it by inspecting the code and trial and error, coming up with a set of preprocessor definitions that appears to work. Here is what that set currently looks like: __WIN32__ _DEBUG _LIB _CRT_SECURE_NO_WARNINGS HAVE_STDIO_H HAVE_STDLIB_H HAVE_STRING_H HAVE_CTYPE_H HAVE_MEMORY_H HAVE_ERRNO_H HAVE_ASSERT_H error_t=int LTDL_OBJDIR=\.\ LTDL_SHLIB_EXT=\.dll\ But let me back up. I thought I was using libtool 2.2.2. It turns out I wasn't. Now that I am, it seems I have other problems. In lt__glibc.h, there are symbols HAVE_ARGZ_H and HAVE_WORKING_ARGZ. Yet, argz.h appears to be included irrespective of them. Is this intentional? (It's a problem here because there is no argz.h.) -- Braden McDaniel e-mail: [EMAIL PROTECTED] http://endoframe.comJabber: [EMAIL PROTECTED] ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Using libltdl with the Microsoft compiler
* Braden McDaniel wrote on Fri, Jun 20, 2008 at 11:03:04AM CEST: In lt__glibc.h, there are symbols HAVE_ARGZ_H and HAVE_WORKING_ARGZ. Yet, argz.h appears to be included irrespective of them. Is this intentional? (It's a problem here because there is no argz.h.) Yes. Libtool provides a replacement argz.h, see libltdl/argz_.h. How come you cannot use a shell environment like MinGW or SFU to build Libtool? You're going to have a pretty rough time emulating the rather involved build logic for it. Cheers, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Using libltdl with the Microsoft compiler
Ralf Wildenhues wrote: * Braden McDaniel wrote on Fri, Jun 20, 2008 at 11:03:04AM CEST: In lt__glibc.h, there are symbols HAVE_ARGZ_H and HAVE_WORKING_ARGZ. Yet, argz.h appears to be included irrespective of them. Is this intentional? (It's a problem here because there is no argz.h.) Yes. Libtool provides a replacement argz.h, see libltdl/argz_.h. I did. But argz_.h is not getting included by lt__glibc.h; argz.h is. Does the configure build copy argz_.h to argz.h? How come you cannot use a shell environment like MinGW or SFU to build Libtool? You're going to have a pretty rough time emulating the rather involved build logic for it. It didn't seem so bad for libtool 1.5; I guess it's gotten quite a bit more complicated for 2.x? Historically (i.e. prior to libtool 2.x), I've found that those shell environments have not been up to the task of building my project with the Microsoft compiler. So I've maintained a Visual C++ project file build for that compiler. I've just recently started using libltdl in my project, though; and I'd like to build it using the same facilities as the rest of the software. Really, I'd be happy to jettison the Visual C++ project file build and build the software with the autotools on Windows. But libtool 1.5 certainly didn't play nicely with the Microsoft compiler; and while I haven't tried 2.2, the traffic I've seen on this list suggests that patches to improve that situation haven't made it in yet. -- Braden McDaniel e-mail: [EMAIL PROTECTED] http://endoframe.com Jabber: [EMAIL PROTECTED] ___ http://lists.gnu.org/mailman/listinfo/libtool
LIBLTDL
For GraphicsMagick I need to maintain a GraphicsMagick-config script which contains such things as the list of libraries necessary to link. The conditionally installed libltdl seems to make this difficult to figure out. If enable_ltdl_convenience is 'yes' then it is wrong to list -lltdl as needed but if enable_ltdl_install is 'yes' and enable_ltdl_convenience is 'no, then it appears that -lltdl should be listed as a dependency. In GraphicsMagick things are even more confusing since for a static build, libltdl is not used at all. It is necessary to build this logic out of internal configure variables. A bunch of existing substitutions will need to change in scripts and Makefiles. Ugh! It seems that the GM 1.2 release is a bit botched since I did not take the time to think through the convoluted logic. I suspect that other packages will be bit by this as well. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: LIBLTDL
Hi Bob, On 3 May 2008, at 17:40, Bob Friesenhahn wrote: For GraphicsMagick I need to maintain a GraphicsMagick-config script which contains such things as the list of libraries necessary to link. The conditionally installed libltdl seems to make this difficult to figure out. I think installing libltdl from a parent package is a bad enough idea that we should consider removing the option. Only libtool itself should really install a libltdl. Sane options are to link against the installed libltdl if it is good enough (whatever that means) or use a convenience libltdl otherwise. If enable_ltdl_convenience is 'yes' then it is wrong to list -lltdl as needed but if enable_ltdl_install is 'yes' and enable_ltdl_convenience is 'no, then it appears that -lltdl should be listed as a dependency. In GraphicsMagick things are even more confusing since for a static build, libltdl is not used at all. It is necessary to build this logic out of internal configure variables. A bunch of existing substitutions will need to change in scripts and Makefiles. Ugh! We should try to wrap this logic up in a macro for libtool-2.4 then. Although, just because libltdl itself provides a lot of flexibility in how it can be built, linked and installed, doesn't mean that the package that uses it needs to provide all those build options too. You might say that whenever GraphicsMagick links against libltdl it will always use a convenience libltdl... It seems that the GM 1.2 release is a bit botched since I did not take the time to think through the convoluted logic. I suspect that other packages will be bit by this as well. Bummer :-( Cheers, Gary -- ())_. Email me: [EMAIL PROTECTED] ( '/ Read my blog: http://blog.azazil.net / )= ...and my book: http://sources.redhat.com/autobook `(_~)_ PGP.sig Description: This is a digitally signed message part ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: LIBLTDL
On Sat, 3 May 2008, Gary V. Vaughan wrote: Sane options are to link against the installed libltdl if it is good enough (whatever that means) or use a convenience libltdl otherwise. I agree with that. Otherwise people may accidentally interfere with a formally installed libltdl maintained by a package manager. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool