I ran all of the load and loadapi tests in the test suite and found a couple of problems in the current implementation that were based on unportable assumptions. The solutions touch to some extent the platform independent code and build requirements, so I'd like to discuss them here before I actually push them. (For the impatient: see the patches near the end of this mail.)
The two problems I discovered and fixed are: 1. How to build the extension shared libraries. The current code and instructions (and also the test suite) simply compile and link the extensions with -shared compiler switch. This assumes the linker lets you get away with unresolved externals (the gmk_* functions provided by Make), and let these references be resolved at run time, when the extension is loaded by dlopen/dlsym. This doesn't work on Windows. There, the linker _must_ see some symbols that allow it to resolve the references _at_link_time_. So if I try the original command used by the load test, i.e.: gcc -I../../.. -g -shared -fPIC -o testload.dll testload.c it complains about unresolved externals and bails out. There are 2 ways of solving this on Windows: . Call the gmk_* functions via pointers (so that the linker doesn't see the function calls at all). These pointers need to be assigned the addresses of the functions at the dlopen time, and they need to have the exact same type as the functions they will call. I did manage to get this to work, but found that this method has a lot of limitations. For example, since the pointer variables need to have the same names as the functions (to make it possible to compile the same extension on Unix and on Windows), these variables need to be static (to put them in gnumake.h header), and the function to be called by dlopen needs also to be static. But using static functions and variables means that an extension that has more than 1 C file will have several unrelated copied of these... So I prefer the second alternative, which is: . Use an import library. An import library is a library of stubs that provide enough info for the linker to be happy at link time, and leave the actual resolution of the references at dlopen time. GCC can create such a library for select functions that are declared with a special type. Then the import library is submitted to the linker when the extension is built, and that's it -- the extension can freely call functions exported by Make, as if they were defined in the extension. (Normally, a shared library exports its functions and an import library is used to link the main program. But our case is the opposite: we need the main program to export some functions. But the mechanism works either way.) This requires minor changes to the build_w32.bat script which builds Make, and also to the test suite (if we want it to be runnable on Windows; I ran the test by hand). Additional changes are needed to put the necessary decorations on the functions exported by Make. It also requires that the import library be distributed with the Make binaries for Windows. 2. When Make decides that it needs to remake a dynamic object, it does so with the object still loaded. Windows does not allow to overwrite a shared library that is being used by some program, so the remake fails. To fix this, I record the pointer returned by dlopen in the file structure of the dynamic object, and call dlclose before running the job to remake it. This means that any features defined by the object cannot be used for remaking the object. I hope this does not impose any real restrictions, since if the object is needed to remake itself, it means that object cannot be created from scratch. That's it. Now I let the patch talk: --- commands.c~2 2013-04-28 06:41:56.000000000 +0300 +++ commands.c 2013-04-30 11:42:46.345909500 +0300 @@ -14,6 +14,8 @@ A PARTICULAR PURPOSE. See the GNU Gener You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>. */ +#include <dlfcn.h> + #include "makeint.h" #include "dep.h" #include "filedef.h" @@ -468,6 +470,12 @@ execute_file_commands (struct file *file set_file_variables (file); + /* If this is a loaded dynamic object, unload it before remaking. + Some systems don't allow to overwrite a loaded shared + library. */ + if (file->dlopen_ptr) + dlclose (file->dlopen_ptr); + /* Start the commands running. */ new_job (file); } --- filedef.h~2 2013-04-29 06:47:10.000000000 +0300 +++ filedef.h 2013-04-30 11:56:47.364106900 +0300 @@ -61,6 +61,8 @@ struct file int command_flags; /* Flags OR'd in for cmds; see commands.h. */ char update_status; /* Status of the last attempt to update, or -1 if none has been made. */ + void *dlopen_ptr; /* For dynamic loaded objects: pointer to + pass to dlclose to unload the object. */ enum cmd_state /* State of the commands. */ { /* Note: It is important that cs_not_started be zero. */ cs_not_started, /* Not yet started. */ --- gnumake.h~1 2013-04-28 06:41:57.000000000 +0300 +++ gnumake.h 2013-04-30 09:58:14.387983000 +0300 @@ -26,13 +26,23 @@ typedef struct unsigned long lineno; } gmk_floc; +#ifdef _WIN32 +# ifdef MAIN +# define GMK_EXPORT __declspec(dllexport) +# else +# define GMK_EXPORT __declspec(dllimport) +# endif +#else +# define GMK_EXPORT +#endif + /* Run $(eval ...) on the provided string BUFFER. */ -void gmk_eval (const char *buffer, const gmk_floc *floc); +void GMK_EXPORT gmk_eval (const char *buffer, const gmk_floc *floc); /* Run GNU make expansion on the provided string STR. Returns an allocated buffer that the caller must free. */ -char *gmk_expand (const char *str); +char * GMK_EXPORT gmk_expand (const char *str); /* Register a new GNU make function NAME (maximum of 255 chars long). When the function is expanded in the makefile, FUNC will be invoked with @@ -49,8 +59,9 @@ char *gmk_expand (const char *str); If EXPAND_ARGS is 0, the arguments to the function will not be expanded before FUNC is called. If EXPAND_ARGS is non-0, they will be expanded. */ -void gmk_add_function (const char *name, - char *(*func)(const char *nm, int argc, char **argv), - int min_args, int max_args, int expand_args); +void GMK_EXPORT gmk_add_function (const char *name, + char *(*func)(const char *nm, + int argc, char **argv), + int min_args, int max_args, int expand_args); #endif /* _GNUMAKE_H_ */ --- load.c~2 2013-04-29 14:52:21.870479300 +0300 +++ load.c 2013-04-30 11:33:49.346863900 +0300 @@ -32,11 +32,13 @@ this program. If not, see <http://www.g static load_func_t load_object (const gmk_floc *flocp, int noerror, - const char *ldname, const char *symname) + const char *ldname, const char *symname, void **dlp) { static void *global_dl = NULL; load_func_t symp; + *dlp = NULL; + if (! global_dl) { global_dl = dlopen (NULL, RTLD_NOW|RTLD_GLOBAL); @@ -46,7 +48,6 @@ load_object (const gmk_floc *flocp, int symp = (load_func_t) dlsym (global_dl, symname); if (! symp) { - void *dlp = NULL; /* If the path has no "/", try the current directory first. */ if (! strchr (ldname, '/') @@ -54,14 +55,14 @@ load_object (const gmk_floc *flocp, int && ! strchr (ldname, '\\') #endif ) - dlp = dlopen (concat (2, "./", ldname), RTLD_LAZY|RTLD_GLOBAL); + *dlp = dlopen (concat (2, "./", ldname), RTLD_LAZY|RTLD_GLOBAL); /* If we haven't opened it yet, try the default search path. */ - if (! dlp) - dlp = dlopen (ldname, RTLD_LAZY|RTLD_GLOBAL); + if (! *dlp) + *dlp = dlopen (ldname, RTLD_LAZY|RTLD_GLOBAL); /* Still no? Then fail. */ - if (! dlp) + if (! *dlp) { if (noerror) DB (DB_BASIC, ("%s", dlerror())); @@ -70,7 +71,7 @@ load_object (const gmk_floc *flocp, int return NULL; } - symp = dlsym (dlp, symname); + symp = dlsym (*dlp, symname); if (! symp) fatal (flocp, _("Failed to load symbol %s from %s: %s"), symname, ldname, dlerror()); @@ -80,7 +81,7 @@ load_object (const gmk_floc *flocp, int } int -load_file (const gmk_floc *flocp, const char **ldname, int noerror) +load_file (const gmk_floc *flocp, const char **ldname, int noerror, void **dlp) { int nmlen = strlen (*ldname); char *new = alloca (nmlen + CSTRLEN (SYMBOL_EXTENSION) + 1); @@ -90,6 +91,8 @@ load_file (const gmk_floc *flocp, const int r; load_func_t symp; + *dlp = NULL; + /* Break the input into an object file name and a symbol name. If no symbol name was provided, compute one from the object file name. */ fp = strchr (*ldname, '('); @@ -165,7 +168,7 @@ load_file (const gmk_floc *flocp, const DB (DB_VERBOSE, (_("Loading symbol %s from %s\n"), symname, *ldname)); /* Load it! */ - symp = load_object(flocp, noerror, *ldname, symname); + symp = load_object(flocp, noerror, *ldname, symname, dlp); if (! symp) return 0; --- loadapi.c~1 2013-04-28 06:41:57.000000000 +0300 +++ loadapi.c 2013-04-30 08:35:33.423280500 +0300 @@ -14,8 +14,6 @@ A PARTICULAR PURPOSE. See the GNU Gener You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>. */ -#include "gnumake.h" - #include "makeint.h" #include "filedef.h" --- makeint.h~1 2013-04-29 06:47:10.000000000 +0300 +++ makeint.h 2013-04-30 12:01:21.405874500 +0300 @@ -48,8 +48,12 @@ char *alloca (); #endif /* Include the externally-visible content. - Be sure to use the local one, and not one installed on the system. */ + Be sure to use the local one, and not one installed on the system. + Define MAIN for proper selection of dllexport/dllimport declarations + for MS-Windows. */ +#define MAIN #include "gnumake.h" +#undef MAIN #ifdef CRAY /* This must happen before #include <signal.h> so @@ -476,7 +480,8 @@ int guile_gmake_setup (const gmk_floc *f /* Loadable object support. Sets to the strcached name of the loaded file. */ typedef int (*load_func_t)(const gmk_floc *flocp); -int load_file (const gmk_floc *flocp, const char **filename, int noerror); +int load_file (const gmk_floc *flocp, const char **filename, int noerror, + void **dlp); #ifdef HAVE_VFORK_H # include <vfork.h> --- read.c~2 2013-04-28 06:41:57.000000000 +0300 +++ read.c 2013-04-30 12:10:07.718054200 +0300 @@ -937,11 +937,12 @@ eval (struct ebuffer *ebuf, int set_defa struct nameseq *next = files->next; const char *name = files->name; struct dep *deps; + void *dlp; free_ns (files); files = next; - if (! load_file (&ebuf->floc, &name, noerror) && ! noerror) + if (! load_file (&ebuf->floc, &name, noerror, &dlp) && ! noerror) fatal (&ebuf->floc, _("%s: failed to load"), name); deps = alloc_dep (); @@ -950,6 +951,7 @@ eval (struct ebuffer *ebuf, int set_defa deps->file = lookup_file (name); if (deps->file == 0) deps->file = enter_file (name); + deps->file->dlopen_ptr = dlp; } continue; --- tests/scripts/features/load~2 2013-04-28 06:41:58.000000000 +0300 +++ tests/scripts/features/load 2013-04-30 12:12:27.409354500 +0300 @@ -21,14 +21,14 @@ print $F <<'EOF' ; #include "gnumake.h" int -testload_gmk_setup () +testload_gmk_setup (gmk_floc *pos) { gmk_eval ("TESTLOAD = implicit", 0); return 1; } int -explicit_setup () +explicit_setup (gmk_floc *pos) { gmk_eval ("TESTLOAD = explicit", 0); return 1; --- w32/compat/posixfcn.c~2 2013-04-30 11:39:33.497473300 +0300 +++ w32/compat/posixfcn.c 2013-04-30 11:39:44.058741000 +0300 @@ -338,6 +338,17 @@ dlsym (void *handle, const char *name) return (void *)addr; } +int +dlclose (void *handle) +{ + if (!handle || handle == INVALID_HANDLE_VALUE) + return -1; + if (!FreeLibrary (handle)) + return -1; + + return 0; +} + #endif /* MAKE_LOAD */ --- w32/include/dlfcn.h~2 2013-04-29 15:48:02.793614200 +0300 +++ w32/include/dlfcn.h 2013-04-30 11:38:27.309442800 +0300 @@ -9,5 +9,6 @@ extern void *dlopen (const char *, int); extern void *dlsym (void *, const char *); extern char *dlerror (void); +extern int dlclose (void *); #endif /* DLFCN_H */ --- build_w32.bat~2 2013-04-28 06:41:56.000000000 +0300 +++ build_w32.bat 2013-04-30 10:02:24.098783700 +0300 @@ -291,7 +291,7 @@ gcc -mthreads -Wall -gdwarf-2 -g3 %OPT% %GUILECFLAGS% -I. -I./glob -I./w32/include -DWINDOWS32 -DHAVE_CONFIG_H -c guile.c :LinkGCC @echo on -gcc -mthreads -gdwarf-2 -g3 -o gnumake.exe variable.o rule.o remote-stub.o commands.o file.o getloadavg.o default.o signame.o expand.o dir.o main.o getopt1.o %GUILEOBJ% job.o read.o version.o getopt.o arscan.o remake.o misc.o hash.o strcache.o ar.o function.o vpath.o implicit.o loadapi.o load.o glob.o fnmatch.o pathstuff.o posixfcn.o w32_misc.o sub_proc.o w32err.o %GUILELIBS% -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -luuid -lodbc32 -lodbccp32 +gcc -mthreads -gdwarf-2 -g3 -o gnumake.exe variable.o rule.o remote-stub.o commands.o file.o getloadavg.o default.o signame.o expand.o dir.o main.o getopt1.o %GUILEOBJ% job.o read.o version.o getopt.o arscan.o remake.o misc.o hash.o strcache.o ar.o function.o vpath.o implicit.o loadapi.o load.o glob.o fnmatch.o pathstuff.o posixfcn.o w32_misc.o sub_proc.o w32err.o %GUILELIBS% -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -luuid -lodbc32 -lodbccp32 -Wl,--out-implib=libgnumake.dll.a @GoTo BuildEnd :Usage echo Usage: %0 [options] [gcc] _______________________________________________ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make