Re: [fpc-pascal] Shared libraries and threadvars
Can such an alternative entry point in the main unit be called by a shared library, i.e. either resolved at load time or with the main binary reopened like a library? Or is the only way to pass a main- program entry point to a shared library by using it as a parameter to a procedure? I can't tell you for *nix systems as I'm not experienced enough regarding their dynamic linkers, Superficial testing suggests that if the code in a .so (Linux, i386) tries to call an extra entry point exported from the main program, that a run-time (libdl) load of the library will fail. I'll keep on tinkering with this, I've not yet retrieved the exact error message and I'm interested in seeing what happens if the .so itself tries to use libdl to reopen the main program. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Shared libraries and threadvars
Mark Morgan Lloyd wrote: Can such an alternative entry point in the main unit be called by a shared library, i.e. either resolved at load time or with the main binary reopened like a library? Or is the only way to pass a main- program entry point to a shared library by using it as a parameter to a procedure? I can't tell you for *nix systems as I'm not experienced enough regarding their dynamic linkers, Superficial testing suggests that if the code in a .so (Linux, i386) tries to call an extra entry point exported from the main program, that a run-time (libdl) load of the library will fail. I'll keep on tinkering with this, I've not yet retrieved the exact error message and I'm interested in seeing what happens if the .so itself tries to use libdl to reopen the main program. Short answer: works on Linux, but at run- (not load-time) only. Longer answer: if a .so has an explicit dependency on the main program, i.e. a procedure marked extern, an attempt to load it using dlopen(...RTLDLAZY) will fail (return zero as a handle). The error message refers to an entry point of the form library_proc_types rather than program_proc_types, and I've not managed to override that using extern or extern/name in a way that makes everything happy. Using LoadLibrary() - dlopen() inside the .so, passing an *empty* string representing the main program, appears to allow exported entry points to be accessed (i.e. a shared library can access a procedure in the main program in exactly the same way that a main program can access a procedure in a shared library). One thing that would be useful would be if DynLibs had a get last error function. I've implemented this myself for Linux and Windows and find it indispensable. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Shared libraries and threadvars
Am 26.06.2012 14:02 schrieb Mark Morgan Lloyd markmll.fpc-pas...@telemetry.co.uk: I'm currently tinkering with shared libraries, using cmem, mainly on Linux. In one case the main-program code is multithreaded, but so far I'm not moving data between threads inside the library. Am I correct in believing that unit-level variables in a shared library might end up being shared between multiple instances of the library, or are they guaranteed to be distinct? If my belief is correct, I presume that I could get around this by using threadvars. But in this case, how would I best implement a variable that I wanted to be common across related threads (e.g. to track the number of a particular object being allocated/deallocated), but distinct across program invocations (i.e. two programs using the same shared library wouldn't clash)? A programs memory always belongs to itself (exceptions through mmap or kernel related thing excluded) and libraries are loaded into the program's memory. Their code sections might only exist once if the OS detects that it had already loaded the library somewhere, but it's data sections and also the memory it allocates during its lifetime belong to the program's memory. This means for you: normal global variables in your library are not shared with other instances of this library (note: if a program loads library A and B and library A also loads B then there is only one instance of the library and its data). They are shared by all threads that use this library though. Having variables shared between different instances of a library is a different topic altogether. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Shared libraries and threadvars
Mark Morgan Lloyd wrote: I'm currently tinkering with shared libraries, using cmem, mainly on Linux. In one case the main-program code is multithreaded, but so far I'm not moving data between threads inside the library. Am I correct in believing that unit-level variables in a shared library might end up being shared between multiple instances of the library, or are they guaranteed to be distinct? If my belief is correct, I presume that I could get around this by using threadvars. But in this case, how would I best implement a variable that I wanted to be common across related threads (e.g. to track the number of a particular object being allocated/deallocated), but distinct across program invocations (i.e. two programs using the same shared library wouldn't clash)? The discussion of libraries in ch16 of the Language reference guide is rather quiet on the semantics. A programs memory always belongs to itself (exceptions through mmap or kernel related thing excluded) and libraries are loaded into the program's memory. Their code sections might only exist once if the OS detects that it had already loaded the library somewhere, but it's data sections and also the memory it allocates during its lifetime belong to the program's memory. Thanks Sven. So to summarise: code might be shared, but this generally won't be relevant since it should be read-only. Data- local or global- won't be shared. This means for you: normal global variables in your library are not shared with other instances of this library (note: if a program loads library A and B and library A also loads B then there is only one instance of the library and its data). They are shared by all threads that use this library though. Having variables shared between different instances of a library is a different topic altogether. OK, so to emphasise that: library B's data will be common, irrespective of the location of the calling routine. Does this apply if the load is being done at runtime, i.e. the program is using dl (or whatever) to load A and B, and A also uses dl to load B? -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Shared libraries and threadvars
Am 26.06.2012 17:15 schrieb Mark Morgan Lloyd markmll.fpc-pas...@telemetry.co.uk: Mark Morgan Lloyd wrote: I'm currently tinkering with shared libraries, using cmem, mainly on Linux. In one case the main-program code is multithreaded, but so far I'm not moving data between threads inside the library. Am I correct in believing that unit-level variables in a shared library might end up being shared between multiple instances of the library, or are they guaranteed to be distinct? If my belief is correct, I presume that I could get around this by using threadvars. But in this case, how would I best implement a variable that I wanted to be common across related threads (e.g. to track the number of a particular object being allocated/deallocated), but distinct across program invocations (i.e. two programs using the same shared library wouldn't clash)? The discussion of libraries in ch16 of the Language reference guide is rather quiet on the semantics. A programs memory always belongs to itself (exceptions through mmap or kernel related thing excluded) and libraries are loaded into the program's memory. Their code sections might only exist once if the OS detects that it had already loaded the library somewhere, but it's data sections and also the memory it allocates during its lifetime belong to the program's memory. Thanks Sven. So to summarise: code might be shared, but this generally won't be relevant since it should be read-only. Data- local or global- won't be shared. Correct. Shared code sections shouldn't matter normally. This means for you: normal global variables in your library are not shared with other instances of this library (note: if a program loads library A and B and library A also loads B then there is only one instance of the library and its data). They are shared by all threads that use this library though. Having variables shared between different instances of a library is a different topic altogether. OK, so to emphasise that: library B's data will be common, irrespective of the location of the calling routine. Does this apply if the load is being done at runtime, i.e. the program is using dl (or whatever) to load A and B, and A also uses dl to load B? It does not matter whether the library is loaded at program start or at runtime. It's always the same mechanism with the exception that in the first case your main function has not yet been called (which normally doesn't matter either). Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Shared libraries and threadvars
Apologies for my lousy message threading, but some are vanishing in the spamtrap. OK, so to emphasise that: library B's data will be common, irrespective of the location of the calling routine. Does this apply if the load is being done at runtime, i.e. the program is using dl (or whatever) to load A and B, and A also uses dl to load B? It does not matter whether the library is loaded at program start or at runtime. It's always the same mechanism with the exception that in the first case your main function has not yet been called (which normally doesn't matter either). Just being cautious :-) One final question if I may: noting Ludo's example elsewhere: In the project.lpr add the following code: Procedure qt_startup_hook;export; Begin End; Exports qt_startup_hook; Can such an alternative entry point in the main unit be called by a shared library, i.e. either resolved at load time or with the main binary reopened like a library? Or is the only way to pass a main-program entry point to a shared library by using it as a parameter to a procedure? I've used such things in a Windows-style .exe so a loader/binder would know how to generate absolute code for an embedded system, but have never tried exploiting it with a real OS. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Shared libraries and threadvars
Am 26.06.2012 20:09 schrieb Mark Morgan Lloyd markmll.fpc-pas...@telemetry.co.uk: One final question if I may: noting Ludo's example elsewhere: In the project.lpr add the following code: Procedure qt_startup_hook;export; Begin End; Exports qt_startup_hook; Can such an alternative entry point in the main unit be called by a shared library, i.e. either resolved at load time or with the main binary reopened like a library? Or is the only way to pass a main-program entry point to a shared library by using it as a parameter to a procedure? I've used such things in a Windows-style .exe so a loader/binder would know how to generate absolute code for an embedded system, but have never tried exploiting it with a real OS. I can't tell you for *nix systems as I'm not experienced enough regarding their dynamic linkers, but I can tell you for Windows systems: If you export functions from a executable (not a DLL) you can get access to these methods from within a library the same way you'd do with a DLL (though I don't know what happens if you load a unrelated executable). As your executable is already loaded no further instance of it will be created. There is also the possibility that you link to the executable at link time (in Pascal using the procedure bla; external 'your.exe' name 'bla' mechanism). I don't know whether you can build a circle here (in the sense of having the exe depend on that DLL as well) as I have not tested that that mechanism yet. Also there is only one case that I definitely know of that uses this link time variant: drivers on NT based systems (which are simply DLLs) link to ntoskrnl.exe and the kernel loads them dynamically. You can see this if you look into the ndk.pas unit of my Native NT port where the constant NTDLL is set to 'ntoskrnl.exe' instead of 'ntdll.dll' if it is compiled with KMODE defined. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
