Re: [Lazarus] Dynamic loading of a custom sqlite library
On Friday 10 of June 2011 10:22:34 Max Vlasov wrote: Looks like it's linux-only patch introduced at around 2004 http://sourceware.org/ml/libc-hacker/2004-09/msg00083.html so was not a good idea to update rtl\unix\dl.pp :) But would be ok to add it to documentation of dynlibs, that someone can use that flag under linux if have similar problem. I had same problem with sqlite3 and double installation of it under fedora 1.system one 2.xulrunner one, always had crashes if firefox is started before I use my app. With that flag it's fixed. zeljko -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Dynamic loading of a custom sqlite library
On Fri, 10 Jun 2011, Maxim Ganetsky wrote: 09.06.2011 13:17, Max Vlasov пишет: Finally found that this was because sqlite3 was in the uses list, so actually it was scenario when fpc linked sqlite dynamically and besides I did my own dynamic loading with my own unit of another version/file of sqlite. So probably there was a conflict with shared data. In my case it looks like everything was ok if the linked library and loaded was the same file (system libsqlite3.so). But if linked was system libsqlite3.so and my loaded was a library with a different name/soname the faults took place. Just curious, such things occasionally can happen from time to time. Is this about the way sqlite library is compiled so no two independent copies of the library coexist dynamically linked/loaded due to intersetion of the shared data. Or this is about the way linux handles them? It is best to ask this question in FPC devel mailing list, this behavior may be a bug. It is not a FPC bug; you're not supposed to use the 2 units at the same time. There is no way the compiler can detect this. At best the linux linker should detect it. Michael.-- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Dynamic loading of a custom sqlite library
2011/6/10 michael.vancann...@wisa.be On Fri, 10 Jun 2011, Maxim Ganetsky wrote: 09.06.2011 13:17, Max Vlasov пишет: Finally found that this was because sqlite3 was in the uses list, so actually it was scenario when fpc linked sqlite dynamically and besides I did my own dynamic loading with my own unit of another version/file of sqlite. So probably there was a conflict with shared data. In my case it looks like everything was ok if the linked library and loaded was the same file (system libsqlite3.so). But if linked was system libsqlite3.so and my loaded was a library with a different name/soname the faults took place. Just curious, such things occasionally can happen from time to time. Is this about the way sqlite library is compiled so no two independent copies of the library coexist dynamically linked/loaded due to intersetion of the shared data. Or this is about the way linux handles them? It is best to ask this question in FPC devel mailing list, this behavior may be a bug. It is not a FPC bug; you're not supposed to use the 2 units at the same time. There is no way the compiler can detect this. At best the linux linker should detect it. Michael, actually what appeared was that two libraries with different filenames and different soname (although compiled from the same sources, one dynamically linked and another dynamically loaded) shared some global data during run-time. I thought that in this case they should look virtually different for the linux loader to separate everything, but actually they were not. What interesting is that a solution was found. It's a flag RTLD_DEEPBIND for dlopen, I discovered it by chance, wrongly assuming RTLD_LOCAL as having $8 (netware libc) while RTLD_LOCAL=0 in linux. In Linux RTLD_DEEPBIND = $8 RTLD_DEEPBIND is not present in any of lazarus/fpc sources, but after I added it to my dlopen call, dynamically loaded library started to work when there was still a dynamically linked sqlite. So they became independent from each other. Quote from http://linux.die.net/man/3/dlopen Place the lookup scope of the symbols in this library ahead of the global scope. This means that a self-contained library will use its own symbols in preference to global symbols with the same name contained in libraries that have already been loaded. This flag is not specified in POSIX.1-2001. I'm not sure this will be helpful for everyone, but maybe it is worth to add this constant to rtl\unix\dl.pp file? Max -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Dynamic loading of a custom sqlite library
On Fri, 10 Jun 2011, Max Vlasov wrote: 2011/6/10 michael.vancann...@wisa.be On Fri, 10 Jun 2011, Maxim Ganetsky wrote: 09.06.2011 13:17, Max Vlasov пишет: Finally found that this was because sqlite3 was in the uses list, so actually it was scenario when fpc linked sqlite dynamically and besides I did my own dynamic loading with my own unit of another version/file of sqlite. So probably there was a conflict with shared data. In my case it looks like everything was ok if the linked library and loaded was the same file (system libsqlite3.so). But if linked was system libsqlite3.so and my loaded was a library with a different name/soname the faults took place. Just curious, such things occasionally can happen from time to time. Is this about the way sqlite library is compiled so no two independent copies of the library coexist dynamically linked/loaded due to intersetion of the shared data. Or this is about the way linux handles them? It is best to ask this question in FPC devel mailing list, this behavior may be a bug. It is not a FPC bug; you're not supposed to use the 2 units at the same time. There is no way the compiler can detect this. At best the linux linker should detect it. Michael, actually what appeared was that two libraries with different filenames and different soname (although compiled from the same sources, one dynamically linked and another dynamically loaded) shared some global data during run-time. I thought that in this case they should look virtually different for the linux loader to separate everything, but actually they were not. What interesting is that a solution was found. It's a flag RTLD_DEEPBIND for dlopen, I discovered it by chance, wrongly assuming RTLD_LOCAL as having $8 (netware libc) while RTLD_LOCAL=0 in linux. In Linux RTLD_DEEPBIND = $8 RTLD_DEEPBIND is not present in any of lazarus/fpc sources, but after I added it to my dlopen call, dynamically loaded library started to work when there was still a dynamically linked sqlite. So they became independent from each other. Quote from http://linux.die.net/man/3/dlopen Place the lookup scope of the symbols in this library ahead of the global scope. This means that a self-contained library will use its own symbols in preference to global symbols with the same name contained in libraries that have already been loaded. This flag is not specified in POSIX.1-2001. Yes, I can see how this can help. I'm not sure this will be helpful for everyone, but maybe it is worth to add this constant to rtl\unix\dl.pp file? It depends. Does it also exist in FreeBSD and Mac OS ? If not, then I think this is not such a good idea. Michael.-- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Dynamic loading of a custom sqlite library
2011/6/10 michael.vancann...@wisa.be I'm not sure this will be helpful for everyone, but maybe it is worth to add this constant to rtl\unix\dl.pp file? It depends. Does it also exist in FreeBSD and Mac OS ? If not, then I think this is not such a good idea. Good point, Looks like it's linux-only patch introduced at around 2004 http://sourceware.org/ml/libc-hacker/2004-09/msg00083.html so was not a good idea to update rtl\unix\dl.pp :) Max -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Dynamic loading of a custom sqlite library
On Wed, Jun 8, 2011 at 6:27 PM, Max Vlasov max.vla...@gmail.com wrote: Hi, I encountered problems using dynamically loaded sqlite library with lazarus on linux. My post about this in the sqlite list is here: http://www.mail-archive.com/sqlite-users@sqlite.org/msg61565.html Maybe someone points me to the right direction It's about loading sqlite library with dlopen, getting all the methods with dlsym and using it. I intentionally use different name and soname to not to conflict with the system name (libsqlite3.so) It works if I use default libsqlite3 library installed with ubuntu package system in dlopen. But if I compile the latest sqlite by myself or use configurable sqlite download with default options, I get SIGSEGV errors inside sqlite code and they appear inside dlopen is called (this is because sqlite3_initialize is defined to be executed when the library is beeing loaded). ... Finally found that this was because sqlite3 was in the uses list, so actually it was scenario when fpc linked sqlite dynamically and besides I did my own dynamic loading with my own unit of another version/file of sqlite. So probably there was a conflict with shared data. In my case it looks like everything was ok if the linked library and loaded was the same file (system libsqlite3.so). But if linked was system libsqlite3.so and my loaded was a library with a different name/soname the faults took place. Just curious, such things occasionally can happen from time to time. Is this about the way sqlite library is compiled so no two independent copies of the library coexist dynamically linked/loaded due to intersetion of the shared data. Or this is about the way linux handles them? Thanks Max -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] Dynamic loading of a custom sqlite library
09.06.2011 13:17, Max Vlasov пишет: Finally found that this was because sqlite3 was in the uses list, so actually it was scenario when fpc linked sqlite dynamically and besides I did my own dynamic loading with my own unit of another version/file of sqlite. So probably there was a conflict with shared data. In my case it looks like everything was ok if the linked library and loaded was the same file (system libsqlite3.so). But if linked was system libsqlite3.so and my loaded was a library with a different name/soname the faults took place. Just curious, such things occasionally can happen from time to time. Is this about the way sqlite library is compiled so no two independent copies of the library coexist dynamically linked/loaded due to intersetion of the shared data. Or this is about the way linux handles them? It is best to ask this question in FPC devel mailing list, this behavior may be a bug. -- Best regards, Maxim Ganetsky mailto:gan...@narod.ru -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
[Lazarus] Dynamic loading of a custom sqlite library
Hi, I encountered problems using dynamically loaded sqlite library with lazarus on linux. My post about this in the sqlite list is here: http://www.mail-archive.com/sqlite-users@sqlite.org/msg61565.html Maybe someone points me to the right direction It's about loading sqlite library with dlopen, getting all the methods with dlsym and using it. I intentionally use different name and soname to not to conflict with the system name (libsqlite3.so) It works if I use default libsqlite3 library installed with ubuntu package system in dlopen. But if I compile the latest sqlite by myself or use configurable sqlite download with default options, I get SIGSEGV errors inside sqlite code and they appear inside dlopen is called (this is because sqlite3_initialize is defined to be executed when the library is beeing loaded). Mostly the faults are always about calling functions accessed from the global structure (sqlite3GlobalConfig) related to mutexes or memory management (I found this when compiled the library with debug info) If I link the same 'problem' library with the sqlite shell (sqlite3), they both work without any problem (so I can do all db manipulations in the shell). So I can not understand what is so special about the default installed library and where should I look further. Thanks in advance, Max -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
