Re: [sqlite] Dynamically loaded sqlite (linux)
On Thu, Jun 9, 2011 at 6:21 PM, Pavel Ivanov wrote: > > I know that I should avoid such things, but just curious, is it something > > inside sqlite that probably makes one dynanmically linked and other > > dynamically loaded library share global data and can this be avoidable? > > No, it's Linux linker who works like that. If you have several > libraries loaded into the process with the same exported symbols then > everybody using those symbols will be linked against ones located in > the library loaded first. Any symbols located in libraries loaded > after that won't be used. This is the default behavior of Linux linker > (it can be changed but AFAIK it's changed very rarely) and this > feature can produce sometimes very nasty results. E.g. if you try to > load the same library located in different directories then > initialization code will be executed in both of them, but they both > will be executed against the same global/static variables. And it > could lead to problems during initialization and definitely will lead > to problems during finalization (like double frees, segmentation > faults etc.). > > Occasionally found the the flag RTLD_DEEPBIND for dlopen helped. I suppose it was a not so old answer for the problems Pavel described. A quote from http://linux.die.net/man/3/dlopen > Place the lookup scope of the symbols in this library ahead of the global > scope. This means that a self-contained library will use its own symbols in > preference to global symbols with the same name contained in libraries that > have already been loaded. This flag is not specified in POSIX.1-2001. > In my case it helped, two instances (one dynamically linked, other loaded) started to act as independent entities without faults But what about the last comment (about POSIX.1-2001). Does it mean some distros could not support it? Max ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Dynamically loaded sqlite (linux)
On Thu, Jun 9, 2011 at 6:21 PM, Pavel Ivanov wrote: > > > So if you ever want to use dlopen() you should be really really > careful to avoid loading the same library several times (even if the > same library have different file names). > > Pavel, thanks for the hint and the information, Max ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Dynamically loaded sqlite (linux)
> I know that I should avoid such things, but just curious, is it something > inside sqlite that probably makes one dynanmically linked and other > dynamically loaded library share global data and can this be avoidable? No, it's Linux linker who works like that. If you have several libraries loaded into the process with the same exported symbols then everybody using those symbols will be linked against ones located in the library loaded first. Any symbols located in libraries loaded after that won't be used. This is the default behavior of Linux linker (it can be changed but AFAIK it's changed very rarely) and this feature can produce sometimes very nasty results. E.g. if you try to load the same library located in different directories then initialization code will be executed in both of them, but they both will be executed against the same global/static variables. And it could lead to problems during initialization and definitely will lead to problems during finalization (like double frees, segmentation faults etc.). So if you ever want to use dlopen() you should be really really careful to avoid loading the same library several times (even if the same library have different file names). Pavel On Thu, Jun 9, 2011 at 9:56 AM, Max Vlasov wrote: > On Tue, Jun 7, 2011 at 9:22 PM, Martin Gadbois wrote: > >> On Tue, Jun 7, 2011 at 12:52 PM, Jay A. Kreibich wrote: >> >> > On Tue, Jun 07, 2011 at 07:47:25PM +0400, Max Vlasov scratched on the >> wall: >> > > Hi, >> > > >> > > I'm trying to use sqlite with linux (Ubuntu, Pascal, Lazarus). I'm >> still >> > not >> > > very familiar with linux development so I might miss something >> essential. >> > > >> > > Two scenarios work ok >> > > - statically linked latest version compiled (3.7.6.3), no options or >> > defines >> > > changed >> > > - Dynamically loaded (dlopen) sqlite used from the installed package >> > > libsqlite3 (libsqlite3.so) >> > >> > That's not how dynamic libraries work (not normally, anyways). >> > Generally you simply tell the compiler/linker to link in the library >> > at build time, and allow the usage of dynamic libs. The dynamic >> > link is then done on application start-up by the OS. In Windows >> > terms, it is like using an .DLL by linking in the associated .lib >> > file. Moving from a static library to a dynamic library requires no >> > code changes. >> > >> >> >> There is a way to do a _good_ shared library. I suggest reading the >> excellent paper: http://www.akkadia.org/drepper/dsohowto.pdf >> >> As for the OP question, do >> gcc -shared -Wl,-init=sqlite3_initialize -o libsqlite.so sqlite3.o >> >> and then link your application with >> gcc -L. -lsqlite -o test test.c >> >> This assumes that libsqlite.so is in your current path: . (thus the -L.) >> >> > > Martin and Jay, thanks for the info. > > The hint helped be to track the problem. > > It appears that my problem was that I forgot to remove dynamic linking of > libsqlite3.so library and had my own dynamic loading of another sqlite > binary. Since the exceptions was when sqlite accessed sqlite global config > (sqlite3GlobalConfig), I suppose they were probably magically sharing the > global data or something like that. > > I know that I should avoid such things, but just curious, is it something > inside sqlite that probably makes one dynanmically linked and other > dynamically loaded library share global data and can this be avoidable? I > thought that two libraries having different filenames and sonames virtually > different for the system, but it looks like they're not. > > Thanks > > Max > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Dynamically loaded sqlite (linux)
On Tue, Jun 7, 2011 at 9:22 PM, Martin Gadbois wrote: > On Tue, Jun 7, 2011 at 12:52 PM, Jay A. Kreibich wrote: > > > On Tue, Jun 07, 2011 at 07:47:25PM +0400, Max Vlasov scratched on the > wall: > > > Hi, > > > > > > I'm trying to use sqlite with linux (Ubuntu, Pascal, Lazarus). I'm > still > > not > > > very familiar with linux development so I might miss something > essential. > > > > > > Two scenarios work ok > > > - statically linked latest version compiled (3.7.6.3), no options or > > defines > > > changed > > > - Dynamically loaded (dlopen) sqlite used from the installed package > > > libsqlite3 (libsqlite3.so) > > > > That's not how dynamic libraries work (not normally, anyways). > > Generally you simply tell the compiler/linker to link in the library > > at build time, and allow the usage of dynamic libs. The dynamic > > link is then done on application start-up by the OS. In Windows > > terms, it is like using an .DLL by linking in the associated .lib > > file. Moving from a static library to a dynamic library requires no > > code changes. > > > > > There is a way to do a _good_ shared library. I suggest reading the > excellent paper: http://www.akkadia.org/drepper/dsohowto.pdf > > As for the OP question, do > gcc -shared -Wl,-init=sqlite3_initialize -o libsqlite.so sqlite3.o > > and then link your application with > gcc -L. -lsqlite -o test test.c > > This assumes that libsqlite.so is in your current path: . (thus the -L.) > > Martin and Jay, thanks for the info. The hint helped be to track the problem. It appears that my problem was that I forgot to remove dynamic linking of libsqlite3.so library and had my own dynamic loading of another sqlite binary. Since the exceptions was when sqlite accessed sqlite global config (sqlite3GlobalConfig), I suppose they were probably magically sharing the global data or something like that. I know that I should avoid such things, but just curious, is it something inside sqlite that probably makes one dynanmically linked and other dynamically loaded library share global data and can this be avoidable? I thought that two libraries having different filenames and sonames virtually different for the system, but it looks like they're not. Thanks Max ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Dynamically loaded sqlite (linux)
On Tue, Jun 7, 2011 at 12:52 PM, Jay A. Kreibich wrote: > On Tue, Jun 07, 2011 at 07:47:25PM +0400, Max Vlasov scratched on the wall: > > Hi, > > > > I'm trying to use sqlite with linux (Ubuntu, Pascal, Lazarus). I'm still > not > > very familiar with linux development so I might miss something essential. > > > > Two scenarios work ok > > - statically linked latest version compiled (3.7.6.3), no options or > defines > > changed > > - Dynamically loaded (dlopen) sqlite used from the installed package > > libsqlite3 (libsqlite3.so) > > That's not how dynamic libraries work (not normally, anyways). > Generally you simply tell the compiler/linker to link in the library > at build time, and allow the usage of dynamic libs. The dynamic > link is then done on application start-up by the OS. In Windows > terms, it is like using an .DLL by linking in the associated .lib > file. Moving from a static library to a dynamic library requires no > code changes. > There is a way to do a _good_ shared library. I suggest reading the excellent paper: http://www.akkadia.org/drepper/dsohowto.pdf As for the OP question, do gcc -shared -Wl,-init=sqlite3_initialize -o libsqlite.so sqlite3.o and then link your application with gcc -L. -lsqlite -o test test.c This assumes that libsqlite.so is in your current path: . (thus the -L.) -- Martin ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Dynamically loaded sqlite (linux)
On Tue, Jun 07, 2011 at 07:47:25PM +0400, Max Vlasov scratched on the wall: > Hi, > > I'm trying to use sqlite with linux (Ubuntu, Pascal, Lazarus). I'm still not > very familiar with linux development so I might miss something essential. > > Two scenarios work ok > - statically linked latest version compiled (3.7.6.3), no options or defines > changed > - Dynamically loaded (dlopen) sqlite used from the installed package > libsqlite3 (libsqlite3.so) That's not how dynamic libraries work (not normally, anyways). Generally you simply tell the compiler/linker to link in the library at build time, and allow the usage of dynamic libs. The dynamic link is then done on application start-up by the OS. In Windows terms, it is like using an .DLL by linking in the associated .lib file. Moving from a static library to a dynamic library requires no code changes. The dlopen() and related functions are for application controlled linking. They're like the LoadLibrary() functions under Windows. -j -- Jay A. Kreibich < J A Y @ K R E I B I.C H > "Intelligence is like underwear: it is important that you have it, but showing it to the wrong people has the tendency to make them feel uncomfortable." -- Angela Johnson ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Dynamically loaded sqlite (linux)
Hi, I'm trying to use sqlite with linux (Ubuntu, Pascal, Lazarus). I'm still not very familiar with linux development so I might miss something essential. Two scenarios work ok - statically linked latest version compiled (3.7.6.3), no options or defines changed - Dynamically loaded (dlopen) sqlite used from the installed package libsqlite3 (libsqlite3.so) But when I try to use shared library compiled from the same sources and use it with dlopen some errors appear. - If I just compile without mapping sqlite3_initialize to init proc, first call to sqlite3GlobalConfig.m.xRoundup(n) in mallocWithAlarm gives SIGSEGV error. - If I map sqlite3_initialize to init, the same happen with the first call to sqlite3_mutex_enter call. The library is build with gcc -g -c -fPIC sqlite3.c gcc -shared -Wl,-init=sqlite3_initialize -o customsqlite.so sqlite3.o What am I missing here? Thanks, Max ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
