Re: [sqlite] SQLITE_CORRUPT recover
"Sabyasachi Ruj" <[EMAIL PROTECTED]> wrote: > I still fail to understand what should I synchronize on. I am *not* sharing > sqlite* across multiple threads. > If you compile SQLite so that it is threadsafe (-DTHREADSAFE=1) and if you do not share sqlite3* pointers across threads, then you should not have to do any synchronization. If you are failing to synchronize threads correctly, then you should expect to see SQLITE_MISUSE errors, not SQLITE_CORRUPT errors. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLITE_CORRUPT recover
You said that you were sharing Sqlite between threads by opening a connection in each thread. Sqlite is a single resource and must be sync'd somehow if it has multiple users. Complaining about it is like complaining that the sky is blue. It is blue because of Rayleigh scattering, a fundamental phenomenon which cannot be changed by wishes. You just have to get used to it. We successfully share Sqlite between threads. Sometimes we use a connection per thread and sometimes a single connection. The single connection is optimal because it enhances the cache usage. We sync with mutexes. The logic is very simple. Sabyasachi Ruj wrote: I still fail to understand what should I synchronize on. I am *not* sharing sqlite* across multiple threads. On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: It is fundamental computer science, CS101 you might say. Pick up a textbook on basic computing. Sabyasachi Ruj wrote: > But can you tell me where is this documented please? > > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > >> >> If you knew the answer then why did you ask the question? You can apply >> regular CS or look for miracles. >> >> You might be better off using a server based RDBMS like Oracle or >> PostgreSQL where the application programmer is insulated from >> synchronization issues. Sqlite has the "lite" in its name for a very >> good reason. It is designed for embedded applications, not as an >> enterprise DBMS. >> >> Sabyasachi Ruj wrote: >> > But the following link >> > http://www.sqlite.org/cvstrac/wiki/wiki?p=MultiThreading >> > says nothing that I have to synchronize at the application level to >> create >> > multiple connections, until the same database connection is being >> shared! >> > Ref: The four points in 'Short Answer' section. >> > >> > BTW: for a DBMS it does not make sense if the application programmer >> has >> to >> > synchronize to create multiple connection. >> > And synhing will have considerable performance drop also. >> > >> > >> > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: >> > >> >> >> >> Threadsafe only means that threads do not access global data elements >> or >> >> that they synchronize (serialize) access to global data. It does >> >> nothing to synchronize threads. That is up to the application >> >> programmer. Sqlite uses POSIX file locks for synchronixation but if >> you >> >> are in a totally threaded environment you can use thread sync >> functions >> >> like mutexes or the finer grained read and write lock thread >> primitives. >> >> >> >> If you are accessing Sqlite across a network file locks are the way to >> >> go, but do depend upon network implementations and settings. If you >> >> have multiple processes on one OS you can sync using semaphores. >> >> >> >> Using textbook style synchronization ensures that you have minimal >> >> problems and optimal performance. >> >> >> >> Sabyasachi Ruj wrote: >> >> > But I think we do not have to take care of synchronizing sqlite >> access. >> >> > sqlite internally does if it is compiled with THREADSAFE=1. >> >> > >> >> > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: >> >> > >> >> >> >> >> >> The problem is fairly straight forward. Sqlite is a single >> resource >> >> >> being shared by multiple thyreads so you just use fundamental >> >> >> synchronization logic as you would when sharing any resource >> between >> >> >> competing threads. >> >> >> >> >> >> Sabyasachi Ruj wrote: >> >> >> > Hi, >> >> >> > >> >> >> > I am using sqlite in a multithreaded environment. I have take >> >> >> > reasonable like not sharing sqlite* handles. I am creating a new >> >> >> sqlite* >> >> >> >> >> >> > for >> >> >> > every thread. >> >> >> > >> >> >> > Where can we get more info on working with SQLite in a >> multithreaded >> >> >> > environment? >> >> >> > >> >> >> > The application is working as a service in windows. >> >> >> > >> >> >> > sqlite3_step() is failing with the following error message: >> >> >> > >> >> >> > * SQL error or missing database >> >> >> > SQL logic error or missing database* >> >> >> > >> >> >> > I am getting this message after running the application for quite >> a >> >> >> long >> >> >> > time (few days). >> >> >> > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I >> get >> >> the >> >> >> > same >> >> >> > error message. >> >> >> > >> >> >> > My application updates the database very 2 mins and the >> corruption >> >> >> happend >> >> >> > randomly >> >> >> > >> >> >> > I dont have any clue how to debug this! >> >> >> > >> >> >> > Thanks. >> >> >> > >> >> >> > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >> >> > >> >> >> >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> >> >> >> > Hi, >> >> >> >> > Is there any way to programmatically fix a corrupted sqlite >> >> >> database? >> >> >> >> > I am using sqlite version 3.3.8 with C APIs >> >> >> >> > >> >> >> >> >> >> >> >> Sometimes VACUUM or REINDEX will help, but usually not. >>
Re: [sqlite] SQLITE_CORRUPT recover
- Make sure you're compiling SQLite with *-DTHREADSAFE=1*. - Make sure that each thread opens the database file and keeps its own sqlite structure. - Make sure you handle the likely possibility that one or more threads collide when they access the db file at the same time: handle * SQLITE_BUSY* appropriately. - Make sure you enclose within transactions the commands that modify the database file, like *INSERT*, *UPDATE*, *DELETE*, and others. Whenever I get the SQLITE_MISUSE or SQLITE_BUSY errors when "it should work", I discover that I have a statement left over in execution mode without it having been sqlite3_reset(...). Perhaps the same is true in your case. andy On 6/18/07, Sabyasachi Ruj <[EMAIL PROTECTED]> wrote: But the following link http://www.sqlite.org/cvstrac/wiki/wiki?p=MultiThreading says nothing that I have to synchronize at the application level to create multiple connections, until the same database connection is being shared! Ref: The four points in 'Short Answer' section. BTW: for a DBMS it does not make sense if the application programmer has to synchronize to create multiple connection. And synhing will have considerable performance drop also. On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > > Threadsafe only means that threads do not access global data elements or > that they synchronize (serialize) access to global data. It does > nothing to synchronize threads. That is up to the application > programmer. Sqlite uses POSIX file locks for synchronixation but if you > are in a totally threaded environment you can use thread sync functions > like mutexes or the finer grained read and write lock thread primitives. > > If you are accessing Sqlite across a network file locks are the way to > go, but do depend upon network implementations and settings. If you > have multiple processes on one OS you can sync using semaphores. > > Using textbook style synchronization ensures that you have minimal > problems and optimal performance. > > Sabyasachi Ruj wrote: > > But I think we do not have to take care of synchronizing sqlite access. > > sqlite internally does if it is compiled with THREADSAFE=1. > > > > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > > > >> > >> The problem is fairly straight forward. Sqlite is a single resource > >> being shared by multiple thyreads so you just use fundamental > >> synchronization logic as you would when sharing any resource between > >> competing threads. > >> > >> Sabyasachi Ruj wrote: > >> > Hi, > >> > > >> > I am using sqlite in a multithreaded environment. I have take > >> > reasonable like not sharing sqlite* handles. I am creating a new > >> sqlite* > >> > >> > for > >> > every thread. > >> > > >> > Where can we get more info on working with SQLite in a multithreaded > >> > environment? > >> > > >> > The application is working as a service in windows. > >> > > >> > sqlite3_step() is failing with the following error message: > >> > > >> > * SQL error or missing database > >> > SQL logic error or missing database* > >> > > >> > I am getting this message after running the application for quite a > >> long > >> > time (few days). > >> > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get > the > >> > same > >> > error message. > >> > > >> > My application updates the database very 2 mins and the corruption > >> happend > >> > randomly > >> > > >> > I dont have any clue how to debug this! > >> > > >> > Thanks. > >> > > >> > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > >> > > >> >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: > >> >> > Hi, > >> >> > Is there any way to programmatically fix a corrupted sqlite > >> database? > >> >> > I am using sqlite version 3.3.8 with C APIs > >> >> > > >> >> > >> >> Sometimes VACUUM or REINDEX will help, but usually not. > >> >> You can also try to recover using: > >> >> > >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB > >> >> > >> >> But that doesn't always work either. The best approach > >> >> is to avoid corruption in the first place. > >> >> -- > >> >> D. Richard Hipp <[EMAIL PROTECTED]> > >> >> > >> >> > >> >> > >> >> > >> > - > >> > >> > >> >> > >> >> To unsubscribe, send email to [EMAIL PROTECTED] > >> >> > >> >> > >> > - > >> > >> > >> >> > >> >> > >> >> > >> > > >> > > >> > >> > >> > >> > - > >> > >> To unsubscribe, send email to [EMAIL PROTECTED] > >> > >> > - > >> > >> > >> > > > > > > > > - > To unsubscribe, send email to [EMAIL PROTECTED] > > - > > -- Sabyasachi
Re: [sqlite] SQLITE_CORRUPT recover
I still fail to understand what should I synchronize on. I am *not* sharing sqlite* across multiple threads. On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: It is fundamental computer science, CS101 you might say. Pick up a textbook on basic computing. Sabyasachi Ruj wrote: > But can you tell me where is this documented please? > > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > >> >> If you knew the answer then why did you ask the question? You can apply >> regular CS or look for miracles. >> >> You might be better off using a server based RDBMS like Oracle or >> PostgreSQL where the application programmer is insulated from >> synchronization issues. Sqlite has the "lite" in its name for a very >> good reason. It is designed for embedded applications, not as an >> enterprise DBMS. >> >> Sabyasachi Ruj wrote: >> > But the following link >> > http://www.sqlite.org/cvstrac/wiki/wiki?p=MultiThreading >> > says nothing that I have to synchronize at the application level to >> create >> > multiple connections, until the same database connection is being >> shared! >> > Ref: The four points in 'Short Answer' section. >> > >> > BTW: for a DBMS it does not make sense if the application programmer >> has >> to >> > synchronize to create multiple connection. >> > And synhing will have considerable performance drop also. >> > >> > >> > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: >> > >> >> >> >> Threadsafe only means that threads do not access global data elements >> or >> >> that they synchronize (serialize) access to global data. It does >> >> nothing to synchronize threads. That is up to the application >> >> programmer. Sqlite uses POSIX file locks for synchronixation but if >> you >> >> are in a totally threaded environment you can use thread sync >> functions >> >> like mutexes or the finer grained read and write lock thread >> primitives. >> >> >> >> If you are accessing Sqlite across a network file locks are the way to >> >> go, but do depend upon network implementations and settings. If you >> >> have multiple processes on one OS you can sync using semaphores. >> >> >> >> Using textbook style synchronization ensures that you have minimal >> >> problems and optimal performance. >> >> >> >> Sabyasachi Ruj wrote: >> >> > But I think we do not have to take care of synchronizing sqlite >> access. >> >> > sqlite internally does if it is compiled with THREADSAFE=1. >> >> > >> >> > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: >> >> > >> >> >> >> >> >> The problem is fairly straight forward. Sqlite is a single >> resource >> >> >> being shared by multiple thyreads so you just use fundamental >> >> >> synchronization logic as you would when sharing any resource >> between >> >> >> competing threads. >> >> >> >> >> >> Sabyasachi Ruj wrote: >> >> >> > Hi, >> >> >> > >> >> >> > I am using sqlite in a multithreaded environment. I have take >> >> >> > reasonable like not sharing sqlite* handles. I am creating a new >> >> >> sqlite* >> >> >> >> >> >> > for >> >> >> > every thread. >> >> >> > >> >> >> > Where can we get more info on working with SQLite in a >> multithreaded >> >> >> > environment? >> >> >> > >> >> >> > The application is working as a service in windows. >> >> >> > >> >> >> > sqlite3_step() is failing with the following error message: >> >> >> > >> >> >> > * SQL error or missing database >> >> >> > SQL logic error or missing database* >> >> >> > >> >> >> > I am getting this message after running the application for quite >> a >> >> >> long >> >> >> > time (few days). >> >> >> > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I >> get >> >> the >> >> >> > same >> >> >> > error message. >> >> >> > >> >> >> > My application updates the database very 2 mins and the >> corruption >> >> >> happend >> >> >> > randomly >> >> >> > >> >> >> > I dont have any clue how to debug this! >> >> >> > >> >> >> > Thanks. >> >> >> > >> >> >> > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >> >> > >> >> >> >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> >> >> >> > Hi, >> >> >> >> > Is there any way to programmatically fix a corrupted sqlite >> >> >> database? >> >> >> >> > I am using sqlite version 3.3.8 with C APIs >> >> >> >> > >> >> >> >> >> >> >> >> Sometimes VACUUM or REINDEX will help, but usually not. >> >> >> >> You can also try to recover using: >> >> >> >> >> >> >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB >> >> >> >> >> >> >> >> But that doesn't always work either. The best approach >> >> >> >> is to avoid corruption in the first place. >> >> >> >> -- >> >> >> >> D. Richard Hipp <[EMAIL PROTECTED]> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> - >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> To unsubscribe, send email to >> [EMAIL PROTECTED] >> >> >> >> >> >> >> >> >> >> >> >> >> >>
Re: [sqlite] SQLITE_CORRUPT recover
It is fundamental computer science, CS101 you might say. Pick up a textbook on basic computing. Sabyasachi Ruj wrote: But can you tell me where is this documented please? On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: If you knew the answer then why did you ask the question? You can apply regular CS or look for miracles. You might be better off using a server based RDBMS like Oracle or PostgreSQL where the application programmer is insulated from synchronization issues. Sqlite has the "lite" in its name for a very good reason. It is designed for embedded applications, not as an enterprise DBMS. Sabyasachi Ruj wrote: > But the following link > http://www.sqlite.org/cvstrac/wiki/wiki?p=MultiThreading > says nothing that I have to synchronize at the application level to create > multiple connections, until the same database connection is being shared! > Ref: The four points in 'Short Answer' section. > > BTW: for a DBMS it does not make sense if the application programmer has to > synchronize to create multiple connection. > And synhing will have considerable performance drop also. > > > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > >> >> Threadsafe only means that threads do not access global data elements or >> that they synchronize (serialize) access to global data. It does >> nothing to synchronize threads. That is up to the application >> programmer. Sqlite uses POSIX file locks for synchronixation but if you >> are in a totally threaded environment you can use thread sync functions >> like mutexes or the finer grained read and write lock thread primitives. >> >> If you are accessing Sqlite across a network file locks are the way to >> go, but do depend upon network implementations and settings. If you >> have multiple processes on one OS you can sync using semaphores. >> >> Using textbook style synchronization ensures that you have minimal >> problems and optimal performance. >> >> Sabyasachi Ruj wrote: >> > But I think we do not have to take care of synchronizing sqlite access. >> > sqlite internally does if it is compiled with THREADSAFE=1. >> > >> > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: >> > >> >> >> >> The problem is fairly straight forward. Sqlite is a single resource >> >> being shared by multiple thyreads so you just use fundamental >> >> synchronization logic as you would when sharing any resource between >> >> competing threads. >> >> >> >> Sabyasachi Ruj wrote: >> >> > Hi, >> >> > >> >> > I am using sqlite in a multithreaded environment. I have take >> >> > reasonable like not sharing sqlite* handles. I am creating a new >> >> sqlite* >> >> >> >> > for >> >> > every thread. >> >> > >> >> > Where can we get more info on working with SQLite in a multithreaded >> >> > environment? >> >> > >> >> > The application is working as a service in windows. >> >> > >> >> > sqlite3_step() is failing with the following error message: >> >> > >> >> > * SQL error or missing database >> >> > SQL logic error or missing database* >> >> > >> >> > I am getting this message after running the application for quite a >> >> long >> >> > time (few days). >> >> > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get >> the >> >> > same >> >> > error message. >> >> > >> >> > My application updates the database very 2 mins and the corruption >> >> happend >> >> > randomly >> >> > >> >> > I dont have any clue how to debug this! >> >> > >> >> > Thanks. >> >> > >> >> > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >> > >> >> >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> >> >> > Hi, >> >> >> > Is there any way to programmatically fix a corrupted sqlite >> >> database? >> >> >> > I am using sqlite version 3.3.8 with C APIs >> >> >> > >> >> >> >> >> >> Sometimes VACUUM or REINDEX will help, but usually not. >> >> >> You can also try to recover using: >> >> >> >> >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB >> >> >> >> >> >> But that doesn't always work either. The best approach >> >> >> is to avoid corruption in the first place. >> >> >> -- >> >> >> D. Richard Hipp <[EMAIL PROTECTED]> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> - >> >> >> >> >> >> >> >> >> >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> >> >> >> >> >> >> >> - >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> > >> >> >> >> >> >> >> >> >> - >> >> >> >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> >> >> >> - >> >> >> >> >> >> >> >> > >> > >> >> >> >> - >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >>
Re: [sqlite] SQLITE_CORRUPT recover
But can you tell me where is this documented please? On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: If you knew the answer then why did you ask the question? You can apply regular CS or look for miracles. You might be better off using a server based RDBMS like Oracle or PostgreSQL where the application programmer is insulated from synchronization issues. Sqlite has the "lite" in its name for a very good reason. It is designed for embedded applications, not as an enterprise DBMS. Sabyasachi Ruj wrote: > But the following link > http://www.sqlite.org/cvstrac/wiki/wiki?p=MultiThreading > says nothing that I have to synchronize at the application level to create > multiple connections, until the same database connection is being shared! > Ref: The four points in 'Short Answer' section. > > BTW: for a DBMS it does not make sense if the application programmer has to > synchronize to create multiple connection. > And synhing will have considerable performance drop also. > > > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > >> >> Threadsafe only means that threads do not access global data elements or >> that they synchronize (serialize) access to global data. It does >> nothing to synchronize threads. That is up to the application >> programmer. Sqlite uses POSIX file locks for synchronixation but if you >> are in a totally threaded environment you can use thread sync functions >> like mutexes or the finer grained read and write lock thread primitives. >> >> If you are accessing Sqlite across a network file locks are the way to >> go, but do depend upon network implementations and settings. If you >> have multiple processes on one OS you can sync using semaphores. >> >> Using textbook style synchronization ensures that you have minimal >> problems and optimal performance. >> >> Sabyasachi Ruj wrote: >> > But I think we do not have to take care of synchronizing sqlite access. >> > sqlite internally does if it is compiled with THREADSAFE=1. >> > >> > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: >> > >> >> >> >> The problem is fairly straight forward. Sqlite is a single resource >> >> being shared by multiple thyreads so you just use fundamental >> >> synchronization logic as you would when sharing any resource between >> >> competing threads. >> >> >> >> Sabyasachi Ruj wrote: >> >> > Hi, >> >> > >> >> > I am using sqlite in a multithreaded environment. I have take >> >> > reasonable like not sharing sqlite* handles. I am creating a new >> >> sqlite* >> >> >> >> > for >> >> > every thread. >> >> > >> >> > Where can we get more info on working with SQLite in a multithreaded >> >> > environment? >> >> > >> >> > The application is working as a service in windows. >> >> > >> >> > sqlite3_step() is failing with the following error message: >> >> > >> >> > * SQL error or missing database >> >> > SQL logic error or missing database* >> >> > >> >> > I am getting this message after running the application for quite a >> >> long >> >> > time (few days). >> >> > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get >> the >> >> > same >> >> > error message. >> >> > >> >> > My application updates the database very 2 mins and the corruption >> >> happend >> >> > randomly >> >> > >> >> > I dont have any clue how to debug this! >> >> > >> >> > Thanks. >> >> > >> >> > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >> > >> >> >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> >> >> > Hi, >> >> >> > Is there any way to programmatically fix a corrupted sqlite >> >> database? >> >> >> > I am using sqlite version 3.3.8 with C APIs >> >> >> > >> >> >> >> >> >> Sometimes VACUUM or REINDEX will help, but usually not. >> >> >> You can also try to recover using: >> >> >> >> >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB >> >> >> >> >> >> But that doesn't always work either. The best approach >> >> >> is to avoid corruption in the first place. >> >> >> -- >> >> >> D. Richard Hipp <[EMAIL PROTECTED]> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> - >> >> >> >> >> >> >> >> >> >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> >> >> >> >> >> >> >> - >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> > >> >> >> >> >> >> >> >> >> - >> >> >> >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> >> >> >> - >> >> >> >> >> >> >> >> > >> > >> >> >> >> - >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> - >> >> >> > > - To unsubscribe, send
Re: [sqlite] SQLITE_CORRUPT recover
If you knew the answer then why did you ask the question? You can apply regular CS or look for miracles. You might be better off using a server based RDBMS like Oracle or PostgreSQL where the application programmer is insulated from synchronization issues. Sqlite has the "lite" in its name for a very good reason. It is designed for embedded applications, not as an enterprise DBMS. Sabyasachi Ruj wrote: But the following link http://www.sqlite.org/cvstrac/wiki/wiki?p=MultiThreading says nothing that I have to synchronize at the application level to create multiple connections, until the same database connection is being shared! Ref: The four points in 'Short Answer' section. BTW: for a DBMS it does not make sense if the application programmer has to synchronize to create multiple connection. And synhing will have considerable performance drop also. On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: Threadsafe only means that threads do not access global data elements or that they synchronize (serialize) access to global data. It does nothing to synchronize threads. That is up to the application programmer. Sqlite uses POSIX file locks for synchronixation but if you are in a totally threaded environment you can use thread sync functions like mutexes or the finer grained read and write lock thread primitives. If you are accessing Sqlite across a network file locks are the way to go, but do depend upon network implementations and settings. If you have multiple processes on one OS you can sync using semaphores. Using textbook style synchronization ensures that you have minimal problems and optimal performance. Sabyasachi Ruj wrote: > But I think we do not have to take care of synchronizing sqlite access. > sqlite internally does if it is compiled with THREADSAFE=1. > > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > >> >> The problem is fairly straight forward. Sqlite is a single resource >> being shared by multiple thyreads so you just use fundamental >> synchronization logic as you would when sharing any resource between >> competing threads. >> >> Sabyasachi Ruj wrote: >> > Hi, >> > >> > I am using sqlite in a multithreaded environment. I have take >> > reasonable like not sharing sqlite* handles. I am creating a new >> sqlite* >> >> > for >> > every thread. >> > >> > Where can we get more info on working with SQLite in a multithreaded >> > environment? >> > >> > The application is working as a service in windows. >> > >> > sqlite3_step() is failing with the following error message: >> > >> > * SQL error or missing database >> > SQL logic error or missing database* >> > >> > I am getting this message after running the application for quite a >> long >> > time (few days). >> > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get the >> > same >> > error message. >> > >> > My application updates the database very 2 mins and the corruption >> happend >> > randomly >> > >> > I dont have any clue how to debug this! >> > >> > Thanks. >> > >> > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> > >> >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> >> > Hi, >> >> > Is there any way to programmatically fix a corrupted sqlite >> database? >> >> > I am using sqlite version 3.3.8 with C APIs >> >> > >> >> >> >> Sometimes VACUUM or REINDEX will help, but usually not. >> >> You can also try to recover using: >> >> >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB >> >> >> >> But that doesn't always work either. The best approach >> >> is to avoid corruption in the first place. >> >> -- >> >> D. Richard Hipp <[EMAIL PROTECTED]> >> >> >> >> >> >> >> >> >> - >> >> >> >> >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> >> >> >> - >> >> >> >> >> >> >> >> >> > >> > >> >> >> >> - >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> - >> >> >> > > - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLITE_CORRUPT recover
But the following link http://www.sqlite.org/cvstrac/wiki/wiki?p=MultiThreading says nothing that I have to synchronize at the application level to create multiple connections, until the same database connection is being shared! Ref: The four points in 'Short Answer' section. BTW: for a DBMS it does not make sense if the application programmer has to synchronize to create multiple connection. And synhing will have considerable performance drop also. On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: Threadsafe only means that threads do not access global data elements or that they synchronize (serialize) access to global data. It does nothing to synchronize threads. That is up to the application programmer. Sqlite uses POSIX file locks for synchronixation but if you are in a totally threaded environment you can use thread sync functions like mutexes or the finer grained read and write lock thread primitives. If you are accessing Sqlite across a network file locks are the way to go, but do depend upon network implementations and settings. If you have multiple processes on one OS you can sync using semaphores. Using textbook style synchronization ensures that you have minimal problems and optimal performance. Sabyasachi Ruj wrote: > But I think we do not have to take care of synchronizing sqlite access. > sqlite internally does if it is compiled with THREADSAFE=1. > > On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: > >> >> The problem is fairly straight forward. Sqlite is a single resource >> being shared by multiple thyreads so you just use fundamental >> synchronization logic as you would when sharing any resource between >> competing threads. >> >> Sabyasachi Ruj wrote: >> > Hi, >> > >> > I am using sqlite in a multithreaded environment. I have take >> > reasonable like not sharing sqlite* handles. I am creating a new >> sqlite* >> >> > for >> > every thread. >> > >> > Where can we get more info on working with SQLite in a multithreaded >> > environment? >> > >> > The application is working as a service in windows. >> > >> > sqlite3_step() is failing with the following error message: >> > >> > * SQL error or missing database >> > SQL logic error or missing database* >> > >> > I am getting this message after running the application for quite a >> long >> > time (few days). >> > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get the >> > same >> > error message. >> > >> > My application updates the database very 2 mins and the corruption >> happend >> > randomly >> > >> > I dont have any clue how to debug this! >> > >> > Thanks. >> > >> > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> > >> >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> >> > Hi, >> >> > Is there any way to programmatically fix a corrupted sqlite >> database? >> >> > I am using sqlite version 3.3.8 with C APIs >> >> > >> >> >> >> Sometimes VACUUM or REINDEX will help, but usually not. >> >> You can also try to recover using: >> >> >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB >> >> >> >> But that doesn't always work either. The best approach >> >> is to avoid corruption in the first place. >> >> -- >> >> D. Richard Hipp <[EMAIL PROTECTED]> >> >> >> >> >> >> >> >> >> - >> >> >> >> >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> >> >> >> - >> >> >> >> >> >> >> >> >> > >> > >> >> >> >> - >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> - >> >> >> > > - To unsubscribe, send email to [EMAIL PROTECTED] - -- Sabyasachi
Re: [sqlite] SQLITE_CORRUPT recover
Threadsafe only means that threads do not access global data elements or that they synchronize (serialize) access to global data. It does nothing to synchronize threads. That is up to the application programmer. Sqlite uses POSIX file locks for synchronixation but if you are in a totally threaded environment you can use thread sync functions like mutexes or the finer grained read and write lock thread primitives. If you are accessing Sqlite across a network file locks are the way to go, but do depend upon network implementations and settings. If you have multiple processes on one OS you can sync using semaphores. Using textbook style synchronization ensures that you have minimal problems and optimal performance. Sabyasachi Ruj wrote: But I think we do not have to take care of synchronizing sqlite access. sqlite internally does if it is compiled with THREADSAFE=1. On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: The problem is fairly straight forward. Sqlite is a single resource being shared by multiple thyreads so you just use fundamental synchronization logic as you would when sharing any resource between competing threads. Sabyasachi Ruj wrote: > Hi, > > I am using sqlite in a multithreaded environment. I have take > reasonable like not sharing sqlite* handles. I am creating a new sqlite* > for > every thread. > > Where can we get more info on working with SQLite in a multithreaded > environment? > > The application is working as a service in windows. > > sqlite3_step() is failing with the following error message: > > * SQL error or missing database > SQL logic error or missing database* > > I am getting this message after running the application for quite a long > time (few days). > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get the > same > error message. > > My application updates the database very 2 mins and the corruption happend > randomly > > I dont have any clue how to debug this! > > Thanks. > > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> > Hi, >> > Is there any way to programmatically fix a corrupted sqlite database? >> > I am using sqlite version 3.3.8 with C APIs >> > >> >> Sometimes VACUUM or REINDEX will help, but usually not. >> You can also try to recover using: >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB >> >> But that doesn't always work either. The best approach >> is to avoid corruption in the first place. >> -- >> D. Richard Hipp <[EMAIL PROTECTED]> >> >> >> >> - >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> - >> >> >> > > - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLITE_CORRUPT recover
But I think we do not have to take care of synchronizing sqlite access. sqlite internally does if it is compiled with THREADSAFE=1. On 6/18/07, John Stanton <[EMAIL PROTECTED]> wrote: The problem is fairly straight forward. Sqlite is a single resource being shared by multiple thyreads so you just use fundamental synchronization logic as you would when sharing any resource between competing threads. Sabyasachi Ruj wrote: > Hi, > > I am using sqlite in a multithreaded environment. I have take > reasonable like not sharing sqlite* handles. I am creating a new sqlite* > for > every thread. > > Where can we get more info on working with SQLite in a multithreaded > environment? > > The application is working as a service in windows. > > sqlite3_step() is failing with the following error message: > > * SQL error or missing database > SQL logic error or missing database* > > I am getting this message after running the application for quite a long > time (few days). > And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get the > same > error message. > > My application updates the database very 2 mins and the corruption happend > randomly > > I dont have any clue how to debug this! > > Thanks. > > On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > >> "Sabyasachi Ruj" < [EMAIL PROTECTED]> wrote: >> > Hi, >> > Is there any way to programmatically fix a corrupted sqlite database? >> > I am using sqlite version 3.3.8 with C APIs >> > >> >> Sometimes VACUUM or REINDEX will help, but usually not. >> You can also try to recover using: >> >>sqlite3 OLD.DB .dump | sqlite3 NEW.DB >> >> But that doesn't always work either. The best approach >> is to avoid corruption in the first place. >> -- >> D. Richard Hipp <[EMAIL PROTECTED]> >> >> >> >> - >> >> To unsubscribe, send email to [EMAIL PROTECTED] >> >> - >> >> >> > > - To unsubscribe, send email to [EMAIL PROTECTED] - -- Sabyasachi
Re: [sqlite] SQLITE_CORRUPT recover
The problem is fairly straight forward. Sqlite is a single resource being shared by multiple thyreads so you just use fundamental synchronization logic as you would when sharing any resource between competing threads. Sabyasachi Ruj wrote: Hi, I am using sqlite in a multithreaded environment. I have take reasonable like not sharing sqlite* handles. I am creating a new sqlite* for every thread. Where can we get more info on working with SQLite in a multithreaded environment? The application is working as a service in windows. sqlite3_step() is failing with the following error message: * SQL error or missing database SQL logic error or missing database* I am getting this message after running the application for quite a long time (few days). And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get the same error message. My application updates the database very 2 mins and the corruption happend randomly I dont have any clue how to debug this! Thanks. On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: "Sabyasachi Ruj" <[EMAIL PROTECTED]> wrote: > Hi, > Is there any way to programmatically fix a corrupted sqlite database? > I am using sqlite version 3.3.8 with C APIs > Sometimes VACUUM or REINDEX will help, but usually not. You can also try to recover using: sqlite3 OLD.DB .dump | sqlite3 NEW.DB But that doesn't always work either. The best approach is to avoid corruption in the first place. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLITE_CORRUPT recover
Hi, I am using sqlite in a multithreaded environment. I have take reasonable like not sharing sqlite* handles. I am creating a new sqlite* for every thread. Where can we get more info on working with SQLite in a multithreaded environment? The application is working as a service in windows. sqlite3_step() is failing with the following error message: * SQL error or missing database SQL logic error or missing database* I am getting this message after running the application for quite a long time (few days). And then if I execute *PRAGMA INTEGRITY_CHECK* on that table, I get the same error message. My application updates the database very 2 mins and the corruption happend randomly I dont have any clue how to debug this! Thanks. On 5/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: "Sabyasachi Ruj" <[EMAIL PROTECTED]> wrote: > Hi, > Is there any way to programmatically fix a corrupted sqlite database? > I am using sqlite version 3.3.8 with C APIs > Sometimes VACUUM or REINDEX will help, but usually not. You can also try to recover using: sqlite3 OLD.DB .dump | sqlite3 NEW.DB But that doesn't always work either. The best approach is to avoid corruption in the first place. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] - -- Sabyasachi
Re: [sqlite] SQLITE_CORRUPT recover
"Sabyasachi Ruj" <[EMAIL PROTECTED]> wrote: > Hi, > Is there any way to programmatically fix a corrupted sqlite database? > I am using sqlite version 3.3.8 with C APIs > Sometimes VACUUM or REINDEX will help, but usually not. You can also try to recover using: sqlite3 OLD.DB .dump | sqlite3 NEW.DB But that doesn't always work either. The best approach is to avoid corruption in the first place. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] SQLITE_CORRUPT recover
Hi, Is there any way to programmatically fix a corrupted sqlite database? I am using sqlite version 3.3.8 with C APIs -- Sabyasachi
