RE: [sqlite] Problems with multiple threads?

2006-06-08 Thread Pat Wibbeler
: [sqlite] Problems with multiple threads? * Pat Wibbeler <[EMAIL PROTECTED]> [2006-06-07 22:55]: > It's entirely possible I'm reading these docs incorrectly, but > this strategy has worked quite well for me. No, I don't see any error in your reading. My apologies; I should have consulted the

Re: [sqlite] Problems with multiple threads?

2006-06-08 Thread drh
"Kervin L. Pierre" <[EMAIL PROTECTED]> wrote: > Hello, > > I was under the impress that we could never > get an SQLITE_BUSY, not even on COMMIT if > we use BEGIN EXCLUSIVE. But this seems to > say that COMMITs on exclusive transactions > can through SQLITE_BUSY?... > You can get an SQLITE_BUSY

Re: [sqlite] Problems with multiple threads?

2006-06-08 Thread Kervin L. Pierre
Hello, I was under the impress that we could never get an SQLITE_BUSY, not even on COMMIT if we use BEGIN EXCLUSIVE. But this seems to say that COMMITs on exclusive transactions can through SQLITE_BUSY?... --- [EMAIL PROTECTED] wrote: > then start the transaction initially with BEGIN >

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread Joe Wilson
--- Nathaniel Smith <[EMAIL PROTECTED]> wrote: > On Wed, Jun 07, 2006 at 01:24:38PM -0400, [EMAIL PROTECTED] wrote: > > If it is inconvenient to rollback and retry the entire transaction, > > then start the transaction initially with BEGIN EXCLUSIVE. This > > will acquire the reserved lock

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread A. Pagaltzis
* Pat Wibbeler <[EMAIL PROTECTED]> [2006-06-07 22:55]: > It's entirely possible I'm reading these docs incorrectly, but > this strategy has worked quite well for me. No, I don’t see any error in your reading. My apologies; I should have consulted the docs instead of going by mailing list posts.

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread Bill KING
Jay Sprenkle wrote: > On 6/7/06, Bill KING <[EMAIL PROTECTED]> wrote: >> I understand why I'm getting the deadlock now, lazy locking, (it's >> against the logical grain of transaction/locking, but that's a whole >> other argument) . Maybe this should be highlighted with big arrows in >> the

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread Bill KING
[EMAIL PROTECTED] wrote: > Bill King <[EMAIL PROTECTED]> wrote: > >> Christian Smith wrote: >> >>> If one transaction already has a read lock, and another transaction >>> has a reserved lock (trying to get a write lock), neither thread can >>> get a write lock. One of the transactions

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread Craig Morrison
Bill King wrote: It would be nice if SQLite told us this. However, SQLite detects the reserved lock and returns SQLITE_BUSY, telling niether transaction much other than to try again. If a reserved lock is detected when trying to promote an existing read lock, this is a deadlock situation

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread A. Pagaltzis
* Pat Wibbeler <[EMAIL PROTECTED]> [2006-06-07 20:50]: > Beginning everything with BEGIN IMMEDIATE should eliminate the > possibility of deadlock, but you will serialize read-only > operations. Why? BEGIN IMMEDIATE acquires a for-read lock. Multiple for-read locks can be acquired concurrently. It

RE: [sqlite] Problems with multiple threads?

2006-06-07 Thread Pat Wibbeler
To: sqlite-users@sqlite.org Subject: RE: [sqlite] Problems with multiple threads? > If it is inconvenient to rollback and retry the entire transaction, then start the transaction initially with BEGIN EXCLUSIVE. > This will acquire the reserved lock immediately (instead of waiting to the

RE: [sqlite] Problems with multiple threads?

2006-06-07 Thread Jiri Hajek
> If it is inconvenient to rollback and retry the entire transaction, then start the transaction initially with BEGIN EXCLUSIVE. > This will acquire the reserved lock immediately (instead of waiting to the first write occurs) and so you will either get an > SQLITE_BUSY right away (when it is a

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread drh
Christian Smith <[EMAIL PROTECTED]> wrote: > On Wed, 7 Jun 2006, Jiri Hajek wrote: > > > However, right after fixing this, I found another problem. It certainly can > > be my fault, but I don't see how could it be: If I don't use transactions, > > multiple threads seem to proceed well, but then

Re: [sqlite] BEGIN and Backup [was [sqlite] Problems with multiple threads?]

2006-06-07 Thread drh
"Pat Wibbeler" <[EMAIL PROTECTED]> wrote: > You can use BEGIN IMMEDIATE or BEGIN EXCLUSIVE depending on the type of > lock you'd like. > If you are just trying to make sure the database does not change while you back it up, then Jay's suggestion of BEGIN IMMEDIATE is the better approach (better

RE: [sqlite] Problems with multiple threads?

2006-06-07 Thread Pat Wibbeler
EGIN and Backup [was [sqlite] Problems with multiple threads?]" Discusses a similar issue. Pat -Original Message- From: Jiri Hajek [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 07, 2006 9:26 AM To: sqlite-users@sqlite.org Subject: RE: [sqlite] Problems with multiple threads? Than

RE: [sqlite] BEGIN and Backup [was Re: [sqlite] Problems with multiple threads?]

2006-06-07 Thread Pat Wibbeler
Message- From: Russell Leighton [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 07, 2006 8:24 AM To: sqlite-users@sqlite.org Subject: [sqlite] BEGIN and Backup [was Re: [sqlite] Problems with multiple threads?] So, this was very enlightening...I have a simple backup function that I now question

Re: [sqlite] BEGIN and Backup [was [sqlite] Problems with multiple threads?]

2006-06-07 Thread Russell Leighton
Thx! [EMAIL PROTECTED] wrote: Russell Leighton <[EMAIL PROTECTED]> wrote: So, this was very enlightening...I have a simple backup function that I now question is correct. It does: - execute "begin" // lock from writes -copy db file to new file byte by byte - execute "commit" //

RE: [sqlite] Problems with multiple threads?

2006-06-07 Thread Christian Smith
On Wed, 7 Jun 2006, Jiri Hajek wrote: However, right after fixing this, I found another problem. It certainly can be my fault, but I don't see how could it be: If I don't use transactions, multiple threads seem to proceed well, but then right after I add BEGIN and COMMIT to some place, all

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread Jay Sprenkle
On 6/7/06, Jiri Hajek <[EMAIL PROTECTED]> wrote: However, right after fixing this, I found another problem. It certainly can be my fault, but I don't see how could it be: If I don't use transactions, multiple threads seem to proceed well, but then right after I add BEGIN and COMMIT to some

RE: [sqlite] Problems with multiple threads?

2006-06-07 Thread Jiri Hajek
Thanks for an additional explanation, I used sqlite3_get_autocommit() for debugging and it helped me to find out that it really was my fault. There was an incorrect processing after COMMIT returned SQLITE_BUSY. So sorry for this. However, right after fixing this, I found another problem. It

Re: [sqlite] BEGIN and Backup [was Re: [sqlite] Problems with multiple threads?]

2006-06-07 Thread Jay Sprenkle
On 6/7/06, Russell Leighton <[EMAIL PROTECTED]> wrote: So, this was very enlightening...I have a simple backup function that I now question is correct. It does: - execute "begin" // lock from writes -copy db file to new file byte by byte - execute "commit" // unlock ...I was

RE: [sqlite] Problems with multiple threads?

2006-06-07 Thread Brannon King
For me, I have a bunch of threads writing to the database. That is the only part I do multithreaded. (All my read queries are handled after all the data is written.) I just use the scoped_lock operator from the Boost library at the top of my function that does the bind and step calls. I pass a

Re: [sqlite] Problems with multiple threads?

2006-06-07 Thread drh
As various people search for application and/or SQLite bugs related to multiple threads and BEGIN, let me try to aid the effort by better describing exactly what BEGIN does and suggesting some debugging tricks. Realize that BEGIN does not actually create any file locks or check to see if any

RE: [sqlite] Problems with multiple threads?

2006-06-06 Thread Pat Wibbeler
to be mine, not SQLite when I see this issue. Pat -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 06, 2006 6:36 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Problems with multiple threads? Bill KING <[EMAIL PROTECTED]> wrote: > Could t

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread Bill KING
Joe Wilson wrote: > --- Bill KING <[EMAIL PROTECTED]> wrote: > >> Outside of, and I use Qt's QReadWriteLock. ->lockForRead()/lockForWrite() >> http://doc.trolltech.com/4.1/qreadwritelock.html >> > > Thanks. > > Many follow-up questions... :-) > > Is this for an application or for the Qt

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread Bill KING
Joe Wilson wrote: > Hi Bill, > > When you say "handle read/write locking [your]self" do > you mean outside of SQLite in your code or by altering > SQLite's source code? > > What algorithm do you employ? > > --- Bill KING <[EMAIL PROTECTED]> wrote: > >> I personally did do all this, this

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread John Stanton
[EMAIL PROTECTED] wrote: Bill KING <[EMAIL PROTECTED]> wrote: Could this be cause by one thread opening a transaction, then a second on a second connection trying to open a transaction on it, and failing to open the transaction file (as it already exists?). No. Each database connection

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread Joe Wilson
Hi Bill, When you say "handle read/write locking [your]self" do you mean outside of SQLite in your code or by altering SQLite's source code? What algorithm do you employ? --- Bill KING <[EMAIL PROTECTED]> wrote: > I personally did do all this, this doesn't solve the issue. As I > mentioned

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread drh
Bill KING <[EMAIL PROTECTED]> wrote: > Could this be cause by one thread opening a transaction, then a second > on a second connection trying to open a transaction on it, and failing > to open the transaction file (as it already exists?). > No. Each database connection (each sqlite3* pointer)

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread Bill KING
[EMAIL PROTECTED] wrote: > "Jiri Hajek" <[EMAIL PROTECTED]> wrote: > >> 1. Occasionally after running Sqlite3_step() I get SQLITE_CANTOPEN ('Unable >> to open the database file') error. I found out that it can be fixed by >> running the query again, i.e. again calling Sqlite3_Prepare(). So this

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread drh
"Jiri Hajek" <[EMAIL PROTECTED]> wrote: > > 1. Occasionally after running Sqlite3_step() I get SQLITE_CANTOPEN ('Unable > to open the database file') error. I found out that it can be fixed by > running the query again, i.e. again calling Sqlite3_Prepare(). So this isn't > a big issue, but still

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread Bill KING
[EMAIL PROTECTED] wrote: > "Jiri Hajek" <[EMAIL PROTECTED]> writes: > > >> 1. Occasionally after running Sqlite3_step() I get SQLITE_CANTOPEN ('Unable >> to open the database file') error. I found out that it can be fixed by >> running the query again, i.e. again calling Sqlite3_Prepare(). So

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread Derrell . Lipman
"Jiri Hajek" <[EMAIL PROTECTED]> writes: > 1. Occasionally after running Sqlite3_step() I get SQLITE_CANTOPEN ('Unable > to open the database file') error. I found out that it can be fixed by > running the query again, i.e. again calling Sqlite3_Prepare(). So this isn't > a big issue, but still I

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread John Stanton
Synchronize you database access so that only one transaction is current and is finalized on cpmpletion. In other words serialize it. You can use a mutex or similar to achieve synchronization. Look back at the recent discussion on this forum. Jiri Hajek wrote: Hello, I'm trying to use

Re: [sqlite] Problems with multiple threads?

2006-06-06 Thread Bill KING
Jiri Hajek wrote: > Hello, > > I'm trying to use SQLite in an application where it's needed to work with > one database in mutliple threads. Based on all the info I read in SQLite > documentation I create a new database connection for every new thread > created. Each thread does some SELECTs,

[sqlite] Problems with multiple threads?

2006-06-06 Thread Jiri Hajek
Hello, I'm trying to use SQLite in an application where it's needed to work with one database in mutliple threads. Based on all the info I read in SQLite documentation I create a new database connection for every new thread created. Each thread does some SELECTs, INSERTs or UPDATEs, but there