Re: [sqlite] shared cache/ test_server.c

2007-07-27 Thread John Stanton
Scott Hess wrote: On 7/27/07, John Stanton <[EMAIL PROTECTED]> wrote: Scott Hess wrote: On 7/26/07, Richard Klein <[EMAIL PROTECTED]> wrote: According to the Mozilla article referenced above, it's even worse than that: *All* cache pages, dirty or not, are freed at the end of *every*

Re: [sqlite] shared cache/ test_server.c

2007-07-27 Thread Scott Hess
On 7/27/07, John Stanton <[EMAIL PROTECTED]> wrote: > Scott Hess wrote: > > On 7/26/07, Richard Klein <[EMAIL PROTECTED]> wrote: > >>According to the Mozilla article referenced above, it's even worse than > >>that: *All* cache pages, dirty or not, are freed at the end of *every* > >>transaction,

Re: [sqlite] shared cache/ test_server.c

2007-07-27 Thread John Stanton
Scott Hess wrote: On 7/26/07, Richard Klein <[EMAIL PROTECTED]> wrote: According to the Mozilla article referenced above, it's even worse than that: *All* cache pages, dirty or not, are freed at the end of *every* transaction, even if the transaction consisted of only read operations. I

Re: [sqlite] shared cache/ test_server.c

2007-07-27 Thread John Stanton
Richard Klein wrote: John Stanton wrote: Richard Klein wrote: Joe Wilson wrote: You've probably read this. It's useful information for any performance minded developer using SQLite: http://developer.mozilla.org/en/docs/Storage:Performance >> [snip] If the above is correct, it

Re: [sqlite] shared cache/ test_server.c

2007-07-26 Thread Scott Hess
On 7/26/07, Richard Klein <[EMAIL PROTECTED]> wrote: > According to the Mozilla article referenced above, it's even worse than > that: *All* cache pages, dirty or not, are freed at the end of *every* > transaction, even if the transaction consisted of only read operations. I believe this is no

Re: [sqlite] shared cache/ test_server.c

2007-07-26 Thread Richard Klein
John Stanton wrote: Richard Klein wrote: Joe Wilson wrote: You've probably read this. It's useful information for any performance minded developer using SQLite: http://developer.mozilla.org/en/docs/Storage:Performance >> [snip] If the above is correct, it is not enough for the

Re: [sqlite] shared cache/ test_server.c

2007-07-26 Thread John Stanton
Richard Klein wrote: Joe Wilson wrote: You've probably read this. It's useful information for any performance minded developer using SQLite: http://developer.mozilla.org/en/docs/Storage:Performance I read it, and I'm now weeping! Below I've reproduced the paragraphs that cause me some

Re: [sqlite] shared cache/ test_server.c

2007-07-26 Thread Richard Klein
Joe Wilson wrote: You've probably read this. It's useful information for any performance minded developer using SQLite: http://developer.mozilla.org/en/docs/Storage:Performance I read it, and I'm now weeping! Below I've reproduced the paragraphs that cause me some consternation:

Re: [sqlite] shared cache/ test_server.c

2007-07-26 Thread John Stanton
I read that. The Mozilla people point out that their advantage was with a large volume of small transactions. They benefit from not destroying the cache between transactions and by conserving memory with large numbers of users. They point out that relaxing the ACID requirements also aids

Re: [sqlite] shared cache/ test_server.c

2007-07-26 Thread Joe Wilson
> Shared cache mode would be better named "persistent cache mode" because > its main effect is to permit one thread to not flush the cache after > each transaction. The people at Mozilla report that they use it and get > better throughput on small transactions. You've probably read this. It's

Re: [sqlite] shared cache/ test_server.c

2007-07-25 Thread John Stanton
Now you have made me worry. If cache can only be shared by connections created in one thread then there is no shared cache. I must investigate this more closely. Perhaps my reading of the documentation included a dose of wishful thinking and a belief that "shared" meant shared! Looking

Re: [sqlite] shared cache/ test_server.c

2007-07-24 Thread Ken
John, According to the Sqlite documentation on sqlite3_enable_shared_cache: "There is no mechanism for sharing cache between database connections running in different threads." This means exactly what I said in the first place: You cannot have a "shared cache" access across threads. I

Re: [sqlite] shared cache/ test_server.c

2007-07-24 Thread Joe Wilson
--- John Stanton <[EMAIL PROTECTED]> wrote: > I think that you misunderstood the shared cache description. Cache is > shared by many connections but connections may not be passed between > threads. Each thread must maintain and use its its own connection. In > our case a thread has an

Re: [sqlite] shared cache/ test_server.c

2007-07-24 Thread John Stanton
I think that you misunderstood the shared cache description. Cache is shared by many connections but connections may not be passed between threads. Each thread must maintain and use its its own connection. In our case a thread has an associated control block and the connection handle

Re: [sqlite] shared cache/ test_server.c

2007-07-23 Thread Richard Klein
Ken wrote: I guess my point was that inside the server thread, once a transaction is entered upon behalf of a client then only that activity may continue and no others. So in my design i only had two choices, re-enqueue the message inside the server until the transactional thread completed or

Re: [sqlite] shared cache/ test_server.c

2007-07-23 Thread Ken
John, The server can maintaine a "shared cache" but if a thread also opens the DB then that execution line will not have a "shared cache" but rather a cache per thread. Only the server thread may open and act upon the connection utilizing a shared cache on behalf of the client. The client may

Re: [sqlite] shared cache/ test_server.c

2007-07-23 Thread John Stanton
That is why the Sqlite locking is not a good fit for a threaded server. Why not use thread locks instead and achieve the synchronization with minimum overhead and latency? You do miss out on a couple of Sqlite features doing that (the pending and reserved locks which help with concurrency

Re: [sqlite] shared cache/ test_server.c

2007-07-23 Thread Ken
John, The sqlite api won't block, it will return a sqlite_busy type error to any other transactions that are attempted? Correct, so there is no sqlite blocking which is a good thing when writing a server. The clients will always block waiting upon a response from the server. The server simply

Re: [sqlite] shared cache/ test_server.c

2007-07-20 Thread John Stanton
Why not just bloock on the transation. That will queue waiting threads, serializing the access to the transaction. Ken wrote: My assumption for the server thread was that it needed to process all incoming requests in transaction order and to not loose outstanding requests. You have two

Re: [sqlite] shared cache/ test_server.c

2007-07-20 Thread Richard Klein
Ken wrote: Richard, You might want to look at src/test_server.c for an example of the shared_cache if you haven't found it already. I'll take a look at it, thanks! Personally, I think it makes a lot of sense (read simpler) to implement independent connections than to implement a

Re: [sqlite] shared cache/ test_server.c

2007-07-20 Thread Ken
My assumption for the server thread was that it needed to process all incoming requests in transaction order and to not loose outstanding requests. You have two choices once a client initiates a transaction: a. reject the incoming request since a transaction is active in the server.

Re: [sqlite] shared cache/ test_server.c

2007-07-20 Thread John Stanton
Ken wrote: Richard, You might want to look at src/test_server.c for an example of the shared_cache if you haven't found it already. Personally, I think it makes a lot of sense (read simpler) to implement independent connections than to implement a server. But I can see why you might want a