> any plans/desires to fix this I hesitate to announce this for reasons I'll explain below, but I have been working on an experimental version on the SQLite pager called the Shadow Pager. The Shadow Pager is intended to address this problem for applications that use SQLite from a single process only, but with several threads. In particular, in SQLite with the Shadow Pager read transactions are never blocked. Furthermore, read transactions do not block write transactions; write transactions DO block other writes; ACID properties are maintained for all transactions.
SQLite with the Shadow Pager will certainly have drawbacks. Databases will grow in size a bit (how much depends on transaction concurrency), but there will be no (well, perhaps O(1), but not O(N)) journaling. Compared with SQLite it will undoubtedly be a bit slower, although it's possible that write transactions will be competitive. The tradeoff is: lower throughput for lower latency. I need this lower latency in some of my applications; you may not need it for yours. [It is certainly feasible to extend the Shadow Pager to multiple processes. However, it would involve significant engineering, and it is not of interest to me right now.] My hesitation in announcing the Shadow Pager project is that there is no guarantee of success, no timetable for completion, and by necessity it must take a low priority position on my schedule (at least until it becomes part of a funded project). In other words, it's vaporware. The present status is that the design is complete, a specification of sorts has been written, and coding has begun. At some point I will be asking for volunteer testers, especially for gathering performance data. If you are interested in serving as a beta tester please send me a private e-mail and I will add you to the list. It will be a couple months at least before testing will commence. My intention is to release the Shadow Pager to the public domain so that D. Richard Hipp can incorporate it into the SQLite distribution if he sees fit. However, D. Richard Hipp has not been asked to endorse nor to review the project. At this point the Shadow Pager is not far enough along to ask for that level of support. e Friday, January 9, 2004, 2:28:50 PM, you wrote: >> -----Original Message----- >> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] >> >> The important thing to understand here, is that you're not stepping >> through a set of results that you have in memory and that >> were extracted >> from the database. You are (at least for simple queries) >> stepping through >> data that is still in the database. Because you're still >> actively reading >> the database the writer thread can't get access until your query is >> finished. If, instead, you want to load the whole results set >> into memory >> before stepping through them consider the sqlite_get_table_* >> functions. >> Alternatively you can collect the results into memory any way >> you like >> before stepping through the structures you've created yourself. > I ran into this yesterday too. This is a pretty severe limitation compared > with other databases I've used, like Postgres, MySQL, and Oracle. Are there > any plans/desires to fix this, or is it considered something that just comes > with the "lightweight" territory? > -Ken > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
