> The underlying error here is that you are attempting to use threads in the > first place. You should never do that. Threads are evil and should be > avoided wherever possible. Use separate processes for concurrency. Threads > in application programs always result in subtle bugs (such as this one) that > are hard to reproduce and waste countless hours of developer time. Just say > "no" to threads. Could not agree with you more. Threads are evil. This is exactly the reason I cannot EVER do long operations in the main thread (and by long I mean more than 20-30 ms). If I will do long operations, my system wont be responsive enough. When is it allowed to use threads? Only to do long, standalone operations, that would block the main thread (such as wal checkpoints). That is exactly what I did in my example program.
What I really wanted to do, is to use long transactions in the main thread and checkpoint in a background thread. long transaction increase the performance (and responsiveness) dramatically, since most of the work is in memory, and when committing, much less pages needs to be written to the DB (since many sql statements update the same pages), so I/O is reduced by order of magnitude. In addition there is no overhead of begin/end transaction for every statement. Currently I am running wal checkpoint in the main thread, every time sql cache is stressed (about 2000 dirty pages). but checkpoint can take a VERY long time (from 400ms to 20secs!) and this time is simply unacceptable. So I want to run checkpoint in another thread. But then I bump into the wal file size problem: http://www.mail-archive.com/sqlite-users@sqlite.org/msg57071.html So, this is what I am trying to solve now, using the method I described in the link above. I hope that one of the SQLite developer can attend to this problem, and would love to hear if you have any intention to do this in the near future. I also would love to hear any comment on the solution I suggested. Yoni. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
