On Sat, Oct 23, 2010 at 9:26 AM, Bob Smith <bobs9...@gmail.com> wrote:
> On Sat, Oct 23, 2010 at 5:56 AM, Richard Hipp <drh at sqlite.org> wrote: > > > > Is that what you are seeing? > > > > Which filesystem are you using? > > > > > > Thanks Richard, this does seem to be what I am seeing. I am definitely > seeing more writes during the subsequent cycles and an explanation > that this is likely due to the writes being able to go faster to the > existing parts of the WAL definitely makes sense. I think that is it. > > I have run and seen this scenario on two machines. One has a linux > ext4 filesystem and the other has a linux ext3 filesystem. > > The WAL mode is a great addition to sqlite. Being able to do > simultaneous reads while writing is fantastic. I have also seen nice > improvements with writing speed. WAL mode has lots of value and the > sqlite developers have done a fantastic job. > > In my scenario of having to support a constant stream of steady writes > (granted, I would hope not with the frequency of my stress test) along > with an unknown frequency of reads that will come in, I do have > concerns about the WAL file consistently growing over a long period of > time (days and weeks). I would love to be able to continue using the > WAL mode, so I need to see what I can do to limit the growth or to > periodically make sure all connections to the database are closed at > the same time to allow the WAL file to periodically truncate. Any > other ideas? > > Best Regards, > > Bob > Give the described conditions, I do not think you need to worry about the WAL file growing without bound. I think that each increase should be smaller than the previous one, and will stop growing all together [for your stress test] when the WAL file has increased to the point that all of the [WAL] writes [during your forced lock conditions time frame] fit in the expanded file. How big, and how long to get there, will depend on the relative speeds of [over] writing the existing file, versus expanding and writing. Of course there are always edge cases. If a read / lock lasts long enough for *another* read to get a lock, then a progressive serious can keep a lock indefinitely, and the WAL file will be forced to continue to expand. The only way to *prevent* that expansion, is to stop writing. Other solutions get outside of SQLITE, like having application logic that starts rejecting new read requests at some point, until all existing read connection have closed, and SQLITE has finished the [long delayed] checkpoint. -- Phil _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
