On Nov 29, 2016, at 9:09 AM, Simon Slavin <slav...@bigfraud.org> wrote: > > >> On 29 Nov 2016, at 4:18pm, Mark Hamburg <mhamburg...@gmail.com> wrote: >> >> Does this make sense? Does it seem useful? (It seems useful to me when I see >> multi-megabyte WAL files.) > > Sorry, but I cannot spare the time right now to analyze the system you laid > out. It usually takes half an hour to diagram out the read and write > procedures and point out where multiple simultaneous ones don’t fit together. > > I can tell you that entire books are written about the difficulties of > simultaneous access to a database, and that I’ve read too many of them. And > that it has been proven many times that there’s no solution. You cannot > design a system which (A) provides up-to-date data to readers (B) allows > writers to get rid of their data immediately without ever locking up and (C) > guarantees that earlier changes to the data are ’saved' before later changes, > thus preserving uncorrupted data in the case of power-cuts, etc.. > > It is possible to implement a version if you drop one of the requirements. > For example, you can have many simultaneous writers as long as you don’t need > any readers. Or you can have many readers as long as you have only one > writer and you don’t need readers to be completely up-to-date. > > You might like to read more about Brewer’s Theorem: > > <https://en.wikipedia.org/wiki/CAP_theorem> > > In the meantime, I’m glad that WAL mode seems to be useful for you, if you > can cope with big journal files until all connections are closed, it’s a good > solution.
What I probably haven't accounted for is what it would take to do an atomic/safe swap of the WAL files in my double WAL scenario. I need to give that some more thought. In the single WAL scenario, what I probably really want is a way for the checkpoint operation on the write connection to do its work then wait for exclusive access — standard reader/writer lock pattern — to do the WAL reset. This would presumably limit the time that the readers were blocked since I would expect the WAL reset to be reasonably quick if the checkpoint copying work were already complete. Furthermore, the write operation would only be blocked for the length of the longest outstanding read so as long I favor big transactions for writes and short queries for reads, the writer shouldn't be blocked for too long either. Are there checkpoint settings that achieve this or do I need to build that logic into my code? Mark _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
