Sorry to bring this up yet again... On Tue, 13 Sep 2005, D. Richard Hipp wrote:
>On Tue, 2005-09-13 at 14:09 -0700, Brett Wilson wrote: >> I'm still concerned about the warnings on the web page about some >> networked file systems not supporting locking. There will be multiple >> DB connections from the same process. They might even be >> multithreaded. Might we have a problem in this case? >> > >Most network filesystems do fcntl locks incorrectly if >at all. An incorrect implementation of fcntl can result >in database corruption. Multiple connections from the same process could (should?) have a single lock manager for all connections. The threads share memory between them, hence intra-thread locking should be simple. The actual lock on the file should be the most restrictive lock reqired by all the threads within a process. Currently, if I'm not mistaken, SQLite has the following relationship between btree, pager and OsFile: btree -> pager -> OsFile btree -> pager -> OsFile btree -> pager -> OsFile What if, instead, we had this relationship: btree -> pager \ btree -> pager -> OsFile btree -> pager / Or perhaps even: btree -> pager \ btree -> pager -> page cache -> OsFile btree -> pager / With the latter, we'd get the benefit that multiple connections can share the page cache for a file, with some sort of per-page shadow for dirty pages, much as is done at the moment. Locking on the file is then done once per-process, and is set at any time to the greatest locking level required, which can be done at the point the locking level changes. Intra-process locking only requires changing the file level locking when there is a change. So, if thread 1 has a shared lock, and thread 2 also requires a shared lock, no change to the lock level on the file is required. If thread 2 then changes to an exclusive lock, the single lock on the single file can be updated from thread 2. Plus, any thread can set the lock on the file to any level, even if the previous level was set by another thread, so the implementation allows sharing of connections between threads. Unfortunately, this would require a major rejigging of the pager code, but could simplify os_unix.c to remove the need for openCnt and lockInfo, as the OsFile of the database file would only close when all connections are idle, and only be (un)locked when the pager lock state changes. Does this sound reasonable? Or am I way off base? > >Apple has contributed patches to SQLite that claim >to fix this problem. Those patches may one day find >their way into the default release. In the meantime, >you can find the patches at: > >http://www.sqlite.org/cvstrac/tktview?tn=1240 > Any way to generate a diff against 3.1.3? I'm having problem getting a sandbox of 3.1.3 from CVS. Are releases tagged? Releases should have a CVS tag. It's difficult to recreate a release after the fact without tags (or at least I'm having difficulty!) Christian -- /"\ \ / ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL X - AGAINST MS ATTACHMENTS / \
