Hello Adam, all,
Cross platform locking is defiantly very important. The goal should be
to use the identical sqlite DB via UNIX, AFP, SMB, NFS and others. My
opinion is that it is not needed to have any additional open parameters,
just use the locking features which are in common between all platforms.
Using an additional TCP/IP server is more complex in term of cross
platform compatibility and may be an overkill in terms of performance,
and will introduce additional problems.
At present the os_unix.c has some problems, I don't recall all its
details so here is just an estimate:
Multiple locks are getting unlocked with one call
- This does not work on AFP and SMB
A Read lock gets turned into a writer lock (and or back?)
- This does not work on AFP, SMB has limited support for it.
The current lock offset will not allow to copy open DB files if the
database gets larger than 0x40000000 bytes. This is because locked
regions cannot be copied under Windows, we changed it to:
#define PENDING_BYTE I64CONST(0x7fffffffffff0000)
#define RESERVED_BYTE I64CONST(0x7fffffffffff0001)
#define SHARED_FIRST I64CONST(0x7fffffffffff0002)
Advisory versus mandatory record locking
AFP and SMB is doing mandatory locking which means other clients cannot
read locked file areas, advisory locking means that we can ready any
data therefore we must call lock first before reading bytes.
The benefit of supporting the OS based locking is that after a program
exits it will automatically cleanup all locks. Windows has some "oplock"
features which will handle remote locks completely on the Windows SMB
clients as long as only one writer uses the file.
It should not be this difficult to use only locking features which are
in common cross major platforms and network file systems.
Helmut Tschemernjak
Adam Swift wrote:
All,
In order to provide locking support for database files mounted from
remote file systems (NFS, SMB, AFP, etc) as well as provide compatible
locking support between database clients both local and remote, I would
like to propose some additions to the existing database file locking
behavior.
I have discussed this briefly with D. Richard Hipp and he has expressed
interest in pursuing it. We would appreciate
feedback/suggestions/concerns on the proposed solutions below.
1. Support a variety of locking mechanisms, from the lowest granularity
(using a <database_name>.lock file) to the highest (the existing
advisory locks). A preliminary list: .lock files, flock, afp locks,
posix advisory locks.
2. Allow clients to specify type of locking (on database open) manually
or ask sqlite to automatically detect the best locking supported by the
file system hosting the database file (automatic detection would not
work in a mixed local/remote file system situation, all clients of a
single database file need to use the same type of locking to avoid
conflicts).
3. Extend the sqlite3_open commands to support URI style path references
in order to specify the file system locking type (as opposed to
modifying the arguments list). After a little poking around on RFC 3986
<http://www.ietf.org/rfc/rfc3986.txt> I'm inclined to specify the
locking choice via the query part of the URI. For example:
file:///mounts/myhost/dbfile.db?locktype=flock
file:///localdisk/otherdbfile.db?locktype=automatic
Thanks in advance for your input.
Adam Swift
