Actually, I knew... I don't think this is a bug, but instead a feature.
Note it has nothing at all to do with closing the file descriptor. In my
below example (taken from a Linux 2.6 system), I don't bother closing
fd#3 before trying to lock again-- and yet my lock succeeds anyway.
open("/tmp/foo1", O_RDWR|O_CREAT, 0666) = 3
fcntl64(3, F_SETLK, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0
open("/tmp/foo1", O_RDWR|O_CREAT, 0666) = 4
fcntl64(4, F_SETLK, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0
close(4) = 0
open("/tmp/foo1", O_RDWR|O_CREAT, 0666) = 4
fcntl64(4, F_SETLK, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0
close(4) = 0
close(3) = 0
However. Consider BSD flock() semantics below:
open("/tmp/foo1", O_RDWR|O_CREAT, 0666) = 3
flock(3, LOCK_EX|LOCK_NB) = 0
open("/tmp/foo1", O_RDWR|O_CREAT, 0666) = 4
flock(4, LOCK_EX|LOCK_NB) = -1 EAGAIN (Resource
temporarily unavailable)
close(4) = 0
open("/tmp/foo1", O_RDWR|O_CREAT, 0666) = 4
flock(4, LOCK_EX|LOCK_NB) = -1 EAGAIN (Resource
temporarily unavailable)
close(4) = 0
close(3) = 0
This works a lot more like the locks you're expecting, but as you're
probably already aware, flock() doesn't work over NFS.
One solution is to use BOTH fcntl() and flock() locks. This won't work
on systems that treat fcntl() and flock() locks as the_same_ (most
notably, certain POSIX wrappers, 1.1-era of Linux kernels, and a few
other boneheaded installations), but it will catch the programming
mistake you mention, and anyway, there are run-time checks you can do
that will detect if flock() and fcntl() are the same.
However.
There exist systems that only have fcntl() or only have flock()-- the
latter are screwed with respect to NFS anyway, so we won't go over them,
but systems that ONLY have fcntl() are going to have a hard-time.
A better solution would be to keep all of your sqlite* structures in a
linked-list and simply _CHECK_ the inode numbers of any other open
databases to see if they match. This will require that SQLite's OS layer
understand threading (or at least, simple mutexes- in order to implement
the structure properly).
And still, a better solution (I really like this one) is to examine ALL
file descriptors LOWER than the one we just received from open() and
just successfully fcntl() locked and fstat() them. If the inode numbers
match, then this process ALREADY opened it- either another sqlite_open()
call, or some bonehead opening the database directly.
The best part about this is it requires relatively little memory, but it
requires a (potentially) large number of system calls. It may be usable
in a fall-back situation.
On Sun, 2004-01-11 at 11:48, D. Richard Hipp wrote:
> It turns out that POSIX has the following ugly misfeature:
> When a file descriptor is closed, all locks on the file that
> the file descriptor points to that are owned by the current
> process are dropped. Even locks created by completely separate
> and independent file descriptors. Who knew?
>
> Consider what this means for SQLite. Suppose you have a process
> that opens a database and starts making changes. Like this:
>
> db = sqlite_open("example1.db", 0, 0);
> sqlite_exec(db, "BEGIN", 0, 0, 0);
> sqlite_exec(db, "INSERT INTO whatever VALUES(1,2,3)", 0, 0, 0);
>
> We are in the middle of a transaction, so the file is open and
> it has a write lock. But before continuing, some obscure
> subroutine in your code does this:
>
> db2 = sqlite_open("example1.db", 0, 0);
>
> In second sqlite_open creates a new file descriptor on the database.
> Then it attempts to open the database for reading, so that it can
> load the database schema. It finds, however, that there is a write
> lock on the file, so it cannot read from it, so it immediately
> closes the file descriptor again. But in closing the file
> descriptor from this second sqlite_open(), POSIX also clears the
> write lock that was created by the first sqlite_open() above.
>
> Thus, the database is left sitting there, half-way updated, with
> no lock on it. Any other unsuspecting process can come along and
> update the database, leading to unimagined corruption.
>
> It will probably take some time to fix this problem. Until it is
> fixed, if you are using SQLite on unix, you should take care to
> never open the same database file more than once in the same
> same process. This also applies to multithreaded processes.
> You should never open the same database more than once in the
> same process - even in two independent threads.
>
> Until I have had a chance to investigate this situation futher,
> you should also avoid having two or more virtual machines (created
> by the sqlite_compile() API) active at once. Finalize one VM
> before creating the next one.
>
> At some point, I'll figure out a way to code around this horribly
> ugly wart in the POSIX advisory file locking. Until then, you
> risk database corruption if you try to do more than one thing at
> a time with the same database in the same process.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
