On 05/14/2014 01:18 AM, [email protected] wrote: > [email protected] wrote: >> Currently if a user (wilfully or accidentally, say, through composition of >> third party libs) opens the same LMDB environment twice within a process, and >> maintains active read transactions at the time one mdb_env_close() is called, >> all reader slots will be deallocated in all environments due to the logic >> around line 4253 that unconditionally clears reader slots based on PID. >> >> I'd like to avoid this in py-lmdb, and seems there are a few alternatives to >> fix it: >> >> 1) Tell users not to do that (doc fix, nobody reads docs) > > This is already in the doc - don't open the same env twice within one process.
1.5) Use something else than fcntl locks on the lock-file descriptor to control whether to re-init the lockfile and its mutexes/semaphores. Like MDB_NOLOCK. Needed to get any other suggestions off the ground, see Caveats in lmdb.h. Once two MDB_env's in a process has the lockfile open, both lose their fcntl locks when either closes it - and the next mdb_env_open() will reset the lockfile. Then both readers and writers are screwed, and more care with reader slots won't help. Well, one MDB_env could leak its lockfile descriptor instead of closing it, or it could give the descriptor to the other MDB_env for a deferred close if it could find it using some global state. (Could work until someone wrote an open-close loop which would run into the max-open-FD limit.) Either way, no fun. Another way would be #ifdef ... more unportable code ... #endif:-( Surely some OSes have come up with sane way to address this problem, there's just no portable way. >> 2) Externally maintain a set of (st_dev, st_ino) keys representing already >> opened environments, and check any attempt to open an environment against >> this >> list, failing if data.mdb's key already appears. > > That would require global library state, which is definitely unwanted. Though in one sense we already have that, which is the problem. Global (well, per-process) lock state on open files and mutexes. >> 3) Disable the mdb_env_close() cleanup loop when MDB_NOTLS is enabled. Since >> no >> reader slot will ever exist in TLS in this mode, the existing mdb_env_close() >> doc "All transactions, databases, and cursors must already be closed before >> calling this function." ensures no readers exist at close time, and the loop >> need not run. > > Could do this, but since it's an incomplete solution, not sure there's any > point. That's MDB_NOLOCK anyway. MDB_NOTLS does have reader table entries. > (...) -- Hallvard
