Gerlando Falauto writes: thanks for you reply. Indeed, readers are long-lived processes on a Linux system. They're threads of a Pyhon Flask application to export data, which uses uses peewee as a wrapper for SQLite. Theoretically, once a request is served, the transaction should be terminated. However, I suspect there is some issue there (it's a pretty complex pipeline to build files to be served on the fly), and for some reason cleanup is not performed correctly. For one thing, I did not program it explicitly. I can of course investigate further, but I was wondering if there was some way of ascertaining whether my suspicion is correct (and later, whether I've fixed the bug or not). In other words, if there's a way of getting a list of the current transactions or locks. I believe this information is stored somewhere in the WAL-index file (.db-shm), because that's the kind of information SQLite needs to decide it can't run a checkpoint (which is believe is what happening).
If would also like to know if killing (possibly with -9) a process with an open handle might accidentally leave the DB locked. You’re welcome. SQLite relies upon process synchronization objects, provided by the operating system, to coordinate access to the database on disk (or whatever the persistent storage medium is). As Keith mentioned, the processes need not be identified in the stored data to do this. Doing so would be problematic for several reasons, among which is that such ephemeral facts do not belong in the persistent data which should be meaningful across power cycles, machines, and even platforms. Your worry about the locking objects being orphaned by ‘kill -9 …’ or other sudden process death is understandable but misplaced. One very well settled convention in operating system design is that such orphaning should not happen, and if it does, that is a serious bug. This is part of the reason that the operating system manages such objects. It tracks which processes are using various resources subject to being used among multiple processes, and releases associated process-to-object links (typically ‘handles’) when a process exits for any reason. I highly doubt any serious operating system provider would release an OS which failed to do this reliably. The problem is (most likely) in the code you suspect, (that “pretty complex pipeline”), or some not-quite-intelligent caching associated with it. Whoever debugs that would benefit from using an equivalent of the “Handle” utility I suggested. (That is a way of getting a list of currently existing process synchronization objects, even ones used as locks.) It is easy for an application to orphan its own references to operating system resources, a sin which only is absolved when it dies or exits. I suspect that when the error is found, there will be some improperly handled error return from SQLite involved, where insufficient cleanup is done and a SQLite connection is effectively orphaned. SQLite itself is very good about not orphaning the objects it holds, but it can only do this when its “connection” object (representing by a pointer) is closed. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
