Re: [sqlite] SQLite between forks
On 26/03/18 13:30, Simone Mosciatti wrote: > However I fail to see how this can be a problem for purely in-memory > database. When a process forks, only the thread that called fork is kept in the new child process. Also note that semaphores (and locks in general) are left in the same state as at the time of the fork call, and are not reset in the child (forked) process. A very simple example of a problem would be if a thread in the parent process is inside a SQLite call and another thread calls fork(). The currently active locks will remain in the child so SQLite operations will just block forever in the child. That can be avoided providing you use no threads, the libraries you use have no threads, and the libraries those libraries use have no threads. That will be increasingly unlikely over time. If you do use threads, then you would have to make things safe by controlling exactly when forks can happen (assuming no libraries thwart you). This is very tedious work that is very difficult to statically check, let alone verify everything is correct in every possible code path. Hence the recommendation to not use SQLite across forks. I address that by making sure only leaf processes do SQLite work. For my Python SQLite wrapper, I added a fork checker to help with testing and make sure you don't use SQLite across forks. The way it is implemented is replacing the SQLite mutex operations (there is a table of them) with ones that check the process id, and then call the original mutex operation. In my test of a benchmark where 100% of the code was doing SQLite operations, there was a slowdown of 1%. https://rogerbinns.github.io/apsw/apsw.html#apsw.fork_checker Roger signature.asc Description: OpenPGP digital signature ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] SQLite between forks
Simone Mosciatti wrote: > it is suggested in several place to don't share a connection between forks. Because of how locking and file handles interact. > However I fail to see how this can be a problem for purely in-memory database. In-memory databases do not use a file handle or file locking. > Different point if the database is backed by a regular file, however, > in such case, couldn't I simply close the connection in the child fork(2) says that | the child does not inherit process-associated record locks from its | parent (fcntl(2)). (On the other hand, it does inherit fcntl(2) open | file description locks and flock(2) locks from its parent.) So closing the child's handle can break the parent's locks. Regards, Clemens ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] SQLite between forks
Hi all, it is suggested in several place to don't share a connection between forks. However I fail to see how this can be a problem for purely in-memory database. If my understanding of fork and sqlite are correct I don't see issues in having a forked child reading a database connection opened before the fork itself. The address space of the two database are different and, at this point, we are already talking about two completely different database. Of course each write in the child (parent) won't be visible by the parent (child), but this is not my concern. Is my reasoning sound and correct? Different point if the database is backed by a regular file, however, in such case, couldn't I simply close the connection in the child and reopen it (maybe in read only mode)? Sorry for this question that may be trivial! Best Regards, Simone ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
