This line stood out: > The main process opens the databases and then forks the other processes which can then perform database operations using the already opened databases.
From https://sqlite.org/howtocorrupt.html#_carrying_an_open_database_connection_across_a_fork_ : > Do not open an SQLite database connection, then fork(), then try to use that database connection in the child process. All kinds of locking problems will result and you can easily end up with a corrupt database. SQLite is not designed to support that kind of behavior. Any database connection that is used in a child process must be opened in the child process, not inherited from the parent. In this kind of situation, I usually use pthread_atfork() callbacks to automate closing databases and then re-opening them in the parent and child. On Thu, Nov 7, 2019 at 9:18 AM Jukka Marin <jukka.ma...@jmp.fi> wrote: > Dear List, > > I'm developing software which keeps parameters and real-time data in > SQLite databases on a x86_64/linux system. I am getting "database > disk image is malformed" errors from SQLite when using select. Some > select operations succeed, some fail. This happens on multiple systems. > > I would like to know if I'm doing something that is known not to work > with SQLite. Background information: > > The software runs on a xen virtual machine. The database files are > on a separate disk (actually, a separate partition of the host linux) > which is directly mounted on the virtual machine (not using NFS) > and uses ext4 filesystem. The whole system runs off a single SSD disk. > > SQLite version is 3.28.0 and it is currently built and statically > linked against the application binary. > > The databases are originally created by a script and the sqlite3 > command. "pragma journal_mode=wal;" is set for all databases. > > The software consists of multiple processes. The main process opens > the databases and then forks the other processes which can then perform > database operations using the already opened databases. No locking is > done in the processes, they rely on the SQLite internal mechanisms. > sqlite3_busy_timeout is set to 2500 for all databases. > > The processes mostly read (select) data from the databases, but also > occasionally add new rows or update existing rows. > > > Is this system with multiple processes accessing the databases > safe with SQLite? If it is, does anyone have ideas of what could > be corrupting the database files? > > These systems are running 24/7, but of course, it _may_ be possible > that someone has disconnected the power supply without shutting down > the system first and causing errors in the database files. > > Is there something I could do to (try to) prevent database corruption > if power is lost? (In software - a UPS is a hardware solution, of > course). > > If the database is corrupted, is there any way of recovering it? > Maybe I should use sqlite3 to dump and recreate the database (of > course, some or all of the data might get lost)? > > Experienced SQLite users, please share your ideas - thank you! :-) > > Jukka Marin > (a beginner with SQLite) > _______________________________________________ > sqlite-users mailing list > sqlite-users@mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
