On Dec 11, 2018, at 04:01, Daniel Alm <dan...@timingapp.com> wrote: > > Hi, > > For the past half year we’ve been receiving reports from users who had > restored their SQLite-based databases from a Time Machine backup. Afterwards, > they would receive "database disk image is malformed” errors. The app also > backs up the user’s data “manually” to a ZIP file every week; those backups > seem to be working fine. We also haven’t received reports from other backup > tools causing issues. I have also suspected a bug in Time Machine, but it is > striking that the issues did seem to start occurring after an update to the > app (luckily, in fact, with the same update that also introduced the “manual” > backups).
Time Machine achieves eventual consistency by restarting when it detects that a file has changed since the backup was started. It does not have special provisions for SQLite database files. Even if the scheduled backup never runs, the conditions under which a database would be captured in an utterly inconsistent state should be vanishingly rare. It would be most useful if you could share a representative database with Richard for analysis. > Changes that we made to our setup in the update that coincided with the > errors occurring: > - Upgraded SQLite from 3.21 to 3.24 (we have since reverted to 3.23.1 in > another update; no improvement). > - Used memory mapping for read accesses via “PRAGMA mmap_size = 1073741824;” > (we have since reverted to “PRAGMA mmap_size = 0;” after reading > http://sqlite.1065341.n5.nabble.com/Re-Database-corruption-and-PRAGMA-fullfsync-on-macOS-td95366.html > > <http://sqlite.1065341.n5.nabble.com/Re-Database-corruption-and-PRAGMA-fullfsync-on-macOS-td95366.html>; > no improvement). > - Using a secondary database via [ATTACH > DATABASE](https://www.sqlite.org/lang_attach.html > <https://www.sqlite.org/lang_attach.html>) (although this also seems to occur > for users without such a database). > > At this point, I am at a loss, especially given that SQLite should be fairly > robust against database corruption. While our app is running in the > background all the time, it is not very write-heavy (~ one transaction per > minute taking just a few milliseconds). Also, the app had been running fine > before the update for a long time without any reports of this issue. I might > be doing something wrong or have changed anything else, but I don’t know > what; if you have any ideas, let me know. > > Any suggestions on what could be the culprit or what else I could try besides > downgrading all the way to SQLite 3.21 would be appreciated. Out of curiosity, why aren't you using the SQLite that comes with the OS? Scott _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
