Re: [sqlite] Data loss during the disk full condition

I haven't seen anything to say what journalling is being used (Rollback, WAL or none). If the latter then SQLite will have nothing to revert to on error. Paul www.sandersonforensics.com skype: r3scue193 twitter: @sandersonforens Tel +44 (0)1326 572786

Re: [sqlite] Data loss during the disk full condition

On 10/29/16, Yuri wrote: >> What does "PRAGMA integrity_check" say about your database? > > It returns "ok", but this file has been opened and written into again. > This suggests that the problem may be somewhere besides in SQLite. If SQLite were getting confused and zeroing

Re: [sqlite] Data loss during the disk full condition

On 29 Oct 2016, at 11:47pm, Yuri wrote: > It opens the DB, applies updates without the explicit transaction, some of > them fail, messages are printed, DB is closed. DB should be left in the valid > state. Is this not what is supposed to happen? I would argue that you should

Re: [sqlite] Data loss during the disk full condition

On 10/29/2016 15:15, Simon Slavin wrote: I believe SQLite handled the failure correctly. It returned SQLITE_FULL and it did not corrupt its database. SQLITE_FULL is documented as an error code, not a warning, in section 1 of My program prints errors,

Re: [sqlite] Data loss during the disk full condition

On 29 Oct 2016, at 10:42pm, Yuri wrote: > On a busy system one write operation might fail with disk-full, while the > rest are fine, because other processes will write and delete data > continuously. This is what I suspect: that SQLite failed to handle one or a > few such

Re: [sqlite] Data loss during the disk full condition

On 10/29/2016 14:32, Simon Slavin wrote: On 29 Oct 2016, at 10:21pm, Yuri wrote: I think you need to have such testcase: On the table with a lot of key/value pairs you run a set of updates and inserts. Random file operations should fail with some low probability with various

Re: [sqlite] Data loss during the disk full condition

On 29 Oct 2016, at 10:21pm, Yuri wrote: > I think you need to have such testcase: On the table with a lot of key/value > pairs you run a set of updates and inserts. Random file operations should > fail with some low probability with various error codes, like disk full. You

Re: [sqlite] Data loss during the disk full condition

On 10/29/2016 13:06, Richard Hipp wrote: >SQLite should preserve data in such case, it should fail operations that >can't be performed, but the old data should absolutely stay intact. That is what is suppose to happen - everything should be preserved. This is tested extensively and there are no

Re: [sqlite] Data loss during the disk full condition

On 10/29/16, Yuri wrote: > > After the reboot the table still existed but was blank. The file was > about the same size, but had large zeroed areas, and no rows in this > table. Other table still had rows. What does "PRAGMA integrity_check" say about your database? -- D.

Re: [sqlite] Data loss during the disk full condition

On 10/29/16, Yuri wrote: > > After the reboot the table still existed but was blank. The file was > about the same size, but had large zeroed areas, and no rows in this > table. Other table still had rows. > > > How could this have happened that data could get lost like that? I

Re: [sqlite] Data loss during the disk full condition

> On Oct 29, 2016, at 11:34 AM, Simon Slavin wrote: > > Really ? An interactive program (or any program) gets a result code it wan't > expecting and you don't want it to shut down ? Really. Apps aren’t supposed to crash. How would you feel if Photoshop or Word or Logic

Re: [sqlite] Data loss during the disk full condition

On 29 Oct 2016, at 7:23pm, Jens Alfke wrote: > On Oct 29, 2016, at 10:44 AM, Simon Slavin wrote: > >> To minimise problems like the one you reported it needs to quit the program >> (preferably with a non-zero exit code) when it gets any unexpected

Re: [sqlite] Data loss during the disk full condition

> On Oct 29, 2016, at 10:44 AM, Simon Slavin wrote: > > To minimise problems like the one you reported it needs to quit the program > (preferably with a non-zero exit code) when it gets any unexpected result. That may be true for a server or a command-line tool, but it's

Re: [sqlite] Data loss during the disk full condition

On 10/29/2016 10:44, Simon Slavin wrote: To minimise problems like the one you reported it needs to quit the program (preferably with a non-zero exit code) when it gets any unexpected result. This particular application, SQLite QSettings backend, can't quit on the disk-full condition because

Re: [sqlite] Data loss during the disk full condition

On 29 Oct 2016, at 6:32pm, Yuri wrote: > It does check error codes, and it prints warnings, but errors don't fail the > overall app. It will keep updating/inserting again and again later. To minimise problems like the one you reported it needs to quit the program (preferably

Re: [sqlite] Data loss during the disk full condition

On 10/29/2016 06:08, Simon Slavin wrote: On 29 Oct 2016, at 8:05am, Yuri wrote: >The application is synchronizing the in-memory key-value table with the disk one using insert/update/delete statements using that key. The in-memory table was full at the moment of reboot,

Re: [sqlite] Data loss during the disk full condition

On 29 Oct 2016, at 8:05am, Yuri wrote: > The application is synchronizing the in-memory key-value table with the disk > one using insert/update/delete statements using that key. The in-memory table > was full at the moment of reboot, You're going to have to explain that.

[sqlite] Data loss during the disk full condition

My disk was full when the system rebooted, and the important SQLite table got blanked at that time. The table is a simple key/value table with the primary key. The application is synchronizing the in-memory key-value table with the disk one using insert/update/delete statements using that key.