Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-28 Thread Simon Slavin
TL;DR: If you want ACID at the OS and storage firmware level, expect to buy expensive server-rated hardware and expect it to be slow. On 28 Jan 2013, at 12:30pm, Phil Schwan wrote: > Arguably more importantly, there's an OS page cache that sits between your > application

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-28 Thread Phil Schwan
I'm not even sure why I'm wading into this; glutton for punishment, I guess. TL;DR: the assumption that a data-journaled file system guarantees the atomicity of individual write()s is, in my experience, not a valid one. Unfortunately this isn't really a topic about which one can draw general

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-28 Thread Richard Hipp
On Sun, Jan 27, 2013 at 12:21 PM, Shuki Sasson wrote: > No confusion here, the atomicity of the FS journal guarantees that the > fwrite will happen in full or not happen at all... > First off, SQLite uses write(), not fwrite(). Secondly, I don't think any modern unix-like

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-28 Thread Shuki Sasson
UFS is not fully journaled FS it jut keeps the metadata. With fully journaled File System that keeps metadata and data there is no possibility to loose unsaved data. Anything that was handed to fwrite and fwrite returned an OK for it is backed by the journal. Read the following:

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-27 Thread Pavel Ivanov
OK. I picked this one: http://www.freebsd.org/doc/en/articles/gjournal-desktop/article.html. It says: A journaling file system uses a log to record all transactions that take place in the file system, and preserves its integrity in the event of a system crash or power failure. Although it is

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-27 Thread Shuki Sasson
Pick up any book about UFS and read about the journal... Shuki On Sun, Jan 27, 2013 at 7:56 PM, Pavel Ivanov wrote: > > So in any file system that supports journaling fwrite is blocked until > all > > metadata and data changes are made to the buffer cache and journal is > >

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-27 Thread Pavel Ivanov
> So in any file system that supports journaling fwrite is blocked until all > metadata and data changes are made to the buffer cache and journal is > update with the changes. Please give us some links where did you get all this info with the benchmarks please. Because what you try to convince us

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-27 Thread Stephan Beal
On Sun, Jan 27, 2013 at 5:53 PM, Shuki Sasson wrote: > Answer: The journal is organized in transactions that each of them is > atomic, so all the buffered cache changes for such operation are put into > the transaction. Only fully completed transaction are replayed when

Re: [sqlite] [sqlite-dev] Can I safely use the pragma synchronization = OFF?

2013-01-27 Thread Shuki Sasson
Hi Pavel, thanks a lot for your answer. Assuming xWrite is using fwrite here is what is going on the File System: In a legacy UNIX File System (UFS) the journaling protects only the metadata (inode structure directory block indirect block etc..) but not the data itself. In more modern File Systems