On 2 February 2016 at 08:22, Stephen Chrzanowski <pontiac76 at gmail.com> wrote:
> On Mon, Feb 1, 2016 at 11:20 AM, Rowan Worth <rowanw at dugeo.com> wrote: > As I indicated in the last paragraph of my mail, I'm not in favour of > > fsync-directory-on-commit in the general case. But that's because I worry > > about the performance impact, not because SQLite is doing its best and at > > the mercy of the OS. I can't remember who in this thread suggested it but > > fsync-directory-on-close seemed like a great compromise. > > *ONLY* if configurable, and *ONLY* if not a default. Most of my apps are > setup at highest risk because the data they contain doesn't matter and I > want performance vs safety. Performance over safety is exactly the reason we are here in the first place. If performance wasn't a factor the OS would go straight to the disk platter in response to a write() call. But if we accept reality, we know that we have no such guarantee the data we just wrote to a file is on the platter before we call fsync(). And secondly, we have no guarantee that the directory structure we just changed (eg. by unlinking the journal) is on the platter before we call fsync() on the directory. The OS/hardware can still lie, sure. But sqlite hasn't given it a chance to tell the truth since doesn't fsync the directory - this is what I meant when I said "hasn't even asked the question." SQLite doesn't ask for the directory update to hit the platter, and then proceeds execution as if it has. To reiterate, I don't have a problem with sqlite's behaviour at all. I think it's perfectly acceptable. But I think it deserves to be acknowledged and understood for what it is; a pragmatic decision by sqlite to improve performance in the general case at the "expense" of a not-strictly-necessary rollback in an obscure failure scenario. It's not fair to meet it with the double standard of "fsyncing the directory every COMMIT would hurt general-case performance too much" and "the OS is responsible for ensuring data gets to disk", because that essentially asks the OS to fsync directory after every *i/o operation*, whether asked to or not. Which would hurt general-case performance much more! I'm right there behind you that it shouldn't be forced on users of sqlite :) -Rowan
