On Fri, Sep 12, 2014 at 6:21 PM, Richard Hipp <d...@sqlite.org> wrote:
> On Fri, Sep 12, 2014 at 8:07 PM, Simon Slavin <slav...@bigfraud.org> > wrote: > > > > one thing that annoys me about SQLite is that it needs to make a > > journal file which isn't part of the database file. Why ? Why can't it > > just write the journal to the database file it already has open ? This > > would reduce the problems where the OS prevents an application from > > creating a new file because of permissions or sandboxing. > > > Where in the database does the journal information get stored? At the > end? What happens then if the transaction is an INSERT and the size of the > content has to grow? Does that leave a big hole in the middle of the file > when the journal is removed? During recovery after a crash, where does the > recovery process go to look for the journal information? If the journal > is at some arbitrary point in the file, where does it look. Note that we > cannot write the journal location in the file header because the header > cannot be (safely) changed without first journaling it but we cannot > journal the header without first writing the journal location into the > header. > > One idea that might work is to interleave the journal information with the > content. So for each page in the database, there is a corresponding page > of journal content. The downside there is that you double the size of the > database file without increasing its storage capacity. > A couple of academic thoughts. 1. If one wanted to embed the journal within the database, would it be adequate to reserve a specific page as the "root" page of the journal, then allocate the remaining pages as normal (either to the journal or the main database)? This does leave the big hole problem so it may still not be ideal, but it would give you a known location to find the beginning of the journal without doubling the database size or requiring an extra file. 2. Building on 1, could sparse files be used to accomplish this? Seek to "really big constant offset" and do all journaling operations at that point, allowing the operating system to manage actual disk allocation? If this were possible, deleting the journal would be a "fast" truncate operation. A custom VFS might be able to provide a proof of concept... hmm. -- Scott Robison _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
