Scott Robison wrote:
On Sat, Sep 13, 2014 at 2:24 PM, Howard Chu <h...@symas.com> wrote:
Scott Robison wrote:
A couple of academic thoughts.
1. If one wanted to embed the journal within the database, would it be
adequate to reserve a specific page as the "root" page of the journal,
then
allocate the remaining pages as normal (either to the journal or the main
database)? This does leave the big hole problem so it may still not be
ideal, but it would give you a known location to find the beginning of the
journal without doubling the database size or requiring an extra file.
Starting with a known location is definitely a step in the right direction.
2. Building on 1, could sparse files be used to accomplish this? Seek to
"really big constant offset" and do all journaling operations at that
point, allowing the operating system to manage actual disk allocation? If
We're talking about implementing a filesystem. "the operating system" is
your own code, in this case, you don't get to foist the work off onto
anyone else.
No, Simon's original question was to the effect of why doesn't SQLite just
use the already open database file for journaling purposes as well.
OK, maybe I missed that, but I thought that question itself arose from how to
use SQLite to implement a filesystem, on a raw partition. And the answer to
that question (operating inside a raw partition) could apply equally well to
operating inside a single file.
If you preassign a fixed maximum size to the file, you could e.g. reserve the
tail of the file for the journal, growing backward toward the head of the
file, while the main data grows the usual direction from the head of the file
toward the tail. This would basically be your (2) above. On HDDs this approach
would have horrible seek latencies but it could work OK on SSDs.
The other point though - like the existing journaling filesystems, you should
not limit yourself to using a single file/storage device. Allow the option of
storing the journal somewhere else - the performance potential is worth it.
My
point 1 was in response to the need to know where the journal file is, so
just pick a dedicated page in the file as the root page of the journal,
allowing the two files to be co-mingled. It doesn't address every possible
bad reason for co-mingling the data, but it would at least answer the
question "how do you find the journal".
My second point was about existing SQLite database files that live in a
file system managed by some operating system. SQLite already foists that
work off on to someone else, this would be no different. It still may be a
bad idea, but that's not the reason why it wouldn't work. :)
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
