Mrs. Brisby wrote:

The linked-list structure of overflow storage is part of the problem. But the fact that SQLite uses synchronous I/O is also a factor. In order to make BLOBs fast in SQLite, I would have to change to a different indexing technique for overflow storage *and* come up with some kind of cross-platform, asynchronous disk read mechanism.

D.R.Morrison (1968)'s PATRICIA would certainly be faster for indexing
large objects.

A key feature of SQLite 3.0 (needed to support internationalization) is the ability of users to specify their own comparison functions then have SQLite use that comparison function to order indices. PATRICIA does not support user-defined comparison functions. Keys in PATRICIA must occur in memcmp() order, as far as I am aware.

Asynchronous read isn't necessary, but vectored reads are. Consider readv() POSIX 1003.1-2001 -- in fact, you could probably make result-fields return a struct iovec * that would "point" to the value within the database.

readv() doesn't help, actually. BLOBs are stored in 1k blocks scattered all over the file. readv() reads a continguous range of bytes - it puts those bytes into scattered buffers but the bytes must originate from a contiguous region of the file. I'd still have to do 1024 sequential readv()s in order to extract a 1MB blob.

-- D. Richard Hipp -- [EMAIL PROTECTED] -- 704.948.4565

--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to