On Mon, 2004-04-19 at 20:36, D. Richard Hipp wrote:
> Mrs. Brisby wrote:
> >>
> >>The linked-list structure of overflow storage is part of the problem.
> >>But the fact that SQLite uses synchronous I/O is also a factor.  In
> >>order to make BLOBs fast in SQLite, I would have to change to a different
> >>indexing technique for overflow storage *and* come up with some kind
> >>of cross-platform, asynchronous disk read mechanism.
> > 
> > 
> > D.R.Morrison (1968)'s PATRICIA would certainly be faster for indexing
> > large objects.
> > 
> A key feature of SQLite 3.0 (needed to support internationalization)
> is the ability of users to specify their own comparison functions then
> have SQLite use that comparison function to order indices.  PATRICIA
> does not support user-defined comparison functions.  Keys in PATRICIA
> must occur in memcmp() order, as far as I am aware.

Why not fold the strings at insert time to keep your indexing simple?
You can still get internationalization, but require the user supply a
function with performs this folding:  -> a for example.

> > Asynchronous read isn't necessary, but vectored reads are. Consider
> > readv() POSIX 1003.1-2001 -- in fact, you could probably make
> > result-fields return a struct iovec * that would "point" to the value
> > within the database.
> > 
> readv() doesn't help, actually.  BLOBs are stored in 1k blocks scattered
> all over the file.  readv() reads a continguous range of bytes - it
> puts those bytes into scattered buffers but the bytes must originate
> from a contiguous region of the file.  I'd still have to do 1024
> sequential readv()s in order to extract a 1MB blob.

My brain fizzled out there for a moment. I don't know where I was.

Sadly you're right. While, POSIX 1003.1-2003 does define aio_read(), it
still is a portability nightmare.

On systems where context switches are cheap, one could use fork() or
posix threads to populate a number of pipes, but doubt this would buy
much (if anything)... ever.

Wouldn't it be nice if poll() actually did something interesting with
regular files? :)

Looks to me like you can either make two I/O policies (or more), sort
your reads/seeks, OR move the blobs into another file :)

To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to