On 5/7/07, Jonathan Vanasco <[EMAIL PROTECTED]> wrote:
Ah, I reread the post. I saw "large lists" and thought "complex data structure", not simple text.
I think we were talking about different things, actually. For reading and writing a large and complex data structure in its entirety, a Storable file is as good as it gets, unless you can rig something with an mmap'ed file. I was talking about reading/writing pieces of it in BDB or MySQL, which should not need Storable.
I didn't know that BDB does shared memory caching.
And no socket overhead too. All the calls are in-process.
> Primary key lookups in MySQL over local sockets are very fast -- > faster than memcached. Really ? I had read that they were about the same, but that mysql selects are blocking & FIFO , while memcached is threaded and supports concurrent access.
Memcached is single-threaded and uses non-blocking I/O. It's a very different approach from a multi-threaded daemon like MySQL, and should scale better ultimately. However, for simple lookups, the network overhead from the TCP socket that memcached requires seems to outweigh any advantages. MySQL can use a pipe instead of a TCP socket, which it does automatically when you connect to a server on localhost. - Perrin
