On 12 Feb 2009, at 17:19, Brian Candler wrote:
Of course, SMTP clients don't mind a small delay before they get
their 250
OK at the end of the message; you can therefore write a number of
batches
and do an fsync() every second or so, as long as you remember not to
send
the acknowledgement back to each client until *after* the fsync has
completed.
CouchDB does the fsync()-a-second currently.
(2) Couch won't let you write a document to disk in chunks; if it
doesn't
get an up-front Content-Length: header then it will buffer the whole
thing
in RAM.
So if you receive very large E-mail messages, you may wish to buffer
them
locally (e.g. in a tempfile on another disk) before sending them as
a single
document to Couch.
Note that you sometimes get an indication of the message size in a
SMTP
transaction, but it's not guaranteed to be accurate; so you won't
know the
true size until you've read it in.
A patch to allow for streaming unknown-length attachments into CouchDB
is in the works.
(3) As you say, messages are stored and deleted frequently. You may
end up
having to compact your message store frequently, which means basically
reading the whole store from start to end and rewriting it to a new
file.
This has to be done when write load isn't too high, to ensure that
it will
complete.
CouchDB's roadmap includes an item about pausing writes to allow
compaction
to catch up on high write loads.
Cheers
Jan
--
