Hans Kristian Rosbach wrote: > On Fri, 2005-02-11 at 11:01, Aaron Stone wrote:
[...] > > So wait -- the MTA takes the message, puts it in a file to queue for > > delivery. Then the MTA reads the file either into a pipe or tcp connection > > to dbmail. Then DBMail puts the message into a file to piece into the > > database. So for every message, we put it on disk twice and into memory > > three times... > > Well, actually the file can be opened with flags that hint to the OS > that this is indeed a temporary file that might just be stored into > memory cache and never even touch the disk (OS will decide depending > on wether enough memory is free and how long it has lived in > memory). Just mmap() the file into memory. The end result is that real memory isn't used at all, but the file can be accessed as if it had been read directly into the memory area returned by mmap(). You can mmap() a chunk at a time if you're concerned about using too much of your process' address space. But realistically, a messageblk should be large enough to give decent performance when inserting a large message into the database and when retrieving it, but should not be so large that the system runs out of memory when lots of concurrent dbmail processes are running. As long as you can read the incoming message as a stream and break it up into messageblks on the fly, you'll be set. Things like this are why you really want to be dealing with a transactional backend. If the message EOFs before it should, running in a transaction gives you the ability to back out of the whole thing easily (just ROLLBACK) if that's appropriate. > > If DBMail became an MTA, then we could write messages to disk in CSV in > > the first place, and use LOAD DATA INFILE (which is standard ANSI, I > > believe) to stuff it into the database once we've done the delivery stuff. Ick. Such an operation is definitely not portable, and probably isn't transactional, either. -- Kevin Brown [EMAIL PROTECTED]
