On Mon, 2003-07-07 at 12:59, Jesse Norell wrote: > > My vision here is a "fastheaders" table which has a message number and > > half a dozen columns defined. On some regular basis, or upon message > > insertion, the messageblks table is scanned and the first entry of each > > new message number, which is the header, is parsed for these fast headers > > and they are stored for future searches and quick/short header listings. > > An idea we've started (not completed) implimenting in weDBmail > along these lines is dynamically parsing/caching the headers. Any > time the message headers are requested, it'll check the "header cache" > table and use entries if found, but if not, it'll parse the headers > from messageblks and use the results while saving appropriate ones in > the cache. Could make the pop3 and imap servers do that as well.
Could you expand a little on what you mean by "dynamically parsing/caching the headers" do you mean writing it to the database, keeping them in application memory? etc... Would be interesting to hear this flushed out a little. The idea of doing this at message header request time somewhat defeats the purpose. I think doing this work at message delivery time is optimal. Yes it adds overhead, but messages arrive in a fairly steady stream thus distributing this load over time as opposed to having to do all this work while the user is waiting. A little work at message delivery time to get get the data into an "optimal" state (still to be determined what optimal really means) will certainly help response times, perhaps with a slight increase in overall server load as a consequence.
