Mikhail Ramendik wrote:
(1) Only select the header messageblk when only the header or a part of
it is to be fetched on IMAP.
That's what the is_header field is going to be used for.
Currently, from what I can gather from the logged queries and the code,
all messageblks are selected.
Note: the code is hard to read and I could find no documentation on the
database, so I may be wrong here. If so please correct me...
hard to read... understatement at large. That code is a total bitch. A messy heap of shit which makes
overcooked spagetti look like soldiers on parade. I actually read that code. Not once, but many times. No
kidding, and not funny so stop laughing.
(2) Create an index for the 'messageblk' field in dbmail_messageblks. At
least MySQL allows this (can anyone tell me if Postgres does?). Then, on
IMAP header field searches, do not load/parse/check; instead, create a
regular expression and do a SELECT with it, selecting only header
blocks. (MySQL specific again, Postgres comments welcome).
well, the messageblks can become largish, and mysql also has some limit here. Too bad innodb doesn't support
full-text searches.
I wonder what kind of index we get when we add one to the messageblk field... I should try that some time. But
not on my main development machine :-)
This will probably result in a *dramatic* speedup, at the cost of some
coding complexity.
This would actually simplify a lot of code.
BTW, why is the "is_header" field unused in dbmail_messageblks? Or at
least I found no place in the code that would use it. Is it redundant?
Forward compatibility. I did commit a patch to cvs-head to start using this field for message insertions, and
have posted a bash/awk script that will fill this field for existing messageblk rows.
If there is no other way to SELECT only header messageblks, it certainly
should be used.
My new message interface code does provide for it's use during message retrieval (dbmail-message.c), but this
api is not yet deployed in the critical parts of the current code base. Small steps and all.
P.S. To be very honest, I did not like the coding style in dbmail. The C
language for this task would not be my choice, but never mind that.
Functions that are hundreds of lines long, with only minimal comments,
are much more problematic.
The original author was probably a very smart guy :-) but a not so very
experienced programmer.
Since Ilja took over things have improved quite a lot, but some code like the mime-parser and _ic_fetch he is
also uncomfortable with.
I actually volunteered to work on that code last summer :-/ and I've already begun splitting up some of those
functions. The recent addition of struct ImapSession in dbmail-imapsession.c is phase one of that refactoring.
The next phase (splitting up and cleaning _ic_fetch) is well underway and currently being tested. Still much
remains to be fixed, simplified, cleaned-up, etc, etc.
Perl or Python (I prefer Python) with their ready-made RFC822 parsing,
along with some DB-expert friends, would help me write an alternative
database store quickly. But I really am not up to implementing protocols
(SMTP, LMTP, POP3, IMAP). If any Python guru here would go for that, we
could try a rewrite :)
I actually started out working on dbmail while studying the twisted framework which has finished imap and pop3
implementations. Should be quite easy to write sql-based storage engines for those interfaces.
But there are some advantages to refactoring the current code. And c is just another language to write code
in. With glib for datatypes, and gmime for message-parsing that code could actually become more fun not only
to write, but better yet, to read again later.
With the low level of output from Ilja recently, I am becoming somewhat concerned that maybe IC&S have decided
to sink their investment in dbmail.
Should I find myself alone in actually working on the current codebase I will re-evaluate dbmail's viability,
and may indeed decide to either go for a complete rewrite in python or move on to other projects.
Aaron, what do *you* think?
--
________________________________________________________________
Paul Stevens mailto:[EMAIL PROTECTED]
NET FACILITIES GROUP PGP: finger [EMAIL PROTECTED]
The Netherlands________________________________http://www.nfg.nl
