Paul J Stevens wrote:
> Please do share your ideas. Anything that will boost perfomance will
> be seriously considered.
I've already implemented one performance booster. Well, at least for
MySQL. Want a patch against 2.0? Or should I pull CVS first? This is the
sole question that prevents me from posting a patch.
> > for (i = 0; i < db_num_rows(); i++) {
> > if (db_get_result(i, 1)[0] == '0')
> > mb->unseen++;
> > if (db_get_result(i, 2)[0] == '1')
> > mb->recent++;
> >
> > mb->seq_list[i] = db_get_result_u64(i, 0);
> > }
> >
> > db_free_result();
> >
> > Well, with db_num_rows() in the tens of thousands one would expect a CPU
> > hog here! Three calls to db_get_result for every message - and
> > db_get_result performs seeking every single time, too.
> And only to count the number of recent_flags, and seen_flags for a certain
> subset of message_idnrs all those
> message rows are actually selected !! Clearly usage of COUNT() comes to mind
> as an optimization.
Actually one must also fill in mb-> seq_list, so one could count as well
- if db_get_result would not perform seeking and retrieving when it does
not have to!
I have this implemented already, and the loop now is through in no time.
Moreover, my version is probably faster than yours - because no extra
COUNT is needed, and because my db_get_result won't perform a single
seek in all this loop! It checks if the row we want is the next row from
the one already fetched, and if so, just fetches it!
> > And so it goes on, doing four queries per message! No wonder it can only
> > process about 20 messages per second, instead of 250 as promised in the
> > README.
>
> Yep. You hit dbmail's sorest spot called _ic_fetch. Not so easy to
> fix. And something I've been working on for
> some months now. Using a better mime-parser will give us better
> model-view separation.
I think that any writing of mime-parsers today is a waste of developer
time. There are many lying ready.
But, what has a mime-parser got to do with the number of queries here?
Joining all the queries, except the one for messageblks, into one or two
big queries for ALL messages can be implemented with no changes in the
mime parser, right? And even that will give us a 3 or 4 times better
performance.
Unfortunately, this time (unlike db_get_result) I can't jump in and
implement it myself, because I don't understand the C code in _ic_fetch.
Yours, Mikhail Ramendik
