On Tue, 19 Feb 2008, Bob Atkins wrote:
A couple of thoughts. On large email boxes, the .mixindex can be very large
so reading the whole thing is 'expensive'. My inbox's index file is 1.2M for
16408 messages.
Correct.
Why is it necessary to read the entire .mixindex before an update?
It isn't strictly necessary for mail delivery; the only benefit is to
ensure that delivery isn't being done to a damaged mailbox (since that may
make the damage worse). The information in the index isn't needed for
mail delivery, other than to make sure that it is sane compared to the
metadata and the size of the current index file.
Now, when a mailbox is actually open in a session, that session needs to
re-read the .mixindex since this is how it discovers new mail and external
expunges.
Currently, these two cases aren't distinguished; and this is an
optimization that can be made.
Is there a
possibility of just reading the last record so you know where it is at?
The last record is only needed as a sanity check; all the information
needed is in the metadata and the sequence record in the index.
Otherwise, it can do an append.
However, reading the last record may be a good idea anyway; it could be
used for a mini sanity check that would catch the most likely forms of
mailbox damage without the cost of doing the entire index.
Then
you could just perform an append rather than a re-write to add the new record
to the file. Same idea for the .mixstatus file. This would also substantially
reduce the lock time and delays for another process that needs to access the
box.
Correct. For the status file, all that is needed is the sequence. In the
append case, we don't care about any of the other data in the status file,
not even for a sanity check.
In theory, if locks are being respected there should be little chance of
corruption - particularly if you are only doing appends when delivering new
messages.
True; in theory.
However, the experience with 10+ years of mbx makes me cautious. I've
seen the "impossible" happen too many times. It's no consolation to the
impacted user that this impossible thing happened due to an operating
system bug or hardware failure.
So, I want to continue with some redundancy and sanity checks, but relax
the extreme level that currently exists. That was my plan from the
beginning.
After all, it doesn't hurt to wear both a belt and suspenders. But after
a while, we can say that the belt and suspenders are good enough, and we
don't also need the seven layers of duct tape also holding the pants up...
;-)
Anyway, I will be looking into this and working on it during the spring.
-- Mark --
http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
_______________________________________________
Imap-uw mailing list
[email protected]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw
