David A Galbraith CIRT wrote:
> The problem I have with doing this, is what happens if the dbm file is
> held on NFS. The great thing aboug Maildir is that it is NFS "robust".
> To add something like this will change that. How does gdbm act if it goes
> to read a record and because of NFS cacheing from another client only
> half the record is there... that sort of thing... Otherwise I would have
> written the code into the maildir.c file for UW's Imap long ago :)
The simple way to get around locking with database files is whenever you want
to update the file, make a complete copy to a unique filename and modify that
file. Then move the temporary file over the original file to do an update. This
keeps you from corrupting the database file without locking, but concurrent
updates get lost.
What if the whole side-by-side index code was written like this:
When you read a maildir, first make your own private copy of the database file.
Then you readdir() all of the cur and new directories and compare with the keys
in the database file. Delete any messages from the database file that don't
exist in the maildir. Then read in the header information from any new messages
found in the Maildir into the database file. From there you are ready to roll.
You see, the problem is that other programs may be delivering and receiving
from the maildir, so we can only really trust the database file to have
information for some messages, but we can't be assured that any message in the
maildir will be in the index. The maildir is authoritative. So, whenever we
want to use the index we update it, which requires reading the directories and
then reading header info from any new messages.. not too bad.
Because we don't trust the index to be complete, we can live with the
concurrent update problem where some updates will get lost. Sometimes messages
will be read into the index twice. No big problem.
However, if you allow people to modify the messages once in the Maildir, then
we have a problem because when the index is rebuilt, each message file has to
be stat()'ed to see if it has changed.. this is an inode lookup so we can
easily end up bouncing all around the filesystem causing lots of seeks in the
hard drive. Not good.
Maildir is really not that well designed for creating an efficient IMAP client.
- David Harris
Principal Engineer, DRH Internet Services
