There has been a long-standing problem when new mail arrives on a Netapp
fileserver, which I've finally got round to investigating. The problem is
that when you make a connection (e.g. POP3), only 40 or so new mails are
visible. On the next POP3 connection, the next 40 or so mails become
visible. And so on. Since I get a lot of spam (~150 per day), if I leave my
mailbox for a while then it can take several downloads to get all my mail.
It seems the problem is to do with the loops for reading new mail:
opendir("new") -> readdir() -> rename() [in loop]
After renaming 40 or so files, the readdir() terminates, probably because
the state of the directory is being changed while it is being read. Arguably
this is a Netapp bug, but I'd like to work around it. I don't mind coding
this, but ideally I'd like Sam to support the idea (i.e. so I don't have to
keep a separate patch set indefinitely)
What I'd do in principle is read in the whole "new" directory first into a
linked list in RAM, then afterwards perform the renames and free the list.
It seems the following places need changing:
- imap/pop3dserver.c: scannew()
- imap/imapscanclient.c: do_imapscan_maildir2() step 5
- maildir/maildir.c: maildir_checknew()
Questions:
(1) sqwebmail uses maildir_checknew; would it be sensible to make the pop3
and imap servers use this function as well? Then the logic can be kept in
one place.
(2) if not, then the minimal set of changes to code would probably be by
implementing a new version of opendir/readdir/closedir which worked by
reading everything into RAM first; otherwise the linked-list handling needs
to go in three places.
Thoughts anyone (esp. Sam?)
Cheers,
Brian.
