I'm using the version of courier-imap that comes with 0.41.
I have been experimenting with a few things, and have noticed that imapd
is almost unbearably slow when moving (actually, COPY 'ing) messages to
a Maildir that is 'large'. By 'large' I qualify that to mean roughly
9000 messages. I am running under Linux 2.4.20, and use the ext3
filesystem, although I've run my experiments on both the IBM jfs and the
ext3 filesystems. The filesystem is not the issue here, however.
Simple trace work shows me that an inordinate amount of time is spent in
the getdents system call -- at times over 70% of the *total time* spent
by the imapd.
Source code analysis shows me that the function do_imapscan_maildir2 is
where most of the time is being spent.
Unless I'm misunderstanding things, by my observations imapd is spending
a great deal of time maintaining the uid database. So much so that it
appears to utterly dominate the time spent in doing COPY operations.
Below is copied an strace -c output of copying 2000 messages to a
mailbox with 9000 messages in it:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
72.83 10.104959 241 41880 getdents64
17.04 2.363832 70 33852 write
2.61 0.361850 8 43916 brk
1.47 0.203354 565 360 link
1.30 0.180502 332 543 connect
1.06 0.147273 68 2166 361 open
0.99 0.138057 59 2349 close
0.79 0.108985 67 1625 723 stat64
The connect calls consist entirely of:
connect(3, {sin_family=AF_INET, sin_port=htons(111),
sin_addr=inet_addr("127.0.0.1")}}, 16) = 0
connect(3, {sin_family=AF_INET, sin_port=htons(797),
sin_addr=inet_addr("127.0.0.1")}}, 16) = 0
connect(5, {sin_family=AF_UNIX, path="/tmp/.famkDJfFd"}, 110) = 0
The write is likely unavoidable, as is the brk and link.
Still, clearly the most expensive portion is the getdents call.
The last two are FAM related.
Not sure why the connect to portmapper. The filesystem is local.
What can be done to improve things here?
I *am* running with IMAP_USELOCKS=1.
I've tried it with IMAP_USELOCKS=0 (default) although not much
difference was noticed.
Sam -- why is it necessary to (seemingly) completely rebuild the UID
db here for COPY? Can that be done less frequently somehow, while still
maintaining UID information?
Also, even more worrysome, is that during this test, I noticed disk
space very very rapidly declining. In ~/Maildir/tmp there were many
800+ KB files of the form:
1047264838.M224342P7612_imapuid_1716.honker.localdomain
The problem was here:
rename("./tmp/1047264752.M614842P7612_imapuid_1458.honker.localdomain",
".gpz2/courierimapuiddb") = -1 EXDEV (Invalid cross-device link)
I did not know it was a requirement that all folders be on the same
system. I would suggest that the proper location for the imapuid
temporary file would be in the destination maildir tmp/, not the INBOX
tmp/.
--
Applying computer technology is simply finding the right wrench to
pound in the correct screw.
Jon Nelson <[EMAIL PROTECTED]>
C and Python Code Gardener
-------------------------------------------------------
This SF.net email is sponsored by: Etnus, makers of TotalView, The debugger
for complex code. Debugging C/C++ programs can leave you feeling lost and
disoriented. TotalView can help you find your way. Available on major UNIX
and Linux platforms. Try it free. www.etnus.com
_______________________________________________
courier-users mailing list
[EMAIL PROTECTED]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
