Sam Varshavchik wrote: > Jonathan Briggs writes: >> I also ran strace on the imapd processes. It's pretty ridiculous, >> and it gets to be obvious why it takes so long. Courier is >> re-opening and re-reading the keywords of every message each time a >> new keyword is added. So OF COURSE the operation is O(n^2). Very bad. > > Unfortunately, I've yet to code the telepathic add-on to Courier that > can figure out that when the IMAP client sends a single command to set > one keyword for one message, that's just the first one of tens of > thousands. ... > So, why don't you take it up with Apple why are they too stupid to > figure out that Mail.app can send a single IMAP command, instead of > tens of thousands of individual commands, that accomplies the same > exact damn thing.
I don't think that Jonathan was complaining that the operation takes a long time because there are a lot of commands. He's complaining because Courier's IMAP server reads the keywords too often. After tracing the process, I'm inclined to agree. I started tracing an open session and set one label, and I see this: (I assume the client is in IDLE when I start the trace) > client sends "DONE\r\n" < server creates a dotlock and reads uiddb < server creates new uid file in /tmp, scans cur and new, and then deletes the file < server reads :list and all of the keywords files < server sends "995 OK IDLE completed\r\n" > client sends "996 uid store 176093 +FLAGS ($label3)\r\n" < server stat()s and open()s a file in cur (presumably the one to be labeled) < server creates a dotlock < server reads :list and all of the keywords files < server saves the new keyword < server sends "* FLAGS ($MDNSent NonJunk $Forwarded $label3 \\Draft \\Answered \\Flagged \\Deleted \\Seen \\Recent)\r\n* OK [PERMANENTFLAGS ($MDNSent NonJunk $Forwarded $label3 \\* \\Draft \\Answered \\Flagged \\Deleted \\Seen)] Limited\r\n* 272 FETCH (UID 176093 FLAGS (\\Seen \\Recent $label3))\r\n996 OK STORE completed.\r\n" > client sends "997 IDLE\r\n" < server prepares new, cur, and courierimapkeywords for monitoring by FAM < server sends "+ entering ENHANCED idle mode\r\n" < server reads a bunch of file names from monitor < server creates a dotlock and reads uiddb < server creates new uid file in /tmp, scans cur and new, and then deletes the file < server reads :list and all of the keywords files < server writes a new :list < server renames one of the files in courierimapkeywords < monitor reports new :list and file in courierimapkeywords < server creates a dotlock and reads uiddb < server creates new uid file in /tmp, scans cur and new, and then deletes the file < server reads :list and all of the keywords files < server prepares new, cur, and courierimapkeywords for monitoring by FAM < server creates a dotlock and reads uiddb < server creates new uid file in /tmp, scans cur and new, and then deletes the file < server reads :list and all of the keywords files < server reads a bunch of file names from monitor < server creates a dotlock and reads uiddb < server creates new uid file in /tmp, scans cur and new, and then deletes the file < server reads :list and all of the keywords files That's where I quit tracing. I could have some of those details wrong, and I don't know the IMAP protocol well enough to know how much of that is strictly required, but it looks to me like: * FAM itself (gamin) may be crazy. It appears to be reporting all of the file names in "cur" immediately after setting up the monitoring. This is probably causing a lot of extra work to be done. * Initially I was confused why courier should get notification about its own modifications, but courier might miss events if it shut down monitoring. It does seem like courier should ignore notifications sent regarding file modifications that it has done, itself. * Finally and most to the point (and please correct me if courier is doing what it must here) it seems like a poor design that courier scans all of the keyword files after commands where it doesn't even report keywords (like "DONE". Moreover, scanning the entire structure repeatedly rather than keeping the keywords in memory is going to create massive disk IO for clients that apply a lot of keywords. If Apple Mail tags everything with "Junk" or "NotJunk", then Courier IMAP is more or less unsuitable for use with Apple Mail. A little bit of a side note: I'd love to contribute more to Courier, and encourage other people to do the same, but I've never been really clear on what kind of vision you have for Courier as a project. Are you interested in attracting new developers to the project rather than being basically the only developer? I think you've expressed disinterest in getting Courier into distributions like Fedora, but I dont' know why (if you ever mentioned it, I've forgotten). ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ courier-users mailing list [email protected] Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
