So, the most recent discussion about IMAP started me thinking ... how bad would a connection for every command really be?
I wrote a command called "imaptest", and I put it into the nmh distribution. It will be build by default, but not installed. It needs a LOT of cleaning up, but it is sufficient to gather some data if you are conversant with the IMAP specification. I started poking around a bit; the Cyrus-SASL archives are available via anonymous IMAP, so this lets you access them: imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody However it should work with any IMAP server that supports the SASL mechanisms we support (it also works with TLS, both STARTTLS and "initial" tls). If you wish to use it, you probably want the -snoop flag so you can see what is going on (we it does no verification of the IMAP syntax, just tries to match up responses). Just to give you an idea, here's what we're talking about in terms of response time. This is basically doing a 'scan last:200' on the Cyrus-SASL archives: % imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody -timestamp +archive.cyrus-sasl 'FETCH 10226:10426 (FLAGS RFC822.SIZE BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)' Connect time: 0.018900 sec Command execution time:: 0.018105 sec Authentication time: 0.037288 sec Command execution time:: 0.068890 sec Command execution time:: 0.056778 sec Total command execution time: 0.125706 sec Command execution time:: 0.018582 sec Total elapsed time: 0.200631 sec There is some variance here. It seems that the DNS TTL for 'cyrus.andrew.cmu.edu' is set to 1 (!) second, so if it has to do a DNS query it takes noticably longer: % imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody -timestamp +archive.cyrus-sasl 'FETCH 10226:10426 (FLAGS RFC822.SIZE BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)' Connect time: 0.433292 sec Command execution time:: 0.017532 sec Authentication time: 0.035130 sec Command execution time:: 0.061797 sec Command execution time:: 0.056097 sec Total command execution time: 0.117972 sec Command execution time:: 0.018482 sec Total elapsed time: 0.605052 sec If you add encryption with -tls, it's actually not so bad (cached DNS this time): % imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody -timestamp +archive.cyrus-sasl 'FETCH 10226:10426 (FLAGS RFC822.SIZE BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)' -tls Connect time: 0.019291 sec Command execution time:: 0.018414 sec Command execution time:: 0.017851 sec Authentication time: 0.113366 sec Command execution time:: 0.054519 sec Command execution time:: 0.057359 sec Total command execution time: 0.111911 sec Command execution time:: 0.018694 sec Total elapsed time: 0.263568 sec And if you decide you want the WHOLE mailbox, it's not really that bad either: % imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody -timestamp +archive.cyrus-sasl 'FETCH 1:10426 (FLAGS RFC822.SIZE BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)' Connect time: 0.019226 sec Command execution time:: 0.017619 sec Authentication time: 0.035386 sec Command execution time:: 0.053753 sec Command execution time:: 0.456180 sec Total command execution time: 0.510001 sec Command execution time:: 0.018752 sec Total elapsed time: 0.583572 sec Even with TLS, not that bad: % imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody -timestamp +archive.cyrus-sasl 'FETCH 1:10426 (FLAGS RFC822.SIZE BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)' -tls Connect time: 0.019436 sec Command execution time:: 0.017595 sec Command execution time:: 0.017777 sec Authentication time: 0.111909 sec Command execution time:: 0.068417 sec Command execution time:: 0.437721 sec Total command execution time: 0.506181 sec Command execution time:: 0.018700 sec Total elapsed time: 0.656586 sec Some things jump out at me after playing with this a bit. A FETCH on a single message is around 18/20ms. If you just do a simple client, with a single FETCH and response for every message, that would be over 200 seconds to fetch the necessary information to do a scan(1) on all 10426 messages in that mailbox. But done as one command it's a half second! That suggests to me that really, a new connection for every mailbox actually might be fine. Now, we'd have to TOTALLY rethink the internal API to make this work; the key there is you need to figure out which messages you were interested in before you started talking to the IMAP server, to minimize the number of network calls. But I think that's achievable. And of course there's the mapping between MH message numbers and IMAP messages, but I think that's not that hard; obviously you'd use UIDs, and I think making it work is relatively straightforward If anyone else wants to play around with imaptest, please try it out! I am sure there are bugs (and features to add). The code is kind of a mess, sadly; improvements welcome in that area as well. --Ken _______________________________________________ Nmh-workers mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/nmh-workers
