On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote: > On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote: > > The best way is to ask for the ENVELOPE and the remaining info using the > > normal BODY.PEEK method. > > Have you actually ever tested this theory? or did you just pull this out > of your proverbial?
I have. The ENVELOPE-way is slightly better because words like "From:" and "To:" are not repeated each time. Although with COMPRESS and/or support for deflate in the TLS/SSL library will probably compress that difference away mostly. Note I didn't test this with compression. I saw a difference of 5% to 10% depending on several factors in bandwidth saving. I usually test all this with wireshark, which doesn't help a lot with compressed data. It would be nice, indeed, to get some analysis of how data is being compressed when compression by either COMPRESS or the SSL library is being done. Note that I haven't yet implemented COMPRESS in camel-lite, but I will do this very soon (I have a CamelStreamGzip thingy ready for this). On top of bandwidth, on pure speed a lot IMAP servers will return your result faster if you only use ENVELOPE. That's because they optimise on the fields that are in ENVELOPE (indexes, etc etc). But then again, there's not enough in ENVELOPE to support all of Evolution's features, so you'll still have to use BODY.PEEK which will most likely slowdown everything back again. > You should note that Thunderbird does not use ENVELOPE, probably because > they found it was LESS efficient to fetch the ENVELOPE + > BODY.PEEK[HEADER.FIELDS (REFERENCES CONTENT-TYPE)] on many existing IMAP > servers in the wild. Probably, indeed. > Having myself tested this in the past, I don't recall it making much > difference one way or the other. On bandwidth, it does. Especially when you have a lot of messages in some folders and the server is not compressing things (no COMPRESS nor TLS/SSL compressing things for you). > Sadly using ENVELOPE breaks on at least one proprietary IMAP server > which occasionally likes to send back responses with NIL as each and > every item, at least until a client has issued a BODY.PEEK fetch > request. Sounds like a bug in that server implementation. > > It should be possible to specify per folder > > which of the headers are to be fetched, to make it really efficient. > > how do you propose calculating the list of headers each folder should > request? which headers would you eliminate conditionally? I'd probably start with the references in case the user doesn't want his headers-view to be threaded, and fetch only the sequence number plus the references header the first time threaded is enabled on that folder and after that (while its enabled) for each summary-item update too. A lot mobile devices's screens are simply not suitable for threaded view anyway: introduces way to much wasted white space. Why fetch this information and waste the user's GPRS bandwidth and therefore often his money too? (for a lot people, GPRS bandwidth costs money per downloaded megabyte) In case the "From" is not displayed, there's no real need to fetch it until the first time the column is added. Same for the dates (date sent and date received, usually only one of both is used by the user). Etc etc etc. > > The imap4 implementation seems to have an ENVELOPE parser, so I guess > > either it does it already or it will use ENVELOPE in future. > > it fetches: > > ENVELOPE BODY.PEEK[HEADER.FIELDS (Content-Type References In-Reply-To)] > > and optionally the mailing-list headers > > I believe it fetches IN-REPLY-TO (which is included in ENVELOPE) > explicitly because some IMAP servers tried to be smart and parse the > IN-REPLY-TO header for us and didn't handle embedded phrases which > certain broken free-software mailers like to send. > > > > > For a mobile device, that works over GPRS, you for example are usually > > not interested (not at all) in the mailinglist specific headers. > > this is mobile-device specific tho, when I was implementing imap4 and > people started trying to use it, it was their #1 complaint (which is why > imap4 conditionally requests the mlist headers... tho that flag may be > hard-coded to TRUE now, I'm not sure) But some people don't use it. Although people more often complain about missing features and not so much about that their E-mail client is wasting their money on GPRS bandwidth, I still believe it's important to support low bandwidth modes too. Also for Evolution, as Evolution is often used with laptops. I do agree, though, that Evolution has a focus difference compared to what I'm doing with Camel. In my opinion is this why splitting Camel from Evolution might help: we could then more easily introduce features for letting Camel switch from low bandwidth mode, to Evolution mode and back. Right now any change that isn't absolutely 100% super pro and for Evolution and Evolution only, gets ignored by you guys at Novell. Usually the explanation goes like this: "Yeah but Evolution has priorities this and Evolution is desktop that ..." The final result of all that this and that is that I just forked it. I do have to say, though, that I'm strongly noticing that more and more tasks of the project are being given to the right people. Maybe things will get better? I'm for example looking forward to cooperating with Matthew on Camel a lot more. > > Note that Tinymail's camel-lite implements all the needed solutions to > > this bandwidth consumption problem. And that its code is, although a lot > > of work because the Evolution maintainers didn't seem interested at the > > time it was happening, definitely portable to upstream. > > > > some of your changes are broken wrt working with all imap servers in the > wild. UID SEARCH, although a lot more bandwidth efficient than UID FETCH for getting a list of UIDS, doesn't work on some servers. For example the Citadel IMAP service used to fail on this, although they have fixed this after they contacted me about the problem. My opinion is that IMAP server developers should ALSO fix their crap. If they want to get users with Mobile devices, they'll have to implement the new Lemonade features, indeed. Although I will put-in some more checks that try to detect whether the IMAP server will incorrectly deal with the changes (so that the old method can be fallen back to), I will not strongly steer Tinymail in the direction of supporting each and every piece of shit IMAP server. If they don't get it right then they should not have users via Tinymail. That's putting the burden of supporting their server-defects on me. > > ps. In my opinion is also the imap4 project getting the majority of > > its design wrong. > > thank you, oh king of software design. My opinion on this kind of "messages to each other", Jeffrey, is that in stead of trying to fight each other, we'd better cooperate and make things better. But then again, this is your choice, not mine. Welcome back to the Camel team, by the way. The Evolution guys can definitely use your experience. -- Philip Van Hoof, software developer home: me at pvanhoof dot be gnome: pvanhoof at gnome dot org http://www.pvanhoof.be/blog _______________________________________________ Evolution-hackers mailing list Evolutionemail@example.com http://mail.gnome.org/mailman/listinfo/evolution-hackers