On Tue, 2003-03-11 at 22:56, Mark Crispin wrote: > That's not the right view. You should instead have "how can I build a > good client within the context of IMAP, without expecting the existing > IMAP world to add facilities to enable me."
That's how I _started_. I looked at the way UIDs allow me to cache message headers on the client so they don't need to be re-downloaded. I looked for an equivalent mechanism for flags, and found it to be absent. Given that even if an extension allowing flag caching _does_ come into existence I'd still have to implement the non-caching variant for compatibility, perhaps I should ignore this particular problem for the moment and come back to it if and when it actually starts to be a significant problem. After all, although Evolution is taking 10 seconds to open certain folders at the moment because it's re-downloading flags, it doesn't actually _need_ all of those flags, so perhaps it'll turn out that having to re-download them isn't quite as much of a showstopper as it is right now, although it'll still be suboptimal. > > Not entirely. I don't necessarily want a _complete_ synchronised state > > -- but neither do I want to have to discard the state I _do_ need, and > > download it again shortly thereafter, just because the server gives me > > no way to be sure that nothing's changed in the meantime. > > I keep my IMAP clients (at least, one at home and one in the office) > running for days at a time. I have two incoming mailboxes and some > newsgroups. All other mailboxes are ones that get updated by my action. I prefer to do the same. The startup time is far from negligible. But I got out of the habit of leaving them running while using wu-imapd, because the imapd would keep killing itself if two clients looked at the same folder at the same time. Now I'm _staying_ out of that habit because Evolution has the same behaviour w.r.t. message flags as you describe below -- it doesn't re-download them in the same session (i.e. the life of one 'evolution' invocation). Which is somewhat unfortunate if the flags are changed by another client, because we never notice till I exit and restart Evo. The _correct_ behaviour, according to IMAP, is to redownload those flags we care about _every_ time the folder is reselected. > > Well, maybe. More fundamentally though, I'd say that this is a > > consequence of living on the wrong end of a 64K ISDN link and hoping > > that somewhere, somewhen, I can eliminate _any_ redundant traffic to the > > IMAP server. > > I once regularly used IMAP over a 2400 bps line, and to this day I still > use IMAP over a CDPD device. Please do not treat me as someone who > doesn't understand the issue of slow lines. I wouldn't claim you don't understand. I'm just trying to express the pain that's a large part of my motivation :) > My claim is not that you should re-download data that you already had. > Rather, you should not re-download anything unless you need it; and by > following a strict "don't need it, don't download it" will buy you more > than efforts to keep the caches of a dozen clients in synchronization with > each other. I am in violent agreement with this. I _really_ don't want to re-download data that I already had. This is why I'm looking for a way to _know_ when I need to do so and when I don't, because currently the status is that I have to re-download it unconditionally. Yes, of course I shouldn't be downloading anything I don't _need_ in the first place, but that is largely an orthogonal issue. Extensions for server-side threading have already allowed clients to get away with requiring far less to be downloaded in the first place; that alleviates but doesn't entirely remove the requirement for being able to cache what we _do_ need without having to redownload it. > > And redownloading message flags which I already had and which haven't > > changed is definitely redundant -- especially if it's doing so just > > because I temporarily selected my inbox to read a new mail therein, > > before returning to the mailing list I was reading a moment ago. > > So why didn't you spawn a separate session for the other mailbox? TCP > sessions are cheap. Well, it's not actually just TCP. It's 'ssh mailhost.$COMPANY.internal exec imapd' where ssh knows that for *.$COMPANY.internal you actually run 'ssh bastionhost.$COMPANY.com exec netcat %h %p' instead of making a TCP connection... So it's not _that_ cheap, and not that fast to start up either. And I can't just open one IMAP connection for _every_ folder I visit in an MUA session -- there have to be limits on the number of connections I open. Besides which, it doesn't fix the case where I _have_ actually closed the MUA and restarted it, or where a temporary network outage has caused a disconnection. But yes, making the client automatically spawn separate IMAP connections in order to maintain state on some folders is a possibility which allows us to work around the lack of caching info. It would have helped us work around a lack of UID too, to a large extent. But it is a workaround, not a real fix. It'll also need some thought on which folders we keep open -- the set of folders is likely to exceed the number of concurrent connections I can make, and a na�ve round-robin behaviour is likely to be pessimal, without ever actually _using_ a folder we already had open from earlier. > > I tend to keep coming back to Pine too, and have tended to use it as an > > example of 'ideal' IMAP behaviour. But it's not quite perfect either -- > > it _could_ cache headers locally but doesn't, and this means that > > sometimes it takes _seconds_ just to draw the screen after I hit PgUp in > > a message index, saturating my link while it's at it. > > I have never had it take more than a second or two, even with CDPD. It > all depends upon the size of the envelopes. It's about five seconds per page (19 mails) for me this morning. How does the size of the envelope matter -- we're only displaying four fields. Are we downloading headers we don't actually need? > Have you noticed that Pine *does* completely cache within a session? As > long as you don't give up the session, Pine will never re-fetch the same > data. I'm not sure what you mean by 'session' in this context. When changing between folders, I find that Pine caches stuff for as long as I have a given folder open, but if I change away and back to that folder, Pine does re-download even the message headers. And, of course, it _has_ to redownload the flags since it can't know if they've changed. -- dwmw2
