Sabahattin Gucukoglu via Courier-imap writes:
Incidentally I have since poked about in imapd’s source and concluded that wherever I got my information about syscall overuse, it was just wrong— fetching flags can be done using just information encoded in the filenames of messages, no stat() required. So that’s a relief. I can only conclude
This is correct. The only limiting factor is the absolute number of messages in a mailbox. A FETCH of FLAGS and UID is sufficient for the client to completely resynchronize itself, however if you have ten thousand messages, the bandwidth does add up, and takes time to process it.
that perhaps sorting or threading was involved by requirement of some broken (non-caching) webmail or other.
IMAP has historically suffered from several built-in design flaws. Just because an IMAP client can go about doing its thing in a certain way doesn't mean that it should, and many IMAP implementations have ignored certain practical realities, resulting in sub-optimal results.
The CONDSTORE extension is a perfect example. It's a perfect IMAP extension, that exhibits the same underlying problems that IMAP itself has. If the goal is to provide for efficient client/server resynchronization, this can be done far, far simpler than all the baggage that goes into CONDSTORE+QRESYNC. There's a huge pile of functionality in CONDSTORE+QRESYNC that has nothing to do with resynchronization, but the server has to implement it; and I doubt whether any client actually even uses it. Why? Well, because CONDSTORE+QRESYNC is an optional extension, and if the server doesn't support it, what are the client's choices? Disable whatever extra functionality the client needs the extension is for? That's not going to fly. No, instead the client will simply fallback to IMAP core functionality, and as long as the client has to code that anyway, there's very little reason to then also implement the extension too, in the client, unless the server can do the extension far more efficiently. But in CONDSTORE's case, most of the extension's functionality seems to be rather dubious, and is a solution for an unclear problem.
As I mentioned, client/server resynchronization is a much simpler task. In your poking of Courier-IMAP's code you must've missed the parallel implementation of an alternative client/server protocol, which I called SMAP, that I did, for some research, a long time ago. It's still there, it just needs to be enabled. And Cone implements it on the client side. And one of its aspects is exactly this: efficient client/server resynchronization.
Resynchronization is a much, much simpler task than the arcane way CONDSTORE+QRESYNC goes about doing it. All that needs to happen is for the server to save the current state, a snapshot of the mailbox, and send an opaque handle to the client, representing the snapshot. The server only needs to save one snapshot per client, and each snapshot simply discards the previous one (well, two most recent snapshots need to be saved, to make this bulletproof).
When the client opens a mailbox it provides the snapshot handle it knows about to the server, which then loads the saved snapshot, and then proceeds and resynchronizes its state, the same way it normally does, and then update the client using the responses it already does, i.e. EXPUNGE, FLAGS of any messages' changed flags, the number of EXISTting messages itself. 99% of what needs to be done, to resynchronize the client/server, already exists as part of the core protocol so why invent a completely new way of doing that?
All of that is somewhat analogous to CONDSTORE's sequence numbers. Except that, from what I can tell, the server must maintain a complete history of all changes to the mailbox, in order to support CONDSTORE+QRESYNC. This seems completely unnecessary itself, too onerous, and it misses the mark entirely. The server only needs to maintain a single per-client snapshot (with the next most recent backup snapshot), identified by a single opaque handle.
pgpSv1mksK7N6.pgp
Description: PGP signature
_______________________________________________ Courier-imap mailing list Courier-imap@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-imap
