On Tue, Oct 01, 2013 at 12:34:38PM -0500, Felipe Contreras wrote:
> Pushing the updates is not the problem for me, as it's usually a
> couple of messages that get updated, it's fetching them. Even if
> nothing changed at all, it takes a long time.
What's your ping round trip time (RTT) to your IMAP server? Right
now, updating the flags is using a sliding window scheme where we send
a large number of flag updates in parallel, and then as the server
acks them, we send more flag updates. I implemented this way back
when because perviously, the flag updates were done using a lock step
protocol, where we wouldn't send the next flag update until the
previous flag update was ack'ed by the server. This was incredibly
slow when I was on a cruise ship network, where each packet had to go
up to a satellite and then back down, which meant it had a very high
RTT (and on top of that, there was some flaky packet loss). Using a
sliding window approach makes a huge difference --- for those of you
who remember the old XMODEM/ZMODME days, it's the difference between
the XMODEM and ZMODEM protocol.
I didn't do this for the message fetching part of the protocol,
because wrapping the state to allow multiple fetches to be done in
parallel and asynchronously was a lot more difficult, and since the
time to download the a larger message versus the overhead caused by
the lock-step protocol made this not as much of a huge win.
Now, the cruise ship network had a very low bandwidth as well as a
high latency time, so that's why the tradeoff was such that it wasn't
worth my effort to try to optimize the message fetch processing in
mbsync. If you have a network connection which has a higher
bandwidth, but a much longer RTT time --- say, if you were an
astronaut on a Marse mission trying to use mbsync, then it would
definitely make sense to fetch multiple messages in parallel.
Using a single UID STORE command for multiple messages, as Maxim
suggested, would also help, but primarily by optimizing the amount of
data being sent over the network connection (for example, if you are
getting charged by the byte). It would also be quite a bit more
complicated to implement, and this wasn't as big of a deal for me
since the cruise ship network charged by connection time, not by the
kilobyte, and I was trying to get the hack done before the Geek Cruise
was over. :-)
Alas, the Geek Cruises are no longer being run, so I haven't had as
much of an incentive to improve mbsync's IMAP protocol efficiency in a
couple of years. These days, I'm privileged to have high bandwidgth,
low-latency network connections most of the time, so it's much less of
an itch for me to scratch.
Cheers,
- Ted
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
isync-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/isync-devel
