Re: [Evolution-hackers] RFClue: Atomic folder updates

2011-12-04 Thread David Woodhouse
On Sun, 2011-12-04 at 10:04 -0700, Sankar P wrote:
> My memory is very rusty and I have not seen Evolution sources in more
> than 3 years now. However, we had a similar situation when we working
> on a GroupWise backend and we used a trick to get the number of
> messages in a folder on startup of Evolution (or when the user
> explicitly presses the getmail button) and try to compare the mail
> counts between summary and server and update if they differ (and
> obviously, after fetching new items in this delta)
> 
> I am not sure if Exchange server has such a count API. If so, it can
> be used without breaking the summary code much. 

We do something similar to that for IMAP — using the counts as a sanity
check and falling back to the old-style non-QRESYNC method of refetching
the flags for *every* message in the folder.

I don't believe we have fallback like that for EWS and ActiveSync. Those
didn't have sane delta-based operation 'tacked on' to an old protocol
which we can fall back to; they were designed to be delta-based from the
*start*, and the client is expected *not* to get itself out of sync.

Even if we could detect a problem in ActiveSync, we'd basically have to
refetch the entire folder to recover — and that would mean that message
IDs all change, so our cache has to be entirely blown away. I *think* we
could keep the cache in EWS and just refetch the message details, but I
still don't like it — it would be a horrid workaround for a bug, if we
ever have to use it.

When the server responds, it tells us "here are a bunch of changes, and
your new 'bookmark' for the folder state is XXX". We should remember
that, atomically. If we crash, our cache on restart should reflect
either the state *before* the response, or the state *after* it. Never
something in between.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] RFClue: Atomic folder updates

2011-12-04 Thread Sankar P
My memory is very rusty and I have not seen Evolution sources in more than 3 
years now. However, we had a similar situation when we working on a GroupWise 
backend and we used a trick to get the number of messages in a folder on 
startup of Evolution (or when the user explicitly presses the getmail button) 
and try to compare the mail counts between summary and server and update if 
they differ (and obviously, after fetching new items in this delta)

I am not sure if Exchange server has such a count API. If so, it can be used 
without breaking the summary code much.

Sankar
http://psankar.blogspot.com 

>>> On 12/3/2011 at 05:35 AM, in message
<1322870713.5191.20.ca...@shinybook.infradead.org>, David Woodhouse
 wrote: 
> We have at least three mail protocols now which are delta-based. That
> is, you have a bookmark, 'sync key' or timestamp which represents what
> the server *last* told you about a given folder. You say to the server
> "what changed since XXX", and get back a list of added/removed/changed
> messages along with a *new* timestamp YYY.
> 
> It's a very efficient way to handle mailbox access, and it's used by at
> least ActiveSync, EWS and IMAP+QRESYNC. In the Exchange protocols it's
> called 'SyncState' or 'SyncKey', and in IMAP it's the HIGHESTMODSEQ.
> (It's never *actually* a timestamp, since wall-clock time is a PITA. But
> it's easy to *think* of it as as timestamp; the modification time on the
> folder).
> 
> The problem is, we need to handle these updates *atomically*. If we
> store the new timestamp before the changed messages, and we crash in the
> middle of doing so, then we might miss out forever on the messages in
> question. We'd restart, go to the server and say "what happened since
> YYY" and we never get told *again* about the messages which came in
> between time XXX and time YYY, that we didn't manage to fetch.
> 
> And if you do it the other way round and store the changed messages
> first, and crash before you store the new timestamp, you get similar
> issues (cf. bug 664637).
> 
> I'm not sure how to fix it. Looking at EWS first, since that's where we
> noticed it, I pondered removing the camel_folder_summary_save_to_db()
> call from the camel_ews_utils_sync_{created,updated,deleted}_items()
> helper functions, so it happens just *once* at the end of the loop in
> ews_refresh_info_sync() and commits all the changes at once. But just
> deferring the camel_folder_summary bits isn't enough, is it? The
> individual message_infos will have a lifetime of their own, and even
> *internally*, camel_folder_summary_save_to_db() doesn't actually write
> things out atomically using transactions in sqlite... does it?
> 
> Any suggestions or insight would be gratefully received...
> 
> -- 
> dwmw2
> 


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers