Re: [Evolution-hackers] API considerations for avoiding data loss in race conditions
On Thu, 2012-12-06 at 10:43 -0500, Matthew Barnes wrote: > On Thu, 2012-12-06 at 16:22 +0100, Christian Hilberg wrote: > > * EClient starts transaction / acquires lock on PIM > > entry > > > > * E-D-S records transaction start / gives away the PIM > > entry lock to the EClient requesting > > > > * EClient dies for any reason > > > > * Started transaction / aquired PIM lock persists in > > E-D-S > > > > What now? Timeout after some time? Welcome back to possible > > raciness (-->does not truly solve our problem). > > This scenario is easily handled by watching the bus name of the client > process that acquired the lock. If the bus name vanishes, release the > lock immediately. > > But we have no such transaction lock at present, do we? Hi, nope, there is no such API for it, and I do not think it's of any use when you consider the eds<->groupware server case. From my point of view the client should take care of the potential issues when modifying/removing the contact/event/ When you look at it then the chain is: EClient <-> eds backend <-> remote server The EClient is a client for the eds backend. The eds backend is a client for the remote server. And you cannot expect remote servers running transaction locks on items when any client decides to take one. Evolution, as one of the EClient user, has capability of notifying user about received changes, at least the calendar part component editor notifies user when the item is opened and the view receives update of it. And that's the way to go. The connection between eds backend and remote server is harder, because the backend cannot talk to user directly right now, but I'm working on a service which will allow it, which should help too. Nonetheless, the conflict strategy is currently chosen by each respective backend coder, rather than left to a user. That said, I would not go with the "transaction locking", it doesn't fix much. I'd rather update book and calendar DBus/backend/EClient API to include necessary parameters for conflict resolution (enum for conflict resolution on modify and expected revision/last-modified for remove operations), and the client should implement what to do, if it gets Out-of-Sync error. The best if there is some centralized place for conflict resolution, thus the EClient users (and backends) will not need to duplicate the code. Though that's probably another question, and a project on its own. Bye, Milan ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] API considerations for avoiding data loss in race conditions
On Thu, 2012-12-06 at 16:22 +0100, Christian Hilberg wrote: > * EClient starts transaction / acquires lock on PIM > entry > > * E-D-S records transaction start / gives away the PIM > entry lock to the EClient requesting > > * EClient dies for any reason > > * Started transaction / aquired PIM lock persists in > E-D-S > > What now? Timeout after some time? Welcome back to possible > raciness (-->does not truly solve our problem). This scenario is easily handled by watching the bus name of the client process that acquired the lock. If the bus name vanishes, release the lock immediately. But we have no such transaction lock at present, do we? Matt ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] API considerations for avoiding data loss in race conditions
Am Donnerstag 06 Dezember 2012, um 16:13:59 schrieb Christian Hilberg: > Hi all! > > Thanks Tristan for bringing an issue up again which is critical for > PIM software, especially when dealing with groupware servers. > > Am Donnerstag 06 Dezember 2012, um 15:27:49 schrieb Tristan Van Berkom: > > Hello all, > > I'd like to raise this issue on the list for feedback regarding this > > bug: https://bugzilla.gnome.org/show_bug.cgi?id=686684 > > > > First let's start with the basic problem statement: > > ~~ > > > > It can happen that two clients modify the same contact simultaneously, > > or... that one client loads a number of contacts into memory and then > > commits that contact to the addressbook without refreshing the contact > > first, this will result in lost data for the given contact. > > [...] > > Very short note: > > * This issue is not limited to contacts, but can as well hit > calendar (event, task, todo) entries. > > * This issue is also not limited to the EClient<->E-D-S case, > but can occur in the E-D-S<->GroupwareServer case as well > in very much the same way (even without E-D-S offline > capability - the "one has it open for editing while the > other opens, edits, and stores"-case applies also in > online mode if we deal with shared PIM folders). > > To me, this calls for a general approach of how to deal > with PIM synchronization conflicts. Oh, and talking about locks/transactions, please consider this: * EClient starts transaction / acquires lock on PIM entry * E-D-S records transaction start / gives away the PIM entry lock to the EClient requesting * EClient dies for any reason * Started transaction / aquired PIM lock persists in E-D-S What now? Timeout after some time? Welcome back to possible raciness (-->does not truly solve our problem). That same scenario is - again - not limited to EClient<->E-D-S, but can also hit E-D-S<->GroupwareServer. Just 2 cent, Christian -- kernel concepts GmbH Tel: +49-271-771091-14 Sieghuetter Hauptweg 48 D-57072 Siegen http://www.kernelconcepts.de/ signature.asc Description: This is a digitally signed message part. ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] API considerations for avoiding data loss in race conditions
Hi all! Thanks Tristan for bringing an issue up again which is critical for PIM software, especially when dealing with groupware servers. Am Donnerstag 06 Dezember 2012, um 15:27:49 schrieb Tristan Van Berkom: > Hello all, > I'd like to raise this issue on the list for feedback regarding this > bug: https://bugzilla.gnome.org/show_bug.cgi?id=686684 > > First let's start with the basic problem statement: > ~~ > > It can happen that two clients modify the same contact simultaneously, > or... that one client loads a number of contacts into memory and then > commits that contact to the addressbook without refreshing the contact > first, this will result in lost data for the given contact. > [...] Very short note: * This issue is not limited to contacts, but can as well hit calendar (event, task, todo) entries. * This issue is also not limited to the EClient<->E-D-S case, but can occur in the E-D-S<->GroupwareServer case as well in very much the same way (even without E-D-S offline capability - the "one has it open for editing while the other opens, edits, and stores"-case applies also in online mode if we deal with shared PIM folders). To me, this calls for a general approach of how to deal with PIM synchronization conflicts. Kind regards, Christian -- kernel concepts GmbH Tel: +49-271-771091-14 Sieghuetter Hauptweg 48 D-57072 Siegen http://www.kernelconcepts.de/ signature.asc Description: This is a digitally signed message part. ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
[Evolution-hackers] API considerations for avoiding data loss in race conditions
Hello all, I'd like to raise this issue on the list for feedback regarding this bug: https://bugzilla.gnome.org/show_bug.cgi?id=686684 First let's start with the basic problem statement: ~~ It can happen that two clients modify the same contact simultaneously, or... that one client loads a number of contacts into memory and then commits that contact to the addressbook without refreshing the contact first, this will result in lost data for the given contact. Example: o Client 'A' loads Contact 'Foo' with any of the apis for fetching contacts from the addressbook. o Client 'B' also loads Contact 'Foo' o Client 'A' calls e_book_client_modify_contact() after modifying the contact's Full Name o Client 'B' modifies contact 'Foo' after modifying one of contact 'Foo's email addresses In this case contact 'Foo' has an updated email address but an old and invalid Full Name. In order to prevent this, we propose that the addressbook should refuse to modify any contact if the passed contact's revision does not match the revision of the contact currently stored by the backend. In the above example; Client 'B' would receive a specific OUT_OF_SYNC error code in reply to calling e_book_client_modify_contact(), in which case it would have to fetch the fresh contact from the address book, modify the email address on the refreshed contact data and retry. There are a couple of API concerns to address, however: ~~ a.) Unsuspecting Applications If we just modify this behaviour then any unsuspecting application may start to mal-function, if for example, that application where to make a series of calls to e_book_client_modify_contact() in a row without refreshing the contact in between calls, that sort of code would start to fail. There are a couple of avenues we can take to address this: 1.) An ESourceExtension decides the desired behaviour of the backend, i.e. should the backend refuse modifications on out-of-sync contacts, or not. This could be disabled for a while giving time to applications to adapt to the new behaviour before enabling it, potentially forwarding the decision to those who configure EDS on the system where they intend to use the addressbook. 2.) We could make some invasive API changes, adding a parameter to e_book_client_modify_contact() in frontend & backend apis deciding what the behaviour should be for each invocation, in this way one could say "Force my data to replace a given contact in the addressbook, even if it's out-of-sync". b.) Synchronization of addressbooks I don't think this is a big problem but worth mentioning in this context: If contacts in EDS have changed since the last synchronization, and the same contacts on another device have also changed, SyncEvolution will detect that the revisions don't match with the last synchronization and will then fetch the updated contact data from EDS before attempting a merge and performing a new commit. c.) Remote backends, and collection backends In the case that a backend does not have the data readily on hand, then enforcing this policy becomes more difficult. For example, what if you currently don't have an internet connection and you want to modify a contact that is stored in some off-site database, in which case you are using some offline cache, and you can't be notified right away if your contact was actually out-of-sync or not. I think as a first step it would be alright to only really enforce the policy in the local 'file' backend, but we should have a plan at least as to how to handle this case in the future. Milan suggested that one of the options for contact modifications can be "Create a Backup Copy if Out-of-Sync Occurs", in this case a remote backend could have 3 options when synchronizing with offsite data: 1.) Drop local modifications if the remote contact has also changed. 2.) Force overwrite the remote contact with the new contact data regardless of revision mismatches. 3.) Create a copy of the contact with a newly invented UID, maintaining both local and remote changes to the same contact and leaving the final decision up to clients. These three options would potentially be an enumeration which would be the newly added argument to e_book_client_modify_contact() d.) Handling revision mismatches on contact removal Without changing current EDS APIs, we cannot enforce this really since the inputs for removing contacts is the UID only, however we could potentially enforce the same rules if we were to require e_book_client_remove_contacts() to provide contact revisions (E_CONTACT_REV) as well as the contact UIDs. I'm particularly interested in Remote/Collection backends and
