[Evolution-hackers] concurrent modifications of items in GUI and EDS database

2009-01-07 Thread Patrick Ohly
Hello!

I'm currently thinking about synchronizing data with SyncEvolution in
the background while the user is active with the Evolution UI. Some
users already do that via cron jobs, but it is known to be problematic
for several reasons, among them change tracking and concurrent editing
of items.

I already discussed this with Ross Burton and Rob Bradford. Now I would
like to check that we haven't missed anything.

The plan for change tracking is to get rid of the dependency on
e_book_get_changes(). I already stopped using e_cal_get_changes()
because it was too inflexible. Instead I'll rely on the REV resp.
LAST-MODIFIED properties: the backend must update these each time an
item is modified. This seems to be supported by most backends. Are there
backends which are known to not support this?

REV and LAST-MODIFIED are treated as opaque revision strings.
SyncEvolution doesn't interpret the content, only compares it against
the last synchronized value.

The problem with concurrent modifications is two-fold.
Stale data in UI:
  * user opens an item in Evolution
  * synchronization starts in the background
  * updates the item in EDS
  * = When the user saves his changes, will he overwrite the more
recent data in EDS? Will he be warned? With Evolution the user
is not warned and his changes overwrite the ones in EDS (tested
with contacts). Evolution should listen for change signals and
warn the user as soon as he has stale data in the edit dialog.
The user then can cancel and reopen the item to redo his
changes. This is unlikely to happen often, so more elaborate
solutions (merging changes, additional buttons to copy from EDS)
should not be necessary. Should I file a bug for this? Anyone
able and willing to work on it?

Stale data in sync:
  * when the sync starts, it builds a list of new/updated/deleted
items
  * user modifies data in EDS
  * this leads to conflicts, f.i. sync modifies item that was
modified by user

Both cases need to be handled by the program which wants to make changes
to EDS data (Evolution, sync engine). To avoid race conditions, support
by EDS would be needed which currently doesn't exist. As a workaround
the following method would reduce the time window in which conflicts can
occur:
  * get revstring before starting to make modifications (when
opening item in UI; when starting sync)
  * before modifying the item, check the revstring again
  * if the same as before, do the modification
  * if different, handle conflict 

A secure solution would have to put the revision check into EDS itself
to make the check and update atomic. The proposal is to add this check
to e_book_commit_contact() and e_cal_modify_object():
  * The caller is expected to include REV resp. LAST-MODIFIED as
read from EDS earlier.
  * The EDS backend compares against the current value before
updating the item. If there is a mismatch, the update is
rejected with a suitable error code.
  * If the values are unset, the update is always executed.

Problem 1: a client cannot tell whether the backend that it is using
implements this check. If the backend doesn't (not yet updated, not
possible), then the client should implement the workaround described
above.

Problem 2: the backend cannot tell whether the client wants to have the
check enabled or not. An older client might send values for
REV/LAST-MODIFIED and might expect to always succeed. Such a client
could be considered broken and refusing the update might be the right
solution.

Problem 3: the Exchange Connector does not update LAST-MODIFIED if the
VEVENT contains it. I consider this a bug and work around it in
SyncEvolution by filtering out the property before importing into EDS.
Adding LAST-MODIFIED unconditionally would break change tracking.

Exact handling of REV and LAST-MODIFIED in the two calls is basically
undefined right now as far as I can tell. I suggest to clarify that:
  * backends should check them as described above
  * backends must update them independently of what the client sends

To allow clients to query whether the backend supports the check I
suggest to add a new capability for e_cal_get_static_capability() and
e_book_check_static_capabilities().

Any comments?

-- 
Bye, Patrick Ohly
--  
patrick.o...@gmx.de
http://www.estamos.de/

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] A Camel API to get the filename of the cache, also a proposal to have one format to rule them all

2009-01-07 Thread Srinivasa Ragavan
The Exchange patch looks fine to me.

-Srini.
On Mon, 2009-01-05 at 12:28 +0100, Philip Van Hoof wrote:
 On Mon, 2009-01-05 at 00:42 +0530, Srinivasa Ragavan wrote:
 
 
  On Fri, 2009-01-02 at 13:25 +0100, Philip Van Hoof wrote:
   Hi there evos,
   
   For an EPlugin that I'm working on I will need a Camel API to get the
   filename of the cache.
  
  Sure and the patch seems fine to me, but the Exchange portion of the
  patch is missing. It should be similar/simple.
 
 Attached.
 
 Let me know when it's all reviewed and/if I can commit it.
 
 
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] concurrent modifications of items in GUI and EDS database

2009-01-07 Thread Stefano Canepa
Il giorno mer, 07/01/2009 alle 12.45 +0100, Patrick Ohly ha scritto:
 Hello!
 
 I'm currently thinking about synchronizing data with SyncEvolution in
 the background while the user is active with the Evolution UI. Some
 users already do that via cron jobs, but it is known to be problematic
 for several reasons, among them change tracking and concurrent editing
 of items.

I started studing a plugin to do the sync on user request. I'm going to
present my idea in another email just to have clearer threads.

Bye
sc

-- 
Stefano Canepa aka sc: s...@linux.it - http://www.stefanocanepa.it
Three great virtues of a programmer: laziness, impatience and hubris.
Le tre grandi virtù di un programmatore: pigrizia, impazienza e
arroganza. (Larry Wall)


signature.asc
Description: Questa è una parte del messaggio	firmata digitalmente
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Cannot run evolution built using jhbuild

2009-01-07 Thread Stefano Canepa
Hi all,
I would like to write an e-plugin to run syncevolution but as I'm new
to evolution development I decide, may be I'm wrong, to start build from
source using jhbuild thinking that this is the quickest way to satisfy
the requirements. I just followed 
http://projects.gnome.org/evolution/build.shtml.

The build process completed with big problems. When I'm running a
Debian unstable with gnome 2.22. When I try to run the new evolution I
get the errors pasted here: http://pastebin.com/m6d825bf5

So my question are:
1) is using jhbuild the correct way to get latest evolution and develop
an e-plugin?
2) what's going wrong the evolution built by jhbuild?

TIA
sc

-- 
Stefano Canepa aka sc: s...@linux.it - http://www.stefanocanepa.it
Three great virtues of a programmer: laziness, impatience and hubris.
Le tre grandi virtù di un programmatore: pigrizia, impazienza e
arroganza. (Larry Wall)


signature.asc
Description: Questa è una parte del messaggio	firmata digitalmente
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Funambol sync from evolution

2009-01-07 Thread Stefano Canepa
Hi all,
I'm trying to develop a plugin to run configure and run syncevolution
on user request. My idea is to have a plugin like the one that sync with
the ipod adding an option on the context menu in calendar and
addressbook. On user request the plugin will run syncevolution,
displaing an error message if there is no syncevolution on the system
and the log and all messages of syncevolution if there are logs or
messages.

I would like to try to develop it in python as I see that evolution is
now capable of loading plugins written in this language, but it should
be no problem in developing in C. Wich language do you suggest to use? 

TIA
sc

-- 
Stefano Canepa aka sc: s...@linux.it - http://www.stefanocanepa.it
Three great virtues of a programmer: laziness, impatience and hubris.
Le tre grandi virtù di un programmatore: pigrizia, impazienza e
arroganza. (Larry Wall)


signature.asc
Description: Questa è una parte del messaggio	firmata digitalmente
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Cannot run evolution built using jhbuild

2009-01-07 Thread Patrick Ohly
On Mi, 2009-01-07 at 22:20 +0100, Stefano Canepa wrote:
   I would like to write an e-plugin to run syncevolution

Excellent! Let me know when I can help with the SyncEvolution side of
things. As far as e-plugins go I don't know anything myself, so we'll
have to depend on the assistance of the Evolution hackers.

  but as I'm new
 to evolution development I decide, may be I'm wrong, to start build from
 source using jhbuild thinking that this is the quickest way to satisfy
 the requirements. I just followed 
 http://projects.gnome.org/evolution/build.shtml.
 
   The build process completed with big problems. When I'm running a
 Debian unstable with gnome 2.22. When I try to run the new evolution I
 get the errors pasted here: http://pastebin.com/m6d825bf5

I don't know jhbuild. I suspect that it builds more libraries and later
when you try to run evolution-cvs, the environment is not set up
correctly so those libraries are not found.

   So my question are:
 1) is using jhbuild the correct way to get latest evolution and develop
 an e-plugin?

It is going to be useful to build Evolution from source without
optimization and with debug information, because then you can more
easily study how the plugin mechanism works (set breakpoints in a
debugger, go up the stack, analyze variables, etc.).

I suggest that you give the following Makefile a try before spending
more time with jhbuild:
http://mad-scientist.us/evolution.html

It builds just Evolution and generates an evolution-env script that
sets up the right environment for you. The makefile depends on recent
enough version of the libraries that Evolution depends on. You shouldn't
have a problem in Debian unstable.

I also used garnome on machines where those libraries had to be compiled
from source. It's more complicated than the makefile.

-- 
Bye, Patrick Ohly
--  
patrick.o...@gmx.de
http://www.estamos.de/

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Funambol sync from evolution

2009-01-07 Thread Patrick Ohly
On Mi, 2009-01-07 at 22:27 +0100, Stefano Canepa wrote:
 On user request the plugin will run syncevolution,
 displaing an error message if there is no syncevolution on the system
 and the log and all messages of syncevolution if there are logs or
 messages.

It wouldn't be hard to create and install a libsyncevolution.so which
contains all the necessary logic to execute a sync. That might be easier
than wrapping a command line tool. On the other hand, Genesis already
does that quite successfully.

   I would like to try to develop it in python as I see that evolution is
 now capable of loading plugins written in this language, but it should
 be no problem in developing in C. Wich language do you suggest to use? 

Genesis is written in Python, so you could take that as example for
writing a syncevolution command line wrapper. I also started with a
Python binding for the relevant SyncEvolution classes that a GUI would
have to interact with. This was an experiment to learn about
Boost.Python. It showed that this is so easy that finishing it should be
a matter of a day perhaps. Want to learn how it was done and perhaps
finish it? ;-)

Browse the source:
https://zeitsenke.de/websvn/listing.php?repname=SyncEvolutionpath=%2Fbranches%2Fpython%2Frev=0sc=0
Check out:
http://zeitsenke.de/svn/SyncEvolution/branches/python/

If you prefer to work with git (easier for external contributors) then I
could also provide a git mirror of the SVN repository.

My suggestion regarding the language would be to check out how easy it
is to write Evolution plugins in C and Python. Pick the one that you
find easier. Python is a nice language. A downside might be that users
of older Evolution releases without that support won't be able to use
your plugin. When was Python support added?

-- 
Bye, Patrick Ohly
--  
patrick.o...@gmx.de
http://www.estamos.de/

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Cannot run evolution built using jhbuild

2009-01-07 Thread Tobias Mueller
Hi,

On 07.01.2009 22:20, Stefano Canepa wrote:
 2) what's going wrong the evolution built by jhbuild?
 
The build itself works properly, right? Running your newly build
Evolution fails, if I read your log correctly.

To run evolution, you should do a jhbuild run evolution. It is not
clear to me, what evolution-cvs is or does.

Also, you should do an export LANG=C to reset the language before you
post any logs, because reading messages in your language might be
difficult for somebody ;-)

What I can guess from the error messages is, that you run an old gconfd
on your host system and don't run the new, dbus-ified GConfD from jhbuild.

Try to run the gconf deamon from your jhbuild prefix. i.e.
jhbuild run /opt/gnome/libexec/gconfd-2  before you run your
evolution-trunk.


Happy Hacking,
  Muelli
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Funambol sync from evolution

2009-01-07 Thread George Farris
On Wed, 2009-01-07 at 23:20 +0100, Patrick Ohly wrote:
 My suggestion regarding the language would be to check out how easy it
 is to write Evolution plugins in C and Python. Pick the one that you
 find easier. Python is a nice language. A downside might be that users
 of older Evolution releases without that support won't be able to use
 your plugin. When was Python support added?
 

Maybe have a look at Vala 

http://live.gnome.org/Vala

Kind of best of both worlds, Python/ C

Cheers


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] concurrent modifications of items in GUI and EDS database

2009-01-07 Thread Suman Manjunath
On Wed, Jan 7, 2009 at 17:15, Patrick Ohly patrick.o...@gmx.de wrote:
 The plan for change tracking is to get rid of the dependency on
 e_book_get_changes(). I already stopped using e_cal_get_changes()
 because it was too inflexible. Instead I'll rely on the REV resp.
 LAST-MODIFIED properties: the backend must update these each time an
 item is modified. This seems to be supported by most backends. Are there
 backends which are known to not support this?

a) Looking at [1], I can't find what a REV property is. Did you mean
to use [2] ?

 The problem with concurrent modifications is two-fold.
 Stale data in UI:
  * user opens an item in Evolution
  * synchronization starts in the background
  * updates the item in EDS
  * = When the user saves his changes, will he overwrite the more
recent data in EDS? Will he be warned? With Evolution the user
is not warned and his changes overwrite the ones in EDS (tested
with contacts). Evolution should listen for change signals and
warn the user as soon as he has stale data in the edit dialog.
The user then can cancel and reopen the item to redo his
changes. This is unlikely to happen often, so more elaborate
solutions (merging changes, additional buttons to copy from EDS)
should not be necessary. Should I file a bug for this? Anyone
able and willing to work on it?

Not so for calendars. When an event is open in the Evolution UI and
the backend modifies it during a refresh/update from server, the user
_is_ warned of an update to the item. If not, the backend is probably
not doing a e_cal_backend_notify_object_modified() which it ought to.
FWIW, see the open bug at [3].

 Stale data in sync:
  * when the sync starts, it builds a list of new/updated/deleted
items
  * user modifies data in EDS
  * this leads to conflicts, f.i. sync modifies item that was
modified by user

 Both cases need to be handled by the program which wants to make changes
 to EDS data (Evolution, sync engine). To avoid race conditions, support
 by EDS would be needed which currently doesn't exist. As a workaround
 the following method would reduce the time window in which conflicts can
 occur:
  * get revstring before starting to make modifications (when
opening item in UI; when starting sync)
  * before modifying the item, check the revstring again
  * if the same as before, do the modification
  * if different, handle conflict

The GroupWise server updates the 'modified' property of the item when
it actually gets modified on the server. For newly created items, it
also adds the 'created' property at the same time.

This behavior invalidates all the 'handle-at-backend' approaches to
fix the apparent bug, like:
* Check if the item is newly-created by looking for a 'modified'
property in the cached-object. This approach does not handle modifying
an older (and already cached from server) item.
* Pushing the 'modified' property from the client to the server upon
creation/modification. This does not help, since the server would
modify the property anyway.
* A series of 'if-else' trying to handle various scenarios. It was
observed that the conditions would fail for at-least one use-case,
mainly since we would not have a persistent 'last-modified' time
unless it is obtained from the server.

These drawbacks probably apply to the Exchange backend too.

 A secure solution would have to put the revision check into EDS itself
 to make the check and update atomic. The proposal is to add this check
 to e_book_commit_contact() and e_cal_modify_object():
  * The caller is expected to include REV resp. LAST-MODIFIED as
read from EDS earlier.
  * The EDS backend compares against the current value before
updating the item. If there is a mismatch, the update is
rejected with a suitable error code.
  * If the values are unset, the update is always executed.

A backend may not have a LAST-MODIFIED property for a particular event
in this use case:
a) create a new appointment in the GW calendar (while online)
b) open the same appointment (before the refresh timeout)

Pushing an update in this case is not correct as the event can easily
be modified by another client during that refresh interval.

We should ensure that the cached object in EDS will always have the
same last-modified property as that on the server before opening the
event in the UI [This probably implies an (expensive?) 'live' re-cache
of the object being opened]. We already listen for changes to the
event and so the user automatically knows if anything changed since
opening the event editor.

We don't continuously listen for updates from the server and hence the
problem I mentioned above would still exist, but that is something I
can live with :)

- Suman

[1] http://tools.ietf.org/html/rfc2445
[2] http://tools.ietf.org/html/rfc2445#section-4.8.7.4
[3]