Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state
On Wed, 2012-04-04 at 21:25 +0100, Philip Withnall wrote: > Nitpicky, but what happens if a backend has to deal with multiple hosts? > The only example I can think of at the moment, and it's a stretch, is > the Google Contacts backend. It connects to one host for authentication, > and a different one for Contacts operations. Most of the time, GOA will > take care of authentication, so this really is a tiny corner case, but I > guess it's worth considering. > > I suppose we would just use the Contacts operations host for the > purposes of "socket-connectable", and treat failure to connect to the > authentication host as a transient authentication error. Agreed. More broadly I'd say "socket-connectable" should point to where the data lives. If that's not reachable then there's really no point in authenticating, whether _that_ host is reachable or not. > I might be tempted to give the user feedback about why a backend is > _not_ writeable (somehow, perhaps a "writable-reason" enumerated > property). This could be useful when setting up a backend: the backend > might report itself as non-writeable, but the user would not know > whether this is because they've made a typo in the backend's URI, or > because they've used a read-only URI instead of a writeable one. Or > something like that. That's worth considering, though it might already be partially covered just by backends returning error messages on failed operations as per normal. That includes open(). > Overall, this set of properties seems to simplify things nicely though. > It also fits in well with the offline buffering stuff Milan and I have > been discussing (with a few other people CCed). Good! That at least validates I'm not _completely_ off the mark. :) Matt ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state
On Wed, 2012-04-04 at 13:11 -0400, Matthew Barnes wrote: > On Tue, 2012-04-03 at 19:10 +0200, Christian Hilberg wrote: > > How about the "service-available" to be set much like the to-be > > "network-available", through GNetworkMonitor, as an EBackend property, > > which, when changed, emits a signal? > > > > Just rough thinking, nothing elaborate as yet - I'll be meditating > > this. :) > > We discussed this briefly on IRC, but just to follow up more formally. > > Having stewed on this overnight I think we're coming at the problem > wrong. The question boils down to "can the backend operate on its data > set or not?" That status is as much as we need to expose to clients, I > think. Network availability and remote server availability factor into > the answer but clients need not care. If a backend is offline-capable, > then the answer -- as far as the client is concerned -- is always "yes". > > Here's the set of properties I propose for EBackend to replace the > current overly simplistic "online" flag: > > > "host-reachable" > >type: boolean perm: read-only default: false > >EBackend itself updates this as a convenience for subclasses, but >this status need not be exposed to client applications. > >For network-based backends, this property is the result of running >g_network_monitor_can_reach() on startup and in response to changing >network conditions or when the "socket-connectable" property changes. > >For local-file backends, the host is assumed to be "localhost" which >is always reachable. So the property will always be TRUE for them. > > > "socket-connectable" > >type: GSocketConnectable perm: read-write default: (see below) > >This is the GSocketConnectable fed to g_network_monitor_can_reach(). > >EBackend itself will initialize this to a GNetworkAddress based on >the host and port settings in the ESource. Subclasses do have the >option of overriding this, however, which is why it's read-write. > >If the pointer is NULL, this is assumed to mean "localhost". > >Setting this property will trigger a "host-reachable" notification >after EBackend runs g_network_monitor_can_reach() on the new value. > >This property could prove to have additional uses in the future as >we further embrace GIO's networking APIs. Nitpicky, but what happens if a backend has to deal with multiple hosts? The only example I can think of at the moment, and it's a stretch, is the Google Contacts backend. It connects to one host for authentication, and a different one for Contacts operations. Most of the time, GOA will take care of authentication, so this really is a tiny corner case, but I guess it's worth considering. I suppose we would just use the Contacts operations host for the purposes of "socket-connectable", and treat failure to connect to the authentication host as a transient authentication error. > "readable" > >type: boolean perm: read-write default: false > >This property is exposed to clients. It indicates the backend's data >is viewable but not necessarily complete, as in the case of a network >outage and not having fully synchronized for offline usage. > >Backends are responsible for updating this themselves. > >Clients are responsible for disabling the relevant UI elements when >this property is FALSE. > > > "writable" > >type: boolean perm: read-write default: false > >This property is exposed to clients. It indicates the backend's data >can be modified, but possibly only locally. Reasons it may be FALSE >include the remote host not being reachable, the service running on >the remote host not being available, or the service forbidding write >access to the data (such as for "On The Web" calendars). > >Backends are responsible for updating this themselves. > >Clients are responsible for disabling the relevant UI elements when >this property is FALSE. I might be tempted to give the user feedback about why a backend is _not_ writeable (somehow, perhaps a "writable-reason" enumerated property). This could be useful when setting up a backend: the backend might report itself as non-writeable, but the user would not know whether this is because they've made a typo in the backend's URI, or because they've used a read-only URI instead of a writeable one. Or something like that. Overall, this set of properties seems to simplify things nicely though. It also fits in well with the offline buffering stuff Milan and I have been discussing (with a few other people CCed). Philip > Under this scheme, client applications don't need to know about network > or service availability -- just whether the backend can currently handle > a particular user action. I think this simplifies things greatly. > > Matt > > ___ > evolution-hackers mailing list > evolution-hackers@gnome.org > To change your list option
Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state
On Tue, 2012-04-03 at 19:10 +0200, Christian Hilberg wrote: > How about the "service-available" to be set much like the to-be > "network-available", through GNetworkMonitor, as an EBackend property, > which, when changed, emits a signal? > > Just rough thinking, nothing elaborate as yet - I'll be meditating > this. :) We discussed this briefly on IRC, but just to follow up more formally. Having stewed on this overnight I think we're coming at the problem wrong. The question boils down to "can the backend operate on its data set or not?" That status is as much as we need to expose to clients, I think. Network availability and remote server availability factor into the answer but clients need not care. If a backend is offline-capable, then the answer -- as far as the client is concerned -- is always "yes". Here's the set of properties I propose for EBackend to replace the current overly simplistic "online" flag: "host-reachable" type: boolean perm: read-only default: false EBackend itself updates this as a convenience for subclasses, but this status need not be exposed to client applications. For network-based backends, this property is the result of running g_network_monitor_can_reach() on startup and in response to changing network conditions or when the "socket-connectable" property changes. For local-file backends, the host is assumed to be "localhost" which is always reachable. So the property will always be TRUE for them. "socket-connectable" type: GSocketConnectable perm: read-write default: (see below) This is the GSocketConnectable fed to g_network_monitor_can_reach(). EBackend itself will initialize this to a GNetworkAddress based on the host and port settings in the ESource. Subclasses do have the option of overriding this, however, which is why it's read-write. If the pointer is NULL, this is assumed to mean "localhost". Setting this property will trigger a "host-reachable" notification after EBackend runs g_network_monitor_can_reach() on the new value. This property could prove to have additional uses in the future as we further embrace GIO's networking APIs. "readable" type: boolean perm: read-write default: false This property is exposed to clients. It indicates the backend's data is viewable but not necessarily complete, as in the case of a network outage and not having fully synchronized for offline usage. Backends are responsible for updating this themselves. Clients are responsible for disabling the relevant UI elements when this property is FALSE. "writable" type: boolean perm: read-write default: false This property is exposed to clients. It indicates the backend's data can be modified, but possibly only locally. Reasons it may be FALSE include the remote host not being reachable, the service running on the remote host not being available, or the service forbidding write access to the data (such as for "On The Web" calendars). Backends are responsible for updating this themselves. Clients are responsible for disabling the relevant UI elements when this property is FALSE. Under this scheme, client applications don't need to know about network or service availability -- just whether the backend can currently handle a particular user action. I think this simplifies things greatly. Matt ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state
> On Wed, 2012-04-04 at 13:32 +0200, Christian Hilberg wrote: > First of all, no, the things discussed here are not going to be > easy, and it raises the question what Evolution actually wants > to be. Does it want to be a fully offline-capable PIM/groupware > client? That means, does it want to support backends which are or > strive to be? Hi, there seems to be a minimum work to be done on evolution's side, the most work will be done on eds part, in respective backends. From what I understand you want the "Synchronize" button for evolution. That should be pretty easy, add clients there based on some capability, with similar functionality like the Send/Receive button in mailer. > Which is the long-term vision for Evolution in this regard? We (I'm sorry for the plural) definitely want to be the best, of course :) > I guess finding a good balance here is difficult. Supplying the user with > a dedicated sync button is a good thing, and should be done, but you're right, > bugreports will come from users who forgot to press the button. A refresh > timeout with a sensible preset seems to be a good thing, too. Maybe it would > be worth it that the refresh can simply be disabled without changing the > preset, > that would help users who go on travel and who know that they'll be on shaky > lines for the next week (is it that way already? /me needs to check). calendar/book can provide a Refresh interval option, most remote do that. That's what you call "preset", I suppose. You cannot force "no checking when I'm on this network" currently. > As for evolution-kolab, sadly, there is no good way to do a "quick check" for > changes, at least I do not have an idea how one could implement one, since the > server does not do any kind of bookkeeping for the clients. One can check IMAP > UIDs, any change on them in a PIM folder indicates that a sync is needed, but > you do have to search for the changed objects then. It could slow down the > open() operation for a cal or book dramatically if the sync was done there, > that's why I did not put a sync point there for evo-kolab (though it could > easily be done, but it may create the impression that the cal/book itself > hangs... and is opening the cal/book cancellable? I don't think so). Yes, the whole opening phase is cancellable, and is fully asynchronous (signal-driven), thus the client says "open me", and it receives confirmation "your request begun as operation X", then it is waiting for a confirmation for "I'm (not) opened now", which can come anytime. Authenticating against the server (asking for a password) is part of the opening phase. This is there since EClient, thus not for a long time. You've right having it part of the open itself is too much, that's why current builtin backends initiate a "delta thread", which takes care of regular updates (based on the 'refresh' interval) and also possible upload of changes. Because it runs in its own thread, then there is no slowdown during the opening phase. Of course, current builtin backends do not use real offline handling, as far as I know. That's the subject to change. Bye, Milan ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state
On Wed, 2012-04-04 at 13:32 +0200, Christian Hilberg wrote: > Hi Milan, > > thanks a lot for joining us and for writing the nice summary! > This is much appreciated. If the mail thread becomes too long > and overly complicated, it may make sense to drop the findings > into a wiki page and work it out from there. > > First of all, no, the things discussed here are not going to be > easy, and it raises the question what Evolution actually wants > to be. Does it want to be a fully offline-capable PIM/groupware > client? That means, does it want to support backends which are or > strive to be? > > Which is the long-term vision for Evolution in this regard? > > > Am Mittwoch 04 April 2012, um 12:24:58 schrieb Milan Crha: > > On Tue, 2012-04-03 at 13:33 -0400, Matthew Barnes wrote: > > > On Tue, 2012-04-03 at 19:10 +0200, Christian Hilberg wrote: > > > > Just rough thinking, nothing elaborate as yet - I'll be meditating > > > > this. :) > > > > > > Rough thinking here too. I'll let it simmer. > > > > Hi, > > this thread is getting quite complicated, and I confess I'm rather lost > > here (the final outcome should be clear, right). But to summarize which > > things are discussed here a bit (or better those I understood): > > > > a) Add an explicit method to synchronize local changes into the server > > b) Add some mechanism to ask user for conflict resolution during a) > > c) Tell backend to work in "offline mode" - do no network operations > > d) Notify client about current "offline mode" being used by the backend > > That pretty much sums it up. > > > -- > > > > ad a) There is agreed about the method addition, and I agree too. Maybe > > a different method prototype would be used (more parameters, see below). > > I suppose, you still tries to write the changes to the server as soon as > > possible, when in online mode, right? It makes sense, I'm only checking. > > Trying to bulk-sync as soon as network comes back online may interfere > with the user's planned workflow (just reading latest mails on a shaky > line), so I would suggest to either leave that to the user (by pressing > the sync button), or to provide a config option. The latter could also > be done on a per-backend basis. > > > -- > > > > ad b) This is quite complicated, the backend cannot rely on gtk+, > > because it would bring the dependency on the factory and the factory may > > not depend on the gtk+, it should be runnable without live desktop, only > > from a terminal. Correct me if I'm wrong. The idea of "another process > > taking care of the user interaction" is, apart of quite complicated, > > also not easy to do, what if you run the server without live desktop, or > > if you run on thin clients, or ... I'm afraid there can be many ways how > > to break this approach. Thus, what about adding a DBus signal on the > > backend for conflict resolution, something like: > >void resolve_sync_conflict ( > > guint sync_op_id, > > const gchar *server_object, > > const gchar *local_object); > > which backend will throw and the client side should response through > > something like this method: > >void sync_conflict_resolved ( > > guint sync_op_id, > > ESyncConflictResolution resolution); > > where ESyncConflictResolution will contain values like: > > Unknown > > ServerWins > > LocalWins > > ... (maybe more, Christian may advise better) > > Of course, the client part should implement this, which is basically > > undoable for all of them (and some even do not use gtk or any user > > interaction at all), thus I would add one parameter to the "synchronize" > > method, the ESyncConflictResolution value, which will pick the desired > > strategy. If it is "Unknown", then the backend can use the signal and > > wait for the method to resolve conflicts (better name from "Unknown" > > would be "Ask"). Of course, clients without user interface will not call > > the "synchronize" method, most likely. > > > > The resolve_sync_conflict() uses strings for objects, and based on the > > EClient type it's either ECalComponent or EVCard as string. > > You are right, it is too easy to forget that E-D-S better not depend > on UI. As for evolution-kolab, if there is no client connected, then > no synchronize() action would be triggered, hence no sync conflict would > occur. Only if there is a client actually requesting objects or a server > synchronization, then the backend would become active and actually *do* > something. Otherwise it would be sitting idle and not be trying to keep > up with server changes (object changed on server -- backend pulls -- > object is changed back on server -- backend pulls --- ... I think a lazy > approach would be the better one here). > > >
Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state
Hi Milan, thanks a lot for joining us and for writing the nice summary! This is much appreciated. If the mail thread becomes too long and overly complicated, it may make sense to drop the findings into a wiki page and work it out from there. First of all, no, the things discussed here are not going to be easy, and it raises the question what Evolution actually wants to be. Does it want to be a fully offline-capable PIM/groupware client? That means, does it want to support backends which are or strive to be? Which is the long-term vision for Evolution in this regard? Am Mittwoch 04 April 2012, um 12:24:58 schrieb Milan Crha: > On Tue, 2012-04-03 at 13:33 -0400, Matthew Barnes wrote: > > On Tue, 2012-04-03 at 19:10 +0200, Christian Hilberg wrote: > > > Just rough thinking, nothing elaborate as yet - I'll be meditating > > > this. :) > > > > Rough thinking here too. I'll let it simmer. > > Hi, > this thread is getting quite complicated, and I confess I'm rather lost > here (the final outcome should be clear, right). But to summarize which > things are discussed here a bit (or better those I understood): > > a) Add an explicit method to synchronize local changes into the server > b) Add some mechanism to ask user for conflict resolution during a) > c) Tell backend to work in "offline mode" - do no network operations > d) Notify client about current "offline mode" being used by the backend That pretty much sums it up. > -- > > ad a) There is agreed about the method addition, and I agree too. Maybe > a different method prototype would be used (more parameters, see below). > I suppose, you still tries to write the changes to the server as soon as > possible, when in online mode, right? It makes sense, I'm only checking. Trying to bulk-sync as soon as network comes back online may interfere with the user's planned workflow (just reading latest mails on a shaky line), so I would suggest to either leave that to the user (by pressing the sync button), or to provide a config option. The latter could also be done on a per-backend basis. > -- > > ad b) This is quite complicated, the backend cannot rely on gtk+, > because it would bring the dependency on the factory and the factory may > not depend on the gtk+, it should be runnable without live desktop, only > from a terminal. Correct me if I'm wrong. The idea of "another process > taking care of the user interaction" is, apart of quite complicated, > also not easy to do, what if you run the server without live desktop, or > if you run on thin clients, or ... I'm afraid there can be many ways how > to break this approach. Thus, what about adding a DBus signal on the > backend for conflict resolution, something like: >void resolve_sync_conflict ( > guint sync_op_id, > const gchar *server_object, > const gchar *local_object); > which backend will throw and the client side should response through > something like this method: >void sync_conflict_resolved ( > guint sync_op_id, > ESyncConflictResolution resolution); > where ESyncConflictResolution will contain values like: > Unknown > ServerWins > LocalWins > ... (maybe more, Christian may advise better) > Of course, the client part should implement this, which is basically > undoable for all of them (and some even do not use gtk or any user > interaction at all), thus I would add one parameter to the "synchronize" > method, the ESyncConflictResolution value, which will pick the desired > strategy. If it is "Unknown", then the backend can use the signal and > wait for the method to resolve conflicts (better name from "Unknown" > would be "Ask"). Of course, clients without user interface will not call > the "synchronize" method, most likely. > > The resolve_sync_conflict() uses strings for objects, and based on the > EClient type it's either ECalComponent or EVCard as string. You are right, it is too easy to forget that E-D-S better not depend on UI. As for evolution-kolab, if there is no client connected, then no synchronize() action would be triggered, hence no sync conflict would occur. Only if there is a client actually requesting objects or a server synchronization, then the backend would become active and actually *do* something. Otherwise it would be sitting idle and not be trying to keep up with server changes (object changed on server -- backend pulls -- object is changed back on server -- backend pulls --- ... I think a lazy approach would be the better one here). > -- > > ad c) The mailer part does this in global, not per-account bases. Maybe > the client may have this done in the same way. Nonetheless, in a > multi-client environment you may make sure that the clients will not
Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state
On Tue, 2012-04-03 at 13:33 -0400, Matthew Barnes wrote: > On Tue, 2012-04-03 at 19:10 +0200, Christian Hilberg wrote: > > Just rough thinking, nothing elaborate as yet - I'll be meditating > > this. :) > > Rough thinking here too. I'll let it simmer. Hi, this thread is getting quite complicated, and I confess I'm rather lost here (the final outcome should be clear, right). But to summarize which things are discussed here a bit (or better those I understood): a) Add an explicit method to synchronize local changes into the server b) Add some mechanism to ask user for conflict resolution during a) c) Tell backend to work in "offline mode" - do no network operations d) Notify client about current "offline mode" being used by the backend -- ad a) There is agreed about the method addition, and I agree too. Maybe a different method prototype would be used (more parameters, see below). I suppose, you still tries to write the changes to the server as soon as possible, when in online mode, right? It makes sense, I'm only checking. -- ad b) This is quite complicated, the backend cannot rely on gtk+, because it would bring the dependency on the factory and the factory may not depend on the gtk+, it should be runnable without live desktop, only from a terminal. Correct me if I'm wrong. The idea of "another process taking care of the user interaction" is, apart of quite complicated, also not easy to do, what if you run the server without live desktop, or if you run on thin clients, or ... I'm afraid there can be many ways how to break this approach. Thus, what about adding a DBus signal on the backend for conflict resolution, something like: void resolve_sync_conflict ( guint sync_op_id, const gchar *server_object, const gchar *local_object); which backend will throw and the client side should response through something like this method: void sync_conflict_resolved ( guint sync_op_id, ESyncConflictResolution resolution); where ESyncConflictResolution will contain values like: Unknown ServerWins LocalWins ... (maybe more, Christian may advise better) Of course, the client part should implement this, which is basically undoable for all of them (and some even do not use gtk or any user interaction at all), thus I would add one parameter to the "synchronize" method, the ESyncConflictResolution value, which will pick the desired strategy. If it is "Unknown", then the backend can use the signal and wait for the method to resolve conflicts (better name from "Unknown" would be "Ask"). Of course, clients without user interface will not call the "synchronize" method, most likely. The resolve_sync_conflict() uses strings for objects, and based on the EClient type it's either ECalComponent or EVCard as string. -- ad c) The mailer part does this in global, not per-account bases. Maybe the client may have this done in the same way. Nonetheless, in a multi-client environment you may make sure that the clients will not fight on this (when one requests backend to stay offline while the other will require it to run in online). Note the factory uses the same backend for each client it connects to it, that's why they can fight. The current "online" property works fine for me for these purposes, but if you really want, then rename it. Still, this should be, from my point of view, just a preference given by the client (user) whether the backend should or should not do any network operations. You've right that there should be distinguishable whether the offline state was initiated by a user, or by connection issues (wifi disconnected, vpn dropped and so on), thus the backend can decide whether the connection it has currently opened with the server is still alive and it can close it properly - otherwise it would just timeout, and these timeouts aren't short. New network-related property comes in mind. -- ad d) For cases like "network unavailable" or "the network available, but the destination server unreachable" the backend can notify clients about its online mode. (There is a "special" case when you connect to the server for the first time, thus the backend may report an error, rather than running in offline mode against the server address which is not reachable, because of typo in its URL.) There is currently a mechanism to notify about onlinity by the backend towards its clients, the only thing is that evolution itself doesn't indicate it in any way to the user in its UI. For example CalDAV calendars can run online, and after it realizes the server is offline, they switch itself to offline mode (it doesn't have any offli
