Re: [Evolution-hackers] evolution-kolab: local cache for offline-work
On Fri, 2010-07-09 at 19:20 +0200, Christian Hilberg wrote: Hi Chen, thank you very much for taking time to clarify things! Much appreciated. On Friday 09 July 2010 at 18:43:32 chen wrote: On Fri, 2010-07-09 at 18:25 +0200, Christian Hilberg wrote: * Does there exist any infrastructure in E-D-S which will help us creating a local email cache via IMAP? Yes its available. CamelFolderSummary stores all the basic contents of an email which needs to be displayed in the MessageList view (label, subject, dates, fromto headers etc.). CamelDataCache stores the contents of the email (entire mime message) into the cache. The imapx provider would use them to store the mail data into cache. Okay, this helps. I think I saw these parts of Camel before, but I was not sure whether the facility provided would be sufficient for our needs. Seems we have what we need here. * Orelse, is there a sensible way to re-use existing caching functionality inside our backend which comes from Evolution, since Email handling resides there, presently? Without weaving knots between Evo and E-D-S which will be prone to failure and unclean, I mean? Current imapx provider implementation is in such a way that, once you access a message using camel_folder_get_message, the message would be completely downloaded into the cache and would be available for offline usage. camel_folder_refresh_info would fetch the summary contents and optionally fetch the message contents also if the folder is marked for offline usage (when the camel url property, 'sync_offline' is set). Again, this is valuable information. If I read this correctly, the IMAPX implementation would handle all caching so we would be left with only the actual synchronization problem. BTW, is there a way to intercept Camel's synchronization with the IMAP server after reconnecting (i.e., after leaving offline mode)? This question is because the PIM emails hold semantics to which Camel will be agnostic (since it thinks it only handles emails), and it could be problematic to cope with PIM sync conflicts if Camel will just sync the emails right away without a possiblitiy for us to hook-in an awful lot of extra sync logic. You could connect to the signal 'changed' on the camel folder. This would give you information on new/deleted/changed messages once the folder is synchronized with the imap server. http://live.gnome.org/Evolution/Camel.Folder - checkout CamelFolderChangeInfo . Btw the contents of the emails (mime message with Xml calendar/contacts data) would be cached in camel (usually under ~/.evolution/mail/imapx/account/folders/folder/ ) while mails are fetched (through camel_folder_get_message). You could delete those contents after converting them to ICal or VCard and storing the data in the respective backends, by using camel_data_cache apis from calendar/address-book backends to remove duplicate data. - Chenthill. And i guess you would be re-using the existing imapx backend from evolution ? By all means! :-) Once you convert the xml data to Ical or VCard, you can make them available for offline usage based on ESource property, 'offline-sync'. Well well, this all sounds promising. [...] Once I'M really clear that we have to do these things fully on our own and we cannot re-use existing infrastructure, I will instantly stop harassing you with this issue and start working. :-) I just want to avoid reinventing the wheel. Oh np :) You can shoot your queries anytime.. Btw to be more clear no one has tried using camel apis from calendar+address-book backends. I think Matthew already mentioned this. But we will give it a try - not many options for us to choose from. :-) It might be a good idea to use separate instances of Camel providers for contacts and calendar data. We'll see. Thanks for all the fish :-) Christian ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] evolution-kolab: local cache for offline-work
On Thu, 2010-07-08 at 15:50 +0200, Christian Hilberg wrote: Hi all, in evolution-kolab, we need to provide full offline capabilities for working with Kolab 2 groupware data in offline mode. Evo 2.30 has an offline mode which works for IMAP email. If I understood correctly, email handling resides within Evolution presently [1], not E-D-S. For our plugin, though, we need to place email stuff in the backend (for handling contact data and calendar data, as these are stored in XML attachments to emails within Kolab 2 context). This means we will need to sync emails (with proper conflict resolution) when re-connecting to the Kolab server which hold calendar data and contact data. What's more, we have a multimaster-situation here. As Matthew already pointed out in [2], this is sort of new terrain we're stepping on. In order to get an idea about how much work it will be to implement this feature, I'd like to know whether there are ideas already concerning caching calendar data and address data this way. Alas, I have not found time yet to dig into existing backends to check whether they do caching already or whether there is any infrastructure provided with Evo/EDS 2.30 which we could use for that matter. We'll be happy to receive any pointers into existing plugins where something like this is already done or to discussions about the issue. Let me try to summarize some pointers for calendar and you could do the same for address-book as well, + Create New ECalBackend for Kolab + Use the camel apis to fetch the calendar folder + Using the CamelFolder, get its contents. You would mime data and you can use the Camel libraries again to parse these contents. + Convert the Xml calendar data into Ics format and store it into the backend So this helps for displaying the contents. Now with creating meetings/appointments. iirc you would need to send them via smtp ? + use the camel_transport apis (look through e_cal_backend_save/send api's) There are many backend implementations already available file, exchange, groupwise etc. and you can refer them.. I would recommend looking through groupwise/exchange and shoot your questions if any.. Since no one has yet tried using camel apis, we do not know if there are any issues while you might face. But we should be able to help you while you progress.. The same holds good for address-book also. Thanks, Chenthill. Thanks in advance, Christian [1] http://mail.gnome.org/archives/evolution-hackers/2010-June/msg00018.html [2] http://mail.gnome.org/archives/evolution-hackers/2010-June/msg00022.html ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] evolution-kolab: local cache for offline-work
On Thursday 08 July 2010 at 15:50:19 Christian Hilberg wrote: Hi all, in evolution-kolab, we need to provide full offline capabilities for working with Kolab 2 groupware data in offline mode. Anything? Anyone? :-) Best regards, Christian -- kernel concepts GbRTel: +49-271-771091-14 Sieghuetter Hauptweg 48Fax: +49-271-771091-19 D-57072 Siegen http://www.kernelconcepts.de/ signature.asc Description: This is a digitally signed message part. ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] evolution-kolab: local cache for offline-work
On Friday 09 July 2010 at 18:01:53 chen wrote: [...] Let me try to summarize some pointers for calendar and you could do the same for address-book as well, + Create New ECalBackend for Kolab + Use the camel apis to fetch the calendar folder + Using the CamelFolder, get its contents. You would mime data and you can use the Camel libraries again to parse these contents. + Convert the Xml calendar data into Ics format and store it into the backend Yeah, this much is clear. There will be a fully-fledged two-way lossless conversion between EDS data types and Kolab2 data types, and the conversion will most probably be done on-the-fly (if we run into performance issues here, we might change plans... :). So this helps for displaying the contents. Now with creating meetings/appointments. iirc you would need to send them via smtp ? + use the camel_transport apis (look through e_cal_backend_save/send api's) Also no worries so far. Let me be a little more specific (I should have been right away, but my mails tend to become lengthy...): * We need to create a local cache for emails which is handled by the plugin backend(s). * Does there exist any infrastructure in E-D-S which will help us creating a local email cache via IMAP? * Orelse, is there a sensible way to re-use existing caching functionality inside our backend which comes from Evolution, since Email handling resides there, presently? Without weaving knots between Evo and E-D-S which will be prone to failure and unclean, I mean? There are many backend implementations already available file, exchange, groupwise etc. and you can refer them.. I would recommend looking through groupwise/exchange and shoot your questions if any.. I'm in the process of having a closer look at them... Since no one has yet tried using camel apis, we do not know if there are any issues while you might face. But we should be able to help you while you progress.. Okay, thanks. Once I'M really clear that we have to do these things fully on our own and we cannot re-use existing infrastructure, I will instantly stop harassing you with this issue and start working. :-) I just want to avoid reinventing the wheel. Best regards, Christian -- kernel concepts GbRTel: +49-271-771091-14 Sieghuetter Hauptweg 48Fax: +49-271-771091-19 D-57072 Siegen http://www.kernelconcepts.de/ signature.asc Description: This is a digitally signed message part. ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] evolution-kolab: local cache for offline-work
On Fri, 2010-07-09 at 18:25 +0200, Christian Hilberg wrote: On Friday 09 July 2010 at 18:01:53 chen wrote: [...] Let me try to summarize some pointers for calendar and you could do the same for address-book as well, + Create New ECalBackend for Kolab + Use the camel apis to fetch the calendar folder + Using the CamelFolder, get its contents. You would mime data and you can use the Camel libraries again to parse these contents. + Convert the Xml calendar data into Ics format and store it into the backend Yeah, this much is clear. There will be a fully-fledged two-way lossless conversion between EDS data types and Kolab2 data types, and the conversion will most probably be done on-the-fly (if we run into performance issues here, we might change plans... :). So this helps for displaying the contents. Now with creating meetings/appointments. iirc you would need to send them via smtp ? + use the camel_transport apis (look through e_cal_backend_save/send api's) Also no worries so far. Let me be a little more specific (I should have been right away, but my mails tend to become lengthy...): * We need to create a local cache for emails which is handled by the plugin backend(s). * Does there exist any infrastructure in E-D-S which will help us creating a local email cache via IMAP? Yes its available. CamelFolderSummary stores all the basic contents of an email which needs to be displayed in the MessageList view (label, subject, dates, fromto headers etc.). CamelDataCache stores the contents of the email (entire mime message) into the cache. The imapx provider would use them to store the mail data into cache. * Orelse, is there a sensible way to re-use existing caching functionality inside our backend which comes from Evolution, since Email handling resides there, presently? Without weaving knots between Evo and E-D-S which will be prone to failure and unclean, I mean? Current imapx provider implementation is in such a way that, once you access a message using camel_folder_get_message, the message would be completely downloaded into the cache and would be available for offline usage. camel_folder_refresh_info would fetch the summary contents and optionally fetch the message contents also if the folder is marked for offline usage (when the camel url property, 'sync_offline' is set). And i guess you would be re-using the existing imapx backend from evolution ? Once you convert the xml data to Ical or VCard, you can make them available for offline usage based on ESource property, 'offline-sync'. There are many backend implementations already available file, exchange, groupwise etc. and you can refer them.. I would recommend looking through groupwise/exchange and shoot your questions if any.. I'm in the process of having a closer look at them... Since no one has yet tried using camel apis, we do not know if there are any issues while you might face. But we should be able to help you while you progress.. Okay, thanks. Once I'M really clear that we have to do these things fully on our own and we cannot re-use existing infrastructure, I will instantly stop harassing you with this issue and start working. :-) I just want to avoid reinventing the wheel. Oh np :) You can shoot your queries anytime.. Btw to be more clear no one has tried using camel apis from calendar+address-book backends. Thanks, Chenthill. Best regards, Christian ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] evolution-kolab: local cache for offline-work
Hi Chen, thank you very much for taking time to clarify things! Much appreciated. On Friday 09 July 2010 at 18:43:32 chen wrote: On Fri, 2010-07-09 at 18:25 +0200, Christian Hilberg wrote: * Does there exist any infrastructure in E-D-S which will help us creating a local email cache via IMAP? Yes its available. CamelFolderSummary stores all the basic contents of an email which needs to be displayed in the MessageList view (label, subject, dates, fromto headers etc.). CamelDataCache stores the contents of the email (entire mime message) into the cache. The imapx provider would use them to store the mail data into cache. Okay, this helps. I think I saw these parts of Camel before, but I was not sure whether the facility provided would be sufficient for our needs. Seems we have what we need here. * Orelse, is there a sensible way to re-use existing caching functionality inside our backend which comes from Evolution, since Email handling resides there, presently? Without weaving knots between Evo and E-D-S which will be prone to failure and unclean, I mean? Current imapx provider implementation is in such a way that, once you access a message using camel_folder_get_message, the message would be completely downloaded into the cache and would be available for offline usage. camel_folder_refresh_info would fetch the summary contents and optionally fetch the message contents also if the folder is marked for offline usage (when the camel url property, 'sync_offline' is set). Again, this is valuable information. If I read this correctly, the IMAPX implementation would handle all caching so we would be left with only the actual synchronization problem. BTW, is there a way to intercept Camel's synchronization with the IMAP server after reconnecting (i.e., after leaving offline mode)? This question is because the PIM emails hold semantics to which Camel will be agnostic (since it thinks it only handles emails), and it could be problematic to cope with PIM sync conflicts if Camel will just sync the emails right away without a possiblitiy for us to hook-in an awful lot of extra sync logic. And i guess you would be re-using the existing imapx backend from evolution ? By all means! :-) Once you convert the xml data to Ical or VCard, you can make them available for offline usage based on ESource property, 'offline-sync'. Well well, this all sounds promising. [...] Once I'M really clear that we have to do these things fully on our own and we cannot re-use existing infrastructure, I will instantly stop harassing you with this issue and start working. :-) I just want to avoid reinventing the wheel. Oh np :) You can shoot your queries anytime.. Btw to be more clear no one has tried using camel apis from calendar+address-book backends. I think Matthew already mentioned this. But we will give it a try - not many options for us to choose from. :-) It might be a good idea to use separate instances of Camel providers for contacts and calendar data. We'll see. Thanks for all the fish :-) Christian -- kernel concepts GbRTel: +49-271-771091-14 Sieghuetter Hauptweg 48Fax: +49-271-771091-19 D-57072 Siegen http://www.kernelconcepts.de/ signature.asc Description: This is a digitally signed message part. ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
[Evolution-hackers] evolution-kolab: local cache for offline-work
Hi all, in evolution-kolab, we need to provide full offline capabilities for working with Kolab 2 groupware data in offline mode. Evo 2.30 has an offline mode which works for IMAP email. If I understood correctly, email handling resides within Evolution presently [1], not E-D-S. For our plugin, though, we need to place email stuff in the backend (for handling contact data and calendar data, as these are stored in XML attachments to emails within Kolab 2 context). This means we will need to sync emails (with proper conflict resolution) when re-connecting to the Kolab server which hold calendar data and contact data. What's more, we have a multimaster-situation here. As Matthew already pointed out in [2], this is sort of new terrain we're stepping on. In order to get an idea about how much work it will be to implement this feature, I'd like to know whether there are ideas already concerning caching calendar data and address data this way. Alas, I have not found time yet to dig into existing backends to check whether they do caching already or whether there is any infrastructure provided with Evo/EDS 2.30 which we could use for that matter. We'll be happy to receive any pointers into existing plugins where something like this is already done or to discussions about the issue. Thanks in advance, Christian [1] http://mail.gnome.org/archives/evolution-hackers/2010-June/msg00018.html [2] http://mail.gnome.org/archives/evolution-hackers/2010-June/msg00022.html -- kernel concepts GbRTel: +49-271-771091-14 Sieghuetter Hauptweg 48Fax: +49-271-771091-19 D-57072 Siegen http://www.kernelconcepts.de/ signature.asc Description: This is a digitally signed message part. ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... http://mail.gnome.org/mailman/listinfo/evolution-hackers
