[Evolution-hackers] improve PHOTO support in libebook/EDS
Hello! It was pointed out before that support for PHOTO data in EDS is sub-optimal because the data has to be encoded as B64 data in the vCard and then gets transmitted like that via D-BUs and stored in Berkley DB. A high-level description of the problem is here: http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements#Contacts:_Store_PHOTO_Data_as_Plain_Files Photos should be transferred over D-Bus via a filepath to a local file rather than transferring the actual photo data over D-Bus. The VCard format already allows this option (See the PHOTO property). We must be careful to keep these local files in sync and to remove them when contacts are removed, and to restrict access to the files from other processes. There should be utility code available to libebook users to convert between the different representations. Goals: 1. Apps should be able to create photo files without ever passing the data to libebook. 2. External PHOTO file and corresponding contact must be kept in sync: 1. If a contact is deleted, the photo also needs to be removed. 2. If a photo file is created and then storing the contact fails, who removes the file? 3. Access to photo data must be limited to processes which have the right permissions. --- There also was a proposal for how to solve this, but I am no longer entirely happy with it. Openismus is working on this at the moment, see https://bugzilla.gnome.org/show_bug.cgi?id=652178 Before diving further into prototyping or implementing a solution, I would like to come to an agreement about the client visible behavior and a rough outline how to implement it. First let me clarify that we need to distinguish two different use cases: 1. The photo app owns and manages a set of photos. A link is added to a contact. If the contact is deleted, the photo must remain because it is part of a slide show. If the photo is moved or renamed, the link remains but is obviously no longer useful (dangling link as in a symbolic link in a file system). 2. Photo data is attached to a contact. EDS becomes the owner for the data and must ensure that it is stored and deleted together with the contact. Apps get access to the photo via its URI, exactly as in the first case. The extension that we are discussing addresses the second point. It must not break the first one. Tristan's plan seems to be to pass the PHOTO data inline in the vCard to the file backend and to strip it out there. Note that this is not optimal in terms of performance: if the data is already available separately (as it is in syncing, for example), then it has to be inlined and transferred via D-Bus (slow). This is acceptable because contacts are read a lot more often than written. But that doesn't stop people from running write benchmarks ;-) I'd only go down that route if it really makes the implementation a lot simpler. From the description in 652178 and private email it doesn't sound like it necessarily does. How about a different approach: * Each EBookClient can request a staging directory. If the backend supports this mechanism, it is created for this client by EDS. If not, the client has to inline PHOTO data as before. * The content of the staging directory is ephemeral. When a client crashes or disconnects, EDS removes the directory and everything inside it. When EDS crashes, it cleans up all staging directories when it restarts. * If a client wants EDS to take ownership of the photo, it creates a file in the staging directory with an arbitrary name and sets the PHOTO URI such that it points to it with an absolute path. * The client submits the contact. At that point it stops being responsible for the file. * The EDS backend checks for such special PHOTO URIs. If it finds one, it copies or moves the file to its permanent location and updates the URI before storing the contact and notifying clients. Note that backends are free to do that however they want. For example, a CardDAV backend might upload the file data to a resource on the HTTP server and set the PHOTO URI to a http:// URL. The file backend chooses a staging directory in the same file system as the permanent photo directory and thus can simply move the file instead of having to copy it. libebook might offer some additional utility code to simplify the handling of such a staging directory and PHOTO URIs. API to be decided, also for the request a staging directory part. -- Bye, Patrick Ohly -- patrick.o...@gmx.de http://www.estamos.de/ ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your
Re: [Evolution-hackers] improve PHOTO support in libebook/EDS
On Wed, 2011-07-06 at 18:15 +0200, Patrick Ohly wrote: Hello! It was pointed out before that support for PHOTO data in EDS is sub-optimal because the data has to be encoded as B64 data in the vCard and then gets transmitted like that via D-BUs and stored in Berkley DB. Hi, Unfortunately I started this off list so I'm just going to append my original off-list proposal at the end of this mail (it may help for people to understand the context better). A high-level description of the problem is here: http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements#Contacts:_Store_PHOTO_Data_as_Plain_Files Photos should be transferred over D-Bus via a filepath to a local file rather than transferring the actual photo data over D-Bus. The VCard format already allows this option (See the PHOTO property). We must be careful to keep these local files in sync and to remove them when contacts are removed, and to restrict access to the files from other processes. There should be utility code available to libebook users to convert between the different representations. Goals: 1. Apps should be able to create photo files without ever passing the data to libebook. 2. External PHOTO file and corresponding contact must be kept in sync: 1. If a contact is deleted, the photo also needs to be removed. 2. If a photo file is created and then storing the contact fails, who removes the file? 3. Access to photo data must be limited to processes which have the right permissions. --- There also was a proposal for how to solve this, but I am no longer entirely happy with it. Openismus is working on this at the moment, see https://bugzilla.gnome.org/show_bug.cgi?id=652178 Before diving further into prototyping or implementing a solution, I would like to come to an agreement about the client visible behavior and a rough outline how to implement it. First let me clarify that we need to distinguish two different use cases: 1. The photo app owns and manages a set of photos. A link is added to a contact. If the contact is deleted, the photo must remain because it is part of a slide show. If the photo is moved or renamed, the link remains but is obviously no longer useful (dangling link as in a symbolic link in a file system). 2. Photo data is attached to a contact. EDS becomes the owner for the data and must ensure that it is stored and deleted together with the contact. Apps get access to the photo via its URI, exactly as in the first case. The extension that we are discussing addresses the second point. It must not break the first one. Tristan's plan seems to be to pass the PHOTO data inline in the vCard to the file backend and to strip it out there. Note that this is not optimal in terms of performance: if the data is already available separately (as it is in syncing, for example), then it has to be inlined and transferred via D-Bus (slow). This is acceptable because contacts are read a lot more often than written. But that doesn't stop people from running write benchmarks ;-) I'd only go down that route if it really makes the implementation a lot simpler. From the description in 652178 and private email it doesn't sound like it necessarily does. How about a different approach: * Each EBookClient can request a staging directory. If the backend supports this mechanism, it is created for this client by EDS. If not, the client has to inline PHOTO data as before. * The content of the staging directory is ephemeral. When a client crashes or disconnects, EDS removes the directory and everything inside it. When EDS crashes, it cleans up all staging directories when it restarts. * If a client wants EDS to take ownership of the photo, it creates a file in the staging directory with an arbitrary name and sets the PHOTO URI such that it points to it with an absolute path. * The client submits the contact. At that point it stops being responsible for the file. * The EDS backend checks for such special PHOTO URIs. If it finds one, it copies or moves the file to its permanent location and updates the URI before storing the contact and notifying clients. Note that backends are free to do that however they want. For example, a CardDAV backend might upload the file data to a resource on the HTTP server and set the PHOTO URI to a http:// URL. The file backend chooses a staging directory in the same file system as the permanent photo directory and thus can simply move the file instead of having to copy it. libebook might offer some additional
