[Gimp-developer] Proposal: cache directory tagging convention
Dear GIMP developers, I'm soliciting feedback, discussion, and hopefully ultimately support for a very simple proposal that I've written up in preliminary form at this page: http://www.brynosaurus.com/cachedir/ The short version: I propose that applications that create and manage on-disk caches of ephemeral or otherwise regenerable content, such as the GIMP's on-disk tile cache, include within their cache directories a standardized cache directory tag file with a specific name and containing a specific header. This way, other software such as backup systems, data transfer utilities, and data management tools can easily (if configured to do so by the user) recognize such cache directories that contain non-precious data, and avoid wasting storage space and/or network bandwidth on copying around and storing them unnecessarily. Writing such a tag file should amount to a most a 5-10 line addition to caching applications such as GIMP, but could substantially benefit the efficiency of other applications. I don't yet know the best venue for discussion of this proposal (I realize Gimp-developer isn't it, since it's a highly cross-application issue that only slightly affects GIMP itself) - please E-mail me if you're interested and I'll point you to the appropriate forum once I find or create it. Or if you like the idea but just would like me to come back later when the proposal has been more widely peer reviewed by others, I'm happy with that too. :) Thanks, Bryan ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Proposal: cache directory tagging convention
On Tue, 20 Jul 2004 14:36:25 +0200, Bryan Ford [EMAIL PROTECTED] wrote: I'm soliciting feedback, discussion, and hopefully ultimately support for a very simple proposal that I've written up in preliminary form at this page: http://www.brynosaurus.com/cachedir/ [...] I don't yet know the best venue for discussion of this proposal (I realize Gimp-developer isn't it, since it's a highly cross-application issue that only slightly affects GIMP itself) - please E-mail me if you're interested and I'll point you to the appropriate forum once I find or create it. Or if you like the idea but just would like me to come back later when the proposal has been more widely peer reviewed by others, I'm happy with that too. :) The best forum to discuss this proposal is probably the xdg-list. This is where things like the thumbnail managing standard (which you mentioned in your proposal) have been discussed. For more info about this list, see: http://www.freedesktop.org/Main/GettingInvolved If your proposal is adopted as a freedesktop standard, then it is likely that it will find its way into the GIMP and many other applications. I have a few comments on your proposal. Not necessarily some things that should be changed, but maybe you could discuss the pros and cons in a rationale section or in the alternative approaches chapter: - Instead of a file containing a MD5 hash or some other signature, you could consider using an empty file or a symbolic link to .. Neither of them is likely to conflict with normal files used by some application. - If you want to have a non-empty file or symbolic link, what about using the full path to the file? If a cache directory is moved by the user (and not by the application that created it, because such an application would update the path) then maybe it should not be ignored anymore by the backup process. So comparing the path stored in the file or in the symbolic link with the current path of the file could be a good way to check if the directory is still a cache directory or if it is now a copy stored somewhere else by the user. - Some people don't like MixedCase for file names and prefer alllowercase or names_with_underscores. -Raphaël ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Proposal: cache directory tagging convention
Hi, Bryan Ford [EMAIL PROTECTED] writes: Dear GIMP developers, I'm soliciting feedback, discussion, and hopefully ultimately support for a very simple proposal that I've written up in preliminary form at this page: http://www.brynosaurus.com/cachedir/ I don't think the .thumbnails directory can be considered ephemeral or otherwise regenerable content. Regenerating all thumbnails can be very difficult and time-consuming. The original data might be on removable media and even if all the images are on disk you certainly don't want to go through the hassle of regenerating them all since you will most probably need a number of different applications to do that. I suggest that you change your proposal so that it explicitely states that the .thumbnails directory should _not_ be tagged as a cache directory. Since GIMP doesn't put swap files into a dedicated directory your proposal will also not work for GIMP swap files (which are admittedly not worth to be archived). Sven ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Proposal: cache directory tagging convention
Hi, Raphaël Quinet [EMAIL PROTECTED] writes: - Some people don't like MixedCase for file names and prefer alllowercase or names_with_underscores. Eeek, what about names-with-hyphens instead? Sven ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Proposal: cache directory tagging convention
On Tuesday 20 July 2004 15:12, Sven Neumann wrote: Bryan Ford [EMAIL PROTECTED] writes: I'm soliciting feedback, discussion, and hopefully ultimately support for a very simple proposal that I've written up in preliminary form at this page: http://www.brynosaurus.com/cachedir/ I don't think the .thumbnails directory can be considered ephemeral or otherwise regenerable content. Regenerating all thumbnails can be very difficult and time-consuming. The original data might be on removable media and even if all the images are on disk you certainly don't want to go through the hassle of regenerating them all since you will most probably need a number of different applications to do that. I suggest that you change your proposal so that it explicitely states that the .thumbnails directory should _not_ be tagged as a cache directory. I fully agree with your opinion that thumbnails shouldn't be considered particularly _ephemeral_. But then I'm not proposing that the presence of a cache directory tag should indicate that the directory be considered ephemeral, or should befair game for systemwide cron jobs or whatever to clear out on a regular basis. In fact, the full proposal on my web site already specifically disallows such an interpretation of cache directory tags. With or without a cache directory tag, it should be entirely up to the application to determine the _longevity_ of the data in its cache(s) for all normal operational purposes. The cache directory tag merely makes the statement that, in the event of an emergency (e.g., the hard drive dies and has to be restored from a backup), nothing critical is lost - the contents of the cache _can_ be regenerated automatically. While thumbnails aren't necessarily ephemeral, they certainly are automatically regenerable - it is the standard behavior for all applications I know of that use them to regenerate them automatically if they disappear for any reason. Sure, the original source material may be on removable media, so some of the thumbnails won't actually be _regenerated_ until the user inserts that particular CD-ROM and browses it again. But even if those thumbnails hadn't been lost, they wouldn't ever be used until that point anyway. Since the thumbnails are effectively useless until and unless the original content re-appears, and they can be regenerated from the original content once it does re-appear, I can't see how their storage matters beyond the performance penalty of having to re-compute the thumbnails from scratch. And that's the definition of a cache: if you lose it, you wait longer for stuff to load next time, but it's not disaster. Aside from all that, though, I'm not going to push for my particular opinion on what does or does not constitute a cache to be adopted by all applications. My purpose at the moment is merely to establish a convention by which applications can tag cache directories as such, if and when the respective developers and/or users feel appropriate. Since GIMP doesn't put swap files into a dedicated directory your proposal will also not work for GIMP swap files (which are admittedly not worth to be archived). I had thought that that was what the ~/.gimp-2.0/tmp directory was for, but on re-running the GIMP I see that I was mistaken. So you're right - in order to adopt the proposed convention for its tile cache, the GIMP would have to make an additional subdirectory and move its swap file into there. Thanks, Bryan ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Proposal: cache directory tagging convention
Hi, Quoting Sven Neumann [EMAIL PROTECTED]: Bryan Ford [EMAIL PROTECTED] writes: I'm soliciting feedback, discussion, and hopefully ultimately support for a very simple proposal that I've written up in preliminary form at this page: http://www.brynosaurus.com/cachedir/ Since GIMP doesn't put swap files into a dedicated directory your proposal will also not work for GIMP swap files (which are admittedly not worth to be archived). If there were some way to tell that the tile cache was ephemeral data, then the proposal would work, wouldn't it? Bryan, is there any demand for something like this in archiving programs? Cheers, Dave. -- Dave Neary Lyon, France ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Proposal: cache directory tagging convention
Hi, we should really move this discussion to xdg-list. Bryan Ford [EMAIL PROTECTED] writes: While thumbnails aren't necessarily ephemeral, they certainly are automatically regenerable - it is the standard behavior for all applications I know of that use them to regenerate them automatically if they disappear for any reason. Sure, the original source material may be on removable media, so some of the thumbnails won't actually be _regenerated_ until the user inserts that particular CD-ROM and browses it again. But even if those thumbnails hadn't been lost, they wouldn't ever be used until that point anyway. Since the thumbnails are effectively useless until and unless the original content re-appears, and they can be regenerated from the original content once it does re-appear, I can't see how their storage matters beyond the performance penalty of having to re-compute the thumbnails from scratch. And that's the definition of a cache: if you lose it, you wait longer for stuff to load next time, but it's not disaster. The point I was trying to make is that thumbnails are _not_ automatically regeneratable, at least not by the applications reading them. There are a couple of file formats that only certain applications understand (such as for example the GIMP XCF format). A file-manager doesn't know how to create a thumbnail for these file formats. It still can use the thumbnail that the application which created the file (GIMP in this expample) wrote to the .thumbnails directory. That's the whole point of the Thumbnail Managing Standard. If the .thumbnails directory is being deleted (or lost in a disk crash), vital information is lost which cannot easily be regenerated. I'd call that a disaster and thus vote for explicitely not tagging the .thumbnails directory as a cache directory. It simply isn't one. Since GIMP doesn't put swap files into a dedicated directory your proposal will also not work for GIMP swap files (which are admittedly not worth to be archived). I had thought that that was what the ~/.gimp-2.0/tmp directory was for, but on re-running the GIMP I see that I was mistaken. So you're right - in order to adopt the proposed convention for its tile cache, the GIMP would have to make an additional subdirectory and move its swap file into there. We could very well do that but it is nevertheless a weak spot in your proposal and I suggest that you think about ways to tag individual files. Sven ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Proposal: cache directory tagging convention
Rapha and Sven: Thanks for bringing the xdg-list to my attention; let's move further follow-up discussion there. Rapha: Thanks for your comments - I've already considered some of those issues a bit, but you're right, they need to be addressed in the proposal. I'll work on it. Sven: The point I was trying to make is that thumbnails are _not_ automatically regeneratable, at least not by the applications reading them. There are a couple of file formats that only certain applications understand (such as for example the GIMP XCF format). A file-manager doesn't know how to create a thumbnail for these file formats. It still can use the thumbnail that the application which created the file (GIMP in this expample) wrote to the .thumbnails directory. That's the whole point of the Thumbnail Managing Standard. If the .thumbnails directory is being deleted (or lost in a disk crash), vital information is lost which cannot easily be regenerated. I'd call that a disaster and thus vote for explicitely not tagging the .thumbnails directory as a cache directory. It simply isn't one. Point conceded - I hadn't considered the case of one application making use of thumbnails that only other applications can generate. Thanks. Bryan ___ Gimp-developer mailing list [EMAIL PROTECTED] http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
