Hi Antony, As you probably know, XFS is a complicated filesystem that was born as the default filesystem on SGI IRIX. There were lots of reasons to use XFS in the IRIX world, namely guaranteed-rate I/O (for video server applications) and rich metadata support for organizing huge collections of media assets, scientific datasets, etc. There is less reason to use it on Linux today, especially if you've also got a RAID controller. If you read the wikipedia article on XFS (http://en.wikipedia.org/wiki/XFS), particularly the section entitled "Write Barriers", you'll see that XFS filesystems have a lot of complicated mount options, some which, if set improperly, can compromise filesystem integrity.
My thinking here is that since you're seeing similar problems with Banshee that the culprit is likely how you're mounting your XFS filesystem or merely using XFS itself. I don't think this is a Shotwell issue, because we don't really do anything exotic in terms of file I/O in Shotwell. You're right that other filesystems have delayed-write capabilities, but things like delayed-write and journaling should be transparent to applications like Shotwell and should ensure filesystem consistency across all standard file operations. In terms of opening files in bulk, versions of Shotwell later than 0.8.0 that support directory monitoring do a lot of background I/O. And of course, we write metadata to files if you've got that option enabled. But none of this is exotic. Like I say, I think your problem is XFS. Cheers, Lucas On Wed, May 4, 2011 at 3:22 PM, Antony Mee <[email protected]> wrote: > Adam, > > First - my hands are up: I suspect you may be right that my fs may be the > culprit (at least partially). The picture library was on an XFS drive and > it's true that delayed allocation could cause data loss. It would require > the process to crash or be interrupted before a sync has occurred though. > I'm going to do some more rigorous testing but would love your comment on my > current train of thought. > > I have had Shotwell crash on me (mostly previous releases), though I don't > recall it doing so on this occasion. Moreover, if I understand correctly, > the delayed write/allocate wouldn't actually be the problem in itself. > Rather, files have to be open for write and bsomething has to happen to > prevent the sync. Or put another way XFS may simply serve to amplify an > underlying issue? > > The sub-set of files that were corrupted corresponds, I think, to the files > that were edited in some way. Does (can) tagging alone cause the image files > to be opened for write in 0.9.2? Or are there other features that can result > in writing to image files in bulk? > > Stepping back I decided to check whether there are any other apparently > corrupt (zero length) files on my drives. note: this system has been running > with moderate with XFS on my home partition for a couple of years. There > were a few - one more small group of photos and a few seemingly related to > the only other app that I use that has a bulk editing feature: Banshee. > Again the files go in groups. Coincidence I wonder? I'm pleased I have good > backups! > > Could I be barking up the wrong tree? How many files in theory could > Shotwell have open for write at any one time? > > Delayed write is likely to be a feature of all new filesystems (ext4 and > btrfs are good examples.) Perhaps there are some implications for developers > of file processing apps? > > Antony > On 2 May 2011 21:17, "Adam Dingle" <[email protected]> wrote: > _______________________________________________ > Shotwell mailing list > [email protected] > http://lists.yorba.org/cgi-bin/mailman/listinfo/shotwell > _______________________________________________ Shotwell mailing list [email protected] http://lists.yorba.org/cgi-bin/mailman/listinfo/shotwell
