Thanks for taking the time to flesh these points out. Comments below: ...
> The compression I see varies from something like 30% > to 50%, very > roughly (files reduced *by* 30%, not files reduced > *to* 30%). This is > with the Nikon D200, compressed NEF option. On some > of the lower-level > bodies, I believe the compression can't be turned > off. Smaller files > will of course get hit less often -- or it'll take > longer to accumulate > the terrabyte, is how I'd prefer to think of it. Either viewpoint works. And since the compression is not that great, you still wind up consuming a lot of space. Effectively, you're trading (at least if compression is an option rather than something that you're stuck with) the possibility that a picture will become completely useless should a bit get flipped for a storage space reduction of 30% - 50% - and that's a good trade, since it effectively allows you to maintain a complete backup copy on disk (for archiving, preferably off line) almost for free compared with the uncompressed option. > > Damage that's fixable is still damage; I think of > this in archivist > mindset, with the disadvantage of not having an > external budget to be my > own archivist. There will *always* be the potential for damage, so the key is to make sure that any damage is easily fixable. The best way to do this is to a) keep multiple copies, b) keep them isolated from each other (that's why RAID is not a suitable approach to archiving), and c) check (scrub) them periodically to ensure that if you lose a piece (whether a bit or a sector) you can restore the affected data from another copy and thus return your redundancy to full strength. For serious archiving, you probably want to maintain at least 3 such copies (possibly more if some are on media of questionable longevity). For normal use, there's probably negligible risk of losing any data if you maintain only two on reasonably reliable media: 'MAID' experience suggests that scrubbing as little as every few months reduces the likelihood of encountering detectable errors while restoring redundancy by several orders of magnitude (i.e., down to something like once in a PB at worst for disks - becoming comparable to the levels of bit-flip errors that the disk fails to detect at all). Which is what I've been getting at w.r.t. ZFS in this particular application (leaving aside whether it can reasonably be termed a 'consumer' application - because bulk video storage is becoming one and it not only uses a similar amount of storage space but should probably be protected using similar strategies): unless you're seriously worried about errors in the once-per-PB range, ZFS primarily just gives you automated (rather than manually-scheduled) scrubbing (and only for your on-line copy). Yes, it will help detect hardware faults as well if they happen to occur between RAM and the disk (and aren't otherwise detected - I'd still like to know whether the 'bad cable' experiences reported here occurred before ATA started CRCing its transfers), but while there's anecdotal evidence of such problems presented here it doesn't seem to be corroborated by the few actual studies that I'm familiar with, so that risk is difficult to quantify. Getting back to 'consumer' use for a moment, though, given that something like 90% of consumers entrust their PC data to the tender mercies of Windows, and a large percentage of those neither back up their data, nor use RAID to guard against media failures, nor protect it effectively from the perils of Internet infection, it would seem difficult to assert that whatever additional protection ZFS may provide would make any noticeable difference in the consumer space - and that was the kind of reasoning behind my comment that began this sub-discussion. By George, we've managed to get around to having a substantive discussion after all: thanks for persisting until that occurred. - bill This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
