Perkeep doesn't have any erasure-coding (Tahoe-LAFS does, see this rant by the project lead: https://tahoe-lafs.org/pipermail/tahoe-dev/2012-March/007185.html).
I think the fault-tolerance story is to keep multiple full replicas and rely on the intrinsic error detection. It would be interesting to make replacing corrupt blobs from another replica automatic, I don't think that's implemented but I may be wrong. On Sat, May 25, 2019 at 2:11 AM Dan Cutting <[email protected]> wrote: > > Reading some more, I discovered the old Camlistore group and found this > thread about integrity checking: > > https://groups.google.com/forum/#!searchin/camlistore/graph|sort:date/camlistore/_KiSv7x1Eh0/w_lik623AwAJ > > So it's good to know that reindexing will verify blobs have not been > corrupted. But if this is done infrequently it still seems possible to lose > data over time due to corruption. > > Some sort of auto-healing mechanism seems like a good feature for storage > that is intended to last 100 years. Is anybody aware of any work that's been > done on this for Perkeep? > > Thanks! > > > > On Saturday, May 18, 2019 at 2:53:19 PM UTC+1, Dan Cutting wrote: >> >> Hi, does Perkeep offer any sort of error detection or correction? >> >> Data sitting on a disk will "rot" over time (bad disks, bad drivers, cosmic >> rays), and it would be good to automatically correct it. >> >> I'm thinking something along the lines of a Reed-Solomon-style blob server. >> >> A simpler possibility might be to rely on the fact that a blob is >> content-addressed and check that the data read hashes back to the content >> address before returning it to the client. Combining this with multiple >> replicas would let the blob server replace bad blobs with good ones when >> detected. >> >> Or does Perkeep already have a story around this kind of thing? I haven't >> been able to find it. >> >> Thanks! >> Dan > > -- > You received this message because you are subscribed to the Google Groups > "Perkeep" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/perkeep/0ab8636f-5e20-454f-86de-160184ba7b14%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- best, Eric eric.pdxhub.org -- You received this message because you are subscribed to the Google Groups "Perkeep" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/CAGTxtWgm9M5VWnjdW7HG29j2jLvgqjWk14X1ESyJfrUoyQkbnw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
