Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

2015-12-01 Thread Karel Gardas
Tinker, what you basically try to describe as Fletcher is kind of how ZFS is working. The Fletcher on the other hand is simple checksumming algorithm. Please read something about ZFS design to know more about it. Now, what I did for RAID1 to become RAID1C is just to divide data area of RAID1 to dat

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

2015-12-01 Thread Tinker
Sorry for the spam - this is my last post before your next response. My best understanding is that within your RAID1C, Fletcher could work as a "CRC32 on steroids", because it would not only detect error when reading sectors/blocks that are broken because they contain inadvertently moved data,

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

2015-12-01 Thread Tinker
Wait, so you say the input for your CRC32 checksum is "metadata>". So every sector/block in your model contains a CRC32 checksum of that, and on every fread() you check that. Does the SR metadata contain the sector index number, so that if the sector index number would have changed inadverten

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

2015-12-01 Thread Tinker
Just to illustrate the case. This is just how I got that it works, please pardon the amateur level on algorithm details here. With the Fletcher checksumming, say that you have the Fletcher checksum in a tree structure of two levels: One at the disk root, one for every 100MB of data on the disk

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

2015-12-01 Thread Tinker
Hi Karel, Glad to talk to you. Why the extra IO expense? About the Fletcher vs not Fletcher thing, can you please explain to me what happens in a setup where I have one single disk with one single RAID partition on it using your disciple, and.. 1) I write a sector/block on some position X

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

2015-12-01 Thread Karel Gardas
I don't know about fletcher, but I'm working on crc32 based checksumming for soft raid1. The basic implementation is ready but I'm not satisfied with write performance in some cases: small files, lots of collisions in chksumming blocks etc. Worst case I see 6-7x slower performance here in compariso

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid? (+better phrasing)

2015-12-01 Thread Tinker
On 2015-12-02 02:27, Chris Cappuccio wrote: Tinker [ti...@openmailbox.org] wrote: Hi! I heard someone was working with implementing Fletcher checksums in softraid. Do you know any updates on this? Karel Gardas was working on an implementation of RAID1C for softraid Last I remember, it need

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid? (+better phrasing)

2015-12-01 Thread Chris Cappuccio
Tinker [ti...@openmailbox.org] wrote: > Hi! > > I heard someone was working with implementing Fletcher checksums in > softraid. > > Do you know any updates on this? > Karel Gardas was working on an implementation of RAID1C for softraid Last I remember, it needs to be pulled out into smaller pi

Any news on Fletcher checksums (=ZFS-style checksums) in softraid? (+better phrasing)

2015-12-01 Thread Tinker
Hi! I heard someone was working with implementing Fletcher checksums in softraid. Do you know any updates on this? Fletcher checksums are how OpenBSD would guarantee that the data you read from disk actually has integrity. What makes Fletcher checksums different from traditional checksumm

Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

2015-12-01 Thread Tinker
Hi! I heard someone was working with implementing Fletcher checksums in softraid. Do you know any updates on this? Fletcher checksums are how OpenBSD would guarantee that the data you read from disk actually has integrity. What makes it different from traditional checksumming is that it n