Thanks for explaining. See this http://www.theregister.co.uk/2009/11/02/zfs_gets_dedupe/
Seems like , ZFS is using sha256 for generating checksums. As you know, it is impossile to find two different set of bytes with same Sha-256 hash value. So ZFS is supposed to be free from deduplication related problems caused by collisions. Also byte by byte comparison seems to be possible in ZFS , if it is said to do so. >>You can tell ZFS to do full byte comparisons rather than relying on the hash if you want full security against hash duplicates: >> zfs set dedup=verify silo >> You can go the other way and use a simpler hashing algorithm to reduce processing overhead and combine it with the verify function to increase overall dedupe speed: >> zfs set dedup=fletcher4,verify silo ( Extract from the above pasted link. ) another related link : https://blogs.oracle.com/pomah/entry/opensolaris_build_128_now_availble On 13 March 2012 10:07, Siju George <[email protected]> wrote: > On Mon, Mar 12, 2012 at 5:23 PM, Basil Kurian <[email protected]> > wrote: > > I don't know much about the inside stuffs of ZFS. > > > > Ok usually it is done by CRC. > Some times CRC is same for two blocks but they have differrent Data > and that is called CRC collission. > This is how hammer resolves it. > > The hash function used for comparing data blocks is CRC-32 > (CRCs > are computed anyways as part of HAMMER data integrity features, > so there's no additional overhead). Since CRC is a weak hash > function a byte-by-byte comparison is done before actual dedup- > ing. In case of a CRC collision (two data blocks have the same > CRC but different contents) the checksum is upgraded to > SHA-256. > > When you dedup in Hammer it gives you the Data about CRC collissions > > # hammer dedup /home > Dedup /home: objspace 8000000000000000:0000 7fffffffffffffff:ffff pfs_id 4 > Dedup /home succeeded > Dedup ratio = 1.22 > 462 GB referenced > 378 GB allocated > 14 MB skipped > 6869 CRC collisions > 0 SHA collisions > 0 bigblock underflows > > Thanks > > Siju > > _______________________________________________ > Indian Libre User Group Cochin Mailing List > http://www.ilug-cochin.org/mailing-list/ > http://mail.ilug-cochin.org/mailman/listinfo/mailinglist_ilug-cochin.org > #[email protected] > -- Regards Basil
_______________________________________________ Indian Libre User Group Cochin Mailing List http://www.ilug-cochin.org/mailing-list/ http://mail.ilug-cochin.org/mailman/listinfo/mailinglist_ilug-cochin.org #[email protected]
