C. Scott Andreas updated CASSANDRA-9264:
    Component/s: Core

> Cassandra should not persist files without checksums
> ----------------------------------------------------
>                 Key: CASSANDRA-9264
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9264
>             Project: Cassandra
>          Issue Type: Wish
>          Components: Core
>            Reporter: Ariel Weisberg
>            Priority: Major
>             Fix For: 4.x
> Even if checksums aren't validated on the read side every time it is helpful 
> to have them persisted with checksums so that if a corrupted file is 
> encountered you can at least validate that the issue is corruption and not an 
> application level error that generated a corrupt file.
> We should standardize on conventions for how to checksum a file and which 
> checksums to use so we can ensure we get the best performance possible.
> For a small checksum I think we should use CRC32 because the hardware support 
> appears quite good.
> For cases where a 4-byte checksum is not enough I think we can look at either 
> xxhash64 or MurmurHash3.
> The problem with xxhash64 is that output is only 8-bytes. The problem with 
> MurmurHash3 is that the Java implementation is slow. If we can live with 
> 8-bytes and make it easy to switch hash implementations I think xxhash64 is a 
> good choice because we already ship a good implementation with LZ4.
> I would also like to see hashes always prefixed by a type so that we can swap 
> hashes without running into pain trying to figure out what hash 
> implementation is present. I would also like to avoid making assumptions 
> about the number of bytes in a hash field where possible keeping in mind 
> compatibility and space issues.
> Hashing after compression is also desirable over hashing before compression.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

Reply via email to