[ https://issues.apache.org/jira/browse/CASSANDRA-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541828#comment-14541828 ]
Branimir Lambov commented on CASSANDRA-9096: -------------------------------------------- Patch up for review [here|https://github.com/apache/cassandra/compare/trunk...blambov:9096-icompressor], tests [here|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-9096-icompressor-testall/]. Improves the {{ICompressor}} interface: - Clarifies buffer support by changing the misleadingly named {{useDirectOutputBuffers}} into a pair of methods supplying the preferred and supported buffer types. - Introduces {{BufferType}} enumeration to avoid passing around buffer type requirements as opaque booleans. - Changes ByteBuffer compression methods to use specified offsets and lengths to avoid having to duplicate for thread-safety. - Makes direct (and memory-mapped) buffers a common ground that all compressors support by implementing a solution for {{DeflateCompressor}}. - Removes {{WrappedByteBuffer}} and the clunkiness around it by providing an upper bound for required compressed buffer size for {{DeflateCompressor}}. - Shifts content a bit in {{CompressorTest}} to make sure offsets are properly tested. Due to [a bug in LZ4-java|https://github.com/jpountz/lz4-java/issues/64] sliced buffers (which we don't currently use) are not included. - Tests all buffer type combinations the compressors claim to support, not just the preferred. At the moment {{SnappyCompressor}} has to duplicate buffers as the version of snappy-java we use does not give any other option. The effect of this should not be noticeable, but if necessary this can be improved by either: - updating to current snappy-java and using unsafe methods to compress using buffer addresses, or - writing a patch for snappy-java to add the required methods, which could easily also include full support for heap buffers. > Improve ByteBuffer compression interface > ---------------------------------------- > > Key: CASSANDRA-9096 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9096 > Project: Cassandra > Issue Type: Improvement > Reporter: Branimir Lambov > Fix For: 3.x > > > Now that we have a few uses of compression/decompression on ByteBuffers it is > time to finalize the interface before it becomes set in stone with 3.0. The > current code has some shortcomings: > - The interface uses the buffers' positions and limits instead of accepting > offset and length as parameters. This necessitates that the buffers be > duplicated before they can be compressed for thread-safety, something that > adds burden to the caller, is prone to being forgotten, and we could > generally do without for performance. > - The direct/non-direct buffer support needs to be more clearly defined. The > current {{useDirectOutputByteBuffers}} is not named well. > - If we don't want to support non-direct buffers everywhere as a fallback, we > should clearly state the decision and rationale. > - How should {{WrappedByteBuffer}} treat direct/indirect buffers? > - More testing is necessary as e.g. errors in {{DeflateCompressor}} were only > caught in CASSANDRA-6809. -- This message was sent by Atlassian JIRA (v6.3.4#6332)