[
https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174958#comment-13174958
]
Sylvain Lebresne commented on CASSANDRA-3623:
---------------------------------------------
When I was originally checking the compression code, I did a few quick stress
tests under yourkit, and in cases where the data was small and thus entirely in
page cache, the cost of creating a RandomAccessFile on each read was taking the
majority of the time. Note that I'm not saying this to justify this ticket
necessarily, since:
# we could use an object pool to avoid that cost
# my tests were really toy test, so they would need confirmation
but mmapping does avoid this cost. Just saying.
I'll also note that Snappy has a way to decompress data from direct ByteBuffer
directly
([snappydoc|http://wiki.snappy-java.googlecode.com/hg/apidocs/org/xerial/snappy/Snappy.html#compress(java.nio.ByteBuffer,
java.nio.ByteBuffer)]) so this could potentially avoid 1 copy (we would go
from page cache to decompressed buffer directly). Of course we should look how
well that works, but again, just to feed the discussion.
> use MMapedBuffer in CompressedSegmentedFile.getSegment
> ------------------------------------------------------
>
> Key: CASSANDRA-3623
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.1
> Reporter: Vijay
> Assignee: Vijay
> Labels: compression
> Fix For: 1.1
>
> Attachments: 0001-MMaped-Compression-segmented-file.patch
>
>
> CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to
> use the MMap and hence a higher CPU on the nodes and higher latencies on
> reads.
> This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
> // TODO refactor this to separate concept of "buffer to avoid lots of read()
> syscalls" and "compression buffer"
> but i think a separate class for the Buffer will be better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
