[
https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257687#comment-15257687
]
Pavel Yaskevich commented on CASSANDRA-5863:
--------------------------------------------
bq. This is not that trivial for rebuffering – the requested length would
normally be bigger than the number of available bytes during the last
{{reBuffer()}} call on the file, thus the changes required to implement this
check are too substantial to be within the scope of this ticket.
Ok, fair enough.
bq. There are other things to take into account:
Let me address the things you mentioned - all of the data is written into
SSTable file in fixed size chucks and most of the rebuffers are done (or have
been done) in the granularity of 64k or what ever compression buffer size was,
since SSTable has compression parameters we might want to have cache work on
the level of sstables instead of individual files, that way we can get access
to some essential metadata. So what I was saying is that cache could hold
already decompressed 64k (or other power of 2 size) aligned buffers, either raw
file data or decompressed data based on the file; backend implementation,
plugged into rebufferer, would mmap or use regular channel read to read buffer
size aligned chunks based on position given to it, in compressed mode cache
would hold decompressed buffers so it doesn't have to share mmap'ed buffers.
ReaderCache can rely on LIRS or LRU as a replacement mechanism for aligned
buffers, so each buffer is going to be reclaimed when either sstable is removed
(another reason to work closely with sstables) or replacement mechanism
indicates that buffer is no longer viable or invalidated manually. Sorry I
can't provide code, so if you think that rethinking this is not worth it, I'm
fine with that.
bq. The logic is clearly defined for this round. This patch is targeted at 3.x
where we can't change sstable format in any way, and support for this format
will be required long in the future.
I wasn't aware of that. [~jbellis] We can't modify SSTable format at all while
in 3.x phrase, even backward compatible changes in the even feature releases?
> In process (uncompressed) page cache
> ------------------------------------
>
> Key: CASSANDRA-5863
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5863
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: T Jake Luciani
> Assignee: Branimir Lambov
> Labels: performance
> Fix For: 3.x
>
>
> Currently, for every read, the CRAR reads each compressed chunk into a
> byte[], sends it to ICompressor, gets back another byte[] and verifies a
> checksum.
> This process is where the majority of time is spent in a read request.
> Before compression, we would have zero-copy of data and could respond
> directly from the page-cache.
> It would be useful to have some kind of Chunk cache that could speed up this
> process for hot data, possibly off heap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
