[
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360980#comment-14360980
]
Benedict edited comment on CASSANDRA-6809 at 3/13/15 7:31 PM:
--------------------------------------------------------------
The idea of making it a "hard limit" instead of the concrete number is to fix
it as something much larger than you would like it to be, but no larger than
you really must expect it to go, so that it can scale gracefully with some
blips and avoid those blips having severe negative repurcussions. My
expectation is that whatever strategy we use here we will also use for
non-compressed once we migrate to manually managed memory buffers, and it seems
to me always having deallocations trail utilisation by 1 (so the first to
become unused we do not deallocate, but if another becomes unused we release
one of the two) probably gives us pretty good behaviour without any extra
tuning knobs
was (Author: benedict):
The idea of making it a "hard limit" instead of the concrete number is to fix
it as something much larger than you would like it to be, but no larger than
you really must expect it to go, so that it can scale gracefully with some
blips and avoid those blips having severe negative repurcussions. My
expectation is that whatever strategy we use here we will also use for
non-compressed once we migrate to manually managed memory buffers, and it seems
to me always having deallocations trail utilisation by 1 (so the first to
become unused we do not deallocate, but if another becomes unused we release
one of the two)
> Compressed Commit Log
> ---------------------
>
> Key: CASSANDRA-6809
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Benedict
> Assignee: Branimir Lambov
> Priority: Minor
> Labels: docs-impacting, performance
> Fix For: 3.0
>
> Attachments: ComitLogStress.java, logtest.txt
>
>
> It seems an unnecessary oversight that we don't compress the commit log.
> Doing so should improve throughput, but some care will need to be taken to
> ensure we use as much of a segment as possible. I propose decoupling the
> writing of the records from the segments. Basically write into a (queue of)
> DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X
> MB written to the CL (where X is ordinarily CLS size), and then pack as many
> of the compressed chunks into a CLS as possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
