[
https://issues.apache.org/jira/browse/CASSANDRA-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152981#comment-15152981
]
Ariel Weisberg commented on CASSANDRA-10971:
--------------------------------------------
The memory mapped implementation doesn't need/want to bound the number of
buffers in flight. Backpressure comes from the operating system which will
block writer threads when there isn't enough free memory to buffer writes.
You are right that this would be simpler if the {{CLSM}} maintained the bound.
It's already being woken up every time a segment is discarded. I'll rewrite it
that way. I'll only have it bound if there is comrpression.
> Compressed commit log has no backpressure and can OOM
> -----------------------------------------------------
>
> Key: CASSANDRA-10971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10971
> Project: Cassandra
> Issue Type: Improvement
> Components: Local Write-Read Paths
> Reporter: Ariel Weisberg
> Assignee: Ariel Weisberg
> Fix For: 3.x
>
>
> I validated this via a unit test that slowed the ability of the log to drain
> to the filesystem. The compressed commit log will keep allocating buffers
> pending compression until it OOMs.
> I have a fix that am not very happy with because the whole signal a thread to
> allocate a segment that depends on a resource that may not be available
> results in some obtuse usage of {{CompleatableFuture}} to rendezvous
> available buffers with {{CommitLogSegmentManager}} thread waiting to finish
> constructing a new segment. The {{CLSM}} thread is in turn signaled by the
> thread(s) that actually wants to write to the next segment, but aren't able
> to do it themselves.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
