[
https://issues.apache.org/jira/browse/CASSANDRA-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149279#comment-13149279
]
Mck SembWever commented on CASSANDRA-3427:
------------------------------------------
Handling jvm memory since upgrading to cassandra-1.0 and enabling compression
is still a headache.
Where i used to be able to run w/ Xmx8G i'm now struggling to run with Xmx20G
(all caches are disabled) and during startup can hit
{noformat}java.lang.OutOfMemoryError: Java heap space
at org.apache.cassandra.utils.BigLongArray.<init>(BigLongArray.java:53)
at org.apache.cassandra.utils.BigLongArray.<init>(BigLongArray.java:39)
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:122){noformat}
I could keep increasing chunk_length (it's already at 256) but this seems
awkward just to get a cluster running smoothly. At minimum the calculations for
memory requirements for cassandra should be re-written if compression is to
take such a large chunk of heap.
> CompressionMetadata is not shared across threads, we create a new one for
> each read
> -----------------------------------------------------------------------------------
>
> Key: CASSANDRA-3427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3427
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Labels: compression
> Fix For: 1.0.2
>
> Attachments: 3427.patch, 3427_v2.patch, CASSANDRA-3427.patch
>
>
> The CompressionMetada holds the compressed block offsets in memory. Without
> being absolutely huge, this is still of non-negligible size as soon as you
> have a bit of data in the DB. Reallocating this for each read is a very bad
> idea.
> Note that this only affect range queries, since "normal" queries uses
> CompressedSegmentedFile that does reuse a unique CompressionMetadata instance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
