[
https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959057#comment-14959057
]
Robbie Strickland commented on CASSANDRA-10449:
-----------------------------------------------
I discovered that an index on one of the tables has a wide row, and I'm
assuming that to be the root of the issue:
Example:
{noformat}
Compacted partition minimum bytes: 125
Compacted partition maximum bytes: 10299432635
Compacted partition mean bytes: 253692309
{noformat}
This seems like a problem in general for indexes, where the original data model
may be well distributed but the index may have unpredictable distribution.
> OOM on bootstrap due to long GC pause
> -------------------------------------
>
> Key: CASSANDRA-10449
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 14.04, AWS
> Reporter: Robbie Strickland
> Labels: gc
> Fix For: 2.1.x
>
> Attachments: system.log.10-05, thread_dump.log
>
>
> I have a 20-node cluster (i2.4xlarge) with vnodes (default of 256) and
> 500-700GB per node. SSTable counts are <10 per table. I am attempting to
> provision additional nodes, but bootstrapping OOMs every time after about 10
> hours with a sudden long GC pause:
> {noformat}
> INFO [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old
> Generation GC in 1586126ms. G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380
> CassandraDaemon.java:223 - Exception in thread
> Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to
> no avail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
