[
https://issues.apache.org/jira/browse/CASSANDRA-9681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606026#comment-14606026
]
Benedict commented on CASSANDRA-9681:
-------------------------------------
Can you please attach your cassandra.yaml and full log file history since the
problems began?
> Memtable heap size grows and many long GC pauses are triggered
> --------------------------------------------------------------
>
> Key: CASSANDRA-9681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9681
> Project: Cassandra
> Issue Type: Bug
> Environment: C* 2.1.7, Debian Wheezy
> Reporter: mlowicki
> Assignee: Benedict
> Priority: Critical
> Fix For: 2.1.x
>
>
> C* 2.1.7 cluster is behaving really bad after 1-2 days.
> {{gauges.cassandra.jmx.org.apache.cassandra.metrics.ColumnFamily.AllMemtablesHeapSize.Value}}
> jumps to 7 GB
> (https://www.dropbox.com/s/vraggy292erkzd2/Screenshot%202015-06-29%2019.12.53.png?dl=0)
> on 3/6 nodes in each data center and then there are many long GC pauses.
> Cluster is using default heap size values ({{-Xms8192M -Xmx8192M -Xmn2048M}})
> Before C* 2.1.5 memtables heap size was basically constant ~500MB
> (https://www.dropbox.com/s/fjdywik5lojstvn/Screenshot%202015-06-29%2019.30.00.png?dl=0)
> After restarting all nodes is behaves stable for 1-2days. Today I've done
> that and long GC pauses are gone (~18:00
> https://www.dropbox.com/s/7vo3ynz505rsfq3/Screenshot%202015-06-29%2019.28.37.png?dl=0).
> The only pattern we've found so far is that long GC pauses are happening
> basically at the same time on all nodes in the same data center - even on the
> ones where memtables heap size is not growing.
> Cliffs on the graphs are nodes restarts.
> Used memory on boxes where {{AllMemtabelesHeapSize}} grows, stays at the same
> level -
> https://www.dropbox.com/s/tes9abykixs86rf/Screenshot%202015-06-29%2019.37.52.png?dl=0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
