[
https://issues.apache.org/jira/browse/CASSANDRA-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626735#comment-14626735
]
Andy Caldwell edited comment on CASSANDRA-9805 at 7/14/15 5:38 PM:
-------------------------------------------------------------------
Not 100% sure, I was monitoring the memory usage and I saw a steadily climbing
sawtooth (caused I assume by Eden collections) until the sawtooth hit the top
of the heap space, at which point it flattened out for a bit (Cassandra's logs
started talking about running low on Heap space and CPU usage climbed
dramatically) and then, a little while later, the memory usage dropped to
basically nothing (the DB was empty for this test), so I'm guessing it's one
then the other?
was (Author: boss_mc):
Not 100% sure, I was monitoring the memory usage and I saw a steadily climbing
sawtooth (caused I assume by Eden collections) until the sawtooth hit the top
of the heap space, at which point it flattened out for a bit (Cassandra's logs
started talking about running low on Heap space) and then, a little while
later, the memory usage dropped to basically nothing (the DB was empty for this
test), so I'm guessing it's one then the other?
> nodetool status causes garbage to be accrued
> --------------------------------------------
>
> Key: CASSANDRA-9805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9805
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Ubuntu 14.04 64-bit. Cassandra 2.0.14.
> Reporter: Andy Caldwell
> Fix For: 2.0.x
>
>
> As part of monitoring our Cassandra clusters (generally 2-6 nodes) we were
> running `nodetool status` regularly (~ every 5 minutes). On Cassandra 1.2.12
> this worked fine and had negligible effect on the Cassandra database service.
> Having upgraded to Cassandra 2.0.14, we've found that, over time, the tenured
> memory space slowly fills with `RMIConnectionImpl` objects (and some other
> associated objects) until we start running into memory pressure and
> triggering proactive and then STW GC (which obviously impact performance of
> the cluster). It seems that these objects are kept around long enough to get
> promoted to tenured from Eden and then don't get considered for collection
> (due to internal reference cycles?).
> Very easy to reproduce, just call `nodetool status` in a loop and watch the
> memory usage climb to capacity then drop to empty after STW. No need to be
> accessing the DB keys at all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
