In all tables SSTable counts is below 30. On Thu, Feb 19, 2015 at 9:43 AM, Carlos Rolo <r...@pythian.com> wrote:
> Can you check how many SSTables you have? It is more or less a know fact > that 2.1.2 has lots of problems with compaction so a upgrade can solve it. > But a high number of SSTables can confirm that indeed compaction is your > problem not something else. > > Regards, > > Carlos Juzarte Rolo > Cassandra Consultant > > Pythian - Love your data > > rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo > <http://linkedin.com/in/carlosjuzarterolo>* > Tel: 1649 > www.pythian.com > > On Thu, Feb 19, 2015 at 9:16 AM, Michał Łowicki <mlowi...@gmail.com> > wrote: > >> We don't have other things running on these boxes and C* is consuming all >> the memory. >> >> Will try to upgrade to 2.1.3 and if won't help downgrade to 2.1.2. >> >> — >> Michał >> >> >> On Thu, Feb 19, 2015 at 2:39 AM, Jacob Rhoden <jacob.rho...@me.com> >> wrote: >> >>> Are you tweaking the "nice" priority on Cassandra? (Type: man nice) if >>> you don't know much about it. Certainly improving cassandra's nice score >>> becomes important when you have other things running on the server like >>> scheduled jobs of people logging in to the server and doing things. >>> >>> ______________________________ >>> Sent from iPhone >>> >>> On 19 Feb 2015, at 5:28 am, Michał Łowicki <mlowi...@gmail.com> wrote: >>> >>> Hi, >>> >>> Couple of times a day 2 out of 4 members cluster nodes are killed >>> >>> root@db4:~# dmesg | grep -i oom >>> [4811135.792657] [ pid ] uid tgid total_vm rss cpu oom_adj >>> oom_score_adj name >>> [6559049.307293] java invoked oom-killer: gfp_mask=0x201da, order=0, >>> oom_adj=0, oom_score_adj=0 >>> >>> Nodes are using 8GB heap (confirmed with *nodetool info*) and aren't >>> using row cache. >>> >>> Noticed that couple of times a day used RSS is growing really fast >>> within couple of minutes and I see CPU spikes at the same time - >>> https://www.dropbox.com/s/khco2kdp4qdzjit/Screenshot%202015-02-18%2015.10.54.png?dl=0 >>> . >>> >>> Could be related to compaction but after compaction is finished used RSS >>> doesn't shrink. Output from pmap when C* process uses 50GB RAM (out of >>> 64GB) is available on http://paste.ofcode.org/ZjLUA2dYVuKvJHAk9T3Hjb. >>> At the time dump was made heap usage is far below 8GB (~3GB) but total RSS >>> is ~50GB. >>> >>> Any help will be appreciated. >>> >>> -- >>> BR, >>> Michał Łowicki >>> >>> >> > > -- > > > > -- BR, Michał Łowicki
