Re: Upgrade from 2.1 to 3.11

2018-08-31 Thread Gosar M
are under control and normal.  This may not be related to upgrade at all!     From: Pradeep Chhetri [mailto:prad...@stashaway.com] Sent: Tuesday, August 28, 2018 3:32 AM To: user@cassandra.apache.org Subject: Re: Upgrade from 2.1 to 3.11   You may want to try upgrading to 3.11.3 instead which has

Re: Upgrade from 2.1 to 3.11

2018-08-31 Thread Mun Dega
t be related to upgrade at all! > > > > > > *From:* Pradeep Chhetri [mailto:prad...@stashaway.com] > *Sent:* Tuesday, August 28, 2018 3:32 AM > *To:* user@cassandra.apache.org > *Subject:* Re: Upgrade from 2.1 to 3.11 > > > > You may want to try upgrading to

RE: Upgrade from 2.1 to 3.11

2018-08-28 Thread ZAIDI, ASAD A
...@stashaway.com] Sent: Tuesday, August 28, 2018 3:32 AM To: user@cassandra.apache.org Subject: Re: Upgrade from 2.1 to 3.11 You may want to try upgrading to 3.11.3 instead which has some memory leaks fixes. On Tue, Aug 28, 2018 at 9:59 AM, Mun Dega mailto:mundeg...@gmail.com>> wrote: I am sur

Re: Upgrade from 2.1 to 3.11

2018-08-28 Thread Brian Spindler
Ma, did you try what Mohamadreza suggested? Have a such a large heap means you are getting a ton of stuff that needs full GC. On Tue, Aug 28, 2018 at 4:31 AM Pradeep Chhetri wrote: > You may want to try upgrading to 3.11.3 instead which has some memory > leaks fixes. > > On Tue, Aug 28, 2018

Re: Upgrade from 2.1 to 3.11

2018-08-28 Thread Pradeep Chhetri
You may want to try upgrading to 3.11.3 instead which has some memory leaks fixes. On Tue, Aug 28, 2018 at 9:59 AM, Mun Dega wrote: > I am surprised that no one else ran into any issues with this version. GC > can't catch up fast enough and there is constant Full GC taking place. > > The

Re: Upgrade from 2.1 to 3.11

2018-08-27 Thread Mun Dega
I am surprised that no one else ran into any issues with this version. GC can't catch up fast enough and there is constant Full GC taking place. The result? unresponsive nodes makeing entire cluster unusable. Any insight on this issue from anyone that is using this version would be appreciated.

Re: Upgrade from 2.1 to 3.11

2018-08-24 Thread Mohamadreza Rostami
You have very large heap,it’s take most of cpu time in GC stage.you should in maximum set heap on 12GB and enable row cache to your cluster become faster. On Friday, 24 August 2018, Mun Dega wrote: > 120G data > 28G heap out of 48 on system > 9 node cluster, RF3 > > > On Thu, Aug 23, 2018,

Re: Upgrade from 2.1 to 3.11

2018-08-23 Thread Mun Dega
Looks like there are a couple of issues open regarding 3.11.2 release: https://issues.apache.org/jira/browse/CASSANDRA-14355 https://issues.apache.org/jira/browse/CASSANDRA-14495 Ma On Thu, Aug 23, 2018 at 10:54 PM Mun Dega wrote: > Interesting. Any other suggestions what other things

Re: Upgrade from 2.1 to 3.11

2018-08-23 Thread Mun Dega
Interesting. Any other suggestions what other things changed that would be major between 2.1 and 3.x? In Change Log all isee is bug fixes and further improvements. On Thu, Aug 23, 2018, 21:37 Gosar M wrote: > we also increased the max_heap_count (sysctl.conf) value to resolve OOO > memory

Re: Upgrade from 2.1 to 3.11

2018-08-23 Thread Gosar M
we also increased the max_heap_count (sysctl.conf) value to resolve OOO memory issue. On Thursday, 23 August 2018, 16:04:20 GMT-7, Mun Dega wrote: 120G data28G heap out of 48 on system9 node cluster, RF3 On Thu, Aug 23, 2018, 17:19 Mohamadreza Rostami wrote: Hi,How much data do

Re: Upgrade from 2.1 to 3.11

2018-08-23 Thread Mun Dega
120G data 28G heap out of 48 on system 9 node cluster, RF3 On Thu, Aug 23, 2018, 17:19 Mohamadreza Rostami < mohamadrezarosta...@gmail.com> wrote: > Hi, > How much data do you have? How much RAM do your servers have? How much do > you have a heep? > On Thu, Aug 23, 2018 at 10:14 PM Mun Dega

Re: Upgrade from 2.1 to 3.11

2018-08-23 Thread Mohamadreza Rostami
Hi, How much data do you have? How much RAM do your servers have? How much do you have a heep? On Thu, Aug 23, 2018 at 10:14 PM Mun Dega wrote: > Hello, > > We recently upgraded from Cassandra 2.1 to 3.11.2 on one cluster. The > process went OK including upgradesstable but we started to

Upgrade from 2.1 to 3.11

2018-08-23 Thread Mun Dega
Hello, We recently upgraded from Cassandra 2.1 to 3.11.2 on one cluster. The process went OK including upgradesstable but we started to experience high latency for r/w, occasional OOM and long GC pause after. For the same cluster with 2.1, we didn't have any issues like this. We also kept