Re: Overload because of hint pressure + MVs

2020-02-11 Thread Surbhi Gupta
a win to go > to 31GB. > > > > R > > > > *From: *Erick Ramirez > *Reply-To: *"user@cassandra.apache.org" > *Date: *Monday, February 10, 2020 at 3:55 PM > *To: *"user@cassandra.apache.org" > *Subject: *Re: Overload because of hint pressure + MV

Re: Overload because of hint pressure + MVs

2020-02-11 Thread Reid Pinchback
g" Subject: Re: Overload because of hint pressure + MVs Message from External Sender Currently the value of phi_convict_threshold is not set which makes it to 8 (default) . Can this also cause hints buildup even when we can see that all nodes are UP ? You can bump it up to 12 to reduce the

Re: Overload because of hint pressure + MVs

2020-02-10 Thread Erick Ramirez
> > Currently the value of phi_convict_threshold is not set which makes it to > 8 (default) . > Can this also cause hints buildup even when we can see that all nodes are > UP ? You can bump it up to 12 to reduce the sensitivity but it's likely GC pauses causing it. Phi convict is the

Re: Overload because of hint pressure + MVs

2020-02-10 Thread Surbhi Gupta
Just to add , we are using 24GB heap size. On Mon, 10 Feb 2020 at 09:08, Surbhi Gupta wrote: > Hi Jon, > > We are on multi datacenter(On Prim) setup. > We also noticed too many messages like below: > > DEBUG [GossipStage:1] 2020-02-10 09:38:52,953 FailureDetector.java:457 - > Ignoring interval

Re: Overload because of hint pressure + MVs

2020-02-10 Thread Surbhi Gupta
Hi Jon, We are on multi datacenter(On Prim) setup. We also noticed too many messages like below: DEBUG [GossipStage:1] 2020-02-10 09:38:52,953 FailureDetector.java:457 - Ignoring interval time of 3258125997 for /10.x.x.x DEBUG [GossipStage:1] 2020-02-10 09:38:52,954 FailureDetector.java:457 -

Re: Overload because of hint pressure + MVs

2020-02-09 Thread Surbhi Gupta
Thanks a lot Jon.. Will try the recommendations and let you know the results On Fri, Feb 7, 2020 at 10:52 AM Jon Haddad wrote: > There's a few things you can do here that might help. > > First off, if you're using the default heap settings, that's a serious > problem. If you've got the

Re: Overload because of hint pressure + MVs

2020-02-07 Thread Jon Haddad
There's a few things you can do here that might help. First off, if you're using the default heap settings, that's a serious problem. If you've got the head room, my recommendation is to use 16GB heap with 12 GB new gen and pin your memtable heap space to 2GB. Set your max tenuring threshold to

Re: Overload because of hint pressure + MVs

2020-02-07 Thread Michael Shuler
That JIRA still says Open, so no, it has not been fixed (unless there's a fixed duplicate in JIRA somewhere). For clarification, you could update that ticket with a comment including your environmental details, usage of MV, etc. I'll bump the priority up and include some possible branchX

Overload because of hint pressure + MVs

2020-02-07 Thread Surbhi Gupta
Hi, We are getting hit by the below bug. Other than lowering hinted_handoff_throttle_in_kb to 100 any other work around ? https://issues.apache.org/jira/browse/CASSANDRA-13810 Any idea if it got fixed in later version. We are on Open source Cassandra 3.11.1 . Thanks Surbhi