RE: MUTATION messages were dropped in last 5000 ms for cross node timeout

2017-08-03 Thread ZAIDI, ASAD A
Hi Akhil, Thank you for your reply. I kept testing different timeout numbers over last week and eventually settled at setting *_request_timeout_in_ms parameters at 1.5minutes for coordinator wait time. That is the number where I donot see any dropped mutations. Also asked developers to tweak

Re: Replacing a Seed Node

2017-08-03 Thread Oleksandr Shulgin
On Thu, Aug 3, 2017 at 3:00 PM, Fd Habash wrote: > Hi all … > > I know there is plenty of docs on how to replace a seed node, but some are > steps are contradictory e.g. need to remote the node from seed list for > entire cluster. > > > > My cluster has 6 nodes with 3 seeds

Replacing a Seed Node

2017-08-03 Thread Fd Habash
Hi all … I know there is plenty of docs on how to replace a seed node, but some are steps are contradictory e.g. need to remote the node from seed list for entire cluster. My cluster has 6 nodes with 3 seeds running C* 2.8. One seed node was terminated by AWS. I came up with this procedure.

Re: Bootstrapping a new Node with Consistency=ONE

2017-08-03 Thread Daniel Hölbling-Inzko
That makes sense. Thank you so much for pointing that out Alex. So long story short. Once I am up to the RF I actually want (RF3 per DC) and am just adding nodes for capacity joining the Ring will correctly work and no inconsistencies will exist. If I just change the RF the nodes don't have the

Re: Bootstrapping a new Node with Consistency=ONE

2017-08-03 Thread Oleksandr Shulgin
On Thu, Aug 3, 2017 at 9:33 AM, Daniel Hölbling-Inzko < daniel.hoelbling-in...@bitmovin.com> wrote: > No I set Auto bootstrap to true and the node was UN in nodetool status but > when doing a select on the node with ONE I got incomplete data. > What I think is happening here is not related to

Re: Bootstrapping a new Node with Consistency=ONE

2017-08-03 Thread Daniel Hölbling-Inzko
No I set Auto bootstrap to true and the node was UN in nodetool status but when doing a select on the node with ONE I got incomplete data. Jeff Jirsa schrieb am Do. 3. Aug. 2017 um 09:02: > "nodetool status" shows node as UN (up normal) instead of UJ (up joining) > > What

Re: Bootstrapping a new Node with Consistency=ONE

2017-08-03 Thread Jeff Jirsa
"nodetool status" shows node as UN (up normal) instead of UJ (up joining) What you're describing really sounds odd. Something isn't adding up to me but I'm not sure why. You shouldn't be able to query it directly until its bootstrapped as far as I know Are you sure you're not joining as a seed

Re: Bootstrapping a new Node with Consistency=ONE

2017-08-03 Thread Daniel Hölbling-Inzko
Thanks Jeff. How do I determine that bootstrap is finished? Haven't seen that anywhere so far. Reads via storage would be ok as every query would be checked by another node too. I was only seeing inconsistencies since clients went directly to the node with Consistency ONE Greetings Jeff Jirsa