Thanks for the detail. I’ll have to remove and then add one back in. It’s my consistency levels that may bite me in the interim.
Thanks Mark 801-705-7115 office From: Jeff Jirsa [mailto:[email protected]] Sent: Friday, October 6, 2017 11:29 AM To: cassandra <[email protected]> Subject: Re: Node failure There's a lot to talk about here, what's your exact question? - You can either remove it from the cluster or replace it. You typically remove it if it'll never be replaced, but in RF=3 with 3 nodes, you probably need to replace it. To replace, you'll start a new server with -Dcassandra.replace_address=a.b.c.d ( http://cassandra.apache.org/doc/latest/operating/topo_changes.html#replacing-a-dead-node ) , and it'll stream data from the neighbors and eventually replace the dead node in the ring (the dead node will be removed from 'nodetool status', the new node will be there instead). - If you're not going to replace it, things get a bit more complex - you'll do some combination of repair, 'nodetool removenode' or 'nodetool assassinate', and ALTERing the keyspace to set RF=2. The order matters, and so does the consistency level you use for reads/writes (so we can tell you whether or not you're likely to lose data in this process), so I'm not giving step-by-steps here because it's not very straight forward and there are a lot of caveats. On Fri, Oct 6, 2017 at 10:20 AM, Mark Furlong <[email protected]<mailto:[email protected]>> wrote: What happens when I have a 3 node cluster with RF 3 and a node fails that needs to be removed? Mark Furlong Sr. Database Administrator [email protected]<mailto:[email protected]> M: 801-859-7427<tel:(801)%20859-7427> O: 801-705-7115<tel:(801)%20705-7115> 1300 W Traverse Pkwy<https://maps.google.com/?q=1300+W+Traverse+Pkwy%0D+Lehi,+UT+84043&entry=gmail&source=g> Lehi, UT 84043<https://maps.google.com/?q=1300+W+Traverse+Pkwy%0D+Lehi,+UT+84043&entry=gmail&source=g> [http://c.mfcreative.com/mars/email/shared-icon/sig-logo.gif]
