I don't disagree with jon. On Wed, Feb 21, 2018 at 10:27 AM, Jonathan Haddad <[email protected]> wrote:
> The easiest way to do this is replacing one node at a time by using > rsync. I don't know why it has to be more complicated than copying data to > a new machine and replacing it in the cluster. Bringing up a new DC with > snapshots is going to be a nightmare in comparison. > > On Wed, Feb 21, 2018 at 8:16 AM Carl Mueller <[email protected]> > wrote: > >> DCs can be stood up with snapshotted data. >> >> >> Stand up a new cluster with your old cluster snapshots: >> >> https://docs.datastax.com/en/cassandra/2.1/cassandra/ >> operations/ops_snapshot_restore_new_cluster.html >> >> Then link the DCs together. >> >> Disclaimer: I've never done this in real life. >> >> On Wed, Feb 21, 2018 at 9:25 AM, Nitan Kainth <[email protected]> >> wrote: >> >>> New dc will be faster but may impact cluster performance due to >>> streaming. >>> >>> Sent from my iPhone >>> >>> On Feb 21, 2018, at 8:53 AM, Leena Ghatpande <[email protected]> >>> wrote: >>> >>> We do use LOCAL_ONE and LOCAL_Quorum currently. But these 8 nodes need >>> to be in 2 different DC< so we would end up create additional 2 new DC and >>> dropping 2. >>> >>> are there any advantages on adding DC over one node at a time? >>> >>> >>> ------------------------------ >>> *From:* Jeff Jirsa <[email protected]> >>> *Sent:* Wednesday, February 21, 2018 1:02 AM >>> *To:* [email protected] >>> *Subject:* Re: Best approach to Replace existing 8 smaller nodes in >>> production cluster with New 8 nodes that are bigger in capacity, without a >>> downtime >>> >>> You add the nodes with rf=0 so there’s no streaming, then bump it to >>> rf=1 and run repair, then rf=2 and run repair, then rf=3 and run repair, >>> then you either change the app to use local quorum in the new dc, or >>> reverse the process by decreasing the rf in the original dc by 1 at a time >>> >>> -- >>> Jeff Jirsa >>> >>> >>> > On Feb 20, 2018, at 8:51 PM, Kyrylo Lebediev <[email protected]> >>> wrote: >>> > >>> > I'd say, "add new DC, then remove old DC" approach is more risky >>> especially if they use QUORUM CL (in this case they will need to change CL >>> to LOCAL_QUORUM, otherwise they'll run into a lot of blocking read repairs). >>> > Also, if there is a chance to get rid of streaming, it worth doing as >>> usually direct data copy (not by means of C*) is more effective and less >>> troublesome. >>> > >>> > Regards, >>> > Kyrill >>> > >>> > ________________________________________ >>> > From: Nitan Kainth <[email protected]> >>> > Sent: Wednesday, February 21, 2018 1:04:05 AM >>> > To: [email protected] >>> > Subject: Re: Best approach to Replace existing 8 smaller nodes in >>> production cluster with New 8 nodes that are bigger in capacity, without a >>> downtime >>> > >>> > You can also create a new DC and then terminate old one. >>> > >>> > Sent from my iPhone >>> > >>> >> On Feb 20, 2018, at 2:49 PM, Kyrylo Lebediev < >>> [email protected]> wrote: >>> >> >>> >> Hi, >>> >> Consider using this approach, replacing nodes one by one: >>> https://mrcalonso.com/2016/01/26/cassandra-instantaneous-in- >>> place-node-replacement/ >>> >>> <https://mrcalonso.com/2016/01/26/cassandra-instantaneous-in-place-node-replacement/> >>> Cassandra instantaneous in place node replacement | Carlos ... >>> <https://mrcalonso.com/2016/01/26/cassandra-instantaneous-in-place-node-replacement/> >>> mrcalonso.com >>> At some point everyone using Cassandra faces the situation of having to >>> replace nodes. Either because the cluster needs to scale and some nodes are >>> too small or ... >>> >>> >> >>> >> Regards, >>> >> Kyrill >>> >> >>> >> ________________________________________ >>> >> From: Leena Ghatpande <[email protected]> >>> >> Sent: Tuesday, February 20, 2018 10:24:24 PM >>> >> To: [email protected] >>> >> Subject: Best approach to Replace existing 8 smaller nodes in >>> production cluster with New 8 nodes that are bigger in capacity, without a >>> downtime >>> >> >>> >> Best approach to replace existing 8 smaller 8 nodes in production >>> cluster with New 8 nodes that are bigger in capacity without a downtime >>> >> >>> >> We have 4 nodes each in 2 DC, and we want to replace these 8 nodes >>> with new 8 nodes that are bigger in capacity in terms of RAM,CPU and >>> Diskspace without a downtime. >>> >> The RF is set to 3 currently, and we have 2 large tables with upto >>> 70Million rows >>> >> >>> >> What would be the best approach to implement this >>> >> - Add 1 New Node and Decomission 1 Old node at a time? >>> >> - Add all New nodes to the cluster, and then decommission old >>> nodes ? >>> >> If we do this, can we still keep the RF=3 while we have 16 >>> nodes at a point in the cluster before we start decommissioning? >>> >> - How long do we wait in between adding a Node or decomissiing to >>> ensure the process is complete before we proceed? >>> >> - Any tool that we can use to monitor if the add/decomission node >>> is done before we proceed to next >>> >> >>> >> Any other suggestion? >>> >> >>> >> >>> >> --------------------------------------------------------------------- >>> >> To unsubscribe, e-mail: [email protected] >>> >> For additional commands, e-mail: [email protected] >>> >> >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] >>> > For additional commands, e-mail: [email protected] >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] >>> > For additional commands, e-mail: [email protected] >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>
