Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Hossein Ghiyasi Mehr]

There isn't any Rollback in real time systems.
It's better to test upgrade sstables and binary on one node. Then

   - if it was OK, upgrade binary on all nodes then run upgrade sstables
   one server at a time.

OR

   - If it was OK, upgrade servers (binary+sstables) one by one.

*---*
*VafaTech <http://www.vafatech.com> : A Total Solution for Data Gathering &
Analysis*
*---*


On Wed, Mar 4, 2020 at 7:38 AM manish khandelwal <
manishkhandelwa...@gmail.com> wrote:

> Should upgradesstables not be run after every node is upgraded? If we need
> to rollback then  we will not be able to downgrade sstables to older
> version.
>
> Regards
> Manish
>
> On Tue, Mar 3, 2020 at 11:26 PM Hossein Ghiyasi Mehr <
> ghiyasim...@gmail.com> wrote:
>
>> It's more safe to upgrade one node before upgrading another node to avoid
>> down time.
>> After upgrading binary and package, run upgradesstables on candidate node
>> then do it on all cluster nodes one by one.
>> *---*
>> *VafaTech <http://www.vafatech.com> : A Total Solution for Data Gathering
>> & Analysis*
>> *---*
>>
>>
>> On Thu, Feb 13, 2020 at 9:27 PM Sergio  wrote:
>>
>>>
>>>- Verify that nodetool upgradesstables has completed successfully on
>>>all nodes from any previous upgrade
>>>- Turn off repairs and any other streaming operations (add/remove
>>>nodes)
>>>- Nodetool drain on the node that needs to be stopped (seeds first,
>>>preferably)
>>>- Stop an un-upgraded node (seeds first, preferably)
>>>- Install new binaries and configs on the down node
>>>- Restart that node and make sure it comes up clean (it will
>>>function normally in the cluster – even with mixed versions)
>>>- nodetool statusbinary to verify if it is up and running
>>>- Repeat for all nodes
>>>- Once the binary upgrade has been performed in all the nodes: Run
>>>upgradesstables on each node (as many at a time as your load will allow).
>>>Minor upgrades usually don’t require this step (only if the sstable 
>>> format
>>>has changed), but it is good to check.
>>>- NOTE: in most cases applications can keep running and will not
>>>notice much impact – unless the cluster is overloaded and a single node
>>>down causes impact.
>>>
>>>
>>>
>>>I added 2 points to the list to clarify.
>>>
>>>Should we add this in a FAQ in the cassandra doc or in the awesome
>>>cassandra https://cassandra.link/awesome/
>>>
>>>Thanks,
>>>
>>>Sergio
>>>
>>>
>>> Il giorno mer 12 feb 2020 alle ore 10:58 Durity, Sean R <
>>> sean_r_dur...@homedepot.com> ha scritto:
>>>
>>>> Check the readme.txt for any upgrade notes, but the basic procedure is
>>>> to:
>>>>
>>>>- Verify that nodetool upgradesstables has completed successfully
>>>>on all nodes from any previous upgrade
>>>>- Turn off repairs and any other streaming operations (add/remove
>>>>nodes)
>>>>- Stop an un-upgraded node (seeds first, preferably)
>>>>- Install new binaries and configs on the down node
>>>>- Restart that node and make sure it comes up clean (it will
>>>>function normally in the cluster – even with mixed versions)
>>>>- Repeat for all nodes
>>>>    - Run upgradesstables on each node (as many at a time as your load
>>>>will allow). Minor upgrades usually don’t require this step (only if the
>>>>sstable format has changed), but it is good to check.
>>>>- NOTE: in most cases applications can keep running and will not
>>>>notice much impact – unless the cluster is overloaded and a single node
>>>>down causes impact.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Sean Durity – Staff Systems Engineer, Cassandra
>>>>
>>>>
>>>>
>>>> *From:* Sergio 
>>>> *Sent:* Wednesday, February 12, 2020 11:36 AM
>>>> *To:* user@cassandra.apache.org
>>>> *Subject:* [EXTERNAL] Cassandra 3.11.X upgrades
>>>>
>>>>
>>>>
>>>> Hi 

RE: [EXTERNAL] Cassandra 3.11.X upgrades

[Durity, Sean R]

I agree – a back out becomes practically very challenging after the second node 
is upgraded, because the new data is written in the new disk format. To satisfy 
the “you must have a backout” rules, I just say that after node 1, I could stop 
that node, wipe the data, downgrade the binaries, and replace that node back to 
the original version (and yes, there could still be consistency problems with 
that). There is no going back after node 2. And I have never needed to try and 
go back, either. Test well in NP, and be ready to tackle any PR problems to 
keep going forward.

Sean Durity

From: Erick Ramirez 
Sent: Tuesday, March 3, 2020 11:35 PM
To: user@cassandra.apache.org
Subject: Re: [EXTERNAL] Cassandra 3.11.X upgrades

Should upgradesstables not be run after every node is upgraded? If we need to 
rollback then  we will not be able to downgrade sstables to older version

You can choose to (a) upgrade the SSTables one node at a time as you complete 
the binary upgrade, or (b) upgrade the binaries on all nodes then perform the 
SSTables upgrade in one hit. The choice is up to you but beware of the 
following caveats:
- there's a performance hit when C* reads "n - 1" versions of SSTables so the 
sooner you do it the better
- upgrading SSTables one node at a time is preferable due to the considerable 
performance hit or schedule it during low traffic periods

The idea of a rollback isn't what you're accustomed to. There is no concept of 
"downgrade" in Cassandra. If you decide to rollback or backout of an upgrade 
implementation, it means that you have to restore your cluster from backups so 
be aware of that too. This is because once you've performed an upgrade, things 
like schema and system tables are generally no longer backward-compatible. 
Also, new incoming mutations are written in the new format which again is not 
backward-compatible. To cut to the chase -- the decision to upgrade the 
SSTables has zero bearing on rollback. Cheers!




The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any damages or losses arising from any inaccuracies, errors, viruses, 
e.g., worms, trojan horses, etc., or other items of a destructive nature, which 
may be contained in this attachment and shall not be liable for direct, 
indirect, consequential or special damages in connection with this e-mail 
message or its attachment.

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Erick Ramirez]

>
> Should upgradesstables not be run after every node is upgraded? If we need
> to rollback then  we will not be able to downgrade sstables to older version
>

You can choose to (a) upgrade the SSTables one node at a time as you
complete the binary upgrade, or (b) upgrade the binaries on all nodes then
perform the SSTables upgrade in one hit. The choice is up to you but beware
of the following caveats:
- there's a performance hit when C* reads "n - 1" versions of SSTables so
the sooner you do it the better
- upgrading SSTables one node at a time is preferable due to the
considerable performance hit or schedule it during low traffic periods

The idea of a rollback isn't what you're accustomed to. There is no concept
of "downgrade" in Cassandra. If you decide to rollback or backout of an
upgrade implementation, it means that you have to restore your cluster from
backups so be aware of that too. This is because once you've performed an
upgrade, things like schema and system tables are generally no longer
backward-compatible. Also, new incoming mutations are written in the new
format which again is not backward-compatible. To cut to the chase -- the
decision to upgrade the SSTables has zero bearing on rollback. Cheers!

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Anthony Grasso]

Manish is correct.

Upgrade the Cassandra version of a single node only. If that node is
behaving as expected (i.e. is in an Up/Normal state and no errors in the
logs), then upgrade the Cassandra version for each node one at a time. Be
sure to check that each node is running as expected. Once the Cassandra
version is upgraded on all nodes in the cluster and all nodes are in a
healthy state, then you upgrade the SSTables.

As Manish has pointed out, you want to be able to rollback the Cassandra
version easily if something goes wrong with the software upgrade.

If you have the disk space, I recommend taking a snapshot of the SSTables
on all nodes first before upgrading the Cassandra software version. The
snapshots are a safe guard incase there is a major problem after upgrading
SSTables.

Regards,


On Wed, 4 Mar 2020 at 15:08, manish khandelwal 
wrote:

> Should upgradesstables not be run after every node is upgraded? If we need
> to rollback then  we will not be able to downgrade sstables to older
> version.
>
> Regards
> Manish
>
> On Tue, Mar 3, 2020 at 11:26 PM Hossein Ghiyasi Mehr <
> ghiyasim...@gmail.com> wrote:
>
>> It's more safe to upgrade one node before upgrading another node to avoid
>> down time.
>> After upgrading binary and package, run upgradesstables on candidate node
>> then do it on all cluster nodes one by one.
>> *---*
>> *VafaTech <http://www.vafatech.com> : A Total Solution for Data Gathering
>> & Analysis*
>> *---*
>>
>>
>> On Thu, Feb 13, 2020 at 9:27 PM Sergio  wrote:
>>
>>>
>>>- Verify that nodetool upgradesstables has completed successfully on
>>>all nodes from any previous upgrade
>>>- Turn off repairs and any other streaming operations (add/remove
>>>nodes)
>>>- Nodetool drain on the node that needs to be stopped (seeds first,
>>>preferably)
>>>- Stop an un-upgraded node (seeds first, preferably)
>>>- Install new binaries and configs on the down node
>>>- Restart that node and make sure it comes up clean (it will
>>>function normally in the cluster – even with mixed versions)
>>>- nodetool statusbinary to verify if it is up and running
>>>- Repeat for all nodes
>>>- Once the binary upgrade has been performed in all the nodes: Run
>>>upgradesstables on each node (as many at a time as your load will allow).
>>>Minor upgrades usually don’t require this step (only if the sstable 
>>> format
>>>has changed), but it is good to check.
>>>- NOTE: in most cases applications can keep running and will not
>>>notice much impact – unless the cluster is overloaded and a single node
>>>down causes impact.
>>>
>>>
>>>
>>>I added 2 points to the list to clarify.
>>>
>>>Should we add this in a FAQ in the cassandra doc or in the awesome
>>>cassandra https://cassandra.link/awesome/
>>>
>>>Thanks,
>>>
>>>Sergio
>>>
>>>
>>> Il giorno mer 12 feb 2020 alle ore 10:58 Durity, Sean R <
>>> sean_r_dur...@homedepot.com> ha scritto:
>>>
>>>> Check the readme.txt for any upgrade notes, but the basic procedure is
>>>> to:
>>>>
>>>>- Verify that nodetool upgradesstables has completed successfully
>>>>on all nodes from any previous upgrade
>>>>- Turn off repairs and any other streaming operations (add/remove
>>>>nodes)
>>>>- Stop an un-upgraded node (seeds first, preferably)
>>>>- Install new binaries and configs on the down node
>>>>- Restart that node and make sure it comes up clean (it will
>>>>function normally in the cluster – even with mixed versions)
>>>>- Repeat for all nodes
>>>>- Run upgradesstables on each node (as many at a time as your load
>>>>will allow). Minor upgrades usually don’t require this step (only if the
>>>>sstable format has changed), but it is good to check.
>>>>- NOTE: in most cases applications can keep running and will not
>>>>notice much impact – unless the cluster is overloaded and a single node
>>>>down causes impact.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Sean Durity – Staff Systems Engineer, Cassandra
>>>>
>>>>
>>>>
>>&

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[manish khandelwal]

Should upgradesstables not be run after every node is upgraded? If we need
to rollback then  we will not be able to downgrade sstables to older
version.

Regards
Manish

On Tue, Mar 3, 2020 at 11:26 PM Hossein Ghiyasi Mehr 
wrote:

> It's more safe to upgrade one node before upgrading another node to avoid
> down time.
> After upgrading binary and package, run upgradesstables on candidate node
> then do it on all cluster nodes one by one.
> *---*
> *VafaTech <http://www.vafatech.com> : A Total Solution for Data Gathering
> & Analysis*
> *---*
>
>
> On Thu, Feb 13, 2020 at 9:27 PM Sergio  wrote:
>
>>
>>- Verify that nodetool upgradesstables has completed successfully on
>>all nodes from any previous upgrade
>>- Turn off repairs and any other streaming operations (add/remove
>>nodes)
>>- Nodetool drain on the node that needs to be stopped (seeds first,
>>preferably)
>>- Stop an un-upgraded node (seeds first, preferably)
>>- Install new binaries and configs on the down node
>>- Restart that node and make sure it comes up clean (it will function
>>normally in the cluster – even with mixed versions)
>>- nodetool statusbinary to verify if it is up and running
>>- Repeat for all nodes
>>- Once the binary upgrade has been performed in all the nodes: Run
>>upgradesstables on each node (as many at a time as your load will allow).
>>Minor upgrades usually don’t require this step (only if the sstable format
>>has changed), but it is good to check.
>>- NOTE: in most cases applications can keep running and will not
>>notice much impact – unless the cluster is overloaded and a single node
>>down causes impact.
>>
>>
>>
>>I added 2 points to the list to clarify.
>>
>>Should we add this in a FAQ in the cassandra doc or in the awesome
>>cassandra https://cassandra.link/awesome/
>>
>>Thanks,
>>
>>Sergio
>>
>>
>> Il giorno mer 12 feb 2020 alle ore 10:58 Durity, Sean R <
>> sean_r_dur...@homedepot.com> ha scritto:
>>
>>> Check the readme.txt for any upgrade notes, but the basic procedure is
>>> to:
>>>
>>>- Verify that nodetool upgradesstables has completed successfully on
>>>all nodes from any previous upgrade
>>>- Turn off repairs and any other streaming operations (add/remove
>>>nodes)
>>>- Stop an un-upgraded node (seeds first, preferably)
>>>- Install new binaries and configs on the down node
>>>- Restart that node and make sure it comes up clean (it will
>>>function normally in the cluster – even with mixed versions)
>>>- Repeat for all nodes
>>>- Run upgradesstables on each node (as many at a time as your load
>>>will allow). Minor upgrades usually don’t require this step (only if the
>>>sstable format has changed), but it is good to check.
>>>    - NOTE: in most cases applications can keep running and will not
>>>notice much impact – unless the cluster is overloaded and a single node
>>>down causes impact.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Sean Durity – Staff Systems Engineer, Cassandra
>>>
>>>
>>>
>>> *From:* Sergio 
>>> *Sent:* Wednesday, February 12, 2020 11:36 AM
>>> *To:* user@cassandra.apache.org
>>> *Subject:* [EXTERNAL] Cassandra 3.11.X upgrades
>>>
>>>
>>>
>>> Hi guys!
>>>
>>> How do you usually upgrade your cluster for minor version upgrades?
>>>
>>> I tried to add a node with 3.11.5 version to a test cluster with 3.11.4
>>> nodes.
>>>
>>> Is there any restriction?
>>>
>>> Best,
>>>
>>> Sergio
>>>
>>> --
>>>
>>> The information in this Internet Email is confidential and may be
>>> legally privileged. It is intended solely for the addressee. Access to this
>>> Email by anyone else is unauthorized. If you are not the intended
>>> recipient, any disclosure, copying, distribution or any action taken or
>>> omitted to be taken in reliance on it, is prohibited and may be unlawful.
>>> When addressed to our clients any opinions or advice contained in this
>>> Email are subject to the terms and conditions expressed in any applicable
>>> governing The Home Depot terms of business or client engagement letter. The
>>> Home Depot disclaims all responsibility and liability for the accuracy and
>>> content of this attachment and for any damages or losses arising from any
>>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
>>> items of a destructive nature, which may be contained in this attachment
>>> and shall not be liable for direct, indirect, consequential or special
>>> damages in connection with this e-mail message or its attachment.
>>>
>>

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Hossein Ghiyasi Mehr]

It's more safe to upgrade one node before upgrading another node to avoid
down time.
After upgrading binary and package, run upgradesstables on candidate node
then do it on all cluster nodes one by one.
*---*
*VafaTech <http://www.vafatech.com> : A Total Solution for Data Gathering &
Analysis*
*---*


On Thu, Feb 13, 2020 at 9:27 PM Sergio  wrote:

>
>- Verify that nodetool upgradesstables has completed successfully on
>all nodes from any previous upgrade
>- Turn off repairs and any other streaming operations (add/remove
>nodes)
>- Nodetool drain on the node that needs to be stopped (seeds first,
>preferably)
>- Stop an un-upgraded node (seeds first, preferably)
>- Install new binaries and configs on the down node
>- Restart that node and make sure it comes up clean (it will function
>normally in the cluster – even with mixed versions)
>- nodetool statusbinary to verify if it is up and running
>- Repeat for all nodes
>- Once the binary upgrade has been performed in all the nodes: Run
>upgradesstables on each node (as many at a time as your load will allow).
>Minor upgrades usually don’t require this step (only if the sstable format
>has changed), but it is good to check.
>- NOTE: in most cases applications can keep running and will not
>notice much impact – unless the cluster is overloaded and a single node
>down causes impact.
>
>
>
>I added 2 points to the list to clarify.
>
>Should we add this in a FAQ in the cassandra doc or in the awesome
>cassandra https://cassandra.link/awesome/
>
>Thanks,
>
>Sergio
>
>
> Il giorno mer 12 feb 2020 alle ore 10:58 Durity, Sean R <
> sean_r_dur...@homedepot.com> ha scritto:
>
>> Check the readme.txt for any upgrade notes, but the basic procedure is to:
>>
>>- Verify that nodetool upgradesstables has completed successfully on
>>all nodes from any previous upgrade
>>- Turn off repairs and any other streaming operations (add/remove
>>nodes)
>>- Stop an un-upgraded node (seeds first, preferably)
>>- Install new binaries and configs on the down node
>>- Restart that node and make sure it comes up clean (it will function
>>normally in the cluster – even with mixed versions)
>>- Repeat for all nodes
>>- Run upgradesstables on each node (as many at a time as your load
>>will allow). Minor upgrades usually don’t require this step (only if the
>>sstable format has changed), but it is good to check.
>>- NOTE: in most cases applications can keep running and will not
>>notice much impact – unless the cluster is overloaded and a single node
>>down causes impact.
>>
>>
>>
>>
>>
>>
>>
>> Sean Durity – Staff Systems Engineer, Cassandra
>>
>>
>>
>> *From:* Sergio 
>> *Sent:* Wednesday, February 12, 2020 11:36 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Cassandra 3.11.X upgrades
>>
>>
>>
>> Hi guys!
>>
>> How do you usually upgrade your cluster for minor version upgrades?
>>
>> I tried to add a node with 3.11.5 version to a test cluster with 3.11.4
>> nodes.
>>
>> Is there any restriction?
>>
>> Best,
>>
>> Sergio
>>
>> --
>>
>> The information in this Internet Email is confidential and may be legally
>> privileged. It is intended solely for the addressee. Access to this Email
>> by anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken or omitted to be
>> taken in reliance on it, is prohibited and may be unlawful. When addressed
>> to our clients any opinions or advice contained in this Email are subject
>> to the terms and conditions expressed in any applicable governing The Home
>> Depot terms of business or client engagement letter. The Home Depot
>> disclaims all responsibility and liability for the accuracy and content of
>> this attachment and for any damages or losses arising from any
>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
>> items of a destructive nature, which may be contained in this attachment
>> and shall not be liable for direct, indirect, consequential or special
>> damages in connection with this e-mail message or its attachment.
>>
>

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Sergio]

   - Verify that nodetool upgradesstables has completed successfully on all
   nodes from any previous upgrade
   - Turn off repairs and any other streaming operations (add/remove nodes)
   - Nodetool drain on the node that needs to be stopped (seeds first,
   preferably)
   - Stop an un-upgraded node (seeds first, preferably)
   - Install new binaries and configs on the down node
   - Restart that node and make sure it comes up clean (it will function
   normally in the cluster – even with mixed versions)
   - nodetool statusbinary to verify if it is up and running
   - Repeat for all nodes
   - Once the binary upgrade has been performed in all the nodes: Run
   upgradesstables on each node (as many at a time as your load will allow).
   Minor upgrades usually don’t require this step (only if the sstable format
   has changed), but it is good to check.
   - NOTE: in most cases applications can keep running and will not notice
   much impact – unless the cluster is overloaded and a single node down
   causes impact.



   I added 2 points to the list to clarify.

   Should we add this in a FAQ in the cassandra doc or in the awesome
   cassandra https://cassandra.link/awesome/

   Thanks,

   Sergio


Il giorno mer 12 feb 2020 alle ore 10:58 Durity, Sean R <
sean_r_dur...@homedepot.com> ha scritto:

> Check the readme.txt for any upgrade notes, but the basic procedure is to:
>
>- Verify that nodetool upgradesstables has completed successfully on
>all nodes from any previous upgrade
>- Turn off repairs and any other streaming operations (add/remove
>nodes)
>- Stop an un-upgraded node (seeds first, preferably)
>- Install new binaries and configs on the down node
>- Restart that node and make sure it comes up clean (it will function
>normally in the cluster – even with mixed versions)
>- Repeat for all nodes
>- Run upgradesstables on each node (as many at a time as your load
>will allow). Minor upgrades usually don’t require this step (only if the
>sstable format has changed), but it is good to check.
>- NOTE: in most cases applications can keep running and will not
>notice much impact – unless the cluster is overloaded and a single node
>down causes impact.
>
>
>
>
>
>
>
> Sean Durity – Staff Systems Engineer, Cassandra
>
>
>
> *From:* Sergio 
> *Sent:* Wednesday, February 12, 2020 11:36 AM
> *To:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Cassandra 3.11.X upgrades
>
>
>
> Hi guys!
>
> How do you usually upgrade your cluster for minor version upgrades?
>
> I tried to add a node with 3.11.5 version to a test cluster with 3.11.4
> nodes.
>
> Is there any restriction?
>
> Best,
>
> Sergio
>
> --
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>

RE: [EXTERNAL] Cassandra 3.11.X upgrades

[Durity, Sean R]

+1 on nodetool drain. I added that to our upgrade automation and it really 
helps with post-upgrade start-up time.

Sean Durity

From: Erick Ramirez 
Sent: Wednesday, February 12, 2020 10:29 PM
To: user@cassandra.apache.org
Subject: Re: [EXTERNAL] Cassandra 3.11.X upgrades

Yes to the steps. The only thing I would add is to run a nodetool drain before 
shutting C* down so all mutations are flushed to SSTables and there won't be 
any commit logs to replay on startup.

Also, the usual "backup your cluster and configuration files" boilerplate 
applies. 



The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any damages or losses arising from any inaccuracies, errors, viruses, 
e.g., worms, trojan horses, etc., or other items of a destructive nature, which 
may be contained in this attachment and shall not be liable for direct, 
indirect, consequential or special damages in connection with this e-mail 
message or its attachment.

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Erick Ramirez]

Yes to the steps. The only thing I would add is to run a nodetool drain
before shutting C* down so all mutations are flushed to SSTables and there
won't be any commit logs to replay on startup.

Also, the usual "backup your cluster and configuration files" boilerplate
applies. 

>

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Sergio]

Should I follow the steps above right?
Thanks Erick!

On Wed, Feb 12, 2020, 6:58 PM Erick Ramirez 
wrote:

> In case you have an hybrid situation with 3.11.3 , 3.11.4 and 3.11.5 that
>> it is working and it is in production what do you recommend?
>
>
> You shouldn't end up in this mixed-version situation at all. I would
> highly recommend you upgrade all the nodes to 3.11.5 or whatever the latest
> version is installed on the nodes. Mixed-versions isn't a tested or
> supported scenario and the cluster's behaviour can be unpredictable. The
> behaviour might not be catastrophic but you don't want to be the one who
> discovers some exotic bug that arises out of that configuration. Cheers!
>
>>

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Erick Ramirez]

>
> In case you have an hybrid situation with 3.11.3 , 3.11.4 and 3.11.5 that
> it is working and it is in production what do you recommend?


You shouldn't end up in this mixed-version situation at all. I would highly
recommend you upgrade all the nodes to 3.11.5 or whatever the latest
version is installed on the nodes. Mixed-versions isn't a tested or
supported scenario and the cluster's behaviour can be unpredictable. The
behaviour might not be catastrophic but you don't want to be the one who
discovers some exotic bug that arises out of that configuration. Cheers!

>

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Sergio]

Thanks everyone!

In case you have an hybrid situation with 3.11.3 , 3.11.4 and 3.11.5 that
it is working and it is in production what do you recommend?



On Wed, Feb 12, 2020, 5:55 PM Erick Ramirez 
wrote:

> So unless the sstable format has not been changed I can avoid to do that.
>
>
> Just to reinforce what Jon and Sean already said, the above assumption is
> dangerous. It is always best to follow the recommended upgrade procedure
> and mixed-versions is never a good idea unless you've received instructions
> from a qualified source to address a specific issue. But as Jon said, we
> wouldn't be on this mailing list otherwise. 
>
> Erick Ramirez  |  Developer Relations
>
> erick.rami...@datastax.com | datastax.com 
> 
>  
>  
>
> 
>
>>

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Erick Ramirez]

>
> So unless the sstable format has not been changed I can avoid to do that.


Just to reinforce what Jon and Sean already said, the above assumption is
dangerous. It is always best to follow the recommended upgrade procedure
and mixed-versions is never a good idea unless you've received instructions
from a qualified source to address a specific issue. But as Jon said, we
wouldn't be on this mailing list otherwise. 

Erick Ramirez  |  Developer Relations

erick.rami...@datastax.com | datastax.com 

 
 



>

RE: [EXTERNAL] Cassandra 3.11.X upgrades

[Durity, Sean R]

Ah - I should have looked it up! Thank you for fixing my mistake.

Sean Durity

-Original Message-
From: Michael Shuler 
Sent: Wednesday, February 12, 2020 3:17 PM
To: user@cassandra.apache.org
Subject: Re: [EXTERNAL] Cassandra 3.11.X upgrades

On 2/12/20 12:58 PM, Durity, Sean R wrote:
> Check the readme.txt for any upgrade notes

Just a quick correction:

NEWS.txt (upgrade (and other important) notes)
CHANGES.txt (changelog with JIRAs)

This is why we list links to these two files in the release announcements.

--
Kind regards,
Michael

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org




The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any damages or losses arising from any inaccuracies, errors, viruses, 
e.g., worms, trojan horses, etc., or other items of a destructive nature, which 
may be contained in this attachment and shall not be liable for direct, 
indirect, consequential or special damages in connection with this e-mail 
message or its attachment.

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Reid Pinchback]

Hi Sergio,

We have a production cluster with vnodes=4 that is a bit larger than that, so 
yes it is possible to do so.  That said, we aren’t wedded to vnodes=4 and are 
paying attention to discussions happening around the 4.0 work and mulling the 
possibility of shifting to 16.

Note though, we didn’t just pick 4 based on blind faith.  There was work done 
to evaluate the state of our token distribution before and after.  It’s pretty 
much true of all the settings we have for C* (or anything else for that 
matter).  We started from a lot of helpful articles and docs and mail threads 
and JIRA issues, but then we evaluated to see what we thought of the results.  
It was the only way for us to start building up some understanding of the 
details.

From: Sergio 
Reply-To: "user@cassandra.apache.org" 
Date: Wednesday, February 12, 2020 at 2:50 PM
To: "user@cassandra.apache.org" 
Subject: Re: [EXTERNAL] Cassandra 3.11.X upgrades

Message from External Sender
Thanks, everyone! @Jon 
https://lists.apache.org/thread.html/rd18814bfba487824ca95a58191f4dcdb86f15c9bb66cf2bcc29ddf0b%40%3Cuser.cassandra.apache.org%3E<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.apache.org_thread.html_rd18814bfba487824ca95a58191f4dcdb86f15c9bb66cf2bcc29ddf0b-2540-253Cuser.cassandra.apache.org-253E=DwMFaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=GkUeet_u9KFI2OlzYvn34AhugRwTmQiTA-Rc9YYBFx8=Mwa2C1h8fwl0Qxfi75pISpAUVX18lhGVyHAjO8hwk3o=>
I have a side response to something that looks to be controversial with the 
response from Anthony.
So is it safe to go to production in a 1TB cluster with vnodes = 4?
Do we need to follow these steps 
https://thelastpickle.com/blog/2019/02/21/set-up-a-cluster-with-even-token-distribution.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__thelastpickle.com_blog_2019_02_21_set-2Dup-2Da-2Dcluster-2Dwith-2Deven-2Dtoken-2Ddistribution.html=DwMFaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=GkUeet_u9KFI2OlzYvn34AhugRwTmQiTA-Rc9YYBFx8=jVKn-7nVIP3_zUh6LjYMG-X1KyUZPcIvV-m0Cy82Ttk=>?
 because from Anthony response what I got that this is just an example and 
vnodes = 4 it is not ready for production.
https://lists.apache.org/thread.html/r21cd99fa269076d186a82a8b466eb925681373302dd7aa6bb26e5bde%40%3Cuser.cassandra.apache.org%3E<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.apache.org_thread.html_r21cd99fa269076d186a82a8b466eb925681373302dd7aa6bb26e5bde-2540-253Cuser.cassandra.apache.org-253E=DwMFaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=GkUeet_u9KFI2OlzYvn34AhugRwTmQiTA-Rc9YYBFx8=r5CPYag3chuqUfol7RR8NKj31MC27P-gONPzaKhkeJ4=>

Best,

Sergio


Il giorno mer 12 feb 2020 alle ore 11:42 Durity, Sean R 
mailto:sean_r_dur...@homedepot.com>> ha scritto:
>>A while ago, on my first cluster

Understatement used so effectively. Jon is a master.



On Wed, Feb 12, 2020 at 11:02 AM Sergio 
mailto:lapostadiser...@gmail.com>> wrote:
Thanks for your reply!

So unless the sstable format has not been changed I can avoid to do that.

Correct?

Best,

Sergio

On Wed, Feb 12, 2020, 10:58 AM Durity, Sean R 
mailto:sean_r_dur...@homedepot.com>> wrote:
Check the readme.txt for any upgrade notes, but the basic procedure is to:

  *   Verify that nodetool upgradesstables has completed successfully on all 
nodes from any previous upgrade
  *   Turn off repairs and any other streaming operations (add/remove nodes)
  *   Stop an un-upgraded node (seeds first, preferably)
  *   Install new binaries and configs on the down node
  *   Restart that node and make sure it comes up clean (it will function 
normally in the cluster – even with mixed versions)
  *   Repeat for all nodes
  *   Run upgradesstables on each node (as many at a time as your load will 
allow). Minor upgrades usually don’t require this step (only if the sstable 
format has changed), but it is good to check.
  *   NOTE: in most cases applications can keep running and will not notice 
much impact – unless the cluster is overloaded and a single node down causes 
impact.



Sean Durity – Staff Systems Engineer, Cassandra

From: Sergio mailto:lapostadiser...@gmail.com>>
Sent: Wednesday, February 12, 2020 11:36 AM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: [EXTERNAL] Cassandra 3.11.X upgrades

Hi guys!

How do you usually upgrade your cluster for minor version upgrades?

I tried to add a node with 3.11.5 version to a test cluster with 3.11.4 nodes.

Is there any restriction?

Best,

Sergio



The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitt

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Michael Shuler]

On 2/12/20 12:58 PM, Durity, Sean R wrote:

Check the readme.txt for any upgrade notes


Just a quick correction:

NEWS.txt (upgrade (and other important) notes)
CHANGES.txt (changelog with JIRAs)

This is why we list links to these two files in the release announcements.

--
Kind regards,
Michael

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Sergio]

Thanks, everyone! @Jon
https://lists.apache.org/thread.html/rd18814bfba487824ca95a58191f4dcdb86f15c9bb66cf2bcc29ddf0b%40%3Cuser.cassandra.apache.org%3E

I have a side response to something that looks to be controversial with the
response from Anthony.
So is it safe to go to production in a 1TB cluster with vnodes = 4?
Do we need to follow these steps
https://thelastpickle.com/blog/2019/02/21/set-up-a-cluster-with-even-token-distribution.html?
because
from Anthony response what I got that this is just an example and vnodes =
4 it is not ready for production.
https://lists.apache.org/thread.html/r21cd99fa269076d186a82a8b466eb925681373302dd7aa6bb26e5bde%40%3Cuser.cassandra.apache.org%3E

Best,

Sergio



Il giorno mer 12 feb 2020 alle ore 11:42 Durity, Sean R <
sean_r_dur...@homedepot.com> ha scritto:

> >>A while ago, on my first cluster
>
>
>
> Understatement used so effectively. Jon is a master.
>
>
>
>
>
>
>
> On Wed, Feb 12, 2020 at 11:02 AM Sergio  wrote:
>
> Thanks for your reply!
>
>
>
> So unless the sstable format has not been changed I can avoid to do that.
>
>
>
> Correct?
>
>
>
> Best,
>
>
>
> Sergio
>
>
>
> On Wed, Feb 12, 2020, 10:58 AM Durity, Sean R 
> wrote:
>
> Check the readme.txt for any upgrade notes, but the basic procedure is to:
>
>- Verify that nodetool upgradesstables has completed successfully on
>all nodes from any previous upgrade
>- Turn off repairs and any other streaming operations (add/remove
>nodes)
>- Stop an un-upgraded node (seeds first, preferably)
>- Install new binaries and configs on the down node
>- Restart that node and make sure it comes up clean (it will function
>normally in the cluster – even with mixed versions)
>- Repeat for all nodes
>- Run upgradesstables on each node (as many at a time as your load
>will allow). Minor upgrades usually don’t require this step (only if the
>sstable format has changed), but it is good to check.
>- NOTE: in most cases applications can keep running and will not
>notice much impact – unless the cluster is overloaded and a single node
>down causes impact.
>
>
>
>
>
>
>
> Sean Durity – Staff Systems Engineer, Cassandra
>
>
>
> *From:* Sergio 
> *Sent:* Wednesday, February 12, 2020 11:36 AM
> *To:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Cassandra 3.11.X upgrades
>
>
>
> Hi guys!
>
> How do you usually upgrade your cluster for minor version upgrades?
>
> I tried to add a node with 3.11.5 version to a test cluster with 3.11.4
> nodes.
>
> Is there any restriction?
>
> Best,
>
> Sergio
>
>
> --
>
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>
>
> --
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>

RE: [EXTERNAL] Cassandra 3.11.X upgrades

[Durity, Sean R]

>>A while ago, on my first cluster

Understatement used so effectively. Jon is a master.



On Wed, Feb 12, 2020 at 11:02 AM Sergio 
mailto:lapostadiser...@gmail.com>> wrote:
Thanks for your reply!

So unless the sstable format has not been changed I can avoid to do that.

Correct?

Best,

Sergio

On Wed, Feb 12, 2020, 10:58 AM Durity, Sean R 
mailto:sean_r_dur...@homedepot.com>> wrote:
Check the readme.txt for any upgrade notes, but the basic procedure is to:

  *   Verify that nodetool upgradesstables has completed successfully on all 
nodes from any previous upgrade
  *   Turn off repairs and any other streaming operations (add/remove nodes)
  *   Stop an un-upgraded node (seeds first, preferably)
  *   Install new binaries and configs on the down node
  *   Restart that node and make sure it comes up clean (it will function 
normally in the cluster – even with mixed versions)
  *   Repeat for all nodes
  *   Run upgradesstables on each node (as many at a time as your load will 
allow). Minor upgrades usually don’t require this step (only if the sstable 
format has changed), but it is good to check.
  *   NOTE: in most cases applications can keep running and will not notice 
much impact – unless the cluster is overloaded and a single node down causes 
impact.



Sean Durity – Staff Systems Engineer, Cassandra

From: Sergio mailto:lapostadiser...@gmail.com>>
Sent: Wednesday, February 12, 2020 11:36 AM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: [EXTERNAL] Cassandra 3.11.X upgrades

Hi guys!

How do you usually upgrade your cluster for minor version upgrades?

I tried to add a node with 3.11.5 version to a test cluster with 3.11.4 nodes.

Is there any restriction?

Best,

Sergio



The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any damages or losses arising from any inaccuracies, errors, viruses, 
e.g., worms, trojan horses, etc., or other items of a destructive nature, which 
may be contained in this attachment and shall not be liable for direct, 
indirect, consequential or special damages in connection with this e-mail 
message or its attachment.



The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any damages or losses arising from any inaccuracies, errors, viruses, 
e.g., worms, trojan horses, etc., or other items of a destructive nature, which 
may be contained in this attachment and shall not be liable for direct, 
indirect, consequential or special damages in connection with this e-mail 
message or its attachment.

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Jon Haddad]

A while ago, on my first cluster, I decided to do an upgrade by adding
nodes running 1.2 to an existing cluster running version 1.1.  This was a
bad decision, and at that point I decided to always play it safe and always
stick to a single version, and never bootstrap in a node running different
version to a cluster.

I would do this even when it's a trivial upgrade, or a single commit is
different only touching a few lines of code.

Why?

I do this because in my opinion it's better to have a single way of doing
things.  Getting in the routine of doing minor upgrades by following Sean's
upgrade checklist will always work.  It's the supported, correct way of
doing things, and if you decide to always follow the same checklist, the
possibility of doing things wrong when there is an
incompatibility decreases.  Think of it as practice.

The only time you should even consider mixed version bootstrap upgrades is
if you know enough to not have to ask the list.  Even then, I go back to my
previous point about practice.  No point in practicing the less safe
version of things.

My 2 cents.

Jon



On Wed, Feb 12, 2020 at 11:02 AM Sergio  wrote:

> Thanks for your reply!
>
> So unless the sstable format has not been changed I can avoid to do that.
>
> Correct?
>
> Best,
>
> Sergio
>
> On Wed, Feb 12, 2020, 10:58 AM Durity, Sean R 
> wrote:
>
>> Check the readme.txt for any upgrade notes, but the basic procedure is to:
>>
>>- Verify that nodetool upgradesstables has completed successfully on
>>all nodes from any previous upgrade
>>- Turn off repairs and any other streaming operations (add/remove
>>nodes)
>>- Stop an un-upgraded node (seeds first, preferably)
>>- Install new binaries and configs on the down node
>>- Restart that node and make sure it comes up clean (it will function
>>normally in the cluster – even with mixed versions)
>>- Repeat for all nodes
>>- Run upgradesstables on each node (as many at a time as your load
>>will allow). Minor upgrades usually don’t require this step (only if the
>>sstable format has changed), but it is good to check.
>>- NOTE: in most cases applications can keep running and will not
>>notice much impact – unless the cluster is overloaded and a single node
>>down causes impact.
>>
>>
>>
>>
>>
>>
>>
>> Sean Durity – Staff Systems Engineer, Cassandra
>>
>>
>>
>> *From:* Sergio 
>> *Sent:* Wednesday, February 12, 2020 11:36 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Cassandra 3.11.X upgrades
>>
>>
>>
>> Hi guys!
>>
>> How do you usually upgrade your cluster for minor version upgrades?
>>
>> I tried to add a node with 3.11.5 version to a test cluster with 3.11.4
>> nodes.
>>
>> Is there any restriction?
>>
>> Best,
>>
>> Sergio
>>
>> --
>>
>> The information in this Internet Email is confidential and may be legally
>> privileged. It is intended solely for the addressee. Access to this Email
>> by anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken or omitted to be
>> taken in reliance on it, is prohibited and may be unlawful. When addressed
>> to our clients any opinions or advice contained in this Email are subject
>> to the terms and conditions expressed in any applicable governing The Home
>> Depot terms of business or client engagement letter. The Home Depot
>> disclaims all responsibility and liability for the accuracy and content of
>> this attachment and for any damages or losses arising from any
>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
>> items of a destructive nature, which may be contained in this attachment
>> and shall not be liable for direct, indirect, consequential or special
>> damages in connection with this e-mail message or its attachment.
>>
>

Re: [EXTERNAL] Cassandra 3.11.X upgrades

[Sergio]

Thanks for your reply!

So unless the sstable format has not been changed I can avoid to do that.

Correct?

Best,

Sergio

On Wed, Feb 12, 2020, 10:58 AM Durity, Sean R 
wrote:

> Check the readme.txt for any upgrade notes, but the basic procedure is to:
>
>- Verify that nodetool upgradesstables has completed successfully on
>all nodes from any previous upgrade
>- Turn off repairs and any other streaming operations (add/remove
>nodes)
>- Stop an un-upgraded node (seeds first, preferably)
>- Install new binaries and configs on the down node
>- Restart that node and make sure it comes up clean (it will function
>normally in the cluster – even with mixed versions)
>- Repeat for all nodes
>- Run upgradesstables on each node (as many at a time as your load
>will allow). Minor upgrades usually don’t require this step (only if the
>sstable format has changed), but it is good to check.
>- NOTE: in most cases applications can keep running and will not
>notice much impact – unless the cluster is overloaded and a single node
>down causes impact.
>
>
>
>
>
>
>
> Sean Durity – Staff Systems Engineer, Cassandra
>
>
>
> *From:* Sergio 
> *Sent:* Wednesday, February 12, 2020 11:36 AM
> *To:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Cassandra 3.11.X upgrades
>
>
>
> Hi guys!
>
> How do you usually upgrade your cluster for minor version upgrades?
>
> I tried to add a node with 3.11.5 version to a test cluster with 3.11.4
> nodes.
>
> Is there any restriction?
>
> Best,
>
> Sergio
>
> --
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>

RE: [EXTERNAL] Cassandra 3.11.X upgrades

[Durity, Sean R]

Check the readme.txt for any upgrade notes, but the basic procedure is to:

  *   Verify that nodetool upgradesstables has completed successfully on all 
nodes from any previous upgrade
  *   Turn off repairs and any other streaming operations (add/remove nodes)
  *   Stop an un-upgraded node (seeds first, preferably)
  *   Install new binaries and configs on the down node
  *   Restart that node and make sure it comes up clean (it will function 
normally in the cluster – even with mixed versions)
  *   Repeat for all nodes
  *   Run upgradesstables on each node (as many at a time as your load will 
allow). Minor upgrades usually don’t require this step (only if the sstable 
format has changed), but it is good to check.
  *   NOTE: in most cases applications can keep running and will not notice 
much impact – unless the cluster is overloaded and a single node down causes 
impact.



Sean Durity – Staff Systems Engineer, Cassandra

From: Sergio 
Sent: Wednesday, February 12, 2020 11:36 AM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Cassandra 3.11.X upgrades

Hi guys!

How do you usually upgrade your cluster for minor version upgrades?

I tried to add a node with 3.11.5 version to a test cluster with 3.11.4 nodes.

Is there any restriction?

Best,

Sergio



The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any damages or losses arising from any inaccuracies, errors, viruses, 
e.g., worms, trojan horses, etc., or other items of a destructive nature, which 
may be contained in this attachment and shall not be liable for direct, 
indirect, consequential or special damages in connection with this e-mail 
message or its attachment.