Hi Paul,
You don't need to plan for or introduce an outage for a rolling upgrade,
which is the preferred route. It isn't advisable to take down an entire
DC to do upgrade.
You should aim to complete upgrading the entire cluster and finish a
full repair within the shortest gc_grace_seconds (default to 10 days) of
all tables. Failing to do that may cause data resurrections.
During the rolling upgrade, you should not run repair or any DDL query
(such as ALTER TABLE, TRUNCATE, etc.).
You don't need to do the rolling upgrade node by node. You can do it
rack by rack. Stopping all nodes in a single rack and upgrade them
concurrently is much faster. The number of nodes doesn't matter that
much to the time required to complete a rolling upgrade, it's the number
of DCs and racks matter.
Cheers,
Bowen
On 24/04/2024 16:16, Paul Chandler wrote:
Hi all,
We have some large clusters ( 1000+ nodes ), these are across multiple
datacenters.
When we perform upgrades we would normally upgrade a DC at a time during a
planned outage for one DC. This means that a cluster might be in a mixed mode
with multiple versions for a week or 2.
We have noticed that during our testing that upgrading to 4.1 causes a schema
mismatch due to the new tables added into the system keyspace.
Is this going to be an issue if this schema mismatch lasts for maybe several
weeks? I assume that running any DDL during that time would be a bad idea, is
there any other issues to look out for?
Thanks
Paul Chandler
