[Wikitech-l] Re: [Ops] codfw wikikube kubernetes cluster upgrade on 2023-02-21

[Alexandros Kosiaris]

Hello everyone,

The cluster was successfully re-initialized today, all services have been
re-pooled and are in service. The cluster is fully operational again and
can be used by deployers.

Regards,

On Wed, Feb 15, 2023 at 1:41 PM Janis Meybohm 
wrote:

> Hello everyone,
>
> TL;DR if you are not deploying services to the codfw wikikube kubernetes
> cluster, you can safely skip this.
>
> Long version:
>
> We will reinitialize the codfw wikikube kubernetes cluster with kubernetes
> version 1.23 on 2023-02-21 09:00-16:00 UTC [1] (the actual process is
> expected
> take a couple of hours within this window).
> The date was chosen for convenience as we will have depooled all
> active/active
> services from codfw for row B switch maintenance [2] anyways. As all
> traffic
> will be drained beforehand we expect no user visible impact. However, for
> the
> duration of the process, the kubernetes cluster will be unavailable to
> deployers and thus efforts to deploy to it will fail or worse, not have
> the
> expected outcomes.
> This is normal until SRE serviceops announces that the cluster is fully
> operational again.
>
> SRE serviceops will be deploying all services before marking the cluster
> as
> usable and pooling traffic back to it, so there will be no need for
> deployers to
> re-deploy their services (apart from those already informed).
>
> [1] https://phabricator.wikimedia.org/T329664
> [2] https://phabricator.wikimedia.org/T327991
>
> Regars,
> Janis Meybohm
>
> ___
> Ops mailing list -- o...@lists.wikimedia.org
> To unsubscribe send an email to ops-le...@lists.wikimedia.org
>


-- 
Alexandros Kosiaris
Principal Site Reliability Engineer
Wikimedia Foundation
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: March 2023 Datacenter Switchover

[Clément Goubert]

Dear Wikitechians,

I would like to remind you that the datacenter switchover will happen
on *Wednesday
March 1st* starting at *14:00 UTC.*

Please refer to the original email for any additional information. As
always, you can reach out to me directly or the SRE team in #wikimedia-sre
on IRC with any question, or through Phabricator.

Thank you,

On Tue, Feb 14, 2023 at 1:58 PM Clément Goubert 
wrote:

> Dear Wikitechians,
>
> On *Wednesday March 1st*, the SRE team will run a planned data center
> switchover, moving all wikis from our primary data center in Virginia to
> the secondary data center in Texas. This is an important periodic test of
> our tools and procedures, to ensure the wikis will continue to be available
> even in the event of major technical issues in our primary home. It also
> gives all our SRE and ops teams a chance to do maintenance and upgrades on
> systems in Virginia that normally run 24 hours a day.
>
> The switchover process requires a *brief read-only period for all
> Foundation-hosted wikis*, which will start at *14:00 UTC on Wednesday
> March 1st*, and will last for a few minutes while we execute the
> migration as efficiently as possible. All our public and private wikis will
> be continuously available for reading as usual, but no one will be able to
> save edits during the process. Users will see a notification of the
> upcoming maintenance, and anyone still editing will be asked to try again
> in a few minutes.
>
> CommRel has already begun notifying communities of the read-only window. A
> similar event will follow a few weeks later, when we move back to Virginia.
> This is currently scheduled for *Wednesday, April 26th*.
>
> If you like, you can follow along on the day in the public
> #wikimedia-operations channel on IRC (instructions for joining here
> ). To report any
> issues, you can reach us in #wikimedia-sre on IRC, or file a Phabricator
> ticket with the *datacenter-switchover* tag (pre-filled form here
> );
> we'll be monitoring closely for reports of trouble during and after the
> switchover. (If you're new to Phab, there's more information at
> Phabricator/Help.) The switchover and its preparation are tracked tracked
> in Phabricator Task T327920 
>
> On behalf of the SRE team, please excuse the disruption, and our thanks to
> everyone in a number of departments who've been involved in planning this
> work for the past weeks. Feel free to reply directly to me with any
> questions.
>
> Thank you,
>
> --
> Clément Goubert (they/them)
> Senior SRE
> Wikimedia Foundation
>


-- 
Clément Goubert (they/them)
Senior SRE
Wikimedia Foundation
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/