On Tuesday 2020-10-27 at 1600 UTC, ToolsDB, the user database service provided 
by Toolforge 
(https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases 
<https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases>) 
will see some service interruption from restarts and will be set to read-only 
mode during a scheduled failover. The failover must be done to protect data and 
minimize downtime while the hypervisor goes down for upgrades.

Tools may need to restart in order to reconnect during this time and nothing 
will be able to write to the database tables until failover is completed.

After failover, services should resume as normal, and we will announce when it 
is done. The process will take some time because we must copy over several 
databases by hand due to three temporarily unreplicated tables being repaired 
while things are in read-only mode and the four databases that are always 
unreplicated being copied 
(https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#ToolsDB_Backups_and_Replication
 
<https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#ToolsDB_Backups_and_Replication>).
 I expect at least half an hour in read-only mode, but things could take longer 
or shorter time depending on volume of data to copy and issues encountered 
during the process.

Details are on Phabricator at https://phabricator.wikimedia.org/T263679 
<https://phabricator.wikimedia.org/T263679>

Brooke Storm
Staff SRE
Wikimedia Cloud Services
[email protected] <mailto:[email protected]>
IRC: bstorm

Attachment: signature.asc
Description: Message signed with OpenPGP

_______________________________________________
Wikimedia Cloud Services announce mailing list
[email protected] (formerly [email protected])
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
_______________________________________________
Wikimedia Cloud Services mailing list
[email protected] (formerly [email protected])
https://lists.wikimedia.org/mailman/listinfo/cloud

Reply via email to