I believe we're back up? On Tue, Nov 10, 2020 at 11:50 AM Stephen LaPorte <stephen.lapo...@gmail.com> wrote:
> Hi all, > > Toolforge is temporarily in read-only mode for planned maintenance. It may > take an hour or longer before Montage is back running. > > I can send an email when we're back up, and please see below for > additional info. > > Thanks, > Stephen > > ---------- Forwarded message --------- > From: Brooke Storm <bst...@wikimedia.org> > Date: Tue, Nov 10, 2020 at 7:52 AM > Subject: Re: [Cloud] [Cloud-announce] 2020-11-10 ToolsDB (User databases > in Toolforge) read-only downtime > To: <cloud-annou...@lists.wikimedia.org> > > > This will be happening in around 10 minutes. ToolsDB will be read-only > until we can get a consistent dump to rebuild replication. > > Brooke Storm > Staff SRE > Wikimedia Cloud Services > bst...@wikimedia.org > IRC: bstorm > > On Nov 6, 2020, at 12:12 PM, Brooke Storm <bst...@wikimedia.org> wrote: > > The ToolsDB service suffered a breakage in replication on 2020-10-27. WMCS > has tried to restore replication of data, but that has been unsuccessful so > far including doing a dump to rebuild replication without downtime. > At this point, we have a new server waiting to become the replica, but to > start the replication process, we need to set the database to read-only for > a full dump. This could easily take more than an hour. During that entire > time, the database will be read-only. > > We will begin at 1600 UTC and finish when it is done. The database is > quite large, but, with it in read-only mode, I hope the backup will not > take terribly long. > > Please see https://phabricator.wikimedia.org/T266587 for additional > information. > > Brooke Storm > Staff SRE > Wikimedia Cloud Services > bst...@wikimedia.org > IRC: bstorm > > > _______________________________________________ > Wikimedia Cloud Services announce mailing list > cloud-annou...@lists.wikimedia.org (formerly > labs-annou...@lists.wikimedia.org) > https://lists.wikimedia.org/mailman/listinfo/cloud-announce > > On Tue, Nov 10, 2020 at 11:21 AM Ciell Wikipedia < > ciell.wikipe...@gmail.com> wrote: > >> Hi, >> >> Thank you for reporting this: you are not the only one with this problem. >> I've alerted the developers of the tool already. >> >> Vriendelijke groet, >> Ciell >> >> >> Op di 10 nov. 2020 om 20:17 schreef Éder Porto <eder.po...@hotmail.com>: >> >>> Hello to all, >>> >>> Today, a few hours ago, the montage tool just stopped working, I can't >>> even login. Is this happening with someone else? Shoul a ticket on >>> Phabricator be open? A issue on gitHub? My error message is as folows: >>> >>> <ExceptionInfo [sqlalchemy.exc.OperationalError: >>> (_mysql_exceptions.OperationalError) (1290, 'The MariaDB server is running >>> with the --read-only option so it cannot execute this statement') [SQL: >>> u'UPDATE users SET last_active_date=%s WHERE users.id = %s'] >>> [parameters: (datetime.datetime(2020, 11, 10, 19, 13, 33, 773317), >>> 47501133L)]] (40 frames, last=Callpoint('defaulterrorhandler', 36, >>> 'MySQLdb.connections', >>> '/data/project/montage/www/python/venv/local/lib/python2.7/site-packages/MySQLdb/connections.py', >>> 65, ' raise errorclass, errorvalue'))> >>> >>> Thanks in advance for any comments, >>> Éder Porto | User:Ederporto >>> _______________________________________________ >>> Wiki Loves Monuments mailing list >>> WikiLovesMonuments@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >>> http://www.wikilovesmonuments.org >>> >> _______________________________________________ >> Wiki Loves Monuments mailing list >> WikiLovesMonuments@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments >> http://www.wikilovesmonuments.org >> > _______________________________________________ > Wiki Loves Monuments mailing list > WikiLovesMonuments@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments > http://www.wikilovesmonuments.org >
_______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org
