Most of tools should be back up now. Expected behavior for the next many hours (Expected ~12-14 hours based on last sync of tools data, but might be much longer)
Quoting from previous announcement [1]: * The tools NFS share (/data/project and /home) will be read-only for the > duration of the maintenance, so no new data or logs will get written to it. > * New jobs cannot be submitted for the whole maintenance window - this > means submitting jobs through cron or tools-mail will not function, > although tools-mail can continue to send emails. > * Current jobs might keep running, but won't get rescheduled if they die. > If they do not die and aren't writing to NFS they should be fine. [1]: https://lists.wikimedia.org/pipermail/labs-l/2016-October/004698.html On Mon, Nov 14, 2016 at 9:31 AM, Madhumitha Viswanathan < [email protected]> wrote: > All of tools is down currently - unexpected side effect of the ongoing > maintenance - we are working on it, I will update once it's back up. > Apologies for the inconvenience caused. > > On Mon, Nov 14, 2016 at 7:59 AM, Madhumitha Viswanathan < > [email protected]> wrote: > >> Hi all, >> >> This is starting in a few minutes. You can follow along progress on >> #wikimedia-labs or at T146154 <https://phabricator.wikimedia.org/T146154> >> . >> >> >> On Fri, Nov 11, 2016 at 10:16 AM, Madhumitha Viswanathan < >> [email protected]> wrote: >> >>> Hi all, >>> >>> This migration is scheduled to happened on Monday 11/14. We will start >>> at 08:00 PST/16:00 UTC. More details here - https://phabricator.wikimedi >>> a.org/T146154. >>> >>> On Tue, Nov 1, 2016 at 11:47 AM, Madhumitha Viswanathan < >>> [email protected]> wrote: >>> >>>> Hi all, >>>> >>>> We had previously announced a disruptive maintenance for tools[1] on >>>> 11/2(tomorrow), but we are going to have to postpone it tentatively to >>>> 11/14/2016. >>>> >>>> Apologies for the late notice, but the server in our secondary >>>> datacenter in Dallas that hosts data backups is having issues since >>>> yesterday[2], and we are currently still debugging and fixing the problems >>>> there. We've decided not to proceed with this migration until our backups >>>> there are online. >>>> >>>> As always, do let us know if there are any questions/concerns here or >>>> on #wikimedia-labs. >>>> >>>> [1]: https://lists.wikimedia.org/pipermail/labs-l/2016-Octob >>>> er/004698.html >>>> [2]: https://phabricator.wikimedia.org/T149567 >>>> >>> >>> >>> >>> -- >>> --Madhu :) >>> >> >> >> >> -- >> --Madhu :) >> > > > > -- > --Madhu :) > -- --Madhu :)
_______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
