We lost a KVM host at around 7:20 UTC. Because we use local storage for instances there are a number of them that are down. Toolforge suffered a few losses but it seems to have been few enough that GridEngine and Kubernetes users are unaffected at this time . The initial task is T187292 (with a list of instances), and an incident report will follow. We hope to recover all of the instances that are down but it will take time to sort through.
-- Chase Pettet chasemp on phabricator <https://phabricator.wikimedia.org/p/chasemp/> and IRC
_______________________________________________ Wikimedia Cloud Services announce mailing list cloud-annou...@lists.wikimedia.org (formerly labs-annou...@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
_______________________________________________ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud