Best Regards, -kenny Kenny Paul, Technical Program Manager kp...@linuxfoundation.org 510.766.5945
> Begin forwarded message: > > From: nore...@statuspage.io > Subject: [The Linux Foundation status] Update : Outage at PDX datacenter > Date: July 13, 2017 at 2:56:52 PM PDT > To: '' <kp...@linuxfoundation.org> > > Outage at PDX datacenter <http://stspg.io/a162b3e5f> > Incident Report for The Linux Foundation > An Incident Update Has Been Posted > All systems are back online and functional. We are continuing to monitor the > situation and are working with our colocation provider to understand the root > cause of the outage and why so many of our systems were affected by it. > > A power outage occurred this morning which caused many of our non-redundant > and redundant systems to go offline. The order which these systems came back > online was important, as we lost access to core components of our > infrastructure (network, DNS, disk arrays, VPNs, 2FA, virtualization hosts, > etc). Because of the severity of the outage, restoring services took longer > than usual. > Jul 13, 21:56 UTC > PREVIOUS UPDATES > Update > We are continuing to troubleshoot the power issues at the data center by > working through our core services and bringing systems back online as quickly > as possible. Next update at 2:45pm. > Jul 13, 20:53 UTC > Update > An LF staff member is now onsite at the Portland data center to triage and > trouble shoot issues. > As this situation is complex and mostly out of our control, the next update > will be at 1:45pm. > Jul 13, 19:46 UTC > Update > We appear to have lost power to a number of out of band systems and servers > which is continuing to cause outages. We believe the power outage may have > affected core routing/switching in the datacenter as well as power to some of > our servers. We are investigating and will continue to identify additional > issues. > Jul 13, 18:45 UTC > Monitoring > Our colo provider has confirmed that there are power issues at the PDX DC3 > datacenter. They currently do not have an ETA for service restores or root > cause identified, but will update us as soon as they do. > > We appear to have many of our services back online at this time, and will be > monitoring the situation until we have confirmed things are all clear. > Jul 13, 18:40 UTC > Investigating > We are investigating what we believe to be an issue with network/switching > equipment in the PDX datacenter. Atmosera is currently "performing > maintenance on the power systems at DC3, our downtown Portland datacenter" > which was not supposed to affect any running systems. We are investigating > and will update as soon as we learn more. > > This affects any sites and infrastructure which are hosted at, or depend on > services hosted in PDX. Primarily gerrit, git, wiki and jira systems. > Jul 13, 18:09 UTC > View status page <https://status.linuxfoundation.org/> or unsubscribe > <https://status.linuxfoundation.org/?unsubscribe=lt5bjqt09cyf> from these > emails. > Powered by StatusPage > <https://www.statuspage.io/?utm_campaign=end_user_notification_emails&utm_source=status.linuxfoundation.org>
_______________________________________________ onap-discuss mailing list onap-discuss@lists.onap.org https://lists.onap.org/mailman/listinfo/onap-discuss
