Does anyone actually moderate outages-discuss? I’ve tried to join multiple times over the years….
Mike Castoro | VP of IT Adrianna Papell Group | 646-723-3885 Chat with me in Teams<https://teams.microsoft.com/l/chat/0/[email protected]> From: Outages <[email protected]> On Behalf Of Jason Kuehl via Outages Sent: Wednesday, June 10, 2020 10:01 AM To: Andy Ringsmuth <[email protected]> Cc: Jason Kuehl via Outages <[email protected]> Subject: Re: [outages] IBM Cloud / NetworkLayer Outage External Email! Didnt know what was a list. On Wed, Jun 10, 2020 at 9:45 AM Andy Ringsmuth via Outages <[email protected]<mailto:[email protected]>> wrote: Could we move further discussion to outages-discuss please? ---- Andy Ringsmuth 5609 Harding Drive Lincoln, NE 68521-5831 (402) 304-0083 [email protected]<mailto:[email protected]> “Better even die free, than to live slaves.” - Frederick Douglas, 1863 > On Jun 10, 2020, at 8:22 AM, Jason Kuehl via Outages > <[email protected]<mailto:[email protected]>> wrote: > > Someone didn't have proper protections in place it looks like. > > On Wed, Jun 10, 2020 at 9:20 AM Mike Bolitho via Outages > <[email protected]<mailto:[email protected]>> wrote: > So another BGP highjacking... > > - Mike Bolitho > > On Wed, Jun 10, 2020, 5:40 AM Greg Dickinson via Outages > <[email protected]<mailto:[email protected]>> wrote: > Found this on Reddit as an initial RFO, so keep in mind the source (although > r/sysadmin is usually pretty good): > > > > “A 3rd party network provider was advertising routes which resulted in our > WorldWide traffic becoming severely impeded. This led to IBM Cloud clients > being unable to log-in to their accounts, greatly limited internet/DC > connectivity and other significant network route related impacts. Network > Specialists have made adjustments to route policies to restore network > access, and alleviate the impacts. The overall incident lasted from 5:55pm - > 9:30pm ET. We will be providing a fully detailed Customer Incident > Report/Root Cause Analysis as soon as possible” > > > > Greg Dickinson > > Network Engineer > > > > > > 234 Goodwin Crest Drive, Suite 500 > > Birmingham, AL 35209 > > Office: 205.917.2407 | Mobile: 205.234.6427 > > Fax: 205.945.0515 | > > [email protected]<mailto:[email protected]> | > BryantBank.com > > Unbeatable Service. Legendary Results. > > Facebook | Twitter > > > > From: Outages > <[email protected]<mailto:[email protected]>> On Behalf > Of Jason Kuehl via Outages > Sent: Tuesday, June 9, 2020 8:38 PM > To: James Brown <[email protected]<mailto:[email protected]>> > Cc: [email protected]<mailto:[email protected]> > Subject: Re: [outages] IBM Cloud / NetworkLayer Outage > > > > > > This Message originates from outside Bryant Bank. Please use caution when > opening this correspondence, attachments or hyperlinks (URLs). If you have > questions, please contact IT Support. Thank you. > > I can't wait for this post mortem. > > > > On Tue, Jun 9, 2020 at 9:21 PM James Brown via Outages > <[email protected]<mailto:[email protected]>> wrote: > > As of a few minutes ago, all US SoftLayer (IBM Cloud) locations look normal > to me. > > > > On Tue, Jun 9, 2020 at 5:27 PM James Brown > <[email protected]<mailto:[email protected]>> wrote: > > Just wanted to share updates: > > > > I got through to someone at IBM who confirmed the level of chaos over there. > It looks like they've started deploying some mitigations because I'm seeing > packet loss drop off but latency is now awful for all US locations. It looks > like IBM has failed over all of their peering to go through their European > facilities because every return packet from every POP is coming from > lon03.networklayer.com<http://lon03.networklayer.com> or > ams02.networklayer.com<http://ams02.networklayer.com> (via Telia). > > > > On Tue, Jun 9, 2020 at 4:12 PM James Brown > <[email protected]<mailto:[email protected]>> wrote: > > It looks like IBM Cloud / SoftLayer / NetworkLayer is having a global outage > since about 15:04 Pacific time affecting routing to AWS, Level 3 / > CenturyLink, and a bunch of other providers. Has anyone been able to get more > details? > > > > -- > > James Brown > > Network Engineer > > > > > > -- > > James Brown > > Engineer > > > > > > -- > > James Brown > > Engineer > > _______________________________________________ > Outages mailing list > [email protected]<mailto:[email protected]> > https://puck.nether.net/mailman/listinfo/outages > > > > > > -- > > Sincerely, > > Jason W Kuehl > Cell 920-419-8983 > [email protected]<mailto:[email protected]> > > > > NOTICE: This electronic mail message and any files transmitted with it are > intended exclusively for the individual or entity to which it is addressed. > The message, together with any attachment, may contain confidential and/or > privileged information. Any unauthorized review, use, print, save, copy, > disclosure or distribution is strictly prohibited. If you have received this > message in error, please immediately advise the sender by reply email and > delete copies. Thank you. > _______________________________________________ > Outages mailing list > [email protected]<mailto:[email protected]> > https://puck.nether.net/mailman/listinfo/outages > _______________________________________________ > Outages mailing list > [email protected]<mailto:[email protected]> > https://puck.nether.net/mailman/listinfo/outages > > > -- > Sincerely, > > Jason W Kuehl > Cell 920-419-8983 > [email protected]<mailto:[email protected]> > _______________________________________________ > Outages mailing list > [email protected]<mailto:[email protected]> > https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list [email protected]<mailto:[email protected]> https://puck.nether.net/mailman/listinfo/outages -- Sincerely, Jason W Kuehl Cell 920-419-8983 [email protected]<mailto:[email protected]>
_______________________________________________ Outages mailing list [email protected] https://puck.nether.net/mailman/listinfo/outages
