Re: [outages] Zayo Transit
Looks like im.aol.com and webmail.aol.com (which both resolve to Akamai IPs) die in the Zayo network still... > On Sep 29, 2017, at 13:07, Andy Litzinger via Outages> wrote: > > We use Zayo for IP access and are still experiencing issues. Here's the > response we've recived from Zayo Support: > > We are experiencing a major network event across multiple peering points from > the global internet table that is impacting traffic throughout the internet. > We are working to mitigate the incoming traffic across these peers at this > time to lessen the impact until the root-cause of the influx can be > identified and halted. We expect to have a resolution within the hour, > however this appears to be incoming to the Zayo global internet routing table > from our major peering partners. We appreciate your patience as our technical > teams are fully engaged to cease impact to our customers at this time and we > will update you as the status changes. We have opened the following > customer-facing bridge to assist with real-time updates as they are known to > our NCC team: > > -andy > >> On Fri, Sep 29, 2017 at 12:59 PM, Adam Gregory via Outages >> wrote: >> Looks like zayo went above their 60k prefix limit causing peers to drop if >> configured to do so. Looks like they updated to 65k per peeringdb. >> >> On Sep 29, 2017, at 3:12 PM, Mike Davis via Outages >> wrote: >> >>> :| sigh... >>> >>> created by Eric Zalesak on Sep 29, 2017 12:02:03 PM >>> Turn up customer was Singtel. We are still seeing sessions dropping after >>> mitigating Singtel. >>> >>> created by Eric Zalesak on Sep 29, 2017 11:55:59 AM >>> Suspected turn up with open policy. >>> On Fri, Sep 29, 2017 at 11:22 AM, Steven Raymond via Outages wrote: We are still seeing problems with Zayo (indirectly) as of now. ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages >>> >>> ___ >>> Outages mailing list >>> Outages@outages.org >>> https://puck.nether.net/mailman/listinfo/outages >> >> ___ >> Outages mailing list >> Outages@outages.org >> https://puck.nether.net/mailman/listinfo/outages >> > > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] Zayo Transit
Last update we received: Dear Zayo Customer, The majority of our peering links with our affected peering partners have restored with ancillary points still left. We are working to clear damping and reset those sessions left and some customers may still experience latency until this is completed. Thank you for your patience and if you are still experiencing a hard down issue, let us know so we can work to resolve it in a timely manner. From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Andy Litzinger via Outages Sent: Friday, September 29, 2017 1:07 PM To: Adam Gregory <adam.greg...@heidmar.com> Cc: outages@outages.org Subject: Re: [outages] Zayo Transit We use Zayo for IP access and are still experiencing issues. Here's the response we've recived from Zayo Support: We are experiencing a major network event across multiple peering points from the global internet table that is impacting traffic throughout the internet. We are working to mitigate the incoming traffic across these peers at this time to lessen the impact until the root-cause of the influx can be identified and halted. We expect to have a resolution within the hour, however this appears to be incoming to the Zayo global internet routing table from our major peering partners. We appreciate your patience as our technical teams are fully engaged to cease impact to our customers at this time and we will update you as the status changes. We have opened the following customer-facing bridge to assist with real-time updates as they are known to our NCC team: -andy On Fri, Sep 29, 2017 at 12:59 PM, Adam Gregory via Outages <outages@outages.org <mailto:outages@outages.org> > wrote: Looks like zayo went above their 60k prefix limit causing peers to drop if configured to do so. Looks like they updated to 65k per peeringdb. On Sep 29, 2017, at 3:12 PM, Mike Davis via Outages <outages@outages.org <mailto:outages@outages.org> > wrote: :| sigh... created by Eric Zalesak on Sep 29, 2017 12:02:03 PM Turn up customer was Singtel. We are still seeing sessions dropping after mitigating Singtel. created by Eric Zalesak on Sep 29, 2017 11:55:59 AM Suspected turn up with open policy. On Fri, Sep 29, 2017 at 11:22 AM, Steven Raymond via Outages <outages@outages.org <mailto:outages@outages.org> > wrote: We are still seeing problems with Zayo (indirectly) as of now. ___ Outages mailing list Outages@outages.org <mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list Outages@outages.org <mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list Outages@outages.org <mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] Zayo Transit
We use Zayo for IP access and are still experiencing issues. Here's the response we've recived from Zayo Support: We are experiencing a major network event across multiple peering points from the global internet table that is impacting traffic throughout the internet. We are working to mitigate the incoming traffic across these peers at this time to lessen the impact until the root-cause of the influx can be identified and halted. We expect to have a resolution within the hour, however this appears to be incoming to the Zayo global internet routing table from our major peering partners. We appreciate your patience as our technical teams are fully engaged to cease impact to our customers at this time and we will update you as the status changes. We have opened the following customer-facing bridge to assist with real-time updates as they are known to our NCC team: -andy On Fri, Sep 29, 2017 at 12:59 PM, Adam Gregory via Outages < outages@outages.org> wrote: > Looks like zayo went above their 60k prefix limit causing peers to drop if > configured to do so. Looks like they updated to 65k per peeringdb. > > On Sep 29, 2017, at 3:12 PM, Mike Davis via Outages> wrote: > > :| sigh... > > created by Eric Zalesak on Sep 29, 2017 12:02:03 PM > Turn up customer was Singtel. We are still seeing sessions dropping after > mitigating Singtel. > > created by Eric Zalesak on Sep 29, 2017 11:55:59 AM > Suspected turn up with open policy. > > On Fri, Sep 29, 2017 at 11:22 AM, Steven Raymond via Outages < > outages@outages.org> wrote: > >> We are still seeing problems with Zayo (indirectly) as of now. >> >> >> ___ >> Outages mailing list >> Outages@outages.org >> https://puck.nether.net/mailman/listinfo/outages >> > > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages > > > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages > > ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] Zayo Transit
Zayo Master Ticket TTN-0001950534 On Fri, Sep 29, 2017 at 12:59 PM, Adam Gregory via Outages < outages@outages.org> wrote: > Looks like zayo went above their 60k prefix limit causing peers to drop if > configured to do so. Looks like they updated to 65k per peeringdb. > > On Sep 29, 2017, at 3:12 PM, Mike Davis via Outages> wrote: > > :| sigh... > > created by Eric Zalesak on Sep 29, 2017 12:02:03 PM > Turn up customer was Singtel. We are still seeing sessions dropping after > mitigating Singtel. > > created by Eric Zalesak on Sep 29, 2017 11:55:59 AM > Suspected turn up with open policy. > > On Fri, Sep 29, 2017 at 11:22 AM, Steven Raymond via Outages < > outages@outages.org> wrote: > >> We are still seeing problems with Zayo (indirectly) as of now. >> >> >> ___ >> Outages mailing list >> Outages@outages.org >> https://puck.nether.net/mailman/listinfo/outages >> > > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages > > > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages > > ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] Zayo Transit
:| sigh... created by Eric Zalesak on Sep 29, 2017 12:02:03 PM Turn up customer was Singtel. We are still seeing sessions dropping after mitigating Singtel. created by Eric Zalesak on Sep 29, 2017 11:55:59 AM Suspected turn up with open policy. On Fri, Sep 29, 2017 at 11:22 AM, Steven Raymond via Outages < outages@outages.org> wrote: > We are still seeing problems with Zayo (indirectly) as of now. > > > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages > ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] Zayo Transit
We are still seeing problems with Zayo (indirectly) as of now. ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] Zayo Transit
Yeah, seeing a huge spike in updates, especially in Chicago. On Fri, Sep 29, 2017 at 2:11 PM, Mike Davis via Outageswrote: > Anyone noticing problems with Zayo transit within the last 15 minutes? > > Peers are about 125k routes lower in Chicago and Amsterdam. > > Mike > > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages > ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] Zayo Transit
Yes, We are seeing tracert’s fail to our destination in New York from France, Sweden, Denmark, Toronto, CA, and Philadelphia, PA. Regards, Ryan > On Sep 29, 2017, at 14:11, Mike Davis via Outageswrote: > > Anyone noticing problems with Zayo transit within the last 15 minutes? > > Peers are about 125k routes lower in Chicago and Amsterdam. > > Mike > ___ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages