Re: [outages] Level3/TW Telecom issues?

2015-08-28 Thread Ryan Gelobter via Outages
To: Bobin Joseph Cc: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete internet in DFW and SAN We were not down but significant

Re: [outages] Level3/TW Telecom issues?

2015-08-28 Thread Rafael Possamai via Outages
...@outages.org] On Behalf Of jungle Boogie via Outages Sent: Thursday, August 27, 2015 2:56 PM To: Bobin Joseph Cc: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else having

Re: [outages] Level3/TW Telecom issues?

2015-08-28 Thread Bobin Joseph via Outages
: outages@outages.orgmailto:outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.orgmailto:outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete internet in DFW and SAN We were

Re: [outages] Level3/TW Telecom issues?

2015-08-28 Thread Bobin Joseph via Outages
, 2015 6:57 AM To: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? So that is how they lowered the traffic on the cards in Chicago. Thanks, Bobin From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Rafael Possamai via Outages Sent: Friday, August 28, 2015 6:26 AM

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Aaron Gross via Outages
mailto:outages@outages.org *Subject:* Re: [outages] Level3/TW Telecom issues? __ __ Was anyone able to get some useful information out of Level3 regarding this yesterday? Also is anyone else seeing similar issues? On Aug 26, 2015 2:40 PM, Rafael Possamai via Outages

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Bobin Joseph via Outages
: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? We are seeing the same things that happened yesterday on our circuits in Milwaukee and Chicago areas. Lots of packet loss and large latency through Chicago exchanges. On Thu, Aug 27, 2015 at 12:33 PM, Bob Hancock via Outages

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Steven Fischer via Outages
*From:* Outages [mailto:outages-boun...@outages.org] *On Behalf Of *Ryan Gelobter via Outages *Sent:* Thursday, August 27, 2015 9:44 AM *To:* Rafael Possamai *Cc:* outages@outages.org *Subject:* Re: [outages] Level3/TW Telecom issues? Was anyone able to get some useful information out

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Bobin Joseph via Outages
: [outages] Level3/TW Telecom issues? Was anyone able to get some useful information out of Level3 regarding this yesterday? Also is anyone else seeing similar issues? On Aug 26, 2015 2:40 PM, Rafael Possamai via Outages outages@outages.orgmailto:outages@outages.org wrote: Here's a graph showing

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Bob Colon via Outages
For me, issues seemed to clear up around 1700 CDT yesterday, but once again I am getting packet loss to the level3 network.. Signature To ensure prompt service, please use Reply All when responding Robert Colon | TCSA Tier 3 | EthoStream LLC | P: (877) 282-2519 x2114 | F: (414) 258-8307 |

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Brooks via Outages
We had 30 minutes of good operation, now the L3 (ex-twt) NOC says it's a transport/dwdm problem, completely unrelated to yesterday.I don't think so, and yesterday's RFO was crap. Here are the main tickets tracking yesterday and today: routing ST1878748 DWDM ST1879913 This is

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Jason Hellenthal via Outages
tier-2 group is investigating the issue at this time. Thanks, Bobin From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Aaron Gross via Outages Sent: Thursday, August 27, 2015 10:55 AM To: Bob Hancock Cc: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Bob Hancock via Outages
@outages.org *Subject:* Re: [outages] Level3/TW Telecom issues? __ __ Was anyone able to get some useful information out of Level3 regarding this yesterday? Also is anyone else seeing similar issues? On Aug 26, 2015 2:40 PM, Rafael Possamai via Outages outages

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Brooks via Outages
@outages.org Subject: Re: [outages] Level3/TW Telecom issues? We are seeing the same things that happened yesterday on our circuits in Milwaukee and Chicago areas. Lots of packet loss and large latency through Chicago exchanges. On Thu, Aug 27, 2015 at 12:33 PM, Bob Hancock via Outages outages

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Gelobter via Outages
Was anyone able to get some useful information out of Level3 regarding this yesterday? Also is anyone else seeing similar issues? On Aug 26, 2015 2:40 PM, Rafael Possamai via Outages outages@outages.org wrote: Here's a graph showing some packet loss: http://imgur.com/KVAzetA On Wed, Aug 26,

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread chris via Outages
: [outages] Level3/TW Telecom issues? Was anyone able to get some useful information out of Level3 regarding this yesterday? Also is anyone else seeing similar issues? On Aug 26, 2015 2:40 PM, Rafael Possamai via Outages outages@outages.org wrote: Here's a graph showing some packet loss: http

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread jungle Boogie via Outages
On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete internet in DFW and SAN We were not down but significant routing issues in Las Vegas. Thanks, Bobin

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread jungle Boogie via Outages
On 27 August 2015 at 12:54, Scott Weeks via Outages outages@outages.org wrote: --- outages@outages.org wrote: Root Cause: An unknown issue in Chicago, IL was impacting IP services. Fix Action: Adjustments were made to resolve the issue and restore services. Summary: The IP NOC responded to

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Gelobter via Outages
issues from 1406CST until we shut down our circuit. -Original Message- From: Outages [mailto:outages-boun...@outages.org] On Behalf Of jungle Boogie via Outages Sent: Thursday, August 27, 2015 2:56 PM To: Bobin Joseph Cc: outages@outages.org Subject: Re: [outages] Level3/TW

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Lucius Kalbfleisch via Outages
internet in DFW and SAN Thanks, Bobin -Original Message- From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Ryan Brooks via Outages Sent: Thursday, August 27, 2015 12:03 PM To: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On Aug 27, 2015

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Brooks via Outages
: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org mailto:outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete internet in DFW and SAN We were not down but significant routing issues in Las

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Craig Gill via Outages
[mailto:outages-boun...@outages.org] On Behalf Of Ryan Brooks via Outages Sent: Thursday, August 27, 2015 12:03 PM To: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On Aug 27, 2015, at 1:37 PM, David Hubbard via Outages outages@outages.org wrote: I'm seeing traffic outbound

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Eric Henson via Outages
: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete internet in DFW and SAN We were not down but significant routing issues in Las Vegas

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Izzy Goldstein - TeleGo via Outages
one of my clients with TW Telecom has been experiencing packet loss since 10Am ish located in NYC On Thu, Aug 27, 2015 at 3:55 PM, jungle Boogie via Outages outages@outages.org wrote: On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else having

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Gelobter via Outages
: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete internet in DFW and SAN We were not down but significant routing issues in Las

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Brooks via Outages
On Aug 27, 2015, at 1:37 PM, David Hubbard via Outages outages@outages.org wrote: I'm seeing traffic outbound from Level 3 Tampa going to Frankfurt Germany currently, so they're definitely doing something wrong. 3 xe-10-3-1.bar1.Tampa1.Level3.net (4.53.172.29) 13.642 ms 13.650 ms

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Bobin Joseph via Outages
: [outages] Level3/TW Telecom issues? On Aug 27, 2015, at 1:37 PM, David Hubbard via Outages outages@outages.org wrote: I'm seeing traffic outbound from Level 3 Tampa going to Frankfurt Germany currently, so they're definitely doing something wrong. 3 xe-10-3-1.bar1.Tampa1.Level3.net

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Brooks via Outages
And that was yesterday. Today it sounds like some serious synergy is under way: IP and Transport Tier III, Operations Engineering and Field Services continue collaboratively working the issue. Update from 2:26pm US/Central. This has been bad for 24 hrs and seems to be impacting peering and

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Jameel Akari via Outages
: Outages [mailto:outages-boun...@outages.org] On Behalf Of Jason Hellenthal via Outages Sent: Thursday, August 27, 2015 2:23 PM To: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? Retracting my statement from yesterday this is still a solid issue here. Yes absolutely still

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Bobin Joseph via Outages
Joseph Cc: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete internet in DFW and SAN We were not down but significant routing issues

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread David Hubbard via Outages
Cc: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org wrote: Anybody else

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Gelobter via Outages
our circuit. -Original Message- From: Outages [mailto:outages-boun...@outages.org] On Behalf Of jungle Boogie via Outages Sent: Thursday, August 27, 2015 2:56 PM To: Bobin Joseph Cc: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48

Re: [outages] Level3/TW Telecom issues?

2015-08-27 Thread Ryan Brooks via Outages
PM To: Bobin Joseph Cc: outages@outages.org mailto:outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? On 27 August 2015 at 12:48, Bobin Joseph via Outages outages@outages.org mailto:outages@outages.org wrote: Anybody else having complete outage on Level3? We lost complete

Re: [outages] Level3/TW Telecom issues?

2015-08-26 Thread Bobin Joseph via Outages
Can confirm. Seeing packet loss in the DFW area. Thanks, Bobin From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Rafael Possamai via Outages Sent: Wednesday, August 26, 2015 1:21 PM To: outages@outages.org Subject: Re: [outages] Level3/TW Telecom issues? Yes, have noticed some

Re: [outages] Level3/TW Telecom issues?

2015-08-26 Thread Rafael Possamai via Outages
Here's a graph showing some packet loss: http://imgur.com/KVAzetA On Wed, Aug 26, 2015 at 3:16 PM, Matt Elliott via Outages outages@outages.org wrote: Anyone else having issues on Level3? We’re seeing packet loss with anything that traverses Chicago. Thanks, Matt This message and

Re: [outages] Level3/TW Telecom issues?

2015-08-26 Thread Rafael Possamai via Outages
Yes, have noticed some packet loss as well. Out of midwest, east, west, NYC as well. On Wed, Aug 26, 2015 at 3:16 PM, Matt Elliott via Outages outages@outages.org wrote: Anyone else having issues on Level3? We’re seeing packet loss with anything that traverses Chicago. Thanks, Matt This