[outages] Xo Infinera 1600G span failure since about 5:15 pm Eastern

[Silvey, David via Outages]

  *   2/4/16 6:01 PM

Alert Detail: Infinera 1600G span failure between Smyrna, GA and Washington, DC
Alert Update: Infinera 1600G span failure between 4000 Highlands Pkwy SE, 
Smyrna, GA and 4301 Connecticut Ave NW, Washington, DC. Transport Operations is 
investigating.

The information contained in this E-mail message is privileged, confidential, 
and may be protected from disclosure;
please be aware that any other use, printing, copying, disclosure or 
dissemination of this communication may be 
subject to legal restriction or sanction. If you think that you have received 
this E-mail message in 
error, please reply to the sender.

This E-mail message and any attachments have been scanned for viruses and are 
believed to be free of any virus or 
other defect that might affect any computer system into which it is received 
and opened. However, it is the 
responsibility of the recipient to ensure that it is virus free and no 
responsibility is accepted by Kelley 
Drye & Warren LLP for any loss or damage arising in any way from its use.
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

[outages] XO point to point circuit from Edison, NJ to Scottsdale AZ down since 11:38 AM Eastern this morning due to Level 3 fiber cut.

[Silvey, David via Outages]

>From XO support:

Dave, your trouble ticket has been updated with the following new information:
Current Status: Received Trouble Report -- Isolated to Network Outage -- 
5/26/2016 1:27:32 PM
Status Details: Level 3 has confirmed a fibercut due to boring work. Repair 
crews and equipment are onsite. Currently there is no ETR.

--Dave

The information contained in this E-mail message is privileged, confidential, 
and may be protected from disclosure;
please be aware that any other use, printing, copying, disclosure or 
dissemination of this communication may be 
subject to legal restriction or sanction. If you think that you have received 
this E-mail message in 
error, please reply to the sender.

This E-mail message and any attachments have been scanned for viruses and are 
believed to be free of any virus or 
other defect that might affect any computer system into which it is received 
and opened. However, it is the 
responsibility of the recipient to ensure that it is virus free and no 
responsibility is accepted by Kelley 
Drye & Warren LLP for any loss or damage arising in any way from its use.
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

Re: [outages] Cogent routing issues, possibly SoCal?

[Silvey, David via Outages]

Dear Cogent Customer,

The outage in Los Angeles, CA  is now resolved.

If you are still experiencing problems with your service, please contact our 
support team at 877-726-4368 so that we can investigate.

We apologize for the inconvenience this outage caused.

Thank You,

Cogent Customer Support
Cogent Communications
supp...@cogentco.com
877-726-4368

David Silvey
dsil...@kelleydrye.com
212-808-5124


On Sep 18, 2016, at 12:03 PM, Darren Schreiber via Outages 
mailto:outages@outages.org>> wrote:

We've seen this since early AM. We called and they said it was scheduled 
mainteance in LAX. Which makes no sense that the next closest hop is Hong 
Kong...

It's also on their status page. 
http://status.cogentco.com/

Eta 11am PT seems like a really long maintenance window...


 Original message 
From: Wally Hass via Outages mailto:outages@outages.org>>
Date: 9/18/16 9:00 AM (GMT-08:00)
To: Bobin Joseph mailto:bobin.jos...@mitchell.com>>, 
outages@outages.org
Subject: Re: [outages] Cogent routing issues, possibly SoCal?

We also saw issues this AM but seems resolved now.

Wally Hass Consultant, Network Thinking Solutions, Inc
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/phone2.png] 
818-707-7705 
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/email1.png] 
wh...@networkthinking.com  | 
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/social.png] 
Skype: 
networkthinking
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/website.png] 
http://www.networkthinking.com/ 
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/address1.png] 
5655 Lindero Canyon Suite 705, Westlake Village, CA 91362

[https://s3.amazonaws.com/images.wisestamp.com/icons_32/facebook.png]
 [https://s3.amazonaws.com/images.wisestamp.com/icons_32/twitter.png] 


From: Outages [outages-boun...@outages.org] 
on behalf of Bobin Joseph via Outages 
[outages@outages.org]
Sent: Sunday, September 18, 2016 08:54
To: outages@outages.org
Subject: [outages] Cogent routing issues, possibly SoCal?

Anybody seeing issues over their Cogent internet ckts? This is an mtr of 
traffic from our Dallas to San Diego site. Looks like traffic is going over to 
Asia before making it back.
Also noticed this from my home in SD to our site in SD.

Is it an issue in and out of the LAX/SoCal region?

Packets 
  Pings
 Host Loss% 
Drop   Rcv  Wrst   Last   Avg  Best StDev  Wrst

4. 
te0-0-0-2.rcr21.dfw04.atlas.cogentco.com 
  0.0%0   637   2.61.7   1.6   1.5   0.1   2.6
 5. 
be2938.ccr21.dfw01.atlas.cogentco.com
  0.0%0   637   4.22.2   2.1   1.9   0.2   4.2
 6. 
be2432.ccr21.mci01.atlas.cogentco.com
  0.2%1   636  16.3   11.9  11.9  11.8   0.2  16.3
 7. 
be3035.ccr21.den01.atlas.cogentco.com
  0.0%0   637  23.9   23.8  23.3  23.2   0.2  23.9
 8. 
be3037.ccr21.slc01.atlas.cogentco.com
  0.0%0   637  34.2   33.6  33.6  33.5   0.2  34.2
 9. 
be2085.ccr21.sea02.atlas.cogentco.com
  0.8%5   632 681.5   54.3  69.0  54.1  80.5 681.5
10. 
te0-0-2-0.ccr11.tyo01.atlas.cogentco.com 
 29.6%  188   448 165.0  150.5 157.6 148.7   3.6 165.0
11. 
te0-4-0-3.ccr21.hkg02.atlas.cogentco.com 
 29.7%  189   447 227.4  217.3 222.3 213.8   3.6 227.4
12. 
te0-4-0-0.ccr21.sin01.atlas.cogentco.com 
 32.5%  207   429 262.6  253.0 257.2 248.8   3.6 262.6
13. 
te0-0-0-8.ccr21.lax01.atlas.cogentco.com 
 47.5%  302   334 4943.  449.9 467.9 445.9 245.7 4943.
14. 
be2540.rcr22.lax04.atlas.cogentco.com
 33.4%  212   423 452.1  422.9 428.3 414.0   8.7 452.1


Thanks,
Bobin Joseph

Re: [outages] Cogent routing issues, possibly SoCal?

[Silvey, David via Outages]

We are also having problems in LA.

From: Nick W [mailto:nickdwh...@gmail.com]
Sent: Monday, September 19, 2016 2:46 PM
To: Silvey, David 
Cc: outages@outages.org
Subject: Re: [outages] Cogent routing issues, possibly SoCal?

Anyone else seeing issues again? I see them from TX to NV/CA. Had to shut down 
our Cogent peer.

Started at 11:35AM Pacific/1:35PM Central... about 10 minutes ago. Don't see 
anything on the Cogent status page.

On Sun, Sep 18, 2016 at 4:48 PM, Silvey, David via Outages 
mailto:outages@outages.org>> wrote:

Dear Cogent Customer,

The outage in Los Angeles, CA  is now resolved.

If you are still experiencing problems with your service, please contact our 
support team at 877-726-4368 so that we can investigate.

We apologize for the inconvenience this outage caused.

Thank You,

Cogent Customer Support
Cogent Communications
supp...@cogentco.com<mailto:supp...@cogentco.com>
877-726-4368

David Silvey
dsil...@kelleydrye.com<mailto:dsil...@kelleydrye.com>
212-808-5124


On Sep 18, 2016, at 12:03 PM, Darren Schreiber via Outages 
mailto:outages@outages.org>> wrote:
We've seen this since early AM. We called and they said it was scheduled 
mainteance in LAX. Which makes no sense that the next closest hop is Hong 
Kong...

It's also on their status page. 
http://status.cogentco.com/<http://status.cogentco.com/>

Eta 11am PT seems like a really long maintenance window...


 Original message 
From: Wally Hass via Outages mailto:outages@outages.org>>
Date: 9/18/16 9:00 AM (GMT-08:00)
To: Bobin Joseph mailto:bobin.jos...@mitchell.com>>, 
outages@outages.org<mailto:outages@outages.org>
Subject: Re: [outages] Cogent routing issues, possibly SoCal?

We also saw issues this AM but seems resolved now.

Wally Hass Consultant, Network Thinking Solutions, Inc

[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/phone2.png]818-707-7705
 
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/email1.png]wh...@networkthinking.com
 <mailto:wh...@networkthinking.com> | 
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/social.png] 
Skype: 
networkthinking<https://webapp.wisestamp.com/sig_iframe?origin=outlook&signature_id=5215201074348032&t=0.09465256404943023#>
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/website.png]http://www.networkthinking.com/
 <http://www.networkthinking.com/>
[https://s3.amazonaws.com/images.wisestamp.com/symbols/grey/small/address1.png]5655
 Lindero Canyon Suite 705, Westlake Village, CA 91362

[https://s3.amazonaws.com/images.wisestamp.com/icons_32/facebook.png]<http://www.facebook.com/NetworkThinkingSolutions/?ref=aymt_homepage_panel>
 [https://s3.amazonaws.com/images.wisestamp.com/icons_32/twitter.png] 
<http://twitter.com/networkthinking>


From: Outages [outages-boun...@outages.org<mailto:outages-boun...@outages.org>] 
on behalf of Bobin Joseph via Outages 
[outages@outages.org<mailto:outages@outages.org>]
Sent: Sunday, September 18, 2016 08:54
To: outages@outages.org<mailto:outages@outages.org>
Subject: [outages] Cogent routing issues, possibly SoCal?
Anybody seeing issues over their Cogent internet ckts? This is an mtr of 
traffic from our Dallas to San Diego site. Looks like traffic is going over to 
Asia before making it back.
Also noticed this from my home in SD to our site in SD.

Is it an issue in and out of the LAX/SoCal region?

Packets 
  Pings
 Host Loss% 
Drop   Rcv  Wrst   Last   Avg  Best StDev  Wrst

4. 
te0-0-0-2.rcr21.dfw04.atlas.cogentco.com<http://rcr21.dfw04.atlas.cogentco.com> 
  0.0%0   637   2.61.7   1.6   1.5   0.1   2.6
 5. 
be2938.ccr21.dfw01.atlas.cogentco.com<http://ccr21.dfw01.atlas.cogentco.com>
  0.0%0   637   4.22.2   2.1   1.9   0.2   4.2
 6. 
be2432.ccr21.mci01.atlas.cogentco.com<http://ccr21.mci01.atlas.cogentco.com>
  0.2%1   636  16.3   11.9  11.9  11.8   0.2  16.3
 7. 
be3035.ccr21.den01.atlas.cogentco.com<http://ccr21.den01.atlas.cogentco.com>
  0.0%0   637  23.9   23.8  23.3  23.2   0.2  23.9
 8. 
be3037.ccr21.slc01.atlas.cogentco.com<http://ccr21.slc01.atlas.cogentco.com>
  0.0%0   637  34.2   33.6  33.6  33.5   0.2  34.2
 9. 
be2085.ccr21.sea02.atlas.cogentco.com<http://ccr21.sea02.atlas.cogentco.com>
  0.8%5   632 681.5   54.3  69.0  54.1  80.5 681.5
10. 
te0-0-2-0.ccr11.tyo01.atlas.cogentco.com<http://ccr11.tyo01.atlas.cogentco.com> 
 29.6%  188   448 165.0  150.5 157.6 148.7   3.6 165.0
11. 
te0-4-0-3.ccr

[outages] XO fiber cut in Georgia, USA

[Silvey, David via Outages]

Received Trouble Report, Isolated to Network Outage
Saw circuit go down at 1:59 PM Eastern time.  We opened a ticket with XO 
support and were told this:  "Fiber cut between Bogart and Smyrna GA Alert 
Update: Local market and Transport Operations are still investigating."

The information contained in this E-mail message is privileged, confidential, 
and may be protected from disclosure;
please be aware that any other use, printing, copying, disclosure or 
dissemination of this communication may be 
subject to legal restriction or sanction. If you think that you have received 
this E-mail message in 
error, please reply to the sender.

This E-mail message and any attachments have been scanned for viruses and are 
believed to be free of any virus or 
other defect that might affect any computer system into which it is received 
and opened. However, it is the 
responsibility of the recipient to ensure that it is virus free and no 
responsibility is accepted by Kelley 
Drye & Warren LLP for any loss or damage arising in any way from its use.
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

Re: [outages] XO fiber cut in Georgia, USA

[Silvey, David via Outages]

XO ticket seems to be because of Level 3 Fiber cut.  Below is XO ticket update 
as of 4:08 PM Eastern.

“Received Trouble Report, Isolated to Network Outage
Fiber event causing Infinera 400G, 800G, 1600G span failure between Smyrna and 
Bogart, GA. Alert Update: The local market tech is on site in the area of the 
fibercut and further isolating. Level 3 has this handed off to their dispatch 
group but n“

That’s all they have.  Sentence ends at the “n”.


From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Silvey, David 
via Outages
Sent: Tuesday, January 31, 2017 3:39 PM
To: outages@outages.org
Subject: [outages] XO fiber cut in Georgia, USA



Received Trouble Report, Isolated to Network Outage
Saw circuit go down at 1:59 PM Eastern time.  We opened a ticket with XO 
support and were told this:  “Fiber cut between Bogart and Smyrna GA Alert 
Update: Local market and Transport Operations are still investigating.”


The information contained in this E-mail message is privileged, confidential, 
and may be protected from disclosure; please be aware that any other use, 
printing, copying, disclosure or dissemination of this communication may be 
subject to legal restriction or sanction. If you think that you have received 
this E-mail message in error, please reply to the sender.

This E-mail message and any attachments have been scanned for viruses and are 
believed to be free of any virus or other defect that might affect any computer 
system into which it is received and opened. However, it is the responsibility 
of the recipient to ensure that it is virus free and no responsibility is 
accepted by Kelley Drye & Warren LLP for any loss or damage arising in any way 
from its use.

The information contained in this E-mail message is privileged, confidential, 
and may be protected from disclosure;
please be aware that any other use, printing, copying, disclosure or 
dissemination of this communication may be 
subject to legal restriction or sanction. If you think that you have received 
this E-mail message in 
error, please reply to the sender.

This E-mail message and any attachments have been scanned for viruses and are 
believed to be free of any virus or 
other defect that might affect any computer system into which it is received 
and opened. However, it is the 
responsibility of the recipient to ensure that it is virus free and no 
responsibility is accepted by Kelley 
Drye & Warren LLP for any loss or damage arising in any way from its use.
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

[outages] AT&T BVOIP outages nationwide

[Silvey, David via Outages]

[cid:image001.png@01D2AC86.81C915C0]

The information contained in this E-mail message is privileged, confidential, 
and may be protected from disclosure;
please be aware that any other use, printing, copying, disclosure or 
dissemination of this communication may be 
subject to legal restriction or sanction. If you think that you have received 
this E-mail message in 
error, please reply to the sender.

This E-mail message and any attachments have been scanned for viruses and are 
believed to be free of any virus or 
other defect that might affect any computer system into which it is received 
and opened. However, it is the 
responsibility of the recipient to ensure that it is virus free and no 
responsibility is accepted by Kelley 
Drye & Warren LLP for any loss or damage arising in any way from its use.
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

[outages] Level3 fiber cut near Dallas

[Silvey, David via Outages]

From XO support:

8:17 pm Eastern:  Level 3 reports that construction and splice crews are on 
site, and are working on a plan for restoration. As we have additional 
information, we will advise you as soon as possible.

6:45 pm Eastern: A fiber cut on a 96 count and a 216 count fiber has been 
located on Level 3?s network and a splicing crew is currently en route. Level 3 
has advised that they have an ETR of approximately 3:30 
GMT for resolution, however this may change as 
repairs proceed


This message is subject to Kelley Drye & Warren LLP's email communication 
policy. 
KDW-Disclaimer: https://protect-us.mimecast.com/s/328mCYEQRJh6KrJysGQXtx
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

[outages] XO circuit down, possible fiber cut

[Silvey, David via Outages]

Network Outage Sys Down Fiber Cut, Isolation in Progress
Cary NMC Transport reports possible fiber cut reporting on Infinera 400G 
network in Memphis TN. OCC has been contacted to dispatch technician to OTDR 
fiber and isolate cut. Impact is still being collected at this time.


This message is subject to Kelley Drye & Warren LLP's email communication 
policy. 
KDW-Disclaimer: https://protect-us.mimecast.com/s/328mCYEQRJh6KrJysGQXtx
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

[outages] Cogent cut on West Coast - seeing high latency to Arizona and East coast

[Silvey, David via Outages]

From Cogent support at 10:22 pm Eastern on 5/13/2015

At this time, vendors are working to identify a splice point for repairs. There 
is currently no ETR. Please reference 
status.cogentco.com for updates on this issue.

To find answers to common technical questions you may have about your Cogent 
service, please review Cogent’s User Guide located 
athttp://www.cogentco.com/Guide/Global20Guide.pdf
To manage your account online go to 
https://ecogent.cogentco.com

Thank You,


-Original Message-


Dear Cogent Customer,

We currently have a fiber cut on the west coast that is impacting many 
customers. You may see packet loss or latency until this is resolved. Our 
master ticket for this outage is 6157894. You can view updates at 
status.cogentco.com.

The information contained in this E-mail message is privileged, confidential, 
and may be protected from disclosure;
please be aware that any other use, printing, copying, disclosure or 
dissemination of this communication may be 
subject to legal restriction or sanction. If you think that you have received 
this E-mail message in 
error, please reply to the sender.

This E-mail message and any attachments have been scanned for viruses and are 
believed to be free of any virus or 
other defect that might affect any computer system into which it is received 
and opened. However, it is the 
responsibility of the recipient to ensure that it is virus free and no 
responsibility is accepted by Kelley 
Drye & Warren LLP for any loss or damage arising in any way from its use.
___
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages