On Mon, Nov 6, 2017 at 11:54 AM, John Neiberger via Outages < [email protected]> wrote:
> Thanks! Every trace I've seen has indicated a problem in Level 3. That > would certainly explain why this is so widespread. > > On Mon, Nov 6, 2017 at 11:52 AM, Dovid Bender <[email protected]> wrote: > >> From NTT in NYC: >> 3. 165.254.26.201 0.0% 1777 0 >> <1777 0>.8 0.8 0.7 10.0 0.3 >> 4. ae-3.r25.nycmny01.us.bb.gin.ntt.net 0.0% 1777 0 >> <1777 0>.7 0.7 0.6 149.4 3.9 >> 5. ae-9.r22.asbnva02.us.bb.gin.ntt.net 18.5% 1777 6 >> <1777 6>.3 6.4 6.1 24.9 1.8 >> 6. ae-1.r05.asbnva02.us.bb.gin.ntt.net 0.0% 1777 5 >> <1777 5>.9 5.9 5.8 15.6 0.5 >> 7. ae-0.a02.asbnva02.us.bb.gin.ntt.net 19.8% 1777 6 >> <1777 6>.5 7.7 6.0 57.0 5.1 >> 8. lag-12.ear2.Washington12.Level3.net 98.6% 1777 >> 7208 <1777 7208>. 7377. 6927. 8791. 368.8 >> 9. ae-2-3.bear1.Boston1.Level3.net 98.1% 1777 131 >> <1777 131>.9 1047. 120.0 6493. 1842. >> 10. 4.68.71.218 45.2% 1777 129 >> <1777 129>.0 133.7 119.6 143.6 4.4 >> 11. be-2-ar01.chartford.ct.hartford.comcast.net 45.9% 1777 139 >> <1777 139>.2 136.3 122.4 146.4 4.5 >> 12. be-31-sur01.waterbury.ct.hartford.comcast.net 43.7% 1777 143 >> <1777 143>.0 137.4 121.7 147.8 4.3 >> 13. 68.87.158.6 44.7% 1777 139 >> <1777 139>.5 137.1 115.0 148.1 4.5 >> 14. c-73-218-149-14.hsd1.ma.comcast.net 45.3% 1776 139 >> <1776 139>.6 146.0 130.5 177.1 5.8 >> 15. ??? >> >> >> On Mon, Nov 6, 2017 at 1:51 PM, Dovid Bender <[email protected]> wrote: >> >>> From DO (NYC): >>> Keys: Help Display mode Restart statistics Order of fields quit >>> Packets >>> Pings >>> Host Loss% Snt >>> Last Avg Best Wrst StDev >>> 1. 104.131.255.253 0.0% 12 >>> 1.1 1.3 0.3 4.1 1.5 >>> 2. 138.197.248.160 0.0% 11 >>> 13.0 1.6 0.3 13.0 3.8 >>> 3. ix-ae-3-0.tcore1.N75-New-York.as6453.net 0.0% 11 >>> 0.4 0.4 0.3 0.8 0.1 >>> 4. if-ae-12-2.tcore2.NTO-New-York.as6453.net 0.0% 11 >>> 8.5 9.8 6.2 40.8 10.3 >>> 5. if-ae-30-2.tcore1.AEQ-Ashburn.as6453.net 0.0% 11 >>> 6.2 8.8 5.9 24.4 5.7 >>> 6. ??? >>> 7. ??? >>> 8. 4.68.71.218 63.6% 11 >>> 27.7 28.4 27.5 30.7 1.6 >>> 9. be-2-ar01.chartford.ct.hartford.comcast.net 40.0% 11 >>> 31.7 32.2 31.7 32.9 0.5 >>> 10. be-31-sur01.waterbury.ct.hartford.comcast.net 40.0% 11 >>> 33.6 32.7 32.2 33.6 0.6 >>> 11. 68.87.158.14 50.0% 11 >>> 33.5 32.5 32.1 33.5 0.6 >>> 12. c-73-218-149-14.hsd1.ct.comcast.net 70.0% 11 >>> 39.1 38.7 38.4 39.1 0.3 >>> 13. ??? >>> >>> From rackspace (Texas) >>> dev (0.0.0.0) >>> Mon Nov 6 18:50:13 2017 >>> Keys: Help Display mode Restart statistics Order of fields quit >>> Packets >>> Pings >>> Host Loss% Snt >>> Last Avg Best Wrst StDev >>> 1. 50.56.237.2 0.0% 11 >>> 1.0 1.1 0.9 1.8 0.3 >>> 2. aggr170a-8-core10.dfw3.rackspace.net 0.0% 11 >>> 0.9 1.1 0.9 1.5 0.2 >>> 3. core10-coreb.coreb.dfw3.rackspace.net 0.0% 11 >>> 3.4 3.5 3.3 3.8 0.1 >>> 4. be32.dcpe2.dfw1.rackspace.net 0.0% 11 >>> 3.6 3.8 3.4 4.4 0.3 >>> 5. be2-mspe4.dfw30.rackspace.net 0.0% 11 >>> 5.5 5.7 5.5 6.6 0.3 >>> 6. ae-22.a01.dllstx04.us.bb.gin.ntt.net 0.0% 11 >>> 8.5 6.3 4.8 9.1 1.7 >>> 7. ae-9.r11.dllstx09.us.bb.gin.ntt.net 0.0% 11 >>> 52.3 43.8 42.8 52.3 2.8 >>> 8. ae-0.r22.dllstx09.us.bb.gin.ntt.net 0.0% 11 >>> 14.0 5.8 4.4 14.0 2.9 >>> 9. ae-1.r22.asbnva02.us.bb.gin.ntt.net 45.5% 11 >>> 43.0 43.8 42.2 49.9 3.0 >>> 10. ae-1.r05.asbnva02.us.bb.gin.ntt.net 0.0% 11 >>> 42.9 43.2 42.9 44.9 0.6 >>> 11. ae-0.a02.asbnva02.us.bb.gin.ntt.net 50.0% 11 >>> 42.6 42.6 42.3 43.0 0.3 >>> 12. ??? >>> 13. ae-2-3.bear1.Boston1.Level3.net 90.0% 11 >>> 170.9 170.9 170.9 170.9 0.0 >>> 14. 4.68.71.218 20.0% 11 >>> 169.2 175.8 169.2 198.9 9.6 >>> 15. be-2-ar01.chartford.ct.hartford.comcast.net 60.0% 11 >>> 175.0 179.3 167.8 190.8 10.0 >>> 16. be-31-sur01.waterbury.ct.hartford.comcast.net 60.0% 11 >>> 180.1 179.1 175.4 183.0 3.2 >>> 17. 68.87.158.10 44.4% 10 >>> 174.8 181.7 174.8 189.7 5.6 >>> 18. c-73-218-149-14.hsd1.ma.comcast.net 60.0% 10 >>> 173.0 181.0 173.0 191.2 8.7 >>> 19. ??? >>> >>> >>> From COX in SoCal >>> >>> 3. wsip-216-231-24-181.oc.oc.cox.net 0.0% 8 >>> 1.8 1.5 1.4 1.8 0.1 >>> 4. ip68-4-11-200.oc.oc.cox.net 0.0% 8 >>> 1.2 1.2 1.2 1.4 0.1 >>> 5. 68.1.1.167 0.0% 8 >>> 2.7 2.8 2.7 2.9 0.1 >>> 6. ae-7.a00.lsanca20.us.bb.gin.ntt.net 0.0% 8 >>> 3.3 3.3 2.7 4.2 0.6 >>> 7. ae-3.r01.lsanca20.us.bb.gin.ntt.net 0.0% 8 >>> 69.4 69.5 69.4 69.8 0.1 >>> 8. ae-8.r23.lsanca07.us.bb.gin.ntt.net 0.0% 8 >>> 2.9 4.6 2.9 12.0 3.1 >>> 9. ae-6.r22.asbnva02.us.bb.gin.ntt.net 12.5% 8 >>> 66.3 66.5 66.2 67.1 0.3 >>> 10. ae-7.r06.asbnva02.us.bb.gin.ntt.net 0.0% 8 >>> 66.4 66.4 66.2 66.6 0.1 >>> 11. ae-1.a02.asbnva02.us.bb.gin.ntt.net 0.0% 8 >>> 66.2 67.2 66.1 72.8 2.3 >>> 12. ??? >>> 13. ??? >>> 14. 4.68.71.218 75.0% 8 >>> 202.6 204.3 202.6 206.0 2.4 >>> 15. be-2-ar01.chartford.ct.hartford.comcast.net 12.5% 8 >>> 202.3 203.3 197.6 208.0 4.4 >>> 16. be-31-sur01.waterbury.ct.hartford.comcast.net 37.5% 8 >>> 192.5 199.6 192.5 205.0 4.9 >>> 17. 68.87.158.14 42.9% 8 >>> 200.4 202.3 197.8 208.6 4.6 >>> 18. c-73-218-149-14.hsd1.ct.comcast.net 42.9% 8 >>> 207.9 208.2 203.0 214.8 4.9 >>> 19. ??? >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On Mon, Nov 6, 2017 at 1:38 PM, John Neiberger via Outages < >>> [email protected]> wrote: >>> >>>> Our on-net tools don't indicate a problem within the network, but I've >>>> seen a couple of traceroutes that indicate a problem on Level 3, which >>>> certainly would affect us. We're trying to nail this down. Any traceroutes >>>> you have would be helpful. >>>> >>>> On Mon, Nov 6, 2017 at 11:35 AM, Brian R. Watters via Outages < >>>> [email protected]> wrote: >>>> >>>>> Same issue is wide spread in California via both COAX and Fiber >>>>> connections (Comcast). >>>>> >>>>> ------------------------------ >>>>> *From: *"Gary Leato via Outages" <[email protected]> >>>>> *To: *"Jim Limmer" <[email protected]> >>>>> *Cc: *[email protected] >>>>> *Sent: *Monday, November 6, 2017 10:27:35 AM >>>>> *Subject: *Re: [outages] cocmast packetloss and high latency >>>>> >>>>> >>>>> we are seeing similiar comcast problems in central illinois rig >>>>> >>>>> On Mon, Nov 6, 2017 at 12:25 PM, Jim Limmer via Outages < >>>>> [email protected]> wrote: >>>>> >>>>>> Yes. And whats worse is my deli nearby uses Comcast business for >>>>>> their credit card reader.. guess I owe them for lunch tomorrow.. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Jim Limmer, CISSP >>>>>> >>>>>> < Information Security Officer >>>>>> >>>>>> Systems and Network Architect > >>>>>> >>>>>> VNA Community Healthcare >>>>>> >>>>>> www.connecticuthomecare.org >>>>>> >>>>>> >>>>>> >>>>>> [image: cid:[email protected]] [image: >>>>>> smCISSP-logo-2lines] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:* Outages [mailto:[email protected]] *On Behalf Of >>>>>> *Audrius >>>>>> Peciukas via Outages >>>>>> *Sent:* Monday, November 06, 2017 1:24 PM >>>>>> *To:* [email protected] >>>>>> *Subject:* [outages] cocmast packetloss and high latency >>>>>> >>>>>> >>>>>> >>>>>> anyone else is seeing lots of packet loss and 100ms+ latency mostly >>>>>> in east coast on comcast network? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> >>>>>> >>>>>> Audrius Peciukas >>>>>> >>>>>> 1Wire Communications >>>>>> >>>>>> http://www.1wirecom.com >>>>>> >>>>>> _______________________________________________ >>>>>> Outages mailing list >>>>>> [email protected] >>>>>> https://puck.nether.net/mailman/listinfo/outages >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Outages mailing list >>>>> [email protected] >>>>> https://puck.nether.net/mailman/listinfo/outages >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *Brian Watters* >>>>> >>>>> >>>>> 559-579-1803 <(559)%20579-1803> >>>>> >>>>> [email protected] >>>>> >>>>> http://www.totlcom.com >>>>> >>>>> *Very satisfied customers are our standard, not our goal.* >>>>> >>>>> >>>>> _______________________________________________ >>>>> Outages mailing list >>>>> [email protected] >>>>> https://puck.nether.net/mailman/listinfo/outages >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Outages mailing list >>>> [email protected] >>>> https://puck.nether.net/mailman/listinfo/outages >>>> >>>> >>> >> > > _______________________________________________ > Outages mailing list > [email protected] > https://puck.nether.net/mailman/listinfo/outages > > It's not just just Level3. I'm seeing PL through XO as well. 5.|-- 97.75.189.72 0 <https://mail.google.com/mail/u/0/72%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A00>.0% 10 1.0 2.9 0.9 12.3 3.9 6.|-- ae1-prv02-rt01.veracityne 0.0% 10 1.5 1.5 1.4 1.6 0.0 7.|-- ip65-46-63-49 <https://mail.google.com/mail/u/0/65-46-63-49>.z 63-46-65 <https://mail.google.com/mail/u/0/63-46-65>.c 0.0% 10 3.0 3.0 2.8 3.1 0.0 8.|-- 216.156.16.24.ptr.us.xo.n 0.0% 10 61.7 60.6 60.3 61.7 0.3 9.|-- 207.88.12.190.ptr.us.xo.n 0.0% 10 60.4 65.5 60.3 96.9 11.6 10.|-- te0-12-0-0 <https://mail.google.com/mail/u/0/0-12-0-0>.rar3.sanjose-c 0.0% 10 60.3 60.8 60.3 61.9 0.3 11.|-- 207.88.12.164.ptr.us.xo.n 0.0% 10 60.3 65.1 60.2 85.1 7.4 12.|-- 207.88.12.213.ptr.us.xo.n 0.0% 10 62.0 61.0 60.2 64.0 1.2 13.|-- 207.88.12.214.ptr.us.xo.n 0.0% 10 60.2 60.4 60.2 61.4 0.0 14.|-- 207.88.14.181.ptr.us.xo.n 0.0% 10 60.1 61.4 60.1 72.4 3.9 15.|-- ??? 100.0 10 0 <https://mail.google.com/mail/u/0/0%C2%A0%20%C2%A0%2010%C2%A0%20%C2%A0%200>.0 0.0 0.0 0.0 0.0 16.|-- 4.69.137.41 40 <https://mail.google.com/mail/u/0/41%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A040>.0% 10 92.4 92.7 91.7 93.3 0.4 17.|-- 4.68.63.18 40 <https://mail.google.com/mail/u/0/18%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%20%C2%A0%2040>.0% 10 120.6 121.1 120.3 122.1 0.0 18.|-- po-1-rur01.orem.ut.utah.c 40.0% 10 121.6 121.7 120.8 122.3 0.4 19.|-- lag-1-acr04.orem.ut.utah. 30.0% 10 121.8 122.4 121.4 123.5 0.4
_______________________________________________ Outages mailing list [email protected] https://puck.nether.net/mailman/listinfo/outages
