I'd suggest setting up a cron job that logs the mtr data on two systems (system A, going from A->B, and system B, going from B->A), so that you can review it if/when it happens again.
I have a small shell script I wrote, which uses a dorky config file of sorts, that can do this for you if you want me to provide it. I use it on my home FreeBSD system, and my FreeBSD VPS system, checking connectivity between them (and to some other places as well). -- | Jeremy Chadwick [email protected] | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | On Sat, Mar 01, 2014 at 12:25:43AM +0000, jay lark wrote: > Correct, no packet loss in that mtr but packet loss is occurring from time to > time. The main point of that particular trace was to show the latency which > is not normal. That same trace now is reporting times around 165ms. The > reverse trace which appears to go back along the same path is around 160ms. > > Again the problem has been off and on for a while now as others here have > said. I'm just adding my data point and have a ticket with level3 as well. > > -----Original Message----- > From: Jeremy Chadwick [mailto:[email protected]] > Sent: Friday, February 28, 2014 5:55 PM > To: jay lark > Cc: '[email protected]' > Subject: Re: [outages] Level3 Routing... > > I see no indication of packet loss in this mtr; others have already explained > how to properly read output. Loss at hop #3 is not indicative of a universal > problem; it would be different if that latency "trickled down" through all > subsequent hops, all the way to the destination. > > I do see indication in this mtr of generally high latency (between > London<->Paris), reaching up to 246ms and averaging around 171ms. I do not > know if that's normal or not, due to my lack of familiarity with said > region/paths. > > However, as I've stated repeatedly [like a broken record], a traceroute/mtr > from point A to B is only half the picture -- you need the same from point B > to A, due to asymmetrical routing being commonplace. > > -- > | Jeremy Chadwick [email protected] | > | UNIX Systems Administrator http://jdc.koitsu.org/ | > | Making life hard for others since 1977. PGP 4BD6C0CB | > > On Fri, Feb 28, 2014 at 10:04:18PM +0000, jay lark wrote: > > We've also see increased packet loss and latency over the past week or so > > from our London Level3 office to our Verizon offices in NY and LA. Routes > > to our ATT office in Chicago have been okay for the most part. > > > > Here's a current trace from London to LAX. Heavy packet loss in > > London. It was in Paris before that :) > > > > My traceroute [v0.80] > > lon-nagios (0.0.0.0) > > Fri Feb 28 21:52:35 2014 > > Resolver: Received error response 2. (server failure)er of fields quit > > > > Packets Pings > > Host > > Loss% Snt Last Avg Best Wrst StDev > > 1. xxxxxxxxxxxxxxxxxxxxxxxxxxx > > 0.0% 468 0.3 0.3 0.2 4.0 0.3 > > 2. xxxxxxxxxxxxxxx > > 0.0% 468 0.9 2.3 0.8 79.1 7.7 > > 3. ae-0-11.edge3.london2.level3.net > > 61.8% 468 1.1 1.6 1.0 18.0 2.5 > > 4. ae-3-3.ebr1.paris1.level3.net > > 0.0% 468 109.9 171.8 103.0 246.0 26.9 > > 5. ae-91-91.csw4.paris1.level3.net > > 0.2% 468 111.7 170.4 99.6 241.8 27.0 > > 6. ae-92-92.ebr2.paris1.level3.net > > 0.0% 468 121.8 170.1 95.8 236.3 27.0 > > 7. ae-41-41.ebr2.washington1.level3.net > > 0.0% 467 118.5 171.4 95.7 241.1 27.1 > > 8. ae-45-45.ebr2.washington12.level3.net > > 0.0% 467 113.4 171.8 94.9 243.5 27.0 > > 9. ae-204-3604.edge1.washington12.level3.net > > 0.0% 467 107.7 171.7 95.4 250.7 27.8 > > ae-201-3601.edge1.washington12.level3.net > > ae-203-3603.edge1.washington12.level3.net > > ae-202-3602.edge1.washington12.level3.net > > 10. 208.51.134.89 > > 0.0% 467 144.0 233.5 124.8 454.1 47.8 > > 11. po1-20g.ar6.dca3.gblx.net > > 0.0% 467 148.8 226.8 125.8 448.9 37.7 > > 12. uunet-2.ar3.dca3.gblx.net > > 0.0% 467 146.1 225.9 126.5 319.5 32.9 > > 13. ??? > > 14. tengige0-6-0-2.gw5.lax7.alter > > 0.0% 467 203.1 290.3 203.1 370.6 31.4 > > 15. whitehousepost-gw.customer.alter.net > > 0.0% 467 210.6 289.2 200.1 368.3 30.6 > > > > > > From: Outages [mailto:[email protected]] On Behalf Of > > [email protected] > > Sent: Tuesday, February 25, 2014 11:00 AM > > To: 'Clayton Zekelman'; 'Michael Morrison'; '[email protected]' > > Subject: Re: [outages] Level3 Routing... > > > > > > We've been having issues in San Jose the past two days on both L3 MPLS and > > Internet circuits. MPLS was resolved yesterday with these notes: > > > > > > > > "Event The IP NOC advised that troubleshooting is ongoing with the > > equipment vendor to further isolate the point of failure. Additionally, it > > has been advised that voice services are impacted and calls are failing > > network wide. Collaborative troubleshooting efforts between the IP and > > Voice NOC and the equipment vendor will continue in an effort to expedite > > service restoral." > > > > > > > > However, we're still having internet issues. > > > > Alex > > > > From: Outages [mailto:[email protected]] On Behalf Of > > Clayton Zekelman > > Sent: Tuesday, February 25, 2014 7:04 AM > > To: Michael Morrison; [email protected]<mailto:[email protected]> > > Subject: Re: [outages] Level3 Routing... > > > > > > That's what we told our customer... by the time they replied, the issue had > > gone away. > > > > At 09:57 AM 25/02/2014, Michael Morrison wrote: > > Content-Language: en-US > > Content-Type: multipart/alternative; > > boundary="_000_0693cfd728744391a68dbd0e93123a64EX01wcilocal_" > > > > As always, trace routes are helpful. We've been having intermittent issue > > with Level 3 routes for the past couple weeks, most issue for us seem to be > > when going from Level 3 to Global Crossing routes. > > > > From: Outages [ mailto:[email protected]] On Behalf Of > > Clayton Zekelman > > Sent: Tuesday, February 25, 2014 6:39 AM > > To: *Network; Luke Rockwell; Petter Bruland > > Cc: [email protected]<mailto:[email protected]> > > Subject: Re: [outages] Level3 Routing... > > > > > > > > We had a report from a customer earlier this morning, but they advised the > > problem went away. > > > > We connect to Level3 at Southfield MI. > > > > At 09:35 AM 25/02/2014, *Network wrote: > > Content-Language: en-US > > Content-Type: multipart/alternative; > > > > boundary="_000_391AAB48E84BB24288BED210F379E1DC4DEA4C02NOCMAILDAG2Inte_" > > Latency issue with Level 3 here in Detroit area > > > > Anton > > > > > > > > From: Outages [ <mailto:[email protected]> > > mailto:[email protected]] On Behalf Of Luke Rockwell > > Sent: Tuesday, February 18, 2014 12:24 PM > > To: Petter Bruland > > Cc: [email protected]<mailto:[email protected]> > > Subject: Re: [outages] Level3 Routing... > > > > nothing on the level 3 network events page I have access to Luke > > > > On Tue, Feb 18, 2014 at 9:17 AM, Petter Bruland < > > [email protected]<mailto:[email protected]>> > > wrote: > > Not seeing any issues with Level3 in Las Vegas. > > > > -Petter > > > > From: Outages [ <mailto:[email protected]> > > mailto:[email protected]] On Behalf Of Nick Olsen > > Sent: Tuesday, February 18, 2014 6:04 AM > > To: Morgan Miskell; [email protected]<mailto:[email protected]> > > Subject: Re: [outages] Level3 Routing... > > > > Not seeing anything homed off of Level 3 Orlando. > > > > Traceroutes to a few common places look fine. > > > > ~474815 Routes from 3356. > > Nick Olsen > > Network Operations > > (855) FLSPEED x106 > > > > ________________________________ > > From: "Morgan Miskell" > > <[email protected]<mailto:[email protected]> > > > Sent: Tuesday, February 18, 2014 9:00 AM > > To: "[email protected]<mailto:[email protected]>" > > <[email protected]<mailto:[email protected]>> > > Subject: [outages] Level3 Routing... > > Anyone seeing Level3 routing issues? We just observed a massive drop > > in routes in the BGP feeds and seeing some strange ICMP issues along > > their backbone in terms of traceroutes along their backbone. They > > might be adding some filters based on the recent DDOS attacks, I can > > ping interfaces but traceroutes go to hell... > > -- > > Morgan A. Miskell > > CaroNet Data Centers > > 704-643-8330 x206 > > ---------------------------------------------------------------------- > > ------ The information contained in this e-mail is confidential and is > > intended only for the named recipient(s). If you are not the intended > > recipient you must not copy, distribute, or take any action or > > reliance on it. If you have received this e-mail in error, please > > notify the sender. Any unauthorized disclosure of the information > > contained in this e-mail is strictly prohibited. > > ---------------------------------------------------------------------- > > ------ _______________________________________________ > > Outages mailing list > > [email protected]<mailto:[email protected]> > > https://puck.nether.net/mailman/listinfo/outages > > _______________________________________________ > > Outages mailing list > > [email protected]<mailto:[email protected]> > > https://puck.nether.net/mailman/listinfo/outages > > > > _______________________________________________ > > Outages mailing list > > [email protected]<mailto:[email protected]> > > https://puck.nether.net/mailman/listinfo/outages > > --- > > > > Clayton Zekelman > > Managed Network Systems Inc. (MNSi) > > 3363 Tecumseh Rd. E > > Windsor, Ontario > > N8W 1H4 > > > > tel. 519-985-8410 > > fax. 519-985-8409 > > _______________________________________________ > > Outages mailing list > > [email protected]<mailto:[email protected]> > > https://puck.nether.net/mailman/listinfo/outages > > > > --- > > > > Clayton Zekelman > > Managed Network Systems Inc. (MNSi) > > 3363 Tecumseh Rd. E > > Windsor, Ontario > > N8W 1H4 > > > > tel. 519-985-8410 > > fax. 519-985-8409 > > > _______________________________________________ > > Outages mailing list > > [email protected] > > https://puck.nether.net/mailman/listinfo/outages > > > _______________________________________________ > Outages mailing list > [email protected] > https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list [email protected] https://puck.nether.net/mailman/listinfo/outages
