The main problem we were seeing is that the main page would only partially load, and you could not browse to other pages. Although it does seem to be working much better now.
Chris -----Original Message----- From: Outages [mailto:[email protected]] On Behalf Of Jeremy Chadwick Sent: Friday, February 21, 2014 6:47 PM To: Chris Stone Cc: [email protected] Subject: Re: [outages] Possible outage affecting Amazon.com? What you're seeing is commonplace too. I would say both ICMP TTL exceeded (used for route/path discovery) and ECHO/ECHO REPLY look to be filtered at border routers, probably border routers that Amazon owns/maintains. While mtr by default uses ICMP exclusively, ICMP TTL exceeded is still used for path discovery, hence people thinking "there must be something wrong". mtr does have a -T flag that uses TCP SYN instead of ICMP ECHO for endpoint reachability, i.e. mtr -T -P 80 www.amazon.com, but all it does for me on FreeBSD is bail out with a hard-to-read error ("bind(): No error: 0") and doesn't reset tty params on failure, resulting in a wonderful mess of a terminal (requires "stty icanon isig iexten echo icrnl ixon brkint onlcr" be run before things begin behaving again). But traceroute -P tcp -p 80 www.amazon.com doesn't improve things either, going back to my theory that TTL exceeded is probably filtered. Testing end-to-end like you did (in a browser, telnet port 80, etc.) is one of the only things one can do in this situation. <cry> I love how network admins still to this day filter all this, making troubleshooting virtually impossible. The mentality I find hilarious; "if we permit it, someone could ping -f the router's IP that responds to TTL exceeded and cause high CPU" -- and if that doesn't work, guess what the packet kids do? Send 200gbit/sec of traffic at your uplink, effectively accomplishing the same result of knocking you offline. And no I will not respond to commentaries about this section of my mail. </cry> -- | Jeremy Chadwick [email protected] | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | On Fri, Feb 21, 2014 at 05:13:45PM -0700, Chris Stone wrote: > Am in Colorado on CenturyLink DSL. When I try and trace to > www.amazon.com, it fails: > > Start: Fri Feb 21 17:07:37 2014 > HOST: orion Loss% Snt Last Avg Best Wrst StDev > 1.|-- 0.0% 10 0.4 0.4 0.3 0.5 0.0 > 2.|-- hlrn-dsl-gw07.hlrn.qwest. 0.0% 10 35.4 38.8 34.9 70.7 11.2 > 3.|-- hlrn-agw1.inet.qwest.net 0.0% 10 35.2 35.6 34.7 40.6 1.7 > 4.|-- dca2-edge-01.inet.qwest.n 50.0% 10 85.0 85.2 84.9 85.6 0.0 > 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 > > > But the web site comes up fine in my browser. Checked IPs and tried > with that and still the same - site comes up fine but trace and pings > fail at > CTL: > > 17:08:49-cls@orion:~/scripts/ctr$ host www.amazon.com www.amazon.com > has address 72.21.215.232 17:09:40-cls@orion:~/scripts/ctr$ mtr > 72.21.215.232 17:09:49-cls@orion:~/scripts/ctr$ mtr --report > --report-cycles=10 72.21.215.232 > Start: Fri Feb 21 17:10:02 2014 > HOST: orion Loss% Snt Last Avg Best Wrst StDev > 1.|-- 0.0% 10 0.5 0.5 0.4 1.3 0.0 > 2.|-- hlrn-dsl-gw07.hlrn.qwest. 0.0% 10 35.4 41.0 35.1 89.4 17.0 > 3.|-- hlrn-agw1.inet.qwest.net 0.0% 10 35.1 35.0 34.9 35.4 0.0 > 4.|-- dca2-edge-01.inet.qwest.n 0.0% 10 85.1 85.2 84.9 85.5 0.0 > 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 > 17:10:17-cls@orion:~/scripts/ctr$ ping 72.21.215.232 PING > 72.21.215.232 (72.21.215.232) 56(84) bytes of data. > ^C > --- 72.21.215.232 ping statistics --- > 1 packets transmitted, 0 received, 100% packet loss, time 0ms > > Same going out our data center in Denver on Cogent: > > HOST: pegasus Loss% Snt Last Avg Best Wrst StDev > 1. core 0.0% 3 0.6 0.8 0.6 1.1 0.3 > 2. fa0-2.na01.b009854-0.den01.a 0.0% 3 0.9 1.8 0.9 2.9 1.0 > 3. vl3509.mag02.den01.atlas.cog 0.0% 3 1.9 1.3 0.9 1.9 0.5 > 4. te0-7-0-5.ccr21.den01.atlas. 0.0% 3 1.1 1.1 1.1 1.1 0.0 > 5. be2129.mpd21.mci01.atlas.cog 0.0% 3 13.1 13.0 12.8 13.1 0.1 > 6. be2064.ccr21.dfw01.atlas.cog 0.0% 3 22.8 23.0 22.8 23.3 0.3 > 7. be2031.ccr21.dfw03.atlas.cog 0.0% 3 23.5 23.9 23.5 24.2 0.4 > 8. ntt.dfw03.atlas.cogentco.com 0.0% 3 23.7 23.9 23.7 24.5 0.4 > 9. ae-1.r08.dllstx09.us.bb.gin. 0.0% 3 25.2 24.2 23.7 25.2 0.8 > 10. ??? 100.0 3 0.0 0.0 0.0 0.0 0.0 > > > > Chris > > > On 02/21/2014 05:00 PM, Dj Padzensky wrote: > > > >Looks like (more) Level3 problems... > > > >--Dj > > > >On Feb 21, 2014, at 3:57 PM, Grant Ridder <[email protected] > ><mailto:[email protected]>> wrote: > > > >>No problems from comcast in Mountain View ca > >> > >> > >>On Fri, Feb 21, 2014 at 3:47 PM, Clayton Dukes <[email protected] > >><mailto:[email protected]>> wrote: > >> > >> I happen to be in Marrakech at the moment and am able to access it. > >> > >> ______________________________________________________________ > >> > >> Clayton Dukes > >> ______________________________________________________________ > >> > >> > >> On Sat, Feb 22, 2014 at 12:36 AM, <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Is anyone else seeing traffic routing all over the place trying > >> to get to > >> Amazon.com <http://Amazon.com>? > >> > >> > >> Chris > >> > >> > >> _______________________________________________ > >> Outages mailing list > >> [email protected] <mailto:[email protected]> > >> https://puck.nether.net/mailman/listinfo/outages > >> > >> > >> > >> _______________________________________________ > >> Outages mailing list > >> [email protected] <mailto:[email protected]> > >> https://puck.nether.net/mailman/listinfo/outages > >> > >> > >>_______________________________________________ > >>Outages mailing list > >>[email protected] <mailto:[email protected]> > >>https://puck.nether.net/mailman/listinfo/outages > > > > > > > >_______________________________________________ > >Outages mailing list > >[email protected] > >https://puck.nether.net/mailman/listinfo/outages > > _______________________________________________ > Outages mailing list > [email protected] > https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list [email protected] https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list [email protected] https://puck.nether.net/mailman/listinfo/outages
