On Tue, Aug 12, 2014 at 5:29 PM, Geoffrey Mina via Outages <[email protected]> wrote: > We were told by Quality Technology Services in Atlanta that they were > "installing additional memory" to correct the issue. Perhaps they meant > they were actually reconfiguring/bouncing the routers.... I know they had > Cisco engineers on site in GA, CA, and NJ today. >
Probably they were recarving the TCAM allocation. Quick summary document for those having the issue: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html This increases the size of the TCAM allocated for v4 routes, at the cost of v6 space. This doesn't actually *solve* the issue, but does at least kick the can down the road a bit longer... TCAM based solutions for FIB, while cheap to implement, leads to pain... W > Either way, after our network link went totally down and came back up, the > issue was resolved. > > > ________________________________ > From: Bill Wichers [[email protected]] > Sent: Tuesday, August 12, 2014 3:27 PM > To: Geoffrey Mina > Cc: Michael Peterman; [email protected] > Subject: Re: [outages] So what is broken > > That's a tcam limit on the 6500/7600 platform. It's not upgradable, but it > is configurable. > > The gsr platform doesn't have a hard limit like this, its just memory (And > is upgradeable), but most with that gear will have already had to upgrade > well before now. > > Sent from my iPhone > > On Aug 12, 2014, at 4:14 PM, "Geoffrey Mina via Outages" > <[email protected]> wrote: > > Cisco gear can't support more than 512K routes... We hit that or close to > that at around 4am EDT today. > > ISPs are scrambling to update memory in their routers. > > > -------- Original message -------- > From: Michael Peterman via Outages > Date:08/12/2014 1:00 PM (GMT-07:00) > To: [email protected] > Subject: [outages] So what is broken > > So is this issue all related to a fiber cut or a DC/Peering point having > issues? > > http://www.thewhir.com/web-hosting-news/liquidweb-among-companies-affected-major-outage-across-us-network-providers > > > > Michael Peterman > > > > > -----Original Message----- > From: Outages [mailto:[email protected]] On Behalf Of > [email protected] > Sent: Tuesday, August 12, 2014 11:00 AM > To: [email protected] > Subject: Outages Digest, Vol 75, Issue 9 > > Send Outages mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > https://puck.nether.net/mailman/listinfo/outages > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific than > "Re: Contents of Outages digest..." > > > Today's Topics: > > 1. Windstream (Mike Walter) > 2. CoreSite Any2 LA (jd) > 3. Re: CoreSite Any2 LA (Paul Stewart) > 4. LastPass.com has a DC outage (Chris Egeland) > 5. Major outages today, not much info at this time > (Hoyle Anderson (AM)) > 6. Re: Major outages today, not much info at this time (Teun Vink) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 12 Aug 2014 13:19:08 +0000 > From: Mike Walter <[email protected]> > To: "'[email protected]'" <[email protected]> > Subject: [outages] Windstream > Message-ID: > <[email protected]> > Content-Type: text/plain; charset="us-ascii" > > Just an FYI, there is a Windstream fiber cut effecting Ohio and Kentucky > customers, no ETA to repair based on their phone message. Still on hold > waiting to open case to find out more. > -Mike > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <https://puck.nether.net/pipermail/outages/attachments/20140812/194c6c4b/attachment-0001.html> > > ------------------------------ > > Message: 2 > Date: Tue, 12 Aug 2014 07:19:04 -0700 > From: jd <[email protected]> > To: "'[email protected]'" <[email protected]> > Subject: [outages] CoreSite Any2 LA > Message-ID: <[email protected]> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Is anyone having issues with Any2 in LA? I have no evidence but I'm hearing > some things, if anyone could debunk. > > Thanks > JD > > > ------------------------------ > > Message: 3 > Date: Tue, 12 Aug 2014 10:42:39 -0400 > From: Paul Stewart <[email protected]> > To: "'[email protected]'" <[email protected]> > Subject: Re: [outages] CoreSite Any2 LA > Message-ID: <d00fa00a.806c1%[email protected]> > Content-Type: text/plain; charset="UTF-8" > > No issues here at this point?. At least that have become evident? > > Paul > > > > On 2014-08-12, 10:19 AM, "jd via Outages" <[email protected]> wrote: > >>Is anyone having issues with Any2 in LA? I have no evidence but I'm >>hearing some things, if anyone could debunk. >> >>Thanks >>JD >>_______________________________________________ >>Outages mailing list >>[email protected] >>https://puck.nether.net/mailman/listinfo/outages >> > > > > > > ------------------------------ > > Message: 4 > Date: Tue, 12 Aug 2014 15:04:58 +0000 > From: Chris Egeland <[email protected]> > To: "[email protected] List" <[email protected]> > Subject: [outages] LastPass.com has a DC outage > Message-ID: <EA49DAFC5FDC1A40B16DEE2D6ED5469A9FD277AB@mail01> > Content-Type: text/plain; charset="us-ascii" > > https://twitter.com/LastPass/status/499199566701658112 > > "Update: One of our data centers went down, we have the service fully > running on another data center & are working to minimize impact." > > I'm getting "An error occurred while attempting to contact the server. > Please check your Internet connection." In my LastPass bar on Firefox. > > So, if you're getting errors with LastPass, this is why. > > > > ________________________________ > > This e-mail and any attachments may contain privileged or confidential > information of Ferrotherm Corporation. This e-mail is intended solely for > the use of the individual or entity to which it is addressed. If you are not > the intended recipient of this e-mail, you are hereby notified that any > copying, distribution, dissemination or action taken in relation to the > contents of this e-mail and any of its attachments is strictly prohibited > and may be unlawful. If you have received this e-mail in error, please > notify the sender immediately and permanently delete the original e-mail and > destroy any copies or printouts of this e-mail as well as any attachments. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <https://puck.nether.net/pipermail/outages/attachments/20140812/8376270e/attachment-0001.html> > > ------------------------------ > > Message: 5 > Date: Tue, 12 Aug 2014 15:20:45 +0000 > From: "Hoyle Anderson (AM)" <[email protected]> > To: "[email protected]" <[email protected]> > Subject: [outages] Major outages today, not much info at this time > Message-ID: > > <7c6fa1d8224c7944a005e6587bdea50a03f...@usispsvexdb03.na.didata.local> > Content-Type: text/plain; charset="utf-8" > > I know this isn?t much help, but there are major problems with multiple ISPs > since around 4-5 AM EST. I really don?t have much detail, but I have sites > that are unreachable from some providers. Looks like Comcast, level3, ATT, > cogent, etc. > > So, it?s probably not just you, but I?m afraid I don?t know who it is. I > heard one report of a datacenter outage. > > Sources: > http://downdetector.com/ > https://twitter.com/liquidweb > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <https://puck.nether.net/pipermail/outages/attachments/20140812/fa57b850/attachment-0001.html> > > ------------------------------ > > Message: 6 > Date: Tue, 12 Aug 2014 17:42:05 +0200 > From: Teun Vink <[email protected]> > To: "Hoyle Anderson (AM)" <[email protected]> > Cc: "[email protected]" <[email protected]> > Subject: Re: [outages] Major outages today, not much info at this time > Message-ID: <1407858125.27157.21.camel@moridin> > Content-Type: text/plain; charset="UTF-8" > > On di, 2014-08-12 at 15:20 +0000, Hoyle Anderson (AM) via Outages wrote: >> I know this isn?t much help, but there are major problems with >> multiple ISPs since around 4-5 AM EST. I really don?t have much >> detail, but I have sites that are unreachable from some providers. >> Looks like Comcast, level3, ATT, cogent, etc. >> >> >> >> So, it?s probably not just you, but I?m afraid I don?t know who it is. >> I heard one report of a datacenter outage. >> > > Hi, > > Some routing tables hit 512K routes today. Some old hardware and software > can't handle that and either crash or ignore newly learned routes. So this > may cause some disturbances in the force. > > HTH, > Teun > > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Outages mailing list > [email protected] > https://puck.nether.net/mailman/listinfo/outages > > > ------------------------------ > > End of Outages Digest, Vol 75, Issue 9 > ************************************** > > _______________________________________________ > Outages mailing list > [email protected] > https://puck.nether.net/mailman/listinfo/outages > > _______________________________________________ > Outages mailing list > [email protected] > https://puck.nether.net/mailman/listinfo/outages > > > _______________________________________________ > Outages mailing list > [email protected] > https://puck.nether.net/mailman/listinfo/outages > _______________________________________________ Outages mailing list [email protected] https://puck.nether.net/mailman/listinfo/outages
