Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On (2014-06-10 12:39 -0500), Blake Hudson wrote: Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), and the FIB contains only the best path entries from the RIB, wouldn't the FIB be at or below 300k? There is nothing to summarize away from global BGP table, if you have number showing less, it's probably counter bug or misinterpretation. Global BGP table, single BGP feed, will take same amount of RIB and FIB. You can see your FIB use in 'show plat hard capa pfc': #show platform hardware capacity pfc Module FIB TCAM usage: TotalUsed %Used 2 72 bits (IPv4, MPLS, EoM) 884736 712819 81% 144 bits (IP mcast, IPv6) 8192019235 23% You're probably bit better off than I am :) -- ++ytti
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On Wed, Jun 11, 2014 at 1:28 AM, Saku Ytti s...@ytti.fi wrote: On (2014-06-10 12:39 -0500), Blake Hudson wrote: There is nothing to summarize away from global BGP table, if you have number showing less, it's probably counter bug or misinterpretation. Global BGP table, single BGP feed, will take same amount of RIB and FIB. [snip] That depends if by summarize they mean filter prefixes longer than say /22, or otherwise...: discard extraneous prefixes from networks that were allocated a /16 network but chose to deaggregate and advertise every /24 -- choosing to accept only the /16 advertisement instead of installing these extra /24 routes in the FIB, then there are plenty of entries to summarize away. -- -JH
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Matthew Petach wrote the following on 6/10/2014 7:03 PM: On the couple Cisco platforms I have available with full tables, Cisco summarizes BGP by default. Since this thread is talking about Cisco gear, I think it's more topical than results from BIRD. One example from a non-transit AS: ASR#sh ip route sum IP routing table name is default (0x0) IP routing table maximum-paths is 32 Route SourceNetworksSubnets Replicates Overhead Memory (bytes) connected 0 10 0 600 1800 static 1 2 0 180 540 application 0 0 0 0 0 bgp x 164817 330796 0 29736780 89210340 External: 495613 Internal: 0 Local: 0 internal 5799 20123680 Total 170617 330808 0 29737560 109336360 I'm not sure you're reading that correctly. 164817+330796 = 495613 That is, the BGP routing table size is the union of the Networks and the Subnets; it's not magically doing any summarization for you. Matt Thank you Matt for directly addressing my question. My interpretation, which seems likely incorrect, was that smaller announcements could be discarded if there was a covering prefix (that otherwise matched the same AS path and other BGP metrics) and that many smaller prefix announcements could be bundled (again, assuming that all BGP metrics were the same between the prefixes). The numbers I was seeing in my routers for subnets coincided closely with the cidr-report's summzarization numbers http://www.cidr-report.org/as2.0/aggr.html, and I assumed the two used the same logic (not magic) to calculate how to reduce routes without losing any routing functionality. Your explanation that I was simply interpreting the numbers incorrectly seems the most logical now that I look again. Thanks, --Blake
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
I haven't seen anyone bring up this point yet, but I feel like I'm missing something... I receive a full BGP table from several providers. They send me ~490k *prefixes* each. However, my router shows ~332k *subnets* in the routing table. As I understand it, the BGP table contains duplicate information (for example a supernet is announced as well as all subnets within that supernet) or excess information (prefix is announced as two /17's instead of a single /16) and can otherwise be summarized to save space in the RIB. It appears to me that the weekly CIDR report shows similar numbers: Recent Table History Date PrefixesCIDR Agg 30-05-14502889 283047 31-05-14502961 283069 01-06-14502836 283134 02-06-14502943 283080 03-06-14502793 283382 04-06-14503177 282897 05-06-14503436 283062 06-06-14503988 282999 In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Thanks, --Blake Drew Weaver wrote the following on 5/6/2014 10:39 AM: Hi all, I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. We are at about 94/95% right now of 512K. For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service. Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does. In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources. Just something to think about before it becomes a story the community talks about for the next decade. -Drew
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Hi Blake, On 10 Jun 2014, at 19:04, Blake Hudson bl...@ispn.net wrote: In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes. You can find more information here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. -- There's no sense in being precise when | Łukasz Bromirski you don't know what you're talking | jid:lbromir...@jabber.org about. John von Neumann |http://lukasz.bromirski.net
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 6/10/14, 10:15 AM, Łukasz Bromirski wrote: Hi Blake, On 10 Jun 2014, at 19:04, Blake Hudson bl...@ispn.net wrote: In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes. You can find more information here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. Until you add multi-as multipath and addpath and a couple VRFs and all of sudden FIB budget blows up. signature.asc Description: OpenPGP digital signature
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM: Hi Blake, On 10 Jun 2014, at 19:04, Blake Hudson bl...@ispn.net wrote: In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes. You can find more information here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), and the FIB contains only the best path entries from the RIB, wouldn't the FIB be at or below 300k? --Blake
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Hello, On 10.6.2014 19:04, Blake Hudson wrote: I haven't seen anyone bring up this point yet, but I feel like I'm missing something... I receive a full BGP table from several providers. They send me ~490k *prefixes* each. However, my router shows ~332k *subnets* in the routing table. As I understand it, the BGP table contains duplicate information (for example a supernet is announced as well as all subnets within that supernet) or excess information (prefix is announced as two /17's instead of a single /16) and can otherwise be summarized to save space in the RIB. many people deaggregate their address space purposely, including large networks like Google, Akamai, Netflix etc. Full list for analysis like who does that is available at http://www.cidr-report.org/as2.0/aggr.html These days also some people split their allocated aggregatable space (PA) with different routing policies for each subnet, substituting old PI addresses (at least in RIPE region). Technically nothing blocks this and politically - it's up to each, what accepts. But some unreachable subnet means problems with customers... There's no summarization in hardware/software from RIB to FIB. From the vendor perspective, they would like to sell you new hardware with larger TCAMs/etc, of course... :-) With regards, Daniel smime.p7s Description: S/MIME Cryptographic Signature
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 10 Jun 2014, at 19:39, Blake Hudson bl...@ispn.net wrote: And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), and the FIB contains only the best path entries from the RIB, wouldn't the FIB be at or below 300k? Because you need to do your own summarization or ask your upstreams to do it for you. Until then, most of transit accepts loosely prefixes in exact length but also longer (i.e. /24 but also both /25s). You’ll see more and more deaggregation with the rise of smaller entities fighting for chance to do some traffic engineering, so be prepared to constant rise of prefixes overall. -- There's no sense in being precise when | Łukasz Bromirski you don't know what you're talking | jid:lbromir...@jabber.org about. John von Neumann |http://lukasz.bromirski.net
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 6/10/14, 10:39 AM, Blake Hudson wrote: Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM: Hi Blake, On 10 Jun 2014, at 19:04, Blake Hudson bl...@ispn.net wrote: In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes. You can find more information here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), Unlikely, just because prefixes could be cidr aggregated doesn't mean they are. the more specifics exist for a reason, in the case of deaggrates with no covering anouncement, well not much you're doing with those. your rib should be the sum of all received routes that you did not filter. and the FIB contains only the best path entries from the RIB, wouldn't the FIB be at or below 300k? a live example of rib size from a router with two transit providers. bird show route count 979842 of 979842 routes for 490932 networks a live example of rib size from a router with one ibgp peer with addpath and three upstream transit providers bird show route count 1471242 of 1471242 routes for 491977 networks --Blake signature.asc Description: OpenPGP digital signature
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
joel jaeggli wrote the following on 6/10/2014 1:10 PM: On 6/10/14, 10:39 AM, Blake Hudson wrote: Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM: Hi Blake, On 10 Jun 2014, at 19:04, Blake Hudson bl...@ispn.net wrote: In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes. You can find more information here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), Unlikely, just because prefixes could be cidr aggregated doesn't mean they are. the more specifics exist for a reason, in the case of deaggrates with no covering anouncement, well not much you're doing with those. your rib should be the sum of all received routes that you did not filter. On the couple Cisco platforms I have available with full tables, Cisco summarizes BGP by default. Since this thread is talking about Cisco gear, I think it's more topical than results from BIRD. One example from a non-transit AS: ASR#sh ip route sum IP routing table name is default (0x0) IP routing table maximum-paths is 32 Route SourceNetworksSubnets Replicates OverheadMemory (bytes) connected 0 10 0 600 1800 static 1 2 0 180 540 application 0 0 0 0 0 bgp x 164817 330796 0 2973678089210340 External: 495613 Internal: 0 Local: 0 internal 579920123680 Total 170617 330808 0 29737560109336360 and the FIB contains only the best path entries from the RIB, wouldn't the FIB be at or below 300k? a live example of rib size from a router with two transit providers. bird show route count 979842 of 979842 routes for 490932 networks a live example of rib size from a router with one ibgp peer with addpath and three upstream transit providers bird show route count 1471242 of 1471242 routes for 491977 networks The RIB counts and memory used by the RIB seem to be nearly identical between a Cisco router with 3 full BGP feeds and another with 1 BGP feed. The differences seem to lie in the memory used by BGP for prefix tracking. If your router has multiple routing tables, or your installing multiple routes for the same prefix in the RIB, then I can understand why your RIB will be larger. These are nice features, but certainly not a requirement for everyone. --Blake
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On Tuesday, June 10, 2014 08:07:35 PM Łukasz Bromirski wrote: Because you need to do your own summarization or ask your upstreams to do it for you. Until then, most of transit accepts loosely prefixes in exact length but also longer (i.e. /24 but also both /25s). A couple of major service providers today permit announcements of any length, provided they are registered in an IRR that they use to build filters. Of course, there are no guarantees that prefixes typically longer than general industry practice permits will see the light of day beyond their network. Mark. signature.asc Description: This is a digitally signed message part.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On Tue, Jun 10, 2014 at 11:36 AM, Blake Hudson bl...@ispn.net wrote: joel jaeggli wrote the following on 6/10/2014 1:10 PM: On 6/10/14, 10:39 AM, Blake Hudson wrote: Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM: Hi Blake, On 10 Jun 2014, at 19:04, Blake Hudson bl...@ispn.net wrote: In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes. You can find more information here: http://www.cisco.com/c/en/us/support/docs/switches/ catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), Unlikely, just because prefixes could be cidr aggregated doesn't mean they are. the more specifics exist for a reason, in the case of deaggrates with no covering anouncement, well not much you're doing with those. your rib should be the sum of all received routes that you did not filter. On the couple Cisco platforms I have available with full tables, Cisco summarizes BGP by default. Since this thread is talking about Cisco gear, I think it's more topical than results from BIRD. One example from a non-transit AS: ASR#sh ip route sum IP routing table name is default (0x0) IP routing table maximum-paths is 32 Route SourceNetworksSubnets Replicates OverheadMemory (bytes) connected 0 10 0 600 1800 static 1 2 0 180 540 application 0 0 0 0 0 bgp x 164817 330796 0 2973678089210340 External: 495613 Internal: 0 Local: 0 internal 579920123680 Total 170617 330808 0 29737560109336360 I'm not sure you're reading that correctly. 164817+330796 = 495613 That is, the BGP routing table size is the union of the Networks and the Subnets; it's not magically doing any summarization for you. Matt
RE: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
My 2c: The obvious thing for me is if people are running a full ipv4 route table on a box only just capable of handling one single table of that size, then really now is the time to asses if you really need to hold that table or just drop to default +internal+peers. If you have multiple up streams and you are using the route tables to do your route selection then great, but that means you need at least 1M capability now, and really 2+ should be your target. In my experience people running a full table on a small capability box normally don't actually need to carry it, or they just need a bigger box.
RE: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
FIB is not the same as RIB... Perfectly happy 6509, many paths, only one full table in the FIB: BGP router identifier XXX , local AS number 11404 BGP table version is 40916063, main routing table version 40916063 494649 network entries using 71229456 bytes of memory 886903 path entries using 70952240 bytes of memory 29 multipath network entries and 58 multipath paths -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Tony Wicks Sent: Tuesday, June 10, 2014 6:45 PM To: 'nanog' Subject: RE: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. My 2c: The obvious thing for me is if people are running a full ipv4 route table on a box only just capable of handling one single table of that size, then really now is the time to asses if you really need to hold that table or just drop to default +internal+peers. If you have multiple up streams and you are using the route tables to do your route selection then great, but that means you need at least 1M capability now, and really 2+ should be your target. In my experience people running a full table on a small capability box normally don't actually need to carry it, or they just need a bigger box.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
The datasheet for the ESP-5 states support for 500,000 IPv4 Prefixes. The TCAM on the ASR1k is different from 6500 and can't be adjusted in the same fashion. You'd have to filter or upgrade the ESP. http://www.cisco.com/c/en/us/products/collateral/routers/asr-1000-series-aggregation-services-routers/data_sheet_c78-450070.html(ctrl+f Table 3) On Thu, May 8, 2014 at 10:55 AM, Michael Dikkema mdikk...@gmail.com wrote: I could never get an definitive answer out of TAC or my account team, but I believe the ASR1002 w/ESP5 is also affected. On Thu, May 8, 2014 at 1:15 AM, Nikolay Shopik sho...@inblock.ru wrote: Asr1002-f may have problem as it limited to 512k iirc On 08 мая 2014 г., at 2:45, Shawn L sha...@up.net wrote: Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. -- Forwarded message -- From: Irwin, Kevin kevin.ir...@cinbell.com Date: Wed, May 7, 2014 at 10:39 AM Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. To: nanog@nanog.org nanog@nanog.org I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
RE: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Irwin, Kevin Sent: Wednesday, May 07, 2014 4:39 PM I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. I would actually be very surprised if someone would hit the 512K limit on ASRs. With 6500/7600 I can understand they've been around for ages and no one anticipated the 512k limit back then. But ASRs? When these where bough engineers must have known that 512k is not going to be enough. I guess one does some reading and tweaking before installing a box as a PE or Internet Edge. adam
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On May 9, 2014, at 2:48 PM, Vitkovský Adam adam.vitkov...@swan.sk wrote: With 6500/7600 I can understand they've been around for ages and no one anticipated the 512k limit back then. Actually, it *was* anticipated. It's just that those who designed the ASIC didn't necessarily envision that it would still be in service, having gone through successive additional minor variations, for quite so long. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Luck is the residue of opportunity and design. -- John Milton
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Asr1002-f may have problem as it limited to 512k iirc On 08 мая 2014 г., at 2:45, Shawn L sha...@up.net wrote: Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. -- Forwarded message -- From: Irwin, Kevin kevin.ir...@cinbell.com Date: Wed, May 7, 2014 at 10:39 AM Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. To: nanog@nanog.org nanog@nanog.org I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
It depends, you can put in a table-map to stop the routes from being installed into the FIB/RIB on an ASR-1K with 2GB of RAM you can then have up to 2 million IPv4 routes. Alternatively, if you are not using your ASR-1k to forward traffic, I think you could also just turn off CEF and have the same result. On 5/8/14, 2:15 AM, Nikolay Shopik sho...@inblock.ru wrote: Asr1002-f may have problem as it limited to 512k iirc On 08 мая 2014 г., at 2:45, Shawn L sha...@up.net wrote: Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. -- Forwarded message -- From: Irwin, Kevin kevin.ir...@cinbell.com Date: Wed, May 7, 2014 at 10:39 AM Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. To: nanog@nanog.org nanog@nanog.org I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On Thursday, May 08, 2014 04:45:09 PM Irwin, Kevin wrote: It depends, you can put in a table-map to stop the routes from being installed into the FIB/RIB on an ASR-1K with 2GB of RAM you can then have up to 2 million IPv4 routes. Helpful only if you don't want to forward traffic through the box, in which case running IOS XE on a VM on a server is a more lasting idea :-). Mark. signature.asc Description: This is a digitally signed message part.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
I know most people have problems with 2 bgp feeds and 4GB RAM on ASR1002-F (as it max installable memory). So I doubt about 2M routes with 2GB RAM. On 08.05.2014 18:45, Irwin, Kevin wrote: on an ASR-1K with 2GB of RAM you can then have up to 2 million IPv4 routes
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On Thursday, May 08, 2014 05:29:08 PM Nikolay Shopik wrote: I know most people have problems with 2 bgp feeds and 4GB RAM on ASR1002-F (as it max installable memory). So I doubt about 2M routes with 2GB RAM. I've never ran the ASR1002-F, but I know some other ASR1000 platforms consume half the memory just for the IOS image upon boot. This makes running a second instance of IOSd on boxes that have a single RP a sure way to lead to a crash when the same box is running BGP (happened to me once, 2nd IOSd never again). Mark. signature.asc Description: This is a digitally signed message part.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On Tue, May 06, 2014 at 03:39:13PM +, Drew Weaver wrote: I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. Closer to? Internap announces 507K prefixes to me today. Coupled with the prefixes I carry in iBGP internally, I've been sitting at 511K for quite some time, and at occasions, exceeded 512K in the last 2 weeks. L3 Forwarding Resources FIB TCAM usage: TotalUsed %Used 72 bits (IPv4, MPLS, EoM) 802816 511848 64% IP routing table maximum-paths is 32 Route SourceNetworksSubnets OverheadMemory (bytes) connected 0 31 30124464 static 1 78 17456 11376 ospf 1 1 310 22392 44784 Intra-area: 110 Inter-area: 160 External-1: 0 External-2: 41 NSSA External-1: 0 NSSA External-2: 0 bgp 23456 167707 343507 3680812875466680 External: 507479 Internal: 3735 Local: 0 internal605413246152 Total 173763 343926 3685098888773456 -- Brandon Ewing(nicot...@warningg.com) pgpNYNuoqXpz4.pgp Description: PGP signature
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
I could never get an definitive answer out of TAC or my account team, but I believe the ASR1002 w/ESP5 is also affected. On Thu, May 8, 2014 at 1:15 AM, Nikolay Shopik sho...@inblock.ru wrote: Asr1002-f may have problem as it limited to 512k iirc On 08 мая 2014 г., at 2:45, Shawn L sha...@up.net wrote: Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. -- Forwarded message -- From: Irwin, Kevin kevin.ir...@cinbell.com Date: Wed, May 7, 2014 at 10:39 AM Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. To: nanog@nanog.org nanog@nanog.org I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 05/06/2014 05:39 PM, Drew Weaver wrote: I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. Thanks for this e-mail with clear subject ;) Did anyone yet calculated roughly when the ipv4 routing table will hit 512K ?
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
Fwd: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. -- Forwarded message -- From: Irwin, Kevin kevin.ir...@cinbell.com Date: Wed, May 7, 2014 at 10:39 AM Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. To: nanog@nanog.org nanog@nanog.org I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
www.pssclabs.com On May 7, 2014, at 6:47 PM, Shawn L sha...@up.net wrote: Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. -- Forwarded message -- From: Irwin, Kevin kevin.ir...@cinbell.com Date: Wed, May 7, 2014 at 10:39 AM Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. To: nanog@nanog.org nanog@nanog.org I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
ASR1k doesn't have fixed TCAM like the 6500 and has a little more wiggle room, but it depends on the ESP you have installed. For example ESP 20 supports around 1,000,000 routes. http://www.cisco.com/c/en/us/products/collateral/routers/asr-1000-series-aggregation-services-routers/data_sheet_c78-450070.html?cachemode=refresh -Pete On Wed, May 7, 2014 at 6:45 PM, Shawn L sha...@up.net wrote: Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. -- Forwarded message -- From: Irwin, Kevin kevin.ir...@cinbell.com Date: Wed, May 7, 2014 at 10:39 AM Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. To: nanog@nanog.org nanog@nanog.org I¹m really surprised that most people have not hit this limit already, especially on the 9K¹s, as it seems Cisco has some fuzzy math when it comes to the 512K limit. Also make sure you have spare cards when you reload after changing the scaling, those old cards don¹t always like to come back. On 5/6/14, 7:01 PM, Larry Sheldon larryshel...@cox.net wrote: On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker) The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and destroy any copies of this document.
RE: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Do the ASR1k routers have this issue as well? I searched around but couldn't find any information. Not really (according to Cisco) - ESP10 - 1,000,000 IPv4 or 500,000 IPv6 routes ESP20 - 4,000,000 IPv4 or 4,000,000 IPv6 routes ESP40 - 4,000,000 IPv4 or 4,000,000 IPv6 routes ESP100-4,000,000 IPv4 or 4,000,000 IPv6 routes (hardware is capable of 8,000,000 routes) ESP200-4,000,000 IPv4 or 4,000,000 IPv6 routes
Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Hi all, I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. We are at about 94/95% right now of 512K. For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service. Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does. In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources. Just something to think about before it becomes a story the community talks about for the next decade. -Drew
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 06/05/2014 16:39, Drew Weaver wrote: In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources. to fix the problem on sup720/rsp720: Router(config)#mls cef maximum-routes ip 768 This requires a reload to take effect. If you don't recarve the TCAM and you accidentally hit the maximum number of prefixes, the entire chassis will go into software forwarding mode and will require a reboot to recover. IOW, there is no way of avoiding a reload, so best plan as soon as possible. More info here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/116132-problem-catalyst6500-00.html This problem also affects ASR9000 boxes running typhoon line cards which by default will only set aside 500k slots for ipv4 prefixes. The fix for these boxes is: 0/RSP0/CPU0:router# admin hw-module profile scale l3 ...followed by appropriate line card reloads. more information at: http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregation-services-routers/116999-problem-line-card-00.html Nick
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On Tue, 6 May 2014, Drew Weaver wrote: Hi all, I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. We are at about 94/95% right now of 512K. For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service. Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does. In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources. Just something to think about before it becomes a story the community talks about for the next decade. I've been configuring 6500/Sup7203bxl's with mls cef maximum-routes ip 768 The only gotcha is, you have to reload for that to be effective. Speaking of which, I've had WS-X6708-10GE cards go bad on reload in a couple of 6500s. I see cisco finally released some more info on their bad memory announcement from several months back: http://www.cisco.com/c/en/us/support/docs/field-notices/637/fn63743.html It seems our gone bad 6708s may be included in this issue. If you don't have enough spare ports or spare cards, this puts you in a somewhat precarious situation. You need to reload to affect the v4/v6 route storage change, but you might lose some blades in the process. -- Jon Lewis, MCP :) | I route | therefore you are _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 5/6/2014 11:39 AM, Drew Weaver wrote: Hi all, I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. We are at about 94/95% right now of 512K. For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service. Yes, a Sup720/PFC3CXL defaults to 512K IPv4 routes, and reconfiguring the FIB requires a reload. So I've been quietly expecting a somewhat serious meltdown when we hit 512K :) Jeff
RE: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
-Original Message- From: Nick Hilliard [mailto:n...@foobar.org] Sent: Tuesday, May 6, 2014 12:11 PM To: Drew Weaver; 'nanog@nanog.org' Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. This problem also affects ASR9000 boxes running typhoon line cards which by default will only set aside 500k slots for ipv4 prefixes. The fix for these boxes is: I believe you mean This problem also affects ASR9000 boxes running ...trident... line cards. Please confirm? -Drew
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 06/05/2014 18:01, Drew Weaver wrote: I believe you mean This problem also affects ASR9000 boxes running ...trident... line cards. Please confirm? er, yes, trident cards, not typhoon cards. typhoon cards are not affected by this. Nick
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
I just recently got four sets off eBay. Purportedly genuine Cisco. A shade over $100. Raid the departmental beer fund. :) -r Vlade Ristevski vrist...@ramapo.edu writes: It would probably be a good time to upgrade the memory on my 7206 NPE-G1 as well (512MB). I was going to replace the router but am going to keep it around for the Fall Semester. Anyone know of any good 3rd party memory modules that are equivalent to the MEM-NPE-G1-1GB? I got a quote for the official Cisco ones last summer and it was around $5,000 lol On 5/6/2014 11:39 AM, Drew Weaver wrote: Hi all, I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. We are at about 94/95% right now of 512K. For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service. Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does. In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources. Just something to think about before it becomes a story the community talks about for the next decade. -Drew Vlad
Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
On 5/6/2014 10:39 AM, Drew Weaver wrote: Just something to think about before it becomes a story the community talks about for the next decade. Like we have for the last two? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker)
