RE: Dual Homed BGP for failover
Thanks to all for the responses, certainly illuminating. I'm now more aware of what I can do and what tools are available. The following makes sense to me: - Take full routing tables and default from both ISPs and decide how I filter the routes that get installed in my routers. - Originally apply the same filters on both and monitor the links to see what the natural distribution is, when we let the BGP process decide how the traffic is routed. Need to think more about which filters to apply here, the SRX210s are quoted as having capacity for 16k routes. - Once we have a better idea of the traffic profiles start changing the filters to preference certain traffic over the higher speed link. One way this might be done, is to filter based on RIPE or ARIN addresses. We are most concerned about maintaining capacity for European traffic, so install RIPE routes on the higher capacity link and ARIN routes on the lower capacity links. - Accept that we are never going to get an ideal distribution of traffic and continue monitoring and adjusting local pref/prepends etc. as and when we need to change the distribution of traffic. Hopefully we don't need to do this that often. Thoughts? Ahmed From: Max Pierson [mailto:nmaxpier...@gmail.com] Sent: 18 January 2011 21:30 To: Jack Carrozzo Cc: Jack Bates; ayousuf0...@gmail.com; nanog group Subject: Re: Dual Homed BGP for failover Me 3's commit confirmed ... maybe someone from Cisco should be watching :) On Tue, Jan 18, 2011 at 3:21 PM, Jack Carrozzo j...@crepinc.com wrote: Yep, the great thing about IOS without 'commit confirmed' is when you remove a bgp filter, it runs out of memory, reboots, brings up peers, runs out of memory, reboots... meanwhile if you're trying to get in over a public interface you're cursing John Chamber's very existence. Not that that's ever happened to me of course... -Jack Carrozzo On Tue, Jan 18, 2011 at 4:19 PM, Jack Bates jba...@brightok.net wrote: On 1/18/2011 3:03 PM, Jack Carrozzo wrote: I don't think this is the case, on IOS at least. Some years ago I was rocking some 7500s with $not_enough ram for multiple full tables, but with a prefix list to accept le 23 they worked fine. On JunOS, I know I can view pre and post filtered bgp updates ingress and egress. I seem to recall seeing similar functionality introduced into IOS, though I'm less certain. It's still always advisable to be careful. :) Jack
RE: Dual Homed BGP for failover
On Wed, 19 Jan 2011 10:23:47 -, Ahmed Yousuf wrote - Accept that we are never going to get an ideal distribution of traffic and continue monitoring and adjusting local pref/prepends etc. as and when we need to change the distribution of traffic. Hopefully we don't need to do this that often. ^ This. You're fighting a loosing battle with such slow links. Given the limited route capacity of your router you might as well set up statics aimed at each link and forget about BGP shaping. Just keep a floating default pointed at each peer. -Randy
RE: Dual Homed BGP for failover
We're doing BGP to announce our PI space and make sure that our PI space is reachable through both ISPs in case one link goes down. This is the primary need to do the BGP here. Unfortunately my boss has requested that we make use of the capacity of both links, rather than pref traffic out of the higher capacity link. -Original Message- From: Randy McAnally [mailto:r...@fast-serv.com] Sent: 19 January 2011 14:00 To: Ahmed Yousuf; 'nanog group' Subject: RE: Dual Homed BGP for failover On Wed, 19 Jan 2011 10:23:47 -, Ahmed Yousuf wrote - Accept that we are never going to get an ideal distribution of traffic and continue monitoring and adjusting local pref/prepends etc. as and when we need to change the distribution of traffic. Hopefully we don't need to do this that often. ^ This. You're fighting a loosing battle with such slow links. Given the limited route capacity of your router you might as well set up statics aimed at each link and forget about BGP shaping. Just keep a floating default pointed at each peer. -Randy
RE: Dual Homed BGP for failover (Ahmed Yousuf)
On 2011/01/19 5:28 PM, nanog-requ...@nanog.org nanog-requ...@nanog.org wrote: Send NANOG mailing list submissions to nanog@nanog.org To subscribe or unsubscribe via the World Wide Web, visit https://mailman.nanog.org/mailman/listinfo/nanog or, via email, send a message with subject or body 'help' to nanog-requ...@nanog.org You can reach the person managing the list at nanog-ow...@nanog.org When replying, please edit your Subject line so it is more specific than Re: Contents of NANOG digest... Today's Topics: 1. Re: NAT-PT or NAT64 in real life (jarod smith) 2. Re: Software DNS hghi availability and load balancer solution (Joe Greco) 3. Re: Software DNS hghi availability and load balancer solution (Joe Abley) 4. Re: Software DNS hghi availability and load balancer solution (InterNetX - J?rgen Gotteswinter) 5. Re: Network Simulators (Ryan Shea) 6. RE: Network Simulators (Gary Gladney) 7. RE: Dual Homed BGP for failover (Randy McAnally) 8. Re: Network Simulators (Carlos Martinez-Cagnazzo) 9. RE: Dual Homed BGP for failover (Ahmed Yousuf) -- Message: 1 Date: Wed, 19 Jan 2011 13:02:33 +0100 From: jarod smith jarod.smo...@gmail.com Subject: Re: NAT-PT or NAT64 in real life To: nanog@nanog.org Message-ID: aanlkting2sossk-ynlovksps4ntrjewcq+itvwkhr...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Thanks for your reply. In summary it's not possible to deployed IPv6 only if I want to access the whole internet :) On Wed, Jan 19, 2011 at 10:18 AM, jarod smith jarod.smo...@gmail.comwrote: Although it would seem that double-stack is still the preferred method of linux distribution, I want my next deployed in IPv6 only. For linux there is NAT-PT tomicki and NAT64 Viagenie. I don't have Cisco equipment although I'd like tested their NAT-PT, even if it's obsolete. Are some of you have installed one of these two implementations in production on recent versions of linux? Is it stable, secure, ... ? Regards -- Message: 2 Date: Wed, 19 Jan 2011 07:17:07 -0600 (CST) From: Joe Greco jgr...@ns.sol.net Subject: Re: Software DNS hghi availability and load balancer solution To: p...@paulgraydon.co.uk (Paul Graydon) Cc: nanog@nanog.org Message-ID: 201101191317.p0jdh74h076...@aurora.sol.net Content-Type: text/plain; charset=us-ascii On 01/18/2011 07:42 AM, Sergey Voropaev wrote: Does any one know software sollutions (free is preferable) like as cisco GSS and F5 BIG-IP? The main point is that DNS-server (or dns server plugin) must be able to monitor server availability (for example by TCP connect) and from DNS-reply depends on it. I know that it is possible by BIND with set of script. But we are trying to find more usable solution with frendly interface. Thanks a lot. If you want to get fancy you could try an Anycast DNS setup, using GNU's Zebra tool to automatically alter routing tables. http://www.netlinxinc.com/netlinx-blog/45-dns/118-introduction-to-anycast -dns.html You wouldn't use Zebra; it isn't actively developed anymore and has not been updated in many years. Use Quagga instead, which is the community-based offshoot. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples. -- Message: 3 Date: Wed, 19 Jan 2011 08:23:09 -0500 From: Joe Abley jab...@hopcount.ca Subject: Re: Software DNS hghi availability and load balancer solution To: Joe Greco jgr...@ns.sol.net Cc: nanog@nanog.org Message-ID: b3aba767-d8dc-4806-a127-ad0bd5138...@hopcount.ca Content-Type: text/plain; charset=us-ascii On 2011-01-19, at 08:17, Joe Greco wrote: You wouldn't use Zebra; it isn't actively developed anymore and has not been updated in many years. Use Quagga instead, which is the community-based offshoot. I don't think this is what the original post was asking about, but for the sake of completeness other alternatives to Zebra/Quagga (when using BGP between anycast origin servers and adjacent routers, e.g. with multipath configured on the routers) are OpenBGPd and BIRD. See earlier suggestions for bedtime reading, also: http://www.merit.edu/mail.archives/nanog/msg06970.html. Joe -- Message: 4 Date: Wed, 19 Jan 2011 14:27:52 +0100 From: InterNetX - J?rgen Gotteswinter juergen.gotteswin...@internetx.de Subject: Re: Software DNS hghi availability and load balancer solution To: nanog@nanog.org Message-ID: 4d36e6d8.9000...@internetx.de Content-Type: text/plain; charset=ISO-8859-1; format=flowed Am 19.01.11 01:01, schrieb david raistrick: On 01/18/2011 09:42 AM, Sergey Voropaev wrote
RE: Dual Homed BGP for failover
On Wed, 19 Jan 2011 14:26:32 -, Ahmed Yousuf wrote We're doing BGP to announce our PI space and make sure that our PI space is reachable through both ISPs in case one link goes down. This is the primary need to do the BGP here. Unfortunately my boss has requested that we make use of the capacity of both links, rather than pref traffic out of the higher capacity link. Understood! you would _still_ take default BGP routes, I was implying more along the lines (in cisco speak): ! Tweak as necessary to get a good balance ip route 0.0.0.0 128.0.0.0 peer1 ip route 128.0.0.0 128.0.0.0 peer2 Set up SLA tracking on the peer IPs to retract the routes if either peer goes down. Either that or get more RAM on your router and go the BGP-only method. -Randy
Re: Dual Homed BGP for failover
You can just accept directly-connected peers from each network (or within 2 AS's, etc) then point a default at each one with different preferences. You can do with with two edges if you like also: iBGP between the edges, and push default into OSPF from both. WRT dynamic load balancing... generally if your network is large enough for two upstreams you'll have a pretty good distribution of flows so once you get the prefs and prepends setup the way you like, thing won't shift that rapidly. In my experience at least... -Jack Carrozzo On Tue, Jan 18, 2011 at 1:32 PM, Ahmed Yousuf ayousuf0...@gmail.com wrote: Hi, I'm looking at a setup where we use BGP to announce PI space to two upstream ISPs. ISP A provides a 30Mb/s connection and ISP B provides a 10Mb/s. Originally the plan was to use ISP B's link as a backup and local pref traffic outbound via ISP A and pref inbound using AS prepend via ISP A. It has now been requested to be able to distribute traffic across both links rather than preference traffic to the higher speed link. We are going to be using Juniper SRX210s to do this. I have some questions: - Is this really a good idea, as the BGP process won't care what the utilisation of the links are and you will see situations where the lower speed link gets used even though the high speed link utilisation is 0? - If we are doing this, I don't want to take a full routing table, I would rather just take the ISPs routes and perhaps their connected customers. One ISP has said they will only provide full routing table or default. I really don't want to take a full table, is receiving default only going to be a problem for my setup? - Any advice on how to avoid situations where the low bandwidth link is being used even though there is 0 utilisation on the high bandwidth link? Thanks Ahmed
Re: Dual Homed BGP for failover
You really limit yourself when you just take a default from a provider. If you take 2 default's (one from each provider) for whatever reason, once you change the local pref on one of them, it's all your traffic outbound or none. I always request a full table + default, so you can filter to best suit your needs. This way, you can just accept /8's and get some sort of balancing at least (even if you just say all even /8's pref'd on one gateway and all odd /8's from the other provider, etc). Of course this won't be symmetrical, but thats the nature eBGP on the internet. You'll have to watch it and adjust as needed so that you won't saturate your slower link. Max On Tue, Jan 18, 2011 at 12:32 PM, Ahmed Yousuf ayousuf0...@gmail.comwrote: Hi, I'm looking at a setup where we use BGP to announce PI space to two upstream ISPs. ISP A provides a 30Mb/s connection and ISP B provides a 10Mb/s. Originally the plan was to use ISP B's link as a backup and local pref traffic outbound via ISP A and pref inbound using AS prepend via ISP A. It has now been requested to be able to distribute traffic across both links rather than preference traffic to the higher speed link. We are going to be using Juniper SRX210s to do this. I have some questions: - Is this really a good idea, as the BGP process won't care what the utilisation of the links are and you will see situations where the lower speed link gets used even though the high speed link utilisation is 0? - If we are doing this, I don't want to take a full routing table, I would rather just take the ISPs routes and perhaps their connected customers. One ISP has said they will only provide full routing table or default. I really don't want to take a full table, is receiving default only going to be a problem for my setup? - Any advice on how to avoid situations where the low bandwidth link is being used even though there is 0 utilisation on the high bandwidth link? Thanks Ahmed
RE: Dual Homed BGP for failover
From: Ahmed Yousuf Sent: Tuesday, January 18, 2011 10:32 AM To: nanog@nanog.org Subject: Dual Homed BGP for failover - Is this really a good idea, as the BGP process won't care what the utilisation of the links are and you will see situations where the lower speed link gets used even though the high speed link utilisation is 0? It is possible. But one thing, and I know it is a semantics nit but it is really important. There is no difference in the speed of the links. There is a difference in the capacity of the two but the traffic flows at the same speed across both. That said, have you actually tried seeing what the natural breakdown of the traffic is? Without any AS prepend or local pref adjustment, what is the natural ratio of traffic on the two links? Generally different ISPs have different connectivity and some destinations will be favored via one path and others via the other path. It might be useful to determine how BGP naturally routes things first and then you can get an idea of what needs adjusting. - If we are doing this, I don't want to take a full routing table, I would rather just take the ISPs routes and perhaps their connected customers. One ISP has said they will only provide full routing table or default. I really don't want to take a full table, is receiving default only going to be a problem for my setup? Interesting. Most ISPs offer default, full, or customer routes. You can take a full table but simply filter out any that aren't from your ISPs ASN or within one hop of it and only install the routes that meet those criteria. In addition to using AS prepending, your providers might offer communities that allow you to control redistribution of your routing information to their peers. You might want to tell the ISP on the smaller link not to announce your routes to a major peer. That major peer will now find its path to you via the larger pipe. - Any advice on how to avoid situations where the low bandwidth link is being used even though there is 0 utilisation on the high bandwidth link? If that happens, it would mean that the world does not see your path via the high bandwidth pipe as being an attractive path. As mentioned above, you might be able to append communities to your routes to the lower bandwidth ISP that control how they redistribute your routes. One example might be something like don't redistribute my routes if you see them coming from another source in which case that ISP only redistributes your routes when they don't see the announcement via the high bandwidth provider and effectively acts as a backup outside of their own AS but you would still receive traffic originated within their AS over the low bandwidth connection. Ahmed G
Re: Dual Homed BGP for failover
On Tue, Jan 18, 2011 at 1:32 PM, Ahmed Yousuf ayousuf0...@gmail.com wrote: It has now been requested to be able to distribute traffic across both links rather than preference traffic to the higher speed link. - Is this really a good idea, as the BGP process won't care what the utilisation of the links are and you will see situations where the lower speed link gets used even though the high speed link utilisation is 0? Hi Ahmed, This really isn't an either/or situation. You can prefer the higher speed link without excluding the lower speed link. One common way to do this (there are better ones but this one is easy) is to prepend the AS path you send and receive on the lower speed link so that it's longer. - If we are doing this, I don't want to take a full routing table, I would rather just take the ISPs routes and perhaps their connected customers. One ISP has said they will only provide full routing table or default. I really don't want to take a full table, is receiving default only going to be a problem for my setup? IMO, that would be a mistake. Taking significantly less than a full table severely limits your options for balancing traffic between the links. - Any advice on how to avoid situations where the low bandwidth link is being used even though there is 0 utilisation on the high bandwidth link? Any particular communication is either going to go through one link or the other. I'm generalizing here, ignoring some subtleties, but if packets between two particular hosts have picked the low speed link, they will take that one instead of the high speed link. So in a sense it isn't possible to prevent that situation. However, you can adjust the preferences for one path versus the other so that you're not leaving either circuit underused overall and the disparity between your circuits (30 and 10) is not enough to cause major performance issues in and of itself. Regards, Bill Herrin -- William D. Herrin her...@dirtside.com b...@herrin.us 3005 Crane Dr. .. Web: http://bill.herrin.us/ Falls Church, VA 22042-3004
Re: Dual Homed BGP for failover
On 1/18/2011 1:00 PM, William Herrin wrote: IMO, that would be a mistake. Taking significantly less than a full table severely limits your options for balancing traffic between the links. It should also be noted that taking a full table, doesn't mean you have to use the full table. Apply filters to smaller routes or long ASPATHs that you don't want, and then assign preferences, communities, prepends, etc as necessary for the routes you actually accept. This means your sync time is longer and you'll have more updates, but it will still keep the local routing table much lower. Jack
RE: Dual Homed BGP for failover
Someone should advise him that if he wants to take in a full BGP routing table that he makes sure his router can handle it! I would hate for him to open the floodgates and his production router shuts down. LOL Date: Tue, 18 Jan 2011 13:12:18 -0600 From: jba...@brightok.net To: b...@herrin.us Subject: Re: Dual Homed BGP for failover CC: ayousuf0...@gmail.com; nanog@nanog.org On 1/18/2011 1:00 PM, William Herrin wrote: IMO, that would be a mistake. Taking significantly less than a full table severely limits your options for balancing traffic between the links. It should also be noted that taking a full table, doesn't mean you have to use the full table. Apply filters to smaller routes or long ASPATHs that you don't want, and then assign preferences, communities, prepends, etc as necessary for the routes you actually accept. This means your sync time is longer and you'll have more updates, but it will still keep the local routing table much lower. Jack
RE: Dual Homed BGP for failover
-Original Message- From: Brandon Kim Sent: Tuesday, January 18, 2011 11:57 AM To: jba...@brightok.net; b...@herrin.us Cc: ayousuf0...@gmail.com; nanog group Subject: RE: Dual Homed BGP for failover Someone should advise him that if he wants to take in a full BGP routing table that he makes sure his router can handle it! I would hate for him to open the floodgates and his production router shuts down. LOL One can take a full feed but filter so only a subset of the routes are actually installed. For example, filter all routes that are more than one AS away from the immediate upstream.
Re: Dual Homed BGP for failover
On 1/18/2011 2:05 PM, George Bonser wrote: One can take a full feed but filter so only a subset of the routes are actually installed. For example, filter all routes that are more than one AS away from the immediate upstream. You should still be careful, as most processors keep a copy of filtered routes as well, so while your forwarding table may not increase, your route processor memory most likely will. I haven't checked, but I presume IOS and Junos have a knob to disable this feature? Jack
Re: Dual Homed BGP for failover
On Tue, Jan 18, 2011 at 3:57 PM, Jack Bates jba...@brightok.net wrote: You should still be careful, as most processors keep a copy of filtered routes as well, so while your forwarding table may not increase, your route processor memory most likely will. I don't think this is the case, on IOS at least. Some years ago I was rocking some 7500s with $not_enough ram for multiple full tables, but with a prefix list to accept le 23 they worked fine. -Jack Carrozzo
Re: Dual Homed BGP for failover
On 1/18/2011 3:03 PM, Jack Carrozzo wrote: I don't think this is the case, on IOS at least. Some years ago I was rocking some 7500s with $not_enough ram for multiple full tables, but with a prefix list to accept le 23 they worked fine. On JunOS, I know I can view pre and post filtered bgp updates ingress and egress. I seem to recall seeing similar functionality introduced into IOS, though I'm less certain. It's still always advisable to be careful. :) Jack
Re: Dual Homed BGP for failover
Yep, the great thing about IOS without 'commit confirmed' is when you remove a bgp filter, it runs out of memory, reboots, brings up peers, runs out of memory, reboots... meanwhile if you're trying to get in over a public interface you're cursing John Chamber's very existence. Not that that's ever happened to me of course... -Jack Carrozzo On Tue, Jan 18, 2011 at 4:19 PM, Jack Bates jba...@brightok.net wrote: On 1/18/2011 3:03 PM, Jack Carrozzo wrote: I don't think this is the case, on IOS at least. Some years ago I was rocking some 7500s with $not_enough ram for multiple full tables, but with a prefix list to accept le 23 they worked fine. On JunOS, I know I can view pre and post filtered bgp updates ingress and egress. I seem to recall seeing similar functionality introduced into IOS, though I'm less certain. It's still always advisable to be careful. :) Jack
Re: Dual Homed BGP for failover
Me 3's commit confirmed ... maybe someone from Cisco should be watching :) On Tue, Jan 18, 2011 at 3:21 PM, Jack Carrozzo j...@crepinc.com wrote: Yep, the great thing about IOS without 'commit confirmed' is when you remove a bgp filter, it runs out of memory, reboots, brings up peers, runs out of memory, reboots... meanwhile if you're trying to get in over a public interface you're cursing John Chamber's very existence. Not that that's ever happened to me of course... -Jack Carrozzo On Tue, Jan 18, 2011 at 4:19 PM, Jack Bates jba...@brightok.net wrote: On 1/18/2011 3:03 PM, Jack Carrozzo wrote: I don't think this is the case, on IOS at least. Some years ago I was rocking some 7500s with $not_enough ram for multiple full tables, but with a prefix list to accept le 23 they worked fine. On JunOS, I know I can view pre and post filtered bgp updates ingress and egress. I seem to recall seeing similar functionality introduced into IOS, though I'm less certain. It's still always advisable to be careful. :) Jack
Re: Dual Homed BGP for failover
On Tue, Jan 18, 2011 at 12:05 PM, George Bonser gbon...@seven.com wrote: -Original Message- From: Brandon Kim Sent: Tuesday, January 18, 2011 11:57 AM To: jba...@brightok.net; b...@herrin.us Cc: ayousuf0...@gmail.com; nanog group Subject: RE: Dual Homed BGP for failover One can take a full feed but filter so only a subset of the routes are actually installed. For example, filter all routes that are more than one AS away from the immediate upstream. I remember in IOS the BGP config should not have soft-reconfiguration inbound for this uplink session, otherwise routing-engine will still keep one copy of full table in memory. -- Michel~
