Location of upstream connections BGP templates
Hey all, I've got a couple of questions that I'd like operational feedback about. . Although we're an ISP, we currently are only an access provider. We don't yet provide any transit services, but the requirement for us to do so may creep up on a very small scale shortly. Nonetheless... I'm on the latter stages of transforming our network from flat to layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I can eliminate ACL/filtering etc from the core, and push it ALL out toward the edge, keeping the core as fast, sleek and maintenance-free as possible. I visualize my transit providers as essentially 'access', not part of my core backbone. What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but I can't see how that makes sense for any provider that provides (mostly) access services. I'm looking for feedback from both large and small providers, just to gain some perspective. Another question, along the same lines, due to recent discussions, I've done a great deal of research on BGP templates, and now want to migrate to them from peer-group. Before I waste lab time configuring things, I just want to ask for feedback based on experience on whether the following makes sense/will work for transition: - configure template structure - 'no' a single neighbour - apply templates to neighbour - the neighbour comes back up - wash, rinse, repeat Steve
Re: Location of upstream connections BGP templates
--- st...@ibctech.ca wrote: From: Steve Bertrand st...@ibctech.ca layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but Border, core, access. Border routers only connect the core to the upstreams. They do nothing else. No acls, just prefix filters. For example, block 1918 space from leaving your network. Block other bad stuff from leaving your network too. Allow in only what you're expecting from the upstream; again 1918 space, etc. They can fat finger like anyone else. Core is for moving bits as efficiently as possible: no acls; no filters. Connect downstream BGP customers to access routers that participate in the iBGP mesh. Filter them only allowing what they're supposed to advertise. They'll mess it up a lot if they're like my customers by announcing everything under the sun. Filter what you're announcing to them. You can fat finger just as well as anyone else. ;-) scott
Re: Location of upstream connections BGP templates
Border/Core/Access is great thinking when your a sales rep for a vendor that sells under power kit. No reason for it any more. -jim On Wed, Feb 17, 2010 at 8:38 PM, Scott Weeks sur...@mauigateway.com wrote: --- st...@ibctech.ca wrote: From: Steve Bertrand st...@ibctech.ca layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but Border, core, access. Border routers only connect the core to the upstreams. They do nothing else. No acls, just prefix filters. For example, block 1918 space from leaving your network. Block other bad stuff from leaving your network too. Allow in only what you're expecting from the upstream; again 1918 space, etc. They can fat finger like anyone else. Core is for moving bits as efficiently as possible: no acls; no filters. Connect downstream BGP customers to access routers that participate in the iBGP mesh. Filter them only allowing what they're supposed to advertise. They'll mess it up a lot if they're like my customers by announcing everything under the sun. Filter what you're announcing to them. You can fat finger just as well as anyone else. ;-) scott
Re: Location of upstream connections BGP templates
--- deles...@gmail.com wrote: From: jim deleskie deles...@gmail.com Border/Core/Access is great thinking when your a sales rep for a vendor that sells under power kit. No reason for it any more. --- I'd imagine it depends on the network size. Collapsing functionality is doable for smaller sizes. Please elaborate. scott
Re: Location of upstream connections BGP templates
I've done it much larger then small networks. Dependency is much more related to gear used then network size. -jim On Wed, Feb 17, 2010 at 8:46 PM, Scott Weeks sur...@mauigateway.com wrote: --- deles...@gmail.com wrote: From: jim deleskie deles...@gmail.com Border/Core/Access is great thinking when your a sales rep for a vendor that sells under power kit. No reason for it any more. --- I'd imagine it depends on the network size. Collapsing functionality is doable for smaller sizes. Please elaborate. scott
Re: Location of upstream connections BGP templates
Ditto Sent from my iPhone On Feb 17, 2010, at 7:38 PM, Scott Weeks sur...@mauigateway.com wrote: --- st...@ibctech.ca wrote: From: Steve Bertrand st...@ibctech.ca layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but Border, core, access. Border routers only connect the core to the upstreams. They do nothing else. No acls, just prefix filters. For example, block 1918 space from leaving your network. Block other bad stuff from leaving your network too. Allow in only what you're expecting from the upstream; again 1918 space, etc. They can fat finger like anyone else. Core is for moving bits as efficiently as possible: no acls; no filters. Connect downstream BGP customers to access routers that participate in the iBGP mesh. Filter them only allowing what they're supposed to advertise. They'll mess it up a lot if they're like my customers by announcing everything under the sun. Filter what you're announcing to them. You can fat finger just as well as anyone else. ;-) scott
Re: Location of upstream connections BGP templates
On 2010.02.17 19:38, Scott Weeks wrote: --- st...@ibctech.ca wrote: layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but Border, core, access. Border routers only connect the core to the upstreams. They do nothing else. No acls, just prefix filters. For example, block 1918 space from leaving your network. Block other bad stuff from leaving your network too. Allow in only what you're expecting from the upstream; again 1918 space, etc. They can fat finger like anyone else. This was my thought. However... no fat-finger accidents using Team Cymru in conjunction with my internal RTBH triggers with uRPF enabled on every single 'edge' interface ;) This was the basis of my original question. I want to keep this setup at edge-only, and don't want to have to apply it within the core gear. Core is for moving bits as efficiently as possible: no acls; no filters. ...which is what I visualize, and essentially want. Connect downstream BGP customers to access routers that participate in the iBGP mesh. Filter them only allowing what they're supposed to advertise. They'll mess it up a lot if they're like my customers by announcing everything under the sun. Filter what you're announcing to them. You can fat finger just as well as anyone else. ;-) I guess I see 'border' and 'access' as the same. Fat-fingering is important to me. My pref-list is created long before I turn up a BGP session to a client, and the peering is tested internally before I allow them to advertise anything (or I advertise anything to them). At this point, I only have one _true_ peer that advertises their space directly to me, and it is tied down to the last bit. I even informed them that I will perform max path, so they will drop if they break it. Not scalable for multiple clients, but I've learnt a lot. I need to learn now how to scale, which is why the second half of my question dealt with templates. One template, less chance for me to fat-finger :) Cheers, STeve
Re: Location of upstream connections BGP templates
On Feb 17, 2010, at 7:10 PM, Steve Bertrand wrote: Hey all, I've got a couple of questions that I'd like operational feedback about. . Although we're an ISP, we currently are only an access provider. We don't yet provide any transit services, but the requirement for us to do so may creep up on a very small scale shortly. Nonetheless... I'm on the latter stages of transforming our network from flat to layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I can eliminate ACL/filtering etc from the core, and push it ALL out toward the edge, keeping the core as fast, sleek and maintenance-free as possible. I visualize my transit providers as essentially 'access', not part of my core backbone. One of the challenges is how do you decide if something is core vs access. If both are the same speed, is there a reason to keep them on different devices? How do you aggregate your customers if they are the same speed as your core? Are there points of savings? I don't know if you're doing T1 aggregation or 10GE, so this is hard to speculate, but I honestly would not spend a lot of time talking to people that have different buckets for a device class. What is core today is always edge in the future. peering edge customer edge core Mean different things to different networks/people. Some see value, others see excess. What do other providers do? Are your transit peers connected directly to the core? I can understand such a setup for transit-only providers, but I can't see how that makes sense for any provider that provides (mostly) access services. I'm looking for feedback from both large and small providers, just to gain some perspective. Another question, along the same lines, due to recent discussions, I've done a great deal of research on BGP templates, and now want to migrate to them from peer-group. Before I waste lab time configuring things, I just want to ask for feedback based on experience on whether the following makes sense/will work for transition: - configure template structure - 'no' a single neighbour - apply templates to neighbour - the neighbour comes back up - wash, rinse, repeat I've done some examples of templates/community based route filtering here: http://puck.nether.net/bgp/ The examples for Cisco and Juniper should help you create a policy that is sane for your network, or at least something to keep you from leaking transit-learned routes to another one of your transits. (This is very common). - Jared
Re: Location of upstream connections BGP templates
On 2010.02.17 19:41, jim deleskie wrote: Border/Core/Access is great thinking when your a sales rep for a vendor that sells under power kit. No reason for it any more. Hi Jim, Unfortunately, I have a mix of EOL Cisco gear in my network, along with other random custom-built software routers, HP Procurve switches etc. To be honest, I am very pleased with what I've learnt over the course of the last two years with my network re-design/build. In my environment, the layered approach is working exceptionally well (and my sales skills would have me recommend a different ISP at the drop of a dime if they could provide what I couldn't ;) Primarily, my transition has led me down the BCP 38 path (and it's associated techniques/side-effects, specifically automated S/RTBH), which aside from IPv6, is the most important thing I believe that I could have accomplished during that time. It would, however, be interesting to learn how the former drawbacks of flat networks have evolved, and what new technologies make them successful once again. Thanks, Steve
Re: Location of upstream connections BGP templates
On 2010.02.17 20:19, Jared Mauch wrote: On Feb 17, 2010, at 7:10 PM, Steve Bertrand wrote: Hey all, I've got a couple of questions that I'd like operational feedback about. . Although we're an ISP, we currently are only an access provider. We don't yet provide any transit services, but the requirement for us to do so may creep up on a very small scale shortly. Nonetheless... I'm on the latter stages of transforming our network from flat to layered. My thinking is that my 'upstream' connections should be moved out of the core, and onto the edge. My reasoning for this is so that I can eliminate ACL/filtering etc from the core, and push it ALL out toward the edge, keeping the core as fast, sleek and maintenance-free as possible. I visualize my transit providers as essentially 'access', not part of my core backbone. One of the challenges is how do you decide if something is core vs access. If both are the same speed, is there a reason to keep them on different devices? Hi Jared, They are not at the same speed. Typically, the majority of the 'access' or 'edge' is 100Mb, whereas the 'core' consists of 1Gb up to four 1Gb agg links. How do you aggregate your customers if they are the same speed as your core? They are not. For instance, I have an SDSL network where max theoretical speeds for each connection is 2048Kb. I consolidate all of these modems/banks into Cat 2900 switches (or equivalent), which terminate into a 2691 (or equivalent) router. The 2961 routers connect directly to two core routers, providing redundancy across the network. Most of these SDSL clients also have a fibre feed out of our same building, advertising address space assigned by us back to us via BGP, with the SDSL as backup-only. The fibre links are 100Mb each. The fibre from the clients terminate within a building down the street, and we have 10 such clients on a pair of fibre. Each pair of fibre run across Gig transceivers to our building. Each client is within it's own vlan, 10 vlans connect to 10 sub-ints on a router from the switch. This 'access' router then is LACP (generally 2gi) to each 'core'. The 'cores' have multi-gig feeds into the 'access' areas that we host. Each 'access' router (or edge as I refer to it as) protects my network with uRPF etc etc. Are there points of savings? I don't know. Keeping all filtering at the edge saves me much time and much effort. BGP templates will also help. The question is, has it helped...yes, it has, tremendously. Flat or layered, it doesn't take me long anymore to identify points of congestion. The project also has helped me identify what I need to express to my large upstream engineers whenever I come under a direct DDoS, as to save *them* time. I don't know if you're doing T1 aggregation or 10GE, so this is hard to speculate, but I honestly would not spend a lot of time talking to people that have different buckets for a device class. What is core today is always edge in the future. peering edge customer edge core Mean different things to different networks/people. Some see value, others see excess. Agreed. I figured that. I can easily see now that edges are different whether you are a transit provider, access provider etc... Another question, along the same lines, due to recent discussions, I've done a great deal of research on BGP templates, and now want to migrate to them from peer-group. Before I waste lab time configuring things, I just want to ask for feedback based on experience on whether the following makes sense/will work for transition: - configure template structure - 'no' a single neighbour - apply templates to neighbour - the neighbour comes back up - wash, rinse, repeat I've done some examples of templates/community based route filtering here: http://puck.nether.net/bgp/ The examples for Cisco and Juniper should help you create a policy that is sane for your network, or at least something to keep you from leaking transit-learned routes to another one of your transits. (This is very common). Yeah, I know it's common. I can't stand seeing my filtering system sinking/holing BOGON, or more specifically my own IP space that is coming back to me. I'm all for being a good net citizen, and am willing to do whatever it takes to ensure that. I guess my question should have been whether I should move my transit providers to the perimeter instead :) Thanks Jared for the feedback, and the link to the templates. Steve
Re: Location of upstream connections BGP templates
Of course all designs are limited to the budget you have to build the network :) On Wed, Feb 17, 2010 at 9:20 PM, Steve Bertrand st...@ibctech.ca wrote: On 2010.02.17 19:41, jim deleskie wrote: Border/Core/Access is great thinking when your a sales rep for a vendor that sells under power kit. No reason for it any more. Hi Jim, Unfortunately, I have a mix of EOL Cisco gear in my network, along with other random custom-built software routers, HP Procurve switches etc. To be honest, I am very pleased with what I've learnt over the course of the last two years with my network re-design/build. In my environment, the layered approach is working exceptionally well (and my sales skills would have me recommend a different ISP at the drop of a dime if they could provide what I couldn't ;) Primarily, my transition has led me down the BCP 38 path (and it's associated techniques/side-effects, specifically automated S/RTBH), which aside from IPv6, is the most important thing I believe that I could have accomplished during that time. It would, however, be interesting to learn how the former drawbacks of flat networks have evolved, and what new technologies make them successful once again. Thanks, Steve
Re: Location of upstream connections BGP templates
On 2010.02.17 20:45, jim deleskie wrote: Of course all designs are limited to the budget you have to build the network :) Heh, yeah, but it's unbelievable what one can learn on an eBay diet when they put their entire heart, soul and dedication into it! Steve
Re: Location of upstream connections BGP templates
Absolutely. I've worked on networks where I'm was amazed on someday we held it all together, but that is truly when you learn the most. -jim On Wed, Feb 17, 2010 at 9:46 PM, Steve Bertrand st...@ibctech.ca wrote: On 2010.02.17 20:45, jim deleskie wrote: Of course all designs are limited to the budget you have to build the network :) Heh, yeah, but it's unbelievable what one can learn on an eBay diet when they put their entire heart, soul and dedication into it! Steve
Re: Location of upstream connections BGP templates
On 2010.02.17 20:48, jim deleskie wrote: Absolutely. I've worked on networks where I'm was amazed on someday we held it all together, but that is truly when you learn the most. I'm very, very happy that there are people out there who can actually see that... Steve
