Re: Peering + Transit Circuits
On 19/Aug/15 01:12, Patrick W. Gilmore wrote: Normally a router gets a packet and sends it on its way without looking at the source. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. No filters to manage, no RIRs to sync, nothing to code, etc. This is what we do, and to make it more interesting, we have 0/0 and ::/0 on these dedicated peering routers pointing to Null0. Mark.
Re: [c-nsp] Peering + Transit Circuits
On 18/Aug/15 15:31, Tim Durack wrote: Not sure if I really want to get into using DSCP bits for basic IP service though. There are use-cases, but they would mostly be internal. Mark.
Re: [c-nsp] Peering + Transit Circuits
On 18/Aug/15 18:02, Tim Durack wrote: Can I ask why you terminate peering and transit on different routers? (Not suggesting that is bad, just trying to understand the reason.) Easier policy enforcement for us. Lowers the chance of you dealing with traffic in ways you don't intend (although that can always be fixed). Spreading both commercial and technical risk, depending on whether you value transit more than peering, or vice versa. Avoiding kinky things with VRF's. Mark.
Re: [c-nsp] Peering + Transit Circuits
On 25/Aug/15 13:58, Scott Granados wrote: If you’re not enabling URPF at the peering routers and edges how do you handle things like RTBH? D/RTBH still works fine. S/RTBH would be an issue, but one could enable uRPF temporarily for that. Mark.
Re: [c-nsp] Peering + Transit Circuits
On 18/Aug/15 14:29, Tim Durack wrote: Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. We do this. Makes policy enforcement easier. Mark.
Re: [c-nsp] Peering + Transit Circuits
On 18/Aug/15 22:43, Nick Hilliard wrote: i'd advise being careful with this approach: urpf at ixps is a nightmare. We don't generally do uRPF at exchange points, for the simple reason that the router is dedicated (meaning it does not carry a full table), and peers leaking your routes to the Internet (which breaks uRPF in this scenario) is a constant scenario. *sigh* Mark.
Re: Peering + Transit Circuits
On Wed, 19 Aug 2015, Max Tulyev wrote: My solution is: 1. Don't care. 2. If some peer steal your transit, and it is noticeable amount of traffic causing some problems for you - investigate and terminate that peer. You forgot 3. Publicly shame on NANOG so that others will think twice before peering or doing any business with them. -- Jon Lewis, MCP :) | I route | therefore you are _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: Peering + Transit Circuits
My solution is: 1. Don't care. 2. If some peer steal your transit, and it is noticeable amount of traffic causing some problems for you - investigate and terminate that peer. On 18.08.15 15:29, Tim Durack wrote: Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated.
Re: Peering + Transit Circuits
Hi, Max -- On 19/08/2015 17:36, Max Tulyev max...@netassist.ua wrote: My solution is: 1. Don't care. 2. If some peer steal your transit, and it is noticeable amount of traffic causing some problems for you - investigate and terminate that peer. Unless this bandwidth fraud is taking place over a public peering LAN (IX). You could find that a non-peer is “stealing bandwidth”. In which case, tell the IX operator (they *do* care, and *do* want to stop abusive or fraudulent behaviour). You can, if paranoid, apply some l2/3 filters to only hear from expected peers at the IX (which prevents non-peers from pointing statics at you, but not peers though.) How paranoid shall we take it ? You can also - with a small enough customer footprint - perhaps put each peer into their own VRF and apply policies which prohibit forwarding except to customer prefixes. -a
Re: [c-nsp] Peering + Transit Circuits
On 18/08/2015 22:10, William Herrin wrote: This technique described isn't URPF, it's simple destination routing. The routes I offer you via BGP are the only routes in my table, hence the only routes I'm capable of routing. If you send me a packet for a _destination_ I didn't offer to you, I can't route it. yep, I hit send too soon. The point I intended to make was that ixp peering in a vrf will only protect you from transit theft, not clandestine peering. If you want to stop third party organisations at an ixp from getting peering by installing static routes, then l2 filters are what you need. Nick
Re: [c-nsp] Peering + Transit Circuits
On Aug 18, 2015, at 8:47 AM, Gert Doering g...@greenie.muc.de wrote: XR doesn't do it at all, hrmph) We have been asking about this as well, it might be worth revisiting. - Jared
Peering + Transit Circuits
Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. -- Tim:
Re: Peering + Transit Circuits
On Aug 18, 2015, at 1:24 PM, William Herrin b...@herrin.us wrote: On Tue, Aug 18, 2015 at 8:29 AM, Tim Durack tdur...@gmail.com wrote: Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. If you have a small number of peers, a separate router carrying a partial table works really well. To expand on this, and answer Tim’s question one post up in the thread: Putting all peer routes on a dedicated router with a partial table avoids the “steal transit” question. The Peering router can only speak to peers and your own network. Anyone dumping traffic on it will get !N (unless they are going to a peer, which is a pretty minimal risk). It has lots of other useful features such as network management and monitoring. It lets you do maintenance much easier. Etc., etc. But mostly, it lets you avoid joining an IX and having people use you as a backup transit provider. -- TTFN, patrick
Re: Peering + Transit Circuits
On Tue, Aug 18, 2015 at 8:29 AM, Tim Durack tdur...@gmail.com wrote: Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. If you have a small number of peers, a separate router carrying a partial table works really well. -- William Herrin her...@dirtside.com b...@herrin.us Owner, Dirtside Systems . Web: http://www.dirtside.com/
Re: Peering + Transit Circuits
On Tue, Aug 18, 2015 at 1:29 PM, Patrick W. Gilmore patr...@ianai.net wrote: On Aug 18, 2015, at 1:24 PM, William Herrin b...@herrin.us wrote: On Tue, Aug 18, 2015 at 8:29 AM, Tim Durack tdur...@gmail.com wrote: Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. If you have a small number of peers, a separate router carrying a partial table works really well. To expand on this, and answer Tim’s question one post up in the thread: Putting all peer routes on a dedicated router with a partial table avoids the “steal transit” question. The Peering router can only speak to peers and your own network. Anyone dumping traffic on it will get !N (unless they are going to a peer, which is a pretty minimal risk). It has lots of other useful features such as network management and monitoring. It lets you do maintenance much easier. Etc., etc. But mostly, it lets you avoid joining an IX and having people use you as a backup transit provider. This has always been my understanding - thanks for confirming. I'm weighing cost-benefit, and looking to see if there are any other smart ideas. As usual, it looks like simplest is best. -- Tim: p.s. Perhaps I should be relieved no one tried to sell me an SDN peering transit theft controller...
Re: [c-nsp] Peering + Transit Circuits
On Tue, Aug 18, 2015 at 4:43 PM, Nick Hilliard n...@foobar.org wrote: On 18/08/2015 20:22, Tim Durack wrote: This has always been my understanding - thanks for confirming. I'm weighing cost-benefit, and looking to see if there are any other smart ideas. As usual, it looks like simplest is best. i'd advise being careful with this approach: urpf at ixps is a nightmare. Hi Nick, This technique described isn't URPF, it's simple destination routing. The routes I offer you via BGP are the only routes in my table, hence the only routes I'm capable of routing. If you send me a packet for a _destination_ I didn't offer to you, I can't route it. URPF is the opposite of that. I'll only accept packets from you with a _source_ address which is included in the routes you sent to me. Regards, Bill Herrin -- William Herrin her...@dirtside.com b...@herrin.us Owner, Dirtside Systems . Web: http://www.dirtside.com/
Re: [c-nsp] Peering + Transit Circuits
Why do I read this thread as “Peering + Transit Circus” manning bmann...@karoshi.com PO Box 6151 Playa del Rey, CA 90296 310.322.8102 On 18August2015Tuesday, at 6:01, Jared Mauch ja...@puck.nether.net wrote: On Aug 18, 2015, at 8:47 AM, Gert Doering g...@greenie.muc.de wrote: XR doesn't do it at all, hrmph) We have been asking about this as well, it might be worth revisiting. - Jared
Re: [c-nsp] Peering + Transit Circuits
On 18/08/2015 21:56, Gert Doering wrote: So how's that stopping one of your bilateral peers from sending you traffic destined elsewhere? it doesn't: you detect it and depeer them. If they force the situation with static routes, the traffic will be dropped. Nick
Re: [c-nsp] Peering + Transit Circuits
On 18/08/2015 20:22, Tim Durack wrote: This has always been my understanding - thanks for confirming. I'm weighing cost-benefit, and looking to see if there are any other smart ideas. As usual, it looks like simplest is best. i'd advise being careful with this approach: urpf at ixps is a nightmare. If you're concerned about transit / peering theft on a shared l2 ixp style fabric, you're far better to use bilateral-only peering with ingress l2 filters at the ixp interface to include or exclude other participants as required. This will stop the problem dead in the water with no side effects. Nick
Re: Peering + Transit Circuits
Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Tim Durack tdur...@gmail.com To: cisco-...@puck.nether.net, nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:29:31 AM Subject: Peering + Transit Circuits Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. -- Tim:
Re: Peering + Transit Circuits
Thank you to everyone who has offered different explanations.. Yes, all it take is one party pooper to spoil a good party... So now the question is (public or private) what is the best practices to protect the network ? :) Faisal Imtiaz Snappy Internet Telecom - Original Message - From: John Osmon jos...@rigozsaurus.com To: nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:30:45 PM Subject: Re: Peering + Transit Circuits On Tue, Aug 18, 2015 at 11:27:53PM +, Faisal Imtiaz wrote: Thanks for the explanation, I am still trying to figure out the realistic business case where doing something like this would make sense to any party. (unless purely malicious or in error). I'm sure others will reply as well, but in case it helps someone googling in years to come... Let's look at ParasiteNet, a content heavy network with three BGP peerings: - Transit provider A via 100Mbps - Transit provider B via 100Mbps - Peer P via 1GBps (who also buys from provider B at 10G) If ParasiteNet needed to push more than 100Mbps to provider B, they might be tempted to route the traffic to peer P, even though peer P didn't advertise those routes. ParasiteNet gets a free ride if peer P doesn't notice what is going on (until they need more than 100Mbps inbound). I've been told of an occurance of this when a private network started peering with an edu network. Once the link was up, an absurd amount of traffic went across the link -- all destined for the Internet rather than the edu network. When the edu network shutdown the link, they were threatened with lawsuits...
Re: Peering + Transit Circuits
Thanks for the explanation, I am still trying to figure out the realistic business case where doing something like this would make sense to any party. (unless purely malicious or in error). Regards Faisal Imtiaz Snappy Internet Telecom From: Pshem Kowalczyk pshe...@gmail.com To: Faisal Imtiaz fai...@snappytelecom.net, Tim Durack tdur...@gmail.com Cc: nanog list nanog@nanog.org, cisco-...@puck.nether.net Sent: Tuesday, August 18, 2015 7:00:35 PM Subject: Re: Peering + Transit Circuits It's actually quite easy. Provider1 is present at Exchange1 and Exchange2, so is Provider2. Provider2 doesn't want to pay for the traffic between Exchange1 and Exchange2, so it points a static route for all prefixes it has in Exchange2 via Provider1's IP address in Exchange1 and does the same in Exchange2. Provider1's router receives traffic, checks where it should go (Exchange2) and it forwards the traffic. So the traffic flows like this: Provider2 (Exchange1) - Provider1 - (Exchange2) Provider2, all due to static routes. kind regards Pshem On Wed, 19 Aug 2015 at 10:38 Faisal Imtiaz fai...@snappytelecom.net wrote: Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Tim Durack tdur...@gmail.com To: cisco-...@puck.nether.net , nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:29:31 AM Subject: Peering + Transit Circuits Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. -- Tim:
Re: Peering + Transit Circuits
On Tue, Aug 18, 2015 at 11:27:53PM +, Faisal Imtiaz wrote: Thanks for the explanation, I am still trying to figure out the realistic business case where doing something like this would make sense to any party. (unless purely malicious or in error). I'm sure others will reply as well, but in case it helps someone googling in years to come... Let's look at ParasiteNet, a content heavy network with three BGP peerings: - Transit provider A via 100Mbps - Transit provider B via 100Mbps - Peer P via 1GBps (who also buys from provider B at 10G) If ParasiteNet needed to push more than 100Mbps to provider B, they might be tempted to route the traffic to peer P, even though peer P didn't advertise those routes. ParasiteNet gets a free ride if peer P doesn't notice what is going on (until they need more than 100Mbps inbound). I've been told of an occurance of this when a private network started peering with an edu network. Once the link was up, an absurd amount of traffic went across the link -- all destined for the Internet rather than the edu network. When the edu network shutdown the link, they were threatened with lawsuits...
Re: Peering + Transit Circuits
Assume you and I are at an IX and peer. Suppose I send you traffic for Comcast. I can do this, even if you do not send me prefixes for Comcast. It requires me to manually configure things, but I can do it. Put another way, you said We will trust everything coming in”. I am saying that perhaps you should not. As Comcast is not one of your customers, you will have to send the packets out your transit provider. You do not get paid when I give you the packets, and you probably pay your transit provider to get to Comcast. So I have gotten something for free, and you are paying for it - i.e. stealing. Normally a router gets a packet and sends it on its way without looking at the source. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. No filters to manage, no RIRs to sync, nothing to code, etc. There are evil ways around this if you do not configure your router properly (e.g. send you a prefix for Comcast with next-hop set to inside your network). But standard network hygiene will stop those. And as has been stated, this doesn’t have anything to do with URPF either. (Not sure why Nick brought that up, he’s smart enough to know what URPF is and runs an exchange himself, so I think he just brain-farted. Happens to us all.) Hope that made it more clear. -- TTFN, patrick On Aug 18, 2015, at 6:35 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Tim Durack tdur...@gmail.com To: cisco-...@puck.nether.net, nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:29:31 AM Subject: Peering + Transit Circuits Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. -- Tim:
Re: Peering + Transit Circuits
Hi Bob, Your point is completely understood... so the next question becomes what are these best practices methods ? :) Faisal Imtiaz Snappy Internet Telecom Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net - Original Message - From: Bob Evans b...@fiberinternetcenter.com To: Faisal Imtiaz fai...@snappytelecom.net Cc: Patrick W. Gilmore patr...@ianai.net, nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 7:36:00 PM Subject: Re: Peering + Transit Circuits Thank You Bob Evans CTO Thank you for the explanation.. However wouldn't a few other other attributes of the traffic show up . e.g. you would have asymmetric traffic.. going out via us, but coming back via a totally another path ? Patrick is correct in the approach you should take. If you don't have much traffic to being with - yes, you are correct that you'll notice a bounce. However, you should build a network so that your average traffic level can grow without having to check things manually. The more you automate the more you and your network are worth. This way you can simply upgrade ports at IX locations in a second without worrying about traffic levels and having to establish new or change existing policies. Thank You Bob Evans CTO BTW, my comment We will trust everything coming in was in ref. to QOS tags. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. In this scenario, the peering router is feeding routes to a Route Reflector ? and not taking in full routes from the route reflector ? But standard network hygiene will stop those. If there are any resources you could point to for these, I would be much obliged.. Thanks Faisal Imtiaz Snappy Internet Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 7:12:23 PM Subject: Re: Peering + Transit Circuits Assume you and I are at an IX and peer. Suppose I send you traffic for Comcast. I can do this, even if you do not send me prefixes for Comcast. It requires me to manually configure things, but I can do it. Put another way, you said We will trust everything coming in�. I am saying that perhaps you should not. As Comcast is not one of your customers, you will have to send the packets out your transit provider. You do not get paid when I give you the packets, and you probably pay your transit provider to get to Comcast. So I have gotten something for free, and you are paying for it - i.e. stealing. Normally a router gets a packet and sends it on its way without looking at the source. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. No filters to manage, no RIRs to sync, nothing to code, etc. There are evil ways around this if you do not configure your router properly (e.g. send you a prefix for Comcast with next-hop set to inside your network). But standard network hygiene will stop those. And as has been stated, this doesn’t have anything to do with URPF either. (Not sure why Nick brought that up, he’s smart enough to know what URPF is and runs an exchange himself, so I think he just brain-farted. Happens to us all.) Hope that made it more clear. -- TTFN, patrick On Aug 18, 2015, at 6:35 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz
Re: Peering + Transit Circuits
Thank You Bob Evans CTO Thank you for the explanation.. However wouldn't a few other other attributes of the traffic show up . e.g. you would have asymmetric traffic.. going out via us, but coming back via a totally another path ? Patrick is correct in the approach you should take. If you don't have much traffic to being with - yes, you are correct that you'll notice a bounce. However, you should build a network so that your average traffic level can grow without having to check things manually. The more you automate the more you and your network are worth. This way you can simply upgrade ports at IX locations in a second without worrying about traffic levels and having to establish new or change existing policies. Thank You Bob Evans CTO BTW, my comment We will trust everything coming in was in ref. to QOS tags. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. In this scenario, the peering router is feeding routes to a Route Reflector ? and not taking in full routes from the route reflector ? But standard network hygiene will stop those. If there are any resources you could point to for these, I would be much obliged.. Thanks Faisal Imtiaz Snappy Internet Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 7:12:23 PM Subject: Re: Peering + Transit Circuits Assume you and I are at an IX and peer. Suppose I send you traffic for Comcast. I can do this, even if you do not send me prefixes for Comcast. It requires me to manually configure things, but I can do it. Put another way, you said We will trust everything coming inâ. I am saying that perhaps you should not. As Comcast is not one of your customers, you will have to send the packets out your transit provider. You do not get paid when I give you the packets, and you probably pay your transit provider to get to Comcast. So I have gotten something for free, and you are paying for it - i.e. stealing. Normally a router gets a packet and sends it on its way without looking at the source. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. No filters to manage, no RIRs to sync, nothing to code, etc. There are evil ways around this if you do not configure your router properly (e.g. send you a prefix for Comcast with next-hop set to inside your network). But standard network hygiene will stop those. And as has been stated, this doesnât have anything to do with URPF either. (Not sure why Nick brought that up, heâs smart enough to know what URPF is and runs an exchange himself, so I think he just brain-farted. Happens to us all.) Hope that made it more clear. -- TTFN, patrick On Aug 18, 2015, at 6:35 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Tim Durack tdur...@gmail.com To: cisco-...@puck.nether.net, nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:29:31 AM Subject: Peering + Transit Circuits Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith
Re: Peering + Transit Circuits
It's actually quite easy. Provider1 is present at Exchange1 and Exchange2, so is Provider2. Provider2 doesn't want to pay for the traffic between Exchange1 and Exchange2, so it points a static route for all prefixes it has in Exchange2 via Provider1's IP address in Exchange1 and does the same in Exchange2. Provider1's router receives traffic, checks where it should go (Exchange2) and it forwards the traffic. So the traffic flows like this: Provider2 (Exchange1) - Provider1 - (Exchange2) Provider2, all due to static routes. kind regards Pshem On Wed, 19 Aug 2015 at 10:38 Faisal Imtiaz fai...@snappytelecom.net wrote: Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Tim Durack tdur...@gmail.com To: cisco-...@puck.nether.net, nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:29:31 AM Subject: Peering + Transit Circuits Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. -- Tim:
Re: Peering + Transit Circuits
On 18 August 2015 at 14:29, Tim Durack tdur...@gmail.com wrote: 4. Don't worry about peers stealing transit. Because both of our transit providers implement source filters. Any packets received with a source IP not in the list of IP ranges registered by us will be dropped by the transit provider. Stealing transit is not practical giving the limitation that you need to use a source address from our ranges. I use ACLs on our end too just to be sure. ACL on the transit to prevent wrong source from leaving our network and ACL on the peering to prevent wrong destination to enter the network. Actually both ACLs are used in both places. The prefix lists used for the ACL need to be maintained in any case. It is the list of routes that we advertise. Regards, Baldur
Re: Peering + Transit Circuits
Thank you for the explanation.. However wouldn't a few other other attributes of the traffic show up . e.g. you would have asymmetric traffic.. going out via us, but coming back via a totally another path ? BTW, my comment We will trust everything coming in was in ref. to QOS tags. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. In this scenario, the peering router is feeding routes to a Route Reflector ? and not taking in full routes from the route reflector ? But standard network hygiene will stop those. If there are any resources you could point to for these, I would be much obliged.. Thanks Faisal Imtiaz Snappy Internet Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 7:12:23 PM Subject: Re: Peering + Transit Circuits Assume you and I are at an IX and peer. Suppose I send you traffic for Comcast. I can do this, even if you do not send me prefixes for Comcast. It requires me to manually configure things, but I can do it. Put another way, you said We will trust everything coming in”. I am saying that perhaps you should not. As Comcast is not one of your customers, you will have to send the packets out your transit provider. You do not get paid when I give you the packets, and you probably pay your transit provider to get to Comcast. So I have gotten something for free, and you are paying for it - i.e. stealing. Normally a router gets a packet and sends it on its way without looking at the source. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. No filters to manage, no RIRs to sync, nothing to code, etc. There are evil ways around this if you do not configure your router properly (e.g. send you a prefix for Comcast with next-hop set to inside your network). But standard network hygiene will stop those. And as has been stated, this doesn’t have anything to do with URPF either. (Not sure why Nick brought that up, he’s smart enough to know what URPF is and runs an exchange himself, so I think he just brain-farted. Happens to us all.) Hope that made it more clear. -- TTFN, patrick On Aug 18, 2015, at 6:35 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Tim Durack tdur...@gmail.com To: cisco-...@puck.nether.net, nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:29:31 AM Subject: Peering + Transit Circuits Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations/DavidSmith-PeeringPolicyEnforcement.pdf ) 4. Don't worry about peers stealing transit. 5. What is peering? Your comments are appreciated. -- Tim:
Re: Peering + Transit Circuits
On Aug 18, 2015, at 7:26 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: Thank you for the explanation.. However wouldn't a few other other attributes of the traffic show up . e.g. you would have asymmetric traffic.. going out via us, but coming back via a totally another path ? So? If I am a content provider, my transit has more out than in. If I can push some of that outbound traffic through you for free, I’ll get the inbound over my transit provider for free since inbound is so low. BTW, my comment We will trust everything coming in was in ref. to QOS tags. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. In this scenario, the peering router is feeding routes to a Route Reflector ? and not taking in full routes from the route reflector ? The peering router has routes from the peers (since it peers directly with them), and routes from your internal network. Not sure where a router reflector comes into this. You can use one, or not, but it’s not relevant to which routes the peering router has. But standard network hygiene will stop those. If there are any resources you could point to for these, I would be much obliged.. There are lots, but don’t have my references with me. There’s 10K+ people on this list, I’m sure someone else has a list they like. :) -- TTFN, patrick - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 7:12:23 PM Subject: Re: Peering + Transit Circuits Assume you and I are at an IX and peer. Suppose I send you traffic for Comcast. I can do this, even if you do not send me prefixes for Comcast. It requires me to manually configure things, but I can do it. Put another way, you said We will trust everything coming in”. I am saying that perhaps you should not. As Comcast is not one of your customers, you will have to send the packets out your transit provider. You do not get paid when I give you the packets, and you probably pay your transit provider to get to Comcast. So I have gotten something for free, and you are paying for it - i.e. stealing. Normally a router gets a packet and sends it on its way without looking at the source. However, if you have a router at the IX which has _only_ peer routes and your routes, that solves the problem. If I send you a packet for Comcast, your peering router will drop it and send an ICMP Network Unreachable. No filters to manage, no RIRs to sync, nothing to code, etc. There are evil ways around this if you do not configure your router properly (e.g. send you a prefix for Comcast with next-hop set to inside your network). But standard network hygiene will stop those. And as has been stated, this doesn’t have anything to do with URPF either. (Not sure why Nick brought that up, he’s smart enough to know what URPF is and runs an exchange himself, so I think he just brain-farted. Happens to us all.) Hope that made it more clear. -- TTFN, patrick On Aug 18, 2015, at 6:35 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: Let me start backwards... To me 'peering' is sharing internal routes and downstream customer routes,and not external ones. IP transit is all of the external routes including internal routes downstream customer routes Having said that. if one is control of what IP Prefixes get advertised to whom... how exactly someone (peers) 'steal' transit ? (If one is not managing the filters well then yes it is possible, but that would be a configuration error ?) Maybe I am naive, to my Peering routes (relationships) are a subset of IP Transit Routes (relationships) Based on above belief... Then Item # 3, becomes the choice of the OP where one can make one of two starting assumptions... We will trust everything coming in and change what we don't like... or We will not trust anything coming in, and change (accept) what we like. Items # 1 2, would be a function of network design, technical requirements (maintenance window) etc etc.. easier to deal with a distributed edge vs all in one when one has to bring anything down for any reason.. I am open to learning and being corrected if any of the above is wrong ! Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Tim Durack tdur...@gmail.com To: cisco-...@puck.nether.net, nanog list nanog@nanog.org Sent: Tuesday, August 18, 2015 8:29:31 AM Subject: Peering + Transit Circuits Question: What is the preferred practice for separating peering and transit circuits? 1. Terminate peering and transit on separate routers. 2. Terminate peering and transit circuits in separate VRFs. 3. QoS/QPPB ( https://www.nanog.org/meetings/nanog42/presentations
Fwd: [c-nsp] Peering + Transit Circuits
-- Forwarded message -- From: Tim Durack tdur...@gmail.com Date: Tue, Aug 18, 2015 at 9:53 AM Subject: Re: [c-nsp] Peering + Transit Circuits To: Rolf Hanßen n...@rhanssen.de Cc: cisco-...@puck.nether.net cisco-...@puck.nether.net On Tue, Aug 18, 2015 at 9:45 AM, Rolf Hanßen n...@rhanssen.de wrote: Hi, you forgot do some interface-ACL-magic that drops peer-traffic that does not have a destination IP in my cool-networks-whitelist. Yup, valid option. I am trying to avoid anything that involves maintaining lists. -- Tim:
Re: [c-nsp] Peering + Transit Circuits
On Tue, Aug 18, 2015 at 8:47 AM, Gert Doering g...@greenie.muc.de wrote: Hi, On Tue, Aug 18, 2015 at 08:29:31AM -0400, Tim Durack wrote: 4. Don't worry about peers stealing transit. 5. What is peering? I'm afraid that the majority of answers will be 4./5., mixed with 6. what? how can peers stell my transit?! We're somewhat into the we'll notice if there is surprisingly high inbound traffic on peering, and then we'll find the peer, and apply appropriate measures camp... (since we're a hosting shop, we have mostly outgoing traffic, so significant amounts of incomnig traffic stick out). But yeah, something more strict might be in order. Thanks for the response. This is what I was guessing. We currently do 2. Terminate peering and transit circuits in separate VRFs. which works well when everything is a VRF but comes at the cost of higher resource usage (RIB FIB.) I was thinking a creative solution might be: 7. DSCP mark packets on peering ingress, police on transit egress. Not sure if I really want to get into using DSCP bits for basic IP service though. (It would be cool if Cisco would understand that hardware forwarding platforms need useful netflow with MAC-addresses in there... ASR9k at least got working MAC-accounting, but more fine grained telemetry would certainly be appreciated. Software IOS can do it, Sup720 cannot do it due to hardware constraints, Sup2T exports MAC addresses taken from random caches in the system but not the inbound packets, XR doesn't do it at all, hrmph) gert -- USENET is *not* the non-clickable part of WWW! // www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025 g...@net.informatik.tu-muenchen.de -- Tim:
Re: [c-nsp] Peering + Transit Circuits
On Tue, Aug 18, 2015 at 8:47 AM, Gert Doering g...@greenie.muc.de wrote: Hi, (It would be cool if Cisco would understand that hardware forwarding platforms need useful netflow with MAC-addresses in there... ASR9k at least got working MAC-accounting, but more fine grained telemetry would certainly be appreciated. Software IOS can do it, Sup720 cannot do it due to hardware constraints, Sup2T exports MAC addresses taken from random caches in the system but not the inbound packets, XR doesn't do it at all, hrmph) At the risk of introducing religion, I will mention sFlow... -- Tim:
Re: [c-nsp] Peering + Transit Circuits
On Tue, Aug 18, 2015 at 9:38 AM, Gert Doering g...@greenie.muc.de wrote: Hi, On Tue, Aug 18, 2015 at 09:32:53AM -0400, Tim Durack wrote: (It would be cool if Cisco would understand that hardware forwarding platforms need useful netflow with MAC-addresses in there... ASR9k at [..] At the risk of introducing religion, I will mention sFlow... Yes... and this is helping exactly why...? Given the overwhelming support for sFlow in (Cisco-) hardware routers used as peering edge? :-) I ask Cisco for sFlow support on a regular basis. Cisco typically respond with some variation of NIH syndrome. Anyway, back to my question :-) -- Tim:
Re: [c-nsp] Peering + Transit Circuits
On Tue, Aug 18, 2015 at 11:25 AM, Scott Granados sc...@granados-llc.net wrote: So in our case we terminate peering and transit on different routers. Peering routers have well flow enabled (the one that starts with a J that’s inline). With NFSEN / NFDUMP we’re able to collect that flow data and look for anomalous flows or other issues. We pretty much detect and then deal with peering issues rather than prevent them with whitelists and so forth but then again we’ve been lucky and not experienced to many issues other than the occasional leakage of prefixes and such which maxprefix handles nicely. Can I ask why you terminate peering and transit on different routers? (Not suggesting that is bad, just trying to understand the reason.) Tim:
