RE: route for linx.net in Level3?
The older school of thought was to put all of the edge interfaces into the IGP, and then carry all of the external routes in BGP. I thought people where doing it because IGP converged faster than iBGP and in case of an external link failure the ingress PE was informed via IGP that it has to find an alternate next-hop. Though now with the advent of BGP PIC this is not an argument anymore. adam
Re: route for linx.net in Level3?
In a message written on Fri, Apr 05, 2013 at 09:32:52AM +0200, Adam Vitkovsky wrote: I thought people where doing it because IGP converged faster than iBGP and in case of an external link failure the ingress PE was informed via IGP that it has to find an alternate next-hop. Though now with the advent of BGP PIC this is not an argument anymore. You're talking about stuff that's all 7-10 years after the decisions were made that I described in my previous e-mail. Tag switching (now MPLS) had not yet been invented/deployed when the first next-hop-self wave occured it was all about scaling both the IGP and BGP. In some MPLS topologies it may speed re-routing to have edge interfaces in the IGP due to the faster convergence of IGP's. YMMV, Batteries not Included, Some Assembly Required. -- Leo Bicknell - bickn...@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ pgp_AJVWtJwTg.pgp Description: PGP signature
Re: route for linx.net in Level3?
oops! i have a host directly on the linx on which i can give you shell access
Re: route for linx.net in Level3?
Yes. In the fallout from the Cloudflare attack of last week it was announced that several IXs were going to stop advertising the address space of their peering lan, which properly does not need to be advertised anyway. Yes, that will cause some minor problems for those who work for and with the companies that peer there, but they are *clients*, and should be able to have other similar arrangements made for them. Cheers, -- jra - Original Message - From: Yang Yu yang.yu.l...@gmail.com To: k...@network-services.uoregon.edu Cc: NANOG list nanog@nanog.org Sent: Wednesday, April 3, 2013 7:20:44 PM Subject: Re: route for linx.net in Level3? I noticed it too this morning from a AS3549 customer. Level 3 LG shows no route for 195.66.232.0/22 on North American sites. On Wed, Apr 3, 2013 at 6:52 PM, John Kemp k...@network-services.uoregon.edu wrote: Having trouble reaching route-views.linx.routeviews.org from AS3582. I'm assuming that some folks stopped carrying this particular linx.net address prefix as of this morning. ?!? $ whois -h whois.cymru.com -v 195.66.241.146 AS | IP | BGP Prefix | CC | Registry | Allocated | AS Name 5459 | 195.66.241.146 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 | LINX-AS London Internet Exchange Ltd. $ dig +short 146.241.66.195.peer.asn.cymru.com TXT 1299 2914 3257 10310 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 -- John Kemp (k...@routeviews.org) RouteViews Engineer NOC: n...@routeviews.org MAIL: h...@routeviews.org WWW: http://www.routeviews.org -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
Re: route for linx.net in Level3?
In a message written on Thu, Apr 04, 2013 at 02:57:11PM -0400, Jay Ashworth wrote: Yes. In the fallout from the Cloudflare attack of last week it was announced that several IXs were going to stop advertising the address space of their peering lan, which properly does not need to be advertised anyway. Well, now that's a big maybe. I was a big advocate for the peering exchanges each having their own ASN and announcing the peering block back in the day, and it seems people may have forgotten some of the issues with unadvertised peering exchange blocks. It breaks traceroute for many people: The ICMP TTL Unreachable will come from a non-routed network (the exchange LAN). If it crosses another network boundary doing uRPF, even in loose mode, those unreachables will be dropped. It also reduces the utility of a tool like MTR. Without the ICMP responese it won't know where to ping, and even if it receives the ICMP it's likely packets towards the LAN IP's will be dropped with no route to host. It has the potential to break PMTU discovery for many people: If a router is connected to the exchange and a lower MTU link a packet coming in with DF set will get an ICMP would-fragment reply. Most vendors source from the input interface, e.g. the exchange IP. Like the traceorute case, if crosses another network boundary doing uRPF, even in loose mode, those ICMP messages will be lost, resulting in a PMTU black hole. Some vendors have knobs to force the ICMP to be emitted from a loopback, but not all. People would have to turn it on. But hey, this is a good thing because a DDOS caused issues, right? Well, not so much. Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, and by extension all of their customers with a default route pointed at them. For the most popular exchanges (AMS-IX, for instance) I suspect the percentage of end users who can reach the exchange LAN without it being explicitly routed to be well over 80%, perhaps into the upper 90% range. So when those boxes DDOS, they are going to all DDOS the LAN anyway. Security through obscurity does not work. This is going to annoy some people just trying to do their day job, and not make a statistical difference to the attackers trying to take out infrastructure. How about we all properly implement BCP 38 instead? -- Leo Bicknell - bickn...@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ pgpfLwy5Hpdxc.pgp Description: PGP signature
Re: route for linx.net in Level3?
On Thu, Apr 4, 2013 at 12:29 PM, Leo Bicknell bickn...@ufp.org wrote: But hey, this is a good thing because a DDOS caused issues, right? Well, not so much. Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, and by extension all of their customers with a default route pointed at them. For the most popular exchanges (AMS-IX, for instance) I suspect the percentage of end users who can reach the exchange LAN without it being explicitly routed to be well over 80%, perhaps into the upper 90% range. So when those boxes DDOS, they are going to all DDOS the LAN anyway. Yes, thats why everyone needs to set up some sanity in their networks. This was presented at an APNIC conference a little while back: http://conference.apnic.net/__data/assets/pdf_file/0018/50706/apnic34-mike-jager-securing-ixp-connectivity_1346119861.pdf hundreds of networks are improperly set up and are being abused (and abusing) to the IXP LANs. Security through obscurity does not work. This is going to annoy some people just trying to do their day job, and not make a statistical difference to the attackers trying to take out infrastructure. This isn't security through obscurity. This is saving the IXP from getting 100's of G's over transit, which should just be for their corporate network. How about we all properly implement BCP 38 instead? Agree.
Re: route for linx.net in Level3?
Leo Bicknell wrote: Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, and by extension all of their customers with a default route pointed at them. Actually, that may not be the case, and probably *should* not be the case. Here's why, in a nutshell: If two regional ISPs on either side of the planet, point default to the same Global ISP, even if they do not peer with that ISP, by using the IX next-hop at IX A (for ISP A), and IX B (for ISP B), then the Global ISP is now giving free on-net transit to A and B. So, it turns out that pretty much the only way to prevent this at a routing level, is to not carry IXP networks (in IGP or IBGP), but rather to do next-hop-self. The other way is to filter at a packet level on ingress, based on Layer 2 information, which on many kinds of IX-capable hardware, is actually impossible. So, when it comes to IXPs: Next-Hop-Self. (BCP 38 actually doesn't even enter into it, oddly enough.) Brian
Re: route for linx.net in Level3?
On 2013-04-04, at 15:53, Brian Dickson brian.peter.dick...@gmail.com wrote: Leo Bicknell wrote: Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, I have experience of several networks where that is not the case. IGP carries routes for loopback and internal-facing interfaces; external-facing interface routes are only known to the local router; pervasive next-hop-self for IBGP. So, no great survey, but don't assume that everybody does things the same way. Joe
RE: route for linx.net in Level3?
First of all I agree with Leo that not advertising IX prefixes permanently causes more problems than it solves. Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it Well if I would peer with such an ISP at London and Frankfurt I could create a GRE tunnel from London to Frankfurt via the other ISP and use it to transport packets that would otherwise have to traverse my backbone. Or if my peer has a router at IX that happens to have full routing view I can just point a static default toward it and have a free transit. Check out: http://www.bcp38.info adam -Original Message- From: Leo Bicknell [mailto:bickn...@ufp.org] Sent: Thursday, April 04, 2013 9:29 PM To: NANOG Subject: Re: route for linx.net in Level3? In a message written on Thu, Apr 04, 2013 at 02:57:11PM -0400, Jay Ashworth wrote: Yes. In the fallout from the Cloudflare attack of last week it was announced that several IXs were going to stop advertising the address space of their peering lan, which properly does not need to be advertised anyway. Well, now that's a big maybe. I was a big advocate for the peering exchanges each having their own ASN and announcing the peering block back in the day, and it seems people may have forgotten some of the issues with unadvertised peering exchange blocks. It breaks traceroute for many people: The ICMP TTL Unreachable will come from a non-routed network (the exchange LAN). If it crosses another network boundary doing uRPF, even in loose mode, those unreachables will be dropped. It also reduces the utility of a tool like MTR. Without the ICMP responese it won't know where to ping, and even if it receives the ICMP it's likely packets towards the LAN IP's will be dropped with no route to host. It has the potential to break PMTU discovery for many people: If a router is connected to the exchange and a lower MTU link a packet coming in with DF set will get an ICMP would-fragment reply. Most vendors source from the input interface, e.g. the exchange IP. Like the traceorute case, if crosses another network boundary doing uRPF, even in loose mode, those ICMP messages will be lost, resulting in a PMTU black hole. Some vendors have knobs to force the ICMP to be emitted from a loopback, but not all. People would have to turn it on. But hey, this is a good thing because a DDOS caused issues, right? Well, not so much. Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, and by extension all of their customers with a default route pointed at them. For the most popular exchanges (AMS-IX, for instance) I suspect the percentage of end users who can reach the exchange LAN without it being explicitly routed to be well over 80%, perhaps into the upper 90% range. So when those boxes DDOS, they are going to all DDOS the LAN anyway. Security through obscurity does not work. This is going to annoy some people just trying to do their day job, and not make a statistical difference to the attackers trying to take out infrastructure. How about we all properly implement BCP 38 instead? -- Leo Bicknell - bickn...@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
Re: route for linx.net in Level3?
Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, yikes! this is quite ill-advised and i don't know anyone who does this, but i think all my competitors should. I have experience of several networks where that is not the case. IGP carries routes for loopback and internal-facing interfaces; i have seen some carry external because, for some reason, they do not want to re-write next-hop at the border. randy
Re: route for linx.net in Level3?
On Thu, Apr 4, 2013 at 1:26 PM, Adam Vitkovsky adam.vitkov...@swan.sk wrote: Check out: http://www.bcp38.info Right on. :-) - ferg -- Fergie, a.k.a. Paul Ferguson fergdawgster(at)gmail.com
Re: route for linx.net in Level3?
Yeah, you wouldn't think that one should fall out. It is possible that my 195.66.241.146 really should be something sitting within: 195.66.232.0/22. I'll have to talk with some of the LINX folks to understand whether they are intending that 195.66.240.0/22 and 195.66.232.0/22 are treated differently. If that's the case, I may need to move off of 195.66.240.0/22. Thanks, John Kemp (k...@routeviews.org) On 4/3/13 4:20 PM, Yang Yu wrote: I noticed it too this morning from a AS3549 customer. Level 3 LG shows no route for 195.66.232.0/22 on North American sites. On Wed, Apr 3, 2013 at 6:52 PM, John Kemp k...@network-services.uoregon.edu wrote: Having trouble reaching route-views.linx.routeviews.org from AS3582. I'm assuming that some folks stopped carrying this particular linx.net address prefix as of this morning. ?!? $ whois -h whois.cymru.com -v 195.66.241.146 AS | IP | BGP Prefix | CC | Registry | Allocated | AS Name 5459| 195.66.241.146 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 | LINX-AS London Internet Exchange Ltd. $ dig +short 146.241.66.195.peer.asn.cymru.com TXT 1299 2914 3257 10310 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 -- John Kemp (k...@routeviews.org) RouteViews Engineer NOC: n...@routeviews.org MAIL: h...@routeviews.org WWW: http://www.routeviews.org
Re: route for linx.net in Level3?
On Thu, Apr 4, 2013 at 1:43 PM, Randy Bush ra...@psg.com wrote: Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, yikes! this is quite ill-advised and i don't know anyone who does this, but i think all my competitors should. Its more common than uncommon. At WIX (Wellington), 64 out of 93 members will carry packets destined to APE (Auckland Exchange). (source: http://conference.apnic.net/__data/assets/pdf_file/0018/50706/apnic34-mike-jager-securing-ixp-connectivity_1346119861.pdf) and this is just New Zealand! Just checked a few exchanges, not just are the IXP ranges being carried, they're being leaked: Equinix SG: $ bgpctl show rib 202.79.197.0/24 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin 202.79.197.0/24 100 0 13335 23947 23947 ? 202.79.197.0/24 100 0 13335 10026 i Any2 LA: bgpctl show rib 206.223.143.0/24 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin 206.223.143.0/24 100 0 13335 9304 i 206.223.143.0/24 100 0 13335 9304 i 206.223.143.0/24 100 0 13335 4635 9304 i 206.223.143.0/24 100 0 13335 9304 i I have experience of several networks where that is not the case. IGP carries routes for loopback and internal-facing interfaces; i have seen some carry external because, for some reason, they do not want to re-write next-hop at the border. randy
Re: route for linx.net in Level3?
On Thu, Apr 4, 2013 at 1:43 PM, Randy Bush ra...@psg.com wrote: Even if the exchange does not advertise the exchange LAN, it's probably the case that it is in the IGP (or at least IBGP) of everyone connected to it, yikes! this is quite ill-advised and i don't know anyone who does this, but i think all my competitors should. Its more common than uncommon. At WIX (Wellington), 64 out of 93 members will carry packets destined to APE (Auckland Exchange). (source: http://conference.apnic.net/__data/assets/pdf_file/0018/50706/apnic34-mike-jager-securing-ixp-connectivity_1346119861.pdf) and this is just New Zealand! Just checked a few exchanges, not just are the IXP ranges being carried, they're being leaked: i am not unhappy by the exchange mesh being carried within a member and being propagated to their customer cone, see my nanog preso of feb 1997 and leo's recent post. it's putting such things in one's igp that disgusts me. as joe said, igp is just for the loopbacks and other interfaces it takes to make your ibgp work. randy
Re: route for linx.net in Level3?
In a message written on Fri, Apr 05, 2013 at 10:01:34AM +0900, Randy Bush wrote: it's putting such things in one's igp that disgusts me. as joe said, igp is just for the loopbacks and other interfaces it takes to make your ibgp work. While your method is correct for probably 80-90% of the ISP networks, the _why_ people do that has almost been lost to the mysts of time. I'm sure Randy knows what I'm about to type, but for the rest of the list... The older school of thought was to put all of the edge interfaces into the IGP, and then carry all of the external routes in BGP. This caused a one level recursion in the routers: eBGP Route-IXP w/IGP Next Hop-Output Interface The Internet then became a thing, and there started to be a lot of BGP speaking customers (woohoo! T1's for everyone!), and thus lots of edge /30's in the IGP. The IGP convergence time quickly got very, very bad. I think a network or two may have even broken an IGP. The solution was to take edge interfaces (really redistribute connected for most people) and move it from the IGP to BGP, and to make that work BGP had to set next-hop-self on the routes. The exchange /24 would now appear in BGP with a next hop of the router loopback, the router itself knew it was directly connected. A side effect is that this caused a two-step lookup in BGP: eBGP-Route-IXP w/Router Loopback Next Hop-Loopback w/IGP Next Hop-Output Interface IGP's went from O(bgp_customers) routes to O(router) routes, and stopped falling over and converged much faster. On the flip side, every RIB-FIB operation now has to go through an extra step of recursion for every route, taking BGP resolution from O(routes) to O(routes * 1.1ish). Since all this happened, CPU's have gotten much faster, RAM has gotten much larger. Most people have never revisited the problem, the scaling of IGP's, or what hardware can do today. There are plenty of scenarios where the old way works just spiffy, and can have some advantages. For a network with a very low number of BGP speakers the faster convergence of the IGP may be desireable. Not every network is built the same, or has the same scaling properties. What's good for a CDN may not be good for an access ISP, and vice versa, for example. -- Leo Bicknell - bickn...@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ pgp7nvnvQHk2B.pgp Description: PGP signature
route for linx.net in Level3?
Having trouble reaching route-views.linx.routeviews.org from AS3582. I'm assuming that some folks stopped carrying this particular linx.net address prefix as of this morning. ?!? $ whois -h whois.cymru.com -v 195.66.241.146 AS | IP | BGP Prefix | CC | Registry | Allocated | AS Name 5459| 195.66.241.146 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 | LINX-AS London Internet Exchange Ltd. $ dig +short 146.241.66.195.peer.asn.cymru.com TXT 1299 2914 3257 10310 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 -- John Kemp (k...@routeviews.org) RouteViews Engineer NOC: n...@routeviews.org MAIL: h...@routeviews.org WWW: http://www.routeviews.org
Re: route for linx.net in Level3?
Hi John, On Apr 4, 2013, at 12:52 AM, John Kemp k...@network-services.uoregon.edu wrote: Having trouble reaching route-views.linx.routeviews.org from AS3582. I'm assuming that some folks stopped carrying this particular linx.net address prefix as of this morning. ?!? Indeed LINX has taken steps recently to reduce the scope and reach of their peering LAN prefix to partially mitigate some types of attack. I'd be happy to help you off-list to get some permanent connectivity back to the machine. Kind regards, Job
Re: route for linx.net in Level3?
I noticed it too this morning from a AS3549 customer. Level 3 LG shows no route for 195.66.232.0/22 on North American sites. On Wed, Apr 3, 2013 at 6:52 PM, John Kemp k...@network-services.uoregon.edu wrote: Having trouble reaching route-views.linx.routeviews.org from AS3582. I'm assuming that some folks stopped carrying this particular linx.net address prefix as of this morning. ?!? $ whois -h whois.cymru.com -v 195.66.241.146 AS | IP | BGP Prefix | CC | Registry | Allocated | AS Name 5459| 195.66.241.146 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 | LINX-AS London Internet Exchange Ltd. $ dig +short 146.241.66.195.peer.asn.cymru.com TXT 1299 2914 3257 10310 | 195.66.240.0/22 | GB | ripencc | 1997-12-01 -- John Kemp (k...@routeviews.org) RouteViews Engineer NOC: n...@routeviews.org MAIL: h...@routeviews.org WWW: http://www.routeviews.org
