Re: icmp rpf
Hi Mark, On Sun, 24 Sep 2006 16:33:30 -0700 (PDT) Mark Kent [EMAIL PROTECTED] wrote: Mark Smith wrote: The non-announcers, because they're also breaking PMTUD. Really? How? Remember, we're not talking about RFC1918 space, where there is a BCP that says we should filter it at the edge. We're talking about public IP space, that just doesn't happen to be announced outside of a particular AS. When a router that can't shove a DF'd packet down a link because the MTU is too small needs to create a ICMP Destination Unreachable, Packet Too Big, Fragmentation Required, it needs to pick a source IP address to use for that ICMP packet, which will be one of those assigned to the router with the MTU problem (I'm fairly sure it's the IP address assigned to the outgoing interface for this ICMP packet, although I don't think it probably matters much). If an upstream router, i.e. on the way back to the sender who needs to resend with a smaller packet, is dropping these packets because they fail RPF, then PMTUD breaks. The result might be connection timeouts at the sender, or possibly after quite a while the sender might try smaller packets and eventually they'll get through (I think Windows might do this). Either way, bad end-user experience. PMTUD as it currently works isn't ideal, as of course there isn't any guarantee that these ICMP Dest Unreachables will get there even in a good network. However, most of the time it works, where as in the scenario you're presenting, it definately won't. Regards, Mark. -- Sheep are slow and tasty, and therefore must remain constantly alert. - Bruce Schneier, Beyond Fear
Re: Topicality perceptions
One of the biggest issues with the list as I've seen from time to time from my perspective, is the definition of operations. So on a quick breakdown of the logical definition of NANOG, I derive Operations of the North American Network. The problem with this stems from far too many bastardizing their own definition of what it should be. Please don't contribute to the bastardization. Section 3 of the NANOG charter states: The purpose of NANOG is to provide forums in the North American region for education and the sharing of knowledge for the Internet operations community. You can read the full charter here: http://www.nanog.org/charter.html By your definition, Cat's recent request for outage information about Telehouse North would be off-topic. But according to the NANOG FAQ here: http://www.nanog.org/listfaq.html outages are on topic. Obviously, network infrastructure tends to span political borders and geographic borders, therefore it is not unusual that Cat has an infrastructure issue in Europe to deal with. On your first point, the fuzziness and lack of clarity of what network operations issues belong on this list, I agree. The FAQ is never posted on the list so it has become an obscure document hidden away on a little-used website. It needs to be promoted more and I think it needs to be updated to communicate more clearly. These are off-topic but I wouldn't trade em for the world. I've learned much from them, as have I from all sorts of posts on topic or not. I agree with you. Unfortunately some old-timers would rather see a return to the old days when network ops and engineering was an obscure passtime only understood by those who knew the secret handshake and were admitted to the inner circle. They forget that NANOG's major role has been in educating the new people who have flooded into the net ops community as the Internet grew and grew and grew. --Michael Dillon
Re: icmp rpf
The non-announcers, because they're also breaking PMTUD. If you're not sure what benefits PMTUD gives, you might want to review this page: http://www.psc.edu/~mathis/MTU/index.html --Michael Dillon
Re: Topicality perceptions
Concur. Nanog has been an on-going education in essentially all aspects of internetworking, routing, data centres, security, spam/malware/abuse. Long may it stay that way. I'd argue that the fuzziness is probably a reflection of the ever-broadening role of IT/telco/netops people and ideas in current organisations. Now, someone mentioned issues with SIP. I'd like to flag that this is going to become a top line operational issue in the next few years, due to the deployment of following technologies: 1) Carrier/Enterprise VoIP 2) Peer-to-Peer VoIP using SIP (see - Gizmo) 3) Concurrent applications using SIP 4) IP Multimedia Subsystem (IMS) in mobile networks (and possibly fixed networks) interworking with each other, PSTN and the public Internet 5) ETSI TISPAN activity (probably the least important of the five) Note that 1 through 3 use SIP as defined by IETF whereas 4 and 5 use the 3GPP/3GPP2/ETSI extensions to it, which may mean they cannot interwork. Further, IMS and various associated technologies employ DNS ENUM to map e164 numbers to SIP URIs, not to speak of ordinary DNS to map URIs to IP addresses. Some DNS security measures previously discussed on NANOG have the effect of filtering ENUM replies. There is also the problem that IMS carriers, as far as anyone knows, are going to operate as private internetworks and do some form of NAT at the Session Border Controller (ie - gateway to the public Internet). How they will handle this at private interconnections with each other is unclear. It is also unclear how connections between a Carrier SIP client with a privately assigned or RFC1918 address and a carrier-land URI, and an open-Internet IETF SIP client with a globally routable address and its own URI, will work. It also seems clear that IMS-adopting carriers will continue to declare themselves as carrier grade, which suggests that the criticality of their private DNS will be very high.
Re: icmp rpf
[ Quotations have been reordered for clarity in the reply ] On 24 Sep 2006, at 22:59, Mark Kent wrote: If so, which of these two nets is unreasonable in their actions/ policies? I don't think either are *unreasonable* in what they've done. Both actions are prima facie reasonable but have an unforeseen synthesis that is undesirable. One of the largest North American network providers filters/drops ICMP messages so that they only pass those with a source IP address that appears in their routing table. This is clearly reasonable as part of an effort to mitigate ICMP based network abuse. In fact, I'd argue that it would probably be reasonable to drop any packet, not just ICMP, based on its absence from the routing table - conditioned on having a full, stable routing table. A smaller North American network provider, with a modest North American backbone, numbers their internal routers on public IP space that they do not announce to the world. Several people have mooted this as good practice on the basis routers do not need to be reachable (as an end system) except by legitimate managers of those routers (i.e. within the AS in question). As a result, traceroutes from big.net into small.net have numerous hops that time out. Traceroutes from elsewhere that go into small.net but return on big.net also have numerous hops that time out. We do all still think that traceroute is important, don't we? On balance, it's small.net that will have to change to rectify this. My argument would be: 1) Big.net's approach is wholly legitimate - deny spoofed packets transit. 2) Small.net's intentions are good, but they are pseudo-spoofing packets. ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. On balance both are acting with good intent, but small.net haven't fully seen the consequences for ICMP in their scheme. Please note that we're not talking about RFC1918 space, or reserved IP space of any kind. Also, think about the scenario where some failure happens leaving big.net with an incomplete routing table, thus breaking traceroute when it is perhaps most needed. Filtering ICMP is always dangerous. If you are going to do it you *must* understand the consequences both to yourself and to others, and also understand the consequences in both normal situations and all possible failure modes. (If I had a penny for every broken PMTU detection I'd seen because of someone's over eager filtering of ICMP...) Thanks, -mark
Re: icmp rpf
On Mon, Sep 25, 2006, Ian Mason wrote: Filtering ICMP is always dangerous. If you are going to do it you *must* understand the consequences both to yourself and to others, and also understand the consequences in both normal situations and all possible failure modes. (If I had a penny for every broken PMTU detection I'd seen because of someone's over eager filtering of ICMP...) Is there a BCP for handling ICMP? I'm walking the Cisco certification path and they're quite vocal about ICMP rate limiting over any kind of filtering on routers/switches. I haven't read their firewall documentation so I'm not sure what they're preaching for PIX/ASA. (Yup, if I had a penny for every PMTU fix-by-unbreaking-ICMP-filtering I've repaired over the last 10 years..) Adrian
Re: icmp rpf
On Sun, Sep 24, 2006 at 02:59:50PM -0700, Mark Kent wrote: A smaller North American network provider, with a modest North American backbone, numbers their internal routers on public IP space that they do not announce to the world. One of the largest North American network providers filters/drops ICMP messages so that they only pass those with a source IP address that appears in their routing table. I would hope they're doing it for more than just ICMP packets. There are numerous nefarious uses of the network with unrouted/spoofed addres space. Various hosts have done bad things (in the past) if they get something like a SYN that appears to be from themselves. Protecting ones customers from spoofed address DoS attacks and leaking of unrouted IP space (1918 or otherwise) that isn't globally reachable I would argue should be, or is a current best practice. The good packets that are dropped in this scenario are sufficent limited (yes, pmtu and these cases of traceroutes, etc..) but there are also well known solutions and workarounds to this as well. It's still hard to get people to fix their deny all icmp policies that some companies have that create troubles for others. I've had issues accessing my own bank website in the past due to p-mtu issues. These aren't places that are easily approachable to resolve the problem in most cases. As a result, traceroutes from big.net into small.net have numerous hops that time out. Others have pointed out how this can be resolved by by using different techniques and still protect the infrastructure. It may be of value for small.net to look at it and see what applies to them. Traceroutes from elsewhere that go into small.net but return on big.net also have numerous hops that time out. We do all still think that traceroute is important, don't we? I agree traceroute is important and valuable. It's one of the things I have asked people to send me in the past for debugging, but isn't the sole source of debugging available. Other techniques can be applied. Did big.net just turn this on, or has it been on for months/years now? - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
New router feature - icmp error source-interface [was: icmp rpf]
On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick
Re: Topicality perceptions
J. Oquendo rambled incoherently, saying in relevant part: William Allen Simpson wrote: Especially as I'm not aware of any Network Operator worth their salt that doesn't have regular contact with their support call centers. Regular contact? As in finding the name of someone who actually has a clue? Not the contact information of some helpdesk goon who doesn't understand the output of a traceroute? As in some helpdesk goon who understands what an AS is? You are a Network Operator, and you hired support personnel that doesn't understand the output of a traceroute and/or what an AS is? Perhaps the real problem here is some folks have lost sight of what it means to be a Network Operator?
Re: NANOG Thread
Well, if anyone wants to add more to it, there are quite a few prominent 'noggers still to cast.
Re: New router feature - icmp error source-interface [was: icmp rpf]
Patrick W. Gilmore wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? I do. I have suggested much the same in the past.
802.3ad/LACP between Administrative Domains
We have a GE link to another SP and bridge a single VLAN ID to connect multiple hosts on each side. We'd like to increase the BW between the two networks, but the other provider cannot support upgrading to 10GE. What are the issues w/ running 802.3ad LACP between two separately managed networks and binding Nx1GE links: - Does this require BPDU forwarding across the links, and share a common STP for the VLAN? We currently filter BPDU's across the single link. - Any known LACP negotiation issues between a Cisco (6500) and Foundry (unkn)? - Is there the potential for layer 2 loops if LACP fails and BPDU's are being filtered across the links? - Bad experiences w/ 802.3ad when loosing one of the two links? Thank you for your input.
Re: NANOG Thread
On Mon, 25 Sep 2006, Alexander Harrowell wrote: Well, if anyone wants to add more to it, there are quite a few prominent 'noggers still to cast. Can I be at the bottom of each thread, for when it really gets into wanker territory? Thanks. - billn
Re: icmp rpf
Jared Mauch wrote: I would hope they're doing it for more than just ICMP packets. yes, loose RPF, but I just care about ICMP. I would argue should be, or is a current best practice. OK, so I must have missed the memo :-) Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF (not just strict RPF on single-homed customers)? Did big.net just turn this on, or has it been on for months/years now? I'm pretty sure it's months and not years. I've noticed it for a while, but it just recently drove me to the point where I'd complain about it. Thanks, -mark
Re: icmp rpf
In response to this: Mark Smith wrote: The non-announcers, because they're also breaking PMTUD. Really? How? Mark Smith replied with two paragraphs, but it's not 100% clear to me that he got the reason why I asked. I asked because his initial statement boiled down to numbering on un-announced space breaks PMTUD... but it doesn't, not by itself (which he later expanded). It only does so in the presence of filtering. I think this is an important point to make because of my interaction with small.net. When I pointed out the timeouts they said that it was because they don't announce the router IP addresses, which is true but not the whole story. I mentioned that some providers in the past numbered on rfc1918 space and traceroute still worked, so that alone was not enough. Then they said it's because of the asymmetric path, and that also is true, but again not enough. A large proportion of traffic is asymmetric, but traceroute still works. Then they gave me an explanation that rested on the fact that the routers will not respond to pings because they are unannounced outside of their world. That too is true, but irrelevant and I told them how Jacobson's traceroute works and told them that *someone* was dropping/filtering the return packets and I'ld like to know who/why. They somewhat implied that it was my fault, and this situation was unique to my net, so I used the big.net looking glass to show how the same things happens from space not associated with my network. (Yes, I should have done this from the outset.) With that they asked big.net, and big.net said they filtered, and that's where we are. My point here is that it took me ten (10) emails with small.net to get this information partly because the small.net support staff had notions in their head premised on too simplistic statements like numbering on un-announced space breaks PMTUD. I wanted to clear this up because this list is likely read by support people at various networks, and it's pretty clear that not all of them are well versed even on something as thoroughly discussed over the ages as traceroute. Thanks, -mark
Re: NANOG Thread
no; what OS and what applications are you using? Anything particularly unusual? On Sep 25, 2006, at 8:55 AM, [EMAIL PROTECTED] wrote: On Mon, 25 Sep 2006, Alexander Harrowell wrote: Well, if anyone wants to add more to it, there are quite a few prominent 'noggers still to cast. Can I be at the bottom of each thread, for when it really gets into wanker territory? Thanks. - billn
Re: icmp rpf
Once upon a time, Mark Kent [EMAIL PROTECTED] said: I think this is an important point to make because of my interaction with small.net. When I pointed out the timeouts they said that it was because they don't announce the router IP addresses, which is true but not the whole story. I mentioned that some providers in the past numbered on rfc1918 space and traceroute still worked, so that alone was not enough. Not announcing their router interface IP space is not any type of security. Anyone directly connected to them (customer or peer) could if they wish statically route that IP space, and any such security would be gone. Unless it is otherwise filtered, any customer with a default route can reach their routers. -- Chris Adams [EMAIL PROTECTED] Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Re: NANOG Thread
On Mon, 25 Sep 2006, Fred Baker wrote: no; what OS and what applications are you using? Anything particularly unusual? Everything is custom. Cisco crust on top, mystery meat on the bottom. (Not to be confused with 'deviled ham.' It's all held together with a couple of Perl brand farm fresh eggs. There are 485 different SNMP-capable thermometers sticking out of my network meat pie, to inform me as to precise freshness throughout the finished product. Not all of them work. - billn
Re: fyi-- [dns-operations] early key rollover for dlv.isc.org
On Fri, 22 Sep 2006 17:01:31 -0700 (PDT), Gregory Hicks [EMAIL PROTECTED] wrote: On Fri, Sep 22, 2006 at 11:39:51PM +, Fergie wrote: Hmmm. It wouldn't have anything to do with prime numbers, now would it? :-) Well, yes, but there are an infinite number of them. Of course, 17 is the most prime of them all. isc.org announced the early key rollover just as a discussion about exponent 3 damage spreads on the cryptography list was heating up. This discussion started with a statement that: I've just noticed that BIND is vulnerable to: http://www.openssl.org/news/secadv_20060905.txt Executive summary: RRSIGs can be forged if your RSA key has exponent 3, which is BIND's default. Note that the issue is in the resolver, not the server. Fix: Upgrade OpenSSL. So I thought that the early key rollover was due to this. Yet it seems to me that this discussion is still recommending that -e 3 be used. There are no known attacks on e=3 *if* everything else is done properly. There have, however, been many different attacks if mistakes are made, such as the implementation attacks here or various problems with the padding scheme. See, for example, http://www.rsasecurity.com/rsalabs/staff/bios/bkaliski/publications/hash-firewalls/kaliski-hash-firewalls-ct-rsa-2002.pdf http://citeseer.ist.psu.edu/746101.html http://citeseer.ist.psu.edu/coppersmith96lowexponent.html Poking through the cryptanalytic literature shows many other problems and near-problems with small exponents and RSA. My conclusion is that e=3 is too fragile -- it's too easy to make mistakes (or do things that are later determined to be mistakes by mathematicians). NIST's latest draft of FIPS-186-3 says: (b) The exponent e shall be an odd positive integer such that 65,537 = e 2**(nlen - 2*security_strength) where nlen is the length of the modulus n in bits. (security_strength appears to be the symmetric system attack work factor, i.e., 128 for AES-128.) They don't give a reason; we can assume, though, that their friends in Ft. Meade specified it. (Why the upper bound? It turns out that you don't want the decryption exponent to be too small, either...) So -- my very strong recommendation is that e=3 be avoided. For efficiency in implementation, numbers of the form 2^2^n+1 are good for e. Numbers of that form are known as Fermat Numbers; see http://en.wikipedia.org/wiki/Fermat_prime . e=5 is almost as vulnerable as e=3, especially for larger RSA moduli. e=17 might be at risk for really large moduli to match large AES keys (see RFC 3766). I don't know why F3 (257) isn't a good choice, but 65537 has been a popular alternative for years. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Re: icmp rpf
On Mon, 25 Sep 2006, Chris Adams wrote: Once upon a time, Mark Kent [EMAIL PROTECTED] said: I think this is an important point to make because of my interaction with small.net. When I pointed out the timeouts they said that it was because they don't announce the router IP addresses, which is true but not the whole story. I mentioned that some providers in the past numbered on rfc1918 space and traceroute still worked, so that alone was not enough. Not announcing their router interface IP space is not any type of security. Anyone directly connected to them (customer or peer) could if they wish statically route that IP space, and any such security would be gone. Unless it is otherwise filtered, any customer with a default route can reach their routers. Nevertheless putting router interface ip address for your network in one specific block is very effective as way to quickly get rid of DoS attack on the router - you simply stop announcing that block but everything else on the network still works. And doing tricks like having primary ip address which not important at all (except for logging traffic actually destined to it) while secondary ip on the same interface is really the one used for inter-connection also works quite well. -- William Leibzon Elan Networks [EMAIL PROTECTED]
Re: icmp rpf
On Sep 25, 2006, at 12:22 PM, Mark Kent wrote: Jared Mauch wrote: I would hope they're doing it for more than just ICMP packets. yes, loose RPF, but I just care about ICMP. I would argue should be, or is a current best practice. OK, so I must have missed the memo :-) It's been all the rage. :) Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF (not just strict RPF on single-homed customers)? I'm wondering why that is relevant. If all those ASNs only filtered on ASN instead of prefix for customer announcements (for instance), would that mean no one should? -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, 25 Sep 2006 09:22:34 -0400 Patrick W. Gilmore [EMAIL PROTECTED] wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? I do. -- Sheep are slow and tasty, and therefore must remain constantly alert. - Bruce Schneier, Beyond Fear
RE: New router feature - icmp error source-interface [was: icmp rpf]
Might this not be a bad idea if the router has interfaces on multiple, separate paths? Such a case may be where one customer or set of traffic routes over a link to ISP A, and other traffic over a link to ISP B, and not all related addresses are portable. In that case the loopback address for the ICMP errors might show from an address that seems to belong to a block from ISP A, but is really traffic that was transported on ISP B. Just trying to come up with possible issues, not saying that's a best practice or anything... -Scott -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 9:23 AM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Sep 25, 2006, at 5:26 PM, Berkman, Scott wrote: Might this not be a bad idea if the router has interfaces on multiple, separate paths? Such a case may be where one customer or set of traffic routes over a link to ISP A, and other traffic over a link to ISP B, and not all related addresses are portable. In that case the loopback address for the ICMP errors might show from an address that seems to belong to a block from ISP A, but is really traffic that was transported on ISP B. Just trying to come up with possible issues, not saying that's a best practice or anything... I doubt it is possible to make a rule / knob / idea / feature / whatever that cannot be misused, or that is applicable to all corner cases. That's why it's a knob. :) If it is applicable to your situation, you should use it. If not, not. But if the knob has enough use in enough situations, then it is probably something we want to push the vendors to implement. -- TTFN, patrick -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 9:23 AM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? You know I was just having this discussion with someone else a couple days ago. It turns out, much to my surprise, that the RFC actually calls for the ICMP error-message packet (as you said, the things that aren't ping etc which require a specific source-address) to originate from the OUTGOING interface used to return the ICMP message to the original sender. After much googling, I can't find any document where this has ever been officially updated either. The defacto industry standard on the other hand has been to use the primary address of the inbound interface, which serves exactly one function: it makes traceroute work. The hack that people use to simulate this functionality normally is: ip address the.fake.ip.here 255.255.255.252 ip address the.real.ip.here 255.255.255.252 secondary (FYI side note the Juniper equivilent is... confusing or non-working, hard to tell which. The tag primary is defined as the source address for local broadcast/multicast, preferred is defined as the source when you have multiple IPs within a subnet. Neither one should work for what we're talking about according to the docs, but if you actually try it... sometimes it works, sometimes it won't, and sometimes the behavior is different if you include only one but not the other :P). This works well for simple external-facing interfaces, things that speak BGP for example, but can confuse things like OSPF etc when you use secondaries on internal interfaces. FWIW I've been asking for more people to implement exactly what you're talking about for years (specifically setting ONLY the ICMP error source interface without risk of screwing up the interface in other ways). You can debate over exactly how to do it (global or per interface, icmp source-interface lo99 vs icmp source-ip 1.2.3.4, etc), but I agree wholeheatedly it should be done. It should be really simple too! (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) Please stop talking about networking on NANOG, you're confusing people. :) -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
RE: New router feature - icmp error source-interface [was: icmp rpf]
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 5:31 PM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: Re: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 5:26 PM, Berkman, Scott wrote: Might this not be a bad idea if the router has interfaces on multiple, separate paths? Such a case may be where one customer or set of traffic routes over a link to ISP A, and other traffic over a link to ISP B, and not all related addresses are portable. In that case the loopback address for the ICMP errors might show from an address that seems to belong to a block from ISP A, but is really traffic that was transported on ISP B. Just trying to come up with possible issues, not saying that's a best practice or anything... I doubt it is possible to make a rule / knob / idea / feature / whatever that cannot be misused, or that is applicable to all corner cases. That's why it's a knob. :) If it is applicable to your situation, you should use it. If not, not. But if the knob has enough use in enough situations, then it is probably something we want to push the vendors to implement. -- TTFN, patrick -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 9:23 AM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick C and J both already have a similar feature, however I'm not sure whether or not they apply to ICMP. They both support PBR for locally originated packets - which, should include if the thought process is correct, ICMP. Perhaps someone with some time to waste can verify this in a lab. http://www.cisco.com/en/US/customer/products/sw/iosswrel/ps1828/products _configuration_guide_chapter09186a00800ca590.html#5406 -Dave
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Sep 25, 2006, at 5:40 PM, Richard A Steenbergen wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? You know I was just having this discussion with someone else a couple days ago. It turns out, much to my surprise, that the RFC actually calls for the ICMP error-message packet (as you said, the things that aren't ping etc which require a specific source-address) to originate from the OUTGOING interface used to return the ICMP message to the original sender. After much googling, I can't find any document where this has ever been officially updated either. The defacto industry standard on the other hand has been to use the primary address of the inbound interface, which serves exactly one function: it makes traceroute work. I have not read the RFC in full, but after chatting with Daniel offline (see, some people actually do talk without posting! :), I believe this only applies to packets addressed to the router. Since packets going -through- the router have absolutely no guarantee what source will be used coming back, I don't seen an issue here. Just change the idea such that it only is used for error messages to packets where the dest addy is not an interface on the router. Also, this makes traceroute -easier- to use. Suddenly all interfaces on the same router have the same IP address, thereby making it easy to tell if two traceroutes intersect, even if they use different interfaces. Oh, and who said RFCs can't be updated? :-) (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) Please stop talking about networking on NANOG, you're confusing people. :) I knew someone would flame me for it. :) -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 04:33:18PM -0700, David Temkin wrote: C and J both already have a similar feature, however I'm not sure whether or not they apply to ICMP. They both support PBR for locally originated packets - which, should include if the thought process is correct, ICMP. Perhaps someone with some time to waste can verify this in a lab. http://www.cisco.com/en/US/customer/products/sw/iosswrel/ps1828/products _configuration_guide_chapter09186a00800ca590.html#5406 The actual path taken for the ICMP generated by the router does not matter, we're just talking about the source address selected by the router. The only reasons that the source address (which reveals a real IP address on a router) matters at all for ICMP error responses are: * So traceroute works (current industry standard behavior is to use the ingress interface IP so you see the forward path in traceroute, not the reverse path, which may be asymmetric. * So your replies don't get thwacked by people doing uRPF strict (i.e. they must come from announced IPs or people doing strict strict with no exception filtering capabilities will block the traceroute responses). * Optionally, allowing naive tools like MTR to ping the IP they discover via traceroute, lest weenies flood your noc with I'm pinging 10lolz! emails. Revealing your interface IPs carries all kinds of DoS/security risks with it, since there are a great many routers out there without good control plane policing functionality (and even some of those that have it, don't really have it :P). Since there is really no legitimate need for people from the outside world to ever communicate with your real interface IPs at all (with the exception of some rate limited ICMP echo/reply due to aforementioned mtr weenies), having the option to hide those real addresses completely in ICMP source address selection is a very good thing for enhancing network security. As I said you can accomplish part of this hack with primary/secondary IPs on interfaces. You can also accomplish some level of filtering by numbering your interfaces out of common blocks which are filtered at your various borders/edges. It's still a pain in the !(*#*, especially if you number your links out of any regional blocks to cut down on asymmetric routing confusion, or have any number of peers who provide /30s from their own IP space. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: New router feature - icmp error source-interface [was: icmp rpf]
At 9:22 AM -0400 9/25/06, Patrick W. Gilmore wrote: Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? It certainly would beat the alternative of no response at all, but one would hope it wouldn't become common practice since it reduces the information returned (e.g. during a traceroute, you'd lose the sometimes useful information from in-addr about what particular interface was involved). /John
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 08:45:49PM -0400, John Curran wrote: At 9:22 AM -0400 9/25/06, Patrick W. Gilmore wrote: Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? It certainly would beat the alternative of no response at all, but one would hope it wouldn't become common practice since it reduces the information returned (e.g. during a traceroute, you'd lose the sometimes useful information from in-addr about what particular interface was involved). Personally I'd hope that if it was implemented, it would support mapping on a per-interface basis (especially for NSP use). That should in theory lead to even more accurate information, since each network would be capable of easily renumbering without impact, and managing their own DNS for every interface. Currently a great many PTRs are out of date because IP blocks supplied by peers, exchange points, or transit providers, are too much of a pain to keep updated when interfaces move etc. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Comcast contact
Can someone from comcast contact me off list please ? Thanks, Ansh Kanwar Lead Network Engineer -- Citrix Online (AS16815) 5385 Hollister Avenue Santa Barbara, CA 93111 USA --
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that told me into which router by which interface I was going. When the owner of the router might not even have known. Or I have had attempts to do this foiled by routers that used an internal loopback IP address. On the whole, then, I guess I would vote, no. -- Joe Yao --- This message is not an official statement of OSIS Center policies.
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, 25 Sep 2006, Joseph S D Yao wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix I think in the original poster's scenario one network was looking to protect their resources/equipment from a majority of the network's ills. It's not unreasonable... atleast not in my mind. It's also not 'security through obscurity' since one of the parties is/was leaking their information OUT, just not 'in' :) something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that What's interesting is that today, in many networks, the usefulness of traceeroute has bee degraded by other non-ip issues (coughmpls/cough) not in ALL cases, but certainly in many you are not seeing quite what you'd expect from the traceroute :(
Re: New router feature - icmp error source-interface [was: icmp rpf]
Chris, So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) http://momo.lcs.mit.edu/traceroute/index.php The best (?) of both worls, but I digress... - ferg -- Chris L. Morrow [EMAIL PROTECTED] wrote: [snip] What's interesting is that today, in many networks, the usefulness of traceeroute has bee degraded by other non-ip issues (coughmpls/cough) not in ALL cases, but certainly in many you are not seeing quite what you'd expect from the traceroute :( -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Tue, 26 Sep 2006, Fergie wrote: Chris, So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) :) depends on the network I guess... I'm not sure it's going to tell you anything about hops hidden by mpls lsp's that don't decrement ttl though...
Re: New router feature - icmp error source-interface [was: icmp rpf]
Ah, but there's the rub... ISPs who are discreet in how they wish their infrastructure to be viewed will continue to engineer methods in which portions are not visible to the public at-large. Somehow, I don't think that will ever go away, so trying to tilt at windmils w.r.t. (paraphrased) ...what interface an ICMP foo is sourced from... is, indeed... tiling at windmills, methinks. :-) Cheers! - ferg -- Chris L. Morrow [EMAIL PROTECTED] wrote: On Tue, 26 Sep 2006, Fergie wrote: Chris, So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) :) depends on the network I guess... I'm not sure it's going to tell you anything about hops hidden by mpls lsp's that don't decrement ttl though... -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
Re: New router feature - icmp error source-interface [was: icmp rpf]
At 10:29 PM 9/25/2006, Chris L. Morrow wrote: On Mon, 25 Sep 2006, Joseph S D Yao wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix I think in the original poster's scenario one network was looking to protect their resources/equipment from a majority of the network's ills. It's not unreasonable... atleast not in my mind. It's also not 'security through obscurity' since one of the parties is/was leaking their information OUT, just not 'in' :) The issue is that the world has changed. Some networks actually expect not only the use of public addresses on router interfaces, but addresses from advertised blocks. Why has the world changed? Because not everyone is friendly on the Internet. There were issues raised in Mobile IP by the drafts that predated the publication of RFC 2267. Why? Well, Mobile IP WG had made the assumption that all IPv4 packet forwarding would be done, without filtering, based solely on the destination IP address. The down side was it made some of the traffic look spoofed. The world changed, people started abusing the Internet in new ways, and the old assumptions no longer held. Mobile IP WG adapted to the new reality that was coming. The longevity of the TCP/IP protocol suite is attributable to the continued effort of many people, not to savant qualities of those who first wrote specifications. something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that What's interesting is that today, in many networks, the usefulness of traceeroute has bee degraded by other non-ip issues (coughmpls/cough) not in ALL cases, but certainly in many you are not seeing quite what you'd expect from the traceroute :( There's no more degradation of usefulness from MPLS than there was from ATM or Frame Relay. Pick your poison for virtualized link layer, and you'll have some degree of difficulty determining and debugging the underlying network without tools to peer under the hood.
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Tue, Sep 26, 2006 at 02:51:21AM +, Fergie wrote: So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) http://momo.lcs.mit.edu/traceroute/index.php The best (?) of both worls, but I digress... That doesn't show any more or less data about the path, just some extra info about the label that is effectively useless to end users. If TTL decrement is not enabled, all of the IP hops are hidden by the tunnel, which is the point Chris was making. But that said, I personally think Cisco MPLS with TTL decrement enabled but returning the the same rtt as the penultimate hop for every IP hop inside the LSP has caused far more harm to every NOC ticket queue on the planet than just hiding the damn things. While we're asking for silly features, I can name a LOT of people who would pay good money for a dedicated ICMP generating processor on Cisco that doesn't spike every time BGP scanner runs. Silencing end users who have figured out how to work traceroute (or worse MTR) is worth its weight in gold. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
decline of customer service
Times have changed, My experience has been recently that ISP's and ASP's have dramatically malnourished their first level support staff which in turn has created a resentful and lazy second teir. I am sick of the It must be your network/cabling/CPE attitude that I am getting from some teir 1 ISP's. I sick of replacing CSU's and checking extended demarcs while some clown in the POP is re-seating cards in the mux. Moreover stop accusing my network of latency issues. I ran the packet capture 100 times and the client is still send a FIN. The reason your application is slow is because your programmers think sockets are something you plug a can opener into. Finally, YOU are my vendor. I pay you money for exceptional service. Thank you for your time.
Re: New router feature - icmp error source-interface [was: icmp rpf]
Joseph S D Yao wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that told me into which router by which interface I was going. When the owner of the router might not even have known. Or I have had attempts to do this foiled by routers that used an internal loopback IP address. On the whole, then, I guess I would vote, no. Why not just do a show ip route? since you can actually verify the information against your routing table. This way you can see when the route was learned, where was it learned from and how long ago it was last updated... the problem is that too many people engineers rely on traceroute... sure traceroute is a wonderful tool, however it is meant to assist you in tracking down the problem. I've seen far too many you are filtering, investigate please when all that has been done is implementing acls and rate limiting. IMO, If you want to implement a non-routable ip space to protect your backbone... go for it if you want to icmp rate limit *i know level3 does this out of both nyc and la* which causes mass threads of we are getting packet loss, please investigate go for it .. if your network engineers are not equipped with the information to how to fully diagnose a network/problem you should think about new hires. Cheers, Payam
