Re: icmp rpf
Possible approach for small.net - ok, you know that big.net will drop any packets sourced from x.x.x.x if there's no route there (loose uRPF for downstream ISPs like small.net, strict uRPF for end-users.) So give them a route. Either give them a route on one of your direct interfaces to them, and then get rid of the packets by ACL or by null-routing it, or if that causes too much trouble, get yourself a 56kbps line from a spare router and advertise it from there.
Re: icmp rpf
On Tue, Sep 26, 2006 at 12:17:27AM -0700, Tony Rall wrote: On Monday, 2006-09-25 at 10:09 MST, Mark Kent [EMAIL PROTECTED] wrote: Mark Smith replied with two paragraphs, but it's not 100% clear to me that he got the reason why I asked. I asked because his initial statement boiled down to numbering on un-announced space breaks PMTUD... but it doesn't, not by itself (which he later expanded). It only does so in the presence of filtering. Which is exactly what one might expect to happen. At least it seems to me that RFC 3704 (BCP 84, http://www.ietf.org/rfc/rfc3704.txt) applies. When your traffic is sourced with dubious addresses, you should expect much of it to disappear. And when this happens, you're hurting your customers and your customers' customers (okay, sometimes it's just your peer's customers - still a concern in my opinion). I think this is the critical point, dubious ip sources have been used/abused in the past and those at big.net that have done something to mitigate the risk to the world from their customers and their customers from the world are doing a community service imnsho ;). I don't see anyone here really advocating spoofed sources (except for perhaps the mobile-ip users out there ;) How many people here have rpf enabled by default on their customer CPE devices they ship? (in your template or whatnot) Do you u-rpf your dsl/cable/dial/fract-t1/t1 customers that are not doing bgp? It's hard to get this implemented across an entire network. At one time I seem to recall someone saying that 7018 was fully bcp-38 compliant, but I could be wrong. While doing u-rpf on your customers won't mitigate attacks against them, it will help mitigate the costs of tracking spoofed attacks across your network infrastructure (which is harder if you're doing mpls) that you incur. Now, i'm guessing i may be the one responsible for the practices of big.net, but i do encourage other big.nets to enable u-rpf strict or loose wherever possible based on their equipment capabilities. Every little bit will help. - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Re: icmp rpf
On Monday, 2006-09-25 at 10:09 MST, Mark Kent [EMAIL PROTECTED] wrote: Mark Smith replied with two paragraphs, but it's not 100% clear to me that he got the reason why I asked. I asked because his initial statement boiled down to numbering on un-announced space breaks PMTUD... but it doesn't, not by itself (which he later expanded). It only does so in the presence of filtering. Which is exactly what one might expect to happen. At least it seems to me that RFC 3704 (BCP 84, http://www.ietf.org/rfc/rfc3704.txt) applies. When your traffic is sourced with dubious addresses, you should expect much of it to disappear. And when this happens, you're hurting your customers and your customers' customers (okay, sometimes it's just your peer's customers - still a concern in my opinion). -- Tony Rall
Re: icmp rpf
At 10:06 25/09/2006, Ian Mason wrote: One of the largest North American network providers filters/drops ICMP messages so that they only pass those with a source IP address that appears in their routing table. This is clearly reasonable as part of an effort to mitigate ICMP based network abuse. As a matter of fact, most ICMP-based attacks don't require spoofing of the source IP address. You do have to spoof the addresses in the original datagram included in the ICMP payload, though. Kindest regards, -- Fernando Gont e-mail: [EMAIL PROTECTED] || [EMAIL PROTECTED] PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
Re: icmp rpf
I asked: Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF (not just strict RPF on single-homed customers)? and Patrick answered: I'm wondering why that is relevant. It's relevant because it was suggested that loose RPF should be a best common practice so I was curious which of those ASes decided that the benefits outweighed the negatives and actually do it. Don't worry, those were randomly chosen AS. I didn't intend to make any suggestion that the answer would be more important to me for that set of ASes than any other. But, you were correct that I wasn't asking the question I really wanted answered. What I wanted to know was, among the attentive nanog membership, which of you think and/or know that any/all of those AS do loose RPF? The motivation here is that, if asked last week, I would have guessed that none of them run loose RPF. But at least one of them does. The two answers, how many actually do plus whether everyone knew it, will help me decide if I need to spend more time reading nanog email and nanog proceedings (or actually go to a meeting), or not... Thanks, -mark
Re: icmp rpf
On Sep 26, 2006, at 11:57 AM, Mark Kent wrote: I asked: Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF (not just strict RPF on single-homed customers)? and Patrick answered: I'm wondering why that is relevant. It's relevant because it was suggested that loose RPF should be a best common practice so I was curious which of those ASes decided that the benefits outweighed the negatives and actually do it. Don't worry, those were randomly chosen AS. I didn't intend to make any suggestion that the answer would be more important to me for that set of ASes than any other. The actual practices of a network are not necessarily a way to look at what best common practices should be. For instance, how many networks are in full compliance with BCP38? Or are you arguing that since essentially no one is compliant, we should scrap the BCP? But, you were correct that I wasn't asking the question I really wanted answered. What I wanted to know was, among the attentive nanog membership, which of you think and/or know that any/all of those AS do loose RPF? The motivation here is that, if asked last week, I would have guessed that none of them run loose RPF. But at least one of them does. The two answers, how many actually do plus whether everyone knew it, will help me decide if I need to spend more time reading nanog email and nanog proceedings (or actually go to a meeting), or not... Good question. waits for answers -- TTFN, patrick
Re: icmp rpf
On Tue, Sep 26, 2006 at 01:41:52PM -0400, Patrick W. Gilmore wrote: For instance, how many networks are in full compliance with BCP38? I've been working towards this on our network for some time but have been hindered by vendor.. uhm, features^Wbugs. eg: halving the TCAM with rpf enabled, one mode globally (loose vs strict) and other challenges. It is hard to imagine that we'll reach that point but that doesn't mean it's not a goal. Or are you arguing that since essentially no one is compliant, we should scrap the BCP? But, you were correct that I wasn't asking the question I really wanted answered. What I wanted to know was, among the attentive nanog membership, which of you think and/or know that any/all of those AS do loose RPF? The motivation here is that, if asked last week, I would have guessed that none of them run loose RPF. But at least one of them does. The two answers, how many actually do plus whether everyone knew it, will help me decide if I need to spend more time reading nanog email and nanog proceedings (or actually go to a meeting), or not... Good question. Well, digging out messages from archives http://www.merit.edu/mail.archives/nanog/2002-05/msg00289.html These features have been available in some form since at least 2002. That has given people at least a 4 year window of time to consider how much to reduce the (quoting barry) noise on the internet. I recall hearing of various root-server operators about what percentage of the packets they get they just can't respond to. This noise has cost to the common infrastructure that is used globally. You wouldn't believe which GTLD operator tried to spin up some government agencies about how bad the reflector attacks were to their infrastructure. It could be interpreted that they wanted a government subsidy to cover these increased infrastructure costs they would have to incur to handle the traffic. This is just one example (recently) of what happens without filters in-place. Not everyone on the list provides access to US Gov't agencies, but if they changed their purchasing to only acquire access from BCP38 compliant providers, would that impact the way you did business? Would it get insert-long-list-of-asns to change their network practices and hardware? I think any reasonable (market based) approaches to help nudge things in the right direction is better than if we were to hear the dreaded R word. That would not be a good situation for most of us. There are plenty that will advocate all sorts of positions, and it's honestly up to us to do the right thing for the right reasons otherwise we may see an even more imperfect solution come our ways. - Jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Re: icmp rpf
Hi Mark, On Sun, 24 Sep 2006 16:33:30 -0700 (PDT) Mark Kent [EMAIL PROTECTED] wrote: Mark Smith wrote: The non-announcers, because they're also breaking PMTUD. Really? How? Remember, we're not talking about RFC1918 space, where there is a BCP that says we should filter it at the edge. We're talking about public IP space, that just doesn't happen to be announced outside of a particular AS. When a router that can't shove a DF'd packet down a link because the MTU is too small needs to create a ICMP Destination Unreachable, Packet Too Big, Fragmentation Required, it needs to pick a source IP address to use for that ICMP packet, which will be one of those assigned to the router with the MTU problem (I'm fairly sure it's the IP address assigned to the outgoing interface for this ICMP packet, although I don't think it probably matters much). If an upstream router, i.e. on the way back to the sender who needs to resend with a smaller packet, is dropping these packets because they fail RPF, then PMTUD breaks. The result might be connection timeouts at the sender, or possibly after quite a while the sender might try smaller packets and eventually they'll get through (I think Windows might do this). Either way, bad end-user experience. PMTUD as it currently works isn't ideal, as of course there isn't any guarantee that these ICMP Dest Unreachables will get there even in a good network. However, most of the time it works, where as in the scenario you're presenting, it definately won't. Regards, Mark. -- Sheep are slow and tasty, and therefore must remain constantly alert. - Bruce Schneier, Beyond Fear
Re: icmp rpf
The non-announcers, because they're also breaking PMTUD. If you're not sure what benefits PMTUD gives, you might want to review this page: http://www.psc.edu/~mathis/MTU/index.html --Michael Dillon
Re: icmp rpf
[ Quotations have been reordered for clarity in the reply ] On 24 Sep 2006, at 22:59, Mark Kent wrote: If so, which of these two nets is unreasonable in their actions/ policies? I don't think either are *unreasonable* in what they've done. Both actions are prima facie reasonable but have an unforeseen synthesis that is undesirable. One of the largest North American network providers filters/drops ICMP messages so that they only pass those with a source IP address that appears in their routing table. This is clearly reasonable as part of an effort to mitigate ICMP based network abuse. In fact, I'd argue that it would probably be reasonable to drop any packet, not just ICMP, based on its absence from the routing table - conditioned on having a full, stable routing table. A smaller North American network provider, with a modest North American backbone, numbers their internal routers on public IP space that they do not announce to the world. Several people have mooted this as good practice on the basis routers do not need to be reachable (as an end system) except by legitimate managers of those routers (i.e. within the AS in question). As a result, traceroutes from big.net into small.net have numerous hops that time out. Traceroutes from elsewhere that go into small.net but return on big.net also have numerous hops that time out. We do all still think that traceroute is important, don't we? On balance, it's small.net that will have to change to rectify this. My argument would be: 1) Big.net's approach is wholly legitimate - deny spoofed packets transit. 2) Small.net's intentions are good, but they are pseudo-spoofing packets. ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. On balance both are acting with good intent, but small.net haven't fully seen the consequences for ICMP in their scheme. Please note that we're not talking about RFC1918 space, or reserved IP space of any kind. Also, think about the scenario where some failure happens leaving big.net with an incomplete routing table, thus breaking traceroute when it is perhaps most needed. Filtering ICMP is always dangerous. If you are going to do it you *must* understand the consequences both to yourself and to others, and also understand the consequences in both normal situations and all possible failure modes. (If I had a penny for every broken PMTU detection I'd seen because of someone's over eager filtering of ICMP...) Thanks, -mark
Re: icmp rpf
On Mon, Sep 25, 2006, Ian Mason wrote: Filtering ICMP is always dangerous. If you are going to do it you *must* understand the consequences both to yourself and to others, and also understand the consequences in both normal situations and all possible failure modes. (If I had a penny for every broken PMTU detection I'd seen because of someone's over eager filtering of ICMP...) Is there a BCP for handling ICMP? I'm walking the Cisco certification path and they're quite vocal about ICMP rate limiting over any kind of filtering on routers/switches. I haven't read their firewall documentation so I'm not sure what they're preaching for PIX/ASA. (Yup, if I had a penny for every PMTU fix-by-unbreaking-ICMP-filtering I've repaired over the last 10 years..) Adrian
Re: icmp rpf
On Sun, Sep 24, 2006 at 02:59:50PM -0700, Mark Kent wrote: A smaller North American network provider, with a modest North American backbone, numbers their internal routers on public IP space that they do not announce to the world. One of the largest North American network providers filters/drops ICMP messages so that they only pass those with a source IP address that appears in their routing table. I would hope they're doing it for more than just ICMP packets. There are numerous nefarious uses of the network with unrouted/spoofed addres space. Various hosts have done bad things (in the past) if they get something like a SYN that appears to be from themselves. Protecting ones customers from spoofed address DoS attacks and leaking of unrouted IP space (1918 or otherwise) that isn't globally reachable I would argue should be, or is a current best practice. The good packets that are dropped in this scenario are sufficent limited (yes, pmtu and these cases of traceroutes, etc..) but there are also well known solutions and workarounds to this as well. It's still hard to get people to fix their deny all icmp policies that some companies have that create troubles for others. I've had issues accessing my own bank website in the past due to p-mtu issues. These aren't places that are easily approachable to resolve the problem in most cases. As a result, traceroutes from big.net into small.net have numerous hops that time out. Others have pointed out how this can be resolved by by using different techniques and still protect the infrastructure. It may be of value for small.net to look at it and see what applies to them. Traceroutes from elsewhere that go into small.net but return on big.net also have numerous hops that time out. We do all still think that traceroute is important, don't we? I agree traceroute is important and valuable. It's one of the things I have asked people to send me in the past for debugging, but isn't the sole source of debugging available. Other techniques can be applied. Did big.net just turn this on, or has it been on for months/years now? - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
New router feature - icmp error source-interface [was: icmp rpf]
On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
Patrick W. Gilmore wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? I do. I have suggested much the same in the past.
Re: icmp rpf
Jared Mauch wrote: I would hope they're doing it for more than just ICMP packets. yes, loose RPF, but I just care about ICMP. I would argue should be, or is a current best practice. OK, so I must have missed the memo :-) Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF (not just strict RPF on single-homed customers)? Did big.net just turn this on, or has it been on for months/years now? I'm pretty sure it's months and not years. I've noticed it for a while, but it just recently drove me to the point where I'd complain about it. Thanks, -mark
Re: icmp rpf
In response to this: Mark Smith wrote: The non-announcers, because they're also breaking PMTUD. Really? How? Mark Smith replied with two paragraphs, but it's not 100% clear to me that he got the reason why I asked. I asked because his initial statement boiled down to numbering on un-announced space breaks PMTUD... but it doesn't, not by itself (which he later expanded). It only does so in the presence of filtering. I think this is an important point to make because of my interaction with small.net. When I pointed out the timeouts they said that it was because they don't announce the router IP addresses, which is true but not the whole story. I mentioned that some providers in the past numbered on rfc1918 space and traceroute still worked, so that alone was not enough. Then they said it's because of the asymmetric path, and that also is true, but again not enough. A large proportion of traffic is asymmetric, but traceroute still works. Then they gave me an explanation that rested on the fact that the routers will not respond to pings because they are unannounced outside of their world. That too is true, but irrelevant and I told them how Jacobson's traceroute works and told them that *someone* was dropping/filtering the return packets and I'ld like to know who/why. They somewhat implied that it was my fault, and this situation was unique to my net, so I used the big.net looking glass to show how the same things happens from space not associated with my network. (Yes, I should have done this from the outset.) With that they asked big.net, and big.net said they filtered, and that's where we are. My point here is that it took me ten (10) emails with small.net to get this information partly because the small.net support staff had notions in their head premised on too simplistic statements like numbering on un-announced space breaks PMTUD. I wanted to clear this up because this list is likely read by support people at various networks, and it's pretty clear that not all of them are well versed even on something as thoroughly discussed over the ages as traceroute. Thanks, -mark
Re: icmp rpf
Once upon a time, Mark Kent [EMAIL PROTECTED] said: I think this is an important point to make because of my interaction with small.net. When I pointed out the timeouts they said that it was because they don't announce the router IP addresses, which is true but not the whole story. I mentioned that some providers in the past numbered on rfc1918 space and traceroute still worked, so that alone was not enough. Not announcing their router interface IP space is not any type of security. Anyone directly connected to them (customer or peer) could if they wish statically route that IP space, and any such security would be gone. Unless it is otherwise filtered, any customer with a default route can reach their routers. -- Chris Adams [EMAIL PROTECTED] Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Re: icmp rpf
On Mon, 25 Sep 2006, Chris Adams wrote: Once upon a time, Mark Kent [EMAIL PROTECTED] said: I think this is an important point to make because of my interaction with small.net. When I pointed out the timeouts they said that it was because they don't announce the router IP addresses, which is true but not the whole story. I mentioned that some providers in the past numbered on rfc1918 space and traceroute still worked, so that alone was not enough. Not announcing their router interface IP space is not any type of security. Anyone directly connected to them (customer or peer) could if they wish statically route that IP space, and any such security would be gone. Unless it is otherwise filtered, any customer with a default route can reach their routers. Nevertheless putting router interface ip address for your network in one specific block is very effective as way to quickly get rid of DoS attack on the router - you simply stop announcing that block but everything else on the network still works. And doing tricks like having primary ip address which not important at all (except for logging traffic actually destined to it) while secondary ip on the same interface is really the one used for inter-connection also works quite well. -- William Leibzon Elan Networks [EMAIL PROTECTED]
Re: icmp rpf
On Sep 25, 2006, at 12:22 PM, Mark Kent wrote: Jared Mauch wrote: I would hope they're doing it for more than just ICMP packets. yes, loose RPF, but I just care about ICMP. I would argue should be, or is a current best practice. OK, so I must have missed the memo :-) It's been all the rage. :) Who among AS1239, AS701, AS3356, AS7018, AS209 does loose RPF (not just strict RPF on single-homed customers)? I'm wondering why that is relevant. If all those ASNs only filtered on ASN instead of prefix for customer announcements (for instance), would that mean no one should? -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, 25 Sep 2006 09:22:34 -0400 Patrick W. Gilmore [EMAIL PROTECTED] wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? I do. -- Sheep are slow and tasty, and therefore must remain constantly alert. - Bruce Schneier, Beyond Fear
RE: New router feature - icmp error source-interface [was: icmp rpf]
Might this not be a bad idea if the router has interfaces on multiple, separate paths? Such a case may be where one customer or set of traffic routes over a link to ISP A, and other traffic over a link to ISP B, and not all related addresses are portable. In that case the loopback address for the ICMP errors might show from an address that seems to belong to a block from ISP A, but is really traffic that was transported on ISP B. Just trying to come up with possible issues, not saying that's a best practice or anything... -Scott -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 9:23 AM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Sep 25, 2006, at 5:26 PM, Berkman, Scott wrote: Might this not be a bad idea if the router has interfaces on multiple, separate paths? Such a case may be where one customer or set of traffic routes over a link to ISP A, and other traffic over a link to ISP B, and not all related addresses are portable. In that case the loopback address for the ICMP errors might show from an address that seems to belong to a block from ISP A, but is really traffic that was transported on ISP B. Just trying to come up with possible issues, not saying that's a best practice or anything... I doubt it is possible to make a rule / knob / idea / feature / whatever that cannot be misused, or that is applicable to all corner cases. That's why it's a knob. :) If it is applicable to your situation, you should use it. If not, not. But if the knob has enough use in enough situations, then it is probably something we want to push the vendors to implement. -- TTFN, patrick -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 9:23 AM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? You know I was just having this discussion with someone else a couple days ago. It turns out, much to my surprise, that the RFC actually calls for the ICMP error-message packet (as you said, the things that aren't ping etc which require a specific source-address) to originate from the OUTGOING interface used to return the ICMP message to the original sender. After much googling, I can't find any document where this has ever been officially updated either. The defacto industry standard on the other hand has been to use the primary address of the inbound interface, which serves exactly one function: it makes traceroute work. The hack that people use to simulate this functionality normally is: ip address the.fake.ip.here 255.255.255.252 ip address the.real.ip.here 255.255.255.252 secondary (FYI side note the Juniper equivilent is... confusing or non-working, hard to tell which. The tag primary is defined as the source address for local broadcast/multicast, preferred is defined as the source when you have multiple IPs within a subnet. Neither one should work for what we're talking about according to the docs, but if you actually try it... sometimes it works, sometimes it won't, and sometimes the behavior is different if you include only one but not the other :P). This works well for simple external-facing interfaces, things that speak BGP for example, but can confuse things like OSPF etc when you use secondaries on internal interfaces. FWIW I've been asking for more people to implement exactly what you're talking about for years (specifically setting ONLY the ICMP error source interface without risk of screwing up the interface in other ways). You can debate over exactly how to do it (global or per interface, icmp source-interface lo99 vs icmp source-ip 1.2.3.4, etc), but I agree wholeheatedly it should be done. It should be really simple too! (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) Please stop talking about networking on NANOG, you're confusing people. :) -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
RE: New router feature - icmp error source-interface [was: icmp rpf]
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 5:31 PM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: Re: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 5:26 PM, Berkman, Scott wrote: Might this not be a bad idea if the router has interfaces on multiple, separate paths? Such a case may be where one customer or set of traffic routes over a link to ISP A, and other traffic over a link to ISP B, and not all related addresses are portable. In that case the loopback address for the ICMP errors might show from an address that seems to belong to a block from ISP A, but is really traffic that was transported on ISP B. Just trying to come up with possible issues, not saying that's a best practice or anything... I doubt it is possible to make a rule / knob / idea / feature / whatever that cannot be misused, or that is applicable to all corner cases. That's why it's a knob. :) If it is applicable to your situation, you should use it. If not, not. But if the knob has enough use in enough situations, then it is probably something we want to push the vendors to implement. -- TTFN, patrick -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Monday, September 25, 2006 9:23 AM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: New router feature - icmp error source-interface [was: icmp rpf] On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? For instance, you could have a loopback99 which is in an announced block, but filtered at all your borders. Then set ip icmp error source-interface loopback99 or something. All error messages from a router would come from this address, regardless of the incoming or outgoing interface. Things like PMTUD would still work, and your / 30s could be in private space or non-announced space or even imaginary^Wv6 space. :) Note I said error messages, so things like TTL Expired, Port Unreachable, and Can't Fragment would come from here, but things like ICMP Echo Request / Reply pairs would not. Perhaps that should be considered as well, but it is not what I am suggesting here. Obviously there's lots of side effects, and probably unintended consequences I have not considered, but I think the good might out- weigh the bad. Or not. Which is why I'm offering it up for suggestion. (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) -- TTFN, patrick C and J both already have a similar feature, however I'm not sure whether or not they apply to ICMP. They both support PBR for locally originated packets - which, should include if the thought process is correct, ICMP. Perhaps someone with some time to waste can verify this in a lab. http://www.cisco.com/en/US/customer/products/sw/iosswrel/ps1828/products _configuration_guide_chapter09186a00800ca590.html#5406 -Dave
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Sep 25, 2006, at 5:40 PM, Richard A Steenbergen wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: On Sep 25, 2006, at 9:06 AM, Ian Mason wrote: ICMP packets will, by design, originate from the incoming interface used by the packet that triggers the ICMP packet. Thus giving an interface an address is implicitly giving that interface the ability to source packets with that address to potential anywhere in the Internet. If you don't legitimately announce address space then sourcing packets with addresses in that space is (one definition of) spoofing. Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? You know I was just having this discussion with someone else a couple days ago. It turns out, much to my surprise, that the RFC actually calls for the ICMP error-message packet (as you said, the things that aren't ping etc which require a specific source-address) to originate from the OUTGOING interface used to return the ICMP message to the original sender. After much googling, I can't find any document where this has ever been officially updated either. The defacto industry standard on the other hand has been to use the primary address of the inbound interface, which serves exactly one function: it makes traceroute work. I have not read the RFC in full, but after chatting with Daniel offline (see, some people actually do talk without posting! :), I believe this only applies to packets addressed to the router. Since packets going -through- the router have absolutely no guarantee what source will be used coming back, I don't seen an issue here. Just change the idea such that it only is used for error messages to packets where the dest addy is not an interface on the router. Also, this makes traceroute -easier- to use. Suddenly all interfaces on the same router have the same IP address, thereby making it easy to tell if two traceroutes intersect, even if they use different interfaces. Oh, and who said RFCs can't be updated? :-) (Unless, of course, I get 726384 you are off-topic replies, in which case I withdraw the suggestion.) Please stop talking about networking on NANOG, you're confusing people. :) I knew someone would flame me for it. :) -- TTFN, patrick
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 04:33:18PM -0700, David Temkin wrote: C and J both already have a similar feature, however I'm not sure whether or not they apply to ICMP. They both support PBR for locally originated packets - which, should include if the thought process is correct, ICMP. Perhaps someone with some time to waste can verify this in a lab. http://www.cisco.com/en/US/customer/products/sw/iosswrel/ps1828/products _configuration_guide_chapter09186a00800ca590.html#5406 The actual path taken for the ICMP generated by the router does not matter, we're just talking about the source address selected by the router. The only reasons that the source address (which reveals a real IP address on a router) matters at all for ICMP error responses are: * So traceroute works (current industry standard behavior is to use the ingress interface IP so you see the forward path in traceroute, not the reverse path, which may be asymmetric. * So your replies don't get thwacked by people doing uRPF strict (i.e. they must come from announced IPs or people doing strict strict with no exception filtering capabilities will block the traceroute responses). * Optionally, allowing naive tools like MTR to ping the IP they discover via traceroute, lest weenies flood your noc with I'm pinging 10lolz! emails. Revealing your interface IPs carries all kinds of DoS/security risks with it, since there are a great many routers out there without good control plane policing functionality (and even some of those that have it, don't really have it :P). Since there is really no legitimate need for people from the outside world to ever communicate with your real interface IPs at all (with the exception of some rate limited ICMP echo/reply due to aforementioned mtr weenies), having the option to hide those real addresses completely in ICMP source address selection is a very good thing for enhancing network security. As I said you can accomplish part of this hack with primary/secondary IPs on interfaces. You can also accomplish some level of filtering by numbering your interfaces out of common blocks which are filtered at your various borders/edges. It's still a pain in the !(*#*, especially if you number your links out of any regional blocks to cut down on asymmetric routing confusion, or have any number of peers who provide /30s from their own IP space. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: New router feature - icmp error source-interface [was: icmp rpf]
At 9:22 AM -0400 9/25/06, Patrick W. Gilmore wrote: Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? It certainly would beat the alternative of no response at all, but one would hope it wouldn't become common practice since it reduces the information returned (e.g. during a traceroute, you'd lose the sometimes useful information from in-addr about what particular interface was involved). /John
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 08:45:49PM -0400, John Curran wrote: At 9:22 AM -0400 9/25/06, Patrick W. Gilmore wrote: Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? It certainly would beat the alternative of no response at all, but one would hope it wouldn't become common practice since it reduces the information returned (e.g. during a traceroute, you'd lose the sometimes useful information from in-addr about what particular interface was involved). Personally I'd hope that if it was implemented, it would support mapping on a per-interface basis (especially for NSP use). That should in theory lead to even more accurate information, since each network would be capable of easily renumbering without impact, and managing their own DNS for every interface. Currently a great many PTRs are out of date because IP blocks supplied by peers, exchange points, or transit providers, are too much of a pain to keep updated when interfaces move etc. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that told me into which router by which interface I was going. When the owner of the router might not even have known. Or I have had attempts to do this foiled by routers that used an internal loopback IP address. On the whole, then, I guess I would vote, no. -- Joe Yao --- This message is not an official statement of OSIS Center policies.
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Mon, 25 Sep 2006, Joseph S D Yao wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix I think in the original poster's scenario one network was looking to protect their resources/equipment from a majority of the network's ills. It's not unreasonable... atleast not in my mind. It's also not 'security through obscurity' since one of the parties is/was leaking their information OUT, just not 'in' :) something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that What's interesting is that today, in many networks, the usefulness of traceeroute has bee degraded by other non-ip issues (coughmpls/cough) not in ALL cases, but certainly in many you are not seeing quite what you'd expect from the traceroute :(
Re: New router feature - icmp error source-interface [was: icmp rpf]
Chris, So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) http://momo.lcs.mit.edu/traceroute/index.php The best (?) of both worls, but I digress... - ferg -- Chris L. Morrow [EMAIL PROTECTED] wrote: [snip] What's interesting is that today, in many networks, the usefulness of traceeroute has bee degraded by other non-ip issues (coughmpls/cough) not in ALL cases, but certainly in many you are not seeing quite what you'd expect from the traceroute :( -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Tue, 26 Sep 2006, Fergie wrote: Chris, So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) :) depends on the network I guess... I'm not sure it's going to tell you anything about hops hidden by mpls lsp's that don't decrement ttl though...
Re: New router feature - icmp error source-interface [was: icmp rpf]
Ah, but there's the rub... ISPs who are discreet in how they wish their infrastructure to be viewed will continue to engineer methods in which portions are not visible to the public at-large. Somehow, I don't think that will ever go away, so trying to tilt at windmils w.r.t. (paraphrased) ...what interface an ICMP foo is sourced from... is, indeed... tiling at windmills, methinks. :-) Cheers! - ferg -- Chris L. Morrow [EMAIL PROTECTED] wrote: On Tue, 26 Sep 2006, Fergie wrote: Chris, So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) :) depends on the network I guess... I'm not sure it's going to tell you anything about hops hidden by mpls lsp's that don't decrement ttl though... -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
Re: New router feature - icmp error source-interface [was: icmp rpf]
At 10:29 PM 9/25/2006, Chris L. Morrow wrote: On Mon, 25 Sep 2006, Joseph S D Yao wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix I think in the original poster's scenario one network was looking to protect their resources/equipment from a majority of the network's ills. It's not unreasonable... atleast not in my mind. It's also not 'security through obscurity' since one of the parties is/was leaking their information OUT, just not 'in' :) The issue is that the world has changed. Some networks actually expect not only the use of public addresses on router interfaces, but addresses from advertised blocks. Why has the world changed? Because not everyone is friendly on the Internet. There were issues raised in Mobile IP by the drafts that predated the publication of RFC 2267. Why? Well, Mobile IP WG had made the assumption that all IPv4 packet forwarding would be done, without filtering, based solely on the destination IP address. The down side was it made some of the traffic look spoofed. The world changed, people started abusing the Internet in new ways, and the old assumptions no longer held. Mobile IP WG adapted to the new reality that was coming. The longevity of the TCP/IP protocol suite is attributable to the continued effort of many people, not to savant qualities of those who first wrote specifications. something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that What's interesting is that today, in many networks, the usefulness of traceeroute has bee degraded by other non-ip issues (coughmpls/cough) not in ALL cases, but certainly in many you are not seeing quite what you'd expect from the traceroute :( There's no more degradation of usefulness from MPLS than there was from ATM or Frame Relay. Pick your poison for virtualized link layer, and you'll have some degree of difficulty determining and debugging the underlying network without tools to peer under the hood.
Re: New router feature - icmp error source-interface [was: icmp rpf]
On Tue, Sep 26, 2006 at 02:51:21AM +, Fergie wrote: So, I'm wondering: What happens when you have a traceroute tool that shows you MPLS-lableled hops, too? :-) http://momo.lcs.mit.edu/traceroute/index.php The best (?) of both worls, but I digress... That doesn't show any more or less data about the path, just some extra info about the label that is effectively useless to end users. If TTL decrement is not enabled, all of the IP hops are hidden by the tunnel, which is the point Chris was making. But that said, I personally think Cisco MPLS with TTL decrement enabled but returning the the same rtt as the penultimate hop for every IP hop inside the LSP has caused far more harm to every NOC ticket queue on the planet than just hiding the damn things. While we're asking for silly features, I can name a LOT of people who would pay good money for a dedicated ICMP generating processor on Cisco that doesn't spike every time BGP scanner runs. Silencing end users who have figured out how to work traceroute (or worse MTR) is worth its weight in gold. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: New router feature - icmp error source-interface [was: icmp rpf]
Joseph S D Yao wrote: On Mon, Sep 25, 2006 at 09:22:34AM -0400, Patrick W. Gilmore wrote: ... Who thinks it would be a good idea to have a knob such that ICMP error messages are always source from a certain IP address on a router? ... I've sometimes thought it would be useful when I wanted to hide a route. But security via obscurity just makes it that much harder to fix something. Many more times than this would have been useful, I've been able to identify at which router a problem was by a 'traceroute' that told me into which router by which interface I was going. When the owner of the router might not even have known. Or I have had attempts to do this foiled by routers that used an internal loopback IP address. On the whole, then, I guess I would vote, no. Why not just do a show ip route? since you can actually verify the information against your routing table. This way you can see when the route was learned, where was it learned from and how long ago it was last updated... the problem is that too many people engineers rely on traceroute... sure traceroute is a wonderful tool, however it is meant to assist you in tracking down the problem. I've seen far too many you are filtering, investigate please when all that has been done is implementing acls and rate limiting. IMO, If you want to implement a non-routable ip space to protect your backbone... go for it if you want to icmp rate limit *i know level3 does this out of both nyc and la* which causes mass threads of we are getting packet loss, please investigate go for it .. if your network engineers are not equipped with the information to how to fully diagnose a network/problem you should think about new hires. Cheers, Payam
Re: icmp rpf
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mark Kent wrote: A smaller North American network provider, with a modest North American backbone, numbers their internal routers on public IP space that they do not announce to the world. One of the largest North American network providers filters/drops ICMP messages so that they only pass those with a source IP address that appears in their routing table. As a result, traceroutes from big.net into small.net have numerous hops that time out. Traceroutes from elsewhere that go into small.net but return on big.net also have numerous hops that time out. We do all still think that traceroute is important, don't we? If so, which of these two nets is unreasonable in their actions/policies? Please note that we're not talking about RFC1918 space, or reserved IP space of any kind. Also, think about the scenario where some failure happens leaving big.net with an incomplete routing table, thus breaking traceroute when it is perhaps most needed. Thanks, -mark - -- This is yet another reason one shouldn't rely on pings traceroutes to perform reachability analysis. regards, /virendra -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFFxP+pbZvCIJx1bcRAnN8AJ0VqiwhNkxUm5MxG8p/hLptiJ1IdQCg7wIB nx2woHkYDzu1+7MBdnOZaEw= =mlPK -END PGP SIGNATURE-
Re: icmp rpf
virendra rode wrote: This is yet another reason one shouldn't rely on pings traceroutes to perform reachability analysis. So, you're in the traceroute is not important camp? (you'll note that in my email I did ask whether we think traceroute is important) Mark Smith wrote: The non-announcers, because they're also breaking PMTUD. Really? How? Remember, we're not talking about RFC1918 space, where there is a BCP that says we should filter it at the edge. We're talking about public IP space, that just doesn't happen to be announced outside of a particular AS. Thanks, -mark
Re: icmp rpf
On Sep 24, 2006, at 4:33 PM, Mark Kent wrote: Remember, we're not talking about RFC1918 space, where there is a BCP that says we should filter it at the edge. We're talking about public IP space, that just doesn't happen to be announced outside of a particular AS. If the intent is to prevent folks from reaching out and touching random network infrastructure devices directly whilst still allowing traceroute to work, iACLs and/or using IS-IS as one's IGP and null- routing the infrastructure blocks at one's various edges achieves the same effect with less potential for breakage: http://www.nanog.org/mtg-0405/mcdowell.html Note that a good infrastructure addressing plan is a prerequisite for both of these methods. --- Roland Dobbins [EMAIL PROTECTED] // 408.527.6376 voice Any information security mechanism, process, or procedure which can be consistently defeated by the successful application of a single class of attacks must be considered fatally flawed. -- The Lucy Van Pelt Principle of Secure Systems Design
Re: icmp rpf
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mark Kent wrote: virendra rode wrote: This is yet another reason one shouldn't rely on pings traceroutes to perform reachability analysis. So, you're in the traceroute is not important camp? (you'll note that in my email I did ask whether we think traceroute is important) - I'm sure its important. All I'm saying is, icmp can get rate-limited (many times it does) which could possibly lead to packet loss and even drops while traversing hops. regards, /virendra Mark Smith wrote: The non-announcers, because they're also breaking PMTUD. Really? How? Remember, we're not talking about RFC1918 space, where there is a BCP that says we should filter it at the edge. We're talking about public IP space, that just doesn't happen to be announced outside of a particular AS. Thanks, -mark -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFFyejpbZvCIJx1bcRAsFXAKDokAbujtIiuvGDXss2Tt5U3CXElQCgkpKG UaS6MDxtWKjdbiLewujDs/Q= =qgo2 -END PGP SIGNATURE-
Re: icmp rpf
[Can we all have a moment of silence for a useful, interesting, and on-topic post?] On Sep 24, 2006, at 5:59 PM, Mark Kent wrote: A smaller North American network provider, with a modest North American backbone, numbers their internal routers on public IP space that they do not announce to the world. One of the largest North American network providers filters/drops ICMP messages so that they only pass those with a source IP address that appears in their routing table. As a result, traceroutes from big.net into small.net have numerous hops that time out. Traceroutes from elsewhere that go into small.net but return on big.net also have numerous hops that time out. We do all still think that traceroute is important, don't we? If so, which of these two nets is unreasonable in their actions/ policies? Who said either was? First: Your network, your rules. Don't expect others to play by your rules. But more importantly, there is nothing that says two perfectly reasonable, rational rules cannot create a problem when intersecting in interesting ways. But if forced, I'd say Small.Net gets my vote for needing correction. I see less wrongness in a networking running what is essentially loose RPF than a network who expects supposedly bogon- sourced packets to be forwarded. (One could argue that non-announced space is bogus.) Just remember, I would only say that if pushed. Normally I would say neither is wrong. Please note that we're not talking about RFC1918 space, or reserved IP space of any kind. Also, think about the scenario where some failure happens leaving big.net with an incomplete routing table, thus breaking traceroute when it is perhaps most needed. In such an instance, I would suggest Big.Net will have far, far larger problems than whether pings get returned from prefixes it can't reach anyway. -- TTFN, patrick