On Tue, 2 Oct 2007, Brian Dickson wrote: > Dean Anderson wrote: > > I think this may be of interest. It was offlist, so I won't identify > > the author I am responding to. > > > [Did you think to perhaps ask the author first? He/she may have been > willing to be identified...]
The author is not on DNSOP. > >>> I. Harm only possible for ENDSO; Update RFC 2671 Instead The > >>> maximum non-EDNS amplification factor is 8 > >>> > >> 8x can be significant. > >> > > Yes. But they can get more than that from a couple hundred+ root > > servers. > > > You miss the point. Your original comment was 100% wrong. You even > admit so, but don't seem to realize it. "Not possible" you wrote. I didn't say 'not possible'. I said 8x. > "Can be" he/she replied. They didn't say 'can be' they said 8x can be significant. > "Yes, but [...]" you wrote. I said they can easily get more from root servers. The low hanging fruit is in the root servers. (authority servers) > The fact that something else is a *bigger* risk, doesn't have any > bearing on whether the first thing is a risk. Yes, it really does. Especially if the bad guy doesn't have to even change his source code to get more bang for his botnet, and doesn't incur any more risk. In fact, using authority servers is _less_ risk to the abuser, because to compose the reflector attacks, s/he has to crack into a server, craft a record, and search 3.7 million IP addresses for a list of reflectors. All of these things leave a forensic trail. Any one of which might lead back to the bad guy. By contrast, searching authority servers for, say, large SPF records isn't suspicious. There is no trail besides the botnet, which is necessary for either case. Authority servers give more bang and less risk, and same code and payload size. One must presume the attacker a complete idiot to make the open reflectors attack look attractive. While it may be the case that some attackers are indeed complete technical idiots, such idiots only seem to know 'ping -f and other rootkit naughties. > Consider: Banks have *way* more money than convenience stores. (That > fact is even common knowledge.) However, it doesn't seem to have much > impact on their respective robbery rates. Banks have correspondingly higher security, and robbing a bank is a federal crime, and engages the FBI and the Secret Service. Robbing a convenience store is a matter for the local cops. You example doesn't fly. > >> Furthermore, the real danger here is the ability to mount a > >> _distributed_ attack, in which large numbers of servers send bogus > >> responses at a rate far beyond that which the original authoritative > >> server > >> could manage. That requires caching servers; a handful of authoritative > >> servers mostly on the same network won't cut it. > >> > > > > Caching servers are not a requirement for a distributed attack. To > > conduct the attack with any servers (caching or authority), one still > > needs a botnet to send the spoofed source packets. This botnet is > > amplified by the same factor, whatever type of server is used. Thus, > > the type of server is irrelevant to the damage caused. > > > In any DDOS attack, the multiplier effect is not unconstrained. > If the devices being used *as* the multipliers by any of the > participants, are shared among participants, > then the aggregate result can, and likely will, be limited by those devices. > A botnet of 100,000 using a set of 20 servers each on a 1Gbps link, can > generate at most 20 * 1Gbps of traffic, > no matter how massive the aggregate rate of the botnet is. > > At some point, there is even a crossover, where the botnet could do more > damage than the so-called multiplier. > If the botnet of 100,000 could each send 33 kbps, i.e dial-up speeds, > that would be 33 * 100,000 kbps, or 33 Gbps. > > More than that set of 20 servers could generate at line rate. For some inexplicable reason, you assume there are only 20 authority servers in the internet. At great effort, a DNS researcher has compiled a list of about 20000 open recursors by brute force search of 3.7 billion IP addressses. Overnight, CAIDA collects about a million or so authority servers by merely walking in-addr.arpa for NS records. > >> For example, if an anycast nameserver address is backed by three > >> servers and one of them is down, but the routing is such that only 10% > >> of requests go to the down server, then on average, clients will see > >> 10% of their requests dropped, not one third. > >> > > > > This is overly simplistic. It is not true for stateful DNS packets, > > because packets can always be routed to multiple anycast instances. > > Earlier, I had thought that this could only happen with PPLB, and in > > theory (RFC 1812 could happen at anytime. I've since tested this > > experimentally. Theory is was right. Even routers that route using flow > > caches expire those cache entries every 60 seconds. Send 2 packets more > > than 60 seconds apart, and they can go to different servers. I've > > detected anycast open recursors this way. > > > The vast majority of equipment used on high-capacity links, is neither > PPLB, nor cache flow switched. It is pre-computed FIB in hardware > (ASIC) switched, via TCAM. Those entries do not expire. They *do* > periodically change when route selection to destinations change, > something that happens often enough on the aggregate - but not often > on individual prefixes. I'm not sure of this. The last time an operator told me something about Cisco 12000 architecture and implementation, I later found out it was wrong. But it appears the behavior of the core is of no consequence. Closer to the edges, where there are consequences, routers use flow caches. I think that's why I can detect anycast open recursors. But the important fact is: I CAN detect anycast open recursors by sending packets at 60+ second intervals, and don't as easily detect them at 30 second intervals. > The PPLB and cache-aging occurs near the edge, on smaller networks. > There are a very limited number of locations where this is even > theoretically visible, and in most cases, solution to this is under > operator control (e.g. deploy software that does CEF/DCEF instead of > cached switching, and possibly upgrade obsolete hardware that is > unable to do CEF which is now end-of-life.) This doesn't matter. If the PPLB or flow cache is injecting packets into different parts of the core with different exits, it doesn't matter whether the core does load balancing or not. The key feature is the fact that packets could exit though more than one router. Unless you are single homed, you have this 'problem'. Its not a 'problem' of the router. All routers follow RFC1812 in theory and in implementation and IPv4 does not have special Anycast host routes. That is a feature of IPv6. > > But to tell how bad the anycast problem is on an authority server (such > > as the root or TLD servers), one needs to identify uniquely (with NSID), > > the instance each query goes to, and measure how often one gets a > > different server. > > > No, one does not need to identify specific servers. It is sufficient > to *disambiguate* servers, to separate out loci in the vernacular. > Once disambiguated, what their identity is, is not of primary > importance. Well, you can't do that either with NSID encryption. The nonce varies over the same server from request to request. (I actually agreed on DNSEXT during NSID discussion to allow hiding of version or whatnot, so long as the nonce was fixed over a long period of time---the root/tld folks still wouldn't agree, and pushed their scheme through) > And, it is not difficult to disambiguate anycast instances, given a > sufficient number of places from which to observe. Traceroutes, ttl > values on dns responses, round trip times, all can be used to help > isolate instances from each other. That is indirect information. Direct information is much better. > It would be nearly impossible to distinguish servers *within* an > anycast instance, e.g. behind any kind of load balancer. One wonders if that is actually 'anycast'. The load balancer is really just a special kind of stateful NAT. > Without anycast, load balancers would in theory also contribute to the > occasional problem of stale cache removal affecting stateful DNS > traffic. Not just in theory, but in practice. Try doing DNS queries > from very close by a given authority server. Do so over tcp, with a > connection that lasts more than, for example, 300 seconds. See if what > you observe looks the same as your alleged "anycast" bogeyman. Stateful NATs and load balancers keep TCP state for an hour+. Otherwise your SSH sessions would drop. I haven't argued against using load balancers, or firewalls. > [ I invite you to put your reputation on the line. Build a tool that > does the above, and sends mail as you, to dnsop. > Run it blindly, i.e. without advance knowledge of the results. View > the results, the same as us, on the list. > I double dog dare you.] I have built a tool. I have run it. And I have detected anycast open recursors. > And what exactly *is* "the anycast problem"? > > You refer to "the anycast problem" here, as if it is common knowledge. > I am not familiar with the actual existence, let along details of, an > anycast problem. Yet you refer to it as *the* anycast problem. I do have to update. It hasn't been updated in a while. http://www.av8.net/IETF-watch/DNSRootAnycast/History.html For most people on this list, this is common knowledge. > > Incorrect, as shown above. Besides, we expect high availability > > from from root servers. Anycast appears to give no better than > > about 97% over TCP under ideal conditions (from a paper on Nanog by > > Anycast HTTP advocate). It might not be that good, even. A figure > > of 90% would be abysmally bad. 3% packet loss is usually > > unservicable > The 90% in the original authors example was meant to illustrate the > measurable impact on reliability, of a specific failure rate of a > component, to show that there isn't always a linear relationship. Yes, I know that 90% was a example. But the 97% was a statistic from a real (optimistic) paper on HTTP anycast presented by a proponent on Nanog. 3% loss is unacceptable performance for root and tld nameservers. --Dean -- Av8 Internet Prepared to pay a premium for better service? www.av8.net faster, more reliable, better service 617 344 9000 _______________________________________________ DNSOP mailing list [email protected] https://www1.ietf.org/mailman/listinfo/dnsop
