Re: Great Suggestion for the DNS problem...?
On Thu, 28 Aug 2008, Brian Dickson wrote: However, if *AS-path* filtering is done based on IRR data, specifically on the as-sets of customers and customers' customers etc., then the attack *can* be prevented. Yes, but I can't do this for everybody else. Doing AS-path and prefix filtering (matching that a certain prefix can only be announced by a certain AS) doesn't scale in IOS for instance. We do prefix filtering for OUR customers, but there is no feasable way for me to do this for everybody else. I think this needs to be fixed, but it involves something new that isn't present today, and I think it needs to involve vendors because they need to produce new code to handle it. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: Great Suggestion for the DNS problem...?
Alex P wrote: *) There is no currently deployable solution to this problem yet. *) Filtering your customers using IRR is a requirement, however, it is not a solution - in fact, in the demonstration, we registered the /24 prefix we hijacked in IRR. RIRs need to integrate the allocation data with their IRR data. -alex [your former moderator] Kind of true. When doing *prefix* filtering, this kind of hijack is not preventable. However, if *AS-path* filtering is done based on IRR data, specifically on the as-sets of customers and customers' customers etc., then the attack *can* be prevented. The as-path prepending depends on upstreams and their peers accepting the prefix with a path which differs from the expected path (if the upstreams register their as-sets in the IRR). If the as-path filter only allows generally-feasible as-paths from customers, where the permitted variations are just N copies of ASfoo (where foo is a member of an as-set), then adding a fake ASbar in the as-path will cause the prefix to be rejected. If you follow the diagram from the presentation, information about the *real* path to the victim, from the perspective of the hijacker, requires that the AS's on that path not see the hijacked prefix as announced by the hijackers. This means that if the AS's traversed are: as-hijacker, as-bar, as-victim, then as-bar must *not* see the hijacked victim, for the attack to work. By adding as-bar into the as-path of the hijacked prefix, the loop-prevention logic of BGP makes sure as-bar can't see the hijacked prefix. So, if the upstreams of as-hijacker reject all prefixes with an as-path which includes as-bar (because as-bar is not a member of any customer's as-set expansion), the attack fails. I hope I haven't managed to confuse folks too much. But, the short answer is: If you use the IRR, the full value is best realized by adding *as-path* filters to the things you build from the IRR data, and applying them to your customers (and peers !!). Oh, and if you already do IRR stuff, it's really quite easy to do. Brian Dickson
Re: Great Suggestion for the DNS problem...?
On Thu, 28 Aug 2008, Brian Dickson wrote: However, if *AS-path* filtering is done based on IRR data, specifically on the as-sets of customers and customers' customers etc., then the attack *can* be prevented. The as-path prepending depends on upstreams and their peers accepting the prefix with a path which differs from the expected path (if the upstreams register their as-sets in the IRR). You are thinking about this specific exploit - which may in fact be stopped by as-path-filtering. However, that's not the problem you are solving. Problem is the hijacking. There are many other ways to reinject traffic closer to victim - will require attacker to work a little harder, but not really fix the problem. (Think, GRE tunnels, no-export, no-export-to-specific-peer, etc). snipped So, if the upstreams of as-hijacker reject all prefixes with an as-path which includes as-bar (because as-bar is not a member of any customer's as-set expansion), the attack fails. What's to stop me from adding as-bar into my as-set? To do what you are describing, you will have to enforce export AS-LEFT and import AS-RIGHT rules on every pair of AS-PATH adjacencies. And I'm not sure if existing tools can do that - or how many existing adjacencies fail that test.
Re: Great Suggestion for the DNS problem...?
Alex Pilosov wrote: On Thu, 28 Aug 2008, Brian Dickson wrote: However, if *AS-path* filtering is done based on IRR data, specifically on the as-sets of customers and customers' customers etc., then the attack *can* be prevented. The as-path prepending depends on upstreams and their peers accepting the prefix with a path which differs from the expected path (if the upstreams register their as-sets in the IRR). You are thinking about this specific exploit - which may in fact be stopped by as-path-filtering. However, that's not the problem you are solving. Problem is the hijacking. There are many other ways to reinject traffic closer to victim - will require attacker to work a little harder, but not really fix the problem. (Think, GRE tunnels, no-export, no-export-to-specific-peer, etc). snipped Very true. Tying allocations and assignments to ASN-of-origin and providing a suitable place to validate such, for building prefix and as-path filters, would do a lot towards further limiting the ability to hijack prefixes - but only to the degree to which providers filter their customers. And that's only on routes - to say nothing of packet filtering (BCP 38)... So, if the upstreams of as-hijacker reject all prefixes with an as-path which includes as-bar (because as-bar is not a member of any customer's as-set expansion), the attack fails. What's to stop me from adding as-bar into my as-set? To do what you are describing, you will have to enforce export AS-LEFT and import AS-RIGHT rules on every pair of AS-PATH adjacencies. And I'm not sure if existing tools can do that - or how many existing adjacencies fail that test. True, there is nothing actually stopping you from doing this. However, (and this is both the key, and a big presumption) changes to as-sets are the kind of thing that automatic provisioning tools should alert on, and require confirmation of, before updating prefix-lists and as-path lists based on the new members of the as-set. While prefixes are routinely added, new transit relationships at a BGP level don't happen all *that* often. The idea isn't just to stop the prefix announcement, it is to trap activity that would otherwise permit the prefix hijack, at the time it occurs and before it takes effect. The closer this occurs to the hijacker, the more likely it is that appropriate responses can be directed at the bad guy, and with a greater likelihood of success (e.g. prosecution). The threat alone of such response may act as a suitable deterrent... As for the filter building and enforcement mechanisms: The inbound as-path filters need to permit the permutations of paths that result from expanding as-sets, but that can be accomplished unilaterally on ingress, without enforcement beyond the immediate peering session. The expansion is for each as-set behind an ASN, there should be a corresponding as-set, and so on... each gets expanded and added to the expansion of the permitted paths via that ASN. Lather, rinse, repeat. All filtering is local, although the more places filtering happens, the better. Brian
Re: Great Suggestion for the DNS problem...?
however, since it is off-topic for nanog ha ha. please stop telling people that they are off topic for nanog. randy
Re: Great Suggestion for the DNS problem...?
* Paul Vixie: Listen on 200 random fake ports (in addition to the true query ports); at first glance, this is brilliant, though with some unimportant nits. It doesn't work OOTB for most users because the spoofed packets never reach the name server process if you don't use the ports to send packets to the authoritative server which is spoofed--the wonders of stateful firewalling.
Re: Great Suggestion for the DNS problem...?
Colin Alston wrote: Why does it use UDP? :P Faster? Smaller? Less code to break? No perceived need for state? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actioInfallibility, and the ability to learn from their mistakes. Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs
Re: Great Suggestion for the DNS problem...?
On Tue, 29 Jul 2008 15:56:19 +0200 Colin Alston [EMAIL PROTECTED] wrote: DNS uses UDP. Ahh yes of course.. Why does it use UDP? :P In this situation, UDP uses one query packet and one reply. TCP uses 3 to set up the connection, a query, a reply, and three to tear down the connection. *Plus* the name server will have to keep state for every client, plus TIMEWAIT state, etc. (Exercise left to TCP geek readers: how few packets can you do this in? For example -- send the query with the SYN+ACK, send client FIN with the query, send server FIN with the answer? Bonus points for not leaving the server's side in TIMEWAIT. Exercise for implementers: how sane can your stack be if you're going to support that?) --Steve Bellovin, http://www.cs.columbia.edu/~smb
Re: Great Suggestion for the DNS problem...?
On Tue, 29 Jul 2008, Steven M. Bellovin wrote: On Tue, 29 Jul 2008 15:56:19 +0200 Colin Alston [EMAIL PROTECTED] wrote: DNS uses UDP. Ahh yes of course.. Why does it use UDP? :P In this situation, UDP uses one query packet and one reply. TCP uses 3 to set up the connection, a query, a reply, and three to tear down the connection. *Plus* the name server will have to keep state for every client, plus TIMEWAIT state, etc. (Exercise left to TCP geek readers: how few packets can you do this in? For example -- send the query with the SYN+ACK, send client FIN with the query, send server FIN with the answer? Bonus points for not leaving the server's side in TIMEWAIT. Exercise for implementers: how sane can your stack be if you're going to support that?) It was advocated as T/TCP in 90s. http://www.kohala.com/start/ttcp.html Not accepted widely: http://en.wikipedia.org/wiki/T/TCP Regads, Janos Mohacsi
Re: Great Suggestion for the DNS problem...?
On Tue, 29 Jul 2008, Steven M. Bellovin wrote: In this situation, UDP uses one query packet and one reply. TCP uses 3 to set up the connection, a query, a reply, and three to tear down the connection. *Plus* the name server will have to keep state for every client, plus TIMEWAIT state, etc. (Exercise left to TCP geek readers: how few packets can you do this in? For example -- send the query with the SYN+ACK, send client FIN with the query, send server FIN with the answer? Bonus points for not leaving the server's side in TIMEWAIT. Exercise for implementers: how sane can your stack be if you're going to support that?) The bittorrent tracker guys seem to run into problems at around 30kk tracker requests per second (TCP), and they say it's mostly setup/teardown (sy usage in vmstat), the tracker hash lookup doesn't take that much. They're trying to move to UDP, currently their workload is approx 5% UDP. I guess TCP DNS workload would be similar in characteristics. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: Great Suggestion for the DNS problem...?
We mainly use UDP for tracker announces, and only use TCP when we have to, and can confirm that the server spends far more time on the TCP setup/teardown than on computing the tracker response. - LP On Jul 29, 2008, at 12:21 PM, Mikael Abrahamsson wrote: On Tue, 29 Jul 2008, Steven M. Bellovin wrote: In this situation, UDP uses one query packet and one reply. TCP uses 3 to set up the connection, a query, a reply, and three to tear down the connection. *Plus* the name server will have to keep state for every client, plus TIMEWAIT state, etc. (Exercise left to TCP geek readers: how few packets can you do this in? For example -- send the query with the SYN+ACK, send client FIN with the query, send server FIN with the answer? Bonus points for not leaving the server's side in TIMEWAIT. Exercise for implementers: how sane can your stack be if you're going to support that?) The bittorrent tracker guys seem to run into problems at around 30kk tracker requests per second (TCP), and they say it's mostly setup/ teardown (sy usage in vmstat), the tracker hash lookup doesn't take that much. They're trying to move to UDP, currently their workload is approx 5% UDP. I guess TCP DNS workload would be similar in characteristics. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Great Suggestion for the DNS problem...?
[ unthreaded to encourage discussion ] On Sat, Jul 26, 2008 at 04:55:23PM -0500, James Hess wrote: Nameservers could incorporate poison detection... Listen on 200 random fake ports (in addition to the true query ports); if a response ever arrives at a fake port, then it must be an attack, read the identified attack packet, log the attack event, mark the RRs mentioned in the packet as poison being attempted for 6 hours; for such domains always request and collect _two_ good responses (instead of one), with a 60 second timeout, before caching a lookup. The attacker must now guess nearly 64-bits in a short amount of time, to be successful. Once a good lookup is received, discard the normal TTL and hold the good answer cached and immutable, for 6 hours (_then_ start decreasing the TTL normally). Is there any reason which I'm too far down the food chain to see why that's not a fantastic idea? Or at least, something inspired by it? Cheers, -- jr 'IANAIE' a -- Jay R. Ashworth Baylink [EMAIL PROTECTED] Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Those who cast the vote decide nothing. Those who count the vote decide everything. -- (Josef Stalin)
Re: Great Suggestion for the DNS problem...?
On 2008/07/28 09:05 PM Jay R. Ashworth wrote: Is there any reason which I'm too far down the food chain to see why that's not a fantastic idea? Or at least, something inspired by it? If NS records pointed to IP's instead of names then this problem might not exist. The root holds glue going up the chain, and you could reject authoritative responses from IP's not listed as authoritative NS for that zone. Ie for karnaugh.za.net, net is looked up from root. Root IP addresses are queried directly, so you know to ignore responses coming from someone else. That gives you net (the same gtld, how convenient) and authoritative IP response for its NS. So you look up za.net and get correct glue and so on. Actually, if glue were always served up the resolution chain then then only crummy glueless delegations would be vulnerable. Anyone feel like redesigning the DNS protocol? Anyone? No? :(
Re: Great Suggestion for the DNS problem...?
[ unthreaded to encourage discussion ] On Sat, Jul 26, 2008 at 04:55:23PM -0500, James Hess wrote: Nameservers could incorporate poison detection... Listen on 200 random fake ports (in addition to the true query ports); if a response ever arrives at a fake port, then it must be an attack, read the identified attack packet, log the attack event, mark the RRs mentioned in the packet as poison being attempted for 6 hours; for such domains always request and collect _two_ good responses (instead of one), with a 60 second timeout, before caching a lookup. The attacker must now guess nearly 64-bits in a short amount of time, to be successful. Once a good lookup is received, discard the normal TTL and hold the good answer cached and immutable, for 6 hours (_then_ start decreasing the TTL normally). Is there any reason which I'm too far down the food chain to see why that's not a fantastic idea? Or at least, something inspired by it? There's a ton of stuff that you can do, I talked a bit about this kind of solution several days ago, see [EMAIL PROTECTED]. The problem is mainly that this is reactive, and primarily applicable to this attack because it's a brute-force. The next attack might be more elegant. Designing in this sort of protection is good AND bad, because on one hand, you do mostly solve the problem, and that's good, but you also encourage people to think of the problem as fixed or my server is not vulnerable, when the only real way to protect against the *next* attack is to make sure that the data is valid, so that's DNSSEC. There are actually more specifically useful things that you can do to mitigate particular aspects of this attack, except that talking about them will also point to some risks that I don't believe have been made public, and I'm going to do my part to keep it that way, at least for a bit longer. The short form, though, is that if you sit there and try to manufacture artificial protection against each new attack as it develops, you will end up with this Rube Goldberg contraption to protect your nameserver from various attacks, and who knows what will break it. View these as very short-term fixes, rather than a correction of the underlying issue. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
RE: Great Suggestion for the DNS problem...?
As you pointed out, the protocol, if properly implemented, addresses this. There should always be Glue (A records for the NS) in a delegation. RFC 1034 even specifies this: 4.2.2 snip As the last installation step, the delegation NS RRs and glue RRs necessary to make the delegation effective should be added to the parent zone. The administrators of both zones should insure that the NS and glue RRs which mark both sides of the cut are consistent and remain so. /snip -Original Message- From: Colin Alston [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 12:20 PM To: Jay R. Ashworth Cc: nanog@nanog.org Subject: Re: Great Suggestion for the DNS problem...? On 2008/07/28 09:05 PM Jay R. Ashworth wrote: Is there any reason which I'm too far down the food chain to see why that's not a fantastic idea? Or at least, something inspired by it? If NS records pointed to IP's instead of names then this problem might not exist. The root holds glue going up the chain, and you could reject authoritative responses from IP's not listed as authoritative NS for that zone. Ie for karnaugh.za.net, net is looked up from root. Root IP addresses are queried directly, so you know to ignore responses coming from someone else. That gives you net (the same gtld, how convenient) and authoritative IP response for its NS. So you look up za.net and get correct glue and so on. Actually, if glue were always served up the resolution chain then then only crummy glueless delegations would be vulnerable. Anyone feel like redesigning the DNS protocol? Anyone? No? :(
Re: Great Suggestion for the DNS problem...?
On Mon, Jul 28, 2008 at 12:35:30PM -0700, Tomas L. Byrnes wrote: As you pointed out, the protocol, if properly implemented, addresses this. There should always be Glue (A records for the NS) in a delegation. RFC 1034 even specifies this: 4.2.2 snip As the last installation step, the delegation NS RRs and glue RRs necessary to make the delegation effective should be added to the parent zone. The administrators of both zones should insure that the NS and glue RRs which mark both sides of the cut are consistent and remain so. /snip A probably important distinction: That's not the protocol, that's the specified implementation framework of the protocol. In general, DNS still works if you screw that up, which is why it's so often screwed up. Cheers, -- jra -- Jay R. Ashworth Baylink [EMAIL PROTECTED] Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Those who cast the vote decide nothing. Those who count the vote decide everything. -- (Josef Stalin)
Re: Great Suggestion for the DNS problem...?
On 2008/07/28 09:52 PM Jay R. Ashworth wrote: On Mon, Jul 28, 2008 at 12:35:30PM -0700, Tomas L. Byrnes wrote: As you pointed out, the protocol, if properly implemented, addresses this. There should always be Glue (A records for the NS) in a delegation. RFC 1034 even specifies this: 4.2.2 snip As the last installation step, the delegation NS RRs and glue RRs necessary to make the delegation effective should be added to the parent zone. The administrators of both zones should insure that the NS and glue RRs which mark both sides of the cut are consistent and remain so. /snip A probably important distinction: That's not the protocol, that's the specified implementation framework of the protocol. In general, DNS still works if you screw that up, which is why it's so often screwed up. Yes it should work. In fact, why *don't* implementations discard authoritative responses from non-authoritative hosts? Or do we? Or am I horribly wrong? There's an argument that IP spoofing can easily derail this, but I'd shift that argument higher up the OSI, blame TCP, and move on to recommending SYN cookies. Even if forged though, if the forged IP returns NS authority glue that doesn't match the source, the lookup still fails. DNSSEC kinda does this verification though, just more complicatedly and more reliant on administrative cooperation, and I've never met a DNS person who is cooperative ;) My suggestion though was more of replacing NS - A - IP with NS - IP That is just a brain fart though. My 0.00264050803375 cents (at current exchange rates).
Re: Great Suggestion for the DNS problem...?
Hello All: From: Paul Vixie [EMAIL PROTECTED] Date: Tue, 29 Jul 2008 01:24:43 + To: Nanog [EMAIL PROTECTED] Subject: Re: Great Suggestion for the DNS problem...? [EMAIL PROTECTED] (Jay R. Ashworth) writes: [ unthreaded to encourage discussion ] On Sat, Jul 26, 2008 at 04:55:23PM -0500, James Hess wrote: Nameservers could incorporate poison detection... Listen on 200 random fake ports (in addition to the true query ports); if a response ever arrives at a fake port, then it must be an attack, read the identified attack packet, log the attack event, mark the RRs mentioned in the packet as poison being attempted for 6 hours; for such domains always request and collect _two_ good responses (instead of one), with a 60 second timeout, before caching a lookup. The attacker must now guess nearly 64-bits in a short amount of time, to be successful. Once a good lookup is received, discard the normal TTL and hold the good answer cached and immutable, for 6 hours (_then_ start decreasing the TTL normally). Is there any reason which I'm too far down the food chain to see why that's not a fantastic idea? Or at least, something inspired by it? at first glance, this is brilliant, though with some unimportant nits. however, since it is off-topic for nanog, i'm going to forward it to the [EMAIL PROTECTED] mailing list and make detailed comments there. -- Still off topic, but perhaps a BGP feed from Cymru or similar to block IP addresses on the list? Regards, Mike
Re: Great Suggestion for the DNS problem...?
What would the ip-blocking BGP feed accomplish? Spoofed source addresses are a staple of the DNS cache poisoning attack. Worst case scenario, you've opened yourself up to a new avenue of attack where you're nameservers are receiving spoofed packets intended to trigger a blackhole filter, blocking communication between your network and the legitimate owner of the forged ip address. Yes, but what about blocking the addresses of recursive resolvers that are not yet patched? That would certainly stop them from being poisoned, and incent their owners to patch... 1/2 :-) Brian Michael Smith wrote: Still off topic, but perhaps a BGP feed from Cymru or similar to block IP addresses on the list? Regards, Mike
