Re: new DNS forwarder vulnerability
[catching up] That's a good question, but I know that during the ongoing survey within the Open Resolver Project [http://openresolverproject.org/], Jared found thousands of CPE devices which responded as resolvers. Not thousands, *tens of millions*. Our estimate from mid-2013 was 32M such devices (detailed in an IMC paper last year; http://www.icir.org/mallman/pubs/SCRA13/). And, that roughly agrees with both the openresolverproject.org numbers and another (not public) study I know of. And, as if that isn't bad enough ... there is a 2010 IMC paper that puts the number at 15M. I.e., the instances of brokenness are getting worse---doubling in 3 years! UGH. allman pgpql33S7o1ct.pgp Description: PGP signature
Re: new DNS forwarder vulnerability
On Apr 2, 2014, at 8:38 AM, Mark Allman mall...@icir.org wrote: [catching up] That's a good question, but I know that during the ongoing survey within the Open Resolver Project [http://openresolverproject.org/], Jared found thousands of CPE devices which responded as resolvers. Not thousands, *tens of millions*. Our estimate from mid-2013 was 32M such devices (detailed in an IMC paper last year; http://www.icir.org/mallman/pubs/SCRA13/). And, that roughly agrees with both the openresolverproject.org numbers and another (not public) study I know of. And, as if that isn't bad enough ... there is a 2010 IMC paper that puts the number at 15M. I.e., the instances of brokenness are getting worse---doubling in 3 years! UGH. One observation: The OpenResolverProject collects responses that come from ports that the query was not sent to (ie: device responds from UDP/12345 not from UDP/53, which obviously is broken and doesn't work, but they actually return DNS payload which can be used for abuse). Some good news though: http://openresolverproject.org/breakdown-graph1.cgi Since the start of 2014 there seem to be new CPE devices out there that are resolving this issue. The linear nature of the line in the decrease doesn't seem to be something like ISPs started blocking udp/53 to customers, which would appear more like a step function. I'm aware of some other studies ongoing to fingerprint CPE and their behaviors/aggregated resolver dependencies. I expect to see some of that data presented at the upcoming DNS-OARC meeting in Warsaw. Getting everyone to update their firmware on devices would go a long way as well. Some vendors have no software QA on this front so add/remove the response on the WAN interface as their releases march forward. - Jared
Re: new DNS forwarder vulnerability
In message c7e435c6-344f-49cd-9152-7a9ef2fa6...@puck.nether.net, Jared Mauch writes: On Apr 2, 2014, at 8:38 AM, Mark Allman mall...@icir.org wrote: [catching up] That's a good question, but I know that during the ongoing survey within the Open Resolver Project [http://openresolverproject.org/], Jared found thousands of CPE devices which responded as resolvers. Not thousands, *tens of millions*. Our estimate from mid-2013 was 32M such devices (detailed in an IMC paper last year; http://www.icir.org/mallman/pubs/SCRA13/). And, that roughly agrees with both the openresolverproject.org numbers and another (not public) study I know of. And, as if that isn't bad enough ... there is a 2010 IMC paper that puts the number at 15M. I.e., the instances of brokenness are getting worse---doubling in 3 years! UGH. One observation: The OpenResolverProject collects responses that come from ports that the query was not sent to (ie: device responds from UDP/12345 not from UDP/53, which obviously is broken and doesn't work, but they actually return DNS payload which can be used for abuse). Some good news though: http://openresolverproject.org/breakdown-graph1.cgi I see axes, legend but no data points. If I hover over various spots on the graph I see data values pop up. Since the start of 2014 there seem to be new CPE devices out there that are resolving this issue. The linear nature of the line in the decrease doesn't seem to be something like ISPs started blocking udp/53 to customers, which would appear more like a step function. I'm aware of some other studies ongoing to fingerprint CPE and their behaviors/aggregated resolver dependencies. I expect to see some of that data presented at the upcoming DNS-OARC meeting in Warsaw. Getting everyone to update their firmware on devices would go a long way as well. Some vendors have no software QA on this front so add/remove the response on the WAN interface as their releases march forward. - Jared -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Re: new DNS forwarder vulnerability
On Fri, Mar 14, 2014 at 5:06 PM, Wayne E Bouchard w...@typo.org wrote: Have we ascertained if there is a typical configuration adjustment that can be made to reduce or eliminate the likelihood of impact? I think your best tactic is: Provide specified DNS resolver cache servers. Don't use CPEs for DNS forwarders. The trouble is a CPE's management/locally-bound IP address is in many cases... often the same IP address that is a NAT address shared with user traffic; instead of a dedicated separate IP address that traffic can be managed and security controlled. Providing you ensure that the CPE's IP bound address is not overloaded or shared with user traffic you might try firewalling destination port 53 to the CPE, except from the proper upstream DNS resolvers, since nothing else should be replying to a DNS request made by the CPE. Look into whether the CPE can use a different, lesser-used UDP port than 53 to forward DNS requests to; use device firewall rules or upstream ACLs to limit which source IP addresses can talk to the service on the CPE's IP. To ascertain effectiveness for a specific CPE, you would need to run a sample exploit with a before and after test. (From the description it sounds as though this is not possible but it doesn't hurt to ask.) -- -JH
Re: new DNS forwarder vulnerability
Why would a CPE have an open DNS resolver from the WAN side? Gary Baribault On 03/14/2014 12:45 PM, Livingood, Jason wrote: Well, at least all this CPE checks in for security updates every night so this should be fixable. Oh wait, no, nevermind, they don't. :-( This is getting to be the vulnerability of the week club for home gateway devices - quite concerning. JL On 3/14/14, 12:05 PM, Merike Kaeo mer...@doubleshotsecurity.com wrote: On Mar 14, 2014, at 7:06 AM, Stephane Bortzmeyer bortzme...@nic.fr wrote: On Fri, Mar 14, 2014 at 01:59:27PM +, Nick Hilliard n...@foobar.org wrote a message of 10 lines which said: did you characterise what dns servers / embedded kit were vulnerable? He said We have not been able to nail this vulnerability down to a single box or manufacturer so it seems the answer is No. It is my understanding that many CPEs work off of same reference implementation(s). I haven't had any cycles for this but with all the CPE issues out there it would be interesting to have a matrix of which CPEs utilize which reference implementation. That may start giving some clues. Has someone / is someone doing this? - merike
Re: new DNS forwarder vulnerability
Why would a CPE have an open DNS resolver from the WAN side? Honest to god, are you new to computers or something? People have been writing just good enough code since the beginning. A resolver package binds to *:53 by default. Some poor firmware guys with no security experience, deadlines, and too few bytes for code storage don't notice or don't know or don't care and install the resolver feature on the firmware that they're designing, then promptly never think about it again because that feature works and is therefore done. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: new DNS forwarder vulnerability
Good question, but the reality is that a lot of them are this way. They just forward everything from any source. Maybe it was designed that way to support DDoS as a use case. Imagine a simple iptables rule like -p udp --dport 53 -j DNAT --to 4.2.2.4 I think some forwarders work this way - the LAN addresses can be reconfigured and so it's probably easier if the rule doesn't check the source address.. or maybe it was designed to work this way on purpose, because it's easy to explain as a 'bug' or oversight, rather than deliberate action. Of course, it's crazy to think that some person or organization deliberately did this so they would have a practically unlimited amount of DoS sources. -Laszlo On Mar 15, 2014, at 4:26 PM, Gary Baribault g...@baribault.net wrote: Why would a CPE have an open DNS resolver from the WAN side? Gary Baribault On 03/14/2014 12:45 PM, Livingood, Jason wrote: Well, at least all this CPE checks in for security updates every night so this should be fixable. Oh wait, no, nevermind, they don't. :-( This is getting to be the vulnerability of the week club for home gateway devices - quite concerning. JL On 3/14/14, 12:05 PM, Merike Kaeo mer...@doubleshotsecurity.com wrote: On Mar 14, 2014, at 7:06 AM, Stephane Bortzmeyer bortzme...@nic.fr wrote: On Fri, Mar 14, 2014 at 01:59:27PM +, Nick Hilliard n...@foobar.org wrote a message of 10 lines which said: did you characterise what dns servers / embedded kit were vulnerable? He said We have not been able to nail this vulnerability down to a single box or manufacturer so it seems the answer is No. It is my understanding that many CPEs work off of same reference implementation(s). I haven't had any cycles for this but with all the CPE issues out there it would be interesting to have a matrix of which CPEs utilize which reference implementation. That may start giving some clues. Has someone / is someone doing this? - merike
Re: new DNS forwarder vulnerability
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 That's a good question, but I know that during the ongoing survey within the Open Resolver Project [http://openresolverproject.org/], Jared found thousands of CPE devices which responded as resolvers. Further work needs to go into fingerprinting these devices to determine the vendor, version, etc., but it is disturbing to see such brokenness. :-/ - - ferg On 3/15/2014 9:26 AM, Gary Baribault wrote: Why would a CPE have an open DNS resolver from the WAN side? Gary Baribault On 03/14/2014 12:45 PM, Livingood, Jason wrote: Well, at least all this CPE checks in for security updates every night so this should be fixable. Oh wait, no, nevermind, they don't. :-( This is getting to be the vulnerability of the week club for home gateway devices - quite concerning. JL On 3/14/14, 12:05 PM, Merike Kaeo mer...@doubleshotsecurity.com wrote: On Mar 14, 2014, at 7:06 AM, Stephane Bortzmeyer bortzme...@nic.fr wrote: On Fri, Mar 14, 2014 at 01:59:27PM +, Nick Hilliard n...@foobar.org wrote a message of 10 lines which said: did you characterise what dns servers / embedded kit were vulnerable? He said We have not been able to nail this vulnerability down to a single box or manufacturer so it seems the answer is No. It is my understanding that many CPEs work off of same reference implementation(s). I haven't had any cycles for this but with all the CPE issues out there it would be interesting to have a matrix of which CPEs utilize which reference implementation. That may start giving some clues. Has someone / is someone doing this? - merike - -- Paul Ferguson VP Threat Intelligence, IID PGP Public Key ID: 0x54DC85B2 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlMkgYQACgkQKJasdVTchbLR1AD9Ey+ISQtaVoJKReLZ6ZzHI7/4 91h+HIQgvazMAne+NMsA/3CCQVw9KG1U6oZdouKexi8ycVw1Y4d4poH+7Yfh4zEh =bFpE -END PGP SIGNATURE-
Re: new DNS forwarder vulnerability
On 14/03/2014 13:45, Mark Allman wrote: - We have found 7--9% of the open resolver population---or 2-3 million boxes---to be vulnerable to this cache poisoning attack. (The variance is from different runs of our experiments.) did you characterise what dns servers / embedded kit were vulnerable? If so, can you share the results? Nick
Re: new DNS forwarder vulnerability
On Mar 14, 2014, at 7:06 AM, Stephane Bortzmeyer bortzme...@nic.fr wrote: On Fri, Mar 14, 2014 at 01:59:27PM +, Nick Hilliard n...@foobar.org wrote a message of 10 lines which said: did you characterise what dns servers / embedded kit were vulnerable? He said We have not been able to nail this vulnerability down to a single box or manufacturer so it seems the answer is No. It is my understanding that many CPEs work off of same reference implementation(s). I haven't had any cycles for this but with all the CPE issues out there it would be interesting to have a matrix of which CPEs utilize which reference implementation. That may start giving some clues. Has someone / is someone doing this? - merike signature.asc Description: Message signed with OpenPGP using GPGMail
Re: new DNS forwarder vulnerability
On 14/03/2014 16:05, Merike Kaeo wrote: Has someone / is someone doing this? someone has, and many CPEs use dnsmasq. current uplink too slow to find references. Nick
Re: new DNS forwarder vulnerability
Well, at least all this CPE checks in for security updates every night so this should be fixable. Oh wait, no, nevermind, they don't. :-( This is getting to be the vulnerability of the week club for home gateway devices - quite concerning. JL On 3/14/14, 12:05 PM, Merike Kaeo mer...@doubleshotsecurity.com wrote: On Mar 14, 2014, at 7:06 AM, Stephane Bortzmeyer bortzme...@nic.fr wrote: On Fri, Mar 14, 2014 at 01:59:27PM +, Nick Hilliard n...@foobar.org wrote a message of 10 lines which said: did you characterise what dns servers / embedded kit were vulnerable? He said We have not been able to nail this vulnerability down to a single box or manufacturer so it seems the answer is No. It is my understanding that many CPEs work off of same reference implementation(s). I haven't had any cycles for this but with all the CPE issues out there it would be interesting to have a matrix of which CPEs utilize which reference implementation. That may start giving some clues. Has someone / is someone doing this? - merike
Re: new DNS forwarder vulnerability
Have we ascertained if there is a typical configuration adjustment that can be made to reduce or eliminate the likelihood of impact? (From the description it sounds as though this is not possible but it doesn't hurt to ask.) On Fri, Mar 14, 2014 at 09:05:00AM -0700, Merike Kaeo wrote: On Mar 14, 2014, at 7:06 AM, Stephane Bortzmeyer bortzme...@nic.fr wrote: On Fri, Mar 14, 2014 at 01:59:27PM +, Nick Hilliard n...@foobar.org wrote a message of 10 lines which said: did you characterise what dns servers / embedded kit were vulnerable? He said We have not been able to nail this vulnerability down to a single box or manufacturer so it seems the answer is No. It is my understanding that many CPEs work off of same reference implementation(s). I haven't had any cycles for this but with all the CPE issues out there it would be interesting to have a matrix of which CPEs utilize which reference implementation. That may start giving some clues. Has someone / is someone doing this? - merike --- Wayne Bouchard w...@typo.org Network Dude http://www.typo.org/~web/
