Re: [dns-operations] note for the peanut gallery Re: underline in TXT's host
于 2012-12-15 2:37, Fred Morris 写道: So therefore let me state that I suggest that the unwary reader SHOULD NOT paste this into a shell to find out. I'll spoil the fun and tell you that it would delete everything in whatever directory you were in, and all of the directories below it. He is kind enough to not saying `rm -rf /` :) ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] note for the peanut gallery Re: underline in TXT's host
On Dec 17, 2012, at 11:55 AM, Mike Hoskins (michoski) micho...@cisco.com wrote: -Original Message- From: Feng He fen...@nsbeta.info Date: Monday, December 17, 2012 3:31 AM To: dns-operations@lists.dns-oarc.net dns-operations@lists.dns-oarc.net Subject: Re: [dns-operations] note for the peanut gallery Re: underline in TXT's host 于 2012-12-15 2:37, Fred Morris 写道: So therefore let me state that I suggest that the unwary reader SHOULD NOT paste this into a shell to find out. I'll spoil the fun and tell you that it would delete everything in whatever directory you were in, and all of the directories below it. He is kind enough to not saying `rm -rf /` :) It's been a long time since I was bored enough to try something like this for fun, but from what I remember on an old Solaris box / might be kinder -- if often fails quickly due to permissions, /dev being processed early, etc. If nothing else, this thread was a great advertisement for backups. My favorite is noticing that emacs, mutt, whatever is not working right. So I do 'ls -al' and notice that .emacs.d, .mutt, whatever is owned by root… Now, obviously, chown'ing each directory is too hard / annoying, so I run: sudo chown -R $USER .* Yup, .* helpfully expands to '.' and '..' before '.emacs.d', and so chown walks up and then back down the fielsystem, chowning *everything* to $USER…. It usually[0] runs for 5 - 10 seconds before I wonder why it's all taking so long and realize the issue…. This biggest problem with this is that I always think that I can recover from this by copying permissions from another machine by doing 'ls -alR /', and then massaging the output with sed and awk and passing that back to chown…. This never results in a reasonable end state… W [0]: Yes, I have in fact done this more than once :-P [1]: Whenever faced with a problem, some people say `Lets use AWK.' Now, they have two problems. -- D. Tilbrook :-) ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs -- Real children don't go hoppity-skip unless they are on drugs. -- Susan, the ultimate sensible governess (Terry Pratchett, Hogfather) ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
[dns-operations] DNS ANY requests from Amazon?
I'm seeing a bunch of DNS ANY requests to my authoritative servers with Amazon EC2 source IPs. I guess somebody is now trying to run an amplification attack against Amazon? This is the first time I've seen Amazon targeted this way; are others seeing this (am I just late to the party)? -- Chris Adams cmad...@hiwaay.net Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble. ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
Hi Chris, Can you forward me any specific IPs, logs, traffic captures, or any other data you have off list? I'll get it to the right Amazon people to investigate. On Mon, Dec 17, 2012 at 11:02 AM, Chris Adams cmad...@hiwaay.net wrote: I'm seeing a bunch of DNS ANY requests to my authoritative servers with Amazon EC2 source IPs. I guess somebody is now trying to run an amplification attack against Amazon? This is the first time I've seen Amazon targeted this way; are others seeing this (am I just late to the party)? -- Chris Adams cmad...@hiwaay.net Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble. ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs -- Colm ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
On 2012-12-17 7:02 PM, Chris Adams wrote: I'm seeing a bunch of DNS ANY requests to my authoritative servers with Amazon EC2 source IPs. I guess somebody is now trying to run an amplification attack against Amazon? This is the first time I've seen Amazon targeted this way; are others seeing this (am I just late to the party)? it's happened before. can you share the addresses? and are you using some form of ratelimiting? paul -- It seems like the rules for automagic completion of incomplete names typed into browsers are going to start to look like those for the game of fizbin. --rick jones ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
I'm seeing a bunch of DNS ANY requests to my authoritative servers with Amazon EC2 source IPs. I guess somebody is now trying to run an amplification attack against Amazon? Highly likely. This is the first time I've seen Amazon targeted this way; are others seeing this (am I just late to the party)? You're just late to the party. This has been going on for months. Steinar Haug, AS 2116 ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
Chris, Yes, many sites are seeing increasing background noise from Internet hosts repetitively submitting DNS queries, especially for ANY. Amplification attacks, or simply burning CPU cycles. It's starting to look like per-client-IP rate-limiting features are necessary, with intelligent defaults, to ensure applications facing the Internet are protected out-of-the-box, while service providers and others with IT staff can adjust the settings where necessary. The current default settings for most applications to provide unlimited response to any IP address, especially for non-stateful protocols (e.g. UDP), is proving to be noisy. Where some customers haven't implemented rate-limiting within BIND, mitigation is available at the O/S and network layer. As an example, there are connection limits that can be enforced with iptables on Linux. Per-source-IP connection limits can also be restricted on Cisco ASA firewalls (and likely other vendor products). There is a patch available for rate-limiting inside BIND. -Original Message- From: dns-operations-boun...@lists.dns-oarc.net [mailto:dns-operations-boun...@lists.dns-oarc.net] On Behalf Of sth...@nethelp.no Sent: Monday, December 17, 2012 2:27 PM To: cmad...@hiwaay.net Cc: dns-operati...@mail.dns-oarc.net Subject: Re: [dns-operations] DNS ANY requests from Amazon? I'm seeing a bunch of DNS ANY requests to my authoritative servers with Amazon EC2 source IPs. I guess somebody is now trying to run an amplification attack against Amazon? Highly likely. This is the first time I've seen Amazon targeted this way; are others seeing this (am I just late to the party)? You're just late to the party. This has been going on for months. Steinar Haug, AS 2116 ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
On 2012-12-17 7:57 PM, Patrick, Robert (CONTR) wrote: ... Where some customers haven't implemented rate-limiting within BIND, mitigation is available at the O/S and network layer. As an example, there are connection limits that can be enforced with iptables on Linux. Per-source-IP connection limits can also be restricted on Cisco ASA firewalls (and likely other vendor products). such rate limits are too coarse-grained for dns authority service. if you limit your request flows rather than your response flows, then your only choice is: too low, where a legitimate client asking a legitimately diverse set of questions, does not get reliable service; or, too high, where an attacker can get enough of your bandwidth directed at a victim to be damaging. OS-level rate limiting also lacks the ability to insert TC=1 responses on a statistical basis, thus transforming rate limiting into transaction delay rather than transaction loss. to make this work without breaking things, the rate limiting logic has to be within the server itself, and it has to be applied to responses not requests. There is a patch available for rate-limiting inside BIND. see http://www.redbarn.org/dns/ratelimits for background, including patches (which are not currently supported by ISC) and a technical note (which looks a bit like an RFC that some day i hope RRL will deserve.) paul ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
It's starting to look like per-client-IP rate-limiting features
are necessary...
There is a patch available for rate-limiting inside BIND.
There is also RRL code for NSD.
Please note that the main thrust of the BIND and NSD rate limiting
code is response rate limiting (RRL) and *NOT* per-client IP address
rate limiting. Per-client rate limiting is generally the best that
can be done with simple firewall rules or access control lists, but
has limitations and can cause harm. While rate limiting by client IP
address stops a reflection attack, it also turns off almost all DNS
service to the client from the server. Temporarily denying name service
to a target has long been a major part of more serious security problems
than denials of service. For example, if you need to fool your target
about the IP address of www.example.com, it's handy to have the several
seconds of a full set of DNS client timeouts to try many DNS transaction
IDs instead only the milliseconds before the real answer arrives.
With RRL (especially with the slip feature), the victim of a reflection
attack often sees no change in DNS services from the rate limiting
server during a reflection attack. With client IP address rate limiting,
the server stops answering practically all requests from the victim.
The current version the BIND RRL patch does have support for
per-client rate limiting, but it exists only to satisfy popular
demand. Its use is a bad idea in most cases.
I've said something like this before but I keep seeing claims that
BIND rate limiting is harmful or bad based on the mistaken notion that
it limits requests by IP addresses instead of limiting responses by
{IP,qname,qtype}.
The other common claim about RRL is that it is too expensive. Never
mind that much bigger servers are using RRL than the servers run by
people expressing that concern.
Vernon Schryverv...@rhyolite.com
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations
dns-jobs mailing list
https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
From: Patrick, Robert (CONTR) robert.patr...@hq.doe.gov I don't disagree that limiting responses is a smarter tact than limiting requests, with respect to making an informed decision prior to discarding traffic. Having zone and query-type plus response data to evaluate the client hash is more information than looking only at source and destination IP address, as may be implemented at a firewall or within the O/S. Some of these data elements could also be tracked by an application-aware firewall. Yes, you could do response rate limiting (RRL) within an application aware firewall by have the firewall do almost of all of the work of your DNS server. For example, your RRL mechanism (whether in a firewall or DNS server) must count all NXDOMAIN responses to a given IP address as identical to avoid spewing GBytes/sec of big signed NXDOMAIN responses about distinct random, invalid domains. `dig +dnssec asdf1234asdf.com @a.gtld-servers.net` gives a 1K NXDOMAIN. Referrals have a similar issue. A firewall that is as DNS aware as that should not waste the computing it has done to know whether to discard the response it computed to count. If things are ok, it should go ahead and send the response. Never mind that consistency, maintenance, and other problems that always come with duplicating things, whether definitions of constants in code or the big chunks of code and data that are a modern DNS server. ... Allow administrators the freedom to set the limit to any value and/or disable the feature, but shipping the product with a smart default may be viewed as a pragmatic step forward in noise reduction. The right RRL value depends on each server's popularity. It might be reasonable to ship DNS software with a default rate limit suitable for modest servers (e.g. 5 or fewer responses/second) and expect big server operators to make adjustments. Continuing to deploy into the wild without any rate-limiting isn't the best approach long term. Neither is tolerating unnecessary open recursive servers and ignoring BCP-38. Vernon Schryverv...@rhyolite.com ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
On Mon, Dec 17, 2012 at 02:57:28PM -0500, Patrick, Robert (CONTR) robert.patr...@hq.doe.gov wrote a message of 36 lines which said: mitigation is available at the O/S and network layer. As an example, there are connection limits that can be enforced with iptables on Linux. The attached mini-HOWTO may be interesting to some. [Not public] If you have a DNS server used for reflection+amplification attack *and* it is a Linux machine *and* you have Netfilter = 1.4 *and* you cannot or does not want to install the patches for BIND or NSD to do rate-limiting (they may provide a better result) *and* the attack is over IPv4 *and* the attacker uses only a few domain names, you could be interested in this technique. Disclaimer: it works for us, it will not work for ever, it works now. The idea is to use the Netfilter u32 module to recognize the attack, then to rate-limit it with the Netfilter hashlimit module. First, get the iptables rules generation script http://www.bortzmeyer.org/files/generate-netfilter-u32-dns-rule.py. Then, look at the traffic so see the pattern: what query type (typically ANY), what query domain name, etc. In the examples, we'll assume QTYPE=ANY, QNAME=example.net. Then, generate the Netfilter rule: iptables -A INPUT -p udp --dport 53 -m u32 \ --u32 $(python generate-netfilter-u32-dns-rule.py --qname example.net --qtype ANY) -j RATELIMITER The RATELIMITER chain can be: iptables -A RATELIMITER -m hashlimit \ --hashlimit-name DNS --hashlimit-above 20/second --hashlimit-mode srcip \ --hashlimit-burst 100 --hashlimit-srcmask 28 -j DROP or you can replace -j RATELIMITER by -j DROP of you want to be radical. There are more options in the generate-netfilter-u32-dns-rule.py script, such as --bufsize=NNN if the attacker uses a fixed EDNS buffer size (some do). There are several ways for the attacker to work around this technique (some obvious and some not so obvious). I will not describe them here but my point is that it works *today*, with *actual* attacks. So, it definitely helps but keep your eyes open, have alternative solutions in place and do not put all your eggs in one basket ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
Re: [dns-operations] DNS ANY requests from Amazon?
On Mon, Dec 17, 2012 at 08:17:18PM +, Paul Vixie p...@redbarn.org wrote a message of 33 lines which said: if you limit your request flows rather than your response flows, then your only choice is: too low, where a legitimate client asking a legitimately diverse set of questions, does not get reliable service; In theory, you're right. In practice, the attacks of *today* are quite simple and quite separate from normal DNS traffic (nobody asks ANY isc.org in the real world, except the attackers). I appreciate the BIND RRL patch and it is obvious to me that we must continue the research in dDoS mitigation, but let's not drop the mitigations techniques that work *today*. (The attackers are not superhuman, they use imperfect techniques.) OS-level rate limiting also lacks the ability to insert TC=1 responses on a statistical basis, thus transforming rate limiting into transaction delay rather than transaction loss. Yes. see http://www.redbarn.org/dns/ratelimits for background, including patches (which are not currently supported by ISC) In actual deployments, some people may be unwilling or unauthorized (corporate policy) to install unofficial patches on a production server. That's why we should not reject blindly the OS-level rate limiters (see my mini-HOWTO in this thread). ___ dns-operations mailing list dns-operations@lists.dns-oarc.net https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
