Re: SNMP DDoS: the vulnerability you might not know you have
The problem isn't the people on this list leaving the public snmp community on their devices, it's the vendors of home routers leaving it there in their devices. Normal end users don't know or even care what snmp is. (nor can we expect them too) A simple scan of a large cable/dsl ISP's address space will likely net you tens of thousands of devices which respond to the public snmp community. Thomas On 13-07-31 10:57 AM, Blake Dunlap iki...@gmail.com wrote: This looks like more a security issue with the devices, not border security issues. If you're seeing replies of that size, it means the devices themselves are set up to allow public queries of their information (not secured by even keys), which no one should be comfortable with. People should never be leaving the public access snmp strings on devices even if they are internal. Edge blocking just masks the real issue. -Blake On Tue, Jul 30, 2013 at 11:25 PM, bottiger bottige...@gmail.com wrote: Before you skim past this email because you already read the Prolexic report on it or some other article on the internet, there are 2 disturbing properties that I haven't found anywhere else online. 1) After sending abuse emails to many networks, we received many angry replies that they monitored their traffic for days without seeing anything (even as we were being attacked) and that their IPs were spoofed and would block us for spamming them. What we discovered was that their firewalls/routers/gateways coming from vendors like Cisco and SonicWall apparently didn't record SNMP traffic going in or out of themselves. We confirmed this multiple times by running a query to an IP that was claimed to be clean and watching the response come 10-60 seconds later because the device was being so heavily abused. 2) SNMP reflection offers the largest amplification factor by far, even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a 68 byte query and received responses of up to 30,000 to 60,000 bytes. The trick is to use GetBulkRequest to start enumerating from the first OID and setting max repetitions to a large number. This is contrary to the other articles online which suggest a much smaller amplification factor with other queries. This protocol is also prevalent in many devices ranging from routers to printers. To solve this problem you should block SNMP traffic coming from outside your network and whitelist outside IPs that require it.
BGP instability?
Hi, Did anyone else see a large amount of instability between around 12:20am and 3:10am? (UTC, May 17th) We saw around 9 million announces per hour during that period come in through all our upstreams (vs an average normal of around 128k per hour). Just curious as to what happened, if anything. Thanks, Thomas
Mitigating DNS amplification attacks
Hi! I was wondering if anyone had any experience with dealing with open resolvers as a web hoster? We currently have some 40,000 ip's that respond to DNS in our AS, the majority of which are not open but do reply with a referral to the root zones. We've been sending emails to our clients but as the servers are not managed by us, there's not much we can do at that level. Recently we've seen a large increase in the number and volume of DNS amplification DDOS's that are being reflected off of our AS. Just today we've seen at least 6 different attacks with between 4 and 10gbps leaving our AS each time. It's not really causing us issues at the moment because we have the capacity, but I'd hate to be on the receiving side. (and indeed, have been on the receiving side in the past, so I know how much it can suck) Has anyone ever tried mitigating/rate-limiting/etc these attacks in the network before? (vs at the server/application level) We have an Arbor peakflow device, but it's not really geared for this scenario I find. It will detect the outgoing attack via the flows, but all we can really do is null-route the victims ip in our AS. Ideally we would need a way to rate-limit DNS packets based on source ip. Maybe a linux box that handles dropping packets from the same source-ip over 1000/sec with some policy-based routing sending the DNS traffic to it? Does such a box exist already? If anyone has any ideas or suggestions, then by all means! There must be a better way to do this, and I'd really like to avoid re-inventing the wheel if it's been invented already. :) Thanks! Thomas
Re: Mitigating DNS amplification attacks
Hi! On 13-04-30 7:57 PM, Dobbins, Roland rdobb...@arbor.net wrote: On May 1, 2013, at 6:43 AM, Thomas St-Pierre wrote: We've been sending emails to our clients but as the servers are not managed by us, there's not much we can do at that level. Sure, there is - shut them down if they don't comply. Most ISPs have AUP verbiage which would apply to a situation of this type. Unfortunately I somehow doubt management is going to look favourably on a request to shut down so many clients. :( The large majority of the servers being used in the attacks are not open resolvers. Just DNS servers that are authoritative for a few domains, and the default config of the dns application does referrals to root for anything else. Yes there are ways of protecting against this on the server itself, but I don't see it happening here given the complexity of many of the solutions. I hate to say it, but if it's not next - next - next - finish, or integrated as an option in one of the common web hosting panels (cPanel, Plesk, etc) people won't do it. We still struggle just getting people to close actual open resolvers, and that is easy to configure. Has anyone ever tried mitigating/rate-limiting/etc these attacks in the network before? (vs at the server/application level) QoS doesn't work, as the programmatically-generated attack traffic 'crowds out' legitimate requests. We have an Arbor peakflow device, but it's not really geared for this scenario I find. Peakflow SP is a NetFlow-based anomaly-detection system which performs attack detection/classification/traceback. Please feel free to ping me offlist about additional system elements which perform attack mitigation. Pinged off-list! Thanks! Thomas
Re: Mitigating DNS amplification attacks
Hi Damian! We offer a DNS hosted solution, most people still use their own servers though. (especially those with control panels such as cPanel or plesk, where it's built-in). As for BCP38, I would love to stop the spoofed packets, however with them coming from our upstreams, (Level3, Cogent, Tata, etc) I don't see how we can. Thanks!, Thomas From: Damian Menscher dam...@google.commailto:dam...@google.com Date: Tuesday, 30 April, 2013 8:32 PM To: Thomas St.Pierre tstpie...@iweb.commailto:tstpie...@iweb.com Cc: Dobbins, Roland rdobb...@arbor.netmailto:rdobb...@arbor.net, NANOG list nanog@nanog.orgmailto:nanog@nanog.org Subject: Re: Mitigating DNS amplification attacks On Tue, Apr 30, 2013 at 5:28 PM, Thomas St-Pierre tstpie...@iweb.commailto:tstpie...@iweb.com wrote: On 13-04-30 7:57 PM, Dobbins, Roland rdobb...@arbor.netmailto:rdobb...@arbor.net wrote: On May 1, 2013, at 6:43 AM, Thomas St-Pierre wrote: We've been sending emails to our clients but as the servers are not managed by us, there's not much we can do at that level. Sure, there is - shut them down if they don't comply. Most ISPs have AUP verbiage which would apply to a situation of this type. Unfortunately I somehow doubt management is going to look favourably on a request to shut down so many clients. :( The large majority of the servers being used in the attacks are not open resolvers. Just DNS servers that are authoritative for a few domains, and the default config of the dns application does referrals to root for anything else. Offering a DNS service to your customers may allow you to provide a good alternative to push those customers onto. You can then manage it properly. But I think DNS isn't the real issue here, it's the fact you're receiving spoofed traffic. I'd start by tracking the attacks backwards through your upstreams, as obviously someone in the path isn't enforcing BCP 38. Stop the spoof capability and the attacks will stop. It requires less effort overall (vs your counterparts at every hosting provider needing to solve the problem for their networks) and provides the best benefit to the victims. Damian
