Re: DDOS solution recommendation
On 11 Jan 2015, at 13:30, Ammar Zuberi wrote: I've done a lot of research into how these attacks actually work and most of them are done by kids who don't really know what they're doing. The really sad part is that in a huge of the cases we see, the attacks are hugely disproportionate - so many servers/services/applications/networks are so brittle and fragile and non-scalable that only a fraction of the pps/bps/cps/qps generated by the attackers would take them down, anyways. Even worse, the attackers who don't know what they're doing routinely achieve their goals, anyways, due to the above plus the unpreparedness of the defenders. I've only run across a handful of organizations which proactively took appropriate defensive measures; most only do so in the aftermath of a successful attack. It's easy to be an Internet supervillain. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
Is anyone maintaining a list of good, bad and ugly providers in terms of how seriously they take things they should like BCP38 and community support and whatever else that's quantifiable? - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: NANOG list nanog@nanog.org Sent: Sunday, January 11, 2015 7:50:22 AM Subject: Re: DDOS solution recommendation I agree with lots said here. But I've said for years (despite some people saying I am confused) that BCP38 is the single most important thing we can do to cut DDoS. No spoofed source means no amplification. It also stops things like Kaminsky DNS attacks. There is no silver bullet. Security is a series of steps (layers as one highly respected security professional has in his .sig). But the most important layer, the biggest bang for the buck we can do today, is eliminated spoofed source. Push on your providers. Stop paying for transit from networks that do not filter ingress, put it in your RFPs, and reward those who do with contracts. Make it economically advantageous to fix the problem, and people will. -- TTFN, patrick On Jan 11, 2015, at 08:46 , Mike Hammett na...@ics-il.net wrote: Well there's going to be two sources of the attack... infested clients or machines setup for this purpose (usually in a datacenter somewhere). Enough people blackhole the attacking IPs, those IPs are eventually going to have a very limited view of the Internet. They may not care of it's a server in a datacenter being used to attack, but an infested home PC would care once they can't get to Google, FaceBook, Instagram, whatever. If the attacker's abuse contact doesn't care, then just brute force of more and more of the Internet being offline to them, they'll figure it out. You hit my honeypot IPs, blackholed for 30 days. You do a DNS request to my non-DNS servers, blackholed for 30 days. Same goes for NTP, mail, web, etc. You have more than say 5 bad login attempts to my mail server in 5 minutes, blackholed for 30 days. You're trying to access various web pages known for home router or Wordpress exploitation, blackholed for 30 days. No point in letting troublemakers (manual or scripted) spend more time on the network than necessary. The more people (as a collective or not) that do this, the better. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 7:24:55 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 20:07, Mike Hammett wrote: but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. Just because we think something, that doesn't make it true. ; The way to stop this stuff is for those millions of end users to clean up their infected PCs. You may want to do some reading on this topic in order to gain a better understanding of the issues involved: https://app.box.com/s/4h2l6f4m8is6jnwk28cg Some of us have been dealing with DDoS attacks for a couple of decades, now. If it were a simple problem, we would've solved it long ago. Here's a hint: scale alone makes any problem literally orders of magnitude more difficult than any given instance thereof. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On 11 Jan 2015, at 20:50, Patrick W. Gilmore wrote: Push on your providers. Stop paying for transit from networks that do not filter ingress, put it in your RFPs, and reward those who do with contracts. Make it economically advantageous to fix the problem, and people will. Concur 100%. Unfortunately, it's only a tiny minority who understand enough to even care - and even when individuals in that tiny minority are influential within large organizations with global impact, all too often they can't get those kinds of measures implemented due to factors and priorities which are beyond their control. As you yourself know, through hard-won experience. ; --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On 11 Jan 2015, at 20:07, Mike Hammett wrote: but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. Just because we think something, that doesn't make it true. ; The way to stop this stuff is for those millions of end users to clean up their infected PCs. You may want to do some reading on this topic in order to gain a better understanding of the issues involved: https://app.box.com/s/4h2l6f4m8is6jnwk28cg Some of us have been dealing with DDoS attacks for a couple of decades, now. If it were a simple problem, we would've solved it long ago. Here's a hint: scale alone makes any problem literally orders of magnitude more difficult than any given instance thereof. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
Well there's going to be two sources of the attack... infested clients or machines setup for this purpose (usually in a datacenter somewhere). Enough people blackhole the attacking IPs, those IPs are eventually going to have a very limited view of the Internet. They may not care of it's a server in a datacenter being used to attack, but an infested home PC would care once they can't get to Google, FaceBook, Instagram, whatever. If the attacker's abuse contact doesn't care, then just brute force of more and more of the Internet being offline to them, they'll figure it out. You hit my honeypot IPs, blackholed for 30 days. You do a DNS request to my non-DNS servers, blackholed for 30 days. Same goes for NTP, mail, web, etc. You have more than say 5 bad login attempts to my mail server in 5 minutes, blackholed for 30 days. You're trying to access various web pages known for home router or Wordpress exploitation, blackholed for 30 days. No point in letting troublemakers (manual or scripted) spend more time on the network than necessary. The more people (as a collective or not) that do this, the better. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 7:24:55 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 20:07, Mike Hammett wrote: but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. Just because we think something, that doesn't make it true. ; The way to stop this stuff is for those millions of end users to clean up their infected PCs. You may want to do some reading on this topic in order to gain a better understanding of the issues involved: https://app.box.com/s/4h2l6f4m8is6jnwk28cg Some of us have been dealing with DDoS attacks for a couple of decades, now. If it were a simple problem, we would've solved it long ago. Here's a hint: scale alone makes any problem literally orders of magnitude more difficult than any given instance thereof. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On 11 Jan 2015, at 20:46, Mike Hammett wrote: Enough people blackhole the attacking IPs, those IPs are eventually going to have a very limited view of the Internet. TCAMs have limits. Not all networks practice anti-spoofing. Not all networks have any visibility whatsoever into their network traffic. Not all networks have security teams. Again, it would probably be advisable to do some reading before you start telling those of us who've been working on this set of problems for the last couple of decades that it's simple, and that we don't know what we're doing. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
Why does it seem like everyone is trying to solve this the wrong way? Do other networks' abuse departments just not give a shit? Blackhole all of the zombie attackers and notify their abuse departments. Sure, most of the owners of the PCs being used in these scenarios have no idea they're being used to attack people, but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. When the unknowing end-user reached out to support over larger and larger parts of the Internet not working, they'd be told to clean up their system. The way to stop this stuff is for those millions of end users to clean up their infected PCs. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Manuel Marín m...@transtelco.net To: nanog@nanog.org Sent: Thursday, January 8, 2015 11:01:47 AM Subject: DDOS solution recommendation Nanog group I was wondering what are are using for DDOS protection in your networks. We are currently evaluating different options (Arbor, Radware, NSFocus, RioRey) and I would like to know if someone is using the cloud based solutions/scrubbing centers like Imperva, Prolexic, etc and what are the advantages/disadvantages of using a cloud base vs an on-premise solution. It would be great if you can share your experience on this matter. Thank you
Re: DDOS solution recommendation
I agree with lots said here. But I've said for years (despite some people saying I am confused) that BCP38 is the single most important thing we can do to cut DDoS. No spoofed source means no amplification. It also stops things like Kaminsky DNS attacks. There is no silver bullet. Security is a series of steps (layers as one highly respected security professional has in his .sig). But the most important layer, the biggest bang for the buck we can do today, is eliminated spoofed source. Push on your providers. Stop paying for transit from networks that do not filter ingress, put it in your RFPs, and reward those who do with contracts. Make it economically advantageous to fix the problem, and people will. -- TTFN, patrick On Jan 11, 2015, at 08:46 , Mike Hammett na...@ics-il.net wrote: Well there's going to be two sources of the attack... infested clients or machines setup for this purpose (usually in a datacenter somewhere). Enough people blackhole the attacking IPs, those IPs are eventually going to have a very limited view of the Internet. They may not care of it's a server in a datacenter being used to attack, but an infested home PC would care once they can't get to Google, FaceBook, Instagram, whatever. If the attacker's abuse contact doesn't care, then just brute force of more and more of the Internet being offline to them, they'll figure it out. You hit my honeypot IPs, blackholed for 30 days. You do a DNS request to my non-DNS servers, blackholed for 30 days. Same goes for NTP, mail, web, etc. You have more than say 5 bad login attempts to my mail server in 5 minutes, blackholed for 30 days. You're trying to access various web pages known for home router or Wordpress exploitation, blackholed for 30 days. No point in letting troublemakers (manual or scripted) spend more time on the network than necessary. The more people (as a collective or not) that do this, the better. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 7:24:55 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 20:07, Mike Hammett wrote: but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. Just because we think something, that doesn't make it true. ; The way to stop this stuff is for those millions of end users to clean up their infected PCs. You may want to do some reading on this topic in order to gain a better understanding of the issues involved: https://app.box.com/s/4h2l6f4m8is6jnwk28cg Some of us have been dealing with DDoS attacks for a couple of decades, now. If it were a simple problem, we would've solved it long ago. Here's a hint: scale alone makes any problem literally orders of magnitude more difficult than any given instance thereof. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On Sun, Jan 11, 2015 at 5:07 AM, Mike Hammett na...@ics-il.net wrote: Why does it seem like everyone is trying to solve this the wrong way? Do other networks' abuse departments just not give a shit? Blackhole all of the zombie attackers and notify their abuse departments. Sure, most of the owners of the PCs being used in these scenarios have no idea they're being used to attack people, but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. When the unknowing end-user reached out to support over larger and larger parts of the Internet not working, they'd be told to clean up their system. The way to stop this stuff is for those millions of end users to clean up their infected PCs. 1. BCP38 protects your neighbor, do it. 2. Protect yourself by having your upstream police Police UDP to some baseline you are comfortable with. 3. Have RTBH ready for some special case. 4. Sleep better at night. I do all of the above for the last 18 months. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Manuel Marín m...@transtelco.net To: nanog@nanog.org Sent: Thursday, January 8, 2015 11:01:47 AM Subject: DDOS solution recommendation Nanog group I was wondering what are are using for DDOS protection in your networks. We are currently evaluating different options (Arbor, Radware, NSFocus, RioRey) and I would like to know if someone is using the cloud based solutions/scrubbing centers like Imperva, Prolexic, etc and what are the advantages/disadvantages of using a cloud base vs an on-premise solution. It would be great if you can share your experience on this matter. Thank you
Re: DDOS solution recommendation
On Sun, Jan 11, 2015 at 08:46:40AM -0600, Mike Hammett wrote: Is anyone maintaining a list of good, bad and ugly providers in terms of how seriously they take things they should like BCP38 and community support and whatever else that's quantifiable? This list sheds some light on antispoofing commitments made by various providers: https://www.routingmanifesto.org/participants/ Kind regards, Job
Re: DDOS solution recommendation
I’m stuck trying to find a virtual router environment that I can play with flowspec on. We do have some Juniper routers, but they are in production and I don’t think I want to touch flowspec on them just yet. Does anyone have any experience or any ideas here? Even openbgpd? On Jan 11, 2015, at 6:58 PM, Roland Dobbins rdobb...@arbor.net wrote: On 11 Jan 2015, at 20:52, Ca By wrote: 1. BCP38 protects your neighbor, do it. It's to protect yourself, as well. You should do it all the way down to the transit customer aggregation edge, all the way down to the IDC access layer, etc. 2. Protect yourself by having your upstream police Police UDP to some baseline you are comfortable with. This will come back to haunt you, when the programmatically-generated attack traffic 'crowds out' the legitimate traffic and everything breaks. You can only really do this for ntp. 3. Have RTBH ready for some special case. S/RTBH and/or flowspec are better (S/RTBH does D/RTBH, too). --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On Sun, Jan 11, 2015 at 09:58:12PM +0700, Roland Dobbins wrote: 2. Protect yourself by having your upstream police Police UDP to some baseline you are comfortable with. This will come back to haunt you, when the programmatically-generated attack traffic 'crowds out' the legitimate traffic and everything breaks. You can only really do this for ntp. You can also consider adding CHARGEN and SSDP. Kind regards, Job
Re: DDOS solution recommendation
Maybe try the Cisco CSR1000v. In the trial mode it won't give you a decent throughput, but should have all features enabled. On 11 January 2015 at 15:02, Ammar Zuberi am...@fastreturn.net wrote: I’m stuck trying to find a virtual router environment that I can play with flowspec on. We do have some Juniper routers, but they are in production and I don’t think I want to touch flowspec on them just yet. Does anyone have any experience or any ideas here? Even openbgpd? On Jan 11, 2015, at 6:58 PM, Roland Dobbins rdobb...@arbor.net wrote: On 11 Jan 2015, at 20:52, Ca By wrote: 1. BCP38 protects your neighbor, do it. It's to protect yourself, as well. You should do it all the way down to the transit customer aggregation edge, all the way down to the IDC access layer, etc. 2. Protect yourself by having your upstream police Police UDP to some baseline you are comfortable with. This will come back to haunt you, when the programmatically-generated attack traffic 'crowds out' the legitimate traffic and everything breaks. You can only really do this for ntp. 3. Have RTBH ready for some special case. S/RTBH and/or flowspec are better (S/RTBH does D/RTBH, too). --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
To quote a presentation I heard at a conference regarding small routers, Buy bigger rooters, bitches. (Yes, I know it isn't that simple, but most of the audience at that conference had purchasing authority.) Not all networks are doing what they're supposed to be (I'm on that list), but if no one ever does anything because not everyone else is, then nothing ever gets done. I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. Security teams? My network has me, myself and I. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 7:51:59 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 20:46, Mike Hammett wrote: Enough people blackhole the attacking IPs, those IPs are eventually going to have a very limited view of the Internet. TCAMs have limits. Not all networks practice anti-spoofing. Not all networks have any visibility whatsoever into their network traffic. Not all networks have security teams. Again, it would probably be advisable to do some reading before you start telling those of us who've been working on this set of problems for the last couple of decades that it's simple, and that we don't know what we're doing. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
Le 11/01/2015 14:50, Patrick W. Gilmore a écrit : I agree with lots said here. But I've said for years (despite some people saying I am confused) that BCP38 is the single most important thing we can do to cut DDoS. No spoofed source means no amplification. It also stops things like Kaminsky DNS attacks. There is no silver bullet. Security is a series of steps (layers as one highly respected security professional has in his .sig). But the most important layer, the biggest bang for the buck we can do today, is eliminated spoofed source. Push on your providers. Stop paying for transit from networks that do not filter ingress, put it in your RFPs, and reward those who do with contracts. Make it economically advantageous to fix the problem, and people will. +1 mh
Re: DDOS solution recommendation
On 11 Jan 2015, at 22:07, Job Snijders wrote: You can also consider adding CHARGEN and SSDP. People run all sorts of strange things on arbitrary ports - like VPNs, for example. It isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
Le 11/01/2015 14:50, Patrick W. Gilmore a écrit : I agree with lots said here. But I've said for years (despite some people saying I am confused) that BCP38 is the single most important thing we can do to cut DDoS. No spoofed source means no amplification. It also stops things like Kaminsky DNS attacks. There is no silver bullet. Security is a series of steps (layers as one highly respected security professional has in his .sig). But the most important layer, the biggest bang for the buck we can do today, is eliminated spoofed source. Push on your providers. Stop paying for transit from networks that do not filter ingress, put it in your RFPs, and reward those who do with contracts. Make it economically advantageous to fix the problem, and people will. +1 mh
Re: DDOS solution recommendation
Hello! If you speaking about ISP filtering you should check your subnets and ASN here: https://radar.qrator.net I was really amazed amount of DDoS bots/amplificators in my network. On Sun, Jan 11, 2015 at 6:47 PM, Michael Hallgren m.hallg...@free.fr wrote: Le 11/01/2015 14:50, Patrick W. Gilmore a écrit : I agree with lots said here. But I've said for years (despite some people saying I am confused) that BCP38 is the single most important thing we can do to cut DDoS. No spoofed source means no amplification. It also stops things like Kaminsky DNS attacks. There is no silver bullet. Security is a series of steps (layers as one highly respected security professional has in his .sig). But the most important layer, the biggest bang for the buck we can do today, is eliminated spoofed source. Push on your providers. Stop paying for transit from networks that do not filter ingress, put it in your RFPs, and reward those who do with contracts. Make it economically advantageous to fix the problem, and people will. +1 mh -- Sincerely yours, Pavel Odintsov
Re: DDOS solution recommendation
On Sun, 11 Jan 2015 22:29:33 +0700, Roland Dobbins said: On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Sounds like RFC1925, section 4 should be top of the list? pgpN98tKGHe8R.pgp Description: PGP signature
Re: DDOS solution recommendation
I didn't necessarily think I was shattering minds with my ideas. I don't have the time to read a dozen presentations. Blackhole them and move on. I don't care whose feelings I hurt. This isn't kindergarten. Maybe you should have tried a little harder to not get a virus in the first place. Quit clicking on male enhancement ads or update your OS occasionally. I'm not going to spend a bunch of time and money to make sure someone's bubble of bliss doesn't get popped. Swift, effective, cheap. Besides, you're only cut off for 30 days. If in 30 days you can prove yourself to be responsible, we can try this again. Well, that or a sufficient support request. Besides, if enough people did hat, the list of blackholes wouldn't be huge as someone upstream already blocked them. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 9:29:33 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On 11 Jan 2015, at 20:52, Ca By wrote: 1. BCP38 protects your neighbor, do it. It's to protect yourself, as well. You should do it all the way down to the transit customer aggregation edge, all the way down to the IDC access layer, etc. 2. Protect yourself by having your upstream police Police UDP to some baseline you are comfortable with. This will come back to haunt you, when the programmatically-generated attack traffic 'crowds out' the legitimate traffic and everything breaks. You can only really do this for ntp. 3. Have RTBH ready for some special case. S/RTBH and/or flowspec are better (S/RTBH does D/RTBH, too). --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
There's the Cisco xRV too, should be decent for playing around with. On 1/12/2015 午前 12:08, Dave Bell wrote: Maybe try the Cisco CSR1000v. In the trial mode it won't give you a decent throughput, but should have all features enabled. On 11 January 2015 at 15:02, Ammar Zuberi am...@fastreturn.net wrote: I’m stuck trying to find a virtual router environment that I can play with flowspec on. We do have some Juniper routers, but they are in production and I don’t think I want to touch flowspec on them just yet. Does anyone have any experience or any ideas here? Even openbgpd? On Jan 11, 2015, at 6:58 PM, Roland Dobbins rdobb...@arbor.net wrote: On 11 Jan 2015, at 20:52, Ca By wrote: 1. BCP38 protects your neighbor, do it. It's to protect yourself, as well. You should do it all the way down to the transit customer aggregation edge, all the way down to the IDC access layer, etc. 2. Protect yourself by having your upstream police Police UDP to some baseline you are comfortable with. This will come back to haunt you, when the programmatically-generated attack traffic 'crowds out' the legitimate traffic and everything breaks. You can only really do this for ntp. 3. Have RTBH ready for some special case. S/RTBH and/or flowspec are better (S/RTBH does D/RTBH, too). --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
Many attacks can use spoofed source IPs, so who are you really blocking? That's why BCP38 as mentioned many times already is a necessary tool in fighting the attacks overall. Phil On 1/11/15, 4:33 PM, Mike Hammett na...@ics-il.net wrote: I didn't necessarily think I was shattering minds with my ideas. I don't have the time to read a dozen presentations. Blackhole them and move on. I don't care whose feelings I hurt. This isn't kindergarten. Maybe you should have tried a little harder to not get a virus in the first place. Quit clicking on male enhancement ads or update your OS occasionally. I'm not going to spend a bunch of time and money to make sure someone's bubble of bliss doesn't get popped. Swift, effective, cheap. Besides, you're only cut off for 30 days. If in 30 days you can prove yourself to be responsible, we can try this again. Well, that or a sufficient support request. Besides, if enough people did hat, the list of blackholes wouldn't be huge as someone upstream already blocked them. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 9:29:33 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
I do love solutions which open larger attack surfaces than they are supposed to close. In the US, we call that a cure worse than the disease. Send packet from random bot with source of Google, Comcast, Akamai, etc. to Mr. Hammett's not-DNS / honeypot / whatever, and watch him close himself off from the world. Voilà! Denial of service accomplished without all the hassle of sending 100s of Gbps of traffic. Best part is he was willing to explain this to 10,000+ of his not-so-closest friends, in a search-engine-indexed manner. -- TTFN, patrick On Jan 11, 2015, at 14:34 , Phil Bedard bedard.p...@gmail.com wrote: Many attacks can use spoofed source IPs, so who are you really blocking? That's why BCP38 as mentioned many times already is a necessary tool in fighting the attacks overall. Phil On 1/11/15, 4:33 PM, Mike Hammett na...@ics-il.net wrote: I didn't necessarily think I was shattering minds with my ideas. I don't have the time to read a dozen presentations. Blackhole them and move on. I don't care whose feelings I hurt. This isn't kindergarten. Maybe you should have tried a little harder to not get a virus in the first place. Quit clicking on male enhancement ads or update your OS occasionally. I'm not going to spend a bunch of time and money to make sure someone's bubble of bliss doesn't get popped. Swift, effective, cheap. Besides, you're only cut off for 30 days. If in 30 days you can prove yourself to be responsible, we can try this again. Well, that or a sufficient support request. Besides, if enough people did hat, the list of blackholes wouldn't be huge as someone upstream already blocked them. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 9:29:33 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: Anyone from EPOCH Internet/MegaPath?
I'm seeing what appear to be old route objects with origin AS14558 on several other registries. I would recommend you review those and reach out to those registries while you are trying to find a Megapath contact. Maybe theres should be a world 'clean up IRR' day. Getting ARIN to wipe the objects under this maintainer should be easy. mntner: MNT-DNDY referral-by:MNT-DNDY descr: Dandy Connections Inc admin-c:SCU23-ARIN tech-c: SCU23-ARIN upd-to: m...@dandy.net mnt-nfy:m...@dandy.net auth: MAIL-FROM m...@dandy.net notify: m...@dandy.net mnt-by: MNT-DNDY changed:m...@dandy.net 20060428 source: ARIN RADB will clean this up for you. route: 209.128.240.0/20 descr: AS14558 proxy-registered route by Cogent origin: AS14558 remarks:Proxy-registered route object remarks:for Cogent customer notify: net...@cogentco.com mnt-by: MAINT-AS174 changed:net...@cogentco.com 20031230 source: RADB route: 209.128.224.0/19 descr: AS14558 proxy-registered route by Cogent origin: AS14558 remarks:Proxy-registered route object remarks:for Cogent customer notify: net...@cogentco.com mnt-by: MAINT-AS174 changed:net...@cogentco.com 20040308 source: RADB route: 76.161.33.0/24 descr: Proxy-registered route object origin: AS14558 remarks:This route object is for a BtN customer route remarks:which is being exported under this origin AS. remarks: remarks:This route object was created because no existing remarks:route object with the same origin was found, and remarks:since some BtN peers filter based on these objects remarks:this route may be rejected if this object is not created. remarks: remarks:Please contact peer...@cais.net if you have any remarks:questions regarding this object. mnt-by: MAINT-AS3491 changed:sajw...@pccwbtn.com 20080620 source: RADB Hopefully you can get a response out of Level3 to clean these out. mntner:DANDY-MNT descr: Dandy.net Maintainer admin-c: MIH1-LEVEL3 tech-c:MIH1-LEVEL3 upd-to:m...@dandy.net mnt-nfy: m...@dandy.net auth: MAIL-FROM m...@dandy.net notify:m...@dandy.net mnt-by:DANDY-MNT changed: scott.gen...@level3.com 20040629 source:LEVEL3 route: 66.159.96.0/20 descr: route object for dandy.com origin:AS14558 mnt-by:DANDY-MNT changed: scott.gen...@level3.com 20040709 source:LEVEL3 route: 209.128.224.0/19 descr: route object for Dandy.net origin:AS14558 mnt-by:DANDY-MNT changed: scott.gen...@level3.com 20040709 source:LEVEL3 route: 75.127.0.0/20 descr: cwie bgp req 20071228 origin:AS14558 mnt-by:DANDY-MNT changed: adam.heb...@level3.com 20071228 source:LEVEL3 route: 66.160.225.0/24 descr: cwie bgp req 20071228 origin:AS14558 mnt-by:DANDY-MNT changed: adam.heb...@level3.com 20071228 source:LEVEL3 Courtney Smith courtneysm...@comcast.net
Anyone from EPOCH Internet/MegaPath?
Hi, The AS number we were assigned by ARIN (AS14558) was previously owned by DANDY and was in the EPOCH routing registry. We get conflicting route generations from IRR due to this, is there anyone that can contact me off-list and get this done or does anyone have any suggestions on how I can go about getting this removed. I’ve already tried to call and email them, everyone seems clueless unfortunately. Ammar.
Re: DDOS solution recommendation
On Sun, Jan 11, 2015 at 6:46 AM, Mike Hammett na...@ics-il.net wrote: You hit my honeypot IPs, blackholed for 30 days. You do a DNS request to my non-DNS servers, blackholed for 30 days. Same goes for NTP, mail, web, etc. You have more than say 5 bad login attempts to my mail server in 5 minutes, blackholed for 30 days. You're trying to access various web pages known for home router or Wordpress exploitation, blackholed for 30 days. I urge caution in building automatic systems to respond to network abuse, lest you have unanticipated consequences. How are you tracing the source for DNS UDP, NTP UDP, etc, requests? Or TCP SYNs? If you say source address in the packet, you might not be doing what you think you're doing. Or for that matter HTTP accesses. Without giving too much discussion, let me point out: 1) You can forge a victim's IP and send packets to a honeypot (or indeed the entire IPv4 internet if you want). You may not want to assume I see a packet with this claimed source being sent to X, so it must be a bad guy and I should block it. 2) Web crawlers will follow links from Bad Guy's Site to your website, even if these links might match an IDS signature on your end. You may not want to block some search engine crawlers. 3) Legitimate recursive DNS servers can be made to connect to any IP address a bad guy wants them to connect to. You may not want to block some ISP's recursive DNS servers. There are good things to do automatically, but make sure you think them through. I used to do click fraud detection 15 years ago - when that was still a new field and we all were inventing our own ways of doing it. I was amazed at the number of ways a bad guy could do an HTTP request from millions of source IPs (hint: they weren't spoofed). I suspect it hasn't gotten better. The internet isn't able to be broken because the people building and running it are idiots. It's able to be broken because breaking things has always been far easier than building them. It takes much more intelligence, skill, and expertise to build a glass window than to throw a brick through one.
Re: DDOS solution recommendation
On Jan 11, 2015, at 15:28 , Colin Johnston col...@gt86car.org.uk wrote: unfortunately chinanet antispam/abuse email box is always full, after a while people block . always check arin/ripe for known good provider blocks and actively exclude from rules They aren't the only ones who never reply to abuse@. ddos protection via careful overview ips rules and active web source ip monitoring works well, the hard part is daily rule updates and blocks until you know most traffic is genuine. No one is advocating never block anything. However, automatic blocking based on a single DNS packet to a non-DNS server is .. let's call it counterproductive. Good hygiene is necessary both on outgoing packets and on blocking. Checking ARIN/RIPE (not APNIC, LACNIC, AFRINIC?) is not even the bare minimum you should be doing. -- TTFN, patrick On 11 Jan 2015, at 19:42, Patrick W. Gilmore patr...@ianai.net wrote: I do love solutions which open larger attack surfaces than they are supposed to close. In the US, we call that a cure worse than the disease. Send packet from random bot with source of Google, Comcast, Akamai, etc. to Mr. Hammett's not-DNS / honeypot / whatever, and watch him close himself off from the world. Voilà! Denial of service accomplished without all the hassle of sending 100s of Gbps of traffic. Best part is he was willing to explain this to 10,000+ of his not-so-closest friends, in a search-engine-indexed manner. -- TTFN, patrick On Jan 11, 2015, at 14:34 , Phil Bedard bedard.p...@gmail.com wrote: Many attacks can use spoofed source IPs, so who are you really blocking? That's why BCP38 as mentioned many times already is a necessary tool in fighting the attacks overall. Phil On 1/11/15, 4:33 PM, Mike Hammett na...@ics-il.net wrote: I didn't necessarily think I was shattering minds with my ideas. I don't have the time to read a dozen presentations. Blackhole them and move on. I don't care whose feelings I hurt. This isn't kindergarten. Maybe you should have tried a little harder to not get a virus in the first place. Quit clicking on male enhancement ads or update your OS occasionally. I'm not going to spend a bunch of time and money to make sure someone's bubble of bliss doesn't get popped. Swift, effective, cheap. Besides, you're only cut off for 30 days. If in 30 days you can prove yourself to be responsible, we can try this again. Well, that or a sufficient support request. Besides, if enough people did hat, the list of blackholes wouldn't be huge as someone upstream already blocked them. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 9:29:33 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On Jan 11, 2015, at 05:07 , Mike Hammett na...@ics-il.net wrote: Why does it seem like everyone is trying to solve this the wrong way? Because it’s what we CAN do. Do other networks' abuse departments just not give a shit? Blackhole all of the zombie attackers and notify their abuse departments. Sure, most of the owners of the PCs being used in these scenarios have no idea they're being used to attack people, but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. When the unknowing end-user reached out to support over larger and larger parts of the Internet not working, they'd be told to clean up their system. The way to stop this stuff is for those millions of end users to clean up their infected PCs. Agreed… However, let’s look at it from an economics perspective… The average residential service provider doesn’t have the resources and doesn’t charge enough to build the resources to deal with this onslaught. It won’t be the service provider that the attacker blames for the initial few disconnections, it will be the websites in question. So, let’s say XYZ.COM http://xyz.com/ is a really popular site with lots of end-users. Some of those end-users are also unknowingly attacking XYZ.COM http://xyz.com/. XYZ.COM http://xyz.com/ black holes those customers (along with all the other zombies attacking them). XYZ.COM http://xyz.com/ gets angry calls from those customers and has no ability to contact the rest. The rest don’t call their ISP or XYZ.COM http://xyz.com/ because they don’t know that they are unsuccessfully trying to reach XYZ.COM http://xyz.com/, so they don’t see the problem. Depending on hold times, etc., XYZ.COM http://xyz.com/ loses some fraction of their customers (who instead of cleaning up their system, move into the second group who don’t care about the problem any more.) The rest may clean up their systems. So, at the cost of some fraction of their customer base and a substantial burden on their call center, XYZ.COM http://xyz.com/ has managed to clean up a relatively small percentage of systems, but accomplished little else. I’m all for finding a way to do a better job of this. Personally, I’d like to see some sort of centralized clearing house where credible reporters of dDOS information could send some form of standardized (automated) report. The clearing house would then take care of contacting the responsible ISPs in a scaleable and useful manner that the ISPs could handle. Because the clearing house would be a known credible source and because they are providing the information in a way that the ISP can more efficiently utilize the information, it MIGHT allow the ISP to take proactive action such as contacting the user and addressing the problem, limiting the user’s ability to send dDOS traffic, etc. However, this would require lots of cooperation and if such a clearing house were to evolve, it would probably have to start as a coalition of residential ISPs. Owen
Re: DDOS solution recommendation
If that were to happen, it'd be for 30 days and it'd be whatever random residential account or APNIC address that was doing it. Not really a big loss. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: NANOG list nanog@nanog.org Sent: Sunday, January 11, 2015 1:42:13 PM Subject: Re: DDOS solution recommendation I do love solutions which open larger attack surfaces than they are supposed to close. In the US, we call that a cure worse than the disease. Send packet from random bot with source of Google, Comcast, Akamai, etc. to Mr. Hammett's not-DNS / honeypot / whatever, and watch him close himself off from the world. Voilà! Denial of service accomplished without all the hassle of sending 100s of Gbps of traffic. Best part is he was willing to explain this to 10,000+ of his not-so-closest friends, in a search-engine-indexed manner. -- TTFN, patrick On Jan 11, 2015, at 14:34 , Phil Bedard bedard.p...@gmail.com wrote: Many attacks can use spoofed source IPs, so who are you really blocking? That's why BCP38 as mentioned many times already is a necessary tool in fighting the attacks overall. Phil On 1/11/15, 4:33 PM, Mike Hammett na...@ics-il.net wrote: I didn't necessarily think I was shattering minds with my ideas. I don't have the time to read a dozen presentations. Blackhole them and move on. I don't care whose feelings I hurt. This isn't kindergarten. Maybe you should have tried a little harder to not get a virus in the first place. Quit clicking on male enhancement ads or update your OS occasionally. I'm not going to spend a bunch of time and money to make sure someone's bubble of bliss doesn't get popped. Swift, effective, cheap. Besides, you're only cut off for 30 days. If in 30 days you can prove yourself to be responsible, we can try this again. Well, that or a sufficient support request. Besides, if enough people did hat, the list of blackholes wouldn't be huge as someone upstream already blocked them. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 9:29:33 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
Hello! But abuse@ contacts is very-very-very hard way to contacting with ASN administrator in case of attack. Big amount of requests to #Nanog about please contact ASN noc with me offlist confirms this. I'm got multiple attacks from well known ISP and I spend about 10-20 hours to contacting they in average. It's unacceptable time We need FAST and RELIABLE way to contacting with noc of attackers network for effective attack mitigation. We need something like RTBH for knocking network admin of remote network. Maybe somebody can create social network for noc's with API ?:) On Sun, Jan 11, 2015 at 11:55 PM, Owen DeLong o...@delong.com wrote: On Jan 11, 2015, at 05:07 , Mike Hammett na...@ics-il.net wrote: Why does it seem like everyone is trying to solve this the wrong way? Because it’s what we CAN do. Do other networks' abuse departments just not give a shit? Blackhole all of the zombie attackers and notify their abuse departments. Sure, most of the owners of the PCs being used in these scenarios have no idea they're being used to attack people, but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. When the unknowing end-user reached out to support over larger and larger parts of the Internet not working, they'd be told to clean up their system. The way to stop this stuff is for those millions of end users to clean up their infected PCs. Agreed… However, let’s look at it from an economics perspective… The average residential service provider doesn’t have the resources and doesn’t charge enough to build the resources to deal with this onslaught. It won’t be the service provider that the attacker blames for the initial few disconnections, it will be the websites in question. So, let’s say XYZ.COM http://xyz.com/ is a really popular site with lots of end-users. Some of those end-users are also unknowingly attacking XYZ.COM http://xyz.com/. XYZ.COM http://xyz.com/ black holes those customers (along with all the other zombies attacking them). XYZ.COM http://xyz.com/ gets angry calls from those customers and has no ability to contact the rest. The rest don’t call their ISP or XYZ.COM http://xyz.com/ because they don’t know that they are unsuccessfully trying to reach XYZ.COM http://xyz.com/, so they don’t see the problem. Depending on hold times, etc., XYZ.COM http://xyz.com/ loses some fraction of their customers (who instead of cleaning up their system, move into the second group who don’t care about the problem any more.) The rest may clean up their systems. So, at the cost of some fraction of their customer base and a substantial burden on their call center, XYZ.COM http://xyz.com/ has managed to clean up a relatively small percentage of systems, but accomplished little else. I’m all for finding a way to do a better job of this. Personally, I’d like to see some sort of centralized clearing house where credible reporters of dDOS information could send some form of standardized (automated) report. The clearing house would then take care of contacting the responsible ISPs in a scaleable and useful manner that the ISPs could handle. Because the clearing house would be a known credible source and because they are providing the information in a way that the ISP can more efficiently utilize the information, it MIGHT allow the ISP to take proactive action such as contacting the user and addressing the problem, limiting the user’s ability to send dDOS traffic, etc. However, this would require lots of cooperation and if such a clearing house were to evolve, it would probably have to start as a coalition of residential ISPs. Owen -- Sincerely yours, Pavel Odintsov
Re: DDOS solution recommendation
You are very confused about how the Internet works. Or did you not understand the words with source of? Wait, maybe you have some magic to tell the actual source of a packet than the 32/128 bits in the source field? Because if you do, you stand to make a few billion dollars, and I'll be one of the first to pay you for it. (I'm specifically excluding things that give hints like TTL incoming interface. To get paid, you need to tell me the ACTUAL source of a spoofed packet.) While I will admit I do not know which of the above is true, my money is on #1. -- TTFN, patrick On Jan 11, 2015, at 16:08 , Mike Hammett na...@ics-il.net wrote: If that were to happen, it'd be for 30 days and it'd be whatever random residential account or APNIC address that was doing it. Not really a big loss. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: NANOG list nanog@nanog.org Sent: Sunday, January 11, 2015 1:42:13 PM Subject: Re: DDOS solution recommendation I do love solutions which open larger attack surfaces than they are supposed to close. In the US, we call that a cure worse than the disease. Send packet from random bot with source of Google, Comcast, Akamai, etc. to Mr. Hammett's not-DNS / honeypot / whatever, and watch him close himself off from the world. Voilà! Denial of service accomplished without all the hassle of sending 100s of Gbps of traffic. Best part is he was willing to explain this to 10,000+ of his not-so-closest friends, in a search-engine-indexed manner. -- TTFN, patrick On Jan 11, 2015, at 14:34 , Phil Bedard bedard.p...@gmail.com wrote: Many attacks can use spoofed source IPs, so who are you really blocking? That's why BCP38 as mentioned many times already is a necessary tool in fighting the attacks overall. Phil On 1/11/15, 4:33 PM, Mike Hammett na...@ics-il.net wrote: I didn't necessarily think I was shattering minds with my ideas. I don't have the time to read a dozen presentations. Blackhole them and move on. I don't care whose feelings I hurt. This isn't kindergarten. Maybe you should have tried a little harder to not get a virus in the first place. Quit clicking on male enhancement ads or update your OS occasionally. I'm not going to spend a bunch of time and money to make sure someone's bubble of bliss doesn't get popped. Swift, effective, cheap. Besides, you're only cut off for 30 days. If in 30 days you can prove yourself to be responsible, we can try this again. Well, that or a sufficient support request. Besides, if enough people did hat, the list of blackholes wouldn't be huge as someone upstream already blocked them. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 9:29:33 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
peeringdb.com is usually quite accurate. -- Stephen On 2015-01-11 4:11 PM, Pavel Odintsov wrote: Hello! But abuse@ contacts is very-very-very hard way to contacting with ASN administrator in case of attack. Big amount of requests to #Nanog about please contact ASN noc with me offlist confirms this. I'm got multiple attacks from well known ISP and I spend about 10-20 hours to contacting they in average. It's unacceptable time We need FAST and RELIABLE way to contacting with noc of attackers network for effective attack mitigation. We need something like RTBH for knocking network admin of remote network. Maybe somebody can create social network for noc's with API ?:) On Sun, Jan 11, 2015 at 11:55 PM, Owen DeLong o...@delong.com wrote: On Jan 11, 2015, at 05:07 , Mike Hammett na...@ics-il.net wrote: Why does it seem like everyone is trying to solve this the wrong way? Because it’s what we CAN do. Do other networks' abuse departments just not give a shit? Blackhole all of the zombie attackers and notify their abuse departments. Sure, most of the owners of the PCs being used in these scenarios have no idea they're being used to attack people, but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. When the unknowing end-user reached out to support over larger and larger parts of the Internet not working, they'd be told to clean up their system. The way to stop this stuff is for those millions of end users to clean up their infected PCs. Agreed… However, let’s look at it from an economics perspective… The average residential service provider doesn’t have the resources and doesn’t charge enough to build the resources to deal with this onslaught. It won’t be the service provider that the attacker blames for the initial few disconnections, it will be the websites in question. So, let’s say XYZ.COM http://xyz.com/ is a really popular site with lots of end-users. Some of those end-users are also unknowingly attacking XYZ.COM http://xyz.com/. XYZ.COM http://xyz.com/ black holes those customers (along with all the other zombies attacking them). XYZ.COM http://xyz.com/ gets angry calls from those customers and has no ability to contact the rest. The rest don’t call their ISP or XYZ.COM http://xyz.com/ because they don’t know that they are unsuccessfully trying to reach XYZ.COM http://xyz.com/, so they don’t see the problem. Depending on hold times, etc., XYZ.COM http://xyz.com/ loses some fraction of their customers (who instead of cleaning up their system, move into the second group who don’t care about the problem any more.) The rest may clean up their systems. So, at the cost of some fraction of their customer base and a substantial burden on their call center, XYZ.COM http://xyz.com/ has managed to clean up a relatively small percentage of systems, but accomplished little else. I’m all for finding a way to do a better job of this. Personally, I’d like to see some sort of centralized clearing house where credible reporters of dDOS information could send some form of standardized (automated) report. The clearing house would then take care of contacting the responsible ISPs in a scaleable and useful manner that the ISPs could handle. Because the clearing house would be a known credible source and because they are providing the information in a way that the ISP can more efficiently utilize the information, it MIGHT allow the ISP to take proactive action such as contacting the user and addressing the problem, limiting the user’s ability to send dDOS traffic, etc. However, this would require lots of cooperation and if such a clearing house were to evolve, it would probably have to start as a coalition of residential ISPs. Owen
Re: DDOS solution recommendation
I know that UDP can be spoofed, but it's not likely that the SSH, mail, etc. login attempts, web page hits, etc. would be spoofed as they'd have to know the response to be of any good. There's more going on than UDP spoofing\amplification. Frankly the most damaging thing to me has been SMTP hijacking. For you to login to my SMTP server and send e-mail out, there's going to be one hell of a conversation going on. However, the thought is that if someone's PC is hijacked and trying to login to my SMTP server, it'll be doing something else later (or even concurrently). Enough deployment (in addition to BCP 38), and most of the threats are mitigated. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: NANOG list nanog@nanog.org Sent: Sunday, January 11, 2015 3:14:27 PM Subject: Re: DDOS solution recommendation You are very confused about how the Internet works. Or did you not understand the words with source of? Wait, maybe you have some magic to tell the actual source of a packet than the 32/128 bits in the source field? Because if you do, you stand to make a few billion dollars, and I'll be one of the first to pay you for it. (I'm specifically excluding things that give hints like TTL incoming interface. To get paid, you need to tell me the ACTUAL source of a spoofed packet.) While I will admit I do not know which of the above is true, my money is on #1. -- TTFN, patrick On Jan 11, 2015, at 16:08 , Mike Hammett na...@ics-il.net wrote: If that were to happen, it'd be for 30 days and it'd be whatever random residential account or APNIC address that was doing it. Not really a big loss. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Patrick W. Gilmore patr...@ianai.net To: NANOG list nanog@nanog.org Sent: Sunday, January 11, 2015 1:42:13 PM Subject: Re: DDOS solution recommendation I do love solutions which open larger attack surfaces than they are supposed to close. In the US, we call that a cure worse than the disease. Send packet from random bot with source of Google, Comcast, Akamai, etc. to Mr. Hammett's not-DNS / honeypot / whatever, and watch him close himself off from the world. Voilà! Denial of service accomplished without all the hassle of sending 100s of Gbps of traffic. Best part is he was willing to explain this to 10,000+ of his not-so-closest friends, in a search-engine-indexed manner. -- TTFN, patrick On Jan 11, 2015, at 14:34 , Phil Bedard bedard.p...@gmail.com wrote: Many attacks can use spoofed source IPs, so who are you really blocking? That's why BCP38 as mentioned many times already is a necessary tool in fighting the attacks overall. Phil On 1/11/15, 4:33 PM, Mike Hammett na...@ics-il.net wrote: I didn't necessarily think I was shattering minds with my ideas. I don't have the time to read a dozen presentations. Blackhole them and move on. I don't care whose feelings I hurt. This isn't kindergarten. Maybe you should have tried a little harder to not get a virus in the first place. Quit clicking on male enhancement ads or update your OS occasionally. I'm not going to spend a bunch of time and money to make sure someone's bubble of bliss doesn't get popped. Swift, effective, cheap. Besides, you're only cut off for 30 days. If in 30 days you can prove yourself to be responsible, we can try this again. Well, that or a sufficient support request. Besides, if enough people did hat, the list of blackholes wouldn't be huge as someone upstream already blocked them. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Roland Dobbins rdobb...@arbor.net To: nanog@nanog.org Sent: Sunday, January 11, 2015 9:29:33 AM Subject: Re: DDOS solution recommendation On 11 Jan 2015, at 22:21, Mike Hammett wrote: I'm not saying what you're doing is wrong, I'm saying whatever the industry as a whole is doing obviously isn't working and perhaps a different approach is required. You haven't recommended anything new, and you really need to do some reading in order to understand why it isn't as simple as you seem to think it is. Security teams? My network has me, myself and I. And a relatively small network, too. If for example ChinaNet's abuse department isn't doing anything about complains, eventually their whole network gets blocked a /32 at a time. *shrugs* Their loss. Again, it isn't that simple. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
On 01/11/2015 03:22 PM, Mike Hammett wrote: I know that UDP can be spoofed, but it's not likely that the SSH, mail, etc. login attempts, web page hits, etc. would be spoofed as they'd have to know the response to be of any good. I encourage you to investigate Triangular Spamming. (http://www.cs.ucr.edu/~zhiyunq/pub/oakland10_triangular_spamming.pdf) The Triangular... technique does specifically that, allow the attacker to ...know the responses In short, the bot receives the reply to the spoofed source IP and forwards information on to the attacker so that it can continue the conversation. In effect, three parties are having a one way conversation in a ring. There's more going on than UDP spoofing\amplification. Frankly the most damaging thing to me has been SMTP hijacking. For you to login to my SMTP server and send e-mail out, there's going to be one hell of a conversation going on. Yes, there is what appears to you to be be a conversation going on. However, the source of what you are hearing is not where you think it's from. -- Grant. . . . unix || die
Re: DDOS solution recommendation
On 01/11/2015 07:42 PM, Mark Andrews wrote: Just because you can only identify one of the two remotes doesn't mean that you can't report the addresses. It is involved in the communication stream. It is very difficult to make a case that the host with the spoofed IP address is attacking you when it is not even sending any traffic to you. The ISP will very likely not see ANY traffic originating from spoofed IP destined to your server. So what you do know is effectively useless. Actually it is coming from where you think it is coming from, just not directly. No, not quite. 1 - Spammer (A) sends packets to server (B) spoofing the source address of the relay (C). (A spoofed as) C - B 2 - Server (B) replies to relay (C) B - C 3 - Relay (C) sends packets to spammer (A). C - A Notice how the relay (C) is never sending packets -to- the server (B). The traffic is NOT coming from the relay (C). This is not a case of the spammer (A) sending to the relay (C) that is then sending the traffic to the server (B). There is no traffic originating from the relay (C) going to the server (B). Thus there is nothing to be caught by the relay's ISP ISP filter. You could even use this technique on ISPs that block outbound traffic to TCP port 25. (Like many cable / DSL providers.) Also notice how the server (B) never knows the spammer's (A) real IP. This is very similar in concept to a Joe Job, but at the TCP layer, not the SMTP application layer. The point of this is that it is possible, and occurring in the wild, to spoof TCP source IP addresses. - So, don't blindly trust the source IP address used for TCP connections. - It is possible (if not practical) to spoof them and have a successfully transmission. -- Grant. . . . unix || die
Re: DDOS solution recommendation
On 11 Jan 2015, at 23:09, valdis.kletni...@vt.edu wrote: Sounds like RFC1925, section 4 should be top of the list? Indeed - as well as section 8. ; --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
In message 54b31bbe.3000...@tnetconsulting.net, Grant Taylor writes: On 01/11/2015 03:22 PM, Mike Hammett wrote: I know that UDP can be spoofed, but it's not likely that the SSH, mail, etc. login attempts, web page hits, etc. would be spoofed as they'd have to know the response to be of any good. I encourage you to investigate Triangular Spamming. (http://www.cs.ucr.edu/~zhiyunq/pub/oakland10_triangular_spamming.pdf) The Triangular... technique does specifically that, allow the attacker to ...know the responses In short, the bot receives the reply to the spoofed source IP and forwards information on to the attacker so that it can continue the conversation. In effect, three parties are having a one way conversation in a ring. Just because you can only identify one of the two remotes doesn't mean that you can't report the addresses. It is involved in the communication stream. There's more going on than UDP spoofing\amplification. Frankly the most damaging thing to me has been SMTP hijacking. For you to login to my SMTP server and send e-mail out, there's going to be one hell of a conversation going on. Yes, there is what appears to you to be be a conversation going on. However, the source of what you are hearing is not where you think it's from. Actually it is coming from where you think it is coming from, just not directly. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Re: DDOS solution recommendation
On Sun, Jan 11, 2015 at 5:07 AM, Mike Hammett na...@ics-il.net wrote: Blackhole all of the zombie attackers and notify their abuse departments. Sure, most of the owners of the PCs being used in these scenarios have no idea they're being used to attack people, but I'd think that if their network's abuse department was notified, either they'd contact the customer about it issue or at least have on file that they were notified. When the unknowing end-user reached out to support over larger and larger parts of the Internet not working, they'd be told to clean up their system. Notification to abuse departments is largely a waste of time, but I've tried it anyway. My records indicate that over the past year I sent 3139 emails covering 24054 known-infected machines regarding 16 distinct incidents. A few machines were cleaned, but the attacks continue. Part of the problem is that most network providers don't have the resources to chase down abuse issues. In one case I informed an ISP of ~70k infected customers. They said their support team couldn't possibly handle that, and took no action. In another case, a well-known ISP was unable to receive my list because they bounced emails over a certain size. I try to bypass the ISP where possible by sending notices directly to users ( http://googleblog.blogspot.com/2011/07/using-data-to-protect-people-from.html and http://googleonlinesecurity.blogspot.com/2012/05/notifying-users-affected-by-dnschanger.html). That has a provable effect, though not as large as one might hope. Your later comment of blackholing is indeed quite effective (I once blackholed 3 IPs at a hosting provider who had ignored 3 abuse complaints over 3 months, and they had the machines cleaned within days), but is a last resort since there can be significant collateral damage (which is, of course, why they suddenly decided to care). I've also encouraged website owners to care by marking their website as infected in Google search results. On Sun, Jan 11, 2015 at 5:50 AM, Patrick W. Gilmore patr...@ianai.net wrote: But I've said for years (despite some people saying I am confused) that BCP38 is the single most important thing we can do to cut DDoS. Yes, agreed. I've been working on this, but unfortunately nobody is ready to take action, often citing hardware limitations. And since nobody is compliant, there's no way to push others to upgrade. On Sun, Jan 11, 2015 at 6:51 AM, Job Snijders j...@instituut.net wrote: On Sun, Jan 11, 2015 at 08:46:40AM -0600, Mike Hammett wrote: Is anyone maintaining a list of good, bad and ugly providers in terms of how seriously they take things they should like BCP38 and community support and whatever else that's quantifiable? This list sheds some light on antispoofing commitments made by various providers: https://www.routingmanifesto.org/participants/ I have traced spoofed-source attacks to providers on that list. I once considered posting a list-of-shame, but it would be too long (and not win any friends here). On Sun, Jan 11, 2015 at 10:09 AM, Joel Maslak jmas...@antelope.net wrote: I urge caution in building automatic systems to respond to network abuse, lest you have unanticipated consequences. I'm always amused at the automation people create. Googlebot is a frequent victim of admins who know perl, but not /robots.txt. Damian
Re: DDOS solution recommendation
On 11 Jan 2015, at 23:33, Mike Hammett wrote: I don't have the time to read a dozen presentations. Then just read one: https://app.box.com/s/r7an1moswtc7ce58f8gg Skip the screenshots entirely, if you want, and just read the textual slides at the beginning and the end. --- Roland Dobbins rdobb...@arbor.net
Re: DDOS solution recommendation
In message 54b34a12.4000...@tnetconsulting.net, Grant Taylor writes: On 01/11/2015 07:42 PM, Mark Andrews wrote: Just because you can only identify one of the two remotes doesn't mean that you can't report the addresses. It is involved in the communication stream. It is very difficult to make a case that the host with the spoofed IP address is attacking you when it is not even sending any traffic to you. It is accepting the reply traffic and forwarding it to the originator. It is directly involved. The ISP will very likely not see ANY traffic originating from spoofed IP destined to your server. They will see the reply traffic and will see the acks increasing etc. So what you do know is effectively useless. Actually it is coming from where you think it is coming from, just not directly. No, not quite. 1 - Spammer (A) sends packets to server (B) spoofing the source address of the relay (C). (A spoofed as) C - B 2 - Server (B) replies to relay (C) B - C 3 - Relay (C) sends packets to spammer (A). C - A Notice how the relay (C) is never sending packets -to- the server (B). The traffic is NOT coming from the relay (C). This is not a case of the spammer (A) sending to the relay (C) that is then sending the traffic to the server (B). There is no traffic originating from the relay (C) going to the server (B). Thus there is nothing to be caught by the relay's ISP ISP filter. You could even use this technique on ISPs that block outbound traffic to TCP port 25. (Like many cable / DSL providers.) Also notice how the server (B) never knows the spammer's (A) real IP. This is very similar in concept to a Joe Job, but at the TCP layer, not the SMTP application layer. The point of this is that it is possible, and occurring in the wild, to spoof TCP source IP addresses. - So, don't blindly trust the source IP address used for TCP connections. - It is possible (if not practical) to spoof them and have a successfully transmission. There is no difference to this than asymetric routing. The address you are presented with is part of the communication path. -- Grant. . . . unix || die -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Recommended L2 switches for a new IXP
Dear Nanog community We are trying to build a new IXP in some US Metro areas where we have multiple POPs and I was wondering what do you recommend for L2 switches. I know that some IXPs use Nexus, Brocade, Force10 but I don't personally have experience with these switches. It would be great if you can share your experience and recommendations. There are so many options that I don't know if it makes sense to start with a modular switch (usually expensive because the backplane, dual dc, dual CPU, etc) or start with a 1RU high density switch that support new protocols like Trill and that supposedly allow you to create Ethernet Fabric/Clusters. The requirements are simple, 1G/10G ports for exchange participants, 40G/100G for uplinks between switches and flow support for statistics and traffic analysis. Thank you and have a great day. Regards
