Re: Dyn DDoS this AM? - dns

[alvin nanog]

On 10/21/16 at 03:21pm, David Birdsong wrote:
> On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush  wrote:
> > anyone who relies on a single dns provider is just asking for stuff such
> > as this.

:-)

> I'd love to hear how others are handling the overhead of managing two dns
> providers.

in my view of ( automated ) dns managment:

Only on the one "master" dns server, make your DNS changes, update the 
serial number for example.com changes and reload the new update zone
file ... notifications goes out to all known slave DNS servers ..

For all the other authorized DNS servers, they should all automatically 
update itself ... magic all dns servers are in sync ...

some folks don't like "master" DNS server vs slaves .. i donno why not ..

but, you do have to configure your "master dns server" properly to 
only allow only authorized slaves access to their dns reccords

similarly, slave DNS servers should only update from it's recognized
master dns server

there should be zero isues with managing 2 dns server or 100 dns servers

before downloading new dns info, Man-in-the-Middle tests with OpenSSL 
certs should be done to confirm the other end is in fact who you think
it is that you're going to be sending dns info to or receiving from

c ya
alvin
http://DDoS-Mitigator.net 

Re: New Office, New Network. Questions.

[alvin nanog]

hi nikolai

- oops.. this got long based on my experiences/opinions :-)

On 07/10/16 at 09:53pm, Nikolai Petrov wrote:
> We are moving to our new offices in two months and I have access to the 
> building already.
> My task is to set up the entire network for the company.
> The previous administrator has left the company and I thought of taking the 
> chance to remove some "technical debt" and make everything from scratch again.

all good ... 

> I was told to move the networks this week 

do you have the routers, switches, cables, few servers for testing ?
has your ISP installed their internet uplink connectivity to the bldg ?

if so, than the above management is on their toes 
otherwise, you'd need to rattle some $$$ loose to buying missing hw :-)

> and I have spent a lot of time thinking about how I should do it.

good ... now's the chance to fix the problems if any ..

> 1. Currently we do not have IPv6 in our network 

implies a learning IPv6 curve ( red flag for possible time-wasting hogs )

if the task is to mvoe the entire "mid-sized" from current bldg to new bdlg, 
i'd suggest use "known/good/working/best-practices" methodology to move
the company.  first get the new bldg with new test servers working
with IPv4 ( the way you want it done ) and "working" the current bldg 
which should take a few hours :-)

than work with IPv6 issues 

> but I have seen the ISP is giving us a "/56 Block" 

good

> which from what I understand is a couple hundred "/64 Subnets".
> I think you can only have /64 subnets in IPv6.

nah ... you can subnet your /56 into whatever you want

> In our IPv4 setup we have 32 addresses,
> four of which I will use for NAT 
> and the remaining needed for online services and servers.

good ... use that to test everything 

since you want or going to use NAT, you have the standard
internal LAN for the bldg can use the standard 10/8 or 
192.168/16 or 172.16/12

so far.. nothing new/special/problematic

> In IPv6 we have a lot of addresses but I am not sure whether 
> I should give an address of the ISP to every device.

why would you want to complicate time-restricted ( 1month )
to get the new bldg working with IPv6 w/out having prior
IPv6 experience ?

remember, "all eyes" will be looking to you to move the
whole company from current bldg to new bldg without delay

> I found that there is an organization that can help avoid collisions 
> in private IPs: https://www.sixxs.net/tools/grh/ula/ .

there should never be any collision in IP#, ipv4 or ipv6

>  From what I can tell it is just a registry, but I am thinking of 
> registering the ranges there and then use these subnets and 
> NAT them to the IPv6 address of the router.

the ISP provides you the range of IPv6 assigned to you

if your current bldg does NOT have IPv6, you might not be
able to easily test the new IPv6 stuff in the new bldg

you might be able to test your new IPv6 connections
at the local coffee shop or other public places but
that's a major security violation since your new IPv6
has no security pre-cautions installed yet

you should be paranoid about trojans/worms/mailware piggie
backing into your new un-secured new bldg IPv6 infrastructure
or IPv4 infrastructure

> However, I noticed something strange. The WAN port of our
> router gets a /64 IPv6 address which is not in our IPv6.

why strange ??

routers get its IP# from dhcpv6 or statically  assigned

> Should I use this for NAT or one of "our" addresses?

you need to fix this problem before continuing .. 
( explain why the IPv6/64 is not what you're expecting )

NAT is NOT the solution ...

> 2. The previous administrator did some bad job in some parts of the network.

:-) that will always be true 90% of the time :-)

some things are always gonna be "bad"

> We have an internal router protocol to move traffic between routers, 
> but in some cases he used NAT instead of adding these subnets to the 
> router protocol. Everything works and all things that have to be 
> reached are reachable, 

if it works .. why is is "bad" ??

there might be dozens of different ways to make things work 
( "things that have to be reachable are reachable" )

> however I think this is bad 

not necessarily a bad thing

> and we should use the router protocol for all parts of the network.

why ?

> I have found two protocols in our router that are good and support 
> IPv6 and they are OSPF and BGP.

there might be more :-)

>  I did not manage to have BGP work 

what part is not working ?

google/yahoo the error messages :-) 

> and it is slow so I am thinking of OSPF. 

sometimes, which works first/better/easiest might be
a good option, thus trying other things is good, but
that can also create more headaches too .. more problems
to (fun) solve

> Do uou think it is a good choice for IPv6 and IPv4? 

i'd work with IPv4 first ... 
and more importantly... there is NO excuse why IPv4 doesn't
or cannot work in the new bldg

after IPv4 works in the new bldg as good as it does in the 

Re: Detecting Attacks

[alvin nanog]

hi su..

On 06/10/16 at 10:39pm, subashini hariharan wrote:
> I am Subashini, a graduate student. I am interested in doing my project in
> Network Security. I have a doubt related to it.

duh... too broad of a subject ... you'd need to be more specific about which
of the hundred's of sub categories ...

> The aim is to detect DoS/DDoS attacks using the application.

good ... sorta specific but not ...

> I am going to
> use ELK (ElasticSearch, Logstash, Kibanna) for processing the logs (Log
> Analytics).

hummm, why that app and not the couple dozen other ways people are using
to detect incoming and/or outgoing DDoS attacks

if the "professor" says "use ELK" ... you have no ther choice ...
if not, there's much better options to detect DDoS attacks ...

( tcpdump -nnvv ) ... if you cannot explain each line, you've got a DDoS problem

> My doubt is regarding how do we generate logs for detecting this attack? As
> I am new to this process, I am not sure about it.

what's the doubt ?? if there is a doubt ... conduct and experiment and 
see if it confirms your expected result or explain why its different
and do more experiments until "its all explained" and no more doubts

> Also, if it is possible to do any other attacks similar to this, you can
> please give a hint about it.

several dozens other types of attacks similar to DDoS, which  takes over
a server or network offline including no-technical-skill required attacks

> Could anyone please help with this, it would be a great help!!

google/yahoo/bing is your assistant ready to give you ALL the 
answer's you need and ant

---

side notes ...
a) if you log all incoming packets ( attacks ), you have increased the
   effectiveness of ddos attacks since you have now gave them the power 
   to fill up your disk, use your cpu, use your memory, use your time 
   to review the logs, etc, etc 

   all of that is bad bad stuff to have the DDoS attackers do to you 

b) for logs, etc, there are dozens of other apps that try to detect
   attacks ( splunk, snort, hundred other apps, including eyeballs )

   why are some methodologies better than others ?

c) detecting DDoS attacks is nice but, what's the point ??
   you're still under attack ... and haven't resolved the issue

   kinda like cooking dinner but not eating it ... you're still starving

d) every computer connected to the internet is under constant 24x7x365
   attacks ... a good "ddos detector" will tell you how much traffic
   is legitimate and how much bandwidth is wasted by the attacks
   and which server and which ports they are attacking, etc etc

   script kiddies are already attacking your network 
   ( the one you're using bnow ) .. it's a free and harmless DDoS attacks
   and you should be able to see what they are doing to you "now"

   if you cannot "see what" they are attacking, you've got a major problem

e) if you want to generate some specific DDoS attacks ..
   use ping, nping, hping, nmap, etc to start  that should
   keep you busy for the next year or few years

   do NOT ever send packets outside to computers you do not own,
   or some ominous looking folks might come looking for you

f) if you want to detect DDoS attacks  post process tcpdump's output

magic pixie dust
alvin
#
# DDoS-Simulator.net
# DDoS-Mitigator.net
#

Re: Webmail / IMAPS software for end-user clients in 2016

[alvin nanog]

hi yta

On 06/08/16 at 06:43pm, Eric Kuhnke wrote:
> openwebmail hasn't been updated since 2006...

yup.. a minor/major issue

> squirrelmail is ancient and barely maintained.

last update ( svn ) was Jun 09, 2016 ( today )
http://squirrelmail.org/download.php

if you like the "latest/greatest" sw ... debian is not always the 
best choice, as their *.deb packages are sometimes too old for the
binaries it's packaging compared to the author's stable releases
( latest stable packages want in the distro:
( kernel, apache, sendmail, postfix, sql, php, perl, dovecot, etc

-
barely maintained etc is not necessarily bad ...
- sometimes, you want stable software that doesn't change every day
or every week

- presumably, "more stable" sw will not have as many bugs that
requires releasing yet another version

- sometimes, time-tested (stable) software, used by thousands of users 
with thousands of different OS/mta/browsers is a a good thing ..

- optionally, to use the latest dev released from yesterday/last week
is equally good, esp for bug fixes and security patches

> Antivirus and antispam are handled by the SMTP system which operates on the
> backend of the webmail, by the time incoming mail gets to dovecot imap
> storage for the user accounts it has already been processed.

okay ... virus and spam can be stopped at many different places 
or even outsourced .. it's not just the MTA's job to do filter it

> Antivirus/antispam handled similarly on other servers for outgoing SMTP
> traffic.

good if you're stopping outgoing virus/spam... :-) don't forget
the incoming virii/wormns too

have fun
alvin

Re: Webmail / IMAPS software for end-user clients in 2016

[alvin nanog]

hi ya 

On 06/08/16 at 06:06pm, Eric Kuhnke wrote:
> If you had to put up a public facing webmail interface for people to use,
> and maintain it for the foreseeable future (5-6 years), what would you use?
>
> Roundcube?
> https://roundcube.net/
- good

> Rainloop?
> http://www.rainloop.net/
- never used
- w/o db support, how you maintain a (real) list of x,000 users and pwd

> Something else?

http://squirrlemail.org
- good

http://openwebmail.org/
- least effort to get webmail running ( esp if time is limited )

http://horde.org
- possibly confusing install process

-
imaps from doveocot.org
( note differences between dovecot-1.x vs dovecot-2.x )

> Requirements:
> Needs to be open souce and GPL, BSD or Apache licensed
> 
> Email storage will be accessed via IMAP/TLS1.2
> 
> Runs on a Debian based platform with apache2 or nginx
> 
> Desktop browser CSS and mobile device CSS/HTML functionality on 4" to 7"
> size screens with Chrome and Safari

- you probably want support for your favorite sql app
- you probably want support for your favorite anti-virus app
- you probably want support for your favorite anti-spam app

http://networknightmare.net/WebMail/

magic pixie dust
alvin
# DDoS-Mitigator.net
#

Re: Building a technical library

[alvin nanog]

hi ya chris

On 05/31/16 at 08:53pm, Ca By wrote:
> On Tuesday, May 31, 2016, Chris Costa  wrote:
> 
> > Looking to develop a technical library for about 15 staff members all under
> > the same roof.  Subject matter would focus around Juniper/Junos, TCP/IP,
> > dwdm, python, java, and expand from there.  The O'reilly Safari service
> > looks rather comprehensive, although at $400/user there may be more value

$400 is good for around 5-8 new books .. 

> > buying hard copies and own them outright (or at least until they walk
> > away). 

books are like paper clips and pens .. somehow they grow legs and like
to play hide-n-seek

> > Are there other online resources that offer a good value?  Other
> > experiences weighing the pros and cons?

i'm one that say "owning" is better than renting under some cases ..
- books is best owned because you probably have to look up the 
  subject matter again and again, or put it into your online notes 

- usually, books comes with examples on DVD, or at least the ones i get

- oreilly series, sam's, dummy series, IDG series, one can go broke :-)
  i have most of the common topics, but not cisco/juniper type stuff
  and most all books have been in dozens of boxes since moving and 
  not unpacked

- google is always good resource but filtering thru various posts/blogs
  implies you know what you're looking for

- barns n noble is usually good for quick browsing 
- computer literacy is always good ( sf bay area ), not sure if they still
  around, similarly for digital guru

> I have always used the public library, libraries i am familiar with have
> ebook offerings that are comparable to safari if not safari itself

public libraries are seriously lacking "good techie books" stuff

university libraries are good too, but not sure if they let anybody browse

another option .. everybody contributes to your "wiki" with pertinent info
of what you were expecting to find in the "rent-a-book" services

happy reading ( n comprehension )
alvin
#
# DDoS-Mitigator.net
# DDoS-Simulator.net
#

Re: DDoS protection: Corero

[alvin nanog]

hi 

On 05/12/16 at 01:21pm, Ragnar Sigurðsson Joensen wrote:
> Quick question. Is there anyone on this list using Corero for DDoS 
> protection? If so I'd much appreciate an off-list review of it. Thanks in 
> advance.

hummm ... just some generic comments when comparing "DDoS protection"

one DDoS solution is NOT necessarily a cost-effective mitigation
against all the various types of DDoS attacks

various types of attacks:

   - tcp-based DDoS attacks on any port are best mitigated with 
   iptables + tarpits ( in-house appliance could handle up to 100gig/sec )

   the attacking zombie bots should crash long before they can 
   affect your servers
   ( 100,000 ddos packet/sec * 2Kbyte/packet * 120sec tcp timeouts )

   - udp-based DDoS attacks are best mitigated by confirming that
   your DNS server/app, NTP server/app, SNMP server/app, NFS, X11,
   etc, etc properly patched and hardened

   your ISP will most likely have to be involved to mitigate
   incoming UDP and ICMP based attacks using various methods
   like flow analysis/collection/mediation, rtbh, bgp, etc

#
#  if you like the idea of just 'drop the packet" or "limit it",
#  then, it's too late as you have already received the DDoS packets
#  and the damage is done ...
#

   - volumetric attacks ( say over 10gigbit/s ) probably will 
  require various data-centers spread across the oceans
  or use the cloud ...

   - you will need a security policy ( infrastructure policy ) 
   to define "legitimate traffic" and possibly incomign DDoS attacks

   simple minded rule:

web servers should only run "apache/etc", all packets to the
65,534 ports are attacks

mail servers should only run "sendmail/etc", all packets to 
the other 65,534 ports are attacks

  - DDoS attacks consisting of silly spam, virii, worms should be 
  non-issues and imho, is easily mitigated w/ dozens of different 
  foss tools and "company/computer/infrastructure policy"

magic pixie dust
alvin
#
# http://DDoS-Mitigator.net . http://DDoS-Simulator.net 
#

Re: how to deal with port scan and brute force attack from AS 8075 ?

[alvin nanog]

hi nanog'ers

On 03/31/16 at 10:20am, valdis.kletni...@vt.edu wrote:
> On Thu, 31 Mar 2016 10:02:05 +0200, "marcel.duregards--- via NANOG" said:
> 
> > We consider port scan and brute force on ssh port as an attack, and even
 
...
> (For the record, our border routers drop inbound SYN on port 22 on *both*
> ipv4 and ipv6 address spaces.  It's amazing how few brute force
> attempts we see on our servers... :)

i think the best way, ( imho ) to discourage random incoming ssh connections
or anything else ( tcp-based ) is to run tarpit on ALL tcp based ports ... 
one obviously would allow incoming 25/tcp traffic to mail servers
and incoming 80/tcp to web servers, etc etc, but otherwise, all
other incoming tcp ports gets unconditionally tarpit'd

we used to get hundreds of thousands of garbage tcp connections per 
minute
which basically disappeared after running tarpits as needed

and the attackers ( port scanners ) pay a penalty for sending useless
packets to tarpit'd ports

fail2ban/etc is okay but it's too limited since i want to deny all tcp 
connections
and specifically only allow certain incoming traffic which is trivial to 
implement with iptables + tarpits

/dev/null incoming packets is okay but it still occupied time/space/buffers
in the pipe and the attackers didn't feel any pain for sending the packets

doing ddos mitigation for your own IP# space is fairly easy to create
various policies ... doing the ddos mitigation for your customers down
the line using your routers can be tricky business and very messy if
either the customer nor isp doesn't change something ( aka more $$$ )

magic pixie dust
alvin
DDoS-Mitigator.net

Re: Fwd: port 123 reflection attacks

[alvin nanog]

hi ya colin

On 12/30/15 at 09:04am, Colin Johnston wrote:
> Where does it say we need to contact home cert instead on your website ?

because cnc...@cert.org.cn asked ?

> verification of what ?

i'd want to see if it's a simple port scan by a script kidddie vs 
a more serious upcoming DOS attack from attackers with a "evil purpose"

they might just be poking around to find vulnerable ntpd servers ?

since there's been no satisfactory answer in 5 days, 
in the meantime, i'd suggest:
- be sure ntpd is properly configured
- be sure to be running the latest ( no known exploits ) ntpd server
- ntpd servers should only be necessary for your servers ...
  and incoming connections from outside should never reach your ntpd
- use an alternative ntpd server/source on a different wire

> HSOFT ranges have been compromised by NTP reflection attacks 

there's a difference between compromized vs port scanning ( probes )

- compromized... hsoft need to fix it ( upgrade and reconfigure ntpd ) 

- probes/scanners ... nothing much you can do other than limit your 
  outgoing ( 123/udp) replies

- there's thousands of probes occuring constantly on various ports ...

> and the NTP servers hosted by HSOFT need to have a NTP update.

they better get going to update their ntpd and configs ... 

i'd rattle hsoft's cage harder ... :-)

> This has been discussed on NANOG and I also sent information in Chinese to 
> aid debug as well.
> 
> Have had no response from HSOFT…

:-)

i wonder what else is occupying their time

magic pixie dust
alvin
# DDoS-Simulator.net

> > From: "cncertcc" 
> > Subject: Re:Fwd: port 123 reflection attacks
> > Date: 30 December 2015 at 08:15:28 GMT
> > To: "Colin Johnston" 
> > 
> > Greetings,
> > Please forward the case to the corresponding CERT you are located in first 
> > to have it transferred to CNCERT after verification. Thanks for your 
> > understanding.
...
> >>> From: Colin Johnston  >>> >
> >>> Subject: port 123 reflection attacks
> >>> Date: 25 December 2015 at 11:19:26 GMT
> >>> To: 16036...@qq.com , i...@cnnic.cn 
> >>> 
> >>> Cc: Colin Johnston >
> >>> 
> >>> please stop the port 123 reflection attacks from 115.47.24.220

Re: Nat

[alvin nanog]

hi folkx

On 12/17/15 at 10:28am, Mark Andrews wrote:
> We need to make IPv4 painful to use.

already is too crowded

> Adding  delay between SYN and SYN/ACK would be one way to achieve this.


change tcp windoow size to 1 byte per packet or decrease from 1500 byte
packets, more traffic they use, slower it becomes 

instead of zero byte as used in tarpits

>  Start at 100ms..200ms and increase it by 100ms each year.

some of verizon's shared IPv4 traffic has delays exceeding 3sec 
and i seen it exceeding 6sec for simple things like using gmail
thru their network

"delays" are built in automatically ...
- too much spam ..
- too much useless video downloads
- too much useless steaming
- too much useless pix
- too much games 

and alll that junk will increase as more people use it

it'd be nice to put these "services" on their own private LAN
and slow just themself down instead of slowing everybody down

pay more $$$ to get better/faster connectivity ...


ducking for cover
alvin

Re: Ransom DDoS attack - need help!

[alvin nanog]

hi

On 12/10/15 at 11:07am, Joe Morgan wrote:
> These are the three e-mail addresses they have contacted me on so far.
> armada.collect...@bk.ru
> melvin.webst...@gmail.com
> luciennemcglyn...@gmail.com

Ian> messages came from a various bitmessage.ch addresses

# i wonder if they all have the same X-Originating-IP" or the ame
# X-Mailer sw which may imply the same script kiddie or the same 
# "group" sending the "i hope they pay up wish list emails"

Barry> I wonder how much of this is due to language difficulties.
Barry>
Barry> Imagine if all your abuse messages and lots of this often informal
Barry> (and formal) documentation was in Chinese or Russian.


i've always thought, since the 80's and 90's that the computers
( PCs, servers, routers ) managed by non-english speaking folks
and non-computer-geeks ( we seem to call them sys admins and 
IT dept nowdays ) will be more susceptable to "take over"
by those that know how to hijack computers/routers w/o being noticed

given that every culture has their criminals ... there is a possibility
that the english speaking criminals are the ones using mis-configured
servers and routers for their benefit and purposes 

side note, some folks are trying to make $$ with viagra and other meds
but, notice that most of that viagra/meds spam s!@#$ is gone 

there are the email marketer non-nonsense ... probably the ones
controlling the zombie bots ( foreign PCs ) spewing out 25% of the 
world's emails

there are very specific attacks from old culture chinese, N koreans, 
russians and other notorious groups ... etc
that are after certain info ( they may not be after $$$ since its
all gov't $$$ to start with ) .. something to protect against 24x7x365

i'd also worry about the well-known anonymous groups that can actualy
carry out the xxxGbps DDoS attacks and take out high profile targets
- they should be sending out their emails from
anonymous servers ... 
- i doubt that google/yahoo could be considered "anonymous"
( non-traceable ) vs throw away temp emails

the nuisance ransoms from script kiddies probably will not
be able to followup, but one did hopefully take preventative
measures spending time and $$$ ... i think they're the ones
asking ( demanding )  for $20 to not the more reasonable
$$$ per specific DDoS multi-national or large local businesses

--

locally, there seems to a modified virus running around infecting 
small business PCs wiping out their silly quickbooks and emails 
contacts unless the small biz pay up $xx,000 within couple days

no warnings or demands by emails ... all automated which also implies
they might not be able to stop the virus even if the ransom was paid

#
# automated, virus controlled ransoms are a very bad thing
#
removing the virus doesn't help .. since it'd already
removed some or all of your email contacts and quickboosk

hopefully they learned NOT to click on attachments

i donno why the biz's books is exposed to the world
and they don't have clean backups thus their panic to call
the local tv stations ..
( i say they hired a bad outsourced IT dept, but than again,
( some folks tend to be lazy and not listen to the IT dept

magic pixie dust
alvin
# DDoS-Mitigator.net
# Unix'ing since 1970's
#

Re: Ransom DDoS attack - need help!

[alvin nanog]

hi jean-f

On 12/08/15 at 11:46pm, Jean-Francois Mezei wrote:
> Since the OP mentioned a "ransom" demand (aka: extortion), should law
> enforcement be contacted in such cases ?

simply saying "these bozo's are attempting to extort $100 from me"
with their email demands probably will not get the law enforcements attention

yes ... only after you have done everything you can and ready to take
the attackers to court but need law enforcement to haul them into court
and/or seize their computers for evidence

- (ntpdate/ntpd) sync your clock so that your logs have accurate time 

- check the ip# of the email servers and routers it came thru

  you may or may not need to worry about spoof'ed ip# since they 
  want you to get hold of them to give um the $$

- contact the abuse@-the-ISP  for each of those routers and servers
- traceroute the IP# of the mail servers 
- "whois IP#" and contact each of the ISPs

- contact the ISPs that provide connectivity to your "drop off point"
  of where you "supposed to pay up" ... we're assuming that the
  dropoff point is NOT controlled/owned by the ddos attackers

- since you know what time/date/etc that they threaten to attack,
  you should verify your data on the backup systems
  ( build a clone and keep it offline )

  everybody ( you, the ISP, cops, etc ) can all be watching the 
  DDoS attacks and tracing it back to the originating script kiddie
  or the entire extortion network

  you should also get secondary connectivity to watch the DDoS attacks
  in progress and trace it back to the originating source

  let them attack ( the honeypot ) so you can trace it back...

  tarpit all the tcp-based services so that you have 2minutes to 
  trace the attacks back to them ... they cannot "hang up" until 
  the tcp connection attempts times out

- when everything is setup ... tell the DDoS attackers the $$$
  is ready for pickup and watch the DDoS attackers attempt to
  collect the $$$ that doesn't really exist

> Is there any experience doing this ? 

yup...

> Are they any help ?

nope if you don't have the info they want see .. 

you should make it easy for them to take action to get court orders 
to haul them in

yup ... if the cops are trying to collect evidence "on the DDoS attackers"
you'd be in luck

yup ... if the DDoS attackers are large enough and/or if they're attacking 
the high profile victims

> In North america, would that mean FBI in USA and RCMP in Canada, or
> local police force which then escalates to proper law enforcement agency ?

escalation starts with you to provide all the necessary info ...
nobody else will be doing that work for you

get hold of the security dept of your ISP  and any other ISP
along the traceroute and whois iP# way back to the DDoS attackers 

ISPs probably have their favorite agents they like to work with
to chase down the xxx-most-wanted DDoS attackers

magic pixie dust
alvin
# DDoS-Mitigator.net

Re: Ransom DDoS attack - need help!

[alvin nanog]

hi joe

On 12/08/15 at 01:24am, Joe Morgan wrote:
> We received a similar ransom e-mail yesterday 

:-)

dont pay real $$$ ... pretend that it was paid and watch for
them to come get the ransom ... never give your real banking info

ask them, where do you send the "$xx,000" mastercard gift card
by fedex/ups/dhl ... law enforcement might get lucky with real 
physical addresses ... once in a while, there are dumb criminals
that show up on tv news

> followed by a UDP flood attack. 

*pout* or not  ... their demo shows they've got the zombie botnet
capable of sending 20+Gbps  law enforcement and ISP security dept 
"should be interested" to trace them down ... but it takes
tons of (their) resources to take the next steps: who is it and
where are the attackers

*pout* ... udp ddos floods are "expensive" to solve ...

unfortunately, you cannot mitigate any incoming UDP-ddos attacks at your
server/router udp mitigation has to be done by"
- somehow, you need to find out who they are etc and legally seize their botnet
- your upstream ISP/peer whom doesn't send it to you
- or you setup and 2nd pipe at a geographically different colo ( cheaper )
- or you first send your udp traffic thru a ( expensive ) ddos scrubber

the idea of "limit" the udp traffic is basically useless, since
udp packets already came down the wire ... 

you should at least not reply to any udp ddos packet 
- don't send "host not available", etc etc

> Here is a sample of the attack traffic we received as well as a
> copy of the ransom e-mail. Thought this might be useful to others who have
> been targeted as well. I will have to talk with our upstream providers to
> get a definitive on the size of the attacks. At the point in time we
> blackholed our ip we were seeing 20+Gbps.
> 
> *Dec/07/2015 5:40:22PM *Here is a summary of the flows to our web server IP
> during the ddos event:

since it is a webserver they're playing with ... there's "dozen" things you
can do to mitigate the UDP flood attacks
- web server should only be running apache ...
  remove ntpd, bind, etc, etc, etc aka, remove the risks of udp amplification
- make sure required things like ntpd/sshd etc are using local non-routable ip#
- long common sense list of stuff to do ... including the 4 points listed above

everybody would want the timezone so they can check their "bandwidth" monitor
to see if 20Gbps hurts them too

> Top 10 flows by packets per pecond for dst IP: 96.43.134.147
>   Duration Proto  Src IP Addr Src Pt Dst Pt  Packets  pps  bps
>  0.001 UDP  175.43.224.99  1900  2245620482.0 M5.8 G
>  0.002 UDP120.199.113.49  1900  5417720481.0 M2.8 G
>  0.002 UDP27.208.164.227  1900  5417720481.0 M2.7 G

what app do yu have that talks to port 1900 ?

these are probably spoof'd src address  but you will never know
until you look up these ip# to see if there is any common link to it
like it all belonging to the same zombie net

for all ListofZombiehosts
do
 - whois 175.43.224.99
 - traceroute 175.43.224.99
done

- udp is primarily used for ntp, dns, nfs, x11, snmp, etc
  if the service is not used, turn off the ntp/bind/nfsd/X11/snmpd daemons

> Top 10 flows by flows per second for dst IP: 96.43.134.147
>   Duration Proto  Src IP Addr Src Pt Dst Pt  Packets  pps  bps
>248.847 UDP  41.214.2.249123  472078.6 M34594  133.4 M
>248.886 UDP91.208.136.126123  637756.7 M26813  103.4 M
>150.893 UDP  85.118.98.253123  472075.1 M33843  130.5 M

they like to play with ntpd ... make sure your NTPd sw is patched

> Top 10 flows by bits per second for dst IP: 96.43.134.147
>   Duration Proto  Src IP Addr Src Pt Dst Pt  Packets  pps  bps
>  0.002 UDP92.241.8.7553  557520481.0 M  12.4 G
>  0.003 UDP190.184.144.7453  183402048  6826668.3 G
>  0.003 UDP190.109.218.6953  634922048  6826668.3 G

they like to play with DNS ... make sure your bind sw is patched and
properly configured ( not open resolver, etc )

> 
> 
> Copy of the e-mail headers:
> 
> Delivered-To: j...@joesdatacenter.com
> Received: by 10.79.27.84 with SMTP id b81csp1190623ivb;
> Mon, 7 Dec 2015 15:32:22 -0800 (PST)

i assume this ip# is your own local lan ?

> X-Received: by 10.25.88.208 with SMTP id m199mr28948lfb.157.1449531142088;
> Mon, 07 Dec 2015 15:32:22 -0800 (PST)
> Return-Path: 

something tangible to trace/monitor

good luck trying to get bk.ru and their ISP to help resolve the ransom issue

traceroute bk.ru
traceroute mail.ru

traceroute 217.69.141.11
traceroute 95.191.131.93

whois 217.69.141.11
whois 95.191.131.93

politely rattle the security cages of the NOC for each of the ISPs that
is listed in traceroute and especially the IP# owner

> Received: from 

Re: Devices with only USB console port - Need a Console Server Solution

[alvin nanog]

hi erik

On 12/07/15 at 10:15pm, Erik Sundberg wrote:
> We have one of these nice new and fancy Cisco ASR920-24SZ, just realized it 
> doesn't have an RJ45 Console port only USB. When we deploy devices at our pop 
> we wire the console port to a terminal\console server, well that doesn't work 
> for a usb console device.
..
> Does cisco make an USB to RJ45 Jack adapter?

cisco bought linksys long long ago whom, along with dozens of others, make 
USB-ethernet gigE dongle

magic pixie dust
alvin
# DDoS-Mitigator.net

Re: Ransom DDoS attack - need help!

[alvin nanog]

hi ya roland

On 12/04/15 at 11:09am, Roland Dobbins wrote:
> On 4 Dec 2015, at 9:34, alvin nanog wrote:
> >all that tcpdump jibberish
> 
> Is entirely unnecessary, as well as being completely impractical on a
> network of any size.

up to a point, probing around at the packet level is un-necessary depending
on what one is looking for as the end result

> Reasonable network access policies for the entities under attack plus flow
> telemetry collection/analysis, S/RTBH, and/or flowspec are a good start,
> along with this:

flows may address some of the DDoS issues but might not cover all
the various DDoS attacks and mitigation options and still stay within the
victims possibly non-existent DDoS mitigation budgets

> This business of attempting to use packet captures for everything is the
> equivalent of your doctor attempting to diagnose the reason you're running a
> fever by using an electron microscope.

sometimes, one does need to be able to crawl, before walking, before
running track vs running marathons or find someone that can run for you

in the case of ddos mitigation, no one solution can mitigate against all
the possible various attacks... mitigation is a multi-layered solutions

- who-what-when-where-how-why-etc:

- one does need to know what servers, ports and hw is being attacked

  it makes DDoS mitigation a lot easier if you know what is under attack
  and orders of magnitude less expensive to mitigate

- one does need to know who is attacking

  if one cannot defend against low level script kiddie ddos attacks, 
  it's unlikely one will survive a ddos attacks from a more skilled attacker
  determined to take out a server or break in etc

  if you can and have defended against all the basic script kiddie ddos attacks,
  then it might make it easier to find the next set of the various
  ddos mitigation options you need to take 

- one does need to know how often, what time, they are attacking

  if they are attacking after hours, some folks might not care compared
  to they attacking during regular business hours

- one does need to know how much traffic the attacks are costing you
  in terms of time and loss of productivity due to wasted bandwidth

  even at 10% of your bandwidth used up by useless DDoS traffic is still
  noticibly annoying if you were to looking to increase network performance

- nobody can really say why they are attacking, other than are you
  a low level fruit for easy picking or a target'd victim for
  many reasons ( paid ransom before, high profile servers, a bank, 
  govt servers, etc ) .. pay once and all the other DDoS ransom attackers
  will come knocking to collect their share

> Start with the BCPs, then move to the macroanalytical.  Only dip into the
> microanalytical when required, and even then, do so very selectively.

yup... selective and escalate the migitation process and procedure

magix pixie dust
alvin

Re: Ransom DDoS attack - need help!

[alvin nanog]

hi "need help"

On 12/03/15 at 03:15am, halp us wrote:
> A company that shall remain anonymous has received a ransom DDoS note from
> a very well known group that has been in the news lately. 

use an email reader that allows you to see all the received email headers
to see which STMP routers they came thru to reach your smtp servers

contact each of the ISP that owns those IP# ranges to forewarn them of
your upcoming DDoS attacks .. if you're/we're lucky, the actual DDoS
attacks would pass thru the same ISPs again

> Recently they've
> threatened to carry out a major DDoS attack if they are not paid by a
> deadline which is approaching. They've performed an attack of a smaller
> magnitude to prove that they're serious.

cool .. more proof that they can carry out an attacks allows you ( law 
enforcement
and the ISP ) to track down who they are, where they come from, etc, etc, etc

since you also kinda know what time/date they will be attacking, the ISP and
law enforcement can be watching for the incoming attacks reverse track the
originating and probably cracked routers ... and hopefully, one-in-a-million
chance to find the ddos-extorter's computers

if the extorter is in the same city ( your local bully ) using the same ISP, 
finding the extorter should be trivial

you can also catch the extorter by "pretending" to have put up the 
and tell the FBI/interpol/ISPs/PayPal/etc to watch the non-existent account
for incoming connections from the extorter ... and keep telling the
extorter the $$$ is there even if they can't seem to get their $$$

> I would really appreciate help in a few areas (primarily with certain
> provider contacts/intros) so we can execute our strategy (which I can't
> reveal here for obvious reasons).

most folks would like to see that you have done your "homework" too 
trying to stop incoming DDoS attacks ... aka, you need to able to provide 
them the necessary info for them to help you ...

run tcpdump and/or etherreal to capture the DDoS attacks

==

---
ALL servers are under kinda harmless script kiddie attacks every second ...
- defend against those ( free ) ddos attacks scenarios
#
# if you cannot figure out how to stop these harmless probes, you're
# gonna be in trouble when the DDoS attacks are intent on their attacks
#
---

Simple things you should do BEFORE getting outside DDoS mitigation help, 
because they will probably ask and probably perform the same thing:

- prepare a ( time, $$$, technical expertise ) budget to stop that DDoS 
attacks

- get the received headers from the extorter's emails
-

- get the ph# and email contacts of your ISP's security dept and 
their peers/uplinks  .. similarly for the ph# of your local FBI/police 
dept

- at a minimum, update patch all servers to today's patch releases
--

- "confirm" means use the FREE online test tools to test your servers
- confirm your DNS servers are NOT open resolvers
- confirm your SMTP servers are NOT open relays

- use the NTP servers from your ISP if you're not sure if your NTPd is 
secure


---
- install IPtables + tarpit to defend against almost all TCP-based 
attacks
-   imho, it is pointless to run iptables without tarpit support
-   http://NetworkNightmare.net/Tarpits/#Install

---

- defending against UDP attacks requires you get help from your ISP
- usually against DNS, NTP, NFS, SNMP, X11, etc

- defending against ICMP attacks requires you get help from your ISP
  
#
# you cannot stop, block, prevent, mitigate UDP-based or ICMP-based
# ddos attacks at your servers .. 
#
# the ddos attack damage ( wasting your time, $$$ and bandwidth ) 
# is already done if it reaches your servers
#

- backup your user ( /home, /etc ) data ...
- build a brand new server from latest distro and restore your data 
from backup

- if you don't have time for all this DDoS stuff and willing to do only 1 
thing,
  install and learn iptables with tarpits on all your servers exposed to the 
internet

- it's trivial or NOT trivial depending on your abilities
- it is trivial ( few minutes/hours work ) for those folks familiar 
with IPtables

http://IPtables-BlackList.net

- if you do decide to go with outside DDoS scrubbers, you definitely will need 
$$$

if you don't have the time but have the $$$, hire a couple different DDoS 

Re: Ransom DDoS attack - need help!

[alvin nanog]

hi lyndon

On 12/03/15 at 05:54pm, Lyndon Nerenberg wrote:
> On Dec 3, 2015, at 5:00 PM, alvin nanog <nano...@mail.ddos-mitigator.net> 
> wrote:
> > run tcpdump and/or etherreal to capture the DDoS attacks
>
>  Of course! If we had only thought of this sooner! 
> :-)

yupperz.. the problem is, capturing is nice, you have all this data ... now 
what ,,

all that tcpdump jibberish needs to be converted and presented in a format
suitable for the bean counters to allocate $$$ to mitigate and minimize the
effects of the "free n hopefully relatively harmless" DDoS attacks occuring
every second

lets assume required services are properly configured and excluded
- acl's only for your own dns queries
- ssh only from specific ip#
- ntp to/from your isp

lets assume you allow incoming ssh only from w.x.y.z ... all other connections 
are DoS attacks
  tcpdump -n -l ! host w.x.y.z and port 22

lets assume mail is your mail server .. all traffic NOT on port 25 are DoS 
attacks
  tcpdump -n -l host mail.example.com and ! port 25

lets assume www is your web server .. all traffic NOT on port 80 are DoS attacks
  tcpdump -n -l host mail.example.com and ! port 80

if you are running all the services ( mail + apache + mysql ) on one servr
the remaining tcp connections are DoS attacks
  tcpdump -n -l host mail.example.com and \( ! port 80 and ! port 80 and ! port 
3306 \)

lets assume dns is your dns server .. i consider all tcp traffic from outside 
as DoS attacks
  tcpdump -n -l tcp host dns.example.com

to see possible udp attacks .. don't forget to exclude your own DNS and NTP 
queries
  tcpdump -n -l udp

to see possible icmp attacks
  tcpdump -n -l icmp

too many gazillions options makes the world go round n round ...
- where does it end :-) ... it doesn't ...

if you get a screenful of data flying by of stuff you don't recognize,
you're probably under light DDoS attacks

magic pixie dust
alvin
http://DDoS-Mitigator.net/cgi-bin/IPtables-GUI.pl

Re: Bluehost.com

[alvin nanog]

hi

On 11/25/15 at 05:19pm, Bob Evans wrote:
> For an ISP type service - it's almost impossible the make it up in volume
> - all you need is one phone call to cost you $10 in support on a $3.50
> service. With that many customers you can imagine how many call to just
> ask what happened or vent after the event is over.
 
a painful reality ... support costs are NOT cheap if one is trying
to keep customers happy 

more customers usually requires more support expenses too and hopeully,
support expenses would start to go down after some critical levels

> I founded a cable modem business prior to docsis standard.

congrats..

> What do they call it when one keeps
> doing the same thing over and over again expecting a different result ?

"the internet"
"there's NO sheriff in town"
"there's a (new) sucker born every second"
"dumb money"
"tax deductions - tax write offs"
"i wanna get involved, me too syndrome"
...

> Low priced services are difficult to make profitable 

- pricing strategy vs customer volume is always a tradeoff
- one can always give well behaved customers their discounts from "normal 
pricing"
- one cannot give "good service" when starting from "lowest possible pricing"

magic pixie dust on dah turkey
alvin

Re: Updated Ookla Speedtest Server Requirements

[alvin nanog]

hi lorell

On 11/09/15 at 03:27pm, Lorell Hathcock wrote:
> Esteemed Legions of NANOG:
> ...
> Here is the link to their recommendations.
> http://www.ookla.com/support/a26461638/
...

> In the mean time I will just test 1 Gbps speeds off a copper GE port, but
> want the SFP+ capability so I don't have to repurchase hardware in the next
> year.

also in the meantime, while waiting for the fiber stuff to be built out,
you could use a $400 copper-based 10gigE pci card too

magix pixie dust
alvin
http://DDoS-Mitigator.net 

Re: Updated Ookla Speedtest Server Requirements

[alvin nanog]

On 11/09/15 at 05:35pm, Josh Luthman wrote:
> You can't get SFP+ PCI cards anyways.  I don't think you can very easily
> get boards with PCI on them, either.

lots of vendors w/ PCIe 10gigE cards w/ copper or SFP+ interface .. 

very few ( almost none ) for explicitly PCI w/ 10gigE and more importantly

i usually get intel chipset based SFP+ PCI-e x8 based dual-10gigE cards
similarly, for dual 10gigE copper for testing on gigE infrastructcure

not many "tyan/supermicro" motherboards with dual-gigE SFP+ connectors
( our primary requirement is dual-nic 10gigE ports squeezed into 1U chassis )

yes, it could be slow ... and most likely not even run at "bit speeds"
quoted by marketing, but not everybody goes around measuring bandwidth speeds

similarly, not much marketing anymore for 3.xGhz or 4.xGhz or 6.xGhz Ghz 
"cpu speeds" and lately, past decade, how many cores can you squeeze into 1sq 
inch
which is trivially easy to count and measure cores and it's performance :-)

PCIe-3.x spec'd at 8Gbit/s  # x8 would easily give you 40Gbit/s
PCIe-2.x spec'd at 4Gbit/s
PCIe-1.x spec'd at 2Gbit/s

PCIe x1 == 2Gbit/s ( per lane )
PCIe x4 == "combined" 8Gbit/s 
PCIe x8 == "combined" 16Gbit/s  ( common PCIe cards )
PCIe x16 == "combined" 32Gbit/s
PCIe x32 == "combined" 64Gbit/s

> I expect he was just typing it out and left the "E" =)

just being brain-dead lazy w/ assumptions and PCI vs PCIe does make a
big difference with network thruput

magic pixie dust
alvin
# Custom 1U chassis w/ zero drives and up to 8 drives


- End forwarded message -

Re: Long-haul 100Mbps EPL circuit throughput issue

[alvin nanog]

hi eric

On 11/05/15 at 04:48pm, Eric Dugas wrote:
...
> Linux test machine in customer's VRF <-> SRX100 <-> Carrier CPE (Cisco
> 2960G) <-> Carrier's MPLS network <-> NNI - MX80 <-> Our MPLS network <->
> Terminating edge - MX80 <-> Distribution switch - EX3300 <-> Linux test
> machine in customer's VRF
> 
> We can full the link in UDP traffic with iperf but with TCP, we can reach
> 80-90% and then the traffic drops to 50% and slowly increase up to 90%.
 
if i was involved with these tests, i'd start looking for "not enough tcp send 
and tcp receive buffers"

for flooding at 100Mbit/s, you'd need about 12MB buffers ... 

udp does NOT care too much about dropped data due to the buffers,
but tcp cares about "not enough buffers" .. somebody resend packet# 1357902456 
:-)

at least double or triple the buffers needed to compensate for all kinds of 
network whackyness: 
data in transit, misconfigured hardware-in-the-path, misconfigured iperfs, 
misconfigured kernels, interrupt handing, etc, etc

- how many "iperf flows" are you also running ??
- running dozen's or 100's of them does affect thruput too

- does the same thing happen with socat ??

- if iperf and socat agree with network thruput, it's the hw somewhere

- slowly increasing thruput doesn't make sense to me ... it sounds like 
something is cacheing some of the data

magic pixie dust
alvin

> Any one have dealt with this kind of problem in the past? We've tested by
> forcing ports to 100-FD at both ends, policing the circuit on our side,
> called the carrier and escalated to L2/L3 support. They tried to also
> police the circuit but as far as I know, they didn't modify anything else.
> I've told our support to make them look for underrun errors on their Cisco
> switch and they can see some. They're pretty much in the same boat as us
> and they're not sure where to look at.
> 

Re: wanted: tool for traffic generation / characteristics / monitoring

[alvin nanog]

hi matthias

On 10/01/15 at 03:41pm, Matthias Flittner wrote:
> Dear colleagues,
> 
> Currently we are looking for a magic tool with which it is possible to
> generate specific (realistic) traffic patterns between client and server
> to analyze (monitor) traffic characteristics (jitter, delay, inter
> arrival times, etc.).

generating traffic and monitoring traffic is usually not done
by the same apps  there's hundreds of monitoring apps
and hundreds of traffic generators

delay is done very nicely by dummynet in FreeBSD or 
(untested by me ) with NS3 in linux 

i don't understand simulating jitter, but, one can always use 
"delay + random number" 

> It would be good if that wanted tool is not only able to generate
> different traffic patterns

if you want to play with the headers ... that'd imply playing with
nmap/hping3/socat and dozens of other equivalent apps

if you're just trying to flood the wire ... nc/socat/iperf etc

> but also is able to collect different traffic
> metrics over time. So that it is possible to create catchy plots. :)

"what metrics" you want to collect and how to you want to see it
would dictate which apps you'd be using
- tcp queue/buffers
- dropped packets
- delays
- retries
- udp vs tcp vs icmp vs ...
- stuff ...

xmit/recv buffers in the hardware, default buffers in the OS and 
buffers in the software apps must all be tuned to the same gigE 
or 10gigE speeds otherwise, whacky stuff will happen

for "catchy plots", you'd want gnuplot so you can (infinitely) zoom in 
into the section you want to see dot-by-dot

for big picture ... netstat, ntop, (not much info) mrtg, etc, etc

big list of apps
Packet-Craft.net/Apps

> Any hints or links would be greatly appreciated.

if you're a proficient python'er, you'd probably like scapy
which can do everything you'd need to customize any packet

magic pixie dust
alvin
#
# Packet-Craft.net/Apps
#

Re: CHP website returning 503

[alvin nanog]

On 09/28/15 at 12:56am, Larry Sheldon wrote:
> On 9/28/2015 00:24, Christopher Morrow wrote:
> >On Mon, Sep 28, 2015 at 12:42 AM,   wrote:
...
> >Are telling me Eric Estrada won't have a loadbalancer deployed for
> >this super critical resource?

both eric and his buddy was distracted by the blondes on the sunny beaches

> I find the cavalier, screw-en attitude instructive.

i've had the worst luck with "cavalier(?) disks" from western digital

> Does anybody know (I didn't ask "care", I can see that) what the function of
> the site is?  What citizen or patrolman services have been lost?

traffic report is "511" on the cell phone ... usually up to date
within the past hour

i think nothing "important" ( need something now ) is lost during 
their website outage ?

i wonder if they use their laptops in the car to submit/file reports
via their websites or just wait till they get into the office.
some cops are still using pencil and paper to write down reports

gov't are notoriously wasteful for their budget to get the
simplest tasks done. 10+ managers and supervisors and past
retired employees in the past 60yrs on pension need their 
salary/pension while 1 new college grad actually gets the tasks done

---

in dealing with cops ... they've usually used their personal cell phones
to make phone calls/inquiries regarding the issues at hand ... 
kinda wierd ... even more cell phone usage during the overhaul of
their outdated radios used by police, fire, ambulance, etc etc

magic pixie dust
alvin

Re: CHP website returning 503

[alvin nanog]

On 09/27/15 at 09:21pm, Joe Hamelin wrote:
> It is late Sunday night.  When would you do maintenance?

even if one was doing maintenance, there is no reason not
to have at least 1 el-cheapo server replying that it's 
under maintenance vs being suspect of other reasons of it
being down

there's gotta be more than 1 server for chp.ca.gov

- do maintenance on the other servers and test before going live
- remove the "maintenance window notification server"

magic pixie dust
alvin

> On Sun, Sep 27, 2015 at 7:50 PM, Grant Ridder 
> wrote:
> 
> > Hey,
> >
> > If anyone from CHP (california highway patrol) is listening, your website
> > is returning a 503.

Re: correlation between ingress and egress traffic in case of volume-based DDoS

[alvin nanog]

hi martin

On 09/23/15 at 07:07pm, Martin T wrote:
> volume-based DDoS attacks should often result with following bandwidth graphs:
> 
> http://s12.postimg.org/gy3eps10t/volume_based_DDo_S_graph.png
> 
> 
> This is a fabricated bps graph for 100GigE port facing an uplink

when you say "fabricated" ... what do you mean ??
- if its actual ( real ) .. than okay for the reasoning
- if its "fabricated" ( sanitized, made up, theory ), than still okay for 
reasoning

as roland says, there's dozens of reasons to explain the results
one sees in graphs  graphs are always subject to different
interpretation depending on different circumstances

> provider. As seen on the image, outgoing traffic drops at the time
> when incoming traffic increases. I could see following reasons for
> this:
> 
> 1) large portion of traffic uses TCP protocol and in case of

in my case, the way i would simulate a tcp-based DDoS attack against
a test victim, outgoing traffic should remain steady state per
the normal tcp traffic ... and tarpit the ddos attacks from 
the victim under the volumetric tcp-based attacks
setup the ingress filters for tarpits

the attacking zombie should run out of kernel memory and 
crash if it tries to sustain huge amt of volumetric tcp-based
ddos attacks

without tarpits, all kinds of whacky network stuff will occur
depending on how the tcp-based ddos attacks are lauached

if its an icmp-based ddos attack, the outgoing traffic may be 
10x or 100x more traffic than incoming icmp-based ddos attack
if the server was not hardened to protect itself from icmp-based attacks
if it was hardened, limit outgoing reply to incoming 
icmp-attacks, than it outgoing reply should be constant
due to normal remaining traffic, but incoming icmp attacks 
can be 10x or 100x or 1000x normal

similarly, if its an udp-based ddos attack, the outgoing traffic
may be 10x or 100x more than the incoming traffic if dns,ntp,snmp,
x11,nfs,etc was not hardened to protect itself from udp-based attacks
if it was hardened, limit outgoing reply to incoming 
udp-attacks, than it outgoing reply should be constant
due to normal remaining traffic, but incoming udp attacks 
can be 10x or 100x or 1000x normal

if its an arp-based ddos attack, outgoing traffic may generate 
the same amt of traffic going out as whats coming in .. hopefully,
you have bought good switches that don't fail-over into hub-mode

launching those volumetric attacks takes a few minutes to figure
out what options to use to create certain type of ddos attacks 
- nc, socat, iperf, etc
- ping, nping, hping, etc
- hundreds of other apps

> congestion(even in one direction), ACK messages are lost and TCP
> congestion avoidance kicks in and as a result it will reduce the cwnd
> which in effect reduce the data TCP sender can send

syn-cookies doesnt kick in until all tcp stack is exhausted
and syn-cookies tries to service all incoming tcp requests

probably a bad thing to attempt to do while under tcp-based ddos attacks

fiddling with send and receive buffers and delays and timeout would add
more fun to the problem and resulting bandwidth graphs

> 2) certain router platforms share some hardware resources both with Tx
> and Rx traffic
> 
> Are those assumptions correct? Are there any other reasons which cause
> outgoing traffic to drop if incoming traffic is very high or the other
> way around?

in the subject, you used "ingress and egress" filters ...
those rules would also definitively affect the resulting bandwidth graphs

if you're in the cloud ... all these thingies still apply to the
guest OS and especially the host OS

magic pixie dust
alvin
#
# DDoS-Mitigator.com
# DDoS-Simulator.net
#

Re: DDoS auto-mitigation best practices (for eyeball networks)

[alvin nanog]

On 09/19/15 at 02:54pm, Frank Bulk wrote:
> Could the community share some DDoS auto-mitigation best practices for
> eyeball networks, where the target is a residential broadband subscriber?

o kie dough kie

> I'm not asking so much about the customer communication as much as
> configuration of any thresholds or settings, such as:
> - minimum traffic volume before responding (for volumetric attacks)

i prefer zero tolerance ...

i tarpit all incoming tco-based attacks and probes that was
not allowed incoming tcp traffic to port 25 or port 80 or port blah

example iptables rules ... linux and iptables  + tarpits is free
# IPtables-BlackList.net/Howto
- ingress filters
- allow established 
- check for blacklist
- limit udp and icmp reply ( tough problem to solve )
- allow to port 80 ( keep webserver separate from dns, smtp, etc )
- tarpit all new tcp incoming connections
- drop all other new incoming connections

- there is no need log millions of ddos attack pacekets
per second unless you want to fill up your disk which
helps the ddos attacks to be a successful attack

- for icmp and udp ... you will need your ISP to help block it

  limiting incoming icmp/udp is sorta pointless since those
  packets already come down the wire

  however, you still do NOT want to respond to those packets either
  so you will have to limit to just a handful per second, little more
  per hour, and higher limit per day

  for icmp ... turn off broadcast ping responses on all devices

  for udp ... make sure the apps are properly configured
dns, snmp, ntp, nfs, x11, etc uses udp

your dns servers might need to be accessible from outside
all other udp-based servers should be internal only

- to protect against arp-based attacks  
build/patch/configure your hardware/routers/switches properly

- install monitoring tools to watch for whatever you're paranoid about
- man-in-the-middle .. trivial to detect and prevent
- sniffers ( hard to detect )

> - minimum time to wait before responding

zero wait ...

> - filter percentage: 100% of the traffic toward target (or if volumetric,
> just a certain percentage)?

you will always, 100% fail volumetric attacks 

> - time before mitigation is automatically removed

you can have iptables remove a particular ddos attacker automatically
or manually

i prefer manually so i can see what it's doing

> - and if the attack should recur shortly thereafter, time to respond and
> remove again

zero wait time  .. zero tolerance per example iptables rules above

> - use of an upstream provider(s) mitigation services versus one's own
> mitigation tools

i haven't found too many ISP willing to allow customers to put
a customer firewall in their facility just before it comes down to 
the wire to customers bldg

this is required if customers want to properly mitigate icmp-based 
and udp-based ddos attacks

> - network placement of mitigation (presumably upstream as possible)
> - and anything else

mitigation solutions should be a gateway firewall and host-based mitigation

if you can install another firewall at the ISP, thats good too
and you still need a gateway firewall and the host based firewall

> I ask about best practice for broadband subscribers on eyeball networks
> because it's different environment than data center and hosting environments
> or when one's network is being used to DDoS a target.

add corp environment, hospitality environment, govt environment, 
etc etc to the list too
- free wifi, hotel based wifi or hardwire is probably
the easiest way to send the unsuspecting victim home 
with a trojan that will phone home ( the attacker )
when the victim plugs the cracked box into the secure
corp network

nah ddos attacks are ddos attacks ... usually harmless ...
it probably doesn't matter to the attackers what they're attacking 

you are constantly under 24x7x365 low level ddos attacks

if you are being targeted by somebody that wants to get you,
you'd have a problem if they're better at attacking than you 
are at defending your servers ...

they're done if they have a bigger budget to pay for all the 
necessary bandwidth needed to take your servers offline

  - if you know who they are, call the ISP and the cops

-

other "basic best practices"
- have a good security policy ... even if just for yourself
hide the laptop in your trunk using a brown bag
and NOT an obvious laptop bag 
- always use encrypted services... never clear text
- use ssh, openssl, smtps, pop3s, imap3s 

- dozens of other best practices security rules
- always have a incremental daily backup that is kept
for months
- always have a hot swap backup just in case
 etc .. etc ...



you should also keep track of who is attacking your servers
so that law enforcement can 

Re: high latency on West Coast?

[alvin nanog]

hi andrei

On 09/18/15 at 11:50am, Florin Andrei wrote:
> I'm seeing 250 ms between California and Oregon. Not just AWS, but also
> between, say, Comcast and AWS.
> 
> Latency from other locations, such as between N. Virginia and Oregon, is
> much lower, about 72 ms in my tests.
> 
> Anyone else experiencing these issues along the west coast?
 
amazon seems confused

from Reno# traceroute 205.251.230.127
-
traceroute to 205.251.230.127 (205.251.230.127), 30 hops max, 60 byte packets
 1  ... deleted ...
 2  * * *
 3  rno-core0-ge.greatbasin.net (207.228.35.136)  10.863 ms  12.237 ms  13.611 
ms
 4  67.138.231.149 (67.138.231.149)  14.963 ms  16.372 ms  17.719 ms
 5  paix01-sfo4.amazon.com (198.32.176.36)  21.914 ms  23.259 ms  26.048 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *

Re: root zone archive

[alvin nanog]

hi

On 09/17/15 at 12:33am, Joe Abley wrote:
...
> I'm particularly interested in zone data that describes the build out of the
> original root zone NS set to nine servers in mid-1994, the renaming under
> the ROOT-SERVERS.NET domain and the subsequent assignment of J, K, L and M.

wouldn't that info be in the files included with bind-0.x 

have fun
alvin

Re: Any Tool to replace Peakflow CP

[alvin nanog]

hi pavel

On 09/06/15 at 06:11pm, Pavel Odintsov wrote:
> Thanks for recommendation, Alvin!
 
just "on the list of flow stuff to look at"
- opensource like fastnetmon is good for techies and solving problems
- commercial products may be what large corp purchasing folks like

i've looked into the flow based products and got more confused :-)

> As author of FastNetMon I will be very glad to hear some feedback
> about my tool and could help with configuration / development :)

cool ...

i went googlin around and found some additional info for the url list

> On Sun, Sep 6, 2015 at 6:22 AM, alvin nanog
> <nano...@mail.ddos-mitigator.net> wrote:
> >
> > hi aluisio
> >
> > On 09/06/15 at 02:01am, Aluisio da Silva wrote:
> >> Hello,
> >>
> >> Does anyone here have a suggestion for a tool to replace Peakflow CP from 
> >> Arbor Networks?
> >
> > # for reference
> > http://www.arbornetworks.com/products
> >
> >> Please if possible you would like hear some suggestions.
> >
> > - sflow based
- sflow based # trademark owned by inmon 
http://www.sflow.org/products  == list of vendors and collectors 
> > http://www.sflow.com/products/floodprotect.php

their blog.sflow.com has lots of feedback and comparisons

> > http://www.inmon.com/technology/sflowTools.php
http://www.inmon.com/products
> >
> > http://www.andrisoft.com/software/wanguard
> > http://www.github.com/FastVPSEestiOu/fastnetmon
> > http://www.packetdam.com
> > http://www.radware.com/Products/DefenseFlow
> >
- netflow based -- netflow prototcol superceded by IPFIX
http://www.cisco.com/go/netflow
 
http://www.openbsd.org/cgi-bin/man.cgi?query=pflow=4=OpenBSD+Current

one day, i'll go poke around at linux-based tools like openbsd's pflow

> >
> > http://nfdump.sourceforge.net
> > http://nfsen.sourceforge.net
> > http://sourceforge.net/projects/panoptis

- openflow based
http://www.opennetworking.org/sdn-resources/openflow
http://www.radware.com/Products/DefenseFlow

> > - jflow based
juniper.net/us/en/products-services/security/
- still can't find the jflow info :-(

magic pixie dust
alvin
#
# DDoS-Mitigator.com/Competitors/#Flow
#

Re: Any Tool to replace Peakflow CP

[alvin nanog]

hi aluisio

On 09/06/15 at 02:01am, Aluisio da Silva wrote:
> Hello,
> 
> Does anyone here have a suggestion for a tool to replace Peakflow CP from 
> Arbor Networks?

# for reference
http://www.arbornetworks.com/products

> Please if possible you would like hear some suggestions.

- sflow based
http://www.sflow.com/products/floodprotect.php
http://www.inmon.com/technology/sflowTools.php

http://www.andrisoft.com/software/wanguard
http://www.github.com/FastVPSEestiOu/fastnetmon
http://www.packetdam.com
http://www.radware.com/Products/DefenseFlow

- netflow based
?cisco url?

http://nfdump.sourceforge.net
http://nfsen.sourceforge.net
http://sourceforge.net/projects/panoptis

- jflow based
?juniper?

magic pixie dust
alvin
#
# DDoS-Mitigator.com
#

> Thanks.
> 
> Aluísio da Silva
> Coordenação de Planejamento e Engenharia
> CTBC
> (34) 3256-2471
> (34) 9976-0471
> www.ctbc.com.br

Re: Software Defined Networking

[alvin nanog]

hi valdis

On 09/04/15 at 06:59pm, valdis.kletni...@vt.edu wrote:
> 
> Does anybody have a citation that legal disclaimers attached to
> publicly posted mail aren't null and void?  Seems to me that
> what they're trying to say is "Sorry, we're too lame to use
> PGP or similar on actually sensitive e-mail"...

i keep wondering why "they" keep using sniffable clear text 
smtp/imap/pop3 instead of at least encrypted version

the problem also is both ends, the sender and the receiver
and all the laptops/desktops need to be configured

more importantly, why not just use https based webmail
or even smpts encrypted google mail where less setup and
configuring would be needed for sender and receiver

have a nice weekend
alvin
#
# DDoS-Mitigator.com
#

Re: DDoS appliances reviews needed

[alvin nanog]

hi ya ramy

- ddos mitigation is NOT one appliance or cloud scrubber ...
  you'd require multiple layers of different ddos mitigation methodologies

On 08/27/15 at 08:22am, Ramy Hashish wrote:
 Thank you Alvin, I have just remembered that I wanted to reply to your 
 previous
 input on Wanguard versus the other vendors in the market, I will reply this
 there.
 
per your comments on the other posts, its okay to disagree etc, etc ...
everybody has their views .. maybe i wasnt clear enough with what i was saying 
too

 I can't get exactly what you are doing, do you have your own mitigation SW? If
 so I would like to know more about it.

you can download the free version for testing ..
http://DDoS-Mitigator.net/Download

 On Wed, Aug 26, 2015 at 8:53 PM, alvin nanog nano...@mail.ddos-mitigator.net
 wrote:
...
 for your reviewing or collecing info from folks ..
 - what's your metrics that is important to you ?
 
 Our important metrics includes but not limited to the following:
 
 - Ability to mitigate all kinds of volumetric DDoS attacks.

mitigating all kinds of volumetric DDoS attacks means you'd probably fail
most of the time ...

the (simulated) ddos attackers can always, 100% of the time, generate more 
traffic that you want to defend against in hw or $$$ or time or staff ...

i say, first defend against all the current DDoS attacks you are getting
every minute of every day  that'd already take lots and lots of time 
and $$$ and resources as is ... and volumizing it, to 1,000,000 packets/sec 
or more 10M or 100M packets/sec doesn't solve anything ...

imho, the only way to defend against any volumetric ddos attacks that 
exceed your bandwidth capacity is to buy more capacity from the ISP
or from (more expensive) DDoS cloud scrubbers

 - Ability to mitigate application level attacks for at least HTTP, HTTPs, SMTP
 and DNS.

it should be trivial to defend against incoming http/https/smtp ddos attacks
- the so called 10 minute problem

defending against DNS is almost equally trivial 
- 53/udp is used for dns queries ...
- 53/tcp is used for zone transfers between primary and secondary DNS 
servers

thus, all incoming  tcp packets to a DNS server are DDoS attacks
except your own primary and secondary dns server ip#

if UDP attacks against DNS servers exceed your bandwidth, again, you have to 
buy more bandwidth or have the ISP or the ddos scrubber cloud drop it for you

- limiting replies to incoming udp or icmp is useless since it already
used your bandwidth, cpu, memory, disk and the expensive staff's time
 
- we're all assuming your DNS server is closed for recursive queries
to prevent DNS amplification attacks ... 

- similary fix/patch NTP servers and ntp clients

- if the ISP doesn't want to help defend against incoming UDP/ICMP
attacks, than you're kinda screwed ... time to find a new ISP

similarly, always turn off replies to ping broadcast from the outside to 
prevent smurf'ing yourself

 - Time-to-detect and time-to-mitigate.

ddos mitigation should be automated ... people cannot watch and defend the
servers watching millions of packets per second flowing thru the servers

more importantlty, if people are looking at the packets and if you're
sending/receiving confidential data, do you really want 3rd party eyeballs
looking at all your packets, and regulations say they should be certified
eyeballs and certified colo facilities too

 - False positives.

hard to avoid which requires careful planning and lots of testing

 - Response time to the management plan.

why would management dictate ddos mitigation policy vs IT security folks 

 - Ability to sniff packets for further analysis with the support.

too much work ... you have million or gazillion ddos packets per second 
to look at and you will NOT be able to see the attack from the legitimate 
packets or more importantly, might not care anymore ...

 - Granularity of detection thresholds.

seem arbitrary to hit some threshold ...

either it IS a ddos attack or it's NOT ... it should be fairly clear

 - Percentage of DDoS attack leakage.

i dont understand the leakage

 - Multitenancy (We are an ISP)
 
good   the customers can help you determine legit traffic from
DDoS traffic

 - Fast to detect/mitigate appliance, no problem to work inline.
 
as is always the case, anytime you have a server inline, be sure
they are crossed connected so that the other server can take over
in case of hw failure

 my requirement: all tcp-based ddos attacks must be tarpit'd ... 
...
 Could you please give more details on this?

tarpit holds any incoming tcp-based connection attempt open for say 2minutes
during which time the attacking server is stuck 

if the zombie ddos attacker sends 100,000 tcp packets/sec against
your webserver or mail server, they'd have to have a lots of kernel memory

( 100,000 packet/sec * 1500byte/packet

Re: DDoS appliances reviews needed

[alvin nanog]

hi ramy

On 08/26/15 at 12:54pm, Aftab Siddiqui wrote:
 
  Anybody here has experienced a PoC for any anti DDoS appliance, or already
  using a anti DDoS appliance in production and able to share his user
  experience/review?
 
 
 only interested in appliance? why not scrubbing services? is it for own use
 (industry reviews before purchase) or some article/publication/research?

see previous similar thread for some real world reviews by folks

http://mailman.nanog.org/pipermail/nanog/2015-April/074410.html

i think a benchmarking ddos lab would be fun to build and publish findings..
to test all the ddos appliances from those competitors willing to participate

---

for your reviewing or collecing info from folks ..
- what's your metrics that is important to you ?
- what (ddos) problems are you trying to resolve ?

- do you want to see the ddos attacks in progress and how you're being attacked
http://ddos-mitigator.net/cgi-bin/IPtables-GUI.pl

- do you want 100% automated ddos defense with zero false positives :-)

my $0.02 ddos experiences n summary over the years, aka mitigation in 
production use ...

usually, arp-based ddos attacks requires fixing your infrastructure, 
  a ddos appliance may not help you

usually, udp and icmp ddos attacks can only be resolved by the ISP or scrubbing 
centers
- if you limit udp/icmp at your appliance, the damage is already done,
since those packets used your bandwidth, cpu, memory, diskspace and 
your time

spoof'd source addresses can only be resolved by having the ISP preventing 
outgoing
spoofed address ( fix egress filters ) at their edge routers

my requirement: all tcp-based ddos attacks must be tarpit'd ... ddos attacks
are now 1% of it's peak a few years ago where firefox google.com wouldn't 
come up

- you must be able to distinguish legit tcp traffic from ddos attacks
which is ez if you build/install/configure the servers properly

i want the attacking zombies and script kiddies to pay a penalty for 
attacking my customer's servers

to sustain a 100,000 tcp packets attack requires lots of kernel memory 
( 100,000 packets * 1500 byte/packet * 120 seconds ) for 2minute tcp 
timeouts 

there are 65,535 tcp they could be attacking ... imho, an ssh-based 
solution
or apache-based solution would be useless ... add another 65,535 udp 
ports

always keep your servers up to date ... patch your OS, apps, etc, etc

volumetric attacks can only be resolved by (expensive) ddos scrubbers or 
installing 
your own geographcially separated colo in usa, europe, asia like the scrubbers 
... 
if you are high profile target, the ddos attackers probably has more bandwidth 
than 
you could afford and the ddos attacks will probably make the evening news

magic pixie dust
alvin
# DDoS-Mitigator.net/Competitors
# DDoS-Mitigator.net/InHouse-vs-Cloud
# DDoS-Simulator.net
#

Re: A multi-tenant firewall for an MSSP

[alvin nanog]

hi 

 On Mon, Aug 17, 2015 at 10:16 AM, Ramy Hashish ramy.ihash...@gmail.com
 wrote:
 
 We are planning to implement a multi-tenant FW/UTM and start providing
 security as a service, I would like to hear if anybody had experience on

that'd be a good thing ... but ...

 this, and if there are any recommendations for the UTM's vendor.

the possible vendors would depend on the answers to your idea of
what is well rounded solution

# fortinet's (possible) competitors
http://ddos-Mitigator.net/Competitors

 People/customers here are more familiar with the Fortigate, however, we
 need to build a well-rounded solution that suits wide range of enterprises'
 business needs.

# i doubt there is one product that provides the well rounded solution

in my world, well rounded solution would imply at least the following:
- anti virus solution  ( one or more products to resolve the virus issue )
- anti spam solution  ( one or more products to resolve the spam issue )
- iptables with tarpit ( protect against the free tcp-based script kiddies 
tests )
- udp limiting at isp ( part of iptables or your edge routers )
- icmp limiting at isp ( part of iptables or your edge routers )
- ingress/egress filters for your downlinks
- geographically distributed colo to mitigate small/medium sized ddos attacks
- regulatory compliance this and certified that vs just anybody ...
- good response time to fix problems reported by competent customer's IT folks
- other things you deem important to provide ..

pixie dust
alvin
#
# ddos-Mitigator.net
# ddos-Simulator.net

Re: Data Center operations mail list?

[alvin nanog]

hi ya martin

On 08/16/15 at 07:57pm, Martin Hannigan wrote:
 On Sun, Aug 16, 2015 at 3:22 PM, Chris Boyd cb...@gizmopartners.com wrote:
 
  There seems to be some traction, with 268 members on the NADCOG list so
  far.

good

 Great! It's a little more complicated than list member count. Consider a
 social media presence as well? While the Facebook isn't designed as mailing
 list or collaboration media, it sort of works communication wise. If there

what kind of social media presence is what keep bouncing off my empty head

i can't stand fb/twitter/other-selfie-stuff/utube/etc
( aka too many fake emails from people i dont know pretending to know me )

 were some service like /. for mailing lists (that I knew about) I would
 really like the community thumbs up/thumbs down approach so you can sift

/. seems to have tied itself with fb ... you'd need a major following
before /. accepts and publish your submissions

any comments on the meetup.com style ??? ( ?? too big ?? too broad ?? )

one can always create the media/methodology that one wants ??

 through quality vs. quantity. The quality issue has been discussed for
 literally years.

quality/quantify for dc content seems okay to start things rolling

magic pixie dust
alvin
#
# Sushi-Critics.net ... social stuff... if you're visiting in silicon valley 
# Food-Critics.net  social stuff... sillicon beach, reno/lasvegas lemme know
#
# ITX-Blades.net ... 10 mini-itx blades in 4U chassis ...
# DDoS-Mitigator.net ..
#

Re: Experience on Wanguard for 'anti' DDOS solutions

[alvin nanog]

hi ramy

On 08/12/15 at 05:28pm, Ramy Hashish wrote:

 Anybody here compared Wanguard's performance with the DDoS vendors in the
 market (Arbor, Radware, NSFocus, A10, RioRey, Staminus, F5 ..)?
 
wouldn't the above comparison be kinda funky comparing software solutions
with hardware appliances and/or cloud scubbers ??

comparisons between vendors should be between sw solutions,
or hw appliances vs other hw, or cloud vs other clouds

wanguard should be compared with other sw options or vendors using
sflow, netflow, jflow, etc etc
http://www.andrisoft.com/software/wanguard
http://bitbucket.org/tortoiselabs/ddosmon
http://www.github.com/FastVPSEestiOu/fastnetmon
http://nfdump.sourceforge.net
http://nfsen.sourceforge.net

wanguard - software solution using sflow
http://www.andrisoft.com/software/wanguard

arbor  hardware/software solutions -- peakflow 
http://www.arbornetworks.com/products/peakflow

radware -- hardware/software/cloud solutions -- defenseflow
http://www.radware.com/products/attack-mitigation-service/
http://www.radware.com/Products/DefenseFlow/

nsfocus -- hardware/cloud solutions
http://www.nsfocus.com/products/

A10 -- hardware solution
http://www.a10network.com/products

riorey --- hardware solution
http://www.riorey.com/riorey-ddos-products

staminus - hardware/cloud solutions
http://www.staminus.net/shield

# and to add to the ddos confusion ..

akamai/prolexic --- hardware/cloud solution

f5  hardware/cloud solutions

http://www.f5.com/resources/white-papers/mitigating-ddos-attacks-with-f5-technology

fortinet -- custom ASIC hardware and cloud solution

http://www.fortinet.com/products/fortiddos/ddos-mitigation-appliances.html

- simulated ddos attacks should include:
  ==
  == you are already getting hourly low level DDoS attacks from your script 
kiddies
  ==try to defend against those mostly harmless attacks first
  ==
#
# some trivial benchmark DDoS attacks to generate -- internally only
#   - never send DDoS packets outside of your bldg/gateway
#
# DDoS-Simulator.net == generate any DDoS packets per your desires
#   - use nc, socat, *perf, nping or hping to generate most of these DDoS 
attacks 
#   - use dsniff/arpspoof to break everything
#
within your own network, send few packets per second attacks
within your own network, send x,000 packets per second attacks
within your own network, send xxx,000 packet per second attacks 
sustained sporadically over hours/days

- arp-based attacks
- udp-based attacks
nping -v -d1 -c 1 --data-length 1511 --rate 12345 --udp 127.0.0.1 
hping -c 1 -d 1511 -i u 81 --rand-source -p 123 -S --udp -p 123 
127.0.0.1 

- icmp-based attacks
ping -c 1 -s 1511 -i 0.8  127.0.0.1 
nping -v -d1 -c 1 --data-length 1511 --rate 12345 --icmp 127.0.0.1 
hping -c 1 -d 1501 --rand-source --file TeraByteFile.bin --icmp 
127.0.0.1 
gazillionPingApps

- tcp-based attacks --- ez to send malicious packets and to defend against
# 10,000 random src add
hping -c 1 -d 1511 -i u 81 --rand-source -xxTCPflags 127.0.0.1 
# -S = set SYN flag
# -F = set FIN flag
# -A = set ACK flag

- application layer tests --- http, ssh, mail and 65,532 other ports
 hping -c 1 -d 1511 -i u 81 --rand-source -p 22 -S 127.0.0.1 
 hping -c 1 -d 1511 -i u 81 --rand-source -p 25 -S 127.0.0.1 
 hping -c 1 -d 1511 -i u 81 --rand-source -p 80 -S 127.0.0.1 
 hping -c 1 -d 1511 -i u 81 --rand-source -p 53 -S --udp 127.0.0.1 

- these attack the servers or client desktop/laptops

- volumetric attacks -- almost everybody will fail this test
- volumetric attacks are pointless, you'll always fail at some point
ping -f
iperf
socat

- send spam  .. mitigated separately ...
- send virus and worms to the list  ... mitigated separately ...

- cloud solutions
- if you have regulatory compliance requirements, your options
  are extemely limited to a few certified amd expensive clouds

- what triggers the packets to go to the cloud for scrubbing

- you do NOT want somebody looking at millions of packets to
  decide to send it off the cloud for scrubbing or not

- you might NOT want to send everything to the cloud and
  incurr un-necessary expenses if you're NOT under xxxGbit/sec 
  DDoS attacks

- ddos mitigation should be able to distinguish legit traffic 
  from real ddos traffic
- eg folks downloading or sending 4GB dvd or larger files
- eg silly folks sending 4GB dvd via emails

  # simplified way to distinguish legit users from ddos attackers

if web servers are running 

Re: Data Center operations mail list?

[alvin nanog]

hi nanog

On 08/11/15 at 04:46pm, Alex Brooks wrote:
 With the lack of interest compared to NANOG (especially seeing how the
 old list simply dried up) it might be best making the list global
 rather than North America only to get the traffic levels up a bit.

there used to be an active isp-xxx.com decades ago
that was taken over by a commerical entity
- the xxx represented various topics
( security, infrastructure, colo, control panels, etc )

those domains are all dead .. i found these new url that may or 
may not be what you're looking for ... i didnt check it out
- https://lists.debian.org/debian-isp/
- http://lists.freebsd.org/mailman/listinfo/freebsd-isp
- http://www.reedmedia.net/lists/bsd-isp/

i'd like to see a NOC contact address for resolving
( DDoS ) issues coming from that ISP ... i'll help out if needed

the list of friendly co-operating NOC contracts probably need to be 
member only protected to protect it from being spam'd ?

pixie dust
alvin
# http://DDoS-Mitigator.net

Re: RES: Exploits start against flaw that could hamstring huge swaths of

[alvin nanog]

hi ya

  On Tue, Aug 4, 2015 at 11:29 AM, Scott Helms khe...@zcorum.com wrote:
   With the (large) caveat that heterogenous networks are more subject to
   human error in many cases.
 
  coughautomate!/cough
 
...

On 08/04/15 at 12:21pm, Christopher Morrow wrote:
 On Tue, Aug 4, 2015 at 11:46 AM, Scott Helms khe...@zcorum.com wrote:
  Automation just means your mistake goes many more places more quickly.
 
 
 and letting people keep poking at things that computers should be
 doing is... much worse. people do not have reliability and
 repeat-ability over time.

ditto ...
computers are experts at listening and repeatatively doing what it's 
told to do ..

 If you fear 'many more places' problems, improve your testing.

i prefer automation .. even if it's wrong, you can look at the script
and see what bad things it did and you should know what to do to fix
the problem and fix the script to prevent it from spreading that mistake 
again

person's standard excuse
if you ask a person(s), what did you do to create this mess, duh... i donno
btw, it's my kids birthday, i needed to be home an hr ago with the cake :-)

hummm... :-)
/standard

-

fwiw
for automation to work:
- folks updating the scripts should be required to know all platforms being 
  used and how its different from each other 

- folks testing the scripts/updates process/proceedures should be paid
  bonuses, even free pizza/beer for finding bugs before release to the 
  your internal world of automated-machines

- always have 3 co-developments boxes for the script develpment and
  to backup each other 

- always have 2 or more test bed boxes for initial releases of new scripts
  where those boxes can also be downgraded back to the previous release
  before the new patches was applied

- if nothing went wrong, there should be minimal issue with release a 
  patch where it doesn't propagate problems automatically to everywhere

  the trick is how good are the eyes/brains that is looking for 
  potential problems of the new releases/patches/updates/etc

- i also say always let clients pull down patches vs pushing it to
  systems that seems un-responsive to avoid having to wait for dead boxes

-
all appps, not just bind, has occasional problems .. changing to something
else doesn't necessarily solve the original bug problem

pixie dust
alvin
# ddos-mitigator.net

Re: Quakecon: Network Operations Center tour

[alvin nanog]

hi mr bugs :-)

On 08/03/15 at 05:38pm, Mr Bugs wrote:
 The WiFi jammers have an interesting MO. They don't throw up static on the
 frequency, that would also block their own wifi. They spoof
 de-authentication packets. I've been looking for a way to detect this kind
 of jamming because my WiFi sucks and I live next to three hotels, what you
 get for living in downtown Atlanta.

i forgot if kismet showed signal strengths of the wifi ap's ...
stronger signal wins over weaker signal strengths

might not be a jamming issue ??  kismet and tcpdump might be able to show
you the packets you're looking for ?

what happens if you put up a properly designed wire mess around the exterior 
windows of your house/condo/aptr??

i'd wag/blindly say the area is probably full of rogue wifi ap's floating around
where evergbody is trying to wardrive each other and pick up un-suspecting
traveling visitor's login and passwd info ... signals bouncing off 
steel/concrete
is not ez to filter out what should be random background white noise if
you're sitting next to the radiating source ..

pixie dust
alvin
# DDoS-Mitigator.net  
# DDoS-Simulator.net  

Re: Quakecon: Network Operations Center tour

[alvin nanog]

hi ethan

On 08/03/15 at 10:58am, Ethan wrote:
 
 Getting bandwidth into the events is a pain. Huge venues are meant for large
 corporate events not lower budget cons and festivals. Venue pricing I
 believe is 750-1500$ per megabit. 100 megabit = $75,000 for the weekend. One
 year I rememeber there being a switch with 8 vlans on it sitting outside the
 back door with 8 clear modems spread out all blinking away.

for connectivity, does the hotels and convention centers still have wifi 
jammers 
so you cannot use your own 56Mbit wifi to get connection to the outside world ? 
if possible, stick a bunch of dark mirrored-glass covered vans outside the event
for wifi access

the expensive part is due to labor unions that control the workers and
everything else working the capitalistic supply and demand model to the max.
the unions disallow you to carry your own gear from your car to the event
which is good and bad ... 

i dont buy their $10 budweiser, $5 water, etc especially when no outside drinks
allowed inside the event

 Geeks get creative.

good thing  and no unions to control what we did/do ...

another ( 40yr old ) boat that has long since sailed since the days
of why we had to fight off the unions in the electronics industrt ... 

pixie dust
alvin

Re: DDOS Simulation

[alvin nanog]

hi roland

- yup... agreed on most all of your points ...

- good referral to prev ddos discussions

- i'm just saying ..
if one cannot defend and know that their ddos mitigation
is working on the low level free script kiddie ddos attacks,
they should not worry about scaling to gigabit/s, 1terabit/sec
or even 100 terabit/s ddos attacks ... 

one has to start somewhere and grow their ddos mitigation and 
ddos attacks knowledge ... i happen to need to know how to
defend my customers in between the free script kiddies and the 
types of attacks that make the papers/new

start with free (thousands) of ddos attack tools and (hundreds)
of free ddos mitigaton tools

- i'm fairly certain i can fill any pipe with jibberish data
  where ddos mitigation might not work as expected  but when the cops 
  come knocking, the ddos attackers are in deeep kah kah, thus requiring
  prior legal paperwork of all those directly and indirectly involved 

have fun
alvin

On 07/30/15 at 03:05am, Roland Dobbins wrote:
 On 30 Jul 2015, at 2:38, alvin nanog wrote:
 
 there is no need to pay people to attack your servers ...
 
 Unless you don't have the expertise to do it yourself.  Again, I advocate an
 organic defense capability and an organic testing capability, but there are
 many organizations which unfortunately don't have these, and they must start
 somewhere.
 
  - tcpdump and wireshark will tell you everything the attackers are
  doing to your network right now that needs to be defended against
 
 On small, single-homed networks, sure.  On networks of any size, this
 doesn't scale.
 
 Flow telemetry scales.
 
 if a mid-level wanna be attacker wants to target your servers, they're
 just as equally easy to mitigate and prevent and probably sending you
 100,000 ddos packets per second because they can ( bigger zombie network
 :-)
 
 100kpps is nothing.  Of course, so many servers/services are so brittle,
 fragile, and non-scalable that most DDoS attacks are overkill by orders of
 magnitude.
 
 if you are being targeted by masters of deception you have no solution
 other than get local law enforcement involved to track down the
 originating
 attackers
 
 I'm not sure who or what 'masters of deception' are in this context, but
 attribution has nothing to do with DDoS defense.
 
 Defending against serious attackers with lots of resources is taking place
 every minute of every hour of every day.  There are many techniques and
 tools available, most of which have been discussed multiple times on this
 list over the years.  Here's one such example:
 
 http://mailman.nanog.org/pipermail/nanog/2010-January/016747.html
 
 all ddos mitigations is almost 100% guaranteed to fail a volumetric
 DDoS attacks 
 
 This is incorrect.
 
 the DDoS attackrs probably have access to a bigger zombie
 network than most major corp ...
 
 This is true, in many cases - and is also not an issue for
 properly-provisioned, coordinated DDoS defense mechanisms and methodologies.
 
 the attackers job is not to get caught and
 is not ez to be hiding if law enforcement wanted to catch them :-)
 
 Again, attribution is a completely separate issue.
 
  nping send 100,000 packets/sec x 65,000byte/packet  192.168.0.0/16
 
 FYI, 'line-rate' for 64-byte packets at 10gb/sec is ~14.8mpps.
 
 by the same premise, if i had to pick ONE ddos mitigation strategy, i'd
 tarpit all incoming TCP-based ddos attacks which should crash the
 attacking zombie server under sustained tcp-based ddos attacks
 
 There is no one tactic (this is not a strategy) which can be picked, as any
 kind of traffic can be used for DDoS attacks.  With regards to TCP-based
 attacks, it's a subset of those which are connection-oriented and are thus
 susceptible to tarpitting-type techniques.
 
 ---
 Roland Dobbins rdobb...@arbor.net

Re: DOCSIS CMTS Systems

[alvin nanog]

hi

On 07/29/15 at 10:59am, Curtis Maurand wrote:
 Seriously nice solutions...both of them.

..

 -Original Message-
 From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Colton Conor
 Sent: Wednesday, July 29, 2015 8:27 AM
 To: NANOG nanog@nanog.org; Scott Helms khe...@zcorum.com
 Subject: DOCSIS CMTS Systems
 
 We are servicing more MDU customers that have older buildings. There is no
 CAT5E installed, so extremely old phone cable or coaxial TV cable seems to
 be our only inside wire options. There is no easy and inexpensive way to
 run new cable, so we must deal with what is available.

is internal corp wifi an option for internal users w/o cat5e ??
- be sure to only allow as many IP# as needed, not the entire 192.168.0.0

pixie dust
alvin

Re: DDOS Simulation

[alvin nanog]

hi roland

On 07/29/15 at 05:47am, Roland Dobbins wrote:
 
 On 29 Jul 2015, at 5:19, alvin nanog wrote:
 
 as previously noted by others, legit corp will ask you for lots of
 legal paperwork  for their get out of jail card for DDoS'ing your
 servers
 and all the other ISP's routers along the way that had to transport
 those gigabyte/terabyte of useless ddos packets
 
 No company can provide a 'get out of jail card' for illegal activities,
 irrespective of how they arrange their paperwork.

oopps, maybe a misunderstanding ... it's an old be careful euphomism(sp?)
and not meant as literal get out of jail ( from monopoly game too )
- it's intended as make sure the corp lawyers are involved that
is requesting the ddos simulation/testing ( aka pen testing )

- managers/employee/contractors cannot say or sign anything
that binds the company to what the managers said/request

- only officers of the company can bind the company that they
will not press charges for the ddos (pen) tests

- po's are usually valid since the CFO is an officer of the company


 DDoS testing across the Internet is a Big No-No due to legal considerations,
 potential liabilities, potential for catastrophic error, etc.

yes, along with all the other isp's involved along the way between
ddos testor and corp-under-test.com

 Doing it across one's own network which one controls is certainly viable.

definitely and should be the place to start

put your ddos simulator hardware in parallel to your cisco/juniper uplink
to the isp and simulate for the next few decades :-)

 There are some companies which do that, and which take a belt-and-suspenders
 approach to ensure that simulated attack traffic doesn't leak, etc.

all computers are under 24x7x365 ddos attacks every minute and they already
provide the free real world and luckily low level DDoS attacks for free

you should figure out how to find those free ddos attacks and how to mitigate
the script kiddies already providing the free initial ddos simulation

there is no need to pay people to attack your servers ...

- tcpdump and wireshark will tell you everything the attackers are 
doing to your network right now that needs to be defended against

# if you are a web server, it is currently under (free) DDoS attack
tcpdump -n -l dst host www.example.com and ! dst port 80

# if you are a mail server, it is currently under (free) DDoS attack
tcpdump -n -l dst host mail.example.com and ! dst port 25

- a small exercise to clean up the tcpdump output

if a mid-level wanna be attacker wants to target your servers, they're
just as equally easy to mitigate and prevent and probably sending you
100,000 ddos packets per second because they can ( bigger zombie network :-)
- you should notice some slow responses from your servers

if you are being targeted by masters of deception you have no solution
other than get local law enforcement involved to track down the originating
attackers

all ddos mitigations is almost 100% guaranteed to fail a volumetric
DDoS attacks  the DDoS attackrs probably have access to a bigger zombie
network than most major corp ... the attackers job is not to get caught and 
is not ez to be hiding if law enforcement wanted to catch them :-)

problem is the attackers have to be bothersome to somebody before
they start chasing down the attackers .. the rest of us has to fend 
for ourself

 Simulated DDoS attacks and testing of defenses should be part of any real
 development environment, along with scalability testing in general.  Sadly,
 this is rarely the case.

yup :-)

 The best way to learn how to defend something is to learn how to attack it.

exactly  you cannot defend against something you don't understand 
or don't know about that attack vector

different folks defintely attack and/or test for different things
- get different folks to do the testing

if i had to pick only one command for the ddos tests  i'd simply 
flood the wire .. everything is now offline ( should be un-responsive )

nping send 100,000 packets/sec x 65,000byte/packet  192.168.0.0/16

nping can create all kinds of headaches since you can attack
almost anything ... most prototcols, most src/dst ip# and ports 

by the same premise, if i had to pick ONE ddos mitigation strategy, i'd
tarpit all incoming TCP-based ddos attacks which should crash the
attacking zombie server under sustained tcp-based ddos attacks

 Organizations with substantial Internet properties should develop their own
 organic capabilities to perform such testing in a safe and responsible
 manner, as it will also enhance the skills needed to defend said properties.
 ---
 Roland Dobbins rdobb...@arbor.net

yup

magic pixie dust
alvin
- http://DDoS-Mitigator.net
- http://DDoS-Simulator.net

Re: DDOS Simulation

[alvin nanog]

hi dovid

On 07/28/15 at 02:31pm, Dovid Bender wrote:
 We are looking for a company that can launch a DDOS attack against the
 solutions we are testing. I don't want a proof of concept from the company
 that will be offering DDOS protection since they can simulate an easy
 attack and then mitigate. I want whom ever we go with to be able to handle
 what ever is thrown at them.

most all ddos simulator folks all sell their own version of a ddos mitigator
appliance or ddos cloud services ... both has good and bad ddos mitigation
features depending on the type of DDoS attacks you are defending against

http://DDoS-Mitigator.net/Competitors

- largest folks ( aka probably legit ) are probably akamai/prolexic,
arbor networks, fortinet, incapsula, radware, etc

as previously noted by others, legit corp will ask you for lots of
legal paperwork  for their get out of jail card for DDoS'ing your servers
and all the other ISP's routers along the way that had to transport
those gigabyte/terabyte of useless ddos packets

imho, most ddos simulator folks will want to know what are you wanting
to simulate   there are easily, say 100,000 attack vectors ...
- attack all your IP#
- attack all ports on each IP#
- various arp flood 
- various icmp flood
- various udp flood
- various tcp flood ( trivial to defend )
- attack specific vulnerabilities already found n not patched

- there are proably thousands of apps that can be used
to launch various DDoS attacks ...

- volumetric icmp DDoS attacks and volumetric udp DDoS attacks will
  most likely take you offline ... almost nothing you can do to 
  stop it, prevent it, block it, etc... your ISP has to do that for you
  or your ISP's larger peer has to get in there too

you will want the ph# of the security guru at the ISP
to help you resolve the issue

i doubt any ddos mitigation will help you and more importantly,
you probably will not want to pay $$$ to the ddos cloud scrubber
to be removing xTB of udp or icmp DDoS attacks

- if you're thinking of ddos attacks as anything that is thrown at them
  against webservers, mail servers, and ssh servers, that is only 3 ports 
  out of 65,535 possible attacks

there is no such thing as anything that can be thrown at them

defending web servers, mail servers and ssh servers can
be script kiddie trivially defended ... as long as it is
properly patched and maintained and built to be defensible

before you ask others to DDoS your servers, have you
already patched apache/sendmail/ssh/openssl, kernels, etc, etc

ddos attackers will be looking for your weakest link,
usually login/pwd from outside wifi access points and 
home offices, hotel ethernet, etc

there is almost zero benefit for volumetric 10TB or 20 TB of
DDoS attacks we read about in the papers against large corp. the only
defense is to build your own geographically separate colo in each
major customer countries in asia, europe, usa, south america, etc

usually the purpose of DDoS attacks is to take your servers offline or
steal/copy/sniff info or hide in your network or launch other attacks

these are easier ( script kiddie ) DDoS attacks and less likely to 
be noticed by your ISP of incoming attacks
- sniff login/passwd from outside ( wifi, home office, etc )
- install keyboard sniffers
- install other trojans ( virii, worm, etc )

endless list of attacks to simulate

pixie dust
alvin
- http://DDoS-Simulator.net

 On Mon, Jul 27, 2015 at 5:40 PM, lobna gouda lobna_go...@hotmail.com
 wrote:
 
  Hello David et Dan,
 
  Are you going to perform the DDOS solution yourself, or you are looking
  for  a company to provide a solution for you. Some companies perform an
  attack simulation for you before buying the product
 

Re: DDOS Simulation

[alvin nanog]

hi dovid

On 07/27/15 at 11:32am, Dovid Bender wrote:
 We are looking into a few different DDOS solutions for a client. We need a
 LEGITIMATE company that can simulate some DDOS attacks (the generic +
 specific to the clients business). Anyone have any recommendations?

i've compiled a fairly comprehensive list is here:

- http://ddos-mitigator.net/Competitors

simulating ddos attacks are fairly easy to do, except one does
have to be careful of process and proceedure and the all important
get out of jail for free card ( let your local ISP techie's know too )

http://DDoS-Simulator.net/Demo
( wrapper gui around *perf/nc/nmap/*ping command options )

ddos mitigation is not a single thing-a-ma-jig, and should
be multi-layered, different solutions solving different DDoS issues

http://ddos-solutions.net/Mitigation/#Howto
- how are they attacking
- who is attacking ( script kiddie vs master of deception )
- what are they attacking
- when are they attacking
- why are they attacking
- ...

# -
# what kind of simulations are you trying to do ??
# -
- volumetric attacks say 10gigabit vs 200gigabit attacks is trivial
- ping flood, udp flood, arp flood, tcp flood, etc, etc

  local appliances with 10/100 gigabit NIC cards should be able to
  generate close to 100 gigabit/sec of ddos attacks

- udp and icmp attacks are harder to mitigate, since those packets
  need to be stopped at the ISP  if it came down the wire to
  the local offices, it already used the bandwidth, cpu, memory,
  time, people, etc, etc

- tcp-based ddos attacks are trivial ( imho ) to defend against with
  iptables + tarpits
if each tcp connection takes 2K bytes, the DDoS attacker 
that is intent on sending large quantity of tcp-based packets
would incur a counter ddos attack using up its own kernel
memory

100,000 tcp packet/sec * 2K byte -- 200M /sec of kernel memory

?? with tcp timeout of 2 minutes implies they'd need 24TB of
?? kernel memory to sustain a 100,000 tcp packet/sec attack 

# live demo of tarpit incoming ddos attacks
http://ddos-mitigator.net/cgi-bin/IPtables-GUI.pl
http://target-practice.net/cgi-bin/IPtables-GUI.pl

# command line options is 100x faster and easier than html 

# to automatically add new incoming ddos attackers
iptables-gui -doadd -addauto

# to automatically remove inactive ddos attackers
iptables-gui -dodel -deluto

ssh based solutions are nice but only works on port 22
http based solutions are nice but only works on port 80

there are 65,533 other ports to defend against DDoS attacks
which is defensible with tarpit

- it is trivial to generate attacks against apache or web browser 
- it is trivial to generate attacks against sendmail or mail reader

- netcat/socat/nc, hping*, nping, etc, etc
- something that you can define source and destination IP#
- something that you can define source and destination port#

- it is harder to generate the various malformed tcp headers

- gui to help set tcp header flags and options for nmap/hping
- http://ddos-simulator.net/Demo/

- spam, virii and worms seems to be in its own category

- another important question for your clients is if they are under
  any govermental regulations which will limit their choices of solutions
- hippa, pci, sox, etc

   inhouse ddos solutions should not have any governmental compliance
   issues

   cloud based ddos solutions and their facilities would have to 
   comply with the various govermental issues 

   both inhouse and cloud based solutions solve some problems

   another 32+ point comparison for inhouse vs cloud based solutions
   - http://ddos-mitigator.net/InHouse-vs-Cloud

thanx
alvin
- http://ddos-mitigator.net
- http://ddos-simulator.net

Re: DDOS Simulation

[alvin nanog]

hi pavel

On 07/28/15 at 12:02am, Pavel Odintsov wrote:
 It's poor man's traffic generator :)
 
that's the best kind :-) 
as long as it gets the job done and you get to control what it does

 My test lab is i7 2600 with 2 port Intel X520 10GE and Intel Xeon E5
 2604 witj 2 port Intel X520 10GE.

nice cpu hw

trick questions for those thinking of generating ddos traffic for testing

- ?? how much memory was needed to run the traffic generator

i assume around 1GB of memory for 1gigE interface and i still
can purposely run out of memory while some apps are running

at 10gigE pci card, 
you'd probably want at least 12GB - 16GB of memory

- some poor mans apps to generate traffic ... start w/ nping or hping

# generate 1,000 Mbit/sec of junk .. floodig is trivial ...
ping -i 0.001 -s 2000  victimIP#
nping --data-length 2000 --rate 1000 victimIP#
socat
iperf ...
#
# generate udp  or icmp or arp or tcp traffic
#
# add options to generate large-sized packets
# add options to generate 10Gbit/sec ( number of packet/sec )
#
# play around with tcp headers
# add options to send MTU=1501 byte but NOT set DF
# add options to send ACK but no request
#
# add options to spoof source and desitination address and ports

#
# if the host machine become un-available, you've got a problem
#
for host in gw dns ntp http smtp
  for protocol in arp icmp udp tcp
nping --protocol [ options ] host.example.com 
# hping is nice too
  done
done

# for bonus arp fun ...
attacker# arpspoof gateway victim
attacker# arpspoof victim gateway

# prevent mitm with: use hard coded arp /etc/ethers for linux

use OpenSSL certs to flag a warning when attacker inserted
itself in between gateway and un-aware victim

pixie dust
alvin
- DDoS-Mitigator.net

 On Mon, Jul 27, 2015 at 11:59 PM,  valdis.kletni...@vt.edu wrote:
  On Mon, 27 Jul 2015 23:32:56 +0300, Pavel Odintsov said:
 
  I would like to recommend MoonGen for generating very high speed
  attacks (I have generated up to 56 mpps/40GE with it).
 
  OK, I'll bite - what hardware were you using to inject that many packets?