Re: SNMP DDoS: the vulnerability you might not know you have

[Saku Ytti]

On (2013-07-31 17:07 -0700), bottiger wrote:

> But realistically those 2 problems are not going to be solved any time
> in the next decade. I have tested 7 large hosting networks only one of
> them had BCP38.

I wonder if it's truly that unrealistic. If we target access networks, it
seems impractical target.

We have about 40k origin only ASNs and about 7k ASNs which offer transit,
who could arguably trivially ACL those 40k peers.

If we truly tried, as a community to make deploying these ACLs easy and
actively reach out those 7k ASNs and offer help, would it be unrealistic to
have ACL deployed to sufficiently large portion of networks to make
spoofing impractical/expensive?


Do we have other approaches? Can we make this ACL dynamic to a degree? Can
we extract ACL information from BGP table?
If origin only ASN advertises prefix to global table anywhere, allow it at
matching 'remote-as' port. Does not look like difficult feature to build,
does not require magic HW support, essentially dynamically built ACL.
After this spoof would require injected trash BGP route, which would also
steal return traffic, making it useless for DoS.


-- 
  ++ytti

Re: nLayer IP transit

[Saku Ytti]

On (2013-08-01 10:00 +1000), Mark Tees wrote:

> I remember reading a while back that customers of nLayer IP transit
> services could send in Flowspec rules to nLayer. Anyone know if that is
> true/current?

Anyone planning to do this might want to be aware that the validation
process of flowspec does not limit actions.

In practice this means, if you do run flowspec to your customers, your
customers likely can inject traffic to arbitrary VRFs.

I feel RFC should have explicitly stated valid actions for validation
process, which operator MAY change, and any other action MUST cause
validation process to fail.


-- 
  ++ytti

Re: SNMP DDoS: the vulnerability you might not know you have

[Mark Andrews]

In message 
, bottiger writes:
> I realize the root cause is security-oblivious designers and one level
> below that, lack of BCP38.
> 
> But realistically those 2 problems are not going to be solved any time
> in the next decade. I have tested 7 large hosting networks only one of
> them had BCP38.

Then name and shame.
 
> To my knowledge it is practically impossible for someone outside a
> multi-homed network to know if they allow spoofing which means it will
> be difficult to punish. It also cost time and money to maintain these
> ACLs, much more than blocking the occasional wide-spread protocol with
> 8000x amplification every couple of years.

Just define a EDNS option which contains the real ip address and
update who-am-i services to look for that when replying, use a
modified DNS cookies or similar to prevent this being used as
amplifier.  You can do this with a experimental EDNS option code
points today.

Send a non-spoofed request then send a spoofed request using the
cookie learnt from the first request.

Standardise it and every nameserver on the planet and be used to
detect if spoofing is possible.  Using DNS puts up to costs for
ISP's that want to block people checking.

The blackhats already check whether spoofing is possible so there
is little point in blocking whitehats checking.

As for time and money to maintain these acls it really shouldn't
now that SIDR is becoming a reality.  Present a properly signed
CERT to the other providers and automatically generate the acls
based on those.  There is a little setup cost to get the system
running and almost no additional recurrent costs to keep it up
to date.  The CERTs need to be generated for sBGP anyway.

> I am here to talk about solutions today. BCP38 has been repeated to
> death and people aren't going to start doing it because I said so. The
> fact that the amplification factor is so high means that you need to
> ensure near 100% conformity even if everyone started to become BCP38
> compliant today.
> 
> On Wed, Jul 31, 2013 at 4:42 PM, Jimmy Hess  wrote:
> > On 7/31/13, Blake Dunlap  wrote:
> >> I bet blocking all SYN packets and non related flow UDP packets to
> >> customers would be even more effective. Why don't we do that and be done
> >> with it instead of playing whack a mole every 3 months when someone finds
> >> some new service that was poorly designed so that it can be used to send a
> >> flood?
> >
> > Because it breaks applications that people are paying to be able to use.
> >
> > The way I see it;  more and more samples keep getting found about
> > protocols abused because networks have not implemented BCP38.
> >
> > The latest SNMP trend is just another uptick to the sample size,  and
> > proof that  Closing off   perfectly OK  recursive DNS services  is
> > totally inadequate and not a useful long-term fix  to the problem of
> > DDoS or IP/UDP reflection attacks.
> >
> > Asking folks to improve the security of access to their SNMP instances
> > is just chasing the latest exploit implementation,  with no attention
> > to the vulnerability or the root cause
> >
> > --
> > -JH
> >
> 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: ARIN WHOIS for leads

[John Curran]

On Jul 31, 2013, at 1:17 PM, Barry Shein  wrote:

> The usual method is to insert "ringers" which would be info which
> points back at non-existant people with valid-looking contact
> information.
> 
> If for example they called a phone number, or several, owned by ARIN
> (or a service they employed) asking for James T Kirk or Diana Prince
> then that would be a problem and should be logged.

There are some interesting non-obvious elements in the database for 
such purposes and we do take action when they are triggered.

FYI,
/John

John Curran
President and CEO
ARIN

Re: SNMP DDoS: the vulnerability you might not know you have

[bottiger]

I realize the root cause is security-oblivious designers and one level
below that, lack of BCP38.

But realistically those 2 problems are not going to be solved any time
in the next decade. I have tested 7 large hosting networks only one of
them had BCP38.

To my knowledge it is practically impossible for someone outside a
multi-homed network to know if they allow spoofing which means it will
be difficult to punish. It also cost time and money to maintain these
ACLs, much more than blocking the occasional wide-spread protocol with
8000x amplification every couple of years.

I am here to talk about solutions today. BCP38 has been repeated to
death and people aren't going to start doing it because I said so. The
fact that the amplification factor is so high means that you need to
ensure near 100% conformity even if everyone started to become BCP38
compliant today.

On Wed, Jul 31, 2013 at 4:42 PM, Jimmy Hess  wrote:
> On 7/31/13, Blake Dunlap  wrote:
>> I bet blocking all SYN packets and non related flow UDP packets to
>> customers would be even more effective. Why don't we do that and be done
>> with it instead of playing whack a mole every 3 months when someone finds
>> some new service that was poorly designed so that it can be used to send a
>> flood?
>
> Because it breaks applications that people are paying to be able to use.
>
> The way I see it;  more and more samples keep getting found about
> protocols abused because networks have not implemented BCP38.
>
> The latest SNMP trend is just another uptick to the sample size,  and
> proof that  Closing off   perfectly OK  recursive DNS services  is
> totally inadequate and not a useful long-term fix  to the problem of
> DDoS or IP/UDP reflection attacks.
>
> Asking folks to improve the security of access to their SNMP instances
> is just chasing the latest exploit implementation,  with no attention
> to the vulnerability or the root cause
>
> --
> -JH
>

Re: nLayer IP transit

[Patrick W. Gilmore]

On Jul 31, 2013, at 20:00 , Mark Tees  wrote:

> I remember reading a while back that customers of nLayer IP transit
> services could send in Flowspec rules to nLayer. Anyone know if that is
> true/current?

Not any more.

-- 
TTFN,
patrick



signature.asc
Description: Message signed with OpenPGP using GPGMail

nLayer IP transit

[Mark Tees]

Howdy listers,

I remember reading a while back that customers of nLayer IP transit
services could send in Flowspec rules to nLayer. Anyone know if that is
true/current?

Thanks,

-- 
Regards,

Mark

Re: SNMP DDoS: the vulnerability you might not know you have

[Jimmy Hess]

On 7/31/13, Blake Dunlap  wrote:
> I bet blocking all SYN packets and non related flow UDP packets to
> customers would be even more effective. Why don't we do that and be done
> with it instead of playing whack a mole every 3 months when someone finds
> some new service that was poorly designed so that it can be used to send a
> flood?

Because it breaks applications that people are paying to be able to use.

The way I see it;  more and more samples keep getting found about
protocols abused because networks have not implemented BCP38.

The latest SNMP trend is just another uptick to the sample size,  and
proof that  Closing off   perfectly OK  recursive DNS services  is
totally inadequate and not a useful long-term fix  to the problem of
DDoS or IP/UDP reflection attacks.

Asking folks to improve the security of access to their SNMP instances
is just chasing the latest exploit implementation,  with no attention
to the vulnerability or the root cause

--
-JH

Revealed: NSA program collects 'nearly everything a user does on the internet'

[Marsh Ray]

Chris Boyd cboyd at gizmopartners.com  Wed Jul 31 15:50:09 UTC 2013
>
> I would guess that it's becasuse many VPN services still support PPTP which 
> can be attacked as outlined here:
> http://www.schneier.com/paper-pptpv2.html
>
> --Chris

That link doesn't even mention the worst vulnerability in PPTP/MS-CHAPv2. 
Strangely, it's only in the PDF version 
http://www.schneier.com/paper-pptpv2.pdf at the bottom of page 6:

> Note also that the MS-CHAP response generation algorithm is also a weak
> link, even when passwords contain adequate entropy. It is clear that the NT
> hash can be recovered with just two DES exhaustive keysearches (about 256
> trial DES decryptions on average)

In other words, PPTP/MS-CHAPv2 is equivalent to encrypting your password with 
*single DES* and sending it over the untrusted network. It doesn't matter how 
strong your plaintext password is. Not only can the passive eavesdropper 
decrypt your VPN-tunneled data, he can obtain the NT hash which is a 
password-equivalent credential allowing him to impersonate the user to log into 
any other network services. 

Moxie Marlinspike and David Hulton described the exploit for it at Defcon 20 
last summer:

Defeating PPTP VPNs and WPA2 Enterprise with MS-CHAPv2
http://www.youtube.com/watch?v=vWXP3DvH8OQ

Moxie's Cloudcracker online service will decrypt your PPTP packet captures 
using an FPGA cluster from Pico Computing. Last I heard, the price was a flat 
fee of $200, although it sometimes goes on sale.
http://www.h-online.com/security/features/A-death-blow-for-PPTP-1716768.html 
http://h-online.com/-1716768

So it's not just the NSA, it's any passive observer with a budget of $200 for a 
one-off or ~$10K for their own hardware capability.

PPTP is old and busted, don't let your friends use it! If you've ever used it, 
change your password. IMHO, if there's any other protocol more deserving of the 
"internet kill switch" I don't know what it is.

- Marsh

(sorry for not threading properly, I just subscribed to reply)

Re: Hilton proxy issue

[Grant Ridder]

Sounds great Jay, thanks!

On Wed, Jul 31, 2013 at 1:31 PM, Jay Moran  wrote:

> I have BCC'd the likely appropriate Hilton contact for you on this
> response so they can take a look at the NANOG emails below regarding their
> Internet proxies to see if it looks like something they can assist with.
> They were able to have some MTA issues corrected last time Hilton came up
> on the NANOG list. Good luck!
> --
> Jay Moran
> http://linked.com/in/jaycmoran
>
> On Wed, Jul 31, 2013 at 4:16 PM, Grant Ridder wrote:
>
>> Better yet, does anyone have any Hilton contacts they could pass my info
>> to?
>>
>> -Grant
>>
>> On Wed, Jul 31, 2013 at 8:54 AM, Grant Ridder > >wrote:
>>
>> > Anyone from Hilton out there?  We are still having this issue.  It is
>> not
>> > a wayport address since I looked and they are not registered under
>> Hilton's
>> > name.
>> >
>> > -Grant
>> >
>> >
>> > On Tue, Jul 16, 2013 at 1:17 PM, Grant Ridder > >wrote:
>> >
>> >> The requests are coming from 167.187.100.202 which is in a /16 assigned
>> >> to Hilton.  As far as i know, the waypoint service has its own
>> netblocks.
>> >>
>> >> -Grant
>> >>
>> >>
>> >> On Tue, Jul 16, 2013 at 1:11 PM, Jared Mauch > >wrote:
>> >>
>> >>>
>> >>> On Jul 16, 2013, at 3:44 PM, Grant Ridder 
>> >>> wrote:
>> >>>
>> >>> > Hi,
>> >>> >
>> >>> > Anyone from Hilton Hotels NOC or related on here?  We are seeing
>> their
>> >>> > internet proxy doing weird things to http requests to servers at
>> >>> $DAYJOB.
>> >>>
>> >>>
>> >>> Many of the hilton properties have migrated to Wayport/"attwifi".  Are
>> >>> you seeing the requests from AT&T/Wayport or from their corporate?
>> >>>
>> >>> (btw, if you're here and with wayport/attwifi, i would be interested
>> in
>> >>> chatting briefly with you).
>> >>>
>> >>> - Jared
>>
>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Ricky Beam]

On Wed, 31 Jul 2013 18:50:18 -0400, Larry Sheldon   
wrote:
But after years of research I will tell you that there is no way to stop  
an avalanche once it has been released at the source.


http://youtu.be/60loeoblu0M

Anyone can make a device and connect it to the internet.

Re: SNMP DDoS: the vulnerability you might not know you have

[Larry Sheldon]

On 7/31/2013 4:29 PM, Blake Dunlap wrote:


It works better to fix the design issues than to play whack a mole
by blocking every imaginable service to your customers that responds
to the public with data larger than a FIN. Like getting their
providers to more proactively police their spew, manufactures to stop
making negligent devices, or implementing more intelligent filter
communication so the only option doesn't begin with calling your
provider and asking them over the phone to block X ip for you since
you're off the internet.

Maybe even look into liability laws for allowing said attacks to
originate from your customers and not doing anything about it, or
being manufacturer of said devices that harm others through their
lack of due diligence implementing proper security. It's still way
more effective than trying to fix the *last instance* of the problem,
instead of it's reasons for enduring as an issue at a global scale.


The first time I became a pariah on NANOG was for postulating precisely 
that view--that if the originators of problems would stop, we would not 
have to figure out how to clean up after them.  But I am past that now 
and out of work.


But it does occur to me for the first time that I can recall, that maybe 
the tremendous efforts to Get Control Of The Intertubes could be 
suckered into doing some good, say be establishing a certification 
authority to test and certify the safety of designs (is Scott B? 
still around) and devise a way to not permit traffic from uncertified 
devices or configurations.


But after years of research I will tell you that there is no way to stop 
an avalanche once it has been released at the source.

--
Requiescas in pace o email   Two identifying characteristics
of System Administrators:
Ex turpi causa non oritur actio  Infallibility, and the ability to
learn from their mistakes.
  (Adapted from Stephen Pinker)

Re: SNMP DDoS: the vulnerability you might not know you have

[bottiger]

This vulnerability has been present ever since SNMP v2 was announced
back in 1993.

There is a reason why the biggest attacks these days are from
protocols that are decades old like DNS and Chargen.

People making widely spread protocols these days are aware of the
problem and are usually able to get their software to auto-update.

Enforcing source IP filtering seems like a lost cause. Not much
progress has been made that I can tell and it is difficult to discover
if the network allows it without being inside it.

I don't see many uses for unsecured SNMP access so this is not like
blocking all syn and udp packets.

On Wed, Jul 31, 2013 at 2:29 PM, Blake Dunlap  wrote:
> I bet blocking all SYN packets and non related flow UDP packets to
> customers would be even more effective. Why don't we do that and be done
> with it instead of playing whack a mole every 3 months when someone finds
> some new service that was poorly designed so that it can be used to send a
> flood?
>
> Yes, I'm being sarcastic above.
>
> This is hardly the first finger of death amplification attack, and I
> strongly doubt it'll be the last. Years ago it was smurf, then Quake
> servers, then DNS, then Battlefield boxes, etc etc. Now it seems to be snmp
> and recently chargen, and tomorrow it'll be some peer 2 peer service, the
> next day it'll be a voice app. It will never end, and breaking the internet
> port by port doesn't do anything to make it better.
>
> I've been the victim of week long DDoS attacks that took down our
> upstreams, not to mention us, and I still maintain the above.
>
> It works better to fix the design issues than to play whack a mole by
> blocking every imaginable service to your customers that responds to the
> public with data larger than a FIN. Like getting their providers to more
> proactively police their spew, manufactures to stop making negligent
> devices, or implementing more intelligent filter communication so the only
> option doesn't begin with calling your provider and asking them over the
> phone to block X ip for you since you're off the internet.
>
> Maybe even look into liability laws for allowing said attacks to originate
> from your customers and not doing anything about it, or being manufacturer
> of said devices that harm others through their lack of due diligence
> implementing proper security. It's still way more effective than trying to
> fix the *last instance* of the problem, instead of it's reasons for
> enduring as an issue at a global scale.
>
> -Blake
>
>
> On Wed, Jul 31, 2013 at 3:46 PM, Dobbins, Roland  wrote:
>
>>
>> On Aug 1, 2013, at 3:11 AM, bottiger wrote:
>>
>> > The most disturbing part is the lack of logging.
>>
>> Flow telemetry can be of use in this instance.
>>
>> ---
>> Roland Dobbins  // 
>>
>>   Luck is the residue of opportunity and design.
>>
>>-- John Milton
>>
>>
>>

Re: SNMP DDoS: the vulnerability you might not know you have

[Blake Dunlap]

I bet blocking all SYN packets and non related flow UDP packets to
customers would be even more effective. Why don't we do that and be done
with it instead of playing whack a mole every 3 months when someone finds
some new service that was poorly designed so that it can be used to send a
flood?

Yes, I'm being sarcastic above.

This is hardly the first finger of death amplification attack, and I
strongly doubt it'll be the last. Years ago it was smurf, then Quake
servers, then DNS, then Battlefield boxes, etc etc. Now it seems to be snmp
and recently chargen, and tomorrow it'll be some peer 2 peer service, the
next day it'll be a voice app. It will never end, and breaking the internet
port by port doesn't do anything to make it better.

I've been the victim of week long DDoS attacks that took down our
upstreams, not to mention us, and I still maintain the above.

It works better to fix the design issues than to play whack a mole by
blocking every imaginable service to your customers that responds to the
public with data larger than a FIN. Like getting their providers to more
proactively police their spew, manufactures to stop making negligent
devices, or implementing more intelligent filter communication so the only
option doesn't begin with calling your provider and asking them over the
phone to block X ip for you since you're off the internet.

Maybe even look into liability laws for allowing said attacks to originate
from your customers and not doing anything about it, or being manufacturer
of said devices that harm others through their lack of due diligence
implementing proper security. It's still way more effective than trying to
fix the *last instance* of the problem, instead of it's reasons for
enduring as an issue at a global scale.

-Blake


On Wed, Jul 31, 2013 at 3:46 PM, Dobbins, Roland  wrote:

>
> On Aug 1, 2013, at 3:11 AM, bottiger wrote:
>
> > The most disturbing part is the lack of logging.
>
> Flow telemetry can be of use in this instance.
>
> ---
> Roland Dobbins  // 
>
>   Luck is the residue of opportunity and design.
>
>-- John Milton
>
>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Dobbins, Roland]

On Aug 1, 2013, at 3:11 AM, bottiger wrote:

> The most disturbing part is the lack of logging. 

Flow telemetry can be of use in this instance.

---
Roland Dobbins  // 

  Luck is the residue of opportunity and design.

   -- John Milton

Re: Hilton proxy issue

[Jay Moran]

I have BCC'd the likely appropriate Hilton contact for you on this response
so they can take a look at the NANOG emails below regarding their Internet
proxies to see if it looks like something they can assist with. They were
able to have some MTA issues corrected last time Hilton came up on the
NANOG list. Good luck!
--
Jay Moran
http://linked.com/in/jaycmoran

On Wed, Jul 31, 2013 at 4:16 PM, Grant Ridder wrote:

> Better yet, does anyone have any Hilton contacts they could pass my info
> to?
>
> -Grant
>
> On Wed, Jul 31, 2013 at 8:54 AM, Grant Ridder  >wrote:
>
> > Anyone from Hilton out there?  We are still having this issue.  It is not
> > a wayport address since I looked and they are not registered under
> Hilton's
> > name.
> >
> > -Grant
> >
> >
> > On Tue, Jul 16, 2013 at 1:17 PM, Grant Ridder  >wrote:
> >
> >> The requests are coming from 167.187.100.202 which is in a /16 assigned
> >> to Hilton.  As far as i know, the waypoint service has its own
> netblocks.
> >>
> >> -Grant
> >>
> >>
> >> On Tue, Jul 16, 2013 at 1:11 PM, Jared Mauch  >wrote:
> >>
> >>>
> >>> On Jul 16, 2013, at 3:44 PM, Grant Ridder 
> >>> wrote:
> >>>
> >>> > Hi,
> >>> >
> >>> > Anyone from Hilton Hotels NOC or related on here?  We are seeing
> their
> >>> > internet proxy doing weird things to http requests to servers at
> >>> $DAYJOB.
> >>>
> >>>
> >>> Many of the hilton properties have migrated to Wayport/"attwifi".  Are
> >>> you seeing the requests from AT&T/Wayport or from their corporate?
> >>>
> >>> (btw, if you're here and with wayport/attwifi, i would be interested in
> >>> chatting briefly with you).
> >>>
> >>> - Jared
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Warren Bailey]

Would it be possible to add SNMP to your (collective cable labs buddies) 
shapers and it would be taken care of prior to it leaving your network but 
after the cmts?


Sent from my Mobile Device.


 Original message 
From: "Livingood, Jason" 
Date: 07/31/2013 10:07 AM (GMT-08:00)
To: bottiger ,nanog@nanog.org
Subject: Re: SNMP DDoS: the vulnerability you might not know you have


A relevant paper was released by the BITAG, see 
http://www.bitag.org/report-snmp-ddos-attacks.php Section 7 includes 
recommendations.

See also this blog post I wrote one day short of a year ago that may be of 
interest: 
http://corporate.comcast.com/comcast-voices/taking-steps-to-prevent-unintentional-network-abuse

A remaining issue out there for the community is taking action to reduce 
spoofing. A related project is the Open Resolver Project at 
http://openresolverproject.org/.

- Jason



On 7/31/13 6:25 AM, "bottiger" 
mailto:bottige...@gmail.com>> wrote:

Before you skim past this email because you already read the Prolexic
report on it or some other article on the internet, there are 2
disturbing properties that I haven't found anywhere else online.

1) After sending abuse emails to many networks, we received many angry
replies that they monitored their traffic for days without seeing
anything (even as we were being attacked) and that their IPs were
spoofed and would block us for spamming them.

What we discovered was that their firewalls/routers/gateways coming
from vendors like Cisco and SonicWall apparently didn't record SNMP
traffic going in or out of themselves. We confirmed this multiple
times by running a query to an IP that was claimed to be clean and
watching the response come 10-60 seconds later because the device was
being so heavily abused.

2) SNMP reflection offers the largest amplification factor by far,
even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
68 byte query and received responses of up to 30,000 to 60,000 bytes.
The trick is to use GetBulkRequest to start enumerating from the first
OID and setting max repetitions to a large number. This is contrary to
the other articles online which suggest a much smaller amplification
factor with other queries.

This protocol is also prevalent in many devices ranging from routers
to printers.

To solve this problem you should block SNMP traffic coming from
outside your network and whitelist outside IPs that require it.

Re: SNMP DDoS: the vulnerability you might not know you have

[Warren Bailey]

Write into your TOS a block for SNMP. Deal with the whiners on a case by case 
basis. Problem solved.


Sent from my Mobile Device.


 Original message 
From: bottiger 
Date: 07/31/2013 1:13 PM (GMT-08:00)
To: Blake Dunlap 
Cc: nanog@nanog.org
Subject: Re: SNMP DDoS: the vulnerability you might not know you have


Public SNMP being exploited for 8000x amplification is a very serious
issue. It is
arguably worse than open email relays.

Not only does it expose critical information from your users
but it offers the largest possible amplified DDoS by far, likely
bigger than DNS when you take into account the amplification size and
ubiquity. It will also cause your user's device to lag.

The most disturbing part is the lack of logging. We have tried
reporting these exploited servers for many weeks and because of the
logging problem most of them never get shut down because they just
assume they were being spoofed. We even had replies threatening to
block us because they thought were because they couldn't see they
were sending anything. When we were reported chargen attacks we
had much more positive responses.

Maybe you could refine the block by denying SNMP requests with the
public string. As network operators some compromises must be
made for a problem of this magnitude instead of just saying that you
should only be the best dumb pipe you can be.

We have seen attacks large enough to disturb 10G uplinks so as network
administrators you should not ignore this issue because you think it
is a small problem affecting only end users. This will affect you once
more people figure out how to get 8000x amplification from it.

It is great news that Comcast is trying to proactively solve this
problem on their network and hope that more networks would follow
their example.

On Wed, Jul 31, 2013 at 8:24 AM, Blake Dunlap  wrote:
> Agreed, but progressively breaking every service on the internet at the edge
> because you think there might possibly be an issue just leads to bad places.
>
> Get better defaults sure, but don't slowly turn the internet into a cable
> distribution system because "they're just users". It's bad enough already,
> don't make it worse trying to solve every issue with the nuclear option
> before trying anything else.
>
> -Blake
>
>
> On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre 
> wrote:
>>
>> The problem isn't the people on this list leaving the public snmp
>> community on their devices, it's the vendors of home routers leaving it
>> there in their devices. Normal end users don't know or even care what snmp
>> is. (nor can we expect them too)
>>
>> A simple scan of a large cable/dsl ISP's address space will likely net you
>> tens of thousands of devices which respond to the "public" snmp community.
>>
>> Thomas
>>
>>
>>
>> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>>
>> >This looks like more a security issue with the devices, not border
>> >security
>> >issues.
>> >
>> >If you're seeing replies of that size, it means the devices themselves
>> > are
>> >set up to allow public queries of their information (not secured by even
>> >keys), which no one should be comfortable with. People should never be
>> >leaving the public access snmp strings on devices even if they are
>> >internal. Edge blocking just masks the real issue.
>> >
>> >
>> >-Blake
>> >
>> >
>> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
>> >
>> >> Before you skim past this email because you already read the Prolexic
>> >> report on it or some other article on the internet, there are 2
>> >> disturbing properties that I haven't found anywhere else online.
>> >>
>> >> 1) After sending abuse emails to many networks, we received many angry
>> >> replies that they monitored their traffic for days without seeing
>> >> anything (even as we were being attacked) and that their IPs were
>> >> spoofed and would block us for spamming them.
>> >>
>> >> What we discovered was that their firewalls/routers/gateways coming
>> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
>> >> traffic going in or out of themselves. We confirmed this multiple
>> >> times by running a query to an IP that was claimed to be clean and
>> >> watching the response come 10-60 seconds later because the device was
>> >> being so heavily abused.
>> >>
>> >> 2) SNMP reflection offers the largest amplification factor by far,
>> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
>> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
>> >> The trick is to use GetBulkRequest to start enumerating from the first
>> >> OID and setting max repetitions to a large number. This is contrary to
>> >> the other articles online which suggest a much smaller amplification
>> >> factor with other queries.
>> >>
>> >> This protocol is also prevalent in many devices ranging from routers
>> >> to printers.
>> >>
>> >> To solve this problem you should block SNMP traffic coming from
>> >> outside your networ

Re: Hilton proxy issue

[Grant Ridder]

Better yet, does anyone have any Hilton contacts they could pass my info to?

-Grant

On Wed, Jul 31, 2013 at 8:54 AM, Grant Ridder wrote:

> Anyone from Hilton out there?  We are still having this issue.  It is not
> a wayport address since I looked and they are not registered under Hilton's
> name.
>
> -Grant
>
>
> On Tue, Jul 16, 2013 at 1:17 PM, Grant Ridder wrote:
>
>> The requests are coming from 167.187.100.202 which is in a /16 assigned
>> to Hilton.  As far as i know, the waypoint service has its own netblocks.
>>
>> -Grant
>>
>>
>> On Tue, Jul 16, 2013 at 1:11 PM, Jared Mauch wrote:
>>
>>>
>>> On Jul 16, 2013, at 3:44 PM, Grant Ridder 
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > Anyone from Hilton Hotels NOC or related on here?  We are seeing their
>>> > internet proxy doing weird things to http requests to servers at
>>> $DAYJOB.
>>>
>>>
>>> Many of the hilton properties have migrated to Wayport/"attwifi".  Are
>>> you seeing the requests from AT&T/Wayport or from their corporate?
>>>
>>> (btw, if you're here and with wayport/attwifi, i would be interested in
>>> chatting briefly with you).
>>>
>>> - Jared
>>
>>
>>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[bottiger]

Public SNMP being exploited for 8000x amplification is a very serious
issue. It is
arguably worse than open email relays.

Not only does it expose critical information from your users
but it offers the largest possible amplified DDoS by far, likely
bigger than DNS when you take into account the amplification size and
ubiquity. It will also cause your user's device to lag.

The most disturbing part is the lack of logging. We have tried
reporting these exploited servers for many weeks and because of the
logging problem most of them never get shut down because they just
assume they were being spoofed. We even had replies threatening to
block us because they thought were because they couldn't see they
were sending anything. When we were reported chargen attacks we
had much more positive responses.

Maybe you could refine the block by denying SNMP requests with the
public string. As network operators some compromises must be
made for a problem of this magnitude instead of just saying that you
should only be the best dumb pipe you can be.

We have seen attacks large enough to disturb 10G uplinks so as network
administrators you should not ignore this issue because you think it
is a small problem affecting only end users. This will affect you once
more people figure out how to get 8000x amplification from it.

It is great news that Comcast is trying to proactively solve this
problem on their network and hope that more networks would follow
their example.

On Wed, Jul 31, 2013 at 8:24 AM, Blake Dunlap  wrote:
> Agreed, but progressively breaking every service on the internet at the edge
> because you think there might possibly be an issue just leads to bad places.
>
> Get better defaults sure, but don't slowly turn the internet into a cable
> distribution system because "they're just users". It's bad enough already,
> don't make it worse trying to solve every issue with the nuclear option
> before trying anything else.
>
> -Blake
>
>
> On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre 
> wrote:
>>
>> The problem isn't the people on this list leaving the public snmp
>> community on their devices, it's the vendors of home routers leaving it
>> there in their devices. Normal end users don't know or even care what snmp
>> is. (nor can we expect them too)
>>
>> A simple scan of a large cable/dsl ISP's address space will likely net you
>> tens of thousands of devices which respond to the "public" snmp community.
>>
>> Thomas
>>
>>
>>
>> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>>
>> >This looks like more a security issue with the devices, not border
>> >security
>> >issues.
>> >
>> >If you're seeing replies of that size, it means the devices themselves
>> > are
>> >set up to allow public queries of their information (not secured by even
>> >keys), which no one should be comfortable with. People should never be
>> >leaving the public access snmp strings on devices even if they are
>> >internal. Edge blocking just masks the real issue.
>> >
>> >
>> >-Blake
>> >
>> >
>> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
>> >
>> >> Before you skim past this email because you already read the Prolexic
>> >> report on it or some other article on the internet, there are 2
>> >> disturbing properties that I haven't found anywhere else online.
>> >>
>> >> 1) After sending abuse emails to many networks, we received many angry
>> >> replies that they monitored their traffic for days without seeing
>> >> anything (even as we were being attacked) and that their IPs were
>> >> spoofed and would block us for spamming them.
>> >>
>> >> What we discovered was that their firewalls/routers/gateways coming
>> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
>> >> traffic going in or out of themselves. We confirmed this multiple
>> >> times by running a query to an IP that was claimed to be clean and
>> >> watching the response come 10-60 seconds later because the device was
>> >> being so heavily abused.
>> >>
>> >> 2) SNMP reflection offers the largest amplification factor by far,
>> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
>> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
>> >> The trick is to use GetBulkRequest to start enumerating from the first
>> >> OID and setting max repetitions to a large number. This is contrary to
>> >> the other articles online which suggest a much smaller amplification
>> >> factor with other queries.
>> >>
>> >> This protocol is also prevalent in many devices ranging from routers
>> >> to printers.
>> >>
>> >> To solve this problem you should block SNMP traffic coming from
>> >> outside your network and whitelist outside IPs that require it.
>> >>
>> >>
>>
>

Re: GTT/Inteliquent/nLayer

[Richard A Steenbergen]

On Wed, Jul 31, 2013 at 09:28:50AM -0400, Tim Durack wrote:
> Any experience/comments on the GTT Global eXpress service? Looks
> interesting but odd. Why would I use a virtual IXP? Who participates?
> 
> Comments on-list or off-list are fine.

This was an old PacketExchange service, essentially just a single large 
VPLS-based global layer 2 virtual IXP service, which combined long-haul 
transport and multi-party interconnection. It's somewhat interesting as 
a concept (since I'm not aware of anyone else offering anything 
similar), but IMHO not the most practical thing in the world, which is 
why it hasn't really been promoted as a new product in many years. If 
you've heard differently, please contact me off-list. :)

-- 
Richard A Steenbergenhttp://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)

Re: ARIN WHOIS for leads

[Barry Shein]

On July 31, 2013 at 08:00 j...@west.net (Jay Hennigan) wrote:
 > 
 > It's surprising how people go out of their way to deny that it's a sales
 > call, and then start trying to sell something.

[NOTE: The anecdote is followed by some practical advice]



I had a guy call and tell the person who answered he was my brother
and there was a family emergency.

I don't have a brother.

I said put him through. He began a sales pitch.

That was quite a few years ago, he probably still talks about what
jerk I am and if so I am proud of it!





THAT SAID, beyond personal tastes, in this context there's really only
one substantive complaint:

Telemarketing info is PAID FOR, particularly in a ready to use list
form.

If they're scraping WHOIS etc for free that's a problem.

Lists can be protected by intellectual property law against such
abuse.

The usual method is to insert "ringers" which would be info which
points back at non-existant people with valid-looking contact
information.

If for example they called a phone number, or several, owned by ARIN
(or a service they employed) asking for James T Kirk or Diana Prince
then that would be a problem and should be logged.

One obvious response is to just bill them a reasonable telemarketing
list rental fee for the entire database and go from there.

Believe it or not this is well-trod ground, people steal or abuse
(e.g., resell w/o permission) telemarketing and mailing list info all
the time.



-- 
-Barry Shein

The World  | b...@theworld.com   | http://www.TheWorld.com
Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada
Software Tool & Die| Public Access Internet | SINCE 1989 *oo*

Re: SNMP DDoS: the vulnerability you might not know you have

[Livingood, Jason]

A relevant paper was released by the BITAG, see 
http://www.bitag.org/report-snmp-ddos-attacks.php Section 7 includes 
recommendations.

See also this blog post I wrote one day short of a year ago that may be of 
interest: 
http://corporate.comcast.com/comcast-voices/taking-steps-to-prevent-unintentional-network-abuse

A remaining issue out there for the community is taking action to reduce 
spoofing. A related project is the Open Resolver Project at 
http://openresolverproject.org/.

- Jason



On 7/31/13 6:25 AM, "bottiger" 
mailto:bottige...@gmail.com>> wrote:

Before you skim past this email because you already read the Prolexic
report on it or some other article on the internet, there are 2
disturbing properties that I haven't found anywhere else online.

1) After sending abuse emails to many networks, we received many angry
replies that they monitored their traffic for days without seeing
anything (even as we were being attacked) and that their IPs were
spoofed and would block us for spamming them.

What we discovered was that their firewalls/routers/gateways coming
from vendors like Cisco and SonicWall apparently didn't record SNMP
traffic going in or out of themselves. We confirmed this multiple
times by running a query to an IP that was claimed to be clean and
watching the response come 10-60 seconds later because the device was
being so heavily abused.

2) SNMP reflection offers the largest amplification factor by far,
even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
68 byte query and received responses of up to 30,000 to 60,000 bytes.
The trick is to use GetBulkRequest to start enumerating from the first
OID and setting max repetitions to a large number. This is contrary to
the other articles online which suggest a much smaller amplification
factor with other queries.

This protocol is also prevalent in many devices ranging from routers
to printers.

To solve this problem you should block SNMP traffic coming from
outside your network and whitelist outside IPs that require it.

Re: Revealed: NSA program collects 'nearly everything a user does on the internet'

[Warren Bailey]

And how many people utilize a VPN for site to site? You can convince me you can 
spin up an Ipsec connection, but at that point your "originating gateway" 
changed from your way to the Internet to the VPN's way. Either.. Way.. You 
still head out in clear channel Internet and get owned elsewhere. I can't see a 
giant "this doesn't work here" sign on much except for Tor.


Sent from my Mobile Device.


 Original message 
From: Chris Boyd 
Date: 07/31/2013 8:52 AM (GMT-08:00)
To: NANOG 
Subject: Re: Revealed: NSA program collects 'nearly everything a user does on 
the internet'



On Jul 31, 2013, at 10:26 AM, "<<\"tei''>>>"  
 wrote:

> - Have I read it correctly.  Can then break into a vpn connection,
> then leach documents that a german in pakistan is sending to his
> office in germany?

I would guess that it's becasuse many VPN services still support PPTP which can 
be attacked as outlined here:
http://www.schneier.com/paper-pptpv2.html

--Chris

Re: Revealed: NSA program collects 'nearly everything a user does on the internet'

[Ken Gilmour]

Don't forget Theo DeRaadt's email about IPSec!
http://marc.info/?l=openbsd-tech&m=129236621626462


On 31 July 2013 16:50, Chris Boyd  wrote:

>
> On Jul 31, 2013, at 10:26 AM, "<<\"tei''>>>"  <
> oscar.vi...@gmail.com> wrote:
>
> > - Have I read it correctly.  Can then break into a vpn connection,
> > then leach documents that a german in pakistan is sending to his
> > office in germany?
>
> I would guess that it's becasuse many VPN services still support PPTP
> which can be attacked as outlined here:
> http://www.schneier.com/paper-pptpv2.html
>
> --Chris
>
>
>

Re: Hilton proxy issue

[Grant Ridder]

Anyone from Hilton out there?  We are still having this issue.  It is not a
wayport address since I looked and they are not registered under Hilton's
name.

-Grant

On Tue, Jul 16, 2013 at 1:17 PM, Grant Ridder wrote:

> The requests are coming from 167.187.100.202 which is in a /16 assigned to
> Hilton.  As far as i know, the waypoint service has its own netblocks.
>
> -Grant
>
>
> On Tue, Jul 16, 2013 at 1:11 PM, Jared Mauch wrote:
>
>>
>> On Jul 16, 2013, at 3:44 PM, Grant Ridder 
>> wrote:
>>
>> > Hi,
>> >
>> > Anyone from Hilton Hotels NOC or related on here?  We are seeing their
>> > internet proxy doing weird things to http requests to servers at
>> $DAYJOB.
>>
>>
>> Many of the hilton properties have migrated to Wayport/"attwifi".  Are
>> you seeing the requests from AT&T/Wayport or from their corporate?
>>
>> (btw, if you're here and with wayport/attwifi, i would be interested in
>> chatting briefly with you).
>>
>> - Jared
>
>
>

Re: Revealed: NSA program collects 'nearly everything a user does on the internet'

[Chris Boyd]

On Jul 31, 2013, at 10:26 AM, "<<\"tei''>>>"  
 wrote:

> - Have I read it correctly.  Can then break into a vpn connection,
> then leach documents that a german in pakistan is sending to his
> office in germany?

I would guess that it's becasuse many VPN services still support PPTP which can 
be attacked as outlined here:
http://www.schneier.com/paper-pptpv2.html

--Chris

Re: SNMP DDoS: the vulnerability you might not know you have

[Enno Rey]

Hi,

On Wed, Jul 31, 2013 at 03:17:37PM +, Thomas St-Pierre wrote:
> The problem isn't the people on this list leaving the public snmp
> community on their devices, it's the vendors of home routers leaving it
> there in their devices. Normal end users don't know or even care what snmp
> is. (nor can we expect them too)
> 
> A simple scan of a large cable/dsl ISP's address space will likely net you
> tens of thousands of devices which respond to the "public" snmp community.

I can confirm this.
we did some enumeration (and discussed the said amplification attack) here:
http://conference.hitb.org/hitbsecconf2007dubai/materials/D1%20-%20Enno%20Rey%20-%20Digging%20into%20SNMP%202007%20-%20An%20Excercise%20on%20Breaking%20Networks.pdf

at the time once you scanned "typical broadband segments" of major European 
carriers, pretty much every address responding to a ping had SNMP "public" 
also. 

we gave the talk several times and demoed the amplification attack (with a 
slightly modified version of this tool: 
https://www.ernw.de/download/snmpattack.pl) against some of our systems, 
abusing $SOME_RANDOM_SEGMENT as amplifiers (we asked to stop [camera] recording 
in those cases where the talks were recorded) and it worked pretty much all the 
time (~20:1 ratio, initiated from the respective conferences' hotel wifi).

thanks

Enno




> 
> Thomas
> 
> 
> 
> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
> 
> >This looks like more a security issue with the devices, not border
> >security
> >issues.
> >
> >If you're seeing replies of that size, it means the devices themselves are
> >set up to allow public queries of their information (not secured by even
> >keys), which no one should be comfortable with. People should never be
> >leaving the public access snmp strings on devices even if they are
> >internal. Edge blocking just masks the real issue.
> >
> >
> >-Blake
> >
> >
> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
> >
> >> Before you skim past this email because you already read the Prolexic
> >> report on it or some other article on the internet, there are 2
> >> disturbing properties that I haven't found anywhere else online.
> >>
> >> 1) After sending abuse emails to many networks, we received many angry
> >> replies that they monitored their traffic for days without seeing
> >> anything (even as we were being attacked) and that their IPs were
> >> spoofed and would block us for spamming them.
> >>
> >> What we discovered was that their firewalls/routers/gateways coming
> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
> >> traffic going in or out of themselves. We confirmed this multiple
> >> times by running a query to an IP that was claimed to be clean and
> >> watching the response come 10-60 seconds later because the device was
> >> being so heavily abused.
> >>
> >> 2) SNMP reflection offers the largest amplification factor by far,
> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> >> The trick is to use GetBulkRequest to start enumerating from the first
> >> OID and setting max repetitions to a large number. This is contrary to
> >> the other articles online which suggest a much smaller amplification
> >> factor with other queries.
> >>
> >> This protocol is also prevalent in many devices ranging from routers
> >> to printers.
> >>
> >> To solve this problem you should block SNMP traffic coming from
> >> outside your network and whitelist outside IPs that require it.
> >>
> >>
> 
> 

-- 
Enno Rey

ERNW GmbH - Carl-Bosch-Str. 4 - 69115 Heidelberg - www.ernw.de
Tel. +49 6221 480390 - Fax 6221 419008 - Cell +49 174 3082474

Handelsregister Mannheim: HRB 337135
Geschaeftsfuehrer: Enno Rey

Troopers 2013 Videos online: 
http://www.youtube.com/user/TROOPERScon?feature=watch

===
Blog: www.insinuator.net || Conference: www.troopers.de
===

RE: SNMP DDoS: the vulnerability you might not know you have

[James Braunegg]

These attacks are known as SAD attacks ;( ... yes very SAD ;(

SNMP Amplification DDoS


Kindest Regards 

James Braunegg
P:  1300 769 972  |  M:  0488 997 207 |  D:  (03) 9751 7616
E:   james.braun...@micron21.com  |  ABN:  12 109 977 666   
W:  www.micron21.com/tv-hosting  T: @micron21



This message is intended for the addressee named above. It may contain 
privileged or confidential information. If you are not the intended recipient 
of this message you must not use, copy, distribute or disclose it to anyone 
other than the addressee. If you have received this message in error please 
return the message to the sender by replying to it and then delete the message 
from your computer.


-Original Message-
From: Blake Dunlap [mailto:iki...@gmail.com] 
Sent: Thursday, August 01, 2013 1:25 AM
To: Thomas St-Pierre
Cc: nanog@nanog.org
Subject: Re: SNMP DDoS: the vulnerability you might not know you have

Agreed, but progressively breaking every service on the internet at the
edge because you think there might possibly be an issue just leads to bad
places.

Get better defaults sure, but don't slowly turn the internet into a cable
distribution system because "they're just users". It's bad enough already,
don't make it worse trying to solve every issue with the nuclear option
before trying anything else.

-Blake


On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre wrote:

> The problem isn't the people on this list leaving the public snmp
> community on their devices, it's the vendors of home routers leaving it
> there in their devices. Normal end users don't know or even care what snmp
> is. (nor can we expect them too)
>
> A simple scan of a large cable/dsl ISP's address space will likely net you
> tens of thousands of devices which respond to the "public" snmp community.
>
> Thomas
>
>
>
> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>
> >This looks like more a security issue with the devices, not border
> >security
> >issues.
> >
> >If you're seeing replies of that size, it means the devices themselves are
> >set up to allow public queries of their information (not secured by even
> >keys), which no one should be comfortable with. People should never be
> >leaving the public access snmp strings on devices even if they are
> >internal. Edge blocking just masks the real issue.
> >
> >
> >-Blake
> >
> >
> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
> >
> >> Before you skim past this email because you already read the Prolexic
> >> report on it or some other article on the internet, there are 2
> >> disturbing properties that I haven't found anywhere else online.
> >>
> >> 1) After sending abuse emails to many networks, we received many angry
> >> replies that they monitored their traffic for days without seeing
> >> anything (even as we were being attacked) and that their IPs were
> >> spoofed and would block us for spamming them.
> >>
> >> What we discovered was that their firewalls/routers/gateways coming
> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
> >> traffic going in or out of themselves. We confirmed this multiple
> >> times by running a query to an IP that was claimed to be clean and
> >> watching the response come 10-60 seconds later because the device was
> >> being so heavily abused.
> >>
> >> 2) SNMP reflection offers the largest amplification factor by far,
> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> >> The trick is to use GetBulkRequest to start enumerating from the first
> >> OID and setting max repetitions to a large number. This is contrary to
> >> the other articles online which suggest a much smaller amplification
> >> factor with other queries.
> >>
> >> This protocol is also prevalent in many devices ranging from routers
> >> to printers.
> >>
> >> To solve this problem you should block SNMP traffic coming from
> >> outside your network and whitelist outside IPs that require it.
> >>
> >>
>
>

Re: Revealed: NSA program collects 'nearly everything a user does on the internet'

[Jorge Amodio]

Interesting that they are showing screen captures of a ppt file.

-Jorge

On Jul 31, 2013, at 9:46 AM, Warren Bailey 
 wrote:

> Tin foil hat Wednesday, limited supplies.
> 
> Revealed: NSA program collects 'nearly everything a user does on the internet'
> 
> http://gu.com/p/3hy4h

Re: Revealed: NSA program collects 'nearly everything a user does on the internet'

[<<"tei''>>>]

On 31 July 2013 16:46, Warren Bailey
 wrote:
> Tin foil hat Wednesday, limited supplies.
>
> Revealed: NSA program collects 'nearly everything a user does on the internet'
>
> http://gu.com/p/3hy4h
>

 - Have I read it correctly.  Can then break into a vpn connection,
then leach documents that a german in pakistan is sending to his
office in germany?
 - So excel documents store MAC address?... time to set them to random
numbers :D
 - What is the red dots in the bottom of the map? satellites?  penguin
powered servers on the south pole?
 - The document make it looks like this exist to spy religious
terrorist and industrial espionage. But who know.   Woah, thats a lot
of red dots in europe. Must be to protect the europeans.



-- 
--
ℱin del ℳensaje.

Re: SNMP DDoS: the vulnerability you might not know you have

[Blake Dunlap]

Agreed, but progressively breaking every service on the internet at the
edge because you think there might possibly be an issue just leads to bad
places.

Get better defaults sure, but don't slowly turn the internet into a cable
distribution system because "they're just users". It's bad enough already,
don't make it worse trying to solve every issue with the nuclear option
before trying anything else.

-Blake


On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre wrote:

> The problem isn't the people on this list leaving the public snmp
> community on their devices, it's the vendors of home routers leaving it
> there in their devices. Normal end users don't know or even care what snmp
> is. (nor can we expect them too)
>
> A simple scan of a large cable/dsl ISP's address space will likely net you
> tens of thousands of devices which respond to the "public" snmp community.
>
> Thomas
>
>
>
> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>
> >This looks like more a security issue with the devices, not border
> >security
> >issues.
> >
> >If you're seeing replies of that size, it means the devices themselves are
> >set up to allow public queries of their information (not secured by even
> >keys), which no one should be comfortable with. People should never be
> >leaving the public access snmp strings on devices even if they are
> >internal. Edge blocking just masks the real issue.
> >
> >
> >-Blake
> >
> >
> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
> >
> >> Before you skim past this email because you already read the Prolexic
> >> report on it or some other article on the internet, there are 2
> >> disturbing properties that I haven't found anywhere else online.
> >>
> >> 1) After sending abuse emails to many networks, we received many angry
> >> replies that they monitored their traffic for days without seeing
> >> anything (even as we were being attacked) and that their IPs were
> >> spoofed and would block us for spamming them.
> >>
> >> What we discovered was that their firewalls/routers/gateways coming
> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
> >> traffic going in or out of themselves. We confirmed this multiple
> >> times by running a query to an IP that was claimed to be clean and
> >> watching the response come 10-60 seconds later because the device was
> >> being so heavily abused.
> >>
> >> 2) SNMP reflection offers the largest amplification factor by far,
> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> >> The trick is to use GetBulkRequest to start enumerating from the first
> >> OID and setting max repetitions to a large number. This is contrary to
> >> the other articles online which suggest a much smaller amplification
> >> factor with other queries.
> >>
> >> This protocol is also prevalent in many devices ranging from routers
> >> to printers.
> >>
> >> To solve this problem you should block SNMP traffic coming from
> >> outside your network and whitelist outside IPs that require it.
> >>
> >>
>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Thomas St-Pierre]

The problem isn't the people on this list leaving the public snmp
community on their devices, it's the vendors of home routers leaving it
there in their devices. Normal end users don't know or even care what snmp
is. (nor can we expect them too)

A simple scan of a large cable/dsl ISP's address space will likely net you
tens of thousands of devices which respond to the "public" snmp community.

Thomas



On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:

>This looks like more a security issue with the devices, not border
>security
>issues.
>
>If you're seeing replies of that size, it means the devices themselves are
>set up to allow public queries of their information (not secured by even
>keys), which no one should be comfortable with. People should never be
>leaving the public access snmp strings on devices even if they are
>internal. Edge blocking just masks the real issue.
>
>
>-Blake
>
>
>On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
>
>> Before you skim past this email because you already read the Prolexic
>> report on it or some other article on the internet, there are 2
>> disturbing properties that I haven't found anywhere else online.
>>
>> 1) After sending abuse emails to many networks, we received many angry
>> replies that they monitored their traffic for days without seeing
>> anything (even as we were being attacked) and that their IPs were
>> spoofed and would block us for spamming them.
>>
>> What we discovered was that their firewalls/routers/gateways coming
>> from vendors like Cisco and SonicWall apparently didn't record SNMP
>> traffic going in or out of themselves. We confirmed this multiple
>> times by running a query to an IP that was claimed to be clean and
>> watching the response come 10-60 seconds later because the device was
>> being so heavily abused.
>>
>> 2) SNMP reflection offers the largest amplification factor by far,
>> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
>> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
>> The trick is to use GetBulkRequest to start enumerating from the first
>> OID and setting max repetitions to a large number. This is contrary to
>> the other articles online which suggest a much smaller amplification
>> factor with other queries.
>>
>> This protocol is also prevalent in many devices ranging from routers
>> to printers.
>>
>> To solve this problem you should block SNMP traffic coming from
>> outside your network and whitelist outside IPs that require it.
>>
>>

Re: ARIN WHOIS for leads

[Jay Hennigan]

On 7/26/13 8:32 AM, Joel M Snyder wrote:

> I also don't see the problem of cold calling when it's obviously for a
> service or product that I am interested in, just as I don't see the
> problem of cold snail-mailing for the same services.  I'm in business,
> and I expect other businesses to try and market to me.  

I have a low tolerance for telemarketers, especially those who scrape
technical lists or databases.  One test I have is to immediately ask,
"Is this a sales call?"

Anything other than a forthright "Yes" gets nowhere.  Weasel words don't
count.  If the first thing they tell me is a lie, I don't want to do
business with them.  If they're honest I might give them a minute or two
to pitch their wares.

It's surprising how people go out of their way to deny that it's a sales
call, and then start trying to sell something.

--
Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net
Impulse Internet Service  -  http://www.impulse.net/
Your local telephone and internet company - 805 884-6323 - WB6RDV

Re: SNMP DDoS: the vulnerability you might not know you have

[Blake Dunlap]

This looks like more a security issue with the devices, not border security
issues.

If you're seeing replies of that size, it means the devices themselves are
set up to allow public queries of their information (not secured by even
keys), which no one should be comfortable with. People should never be
leaving the public access snmp strings on devices even if they are
internal. Edge blocking just masks the real issue.


-Blake


On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:

> Before you skim past this email because you already read the Prolexic
> report on it or some other article on the internet, there are 2
> disturbing properties that I haven't found anywhere else online.
>
> 1) After sending abuse emails to many networks, we received many angry
> replies that they monitored their traffic for days without seeing
> anything (even as we were being attacked) and that their IPs were
> spoofed and would block us for spamming them.
>
> What we discovered was that their firewalls/routers/gateways coming
> from vendors like Cisco and SonicWall apparently didn't record SNMP
> traffic going in or out of themselves. We confirmed this multiple
> times by running a query to an IP that was claimed to be clean and
> watching the response come 10-60 seconds later because the device was
> being so heavily abused.
>
> 2) SNMP reflection offers the largest amplification factor by far,
> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> The trick is to use GetBulkRequest to start enumerating from the first
> OID and setting max repetitions to a large number. This is contrary to
> the other articles online which suggest a much smaller amplification
> factor with other queries.
>
> This protocol is also prevalent in many devices ranging from routers
> to printers.
>
> To solve this problem you should block SNMP traffic coming from
> outside your network and whitelist outside IPs that require it.
>
>

Revealed: NSA program collects 'nearly everything a user does on the internet'

[Warren Bailey]

Tin foil hat Wednesday, limited supplies.

Revealed: NSA program collects 'nearly everything a user does on the internet'

http://gu.com/p/3hy4h



Sent from my Mobile Device.

SNMP DDoS: the vulnerability you might not know you have

[bottiger]

Before you skim past this email because you already read the Prolexic
report on it or some other article on the internet, there are 2
disturbing properties that I haven't found anywhere else online.

1) After sending abuse emails to many networks, we received many angry
replies that they monitored their traffic for days without seeing
anything (even as we were being attacked) and that their IPs were
spoofed and would block us for spamming them.

What we discovered was that their firewalls/routers/gateways coming
from vendors like Cisco and SonicWall apparently didn't record SNMP
traffic going in or out of themselves. We confirmed this multiple
times by running a query to an IP that was claimed to be clean and
watching the response come 10-60 seconds later because the device was
being so heavily abused.

2) SNMP reflection offers the largest amplification factor by far,
even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
68 byte query and received responses of up to 30,000 to 60,000 bytes.
The trick is to use GetBulkRequest to start enumerating from the first
OID and setting max repetitions to a large number. This is contrary to
the other articles online which suggest a much smaller amplification
factor with other queries.

This protocol is also prevalent in many devices ranging from routers
to printers.

To solve this problem you should block SNMP traffic coming from
outside your network and whitelist outside IPs that require it.

Re: File transfer speed between Hong Kong and Johannesburg, South Africa

[Luan Nguyen]

Just a note on this thread, we got everything sorted out. There was a
little asymmetric routing going on, but the great folks at HGC was very
quick in helping us fix this.
We had some problem with HGC support at the Hutch before, but they are
great and fast now. At the other end in Johannesburg, we have our sister
company and Ruhann was great with helping us out.
In the end, with the Riverbed we get 1.5MBytes/sec bidirectional for SMB2
(active/active SQL database log shipping). With High-Speed TCP setting, we
could get to 7MBytes/sec.

Regards,

-Luan


On Thu, Jul 11, 2013 at 9:47 PM, Larry Sheldon  wrote:

> On 7/11/2013 10:32 AM, Clayton Zekelman wrote:
>
>>
>> It all depends on the air speed velocity of an unladen swallow, and
>> varies if it is African or Eurpoean.
>>
>> In all seriousness, you need to know the speed and latency of the link
>> before that question can be answered.
>>
>> At 10:04 AM 11/07/2013, Luan Nguyen wrote:
>>
>>> Hello folks,
>>>
>>> Does anyone know what's the average speed for windows file transferring
>>> (SMB2) between Hong Kong and Johannesburg?
>>> Any guide on how to calculate/estimate this?
>>>
>>
> An old-timers-uncalibrated-guess"  If you are going to do large transfers
> using a protocol designed (for want of a term) for local area networks with
> near-fault free performance over a long multihop network, you are not going
> to like it.
>
> What criteria drive the selection of such an unlikely protocol?
>
>
> --
> Requiescas in pace o email   Two identifying characteristics
> of System Administrators:
> Ex turpi causa non oritur actio  Infallibility, and the ability to
> learn from their mistakes.
>   (Adapted from Stephen Pinker)
>
>

GTT/Inteliquent/nLayer

[Tim Durack]

Any experience/comments on the GTT Global eXpress service? Looks
interesting but odd. Why would I use a virtual IXP? Who participates?

Comments on-list or off-list are fine.

-- 
Tim:>

Re: vodafone contact

[Christopher Morrow]

got an example item to test with too? I'm sure they'll want to see that as well.

On Tue, Jul 30, 2013 at 3:08 PM, ryanL  wrote:
> the common transit point for this problem is vodafone backone:
>
> aut-num:AS3209
> as-name:VODANET
>
>
> On Tue, Jul 30, 2013 at 2:13 PM, Nick Hilliard  wrote:
>
>> On 30/07/2013 18:34, ryanL wrote:
>> > anyone hanging out from vodafone in europe? or anyone know someone over
>> at
>> > vodafone? we are having goofy issues with mobile clients on your LTE
>> > network. we're having to dump mtu and advmss a whole bunch to make things
>> > work. wondering if you'd be willing to chat offline.
>>
>> "vodafone europe" is mostly run on a per country basis.  You'll need to
>> specify which asn + country you're talking about
>>
>> Nick
>>
>>

Re: which firewall product?

[Christopher Morrow]

On Tue, Jul 30, 2013 at 6:57 PM, Owen DeLong  wrote:

> I believe Bill is looking for DPI on forwarded traffic and not to decapsulate 
> the traffic prior to inspection.

oh! dpi? just use sandvine? comcast says that the work well...