Re: Microsoft SNDS contact with a clue?

[Brian Rak via NANOG]

Unfortunately, that's the generic SNDS support email I have been talking 
to.  They don't appear to be able to actually fix anything.


On 6/25/2020 3:03 PM, Udeme Ukutt wrote:
Brian, try msn-s...@microsoft.com <mailto:msn-s...@microsoft.com>. 
It's been awhile I've had to use it, but it's worth a shot.


Udeme
Postmaster at LinkedIn

On Thu, Jun 25, 2020 at 12:00 PM Brian Rak via NANOG <mailto:nanog@nanog.org>> wrote:


Is there anyone around from Microsoft that can help me with a SNDS
verification issue?

I'm having problems where the verification system is trying to use
the
wrong WHOIS server, and the responses I'm getting from support don't
really indicate they understand the issue.

I also have a different problem where I have a /17 and support keeps
telling me I need to go and verify all the /23's within it...
which is
going to take quite awhile.

Microsoft SNDS contact with a clue?

[Brian Rak via NANOG]

Is there anyone around from Microsoft that can help me with a SNDS 
verification issue?


I'm having problems where the verification system is trying to use the 
wrong WHOIS server, and the responses I'm getting from support don't 
really indicate they understand the issue.


I also have a different problem where I have a /17 and support keeps 
telling me I need to go and verify all the /23's within it... which is 
going to take quite awhile.

Re: [EXTERNAL] Re: Microsoft SNDS contact

[Brian Rak]

Yea, that's the email we've been using (that's trying to tell us to just 
split it into /24s)


On 7/3/2019 10:27 AM, Udeme Ukutt wrote:
Hey Brian - try msn-s...@microsoft.com 
<mailto:msn-s...@microsoft.com>. IIRC that's more geared towards JMRP, 
but I think there's a chance.


Udeme
Postmaster at Wish

On Wed, Jul 3, 2019 at 10:14 AM Brian Rak <mailto:b...@gameservers.com>> wrote:



On 7/3/2019 10:09 AM, Hansen, Christoffer wrote:
> On 03/07/2019 15:50, Hansen, Christoffer wrote:
>>
https://sendersupport.olc.protection.outlook.com/snds/addnetwork.aspx
> E.g. with asn 20473. Key that in. I can select the address
fetched from
> a background WHOIS lookup by MS Smart Network Data Service. For the
> confirmation email to be sent to.
>

We've tried this approach in the past, but it ends up dragging in
a lot
of IP space that we're announcing on behalf of customers. This is
less
then ideal, as then we either have to go back and manually remove all
the customer owned IP space, or deal with a bunch of noise from it.
We'd be willing to accept that as a solution if it were a one-off
thing,
but it's a lot of extra work to do every time we acquire more IP
space.

Re: [EXTERNAL] Re: Microsoft SNDS contact

[Brian Rak]

On 7/3/2019 10:09 AM, Hansen, Christoffer wrote:

On 03/07/2019 15:50, Hansen, Christoffer wrote:

https://sendersupport.olc.protection.outlook.com/snds/addnetwork.aspx

E.g. with asn 20473. Key that in. I can select the address fetched from
a background WHOIS lookup by MS Smart Network Data Service. For the
confirmation email to be sent to.



We've tried this approach in the past, but it ends up dragging in a lot 
of IP space that we're announcing on behalf of customers. This is less 
then ideal, as then we either have to go back and manually remove all 
the customer owned IP space, or deal with a bunch of noise from it.  
We'd be willing to accept that as a solution if it were a one-off thing, 
but it's a lot of extra work to do every time we acquire more IP space.

Microsoft SNDS contact

[Brian Rak]

We've been trying to get SNDS access for our IP space, and we keep 
running into issues where the SNDS site is unable to determine what 
emails it should use to authorize access.  SNDS support has so far been 
very unhelpful, they keep trying to tell us to submit the space as 
individual /24's, which is less then helpful when we have multiple /18s 
we're trying to set up.


Does anyone have a contact at Microsoft that can help?

Re: softlayer.com

[Brian Rak]

I've been trying to reach them regarding an abuse issue, and have 
similarly had no actual luck in reaching their abuse/noc contacts.


On 3/21/2019 9:07 PM, and...@paolucci.ca wrote:
SoftLayer was aquirred by IBM, maybe reaching out to their NOC or 
support would be fruitful. IBM's DNS team is indeed mentioned in 
SoftLayers WHOIS info.


Have you attempted email the addresses listed in the WHOIS for their ASN?

network:Tech-Contact;I:sysadm...@softlayer.com  
network:Abuse-Contact;I:ab...@softlayer.com  
network:Updated-By:ipad...@softlayer.com  

*Registrant Contact*
Registrant Name
Domain Administrator
Registrant Organization
Softlayer Technologies, Inc.
Registrant Street
4849 Alpha Road
Registrant City
Dallas
Registrant State/Province
TX
Registrant Postal Code
75244
Registrant Country
USUnited States
Registrant Phone
+1.2144420600
Registrant Email
bjohn...@softlayer.com 

*Administrative Contact*
Admin Name
Grace Micewicz
Admin Organization
International Business Machines Corporation
Admin Street
New Orchard Road
Admin City
Armonk
Admin State/Province
NY
Admin Postal Code
10504
Admin Country
USUnited States
Admin Phone
+1.9147654227
Admin Fax
+1.9147654370
Admin Email
dns...@us.ibm.com 


Regards.
Andrew Paolucci



‐‐‐ Original Message ‐‐‐
On Thursday, March 21, 2019 3:39 PM, John Alcock  wrote:


Still looking for anyone from softlayer.com 

It has been a challenge.  Anything hosted by softlayer.com 
 is being blocked.


Here is a small list so far

windowbook.tpondemand.com 
ahainstructornetwork.americanheart.org 


clover.com 
Cebroker.com
Softlayer.com
indeed.com  & Enforce Staffing

It is growing every day.

John


On Wed, Mar 20, 2019 at 12:35 PM John Alcock > wrote:


Afternoon,

Thought I would start a new thread.  After researching,
traceroutes, etc, I think I found my problem.

9 out of the 10 sites that subscribers on my new block is being
hosted by softlayer.

Anyone on the list have contacts with softlayer. Right now I have
an email to abuse.  The support line will not help me out.

John

Level3 IRR contact

[Brian Rak]

I'm trying to get some old IRR objects removed from the LEVEL3 database, 
and not having much luck.


Their support guys silently closed my ticket and then had our account 
manager email us directly basically saying "we don't what you want us to 
do".


I used to use routing@level3 to get this done, however they don't seem 
to reply anymore.


http://www.irr.net/docs/list.html directs me to r...@level3.net, which has 
an autoreply that says "open a ticket"

Re: ALTDB - Getting records removed

[Brian Rak]

Are you referring to auto-dbm@ email, or the db-admin@ one?  I emailed 
db-admin@ about 15 hours ago, and haven't heard back (although it didn't 
bounce this time!)  Not sure what sort of response time to expect from a 
free service though.



On 5/16/2018 12:17 PM, mike.l...@gmail.com wrote:

As stated yesterday, email was fixed on AltDB yesterday. Please try again.

Thanks,
Mike


On May 16, 2018, at 08:55, Delacruz, Anthony B 
 wrote:

Ditto also interested have dozens of old entries from previous delegations would 
like to see cleaned up but my google-foo tells me it's been a nonresponsive black 
hole several years now that probably should just go away if it's not going to be 
maintained properly. I think my favorite is the "Is anyone still maintaining 
altdb.net? thread from April 2011.

-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of John Hurley
Sent: Saturday, May 12, 2018 11:16 AM
To: nanog@nanog.org
Subject: ALTDB - Getting records removed

Hi All,

Recently acquired a new 2-byte AS number from ARIN. It had a previous owner
whom had records setup at ALTDB.

I've sent emails to request removal but haven't heard anything back.

Any tips or a different venue I can use to get in touch with the altdb
folks?


This communication is the property of CenturyLink and may contain confidential 
or privileged information. Unauthorized use of this communication is strictly 
prohibited and may be unlawful. If you have received this communication in 
error, please immediately notify the sender by reply e-mail and destroy all 
copies of the communication and any attachments.

AltDB bouncing emails

[Brian Rak]

I've been trying to get some super old entries removed from altdb, 
however the db-admin email bounces:


The mail system : host 
pobox.rubinbroadcasting.com[65.50.205.32] said: 550

    5.7.1 Unable to relay (in reply to RCPT TO command)

Is there another contact here?


Also, if anyone from Internap/Voxel.net is around... I've been trying to 
find someone with a clue to remove some IRR entries, ref ticket 3190091

Centurylink SOC contact?

[Brian Rak]

Does anyone have a contact for the SOC at centurylink?  I've tried 
soc@centurylink and noc@centurylink, with no answer.


For whatever reason, they're mangling IP address in abuse reports, which 
requires us to manually review every report.  We'd really like them to 
stop, and just include the IP address in the body of the report.


They seem to be the only ones that do this, pretty much all the other 
reports we get list a normal IP address.

Re: PlayStationNetwork blocking of CGNAT public addresses

[Brian Rak]

Single IP per email: automated, zero time at all.

Multiple IPs per email: manual process, minutes per IP.


On 9/22/2016 9:34 AM, Suresh Ramasubramanian wrote:
Considering that there are likely to be many such emails - just how 
much time is it going to take your abuse desk staffer to just parse 
out those IPs from whatever log that they send you?


And how much time would processing say 50 individual emails take 
compared to 50 IPs in a single email?


--srs

On 22-Sep-2016, at 6:58 PM, Brian Rak <b...@gameservers.com 
<mailto:b...@gameservers.com>> wrote:


We've also started ignoring their abuse emails, for the same reason. 
 Their abuse emails at one point contained the line:


> P.S. If you would prefer an individual email for each IP address on this 
list, please let us know.

But, they didn't respond after we contacted them requesting it (and 
that line has since been removed).

Re: PlayStationNetwork blocking of CGNAT public addresses

[Brian Rak]

On 9/22/2016 8:10 AM, Baldur Norddahl wrote:

On 22 September 2016 at 10:42, Alexander Maassen 
wrote:


So you ignore/don't deal with the abuse coz it's shipped in a format you
refuse to handle?

And you don't even bother telling the reporter you would like it in a per
ip format? Or make attempts to make it work the way they report it (split
out the ip's and modify the to be forwarded mail to only contain the ip's
belonging to that customer)


You will have to remember that these are automated mails from the reporter.
If I write them back it goes into their bit bucket, because they do not
really care enough to bother replying. I am betting they are sending out
thousands mails each day and they can not handle manually replying to all
of that. In the same way we receive a large amount of automated mail so we
have to be able to handle it automatically. Send me something sane and I
will make a script that forwards it. Send me something unusable and I wont
- but I will not do manual handling of your automated mail.

All I am trying to do here is tell people that send abuse mails not to
combine multiple abuse complaints in one mail, because that makes it harder
for everybody and makes it more likely that your mail will be dropped as
too much work. Double so if your abuse mails is from an automated system,
because I will try to match your automated system with my own. However it
is much harder to make a system that can edit your complaint and duplicate
it to several recipients, than it is to run a simple filter that just
forwards the mail as is.

As to PSN they will usually send multiple mails if the abuse is ongoing. At
some point they will send a mail with just one IP and that one gets
forwarded. So we are dropping some of the mails, but the users eventually
get notified anyway. It is not ideal but it works.

Regards,

Baldur


We've also started ignoring their abuse emails, for the same reason.  
Their abuse emails at one point contained the line:


> P.S. If you would prefer an individual email for each IP address on 
this list, please let us know.


But, they didn't respond after we contacted them requesting it (and that 
line has since been removed).

Re: Comcast postmaster?

[Brian Rak]

Taken care of, thanks!


On 7/11/2016 2:46 PM, Brian Rak wrote:
Is there anyone here that can put me in touch with a Comcast mail 
server administrator?  It seems that they've firewalled off some of 
our IPv6 space, and I can't seem to find any contact information.


Interestingly, I can't even fill out their blocklist removal form, 
because it only accepts IPv4 addresses.

Comcast postmaster?

[Brian Rak]

Is there anyone here that can put me in touch with a Comcast mail server 
administrator?  It seems that they've firewalled off some of our IPv6 
space, and I can't seem to find any contact information.


Interestingly, I can't even fill out their blocklist removal form, 
because it only accepts IPv4 addresses.

RADb Outage?

[Brian Rak]

whois.radb.net seems to have been down since sometime last night, has 
anyone else seen problems with this?


It seems the web interface still works, but that's not very useful for 
scripts.

Re: EyeBall View

[Brian Rak]

So you've invented RIPE ATLAS?

On 10/25/2015 3:49 PM, Dovid Bender wrote:

All,

I had an idea to create a product where we would have a host on every EyeBall 
network. Customers could then connect to these hosts and check connectivity 
back to their network. For instance you may want to see what the speed is like 
from CableVision in central NJ to your network in South Florida or the latency 
etc. I go large scale I wanted to know how much demand there was for such a 
service.


Regards,

Dovid

Re: Multiple vendors' IPv6 issues

[Brian Rak]

On 5/27/2015 3:20 PM, Jared Mauch wrote:

On Tue, May 26, 2015 at 04:19:25PM -0700, David Sotnick wrote:

Hi NANOG,

The company I work for has no business case for being on the IPv6-Internet.
However, I am an inquisitive person and I am always looking to learn new
things, so about 3 years ago I started down the IPv6 path. This was early
2012.

Fast forward to today. We have a /44 presence for our company's multiple
sites; All our desktop computers have been on the IPv6 Internet since June,
2012 and we have a few s in our external DNS for some key services —
and, there have been bugs. *Lots* of bugs.

Now, maybe (_maybe_) I can have some sympathy for smaller network companies
(like Arista Networks at the time) to not quite have their act together as
far as IPv6 goes, but for larger, well-established companies to still have
critical IPv6 bugs is just inexcusable!

My current favorites are:

https://tools.cisco.com/bugsearch/bug/CSCut62344

Which doesn't allow you to see the neighbors on an interface.  this is fun
when diagnosing qemu/kvm issues with the macvtap and hosts with ipv6.
turns out you to 'fix it' you need to make the macvtap interface promisc
as the icmpv6 messages don't make it through the macvtap driver to the VM
breaking neighbor discovery.
You don't need full promisc mode, just the (poorly documented) 
allmulticast option (ip link set dev $macvtap allmulticast on)

Re: Comcast residential DNS contact

[Brian Rak]

Shouldn't everyone be on IPv6 these days anyway ;)

On 12/3/2014 10:28 AM, Jared Mauch wrote:

So have A record queries. Do you filter those as well?

Jared Mauch


On Dec 3, 2014, at 9:08 AM, Stephen Satchell l...@satchell.net wrote:


On 12/03/2014 04:04 AM, Niels Bakker wrote:
* shortdudey...@gmail.com (Grant Ridder) [Wed 03 Dec 2014, 12:54 CET]:

Both of Google’s public DNS servers return complete results every time
and one of the two comcast ones works fine.

If this is working by design, can you provide the RFC with that info?

An ANY query will typically return only what's already in the cache.  So
if you ask for MX records first and then query the same caching resolver
for ANY it won't return, say, any TXT records that may be present at the
authoritative nameserver.

This could be implementation dependent, but Comcast's isn't wrong, and
you should not rely on ANY queries returning full data.  This has been
hashed out to tears in the past, for example when qm**l used to do these
queries in an attempt to optimise DNS query volumes and RTT.

At the ISP I consult to, I filter all ANY queries, because they have
been used for DNS amplification attacks.

Re: DDOS, IDS, RTBH, and Rate limiting

[Brian Rak]

On 11/22/2014 11:18 AM, Denys Fedoryshchenko wrote:

On 2014-11-22 18:00, freed...@freedman.net wrote:

We see a lot of Brocade for switching in hosting providers, which makes
sFlow easy, of course.
Oh, Brocade, recent experience with ServerIron taught me new lesson, 
that i can't
do bonding on ports as i want, it has limitations about even/odd port 
numbers and

etc.
Most amazing part i just forgot, that i have this ServerIron, and it 
is a place where
i run DDoS protection (but it works perfectly over tap way). Thanks 
for reminding

about this vendor :)


I just hope you're not talking FCX's if you upgrade those to 8.x 
firmware, you'll lose sflow on the 10gb ports.  Once you upgrade, they 
send a corrupted sflow packet, and at *far* less then the rate that you 
configure.  Even if you adjust your parser to compensate for the corrupt 
packet, they're still dropping the large majority of samples, making 
sflow pretty much useless.


It's been several months since we reported this, and we're still waiting 
on a fix.

Re: Reporting DDOS reflection attacks

[Brian Rak]

Also, abusix is not completely accurate (and they've never responded to 
my emails reporting problems).  For example, any IPs from apnic and 
nic.ad.jp return the registry's abuse address, which doesn't do anything.


Don't forget about all the providers with incorrect abuse contacts, or 
providers that require you to fill out some form, or providers that 
auto-respond with messages saying it's not their IP space (I'm looking 
at you charter... 71.90.222.x is definitely your space, despite what 
your abuse system thinks).


Some tips:
1) Verify the servers are still vulnerable.  This is pretty 
straightforward, and saves everyone involved some time
2) Your abuse emails should include tcpdump-like output (or you'll get 
tons of replies asking for logs)
3) Sticking to one abusive IP per email seems to get the best response 
rate (or you confuse all the automated systems for parsing these)
4) We provide instructions for fixing the issue for some common 
software... this seems to help some of the people who have no idea what 
they are doing.
5) Make sure you don't send this from your email address.  It should 
definitely be it's own mailbox due to volume of bounces and autoreplies 
you'll see.


Don't expect that sending abuse emails is going to have any noticeable 
effect on the size of the attacks you see.  The openresolverproject 
stats show the scope of the issue: 
http://openresolverproject.org/breakdown.cgi


On 11/8/2014 5:48 PM, Damian Menscher wrote:

I've used https://abusix.com/contactdb.html

Be prepared for a lot of backscatter.  You'll get autoresponders, automated
ticketing systems sending frequent updates, bounce messages (from full
abuse@ inboxes), and be surveyed for how well they're not performing.

Also, be prepared for ISPs / hosting providers to ask for additional
information, like logs proving the attack came from their customer.

Oh, and be prepared to feel sorry for their customers whose VMs are deleted
for hacking, rather than being informed of their misconfiguration.

On the bright side, some 10% will actually correct the problem, thereby
costing the attacker a few minutes of work to re-scan for active
amplifiers. :P

Damian
Professional Pessimist

On Fri, Nov 7, 2014 at 10:56 AM, srn.na...@prgmr.com wrote:


Like most small providers, we occasionally get hit by DoS attacks. We got
hammered by an SSDP
reflection attack (udp port 1900) last week. We took a 27 second log and
from there extracted
about 160k unique IPs.

It is really difficult to find abuse emails for 160k IPs.

We know about abuse.net but abuse.net requires hostnames, not IPs for
lookups and not all IP
addresses have valid DNS entries.

The only other way we know of to report problems is to grab the abuse
email addresses is whois.
However, whois is not structured and is not set up to deal with this
number of requests - even
caching whois data based on subnets will result in many thousands of
lookups.

Long term it seems like structured data and some kind of authentication
would be ideal for reporting
attacks. But right now how should we be doing it?

Re: large BCP38 compliance testing

[Brian Rak]

On 10/2/2014 6:10 AM, Mikael Abrahamsson wrote:


Hi,

To fix a lot of the DDOS attacks going on, we need to make sure BCP38 
compliance goes up. Only way to do this I can think of, is large scale 
BCP38 testing. One way of doing this, is to have large projects such 
as OpenWRT, RIPE Atlas project, perhaps even CPE vendors, implement 
something that would spoof 1 packet per day or something to a known 
destination, and in this packet the real source address of the 
packet is included.


I have been getting pushback from people that this might be illegal. 
Could anyone please tell me what's illegal about trying to send a 
packet with a random source address?


If we can get consensus in the operational world that this is actually 
ok, would that help organisations to implement this kind of testing. I 
could see vendors implement a test like help verify network stability 
and compliance, these tests are anonymous checkbox during the initial 
install, or something like this.


Why isn't this being done? Why are we complaining about 300 gigabit/s 
DDOS attacks, asking people to fix their open resolvers, NTP servers 
etc, when the actual culprit is that some networks in the world don't 
implement BCP38?




A lot of the discussion on BCP38 seems to be around providers who are 
unintentionally allowing IP spoofing.


What about providers who knowingly allow IP spoofing, because it's 
profitable?


There's a provider that basically caters to the DDOS-as-a-service 
industry by selling servers with unmetered connections, where they allow 
IP spoofing. (If you've ever looked into this at all, you know exactly 
who I'm talking about).

Re: where to go to understand DDoS attack vector

[Brian Rak]

On 8/26/2014 12:52 PM, me wrote:


On 08/26/2014 07:58 AM, Roland Dobbins wrote:
On Aug 26, 2014, at 8:37 PM, John York jo...@griffintechnology.com 
wrote:


In this case, 17 is both the protocol and port number. Confusing 
coincidence :)

Not in this output which the OP sent to the list:

8:33:58.482193 IP (tos 0x0, ttl 56, id 0, offset 0, flags [DF], 
proto UDP (17), length 29) x.x.x.x.2072  x.x.x.x.27015: UDP, length 1
0x: 4500 001d  4000 3811 088c cf9a 3b8c 
E.@.8 mailto:E.@.8.;.
0x0010: 405e eebf 0818 6987 0009 10f8 4300  
@^i.C...
0x0020:          
..


18:33:58.482193 IP (tos 0x0, ttl 56, id 0, offset 0, flags [DF], 
proto UDP (17), length 29) x.x.x.x.2072  x.x.x.x.27015: UDP, length 1
0x: 4500 001d  4000 3811 088c cf9a 3b8c 
E.@.8 mailto:E.@.8.;.
0x0010: 405e eebf 0818 6987 0009 10f8 4300  
@^i.C...
0x0020:          
..
18:33:58.484625 IP (tos 0x0, ttl 56, id 0, offset 0, flags [DF], 
proto UDP (17), length 29) x.x.x.x.2072  x.x.x.x.27015: UDP, length 1
0x: 4500 001d  4000 3811 088c cf9a 3b8c 
E.@.8 mailto:E.@.8.;.
0x0010: 405e eebf 0818 6987 0009 10f8 4300  
@^i.C...
0x0020:          
..
18:33:58.486137 IP (tos 0x0, ttl 56, id 0, offset 0, flags [DF], 
proto UDP (17), length 29) x.x.x.x.2072  x.x.x.x.27015: UDP, length 1
0x: 4500 001d  4000 3811 088c cf9a 3b8c 
E.@.8 mailto:E.@.8.;.
0x0010: 405e eebf 0818 6987 0009 10f8 4300  
@^i.C...
0x0020:          
..

Source port 2072, destination port 27015.


Been awhile since I got to dig into hex tcpdump but spent the time 
anyway. A UDP data segment that is 9 bytes long and only contains a 
C (0x43) ? And looks like to a Steam/Half-life (27015) gaming port. 
Not sure what the C is used for with those systems but guessing it's 
some sort of request?


It's pretty tough to say without knowing exactly what game is running 
there.  While 27015 was originally used for Half Life, it's been used by 
a wide range of games at this point.  Pretty much all the Valve games 
use this port, as well as a number of third party games that are based 
on the Steamworks SDK.


Trying to figure out exactly what the game server thinks the packet is 
is not likely to help you figure out why it's being sent.  You should 
instead be figuring out why your IPMI controller is compromised.  It 
could also be reflection, 2072 is within the port range that is usually 
used for KVM or remote media by the IPMI controllers (though, they're 
usually TCP and not UDP).

Re: where to go to understand DDoS attack vector

[Brian Rak]

On 8/26/2014 8:28 PM, Larry Sheldon wrote:

On 8/26/2014 08:31, Roland Dobbins wrote:


On Aug 26, 2014, at 8:26 PM, Stephen Satchell l...@satchell.net wrote:


qotd17/udp  quote


No, that's the protocol number - 17 is UDP - not the port number.



Really?

http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers

udp DID used to be protocol 17, but it is a fact that quotd runs on 
udp port 17.




Yes, he is correct.  This is not UDP port 17.

 8:33:58.482193 IP (tos 0x0, ttl 56, id 0, offset 0, flags [DF], proto 
UDP (17), length 29) x.x.x.x.2072  x.x.x.x.27015: UDP, length 1


Protocol: UDP (IP protocol 17)
Source Port: 2072
Dest Port: 27015

What protocol is UDP now, if it's not 17?

Re: ipmi access

[Brian Rak]

The kernel is the least of your worries here.

This is what you can expect from the Supermicro controllers:

Linux Kernel 2.6.17.13
Lighttpd 1.4.32
pcre 8.31
pcre 8.33
msmtp 1.4.16
tree 1.5.2.2
flex 2.5.35
readline 5.2
termcap 1.3.1
BIND 9.8.1-P1
busybox 1.12.0
ntp 4.2.4p4
openssl 0.9.8h
openlldp 0.3alpha
wide-dhcpv6 20080615
openldap 2.4.11
zlib 1.2.3
glibc 2.3.5
gcc 3.4.4
libxml2 2.6.32

On 6/2/2014 8:33 AM, Jeroen Massar wrote:

On 2014-06-02 14:23, Paul S. wrote:
[..]

On most ATEN chip based BMC boards from Supermicro, it includes a UI to
iptables that works in the same way.

You could put it on a public net, allow your stuff and DROP 0.0.0.0/0.

But unless you have servers with those, I think the best way to go is
putting them on internal IPs and then using some sort of a VPN.

While you are typing the iptables command, do a check of the software
versions, typically they are running a decade old kernel and a lot of
unpatched software that is exposed. You really do not want to run that
on the Interwebs, just the idea of any packet arriving to such a kernel
is scary.


Relevant good reads:
http://michael.stapelberg.de/Artikel/supermicro_ipmi_openvpn
https://plus.google.com/+TobiasDiedrich/posts/Bq44KkBT3vK

The first URL references 2.6.17, yes... *2.6.17* is the CURRENT version
of the kernel running on most IPMIs out there.

http://kernelnewbies.org/Linux_2_6_17 - Released 17 June, 2006

8 years... ouch, yeah, no way that is going to be attached to a public
network...

Thus please, don't shoot yourself in the foot with that and more
importantly don't shoot the rest of the Internet in the foot as they'll
receive the packets.


Note: the IPMI that Michael describes is on a unrouted VLAN, the access
to the OpenVPN port that he runs on the IPMI happens through SSH on a
jumpbox which is ACLd away.

Greets,
  Jeroen

   (who is still awaiting for Zeus4IPMI)

Re: ipmi access

[Brian Rak]

They do publish it.  The problem is, it's not documented, and it takes a 
bunch of work to get into a usable state.See 
ftp://ftp.supermicro.com/GPL/SMT/SDK_SMT_X9_317.tar.gz


Plus, the firmware environment is pretty hostile.  If you flash some bad 
firmware, your only option is to desolder the IPMI flash chip and 
program it externally.  It cannot be reprogrammed in circuit, and 
there's no recovery method.


On 6/2/2014 1:32 PM, Nikolay Shopik wrote:


On 02/06/14 20:56, Christopher Morrow wrote:

so... as per usual:
   1) embedded devices suck rocks
   2) no updates or sanity expected anytime soon in same
   3) protect yourself, or suffer the consequences

seems normal.


So I wonder why vendors don't publish source code of these ipmi 
firmware in first place? Like supermicro from what we know its 99% is 
open source stuff.

Re: ipmi access

[Brian Rak]

On 6/2/2014 3:47 PM, shawn wilson wrote:

On Mon, Jun 2, 2014 at 3:19 PM, Nikolay Shopik sho...@inblock.ru wrote:


Java only used for mouting images. KVM is transfered via VNC protocol iirc.

They're not re-inventing the wheel, but I think KVM is generally some
VNC stream embedded in http(s) which VNC clients can't seem to
understand (at least, at a glance, I haven't been able to connect to
iLo, DRAC, Spider, or Tyan IPMI from outside the Java app).
No, at least on SuperMicro it's a hacked up VNC protocol.  It's not 
embedded in HTTP/HTTPS, it just uses HTTP/HTTPS to fetch the Java app.


I say hacked up because it's got a custom auth method, and a whole bunch 
of undocumented extensions.  I looked into implementing support in noVNC 
for it, but reverse engineering a binary protocol is a bit beyond me.


It's also annoying because it claims to be a TightVNC server (and uses 
TightVNC auth/tunneling)... I was so hopeful that would just work.  It 
looks like they took the TightVNC code, and just made a bunch of changes 
with no regard for the specification.

Re: crave your indulgence

[Brian Rak]

This seems like a perfect use for ATLAS: https://atlas.ripe.net/

On 5/27/2014 2:28 PM, manning bill wrote:

If you wouldn’t mind a quick tracerooute -  Can you confirm reachability to the 
following:

2001:500:84::b

Thanks in advance.

/bill
Neca eos omnes.  Deus suos agnoscet.

Re: Question for service providers regarding tenant use of public IPv4 on your infrastructure

[Brian Rak]

On 4/28/2014 4:18 PM, Cliff Bowles wrote:

(accidentally sent this to nanog-request earlier, sorry if there is a double 
post)

We are an enterprise and we do not yet have a sophisticated service-provider 
model yet for billing, capacity-management, or infrastructure consumption. We 
have a few vBlocks that we consume internally for IT/business needs. Recently, 
the decision was made to start offering our infrastructure to partner 
businesses to deploy their apps on, which will then be made available to their 
customers.

The ingress/egress, the virtualization and even the orchestration part are 
essentially covered. We've tackled the security part as well. However, we have 
some tenants that want to egress to the internet locally rather than backhaul 
the traffic to their premise. Naturally, we could ask each tenant to provide 
their own internet for this, but the business wants to explore a dedicated, 
customer-only internet and chargeback/showback.

My question is: how are cloud providers handling the use of their IP space when 
they don't have full control over what their tenants are doing? More 
specifically, if you own a large block of IPs, how do you prevent business 
impact (or other tenant impact) if one tenant does something that causes an 
upstream ISP to blacklist/block? We don't want to put more controls in path 
between the tenant and the internet, we just want to know how to manage 
upstream relations.
If you're allocating individual customers their own subnets, make sure 
you report these allocations to ARIN (via SWIP).  This will make the 
whois results more accurate, so you'll hopefully just end up with the 
individual customer getting blacklisted, rather then your entire range.  
Make sure you actually respond to abuse complaints in a timely fashion.  
If you're actually responsive to abuse complaints, it's a lot less 
likely you'll end up with all of your subnets blacklisted.



I'm guessing Amazon and other similar providers have some arrangements with 
peering ISPs and law-enforcement to ensure that there is consultation before 
action is taken?
I doubt it.  Most of Amazon's EC2 IP ranges are on various blacklists.  
There's really no feasible way for them to keep all their IPs off 
blacklists, so I suspect they've just given up trying.

Or do ISPs put some level of security between their tenants and the internet to 
prevent this? I've been told that the majority do not have any intelligent 
filtering beyond bogon-lists. I'd imagine that would cause huge operational 
overhead and frustrate the tenants.
You should try to block whatever abuse you can, especially if you're 
going to be offering 'cloud' servers to the public.  Get some routine 
security scans going (start off with the basics, look for open 
resolvers, vulnerable NTP servers, open chargen servers, SNMP servers 
with default communities) and alert your customers whenever you detect 
something.


It should go without saying, but make sure your users cannot spoof IP 
addresses.

Re: NTP DRDos Blog post

[Brian Rak]

That's not a new term.

http://en.wikipedia.org/wiki/DRDOS
DRDoS, a type of network attack named Distributed Reflection Denial of 
Service.

http://en.wikipedia.org/wiki/Distributed_Reflection_Denial_of_Service#Reflected_.2F_Spoofed_attack

On 2/20/2014 11:14 AM, Niels Bakker wrote:

* st...@ntp.org (Harlan Stenn) [Thu 20 Feb 2014, 00:38 CET]:

I'd love to hear any feedback about the post.


Don't invent new terms like DrDos.


-- Niels.

Re: OpenNTPProject.org

[Brian Rak]

Rate limitings been in place for quite some time, but I believe it's 
only for actual time queries.   This DDOS uses monlist, which isn't 
subject to the same rate limits.


You've disabled monlist now, so I bet you'll no longer need all the rate 
limiting IPTables rules. (Though, you'll still see the incoming garbage 
for awhile, but NTPD will just discard it so it shouldn't cause problems).


On 2/17/2014 2:23 AM, Pete Ashdown wrote:

On 2/16/14, 7:38 PM, Brian Rak wrote:

Seriously, just fix your configuration.  The part of NTP being abused
is completely unrelated to actually synchronizing time.  It's a
management query, that has no real reason to be enabled remotely. You
don't even need to resort to iptables for this, because NTPD has built
in rate limiting (which isn't enabled for management queries, but
those are trivial to disable).

Thanks for the tip, monitoring is off.  I was under the impression that
rate-limiting hadn't made it into a stable version of ntpd yet.  Is that
incorrect?

Re: OpenNTPProject.org

[Brian Rak]

Seriously, just fix your configuration.  The part of NTP being abused is 
completely unrelated to actually synchronizing time.  It's a management 
query, that has no real reason to be enabled remotely. You don't even 
need to resort to iptables for this, because NTPD has built in rate 
limiting (which isn't enabled for management queries, but those are 
trivial to disable).


$ ntpdc -c monlist -n clock.xmission.com
remote address  port local address  count m ver code avgint  
lstint

===
173.209.207.23342422 198.60.22.240   4727 3 3 0  0   0
24.155.184.100 45285 198.60.22.240 11 3 4 0  6   0
107.0.41.2 48625 198.60.22.240264 3 4 0  5   0
67.108.239.31  40642 198.60.22.240  77084 3 3 0  0   0
177.65.149.237 62212 198.60.22.240   1085 3 1 0  0   0
209.64.161.162 44786 198.60.22.240 19 3 4 0  7   0
103.7.36.3851618 198.60.22.240  4 3 3 0  8   0
173.209.207.21850616 198.60.22.240   4731 3 3 0  0   0
69.61.203.25   20766 198.60.22.240  16379 3 4 0  1   0
68.188.251.223   478 198.60.22.240  2 1 3 0  0   0
75.82.183.104123 198.60.22.240  1 3 4 0  0   0
63.64.124.129  52839 198.60.22.240 150867 3 4 0  0   0
65.201.33.150151 198.60.22.240393 3 2 0  3   0
124.228.119.10524687 198.60.22.240 31 3 3 0  4   0
64.191.150.130   319 198.60.22.2404494361 3 2 0  0   0
76.102.124.27123 198.60.22.240  2 3 4 0  0   0
72.235.200.183   123 198.60.22.240  1 3 4 0  0   0
50.73.42.121   10398 198.60.22.240 11 3 3  0 14   0
63.64.124.144  26984 198.60.22.2405823740 3 4 0  0   0
71.5.8.194 44699 198.60.22.240  3 3 4 0  0   0
143.112.64.21320 198.60.22.240182 1 3 0  6   0
72.235.19.125123 198.60.22.240  1 3 4 0  0   0
198.237.66.2   10471 198.60.22.240499 3 3 0  3   0
12.108.21.226357 198.60.22.240 10 1 3  0 14   0
174.47.116.250   463 198.60.22.240 24 3 4 0  5   0
72.1.71.73   738 198.60.22.240 19 3 3 0  8   0
67.136.57.101026 198.60.22.240243 3 3 0  5   0
64.199.163.5 306 198.60.22.240231 3 4 0  4   0
70.77.76.153   32188 198.60.22.240  1 3 4 0  0   0

There is no excuse to still be running a NTP server with monlist 
enabled.  Fix your configuration, and you don't need IPTables rules.




On 2/16/2014 1:29 PM, Pete Ashdown wrote:

Just in case you run a legitimate open NTP server, this iptable stanza
helps immensely:

## rate limit ntp
$IPTABLES -N NTP
$IPTABLES -N BLACKHOLE
$IPTABLES -A BLACKHOLE -m recent --set --name ntpv4blackhole --rsource
$IPTABLES -A BLACKHOLE -j DROP
$IPTABLES -A NTP -m recent --update --seconds 5 --hitcount 20 --name
ntpv4 --rsource -j BLACKHOLE
$IPTABLES -A NTP -m recent --update --seconds 5 --hitcount 2 --name
ntpv4blackhole --rsource -j DROP
$IPTABLES -A NTP -m recent --set --name ntpv4 --rsource -j ACCEPT
$IPTABLES -A INPUT -p udp -m udp --dport 123 -j NTP


I've found that blocking TCP destination NTP to client servers/networks
blocks legitimate NTP synchronization for their clients.   Although I
wish they'd all just use my on-network NTP server, I can't assume they
will.  Does anyone have a list or source of pool and vendor
(Apple/Microsoft/etc) servers so I can permit based on source before
blocking based on destination port?

Re: looking for a tool...

[Brian Rak]

pmacct

On 2/4/2014 12:34 AM, Mike wrote:

Hello,

I was wondering if anyone could point me in the direction of a 
tool capable of sniffing (or reading pcap files), and reporting on lan 
station thruput in terms of bits per second. Ideally I'd like to be 
able to generate a sorted report of the top users and top thruputs 
observed and so forth. The traffic is pppoe and I need to monitor it 
at a specific switchport where I can arrange span.


Thank you.

Re: TWC (AS11351) blocking all NTP?

[Brian Rak]

Huh?  The issue with NTP relates to the monlist command (and a few 
others).  These are management queries, and are not critical to the 
operation of a NTP server.  You can disable these quite easily, and 
still run a NTP server that provides accurate time services.



On 2/3/2014 9:14 AM, TGLASSEY wrote:
How about this - I have proposed to NIST we start filtering - realize 
that the NIST ITS program itself was  setup to run NTP in an open 
access mode - we host a dozen or so of those systems and so we get hit 
all the time.


The solution is actually not running timing services across UDP 
because of the hand shaking over the open Internet - and that 
obviously isnt going to happen.


My suggestion is that for those that need access we set up VLAN 
trunked private networking models to your ISP MPOE as it were in a 
digital context.


If this interests you contact me off list.

Todd Glassey - USTiming.ORG

On 2/2/2014 7:17 PM, ryang...@gmail.com wrote:
I'd hate to think that NetOps would be so heavy handed in blocking 
all of UDP, as this would essentially halt quite a bit of audio/video 
traffic. That being said, there's still quite the need for protocol 
improvement when making use of UDP, but blocking UDP as a whole is 
definitely not a resolution, and simply creating a wall that not only 
keeps the abusive traffic out, but keeps legitimate traffic from 
flowing freely as it should.

Sent on the TELUS Mobility network with BlackBerry

-Original Message-
From: Cb B cb.li...@gmail.com
Date: Sun, 2 Feb 2014 15:09:55
To: Matthew Petachmpet...@netflight.com
Cc: nanog@nanog.org
Subject: Re: TWC (AS11351) blocking all NTP?

On Feb 2, 2014 2:54 PM, Matthew Petach mpet...@netflight.com wrote:

On Sun, Feb 2, 2014 at 2:17 PM, Cb B cb.li...@gmail.com wrote:


On Feb 2, 2014 8:35 AM, Jonathan Towne jto...@slic.com wrote:

The provider has kindly acknowledged that there is an issue, and are
working on a resolution.  Heads up, it may be more than just my

region.

And not just your provider, everyone is dealing with UDP amp attacks.

These UDP based amp attacks are off the charts. Wholesale blocking of
traffic at the protocol level to mitigate 10s to 100s of gigs of ddos
traffic is not knee jerk, it is the right thing to do in a world 
where

bcp 38 is far from universal and open dns servers, ntp, chargen, and
whatever udp 172 is run wild.

People who run networks know what it takes to restore service. And
increasingly, that will be clamping ipv4 UDP in the plumbing,  both
reactively and proactively.



Please note that it's not that UDP is at fault here; it's
applications that are structured to respond to small
input packets with large responses.


I dont want to go into fault, there is plenty of that to go around.


If NTP responded to a single query with a single
equivalently sized response, its effectiveness as
a DDoS attack would be zero; with zero amplification,
the volume of attack traffic would be exactly equivalent
to the volume of spoofed traffic the originator could
send out in the first place.

I agree the source obfuscation aspect of UDP can be
annoying, from the spoofing perspective, but that
really needs to be recognized to be separate from
the volume amplification aspect, which is an application
level issue, not a protocol level issue.

Source obfuscation is not annoying. Combined with amplification, it 
is the
perfect storm for shutting down networks... whereby the only solution 
is to

shutdown ipv4 udp. Or wave the magic wand that makes bcp38 universal,
patches boxes, and so on.

My point is: dont expect these abbused services on UDP to last. We have
experience in access networks on how to deal with abused protocols. 
Here is

one reference

http://customer.comcast.com/help-and-support/internet/list-of-blocked-ports/ 



My crystal ball says all of UDP will show up soon.

CB


Thanks!

Matt
PS--yes, I know it would completely change the
dynamics of the internet as we know it today to
shift to a 1:1 correspondence between input
requests and output replies...but it *would*
have a nice side effect of balancing out traffic
ratios in many places, altering the settlement
landscape in the process.  ;)

Re: TWC (AS11351) blocking all NTP?

[Brian Rak]

On 2/3/2014 2:46 PM, Dobbins, Roland wrote:

On Feb 4, 2014, at 12:11 AM, Brian Rak b...@gameservers.com wrote:


You can disable these quite easily, and still run a NTP server that provides 
accurate time services.

Concur 100% - although it should be noted that 1:1 reflection without any 
amplification is also quite useful to attackers.


That's true, but there are countless services out there that could be abused in 
such a way.  It's pretty much the same issue with DNS, even authoritative-only 
servers can be abused for reflection.  Securing everything that could possibly 
be used for reflection is going to be a long and painful process, preventing 
this specific amplification attack is pretty easy.

NTP clients have a long history of poor implementations, so the server already 
has rate limiting built in.  While rate limiting outgoing replies isn't a 
perfect solution, it's significantly better then no rate limiting (for the 
curious, add 'limited' to your 'restrict default' lines to enable rate 
limiting.  This doesn't help with the current amplification issues, but will 
help should someone just be abusing NTP servers for reflection).