Re: no you can't configure your router w/ this

[Warren Kumari]

On Jun 22, 2010, at 7:07 PM, Adam LaFountain wrote:



sigh... where was this useful data 10 years ago!

http://www.fcc.gov/worldtravel/



Even more entertaining is the reboot.fcc.gov (Beta)


Bah, more like Alpha if you ask me -- I clicked link MULTIPLE times  
and the FCC didn't reboot -- can I file a bug somewhere?


W


in the top right
corner.  I wonder if they have a reboot.ftc.gov link as well; that  
might

actually be more useful.

Re: no you can't configure your router w/ this

[Warren Kumari]

On Jun 23, 2010, at 7:56 PM, Christopher Morrow wrote:

On Wed, Jun 23, 2010 at 1:45 PM, Warren Kumari war...@kumari.net  
wrote:


On Jun 22, 2010, at 7:07 PM, Adam LaFountain wrote:



sigh... where was this useful data 10 years ago!

http://www.fcc.gov/worldtravel/



Even more entertaining is the reboot.fcc.gov (Beta)


Bah, more like Alpha if you ask me -- I clicked link MULTIPLE times  
and the

FCC didn't reboot -- can I file a bug somewhere?


how do you know it wasn't rebooted? Did you see the lights in the
building blink?


1: Lights in the building blink? No, the radio didn't turn off --  
thats what they do, isn't it?
2: Even if it *did* reboot, the UI implementation is poor -- I refuse  
to believe that it can reboot instantly, and there was no feedback  
provided. There should be some thing like Rebooting now... The FCC  
will be back in 10s... 9s... 8s...


W

Re: Monitoring Tools

[Warren Kumari]

On Aug 19, 2010, at 6:23 AM, Phil Regnauld wrote:

 jacob miller (mmzinyi) writes:
 Am looking for an opensource network monitoring tool with ability to create 
 different views for different users.
 
 
Hi Jacob,
 
What kind of network monitoring ?  Bandwidth utilization, service
availability, RTT, statistics data collection, ... ?
 
There are tons of open source software tools out there:
 
Nagios (www.nagios.org)
Zabbix (www.zabbix.com)
OpenNMS (www.opennms.org)
ZenOSS (www.zenoss.com)
SmokePing (http://oss.oetiker.ch/smokeping/)
Cacti (www.cacti.netl)
NetFlow Dashboard (http://trac.netflowdashboard.com/netflowdashboard/)
NFSen (http://nfsen.sourceforge.net/)
 
 
etc...
 
Depends on what you want to achieve!

Yes, yes it does...

This is a (dated, but still good) introduction that you might want to read: 
http://www.nanog.org/meetings/nanog26/presentations/stephen.pdf

Joe Abley and Stephen Stuart from NANOG26!

W

 
Cheers,
Phil
 
 

Re: Should routers send redirects by default?

[Warren Kumari]

On Aug 24, 2010, at 4:32 PM, William Herrin wrote:


On Fri, Aug 20, 2010 at 1:20 PM, Christopher Morrow
christopher.mor...@gmail.com wrote:

Polling a little bit here, there's an active discussion going on
6...@ietf about whether or not v6 routers should:
 o be required to implement ip redirect functions (icmpv6 redirect)
 o be sending these by default


Hi Chris,

If you don't mind, I'd like to ask a similar question whose answers
might be instructive for the question you asked:


Forgetting all of the theoretical constructs for a moment, has anyone
here personally encountered an operational scenario in which ICMP
redirects solved a problem for you that you would otherwise have found
difficult or intransigent? Without naming names, would you describe
the scenario's details, explain the problem that would have existed
absent redirects and explain how redirects solved it for you?



I have, but it was a long long time ago (~1997), and it was a stupid  
problem


We had a bunch of hosts on a LAN - their default GW was an AGS+  
connected to provider X. Also on the same network was a Bay Networks  
BCN (AFAIR) connected to provider Y.


In general most flows were relatively long lived (some NNTP, some  
FTP.. oh, and Quake!). There was no reasonable way to inform the hosts  
if provider X went away. The AGS+ would also run a bit too hot if it  
had to accept all of the traffic and then punt the relevant parts over  
to the BCN


Unrelated, but this network also did static IPs for dial customers  
(who could dial into one of ~lots of RAS boxes) -- this meant that the  
RAS boxen has to inject /32s into OSPF for each customer -- this meant  
that if certain routers (like the AGS+) bounced there was enough churn  
that other routers would fall over (the BCN would hit some watchdog  
and fall over, and if you tried to bring it up into a network that was  
already converged it would run out of RAM and happily drop into some  
debugger console).


Fun times...

W





Thanks,
Bill Herrin






--
William D. Herrin  her...@dirtside.com  b...@herrin.us
3005 Crane Dr. .. Web: http://bill.herrin.us/
Falls Church, VA 22042-3004



--
She'd even given herself a middle initial - X - which stood for  
someone who has a cool and exciting middle name.


-- (Terry Pratchett, Maskerade)

Re: Idea's for donating/recycling server hardware [Off-Topic]

[Warren Kumari]

On Aug 26, 2010, at 10:03 PM, Suresh Ramasubramanian wrote:


There's also http://www.nsrc.org at UOregon - as good a home as any


, and better than most,


for any gear you want to trash.

On Fri, Aug 27, 2010 at 4:35 AM, Wil Schultz wschu...@bsdboy.com  
wrote:

I apologize for being somewhat off topic...

I've got a fair amount of SPARC hardware (v210 through v490) and  
32bit HP DL360-380 hardware that I'm looking for creative ways to  
dispose of or to donate.


It seems like a waste to send it to metal scrap, if anyone has a  
more creative way of disposal please contact me off list. Local to  
San Francisco.


*disclaimer, contributions cannot go to religious or political  
organizations per corp policy*


Thanks!

-wil





--
Suresh Ramasubramanian (ops.li...@gmail.com)



--
I try to be good hard-worker-man, but refrigemater so messy, so so  
messy.

-- NewsRadio.

Re: Did your BGP crash today?

[Warren Kumari]

On Aug 27, 2010, at 5:37 PM, bmann...@vacation.karoshi.com wrote:



come on Chris,  is the Internet an experiment or not? :)
one would think that a responsible party would have made
efforts to let others in the playground know they were
going to try something different that could have ramifications
on an unkown distribution of some code bases.


I'm assuming that they weren't really expecting this to cause  
issues... Where does one draw the line? I'm planning on announcing  
x.y.z.0/20 later in the week -- x, y and z are all prime and the sum  
of all 3 is also a prime. There is a non-zero chance that something  
somewhere will go flooie, shall I send mail now or later?


Also, I would prefer that this gets discovered and dealt with (in this  
case by stopping the announcement :-)) than having folk not willing to  
try things and ending up with a weaponized version...


W




I'm not asking my vendor or (in the case of OSS) me to run
full bit sweeps... but a heads up to some of the known
ops lists would have been not only welcome but expected.

as usual, YMMV

--bill


On Fri, Aug 27, 2010 at 04:11:32PM -0400, Christopher Morrow wrote:
On Fri, Aug 27, 2010 at 4:07 PM, Mike Gatti ekim.it...@gmail.com  
wrote:

where's the change management process in all of this.
basically now we are going to starting changing things that can
potentially have an adverse affect on users without letting anyone  
know

before hand  Interesting concept.


you are running bgp, you are connected to the 'internet'... congrats
you are part of the experiment.

I suppose one view is that at least it wasn't someone with ill
intent, or a misconfigured mikrotek!

(you are asking your vendors to run full bit sweeps of each protocol
in a regimented manner checking for all possible edge cases and
properly handling them, right?)

-chris


On Aug 27, 2010, at 3:33 PM, Dave Israel wrote:



On 8/27/2010 3:22 PM, Jared Mauch wrote:
When you are processing something, it's sometimes hard to tell  
if something
just was mis-parsed (as I think the case is here with the  
missing-2-bytes)
vs just getting garbage.  Perhaps there should be some way to  
re-sync when
you are having this problem, or a parallel keepalive path  
similar to
MACA/MCAS/MIDCAS/TCAS between the devices to talk when something  
bad is

happening.


I know it wasn't there originally, and isn't mandatory now, but  
there is
an MD5 hash that can be added to the packet.  If the TCP hash  
checks

out, then you know the packet wasn't garbled, and just contained
information you didn't grok.  That seems like enough evidence to  
be able

to shrug and toss the packet without dropping the session.

-Dave





=+=+=+=+=+=+=+=+=+=+=+=+=
Mike Gatti
ekim.it...@gmail.com
=+=+=+=+=+=+=+=+=+=+=+=+=











--
What our ancestors would really be thinking, if they were alive today,  
is: Why is it so dark in here?


-- (Terry Pratchett, Pyramids)

Re: largest OSPF core

[Warren Kumari]

On Sep 2, 2010, at 11:11 AM, Nick Hilliard wrote:


On 02/09/2010 13:20, lorddoskias wrote:
I'm just curious - what is the largest OSPF core (in terms of  
number of

routers) out there?


You don't expect anyone to actually admit to something like this? :-)


Of course I do -- 'tis much for your reputation to have wrangled a  
poorly designed, ugly network under control than to have only worked  
at places with smooth sailing I *don't* expect the owner /  
designers of these to come forward, rather those who inherited a pile  
of choss to share war stories...  :-P



I worked on a network that had 350 routers in an (non-zero) area.  
Now, ~350 routers in an area doesn't sound *that* impressive, but on  
average these devices had 6 interfaces in OSPF, and many of these  
links were of the form:


[router A]-- {GRE} --- [firewall]-- {GRE in IPSEC} --- 
[Internet]--- {GRE in IPSEC} ---[firewall]---{GRE} --- [router B]


Routers A and B would form an OSPF adjacency. Much of this was an  
overlay network (over the Internet) and so the firewalls would build  
IPSec tunnels. Of course, said firewalls would not pass OSPF, so we  
had to build GRE tunnels between routers A and D and run OSPF over  
those -- traffic would enter the router, get encapsulated in GRE and  
then the GRE would be encapsulated in IPSec and tossed into the void
In other places (in the same OSPF area) we would purchase parallel  
T1 / E1s that we would run MLPPP over, and / or plain DS3s.
Oh, did I mention that network was primarily to support international  
call centers that had been outsourced to wherever was *really* cheap,  
and that many places with very cheap labor have very poor  
infrastructure? It was not uncommon to have interfaces that would  
bounce 5 or 10 times a day*



W

*: And yes, we did 'ave to get up out of shoebox at twelve o'clock at  
night and lick road clean wit' tongue. We had two bits of cold gravel,  
worked twenty-four hours a day at mill for sixpence every four years,  
and when we got home our Dad would slice us in two wit' bread knife.




Nick



--
It's a mistake trying to cheer up camels. You might as well drop  
meringues into a black hole. -- Terry Prachett

Re: Routers in Data Centers

[Warren Kumari]

On Sep 24, 2010, at 6:22 AM, Venkatesh Sriram wrote:


Hi,

Can somebody educate me on (or pass some pointers) what differentiates
a router operating and optimized for data centers versus, say a router
work in the metro ethernet space? What is it thats required for
routers operating in data centers? High throughput, what else?



While this question has many dimensions and there is no real  
definition of either I suspect that what many people mean when they  
talk about a DC routers is:

Primarily Ethernet interfaces
High port density
Designed to deal with things like VRRP / VLAN / ethernet type features.
Possibly CAM based, possibly smaller buffers.
Less likely to be taking full routes.

This is very similar to the religious debate about What's the  
difference between a 'real' router and a L3 switch?


Just my 2 cents.
W




Thanks, Venkatesh



--
Consider orang-utans.
In all the worlds graced by their presence, it is suspected that they  
can talk but choose not to do so in case humans put them to work,  
possibly in the television industry. In fact they can talk. It's just  
that they talk in Orang-utan. Humans are only capable of listening in  
Bewilderment.

-- Terry Practhett

Re: Choice of network space when numbering interfaces with IPv6

[Warren Kumari]

On Oct 16, 2010, at 10:55 PM, Kevin Oberman wrote:

 Date: Sun, 17 Oct 2010 01:56:28 +0100
 From: Randy Bush ra...@psg.com
 
 http://www.ietf.org/internet-drafts/draft-ietf-6man-prefixlen-p2p-00.txt
 Drafts are drafts, and nothing more, aren't they?
 
 must be some blowhard i have plonked
 
 Drafts are drafts. Even most RFCs are RFCs and nothing more. Only a
 handful have ever been designated as Standards. I hope this becomes
 one of those in the hope it will be taken seriously. (It already is by
 anyone with a large network running IPv6.)
 
 juniper and cisco implement today
 
 Unfortunately, a couple of other router vendors whose top of the line
 units I have tested recently did not.

Simple Matter of Programming ;-)

Please suggest to said vendors that they implement this -- IMO it's the right 
way to do it...

W

 -- 
 R. Kevin Oberman, Network Engineer
 Energy Sciences Network (ESnet)
 Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
 E-mail: ober...@es.netPhone: +1 510 486-8634
 Key fingerprint:059B 2DDF 031C 9BA3 14A4  EADA 927D EBB3 987B 3751
 

Re: co-location and access to your server

[Warren Kumari]

On Jan 12, 2011, at 3:49 PM, david raistrick wrote:

 On Wed, 12 Jan 2011, Jeroen van Aart wrote:
 
 What is considered normal with regards to access to your co-located 
 server(s)? Especially when you're just co-locating one or a few servers.
 
 For less than 1 rack, or specialty racks with lockable sections (1/2 or 1/3 
 or 1/4 racks with their own doors), I'd consider any physical access to 
 simply be a plus.  I wouldn't expect any at all.   You're not paying for 
 enough space to justify the costs involved in 24x7 independant access, and 
 the risks to other customers gear.
 
 
 When you get a full rack+, or cage+, I'd expect unfettered 24x7 access since 
 your gear should be seperated and secured from other folks gear.

You would think so, wouldn't you?

Many years ago I had a cage in 811 10th, with the usual pile 'o goodies in 
it... I have simple script (aka tail -f | grep -v ;-)) that I leave running 
in the background that tails syslog and only shows me interesting messages.
One day I notice messages scrolling by, so I go see what is grumping about.

Apparently the CF / PCMCIA card in one of the Cisco 7507s has just unmounted.
No! Wait, it's back. Nope, gone again. Back. Gone! Back! Yay! It's back... 
Whoop, I lied, gone still gone... still gone...

Bah, I figure that the card has just died and the appearing / disappearing 
trick was just the death rattle, so I take a wander over, and notice that it 
didn't just unmount, it's completely missing...
I manage to get one of the security folk to pull the camera footage for around 
that time and I see some chappie wanding up and down the aisles, looking in 
though the mesh at everyone's toys. After the third or forth circuit past our 
cage he suddenly perks up and hustles off camera. He comes back 2 minutes later 
with a broom and proceeds to poke the handle through the mesh and bang on the 
back of the router. Eventually he manages to thwack the eject button hard 
enough and the flash drops onto the floor -- he wiggles it over, slides it 
under the edge of the cage, grins like a monkey and scampers back to his cage...

I guess when you *really* needs some flash, you *really* needs some flash...

W

(I have also learnt the hard way not to use the edge of the cage as cable 
management...)



 Some specialty providers would be exceptions, of course (ie, I used to colo 
 gear inside tv stations, satellite downlink stations, etc).
 
 
 Telecom colo (switch and network gear in a dedicated but shared space for 
 providers providing service) would be an exception, of course.
 
 
 --
 david raistrickhttp://www.netmeister.org/news/learn2quote.html
 dr...@icantclick.org http://www.expita.com/nomime.html
 
 

Re: DSL options in NYC for OOB access

[Warren Kumari]

On Jan 24, 2011, at 6:22 PM, Nathan Eisenberg wrote:

 You can get a CLEAR WiMAX fixed modem with static IP address for $50
 (USD) monthly, or less if you opt for the low-bandwidth plan.
 
 I wouldn't dare rely on something of that nature for a lifeline connection.  
 I'd spring for the extra $30/mo.  It's expensive, but there ain't nothin' 
 like a physical cable when it's 3AM on a Sunday.
 
 Nathan
 
 

Re: IPv4 address shortage? Really?

[Warren Kumari]

On Mar 7, 2011, at 8:48 PM, Mark Andrews wrote:

 
 This has been thought of before, discussed and rejected.

But has this: 
http://tools.ietf.org/id/draft-terrell-math-quant-ternary-logic-of-binary-sys-12.txt
 ?

Please read and explain *exactly* why it doesn't work...

W



 
 In message 1299498200.29652.40.ca...@kotti.kotovnik.com, Vadim Antonov 
 writes
 :
 I'm wondering (and that shows that I have nothing better to do at 3:30am
 on Monday...) how many people around here realize that the plain old
 IPv4 - as widely implemented and specified in standard RFCs can be
 easily used to connect pretty much arbitrary number (arbitrary means
 2^256) of computers WITHOUT NETWORK ADDRESS TRANSLATION.  Yes, you hear
 me right.
 
 And, no, it does not require any changes any in the global routing
 infrastructure - as implemented now, and most OS kernels (those which
 aren't broken-as-designed, grin) would do the trick just fine.  None of
 that dual-stack stupidity, and, of course, no chicken-and-egg problem if
 the servers and gateways can be made to respect really old and
 well-established standards.
 
 DNS and most applications would need some (fairly trivial) updating,
 though, to work properly with the extended addressing; and sysadmins
 would need to do tweaks in their configs since some mythology-driven
 security can get in the way.  But they don't have to do that en mass
 and all at once.
 
 The most obvious solution to the non-problem of address space shortage
 is the hardest to notice, ain't it?
 
 --vadim
 
 P.S. Hfr YFEE gb ebhgr orgjrra cevingr nqqerff fcnprf bire choyvpnyyl
 ebhgrq fcnpr, Yhxr. Guvax bs cevingr nqqerff ovgf nf n evtug-fvqr
 rkgrafvba gb gur sbhe-bpgrg choyvp nqqerff.
 
 P.P.S. Gb rkgraq shegure, nygreangr gjb qvfgvapg cevingr nqqerff fcnprf,
 nf znal gvzrf nf lbh pna svg vagb gur urnqre.
 
 
 -- 
 Mark Andrews, ISC
 1 Seymour St., Dundas Valley, NSW 2117, Australia
 PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org


W

PS: :-)   doh! ROT13 fails to be interesting on punctuation 

Re: KVM

[Warren Kumari]

On Apr 23, 2013, at 5:36 PM, shawn wilson ag4ve...@gmail.com wrote:

 I'm looking at an IP-KVM. I don't need anything high res as I only
 need to see Linux consoles, BIOS, and RAID. What I am looking for:
 Non-Java client that runs on Linux (or a WebUI that will deploy a
 decent RDP or VNC session over SSL).
 Decent/configurable key mappings (ie, I've had a KVM a while ago where
 you had to pull down a menu for F-keys - not cool).
 Decently priced dongles (say ~$100?)
 
 I started looking at the Raritan devices (which can be found really
 cheap on ebay) but I only see a Java client and no mention of
 installing a client on Linux.
 

Related -- kinda.

A while back someone used to sell a cable / thingie that would allow you to use 
your laptop as a keyboard and monitor. Basically it had a VGA / HDMI and PS/2 
port on one side, and a USB port on the other -- you'd plug the USB into your 
laptop (and run some client) and the VGA / PS/2 into a server, machine, 
whatever. Whatever the server sent would show up on the laptop -- basically 
this means you can avoid having a crash cart. I've done a crappy job of 
explaining it, but does anyone know what I'm on about? Who made this? It is 
still available?

W


--
Have you got any previous convictions?

Well, I dunno... I suppose I used to believe very firmly that a penny saved is 
a penny earned--
-- Terry Pratchett

Re: KVM

[Warren Kumari]

On Apr 26, 2013, at 4:52 PM, John Mason john.mason...@gmail.com wrote:

 http://www.startech.com/Server-Management/KVM-Switches/Portable-USB-PS-2-KVM-Console-Adapter-for-Notebook-PCs~NOTECONS01
 

Oh yeah, that's the one… $470.. Now I remember why I didn't buy one when I 
first saw it…

W

 
 On Fri, Apr 26, 2013 at 4:49 PM, Warren Kumari war...@kumari.net wrote:
 
 On Apr 23, 2013, at 5:36 PM, shawn wilson ag4ve...@gmail.com wrote:
 
  I'm looking at an IP-KVM. I don't need anything high res as I only
  need to see Linux consoles, BIOS, and RAID. What I am looking for:
  Non-Java client that runs on Linux (or a WebUI that will deploy a
  decent RDP or VNC session over SSL).
  Decent/configurable key mappings (ie, I've had a KVM a while ago where
  you had to pull down a menu for F-keys - not cool).
  Decently priced dongles (say ~$100?)
 
  I started looking at the Raritan devices (which can be found really
  cheap on ebay) but I only see a Java client and no mention of
  installing a client on Linux.
 
 
 Related -- kinda.
 
 A while back someone used to sell a cable / thingie that would allow you to 
 use your laptop as a keyboard and monitor. Basically it had a VGA / HDMI and 
 PS/2 port on one side, and a USB port on the other -- you'd plug the USB into 
 your laptop (and run some client) and the VGA / PS/2 into a server, machine, 
 whatever. Whatever the server sent would show up on the laptop -- basically 
 this means you can avoid having a crash cart. I've done a crappy job of 
 explaining it, but does anyone know what I'm on about? Who made this? It is 
 still available?
 
 W
 
 
 --
 Have you got any previous convictions?
 
 Well, I dunno... I suppose I used to believe very firmly that a penny saved 
 is a penny earned--
 -- Terry Pratchett
 
 
 
 
 

--
It's a mistake trying to cheer up camels. You might as well drop meringues into 
a black hole. -- Terry Prachett

Re: in urgent need of router w/T3 interface

[Warren Kumari]

I suspect that you might have more luck if you mentioned where you  
are, how far you would be willing to drive to pick one up and how long  
you would need to use it for...


For example, I could probably loan you an old 7200 that would fit the  
bill, but I'm in VA which probably wouldn't work out for you...


Feel free to mail me privately if you don't have any luck elsewhere  
and I'll pull it out of storage, make sure it is still a happy camper,  
etc...


W
On May 22, 2009, at 8:58 AM, Adam Goodman wrote:

I have an urgent need for a router to replace the one that crocked  
last

night. If you have a router to sell or lend please contact me off list

It could be the following or equivalent of  a Cisco 7000 with the  
following

interfaces:
1x T3 clear channel (like a PA-T3)
1x FastEthernet

(Alternatively I can also use a wanPMC-C1T3 card with a PCI adapter.)

Thank you,
-Adam

Adam Goodman
E: a...@wispring.com
C: 801.971.1856

Re: problems with cisco 7200 and PA-T3

[Warren Kumari]

On May 28, 2009, at 6:43 PM, Adam Goodman wrote:

Just installed a cisco 7204vxr with a DS3 interface. we are not  
getting more

than 5Mbits.


So, how are you testing? A single flow maybe?





show interface is not reporting any errors. the provider tech put a  
piece

test equipment on the circuit and sees errors.



What errors are they seeing? How are they testing, etc?

Anyway, cisco-nsp is thad away

W

Does anyone else use a cisco 7200 with a DS3 interface that we might  
be able

to speak with?

Please hit me off list

Thank you,
Adam
801.971.1856


--
After you'd known Christine for any length of time, you found yourself  
fighting a desire to look into her ear to see if you could spot  
daylight coming the other way.


-- (Terry Pratchett, Maskerade)t





smime.p7s
Description: S/MIME cryptographic signature

Re: Opensource or Low Cost NMS for Server Hardware / Application Monitoring

[Warren Kumari]

For networking stuff, see Joe Abley and Stephen Stuart's NANOG 26  
Tutorial Managing IP Networks with Free Software  -- http://www.nanog.org/meetings/nanog26/abstracts.php?pt=Nzg1Jm5hbm9nMjY=nm=nanog26
Direct link to PDF: http://www.nanog.org/meetings/nanog26/presentations/stephen.pdf 
  -- it's from 2002 and so a little out of date, but still a great  
read.


As for server / application / random other stuff (like printers and  
ups's and IP camera and the like), Zenoss is great -- its clean,  
simple, fast(ish), easy  and pretty -- the last one happens to be  
important for some folks (esp in the enterprise world...)



W


On Jul 22, 2009, at 12:42 AM, Roland Dobbins wrote:



On Jul 22, 2009, at 11:34 AM, Stefan wrote:


WebNM + Denika + Logalot - set of
toolshttp://www.plixer.com/products/index.php


nfdump/nfsen, Stager, RANCID, RCS, CVS, or Subversion - these should  
all be included in any list of useful open-source tools for network  
operators, IMHO.


---
Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com

   Unfortunately, inefficiency scales really well.

   -- Kevin Lawton

ISP Security BOF -- NAOG 47

[Warren Kumari]

The time has come, the Walrus said,
To talk of many things, like how NANOG 47 is fast approaching and  
how I am *sure* that you would like to participate...


This is *your* chance to talk about interesting security related  
topics and provide some feedback on what you would (and would not)  
like to hear about...


Some security thing been buggin' you all year? Some topic that you  
feel strongly about and would like a change to inform others about?  
Step right up and give a talk -- this BOF is traditionally fairly laid  
back and easy going, so its a low stress introduction to presenting...


Slides are welcome, but not required...

W

Re: Data Center testing

[Warren Kumari]

On Aug 24, 2009, at 9:38 AM, Dan Snyder wrote:

We have done power tests before and had no problem.  I guess I am  
looking
for someone who does testing of the network equipment outside of  
just power
tests.  We had an outage due to a configuration mistake that became  
apparent

when a switch failed.


So, one of the better ways to make sure that your failover system is  
working when you need it is just to do away with the concept of a  
failover system  and make your failover system be part of your  
primary system

.
This means that your failover system is always passing traffic and you  
know that it is alive and well -- it also helps mitigate the pain when  
a device fails (you are sharing the load over both systems and so only  
half as much traffic gets disrupted). Scheduled maintenance is also  
simpler and less stressful as you already know that your other path is  
alive and well.


Your design and use case dictates how exactly you implement this, but  
in general it involves things like tuning your IGP so you are using  
all your links, staggering VLANs if you rely on them, multiple VRRP  
groups per subnet, etc.


This does require a tiny bit more planning during the design phase,  
and also requires that you check every now and then to make sure that  
you are actually using both devices (and didn't, for example, shift  
traffic to one device and then forget to shift it back :-)).
It also requires that you keep capacity issues in mind --  in a  
primary and failover scenario you might be able to run devices fairly  
close to capacity, but if you are sharing the load you need to keep  
things under 50% (so when you *do* have a failure the remaining device  
can handle the full load) -it's important to make this clear to the  
finance folks before going down this path :-)


W


 It didn't cause a problem however when we did a power
test for the whole data center.

-Dan


On Mon, Aug 24, 2009 at 9:31 AM, Ken Gilmour ken.gilm...@gmail.com  
wrote:



I know Peer1 in vancouver reguarly send out notifications of
non-impacting generator load testing, like monthly. Also InterXion
in Dublin, Ireland have occasionally sent me notification that there
was a power outage of less than a minute however their backup
successfully took the load.

I only remember one complete outage in Peer1 a few years ago... Never
seen any outage in InterXion Dublin.

Also I don't ever remember any power failure at AiNet (Deepak will
probably elaborate)

2009/8/24 Dan Snyder sliple...@gmail.com:
Does any one know of any data centers that do failure testing of  
their

networking equipment
regularly? I mean to verify that everything fails over properly  
after

changes have been made over
time.  Is there any best practice guides for doing this?

Thanks,
Dan





--
Does Emacs have the Buddha nature? Why not? It has bloody well  
everything else!

Re: Route table prefix monitoring

[Warren Kumari]

On Sep 10, 2009, at 7:23 AM, Joel Jaeggli wrote:




Olsen, Jason wrote:

Howdy all,



What I'm left thinking is that it would have been great if we'd had a
snapshot of our core routing table as it stood hours or even days  
prior

to this event occurring, so that I could compare it with our current
broken state, so the team could have seen that subnet in the core
table and what the next hop was for the prefix.  Are there any tools
that people are using to track when/what prefixes are added/withdrawn
from their routing tables, or to pull the routing table as a whole at
regular intervals for storage/comparison purposes?  It looks like
there's a plugin for NAGIOS, but I'm looking for suggestions on any
other tools (commercial, open source, home grown) that we might  
take a

look at.  For reference, we are running Cisco as well as Juniper kit.


Periodic table dumps, or even a log of the updates from a quagga  
router

inside your infrastructure could provide this information. That in a
nutshell is what routeviews and other collectors do for the dfz  
routing

table.


There is also an Internet draft for the BGP Monitoring Protocol (hhttp://tools.ietf.org/html/draft-ietf-grow-bmp-02) 
.
This draft provides for a method whereby the BGP speakers export their  
received updates to a central collector. This allows you to get route  
views in (more) real time, with no more screen scraping (and probably  
much lower CPU as well). Personally I think its an awesome idea and is  
something that we have need for a long long time (over the years I  
must have written 7-8 screen scrapers to get BGP RIB info, and they  
always suck).




Draft Abstract:
This document proposes a simple protocol, BMP, which can be used to  
monitor BGP sessions.
BMP is intended to provide a more convenient interface for obtaining  
route views for research purpose than the screen-scraping approach in  
common use today.
The design goals are to keep BMP simple, useful, easily implemented,  
and minimally service-affecting. BMP is not suitable for use as a  
routing protocol.



W






Feel free to drop me your thoughts off-list.



Thank you for any insight ahead of time,



-Jason Feren Olsen






For every complex problem, there is a solution that is simple, neat,  
and wrong.

-- H. L. Mencken

Re: Google Pagerank and Class-C Addresses

[Warren Kumari]

On Sep 21, 2009, at 2:01 PM, William Pitcock wrote:


On Mon, 2009-09-21 at 18:18 +0200, Sebastian Wiesinger wrote:

Hello Nanog,

I'm looking into a weird request which more and more customers have.
They want different Class C addresses, by which they mean IPs in
different /24 subnets.

The apparent reason for this is that Google will rank links from
different /24 higher then links from the same /24. So it's a SEO
thingy.



They are wrong.  Unfortunately, this is a rumour that is being  
cashed in

greatly by companies like GotWebHost.com, which offer SEO hosting.
They may honestly believe that this is true, it is not.  Infact, IPs
have nothing to do at all, with PageRank, and don't let any of these  
SEO

crackheads tell you otherwise.

A google employee blogged about this topic at:
http://www.mattcutts.com/blog/myth-busting-virtual-hosts-vs-dedicated-ip-addresses/


Yes, and I'll second this -- PageRank does not in any way get improved  
by hosting on multiple IPs (or different ranges or Class A's or Class- 
C's[0] or swamp space or space from different RIRs or premium  
addresses (?!) or anything like that...).






I googled a bit and found pages after pages of FUD and such great
things as the Class C Checker:  This free Class C Checker tool
allows you to check if some sites are hosted on the same Class C IP
Range.

My question is: Is there any proof that Google does differentiate
between /24s, or even better is there any proof that this isn't the
case?


There's Matt's word and Craig Silverstein's word and (not that it  
count for as much) my word -- PageRank does NOT differentiate between / 
24's.


Google has stated this multiple times and we have nothing to gain by  
lying or making things up -- the SEO folks on the other hand have a  
large incentive to claim that IPs *do* make a difference as they sell  
this as a service...


W

[0]: Yes, yes, I know, settle down



I will not give a customer space from different address blocks
just because he read it in a SEO magazine.


As said above: No, it is not true.  Further, SEO is mostly a load of
bullshit that only delivers temporary results, as the search engines
will change their algorithms, etcetera.



Perhaps someone from Google itself can answer this question?

Also how do you handle such requests? I expect I'm not the only one
who gets them.


It depends on how much money they pay me.

If they pay me a lot of money, then I will likely give them what they
want.  If not, well, that's too bad for them.

It doesn't matter to me, regardless, provided that they aren't  
violating

my AUP by you know, spamming or something along those lines.  In those
cases, well, they probably wouldn't be asking for more IPs, because  
they

would be offline.

William
--
William Pitcock SystemInPlace - Simple Hosting  
Solutions

1-866-519-6149 http://www.systeminplace.net/
Follow us on Twitter:   http://www.twitter.com/systeminplace




--
No man is an island, But if you take a bunch of dead guys and tie them  
together, they make a pretty good raft.

--Anon.

Re: bgp update destroying transit on redback routers ?

[Warren Kumari]

On Dec 1, 2011, at 3:36 PM, Christopher Morrow wrote:

 On Thu, Dec 1, 2011 at 3:23 PM, Igor Ybema i...@ergens.org wrote:
 http://tools.ietf.org/html/draft-wkumari-idr-as0-01
 
 one of the reasons the above was written...
 
 That does not include when ASN=0 is used in the aggregator attribute.
 Could you add that?
 
 that's a warren question...

http://tools.ietf.org/html/draft-wkumari-idr-as0-01 has been replaced with 
http://tools.ietf.org/html/draft-ietf-idr-as0-00 -- which does include it.

Thanks all,
W

Arriving early...

[Warren Kumari]

Hi there all,

I'm arriving on Friday evening -- was wondering who all might be around on 
Saturday? 
Anyone interested in doing something? Sightseeing, wandering around, etc?


W
--
Some people are like Slinkies..Not really good for anything but they still 
bring a smile to your face when you push them down the stairs.

NANOG 44 (Los Angeles): ISP Security BOF

[Warren Kumari]

Hi all,

NANOG 44 is fast approaching and once again we are looking for topics  
for the ISP Security BOF.
If you have any security related topics that you would like to hear  
about, not hear about, or (best of all) speak about, please let me  
know as soon as possible...


This is your chance to air your views --- slides are welcome but not  
required.


Danny McPherson and I are going to be moderating this year...

W

Re: NANOG 44 (Los Angeles): ISP Security BOF

[Warren Kumari]

Hello all,

NANOG 44 is now less than a week away.
Here is the current program for the ISP Security BOF (NANOG 44,  
October 13, 2008, 4:30 PM - 6:00 PM) -- as always, the program at this  
point is still somewhat fluid and subject to change.



16:30 - 16:45: Stealing the Internet -- Anton Kapela

In Stealing the Internet Kapela will describe a method where an
attacker exploits the BGP routing system to facilitate transparent  
interception of IP packets.
The method will be shown to function at a scale previously thought by  
many as unavailable.
The talk highlights a new twist in sub-prefix hijacking that he  
demonstrated at Defcon 16:
using intrinsic BGP logic to hijack network traffic and simultaneously  
create a 'bgp shunt towards
the target network. This method will be shown to preserve end-to-end  
reachability while creating
a virtual 'wire tap' at the attackers network. He'll cover additive  
TTL modification and
transparent-origin-AS as a means for the attacker to obscure the  
interception.


There will not be a live demonstration of the hijack or interception
methods.

--

16:45 - 17:00: An interim solution to the threat of DNS cache  
poisoning while waiting for DNSSEC. -- Rodney Joffe


--

17:00 - 17:15: Next steps in IRR/X509 --Barry Raveendran Greene,  
Jason Schiller.


-

17:15 - 17:30: Esthost's response to the 'Hostexploit report' --  
Konstantin Poltev (Esthost, Inc).


We are still waiting for the official title / abstract for this talk,  
so this is a temporary title




17:30 - 17:45: Early Survey Results and Some Attack Statistics --   
Danny McPherson.


-

There are 15 minutes left over at the end of the agenda as I'm sure  
some talks will run over their alloted time.


Hopefully this agenda is interesting and you are looking forward to  
the BOF



See you there,
W

Re: NANOG 44 (Los Angeles): ISP Security BOF

[Warren Kumari]

Hi all,

Well, Esthost has decided that they no longer wish to present their  
side of the story, and so their talk has been removed from the  
agenda :-)


This also means that that the more, erm,  operational talks have been  
lengthened and so won't feel quite as rushed...


The revised agenda is below:

4:30 - 4:50: Stealing the Internet -- Anton Kapela
--

4:50 - 5:10: An interim solution to the threat of DNS cache poisoning  
while waiting for DNSSEC. -- Rodney Joffe


--

5:10 - 5:30: Next steps in IRR/X509 --Barry Raveendran Greene, Jason  
Schiller.


--

5:30 - 5:50: Early Survey Results and Some Attack Statistics --   
Danny McPherson.



I will get this (with some abstracts) posted on the NANOG 44 site soon.

Thanks to everyone who will be presenting, and I look forward to  
seeing y'all there!


W


On Oct 6, 2008, at 2:05 PM, Warren Kumari wrote:


Hello all,

NANOG 44 is now less than a week away.
Here is the current program for the ISP Security BOF (NANOG 44,  
October 13, 2008, 4:30 PM - 6:00 PM) -- as always, the program at  
this point is still somewhat fluid and subject to change.



16:30 - 16:45: Stealing the Internet -- Anton Kapela

In Stealing the Internet Kapela will describe a method where an
attacker exploits the BGP routing system to facilitate transparent  
interception of IP packets.
The method will be shown to function at a scale previously thought  
by many as unavailable.
The talk highlights a new twist in sub-prefix hijacking that he  
demonstrated at Defcon 16:
using intrinsic BGP logic to hijack network traffic and  
simultaneously create a 'bgp shunt towards
the target network. This method will be shown to preserve end-to-end  
reachability while creating
a virtual 'wire tap' at the attackers network. He'll cover additive  
TTL modification and
transparent-origin-AS as a means for the attacker to obscure the  
interception.


There will not be a live demonstration of the hijack or interception
methods.

--

16:45 - 17:00: An interim solution to the threat of DNS cache  
poisoning while waiting for DNSSEC. -- Rodney Joffe


--

17:00 - 17:15: Next steps in IRR/X509 --Barry Raveendran Greene,  
Jason Schiller.


-

17:15 - 17:30: Esthost's response to the 'Hostexploit report' --  
Konstantin Poltev (Esthost, Inc).


We are still waiting for the official title / abstract for this  
talk, so this is a temporary title




17:30 - 17:45: Early Survey Results and Some Attack Statistics --   
Danny McPherson.


-

There are 15 minutes left over at the end of the agenda as I'm sure  
some talks will run over their alloted time.


Hopefully this agenda is interesting and you are looking forward to  
the BOF



See you there,
W

Re: The DDOS problem security BOF: Am i mistaken?

[Warren Kumari]

On Oct 14, 2008, at 12:08 PM, Scott Doty wrote:

First, the good news:  so far, the NANOG conference has been very  
valuable and
content-rich, covering a lot of issues that need to be discussed.   
For that, I am grateful.


But now, the bad news(?):  Maybe it's just me  my paranoia, but do  
I detect

an inkling of murk spam going on with some presentations?


I fully agree with you -- some talks are thinly (or not so thinly)  
veiled attempts to convince you to buy a vendor's shiny, new solution.  
There are a large number of reasons for this, and the Program  
Committee works hard (and I think is doing a great job) to limit the  
amount of sales pitch but A: there are a limited number of talks and  
B: many vendors are unable to resist trying to spin their product. I  
suggest that if you have a topic that you would like to present (and  
will keep it sales free) you resent it to the PC.


I *do* however disagree with you that this happened in the talks to  
which you are referring...





Because there seems to be a fundamental misunderstanding, either on  
my part,
or the part of certain vendors: I'm hear to discuss ideas  freely  
share

them, and they are here to discuss (it would seem) their products.


Once again, great -- please submit a talk to the PC and they will  
review it. The PC is always looking for good talks...



Sometimes
both goals coincide, and that is fine...but...

When a vendor at the security BOF starts showing documents that are  
company
confidential, and trying to whip up a climate of fear, that we  
should all
deploy their product in front of our recursive name servers, i get  
this

funny feeling that I am being murk spammed.


Hmmm... The vendor that you are referring to provides authoritative  
DNS for many domains (and, at least some of them I view as  
important, meaning that I would prefer a correct response!). Yes, I  
am sure that he would be happy to have you as a customer and, yes,  
this is feature that differentiates his company, but I did not get the  
impression AT ALL that he was trying to sell his service, but rather  
provide better service to his existing customers, even going so far as  
to provide free devices to people who run large recursive resolvers.  
This helps both his existing customers (who, yes, will be more likely  
to continue using him), but, more importantly helps me as an end user  
feel a little comfortable that the page that I am getting is the  
correct page...





Perhaps that is my own perspective ( paranoia?), but I found the CERT
gentleman's call to monitor icmp backscatter on our authoritative
nameservers far more informative -- and open.

But I was disappointed with two vendors and their presentations: the  
first
had the tactic of saying DNSSEC is the actual solution when asked  
about
why their product would be necessary...completely ignoring the fact  
that
their proprietary interim solution was by no means the only way to  
prevent

cache poisoning attacks.


I may be mistaken, but I didn't get the impression that he believed  
that his solution was the only one -- he repeatedly pointed out that  
DNSSEC is the correct solution and this his solution does not solve  
all of the problems that DNSSEC would -- however, DNSSEC is FAR from  
being fully deployed.



 Indeed, I would daresay it isn't the best, either
by a BCP perspective, or a cost analysis perspective.





To put a finer point on this, i should say that i found myself  
discomforted
by a presentation suggesting that I should put their proprietary  
appliances
between my recursive name servers  the Net, and I am grateful that  
Mr.

Vixie stood up and said that there are other ways of dealing with the
problem.



Hmmm.. We must have VERY different recollections -- I don't remember  
him mentioning how much this would cost, other than that he would be  
give away some to the biggest wins first. Without knowing how much  
these widgets will be, it is not possibly to do a cost comparison, but  
don't discount just how expensive engineering time is, and just how  
hard it is to find competent DNS folks able to deploy something else.


I have chatted with many people about the state of their DNS  
infrastructure -- many people don't care, many people DO care but just  
don't have the cycles to properly maintain it, many have weird  
internal politics around them, and many just don't have the knowledge.  
Some of these are hard to solve, the lack of knowledge is probably the  
easiest, so I would welcome any how0-to, etc guides that would feel  
like writing



Then there was the gentleman with the DDOS detection/mitigation  
appliance,
who flipped through several graphs, which were intended to show the  
number
of each type of attack.  It's unfortunate that there wasn't more  
time for
questions, because I really wanted to ask why http GET and  
spidering

attacks weren't listen on their graphs...more on that in a second.



Hmmm, probably some of this is my 

Re: Google Contact?

[Warren Kumari]

Hi,

Has this been addressed?

W
On Oct 28, 2008, at 1:18 PM, Chris Murray wrote:


Hi All -

I'm having an issue with Google at the moment, could somebody from  
google ping me off list?


Thanks!

--
Chris Murray
Network Administrator
Stargate Connections Inc.
www.stargate.ca

Re: Google Contact?

[Warren Kumari]

Warren would like to recall the message: Google Contact? because he  
didn't mean to click Reply All...


Yup, it has been / is being addressed, and no, I don't really believe  
that the magic mail elves are going to go scrummaging around your  
mailboxes and delete the message...


W

On Oct 28, 2008, at 10:36 PM, Warren Kumari wrote:


Hi,

Has this been addressed?

W
On Oct 28, 2008, at 1:18 PM, Chris Murray wrote:


Hi All -

I'm having an issue with Google at the moment, could somebody from  
google ping me off list?


Thanks!

--
Chris Murray
Network Administrator
Stargate Connections Inc.
www.stargate.ca

NANOG 45: ISP Security BOF - Call for participants

[Warren Kumari]

Hello and Happy New Years all,

NANOG 45 is fast approaching and so here is the call for participants  
for the ISP Security BOF.


This is *your* chance to talk about interesting security related  
topics and provide some feedback on what you would (and would not)  
like to hear about...


Some security thing been buggin' you all year? Some topic that you  
feel strongly about and would like a change to inform others about?  
Step right up and give a talk...


Slides are welcome, but not required...


W

Smart hands around Dulles airport / northern VA.

[Warren Kumari]

Hi all,

This is a mail that I have been meaning to send ever since I moved  
back to the NoVA area, but have only gotten around to now...


Many years ago I used to provide emergency, smart hands type  
assistance to those in need, but had to give this up when I moved out  
of the area. Anyway, I'm back and am willing to start doing this  
again


This is primarily for those cases where you would normally have to fly  
someone out to have them replace a line-card or two, hook up a few  
cables, maybe swap a disk in an array, etc. This is not for those  
cases where you simple need someone to push the reset button, nor for  
rebuilding your entire cage from scratch...


Anyway, if you have gear here and think that you might need to take me  
up on this, drop me a mail and I'll give you my direct contact info...


If you like this idea, and are willing to also provide this sort of  
thing to the community (either in this, or in another area), please  
let me know -- I'll look into setting up a website / mailing list /  
something...



Important disclaimers and limitations:
1: I do not want (nor may I accept) any compensation for this, other  
than good-will and a hope that you might help out someone else in need  
(AKA, I miss the days when this industry was more co-operative).

2: $day_job comes first.
3: If you are a competitor of $day_job you are probably out of luck.
4: If you wants me to plug in a new device, its your responsibility to  
make sure that you have sufficient power.

5: If you are a shady spammer, don't bother...

FAQ:
Q: What!  Are you crazy? I'd never let a stranger into my cage!
A: Huh, neither would I, but some people are less paranoid than us  
and / or know and trust me.


Q: Why are you doing this again? What's in it for you?
A: Back in the day, this industry used to be much more friendly and  
people would often go out of their way to help others. I fully  
understand why this has changed over time, but, well, it sucks.. I'd  
like to try and bring back some of the community feeling.


Q: How did you end up so handsome!?
A: First off, thank you. It was mainly luck

Q: I am a small company looking to build a network for my 200 sales  
people (or something similar). Can I hire you as a consultant?
A: Sorry, no. I have a day job that keeps me busy and entertained. I  
am not a consultant, nor do I have any wish to be one (and can think  
of few things worse). Oh, and I'm assuming that you meant that you own  
and / or work at a small company and not that you *are* a small  
company, because that would be weird.



W


--
Never criticize a man till you've walked a mile in his shoes.  Then if  
he didn't like what you've said, he's a mile away and barefoot.






smime.p7s
Description: S/MIME cryptographic signature

Re: expectations for bgp peering?

[Warren Kumari]

On Jan 21, 2009, at 12:25 AM, mike wrote:


Hello,

So I am just wondering what my expecations should be in a bgp  
peering scenario where I am multihomed with my own ASN and arin  
assigned ip space. At issue is the fact that my backup isp forced me  
to use ebgp multihop to peer with a router internal to their network  
and not the border router I am directly attached to, and secondly,  
that they say I am not allowed to prepend at all - they will do it  
for me, and from the looks of things they have established a route- 
map that just prepends their AS 6 times to my announcement.


Hmmm, this is distinctly unusual.

I'd suspect that the person that you are talking to is a: very new to  
BGP and is just applying the wrong canned route-map or b: the person  
is a little less new to BGP and has reached the Oooh, now I know that  
I'm doing and can twiddle the knobs with the best of them stage. I'd  
suggest trying to find someone else there to talk to


Unless you have specifically bought the service as a backup service  
(and they are clumsily (and poorly) trying to make sure that you don't  
use it as your primary path) I cannot think of why your ISP would do  
this. This also seems a bit worrying  -- either they have enough  
capacity to carry your traffic when you need them to (and so should be  
happy to let you use them and bill you for the bits) or they don't and  
you will be unhappy when your primary goes away.


Are they really really cheap? If you need a backup ISP for  
regulatory reasons and don't really care, thats fine. If however you  
want good performance when your primary goes away, I'd suggest looking  
into this more...







  This smells of bad engineering. I have looked up the bgp report  
for my provider and they have 0 downstream AS's,


Yeah, that is worrying

and the week that this project has taken (and it's still not up and  
working) has left me with less than absolute confidence in the  
provider. I want to know if anyone has an opinion on ebgp multihop  
for external customers, and wether I should really have an  
expectation to be able to assign my prepends as suits my needs?


While multihop is not in itself an issue, it does give one pause and  
it is worth finding out the reason. But, yes, you should expect to be  
able to prepend at will...


W


Are there any conditions that could make this fail that I should be  
aware of?


Mike-





smime.p7s
Description: S/MIME cryptographic signature

Re: Remote hands site or list?

[Warren Kumari]

On Mar 25, 2009, at 7:07 PM, Christopher Morrow wrote:

On Wed, Mar 25, 2009 at 1:58 PM, virendra rode virendra.r...@gmail.com 
 wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Owen Roth wrote:

Hello nanog mailing list,

I was curious how one would go about looking for certain types of  
remote
hands by geography (ie coaxial runs in Phoenix, AZ). Is there  
another

mailing list or a web site that recommends itself?


- 
http://nanog.cluepon.net/index.php/Hands



wkumari also had a list started I believe as well... sadly I can't
find his link now :( Warren where did you hide it?



http://www.ne-where.com/cgi-bin/mailman/listinfo/ne-where

I created this list so that people in the community could provide  
remote hand for each other and just to try and recapture some of the  
community spirt, but, well, got sidetracked.


This is the first public list that I am running, feel free to provide  
feedback...


On an unrelated note -- I recently ended with a bunch of those  
expensive Juniper DS3 cables (BNC to SMZ -- CBL-SMZ-BNC-M-S) -- long  
story, summary is that someone didn't pay their storage fee and I  
snagged them before they got tossed.


Does anyone have a need for them?
They will be distributed in the following order:
1: People that I know (and that haven't called me stupid recently).
2: People in the Reston, VA area (who will pick them up so I don't  
have to ship them).

3: People who do good things for the Internet (- as decided by me).
4: Everyone else.

Please only ask for them if you actually want / need them -- if you  
are just going to dump them, I can do that myself.

Free to a good home, limit one per person.

W
P.S:  Yes, i had though of giving them away on juniper-nsp, but, well,  
didn't want to...





smime.p7s
Description: S/MIME cryptographic signature

Re: The Confiker Virus.

[Warren Kumari]

On Apr 1, 2009, at 12:01 PM, Jason Iannone wrote:


What's the virus doing with all of those domain names?


http://lmgtfy.com/?q=conficker




On Wed, Apr 1, 2009 at 8:38 AM, Michael Holstein
michael.holst...@csuohio.edu wrote:



Of the 50,000 DNS names generated for today ..


Additional info ..

Top 10 ASN by number/name :

5680 -- 1280 ISC-AS1280 Internet Systems Consortium, Inc. 2820  
-- 1668
AOL-ATDN - AOL Transit Data Network2737 -- 23028 TEAM-CYMRU -  
Team Cymru

Inc. 404 -- 760 University of Vienna, Austria  20 -- 1887
NASK-ACADEMIC NASK10 -- 4134 CHINANET-BACKBONE No.31,Jin- 
rong Street
  7 -- 21844 THEPLANET-AS - ThePlanet.com Internet Services,  
Inc.5
-- 8560 ONEANDONE-AS 11 Internet AG  4 -- 12306 PLUSLINE  
Plus.Line AG

IP-Services  3 -- 26496 PAH-INC - GoDaddy.com, Inc.
So you can tell the good guys are still at it pre-registering the  
bulk of

the conflickr-related domain names.

Cheers,

Michael Holstein
Cleveland State University








smime.p7s
Description: S/MIME cryptographic signature

Call for participants, NANOG 46: ISP Security BOF

[Warren Kumari]

Hello all,

So, for once in my life I have not left things till the last minute :-)

NANOG 46 is still a ways off, but I'd like to invite y'all to start  
thinking about topics for the ISP Security BOF, either things that you  
would like to present, or things that you are interested in and would  
like to see someone else present.


You (Yes! You!) can give a security related presentation at the  
upcoming NANOG, thereby earning fame, respect and adoration from your  
friends and colleagues.


W
--
When it comes to glittering objects, wizards have all the taste and  
self-control of a deranged magpie.

-- Terry Pratchett

Re: Config Backup / Inventory

[Warren Kumari]

On Apr 24, 2009, at 4:25 AM, Joshua Eyres wrote:


Hi,

I am looking for a bit of advice around configuration backup /  
inventory. We

currently have a large multi-vendor network which is currently managed
through two separate tools (rancid - http://www.shrubbery.net/rancid  
and ns4
- http://www.noodles.org.uk/ns4.html). Both tools do the job very  
well, but
management have asked that we look for commercial alternatives that  
have a

proper support organisation looking after them.


Yay for management...




IP Service Activator has been mentioned as something which can fit  
this
role, but I haven't had any experience and I haven't heard a lot of  
good
things about it. We are looking for a tool which is flexible that  
allows
configuration backup to textual form for easy restoration as well as  
the

ability to deploy scripted changes to the network quickly.

Do people generally use free tools for network management or are there
viable commercial alternatives?


A large large number of people use things like RANCID and / or some  
homegrown things... The people who are using commercial products (for  
the above role) are usually doing so because they were saddled with  
these requirements by management and / or are a windows shop..


As you say, RANCID is working well, maybe if you explain the costs  
involved in installing / migrating to and supporting a commercial  
product your management will see things saner?


W




Thanks,
Josh




smime.p7s
Description: S/MIME cryptographic signature

Re: P2P agents for software distribution - saving the WAN from meltdown?!?

[Warren Kumari]

On Jun 18, 2008, at 10:42 AM, Adrian Chadd wrote:


On Tue, Jun 17, 2008, Christopher Morrow wrote:


most of the larger free-nix's do BT downloads on release day(s).
Revision3 distributes their content via BT. There were rumors of
Disney and Apple moving to BT models for their content distribution  
at

one point as well.


random type=idea from tonight
If only there was a way for a SP to run a BitTorrent type service for
their clients, subscribing the BT server(s) to known-good (ie, not  
warez-y)

torrents pre-seeded from trusted sources and then leaving it the hell
alone and not having to continuously dump specific torrent files into
it.
/random



Ah, if only there was a way for my SP to go and look all over the web  
and figure out what pages are acceptable for me to browse and block  
out all of the other stuff like porn and warez and phishing --- and  
other objectionable content like creationism / evolution [delete  
whichever is appropriate ], those bastard [insert your least favorite  
ethnic / religious group here ] and any mention of [insert political  
party]. Oh, and anything to do with clowns, they freak me out...



Yes, P2P is not the web, but the general principle still applies -- I  
don't think that handing over the censorship keys to my ISP is a  
reasonable solution...

W





Hm!



Adrian




--
Do not meddle in the affairs of wizards, for they are subtle and quick  
to anger.

-- J.R.R. Tolkien

Re: Possible explanations for a large hop in latency

[Warren Kumari]

On Jun 26, 2008, at 11:36 PM, Randy Bush wrote:


Frank Bulk - iNAME wrote:
Just google tbr1.sl9mo.ip.att.net and it's clear that high  
latency through
that point has occurred before.  And guess what kind of customer  
complained

to me about the latency?  A gamer.


you can pay a lot of money for the net propagation anomaly detection
services that gamers give you for free.




Many years ago I worked for a small Mom-and-Pop type ISP in New York  
state (I was the only network / technical person there) -- it was a  
very free wheeling place and I built the network by doing whatever  
made sense at the time.


One of my favorite customers (Joe somebody) was somehow related to  
the owner of the ISP and was a gamer. This was back in the day when  
the gaming magazines would give you useful tips like Type 'tracert  
$gameserver' and make sure that there are less than N hops.  Joe  
would call up tech support, me, the owner, etc and complain that there  
was N+3 hops and most of them were in our network. I spent much time  
explaining things about packet-loss, latency, etc but couldn't shake  
his belief that hop count was the only metric that mattered.


Finally, one night he called me at home well after midnight (no, I  
didn't give him my home phone number, he looked me up in the  
phonebook!) to complain that his gaming was suffering because it was  
too many hops to get out of your network. I finally snapped and  
built a static GRE tunnel from the RAS box that he connected to all  
over the network -- it was a thing of beauty, it went through almost  
every device that we owned and took the most convoluted path I could  
come up with. Yay!, I figured, now I can demonstrate that latency  
is more important than hop count and I went to bed.


The next morning I get a call from him. He is ecstatic and wildly  
impressed by how well the network is working for him now and how great  
his gaming performance is. Oh well, I think, at least he is happy  
and will leave me alone now. I don't document the purpose of this GRE  
anywhere and after some time forget about it.


A few months later I am doing some routine cleanup work and stumble  
across a weird looking tunnel -- its bizarre, it goes all over the  
place and is all kinds of crufty -- there are static routes and policy  
routing and bizarre things being done on the RADIUS server to make  
sure some user always gets a certain IP... I look in my pile of notes  
and old configs and then decide to just yank it out.


That night I get an enraged call (at home again) from Joe *screaming*  
that the network is all broken again because it is now way too many  
hops to get out of the network and that people keep shooting him...


What I learnt from this:
1: Make sure you document everything (and no, the network isn't  
documentation)

2: Gamers are weird.
3: Making changes to your network in anger provides short term  
pleasure but long term pain.


-

W




randy



--
Do not meddle in the affairs of dragons, for you are crunchy and taste  
good with ketchup.

Re: ICANN opens up Pandora's Box of new TLDs

[Warren Kumari]

On Jun 30, 2008, at 12:54 PM, [EMAIL PROTECTED] wrote:


On Sun, 29 Jun 2008 17:55:53 EDT, Tuc at T-B-O-H.NET said:


220 Sending HELO/EHLO constitutes acceptance of this agreement


Even in a UCITA state that has onerous rules regarding shrink- 
wrapped EULA
terms, I think you'd have a very hard time getting a court to  
enforce an
alleged contract based on this.  And it's different from the usual  
suggestion
to put all activity may be monitored in your telnet/ssh login  
banners, because
there's an expectation that the human will look at a login banner  
when they
login, but there's no expectation that an SMTP server will look at  
the 220
banner any further than checking the first digit is a '2' (go read  
the section

on SMTP reply codes in RFC2821).

Feel free to cite any relevant case law (in fact, even the case law on
login banners read by humans is a tad skimpy - in most cases, it  
does nothing
for intruders, but it protects you from your own users complaining  
their

privacy was violated)...



I have found the biggest advantage of banners to be the fact that you  
learn to recognize your own devices *before* typing your password...


It you *always* have a banner on *all* of your devices, you quickly  
learn to expect them...


For example:
ssh router1.example.net
**
* This device belongs to example.net. Don't login if you
* are not supposed to be here... Blah blah blah.
* 
*
[EMAIL PROTECTED]'s password:

versus:
ssh router1.exsmple.net
[EMAIL PROTECTED]'s password:


Having a cute, customized banner (not the default from the standard  
security templates) helps with this...


W

--
If the bad guys have copies of your MD5 passwords, then you have way  
bigger problems than the bad guys having copies of your MD5 passwords.

-- Richard A Steenbergen

Re: Hardware capture platforms

[Warren Kumari]

On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote:


Hubs sure are fun...



This might be a stupid question, but where can one get small hubs  
these days? All of the common commodity (eg:  4 port Netgear) hubs  
these days are actually switches.


What I am looking for is:
Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.)
Cheap
Simple
10/100/1000Mbps

While a tap would work, I'd prefer a hub because I can then use it to  
connect machines together in a pinch.


W
---

In the past I have bought some cheap 4 port commodity switches (form  
Circuit City or somewhere similar), found the datasheet for the  
chipset (it was a Broadcom something or other) and tied the pin to  
ground that disables the learning mode (actually, I think that the pin  
just set the size of the learning table to be 0 entries).  While this  
works, doing it once was more than enough :-)


I would trunk the ports you are monitoring, and run the port monitor  
on

the trunk port instead (one trunk port, one port per VLAN, plus one
span) which will help with your density. This is assuming the analysis
software you have can read the dot1q tags, but means you do not need  
to

burn two ports per monitor.

-Original Message-
From: James Pleger [mailto:[EMAIL PROTECTED]
Sent: Tuesday, July 29, 2008 19:26
To: [EMAIL PROTECTED]
Subject: Re: Hardware capture platforms

There are several things that you can do with open source solutions,
however looking at the data may be a bit more difficult than something
like Network Generals or Solera Networks capture appliances. It is
still doable and is definitely much much cheaper...

Something you might want to look into is traffic aggregation with a
switch or hub. You can buy an Allied Telesyn switch and basically turn
it into a hub by disabling switchport learning. Just an idea.

You can use regular old tcpdump with the -C option to rotate logs

tcpdump -i blah -s0 -C filesize to rotate, etc.

or you can use Daemonlogger which does pretty much the same thing...

http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html


On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius  
[EMAIL PROTECTED]

wrote:

Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and
especially his books (Tao of Network Security Monitoring and  
Extrusion

Detection) are the best sources I have ever found, concerning [not

only]

taps and[/but] so much more on the subject - proper usage and best
methodologies and practices for network monitoring (and not only for
security!!!)


Stefan

On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow

[EMAIL PROTECTED]

wrote:


On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch  
[EMAIL PROTECTED]

wrote:

Check out packet forensics depending on what your ultimate

requirements

are.




I would also add a 'see packet forensics'...


On Jul 29, 2008, at 7:10 PM, John A. Kilpatrick

[EMAIL PROTECTED]

wrote:



We've deployed a bunch taps in our network and now we need a

platform on

which to capture the data.  Our bandwidth is currently pretty low

but

I've

got 8 links to tap, which means I need 16 ports.  Has anyone done

any

research on doing accurate packet capture with commodity hardware?


--
John A. Kilpatrick
[EMAIL PROTECTED]Email|

http://www.hypergeek.net/

[EMAIL PROTECTED]  Text pages|  ICQ: 19147504
  remember:  no obstacles/only challenges















--
Build a man a fire, and he'll be warm for a day. Set a man on fire,  
and he'll be warm for the rest of his life. -- Terry Pratchett

Re: Hardware capture platforms

[Warren Kumari]

On Jul 31, 2008, at 12:31 PM, Jay R. Ashworth wrote:


On Wed, Jul 30, 2008 at 02:47:11PM -0400, Jon Kibler wrote:

Hubs are still available that are REAL hubs. I got 4 netgears about a
year ago and they are still available.

However, there is a problem with your specification: No hub (that I  
am

aware of) can do 1Gbps. All hubs are 10/100 AFAIK.


Ok, so I guess what I am speaking is not strictly a hub, it is a non- 
learning bridge (single collision domain per port, full duplex, etc).
There used to be a bunch of devices sold like this -- there were a few  
really cheap chipsets (AFAIR, Vitesse SparX VSCsomething was one of  
them -- basically a standard switch chipset that they shaved a few  
cents off because there was no learning logic / memory) that many  
people used in cheap hubs... I still have some of these somewhere  
and will rip the lid off to figure out exactly what it was so I can  
get some more...






And, note carefully: some dual-speed hubs are actually a 10BT hub  
and

a 100BT hub *with a switch between them*.  I forget which brand I
caught this on, but it bit me a couple of years back.

Which speed cable you plug in determines which hub you're talking to.


I see your weird hub story and raise you one:

I went along to one of my wife's clients to help lug a  printer up the  
stairs... We get it on the desk and I go to plug in the Ethernet port  
-- I follow some cables and find this small white switch jammed behind  
a photocopier -- I pull it out and it has, emblazoned in large red  
letters  on the front, 10/100 Hub with Switch -- this was back in  
the day when switches were still cool... I turn it around, and on the  
back there is... a switch, one side is marked 10M and the other is  
marked 100M... After I stopped laughing I tested it, and sure  
enough, its a standard hub, and you can make the ports either run at  
10Mbps or 100Mbps by flipping the switch... I *really* wish I had  
replaced and kept it...


W



Yes, it's weird.

Cheers,
-- jra
--
Jay R. Ashworth   Baylink  [EMAIL PROTECTED]
Designer The Things I  
Think   RFC 2100
Ashworth  Associates http:// 
baylink.pitas.com '87 e24
St Petersburg FL USA  http://photo.imageinc.us +1  
727 647 1274


 Those who cast the vote decide nothing.
 Those who count the vote decide everything.
   -- (Josef Stalin)



--
Do not meddle in the affairs of wizards, for they are subtle and quick  
to anger.

-- J.R.R. Tolkien

Re: I don't need no stinking firewall!

[Warren Kumari]

On Jan 10, 2010, at 1:32 AM, Dobbins, Roland wrote:



On Jan 10, 2010, at 1:22 PM, harbor235 wrote:

Again, a firewall has it's place just like any other device in the  
network, defense in  depth is a prudent philosophy to reduce the  
chances of compromise, it does not eliminate it nor does any  
architecture you can think of, period




Bah, I was trying not to get sucked into the roaring vortex of this  
thread, but I think that folks are ignoring one of the primary  
benefits of firewalls:

Quite simply, its this:

I can now place a checkbox in the Is there a firewall? column of the  
insert random acronym here audit.


While it may be fun to rail against the stupidity, after the Nth time  
that you have had the This is in no way going to help improves  
security and will actually decrease it argument, you realize that, if  
you want to get real work done, you need to choose your battles.


In may cases the auditor knows that the firewall may not make thing  
better, and may make them worse, but he has a set of guidelines that  
the contracting company he is working for dictates, and he needs to  
see the widget to sign on the dotted line. I have had auditors  
cheerfully point out that the way that their specific requirement is  
worded, a commodity CPE device plugged into port somewhere will fully  
satisfy their requirements and did I know that BestBuy has them on  
sale this week?





W



What a ridiculous statement - of course it does.

*The place of the stateful firewall is in front of clients, not  
servers*.


I'm not going to continue the unequal contest of pitting real-world  
operational experience against Confused Information Systems Security  
Professional brainwashing.  One can spout all the buzzwords and  
catchphrases one wishes, but at the end of the day, it's all dead  
wrong - and anyone naive enough to fall for it is setting himself up  
for a world of hurt.


---
Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com

   Injustice is relatively easy to bear; what stings is justice.

   -- H.L. Mencken








smime.p7s
Description: S/MIME cryptographic signature

Re: New netblock Geolocate wrong (Google)

[Warren Kumari]

On Jan 18, 2010, at 5:27 PM, Rosenberry, Eric wrote:

 I just lit up a new IP netblock (assigned directly from ARIN) and the 
 companies that provide Geolocate databases do not have the correct location 
 information available yet.
 
 Specifically Maxmind (http://www.maxmind.com/) thinks we are in Canada and 
 IP2LOCATION (http://www.ip2location.com/) has no data.
 
 For the most part this is benign or at worst slightly impacting since I often 
 get redirected to global load balance nodes up in Canada instead of locally 
 in the North West, however, the more major issue I am running into is that 
 Google chooses to redirect all my users to 
 www.google.cahttp://www.google.ca.
 
 So my questions to others are:
 
 1.   How do I get my data updated in all of the geolocate providers 
 databases as quickly as possible?
 
 2.   What geolocate database does Google use (is it homegrown?) and how 
 do I get them to update my data?

http://www.google.com/support/websearch/bin/request.py?contact_type=ip

If it is urgent / lots of users are grumping at you, feel free to send me an 
off-list mail including the information asked for on that page and I'll follow 
up internally. 

Something that I have often wondered is how folks would feel about publishing 
some sort of geo information in reverse DNS (something like LOC records, with 
whatever precision you like) -- this would allow the folks that geo stuff to 
automagically provide the best answer, and because you control the record, you 
can specify whatever resolution / precision you like. Based upon the sorry 
state of existing reverse, I'm suspecting that there is no point

W

 
 Thanks.
 
 -Eric
 ___
 Eric Rosenberry
 Sr. Network Engineer | Chief Bit Plumber
 
 
 
 iovation
 111 SW Fifth Avenue
 Suite 3200
 Portland, OR 97204
 www.iovation.comhttp://www.iovation.com/
 
 The information contained in this email message may be privileged, 
 confidential and protected from disclosure. If you are not the intended 
 recipient, any dissemination, distribution or copying is strictly prohibited. 
 If you think that you have received this email message in error, please 
 notify the sender by reply email and delete the message and any attachments.

--
Beware that the most effective way for someone to decrypt your data may be 
with rubber hose. --- SSH 1.2.12 README

Re: New netblock Geolocate wrong (Google)

[Warren Kumari]

On Jan 18, 2010, at 8:38 PM, Steven Bellovin wrote:

 
 On Jan 18, 2010, at 8:22 PM, Warren Kumari wrote:
 
 Something that I have often wondered is how folks would feel about 
 publishing some sort of geo information in reverse DNS (something like LOC 
 records, with whatever precision you like) -- this would allow the folks 
 that geo stuff to automagically provide the best answer, and because you 
 control the record, you can specify whatever resolution / precision you 
 like. Based upon the sorry state of existing reverse, I'm suspecting that 
 there is no point
 
 I don't think that that works.  Apart from the problem that you allude to -- 
 people not bothering to set it up in the first place -- IP geolocation is 
 often used for certain forms of access control and policy enforcement.  For 
 example: Regular Season Local Live Blackout: All live, regular season games 
 available via MLB.TV, MLB.com At Bat 2009 and certain other MLB.com 
 subscription services are subject to local blackouts. Such live games will be 
 blacked out in each applicable Club's home television territory, regardless 
 of whether that Club is playing at home or away. 
 (http://www.mlb.com/mediacenter/).  EBay has apparently used IP geolocation 
 (poorly) to control access to certain auctions for items that are illegal in 
 certain jurisdictions or that cannot be exported.

Ah, yes, sorry, I guess I didn't fully explain this...

This wouldn't (well, shouldn't) be used as an authoritative source -- it would 
simple be yet another signal that could be used, and would provide (if the ISP 
so chose) higher resolution.

If you think that the IP is in Uzbekistan and traceroutes, whois and RTT all 
seem to agree with that, but the published LOC type record claims that it is 
just down the road from you in NJ then, well, you would be silly to believe it.
Folks who are currently using geolocation for policy (like MLB.com) must[0] 
realize that this is a fundamentally flawed approach and is only effective 
against a non-determined audience, mustn't they? TOR / proxies / etc will all 
happily get around this blocking and seem much easier for the average user than 
poking at DNS.

W

[0]: Ok, they probably don't, but 



 
   --Steve Bellovin, http://www.cs.columbia.edu/~smb
 
 
 
 
 

--
She'd even given herself a middle initial - X - which stood for someone who 
has a cool and exciting middle name.

-- (Terry Pratchett, Maskerade)

Re: Yahoo and IPv6

[Warren Kumari]

On May 9, 2011, at 9:14 PM, Owen DeLong wrote:

 
 On May 9, 2011, at 9:25 AM, valdis.kletni...@vt.edu wrote:
 
 On Mon, 09 May 2011 18:16:20 +0300, Arie Vayner said:
 Actually, I have just noticed a slightly more disturbing thing on the Yahoo
 IPv6 help page...
 
 I have IPv6 connectivity through a HE tunnel, and I can reach IPv6 services
 (the only issue is that my ISP's DNS is not IPv6 enabled), but I tried to
 run the Start IPv6 Test tool at http://help.yahoo.com/l/us/yahoo/ipv6/ and
 it says:
 We detected an issue with your IPv6 configuration. On World IPv6 Day, you
 will have issues reaching Yahoo!, as well as your other favorite web sites.
 
 The *really* depressing part is that it says the same thing for me, on a 
 *known*
 working IPv6 network.
 
 FWIW, it is happy with my connection and consistently reports positive 
 results.
 
 I'm running my own addresses through HE tunnels and tunnels
 to Layer42.
 
 The tunnels ride over Comcast and Raw Bandwidth DSL.


Yup -- while not perfect, the Yahoo! testing has been working well for me.

Yahoo has to tread a very careful line between giving too little and too much 
information --  I have tried walking a few non-technical folk through 
troubleshooting their v6 connectivity by phone and it is really very hard to 
do, and that is interactively. Writing something that someone can download, 
print and then follow is nigh impossible. No matter how well this guide is 
written, a number of folk will manage to screw it up, and of *course* that will 
be Yahoo's fault

Jason's page at http://test-ipv6.com/ gives way way more information (and the 
page at http://ipv6-test.com/ also gives some more), both of these pages are 
much too complex for the average user.

W
 
 And then when I retry it a few minutes later, with a tcpdump running, it 
 works.
 
 And then another try says it failed, though tcpdump shows it seems to work.
 
 For what it's worth, the attempted download  file is:
 
 % wget http://v6test.yahoo.com/eng/test/eye-test.png
 --2011-05-09 11:44:39--  http://v6test.yahoo.com/eng/test/eye-test.png
 Resolving v6test.yahoo.com... 2001:4998:f00d:1fe::2000, 
 2001:4998:f00d:1fe::2002, 2001:4998:f00d:1fe::2003, ...
 Connecting to v6test.yahoo.com|2001:4998:f00d:1fe::2000|:80... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: unspecified [image/png]
 Saving to: `eye-test.png.1'
 
   [ =   ] 2,086   --.-K/s   in 0s  
 
 2011-05-09 11:44:39 (154 MB/s) - `eye-test.png.1' saved [2086]
 
 Looking at the Javascript that drives the test, it appears the *real* problem
 is that they set a 3 second timeout on the download - which basically means
 that if you have to retransmit either the DNS query or the TCP SYN, you're
 dead as far as the test is concerned.
 
 Well, if you're having to retransmit those intermittently, then, it does seem 
 you
 have some level of brokenness with your network, no?
 
 Owen
 
 

Re: Yahoo and IPv6

[Warren Kumari]

On May 10, 2011, at 12:37 PM, Igor Gashinsky wrote:

 On Tue, 10 May 2011, Iljitsch van Beijnum wrote:
 
 :: On 9 mei 2011, at 21:40, Tony Hain wrote:
 :: 
 ::  Publicly held corporations are responsible to their shareholders to get
 ::  eyeballs on their content. *That* is their job, not promoting cool new
 ::  network tech. When you have millions of users hitting your site every
 ::  day losing 1/2000 is a large chunk of revenue.
 :: 
 :: Nonsense. 0.05% is well below the noise margin for anything that involves 
 humans.
 
 I assure you, it is not. 0.005% might be in the noise, but 0.05% is 
 quite measurable given a large enough audience.
 
 ::  The fact that the big
 ::  players are doing world IPv6 day at all should be celebrated, promoted,
 ::  and we should all be ready to take to heart the lessons learned from
 ::  it.
 :: 
 :: I applaud the first step, but I'm bothered by the fact that no second step 
 is planned.
 
 Just because it's not public, doesn't mean that it hasn't been planned :)
 
 Most of us want to see the data that we get from the first step, before 
 making the decision on which second step to take, I'm sure most people 
 can understand that.


Argck, I cannot believe that I am going to do this, let alone publicly, but 
here goes...

Igor is right on both counts here -- 0.05% is definitely noticeable at these 
sorts of scale, and I'd be shocked if Yahoo didn't have a set of alerts that 
fire if projections differ from actual traffic by this amount. I'm also a 
little surprised that you figured that there were no plans past the event -- 
much of the point of this is for data gathering -- did you figure folk were 
just going to gather the data and then ignore it?

Ok, that fully used up my agreeing with Igor quota for the year...

W

 
 -igor
 

Clearing DF bits...

[Warren Kumari]

Hi there all,

Years ago it used to be a somewhat common practice to clear the DF bit on 
packets, either on all packets, or just on those that that you were going to 
shove through a tunnel (I think the netscreen command was something like set 
vpn foo df-bit clear, cisco had something funky with policy routing IIRC,etc).

This was done both to deal with multiple encapsulations and for the folk that 
block all ICMP for security reasons.

Is this practice still common / do you know of anyone still doing it?

W

Re: Had an idea - looking for a math buff to tell me if it's possible with today's technology.

[Warren Kumari]

On May 19, 2011, at 11:42 AM, Landon Stewart wrote:

 On Thu, May 19, 2011 at 5:05 AM, Vitkovsky, Adam 
 avitkov...@emea.att.comwrote:
 
 inverse problem
 This is what I believe Landon meant in his original post
 
 Everybody started talking about compression -but that is I believe sending
 the result of the function -where both nodes know the function
 
 But how hard if at all possible is to figure out a function(or set of
 functions) and variables that describe the given data
 
 And than just send those functions and variables to the other node
 And let it to recompute the original file
 
 Complex function can be represented by simple numbers to shrink down the
 amount of data to be sent over the wire
 
 If the file is: 1048576
 
 -than that coule be represneted via:
 1*1
 X=2
 Y=10
 Where both nodes would know that 1 = x^y
 
 
 Just wanted to say yes, this is entirely what I meant.  Of course the
 smaller the file the more pointless it gets but still...  If the file was
 1GB instead of just 7 bytes I'm wondering if a regular old workstation could
 put it back together in any reasonable amount of time with the equation.

While many folk have said You've just invented compression, I'm going to be a 
little more specific -- Wavelet compression.

W
 
 
 -- 
 Landon Stewart lstew...@superb.net
 SuperbHosting.Net by Superb Internet Corp.
 Toll Free (US/Canada): 888-354-6128 x 4199
 Direct: 206-438-5879
 Web hosting and more Ahead of the Rest: http://www.superbhosting.net
 

Re: ICANN to allow commercial gTLDs

[Warren Kumari]

On Jun 17, 2011, at 9:13 PM, David Conrad wrote:

 On Jun 17, 2011, at 4:04 PM, Owen DeLong wrote:
 I really don't think that namespace issues are part of the role for the ASO 
 AC.
 
 Why do you think there is an ASO?
 
 This is clearly a problem for ICANN's disaster-ridden domain-name side, and 
 not
 for the ASO/NRO side of things.
 
 Because there is clearly no inter-relation between domains and address and the
 operation of the Internet.
 
 Operationally, it's a horrible idea, but,
 most of us in layers 1-4 stopped paying much attention to the disasters 
 happening
 at ICANN for DNS along time ago as we sort of came to believe that we didn't 
 have
 enough money to bribe^h^h^h^h^hinfluence the right people in a sufficiently
 meaningful way to make our voices heard.
 
 Aren't you one of the folks who state that if you don't participate in PPML 
 then
 you have no reason to criticize ARIN policies?

+1 -- If you haven't bothered to be involved, you have lost the right to 
kvetch… If enough operational folk had bothered to stay involved, ICANN would 
be more operational. Claiming that it is all driven by money is a cop out. Yes, 
it's very political, yes there are LOTS of lawyers and policy folk, yes the 
atmosphere is not fun, yes the registries and registrars are the big players 
(because they have bothered to play), but technical folk CAN and DO make a 
difference…

Warren serves on the SSAC Kumari

 
 Regards,
 -drc
 
 

Re: How to begin making my own ISP?

[Warren Kumari]

On Sep 16, 2011, at 2:53 PM, valdis.kletni...@vt.edu wrote:

 On Fri, 16 Sep 2011 18:42:18 -, bmann...@vacation.karoshi.com said:
 Configure Quagga w/ the obtained ASN and announce the IP prefix(es).
 
 TaDa ...  You are an ISP!
 
 Now all you need is a business plan that pays for the rack space. ;)

http://www.monkeybagel.com/consult.html

I'd also recommend reading the Systems Hardware Integration Tasks linked to 
from: http://www.monkeybagel.com/process.html

W

Does anyone know Jared's birthday?

[Warren Kumari]

Yup, I did think it was worth asking the entire list.

W

Re: Does anyone know Jared's birthday?

[Warren Kumari]

On Wednesday, June 4, 2014, manning bill bmann...@isi.edu wrote:

 did you ask Jared?


Yup.

And he updated it on Facebook to throw us off the scent...

W



 /bill
 Neca eos omnes.  Deus suos agnoscet.

 On 4June2014Wednesday, at 12:15, Warren Kumari war...@kumari.net
 javascript:; wrote:

  Yup, I did think it was worth asking the entire list.
 
  W

Re: Mikrotik RouterBoard and Ubiquiti Networks Routing and Switching Solutions

[Warren Kumari]

On Mon, Aug 11, 2014 at 8:22 PM, Colton Conor colton.co...@gmail.com wrote:
 I am interested to hear opinions on Mikrotik and Ubiquiti Networks routing
 and switching products. I know both hardware providers are widely deployed
 in WISP networks, but I am less interested in their wireless solutions and
 more in their wired products.


Probably not the experiences you are looking for, but I replaced my
home CPE (a Netscreen SSG) with a Ubiquiti Edge Router -- there was a
very small learning curve (their CLI is different -- feels like
somewhat less polished JunOS to me, some simply things like completion
don't work), but after 15 minutes or so was all set. Sine then it has
remained perfectly stable, has a pretty GUI in case you want a quick
graph of bandwith, etc.


We have also used them when building the IETF network to start
pre-announcing the space (we go to the location a few weeks early,
test the circuits and BGP peerings, and then start announcing the
space - this helps some with some geo-location systems). We have also
used them when cutting over the guest rooms (when we cut over hotel
guest rooms to the IETF infrastructure and space, we sometimes
continue to route and NAT the hotels (RFC1918) space for a while so
that folk who still have a DHCP address can continue to work until
their lease expires).


W

 I know most of their switches and routers are software based, but that
 might not necessarily be a bad thing since everyone is going to SDN
 anyways. Their products are 1/10th or less of the cost of
 the equivalent Cisco/Juniper products.

 How stable and feature rich are both of their platforms? How do both of
 their command line interfaces compare to Cisco or Juniper? Is it easy to
 train a Cisco tech how to use a Mikrotik or Ubiquiti Networks product?


 *Ubiquiti Networks software is based on a version of Vyatta I believe. As
 many of you know Vyatta was bought by Brocade. I have heard that Vyatta is
 very Juniper OS like. *Ubiquiti just release a line of switches that have
 an amazing price and seem to support wire speed switching. Their EdgeRouter
 is supposedly faster than Mikrotiks solutions. They are also traded on the
 stock market, and seem to be doing well as a company.
 http://www.ubnt.com/products/

 Mikrotik also seems to make routers and switches. I am not sure what their
 software is based on, but it does support advanced features such as MPLS.
 Not sure about their switches, but they seem to be dirt cheap! What is
 their command line interface like? I couldn't find any financial
 information on this company, but they seem to be located in Latvia?
 http://routerboard.com/

 Does anyone have any meaningful insight to both companies? Why haven't they
 made a dent in the switching and router market with their amazing price
 points? Am I missing something here?

Re: So Philip Smith / Geoff Huston's CIDR report becomes worth a good hard look today

[Warren Kumari]

On Wed, Aug 13, 2014 at 1:40 AM,  valdis.kletni...@vt.edu wrote:
 On Wed, 13 Aug 2014 08:08:04 +0300, Hank Nussbacher said:

 We went with 768 - enough time to replace the routers with ASR9010s.  It is
 merely a stop-gap measure to give everyone time to replace their routers in
 an orderly fashion.

 The same people who, knowing the 6509 had this default config issue, and
 neither replaced the gear nor did the reconfig to buy time *before* the
 wall got hit, are going to replace said 6509 in orderly fashion?


Sadly enough:
A: not everyone knew about the issue - there are a large number of
folk running BGP on 65xx and taking full tables who are not plugged
into NANOG / the community. In many cases they are single homed
enterprise folk, but run BGP anyway (because com consultant set it up,
some employee with clue did it years ago and then left, etc).

B: they *did* know about the issue, but convincing management to spend
the cash to buy hardware that doesn't suck was hard, because
everything is working fine at the moment -- some folk needed things
to fail spectacularity to be able to justify shelling out the $$$ (
yes, they could recard the TCAM, but they are using this as an excuse
to get some real gear)...

Am I overly cynical, or does this all work out perfectly for some
vendors? I'm guessing that a certain vendor is going to see a huge
number of orders for new equipment, for an event that could have been
(and was) easily predicted... Here, buy my widget... and then you'll
come back in a few years and buy another one.. mwahahahah.
Yup, folk purchasing these *should* have known (not like there was no
discussions of this), but, well, not everyone spends all day reading
NANOG / RIPE / CIDR report...

W



 Hank, you gotta learn to wear respiratory apparatus when working near
 open containers of magic router pixie dust - that stuff can screw you up
 if you inhale it. :)



-- 
--
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

An update from the ICANN ISPCP meeting...

[Warren Kumari]

Those of y'all who were at NANOG62 may remember a presentation from the ICANN
Internet Service Provider and Connectivity Providers Constituency (ISPCP).

I feel somewhat bad because I misunderstood what they were sayingin,
and kinda lost my cool during the preso.  Anyway, the ISPCP met at
ICANN 51 last week. Unfortunately I was not able to attend, but the
meeting audio stream is posted at:
http://la51.icann.org/en/schedule/tue-ispcp

If you'd rather read than listen, the transcript is posted here:
http://la51.icann.org/en/schedule/tue-ispcp/transcript-ispcp-14oct14-en.pdf

I snipped a bit that mentions NANOG:

The next outreach experience that we had was at NANOG. NANOG, as you
may know, is the North American Network Operators Group, an area where
we really wanted to make an impact because it is the network operators
groups that can really bring the insight that we need to act on being a unique
and special voice within the ICANN community on issues that matter to ISPs
around some of the things that are on our agenda today, such as universal
access, such as name collisions. And we wanted to get more technical voices
in the mix and more resources in the door so that we could make a better
impact there.
A lot of what we received when we stood up to give our presentation were
messages from people who had attempted to engage in ICANN in the past or
attempted to engage in the ISPCP in the past and had had very difficult time
doing. They said when you come into this arena you spend so much time
talking about process, so much time talking about Whois and what board
seats, about what needs to happen around transparency. I'm a technical guy,
I want to focus on technical issues and I don't have a unique venue for being
able to do that.
So we spent some time as a group trying to figure out how we can address
that because we do need those voices. Our goal has been to take the
feedback that we receive from NANOG and create an action plan to make
sure that we can pull in voices like that and go back to the NOG community,
go back to the technical operators community, bring them on board and say
we've got a different path for you.



Anyway, go listen / read the full transcript if you are so inclined...

W


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: An update from the ICANN ISPCP meeting...

[Warren Kumari]

On Thu, Oct 23, 2014 at 6:15 PM, Eric Brunner-Williams
brun...@nic-naa.net wrote:
 some history.

 at the montevideo icann meeting (september, 2001), there were so few
 attendees to either the ispc (now ispcp) and the bc (still bc), that these
 two meetings merged. at the paris icann meeting (june, 2008) staff presented
 an analysis of the voting patters of the gnso constituencies -- to my
 non-surprise, both the bc and the ispc votes (now ispcp) correlated very
 highly with the intellectual property constituency, and unlike that
 constituency, originated very little in the way of policy issues for which
 an eventual vote was recorded. in other words, the bc and ispc were, and for
 the most part, imho, remain captive properties of the intellectual property
 constituency.

 this could change, but the isps that fund suits need to change the suits
 they send, the trademark lawyer of eyeball network operator X is not the vp
 of ops of network operator X.

Unless folk here *like* having their views represented as being
aligned with intellectual property folk?

Well, do you? If not, come to an ICANN meeting and say so...

W



 meanwhile, whois, the udrp, and other bits o' other-people's-business-model
 take up all the available time.

 eric



 On 10/23/14 2:58 PM, Warren Kumari wrote:

 Those of y'all who were at NANOG62 may remember a presentation from the
 ICANN
 Internet Service Provider and Connectivity Providers Constituency (ISPCP).

 I feel somewhat bad because I misunderstood what they were sayingin,
 and kinda lost my cool during the preso.  Anyway, the ISPCP met at
 ICANN 51 last week. Unfortunately I was not able to attend, but the
 meeting audio stream is posted at:
 http://la51.icann.org/en/schedule/tue-ispcp

 If you'd rather read than listen, the transcript is posted here:

 http://la51.icann.org/en/schedule/tue-ispcp/transcript-ispcp-14oct14-en.pdf

 I snipped a bit that mentions NANOG:

 The next outreach experience that we had was at NANOG. NANOG, as you
 may know, is the North American Network Operators Group, an area where
 we really wanted to make an impact because it is the network operators
 groups that can really bring the insight that we need to act on being a
 unique
 and special voice within the ICANN community on issues that matter to ISPs
 around some of the things that are on our agenda today, such as universal
 access, such as name collisions. And we wanted to get more technical
 voices
 in the mix and more resources in the door so that we could make a better
 impact there.
 A lot of what we received when we stood up to give our presentation were
 messages from people who had attempted to engage in ICANN in the past or
 attempted to engage in the ISPCP in the past and had had very difficult
 time
 doing. They said when you come into this arena you spend so much time
 talking about process, so much time talking about Whois and what board
 seats, about what needs to happen around transparency. I'm a technical
 guy,
 I want to focus on technical issues and I don't have a unique venue for
 being
 able to do that.
 So we spent some time as a group trying to figure out how we can address
 that because we do need those voices. Our goal has been to take the
 feedback that we receive from NANOG and create an action plan to make
 sure that we can pull in voices like that and go back to the NOG
 community,
 go back to the technical operators community, bring them on board and say
 we've got a different path for you.



 Anyway, go listen / read the full transcript if you are so inclined...

 W






-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Oh dear, we've all been made redundant...

[Warren Kumari]

Found on Staple's website:
http://www.staples.com/NetReset-Automated-Power-Cycler-for-Modems-and-Routers/product_1985686

Fixes all issues, less downtime, less stress...
Improves performance, eliminates buffering...
It slices, it dices in teeny, tiny slices.
It makes mounds of julienne fries in just seconds.
...

Description - copied here for convenience:

All the issues associated with the Internet being down can be solved by
power cycling the modem and router. But that can be hard to do! NetReset
resolves network issues by offering sequential power cycling. This means
that when the modem and router are plugged into the device, they are
powered up at different times. The modem is powered up first, then a minute
later, the router is powered up. This rebooting will occur at initial
setup, every 24 hours and after a power failure. Do you have a modem/router
combo? No problem! NetReset will also power cycle the modem/router combo.


Automatically resets user's Internet every 24 hours
Maximizes Internet speed & reliability
Eliminates media stream buffering
Hands-free Internet reset
Resets hard-to-reach modem/router
Less Internet downtime
Less daily stress
No need to manually reset
Reset occurs at programmed time
Updated information from Internet service provider
Proper reboot after a power failure
Resetting allows equipment to auto-correct issues

Re: sub $500-750 CPE firewall for voip-centric application

[Warren Kumari]

Yeah, the EdgeRouter series do not suck.
Fast, stable, easy to manage (although the broken tab completion drives me
nuts ('sho ip route' should just work, I'm too old to retrain my
fingers...) - other than that they are great...

W

On Thu, May 5, 2016 at 8:28 PM Jared Mauch  wrote:

>
> > On May 5, 2016, at 4:52 PM, Javier J  wrote:
> >
> > I'm a fan of the EdgeRouterLite3
> >
> >
> > I don't manage many small businesses networks anymore because we now do
> > only 100% cloud and remote work but I started deploying them to all my
> old
> > clients I still have on retainer.
> >
> >
> > It is a wonderful solid set it, and forget it device and you can manage
> it
> > with ssh (it is basically running a fork of Vyatta under the hood on
> Cavium
> > hardware which is nice because it does lots of hardware offload like any
> > other enterprise device.)
>
> I’ll +1 the Edgerouter series.  They are cheap and hit the right price
> performance ratio for most homes.
>
> You can do site-to-site IPSEC VPN stuff and easily SSH + tcpdump if
> necessary.
>
> If you are looking for more complex blocking rules and services, you need
> to be
> looking at something like the Deteque DNS service or the Cisco/OpenDNS
> services
> instead to nuke outbound malware connections and such.
>
> - Jared
>
>

NIST looking for comments on "Secure Interdomain Traffic Exchange – BGP Robustness and DDoS Mitigation: NIST Releases Draft NIST SP 800-189"

[Warren Kumari]

Hey all,

NIST is looking for comments on "Secure Interdomain Traffic Exchange – BGP
Robustness and DDoS Mitigation: NIST Releases Draft NIST SP 800-189"

They recently extended the deadline for comments to March 15, 2019, and so
it looks like they would really like feedback

-

From: NIST Computer Security Division 
Subject: Secure Interdomain Traffic Exchange – BGP Robustness and DDoS
Mitigation: NIST Releases Draft NIST SP 800-189

NIST has released Draft NIST Special Publication (SP) 800-189, Secure
Interdomain Traffic Exchange: BGP Robustness and DDoS Mitigation, which
provides technical guidance and recommendations for deploying technologies
that improve the security of interdomain traffic exchange. The document
focuses on securing the interdomain routing control (i.e., Border Gateway
Protocol) traffic as well as mitigating Distributed Denial of Service
(DDoS) attacks.  It is intended to guide information security officers and
managers of federal enterprise networks. The guidance also applies to the
network services of hosting providers (e.g., cloud-based applications and
service hosting) and Internet Service Providers (ISPs) when they are used
to support federal IT systems. The guidance will also be useful for
enterprise and transit network operators and equipment vendors in general.

A public comment period for this document is open until March 15, 2019.
Email Comments to: sp800-...@nist.gov

Publication Details:
https://csrc.nist.gov/publications/detail/sp/800-189/draft

CSRC Update:
https://csrc.nist.gov/news/2018/nist-releases-draft-sp-800-189-for-comment


Warren.


--
I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
   ---maf

Re: DOs and DONTs for small ISP

[Warren Kumari]

On Mon, Jun 3, 2019 at 11:34 PM Brandon Martin  wrote:
>
> On 6/3/19 9:56 AM, Jon Lewis wrote:
> > 3) Don't advertise one transit provider's routes to another.  Each should
> > be filtering your routes, but you never know.  Come up with, and use
> > BGP communities to control route propagation.  As you grow, it sucks
> > having to update prefix-list filters in multiple places every time
> > something changes...like a new customer with their own IPs.
>
> To reiterate all this, FILTER EVERYTHING.
>
> To start with, explicitly specify in a route-map or similar everything
> you want to advertise.  I usually create a separate route-map for each
> transit/peer and include what I want to advertise via prefix lists (for
> my IP space) and/or communities (for downstream BGP-speaking customers
> if anticipated).

I think a related *principle* is: "Build everything as though you are
expecting to scale."

This doesn't mean "spend lots of money to buy huge
[routers|servers|commercial software|], but rather "when you plan
your addressing structure and routing policies and monitoring and
device config generation and... keep the in mind the question "If this
suddenly takes off, and I hire N more people to run this, can I
explain to them how it works? Do I have documentation I can point them
at or is it stuck in my head / on the devices? If I need to add
another M customers in the next month, can I do that easily?".

This is related to the FILTER EVERYTHING -- when you turn up a new
customer / peer / transit / whatever, you shouldn't be sitting around
trying to figure out how you will write their route-map /
policy-options -- this leads to weird one-offs, and quick hacks.
Instead you should have policies already largely designed and simply
plug in their prefixes (or, better yet, use bgpq3 or similar to build
and populate these). Obviously there will be some cases where a new
connection does require some special handling, but that *should* just
be a plugin/chain in an existing policy-statement. Related to this is
how you end up naming things -- I recently found 9 variants of
firewall-filters which basically do:

filter ACCEPT {
   term ACCEPT {
then accept;
  }
}
named things like: ACCEPT, ACEPT, Accept, Allow, Permit_all,
AcceptAll, dontdrop [0].

Obviously, there is a tension in the "design for scale" - while it
would be great to design a complete automation system so that
everything from installing a new customer to a new sites is simply
typing 'make ' and having everything pull from a database, at
some point you will need to actually build a network, or you'll never
have customers :-) Just keep in mind that "Am I building myself into a
corner here?". E.g it only takes 10 or 15 minutes to install something
like NetBox to keep track of addresses (and prefixes and racks and
connections and ...) -- stuffing this in a spreadsheet might save you
a few minutes *now*, but will this scale? Can $new_person easily
figure it out?


W
[0]: My personal favorite is:
filter Accept_All {
term Accept {
then {
count dropped;
reject;
}
}
term filter_ {
from {
prefix-list {
;
   }
}
then accept;
}
term NEXT {
then log;
}
}

Presumably this all made sense to 
when they stuck it in at 3AM to deal with some crazy issue, but...



>
> When you turn on the session, check what you're squawking AND WHAT
> YOU'RE FILTERING.  You shouldn't be filtering anything you don't expect.
>   Belt + suspenders.
>
> The same goes for anything you accept.  Obviously for a blended full
> transit BGP edge router, you're probably going to accept almost
> everything.  But if you only want default + on-net, try to filter using
> communities from the peer, etc.  Again, right when you turn on the
> session, "sh ip bgp ... filtered" of whatever's equivalent on your
> platform.  If you're filtering something you don't expect to be
> receiving at all, figure out where the misunderstanding or
> misconfiguration lies.
>
> And of course it goes without saying that, if you've got BGP speaking
> customers, you filter the heck out of them.  Use ROAs and/or RPKI if you
> can to automatically generate filter lists.  Encourage your upstreams to
> do the same if they're filtering you (and they probably are, or at least
> should be, if you're new).  Remember that you are responsible for every
> route you advertise, at the end of the day, even if you only advertised
> it because a downstream network made a boo-boo and you didn't filter it.
>
> Filters are useful on your IGP, too, but there's so many ways to set all
> that up that it's a bit more difficult to come up with nearly universal
> best practices.  Generally speaking, be careful with redistribution,
> never distribute BGP into IGP or vice versa unless you have a really,
> really good reason to, and consider filters between both IGP
> areas/regions or protocols (e.g. RIP 

Re: Power cut if temps are too high

[Warren Kumari]

I used to work for a small, fairly crappy ISP -- the "datacenter" was
a converted brick garage / loading dock. In order to provide cooling,
they had chipped out a bunch of bricks, and mounted in 8 or so AC
units, all in a line.

We monitored everything with WhatsUp Gold[0] - one (hot) night I'm
oncall, and at 3:30AM I get an alert that the environmental sensors on
one of the routers thinks it's too hot. I'm tired and grumpy, and it's
only slightly too hot, so I ack it and go back to bed. A short while
later I get paged again - another router now thinks it is
uncomfortably warm. Still grumpy, so I ack that too, and back to bed.
Sure enough, 20 minutes later, another page Fine, I get dressed,
drive over to the location -- and realize that bricks / mortar are
strong in compression, but weak in tension - the AC window units have
been quietly vibrating for many years, and the entire row of bricks
above the AC units has popped out. All the AC units are lying outside
the building on the grass, still running :-) I stared at them for
a bit, unsure what to do -- so I turned them off, bumped up the
monitoring levels, and went back to bed... Next day we blocked up the
hole, installed some temporary chillers, and then finally installed
real colling

There isn't much point to this story, but I've got a cold, and wanted
to share... :-P

W
[0]: Wow, I just realized that WUG still exists... huh.

On Tue, May 28, 2019 at 9:13 AM Thomas Bellman  wrote:
>
> On 2019-05-27 18:18 +, Mel Beckman wrote:
>
> > Before the trigger temperature is reached, the NMS would have sent
> > various escalating alarms to on call staffers, who hopefully would
> > intervene before this point.
>
> Would they actually have time to react and do something?  In our
> datacenters, we reach our cut-off temperature in about 20 minutes
> if cooling stops.
>
>
> > This system has triggered one time, successfully shutting down the data
> > center on a holiday weekend when people missed their notifications, and
> > undoubtedly saved a lot of hard drives. When we got to the room the
> > temperature was over 115°, but the power was cut at 95°.
>
> Presumably that was °F, not °C.
>
> I have heard from people who did *not* have automatic cutting of the
> power at high temperatures.  Their computer room reached 100°C in
> places; some keyboards apparently looked like a certain Salvador Dali
> painting afterwards...  (But I think they had very few actual servers
> or disk drives breaking.)  The reason it didn't get even hotter, was
> that as temperature rose, servers started overheating and shut them-
> selves down, thus lowering power disippation more and more.
>
>
> Our system for cutting power at high temperatures is part of the PLC
> monitoring power and temperature in the computer rooms.  It sends a
> signal to the large breakers connecting the power subcentrals (where
> all the 16A fuses are) to the power rail feeding the room.  I believe
> our PLCs are from Schneider Electric, but anyone who delivers PLCs
> for controlling power and cooling in a datacenter should be capable
> or programming their PLCs to do the same.  You just need to remember
> putting it in the specifications when you contract the building. :-)
>
>
> /Bellman
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: DOs and DONTs for small ISP

[Warren Kumari]

On Mon, Jun 3, 2019 at 1:09 PM Fletcher Kittredge  wrote:
>
>
> Here is your checklist in descending order of importance:
>
> market opportunity
> finding the right partners (see below)
> financial
> sales and marketing
> organizational capacity and HR
> legal, regulatory
> capital acquisition
> security
> ...
> ...
> ...
> technical including equipment selection, routing policy, filtering, etc
>
> It is a stone cold lock that the success of your new ISP will governed by 
> factors other than technical. Your most important task is to find competent  
> financial and marketing people you can respect and trust. If the market 
> opportunity exists and you find them, you will succeed. If you don't, all the 
> technical excellence in the world won't help you. The road is littered with 
> technically excellent companies that failed.

Indeed, but you *also* need to have some technical clue. Two or three
years ago a friend and I tried to start a local wireless ISP -- I was
doing this purely as a "My home Internet access sucks, and I'll
happily donate time, equipment, IP space and some startup capital to
fix this" play -- unfortunately it turns out that he and I had very
different ideas on, well, basically everything. I wanted an actual
architecture / design, and diagrams and routin' and such. He was much
more of "We don't need a list of IPs, if I ping it and can't reach it
it must be free" / "routing is too hard, let's just put it all in a
switch and... um... NAT!". I wanted a plan, and was willing to put in
the time and effort to build Ansible / Puppet / an NMS / AAA, etc, he
was more seat-of-the-pants.

But yes, even if we had good technology this would have failed - there
was no real business plan (other than "The current provider is really
bad, if we build something else, people will be breaking down the door
to sign up"), no real marketing plan (see previous), etc.

He was also a bit of a gun nut, and so would arrive at customers with
a (holstered) firearm belted on -- even in Virginia this was not a
winning business move.

Starting a successful ISP is this day and age is hard - make sure
that, if you do it, you and whoever you are doing this with are
compatible, are both committed, and have similar views on things...

W


>
>
>
> On Mon, Jun 3, 2019 at 8:05 AM Mehmet Akcin  wrote:
>>
>> hi there,
>>
>> I know there are folks from lots of small ISPs here and I wanted to check-in 
>> on asking few advice points as I am involved building an ISP from 
>> green-field.
>>
>> Usually, it's pretty straight forward to cover high-level important things, 
>> filters, routing policies, etc.but we all know the devil is in the details.
>>
>> I am putting together a public DOs and DONTs blog post and would love to 
>> hear from those who have built ISPs and have recommendations from Billing to 
>> Interconnection, Routing policy to Out of the band  & console setup, 
>> Software recommendations, etc. Bottom line is that I would like to publish a 
>> checklist with these recommendations which I hope will be useful for all.
>>
>> thanks in advance for your help and recommendation.
>>
>> Mehmet
>>
>>
>
>
> --
> Fletcher Kittredge
> GWI
> 207-602-1134
> www.gwi.net



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: SSL VPN

[Warren Kumari]

OpenVPN AS?

I’ve been running it for ~20 users for many years — it just works, has
clients for many OSes, etc.

W

On Sat, Jun 1, 2019 at 10:54 AM Mehmet Akcin  wrote:

> Hey there
>
> I am trying to choose SSL VPN for a remote office 3-4 people max each any
> given time.
>
> I have looked at Pulse and Cisco, and wanted to check in here for
> recommendations on latest trends.
>
> Trying to get a solution easy to manage and won’t break the bank with
> licenses when team grows to 10.
>
> Thanks in advance.
>
> Mehmet
> --
> Mehmet
> +1-424-298-1903
>
-- 
I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
   ---maf

Re: DOs and DONTs for small ISP

[Warren Kumari]

On Mon, Jun 3, 2019 at 2:55 PM Fletcher Kittredge  wrote:
>
>
> I would respectfully point out that my point about the importance of finding 
> the right partners. For you, sounds like it was good to have opportunity to 
> get out of this venture.


Oh, goodness yes -- however, I *still* have barely working Internet
access at that location (it's basically a weekend home) -- I've
somewhat started down the Jared Mauch "buy used vibratory plow, trench
(house is off a dirt road, neighbors won't have an issue with right of
way if I provide them free bits!), install fiber" path. My driveway is
@1/4 mile, the dirt road is 0.6 miles and at the end of the road there
is (what used to be) Quest fiber. Unfortunately the fiber was
originally run for Mount Weather
(https://en.wikipedia.org/wiki/Mount_Weather_Emergency_Operations_Center)
and no-one I've asked seems to know who controls it now, and or swears
that it doesn't exist (although the local Miss Utility / VA811 will
come mark it if you call).

Some dark nights, when trying to use the network while everyone is
using NetFlix, and the RTT for my SSH sessions starts exceeding
1,500ms I strongly consider walking to the end of the road with a
nice, sharp shovel, and then talking to the splicers when they come
repair the damage :-)

W

>
> On Mon, Jun 3, 2019 at 2:40 PM Warren Kumari  wrote:
>>
>> On Mon, Jun 3, 2019 at 1:09 PM Fletcher Kittredge  wrote:
>> >
>> >
>> > Here is your checklist in descending order of importance:
>> >
>> > market opportunity
>> > finding the right partners (see below)
>> > financial
>> > sales and marketing
>> > organizational capacity and HR
>> > legal, regulatory
>> > capital acquisition
>> > security
>> > ...
>> > ...
>> > ...
>> > technical including equipment selection, routing policy, filtering, etc
>> >
>> > It is a stone cold lock that the success of your new ISP will governed by 
>> > factors other than technical. Your most important task is to find 
>> > competent  financial and marketing people you can respect and trust. If 
>> > the market opportunity exists and you find them, you will succeed. If you 
>> > don't, all the technical excellence in the world won't help you. The road 
>> > is littered with technically excellent companies that failed.
>>
>> Indeed, but you *also* need to have some technical clue. Two or three
>> years ago a friend and I tried to start a local wireless ISP -- I was
>> doing this purely as a "My home Internet access sucks, and I'll
>> happily donate time, equipment, IP space and some startup capital to
>> fix this" play -- unfortunately it turns out that he and I had very
>> different ideas on, well, basically everything. I wanted an actual
>> architecture / design, and diagrams and routin' and such. He was much
>> more of "We don't need a list of IPs, if I ping it and can't reach it
>> it must be free" / "routing is too hard, let's just put it all in a
>> switch and... um... NAT!". I wanted a plan, and was willing to put in
>> the time and effort to build Ansible / Puppet / an NMS / AAA, etc, he
>> was more seat-of-the-pants.
>>
>> But yes, even if we had good technology this would have failed - there
>> was no real business plan (other than "The current provider is really
>> bad, if we build something else, people will be breaking down the door
>> to sign up"), no real marketing plan (see previous), etc.
>>
>> He was also a bit of a gun nut, and so would arrive at customers with
>> a (holstered) firearm belted on -- even in Virginia this was not a
>> winning business move.
>>
>> Starting a successful ISP is this day and age is hard - make sure
>> that, if you do it, you and whoever you are doing this with are
>> compatible, are both committed, and have similar views on things...
>>
>> W
>>
>>
>> >
>> >
>> >
>> > On Mon, Jun 3, 2019 at 8:05 AM Mehmet Akcin  wrote:
>> >>
>> >> hi there,
>> >>
>> >> I know there are folks from lots of small ISPs here and I wanted to 
>> >> check-in on asking few advice points as I am involved building an ISP 
>> >> from green-field.
>> >>
>> >> Usually, it's pretty straight forward to cover high-level important 
>> >> things, filters, routing policies, etc.but we all know the devil is in 
>> >> the details.
>> >>
>> >> I am putting together a public DOs and DONTs blog post and would love to 
>> >> hear from those who 

Re: someone is using my AS number

[Warren Kumari]

On Thu, Jun 13, 2019 at 9:59 AM Joe Abley  wrote:
>
> Hey Joe,
>
> On 12 Jun 2019, at 12:37, Joe Provo  wrote:
>
> > On Wed, Jun 12, 2019 at 04:10:00PM +, David Guo via NANOG wrote:
> >> Send abuse complaint to the upstreams
> >
> > ...and then name & shame publicly. AS-path forgery "for TE" was
> > never a good idea. Sharing the affected prefix[es]/path[s] would
> > be good.
>
> I realise lots of people dislike AS_PATH stuffing with other peoples' AS 
> numbers and treat it as a form of hijacking.
>

Actually, I've been meaning to start a thread on this for a while.

I have an anycast prefix - at one location I'm a customer of a
customer of ISP_X &  ISP_Y & ISP_Z. Because ISP_X prefers customer
routes, any time a packet touches ISP_X, it goes to this location,
even though it is (severely) suboptimal -- things would be better if
ISP_X didn't accept this route in this location.

Now, the obvious answer of "well, just ask your provider in this
location to not announce it to ISP_X. That's what communities / the
telephone were invented for!" doesn't work for various (entirely
non-technical) reasons...

Other than doing path-poisoning can anyone think of a way to
accomplish what I want? (modulo the "just become a direct customer
instead of being a customer of a customer" or "disable that site", or
"convince the AS upstream of you to deploy communities / filters").
While icky, sometimes stuffing other people's AS in the path seems to
be the only solution...

W


> However, there's an argument that AS_PATH is really just a loop-avoidance 
> mechanism, not some kind of AS-granular traceroute for prefix propagation. In 
> that sense, stuffing 9327 into a prefix as a mechanism to stop that prefix 
> being accepted by AS 9327 seems almost reasonable. (I assume this is the kind 
> of TE you are talking about.)
>
> What is the principal harm of doing this? Honest question. I'm not advocating 
> for anything, just curious.
>
>
> Joe
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: someone is using my AS number

[Warren Kumari]

On Thu, Jun 13, 2019 at 11:37 AM Jared Mauch  wrote:
>
> You also may not know who allows their own ASN inbound as well. It certainly 
> is a mixed bag.
>
> I do consider poisoning at best horrible hygiene and at worst evidence of 
> malicious intent.

Yes, I fully agree it it bletcherous -- which is why I'm looking for
something less ugly...

>
> Good filtering isn’t just prefix or AS path based it’s both.
>
> Best filtering is pinning the prefix to a specific ASN.
>
> Sent from my iCar
>
> On Jun 13, 2019, at 11:24 AM, Job Snijders  wrote:
>
> On Thu, Jun 13, 2019 at 11:18 Warren Kumari  wrote:
>>
>> On Thu, Jun 13, 2019 at 9:59 AM Joe Abley  wrote:
>> >
>> > Hey Joe,
>> >
>> > On 12 Jun 2019, at 12:37, Joe Provo  wrote:
>> >
>> > > On Wed, Jun 12, 2019 at 04:10:00PM +, David Guo via NANOG wrote:
>> > >> Send abuse complaint to the upstreams
>> > >
>> > > ...and then name & shame publicly. AS-path forgery "for TE" was
>> > > never a good idea. Sharing the affected prefix[es]/path[s] would
>> > > be good.
>> >
>> > I realise lots of people dislike AS_PATH stuffing with other peoples' AS 
>> > numbers and treat it as a form of hijacking.
>> >
>>
>> Actually, I've been meaning to start a thread on this for a while.
>>
>> I have an anycast prefix - at one location I'm a customer of a
>> customer of ISP_X &  ISP_Y & ISP_Z. Because ISP_X prefers customer
>> routes, any time a packet touches ISP_X, it goes to this location,
>> even though it is (severely) suboptimal -- things would be better if
>> ISP_X didn't accept this route in this location.
>>
>> Now, the obvious answer of "well, just ask your provider in this
>> location to not announce it to ISP_X. That's what communities / the
>> telephone were invented for!" doesn't work for various (entirely
>> non-technical) reasons...
>>
>> Other than doing path-poisoning can anyone think of a way to
>> accomplish what I want? (modulo the "just become a direct customer
>> instead of being a customer of a customer" or "disable that site", or
>> "convince the AS upstream of you to deploy communities / filters").
>> While icky, sometimes stuffing other people's AS in the path seems to
>> be the only solution...
>
>
>
> Given the prevalence of peerlock-style filters at the transit-free club, 
> poisoning the path may result in a large outage for your prefix rather than a 
> clever optimization.

Er, let me think about this -- if I have 3 locations, A, B, and C, and
at location A (the problematic one) I announce prefix 192.0.2.0/24
with ISP_X in the path, and at locations B and C I just prepend my AS#
(to keep path lengths roughly the same), even if ISP_X, ISP_Y, ISP_Z
(and others) enable peerlock, AFAICT, it will only be location A which
might get filtered, yes?

> Poisoning paths is bad for all parties involved.

Not disagreeing - I'd love to tag my routes with community
1234:, or 1233:, but without
useful levers, what do I pull? Unlike normally, I'm not arguing just
for the sake of arguing, I'm a lookin' for suggestions...
W


>
> Kind regards,
>
> Job



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Cost effective time servers

[Warren Kumari]

On Thu, Jun 20, 2019 at 11:00 AM Mel Beckman  wrote:
>
> I use the $300 GPS-based TM1000A from TimeMachinesCorp.com. Gets Stratum-1 
> time from GPS satellites and distributes it. Usually I relay this through a 
> handful of local time servers to spread out the load, but it can handle 
> hundreds of queries per minute, so it’s reasonable to use as a primary source 
> even in moderate-sized data centers.
>
> I’ve put in a ton of them, and in most installations I buy two for 
> redundancy. The GPS antenna works from a window in most instances .

I recently fell down the high precision time rabbithole, and now have
3 GPS units (a Truetime, a Symmetricom S250 and a LeoNTP), 3 Cesuim
Primary Reference sources (an FTS4060, and 2 PRS-50s), and an
assortment rubidium units.

One of the "standard" solutions is one of the Microsemi (Symmetricom)
SyncServer's, but these can be expensive -- I've been much happies
with the LeoNTP (
http://www.leobodnar.com/shop/index.php?main_page=product_info_id=272
) -- they are small, they are cheap, and they fast, they are "accurate
enough", and they just work. I've got one on my desk, with a cheap
(car) GPS antenna dangling out the window, and it syncs and runs
happily. A friend of mine has stuffed one in an IP68 box and it's
hanging happily on the side of a TV tower in the elements with no
issues...

I get mine from airspy.us - $349 + antenna.

W


>
>  -mel beckman
>
> > On Jun 20, 2019, at 7:53 AM, David Bass  wrote:
> >
> > What are folks using these days for smaller organizations, that need to 
> > dole out time from an internal source?



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Cost effective time servers

[Warren Kumari]

On Thu, Jun 20, 2019 at 11:42 AM Mel Beckman  wrote:
>
> Warren,
>
> I like the cheap price of the LeoNTP. The only reason I prefer the Tm1000a is 
> that it has an embedded web server, which lets me monitor the satellite 
> constellation visibility. Otherwise, except for oven-controller time clocks, 
> it seems obvious that the $2000+ GPS NTP servers are overpriced overkill :)

Yup, that is a very good point -- the LeoNTP has a small LED interface
and a rotary encoder for configuration and monitoring, but it doesn't
have a web UI.
>From the FAQ:
"Q/ Can I configure it via HTTP/Telnet ?
A/ No. Running a web server on this device although entirely possible
would reduce the performance of the unit. Therefore we took the
decision to just do configuration via the front panel."

This is indeed a really annoying limitation - in the past I've ended
up pointing a webcam at the LCD, but that is (obviously) suboptimal.
I also forgot to mention that it doesn't (yet) do v6, but that might
be added in future firmware versions...

W



>
> -mel via cell
>
> > On Jun 20, 2019, at 8:31 AM, Warren Kumari  wrote:
> >
> >> On Thu, Jun 20, 2019 at 11:00 AM Mel Beckman  wrote:
> >>
> >> I use the $300 GPS-based TM1000A from TimeMachinesCorp.com. Gets Stratum-1 
> >> time from GPS satellites and distributes it. Usually I relay this through 
> >> a handful of local time servers to spread out the load, but it can handle 
> >> hundreds of queries per minute, so it’s reasonable to use as a primary 
> >> source even in moderate-sized data centers.
> >>
> >> I’ve put in a ton of them, and in most installations I buy two for 
> >> redundancy. The GPS antenna works from a window in most instances .
> >
> > I recently fell down the high precision time rabbithole, and now have
> > 3 GPS units (a Truetime, a Symmetricom S250 and a LeoNTP), 3 Cesuim
> > Primary Reference sources (an FTS4060, and 2 PRS-50s), and an
> > assortment rubidium units.
> >
> > One of the "standard" solutions is one of the Microsemi (Symmetricom)
> > SyncServer's, but these can be expensive -- I've been much happies
> > with the LeoNTP (
> > http://www.leobodnar.com/shop/index.php?main_page=product_info_id=272
> > ) -- they are small, they are cheap, and they fast, they are "accurate
> > enough", and they just work. I've got one on my desk, with a cheap
> > (car) GPS antenna dangling out the window, and it syncs and runs
> > happily. A friend of mine has stuffed one in an IP68 box and it's
> > hanging happily on the side of a TV tower in the elements with no
> > issues...
> >
> > I get mine from airspy.us - $349 + antenna.
> >
> > W
> >
> >
> >>
> >> -mel beckman
> >>
> >>> On Jun 20, 2019, at 7:53 AM, David Bass  wrote:
> >>>
> >>> What are folks using these days for smaller organizations, that need to 
> >>> dole out time from an internal source?
> >
> >
> >
> > --
> > I don't think the execution is relevant when it was obviously a bad
> > idea in the first place.
> > This is like putting rabid weasels in your pants, and later expressing
> > regret at having chosen those particular rabid weasels and that pair
> > of pants.
> >   ---maf



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Time and Timing Servers

[Warren Kumari]

On Thu, Jul 11, 2019 at 10:30 AM Mike Hammett  wrote:

> There were a lot of NTP threads several weeks ago, but I didn't get an
> answer to my question amongst all of the other chatter.
>
> I'm looking for a device that can receive GPS inside a building without
> the assistance of an external antenna (Frontier says they no longer allow
> external antenna), will provide traditional NTP services, and will provide
> a timing signal that my Metaswitch can work with.
>
> I know that MicroSemi via Symmetricom makes these kinds of devices, but
> I'm hoping to look at multiple manufacturers and compare.
>

I have a Symmetricom S250 with the Rb option -- it has an active antenna;
while it does *technically* work inside buildings it really needs to be
jammed right up against a window to work. In my (top floor in my house)
office it gets no reception unless against a window...

W



>
>
> Thanks.
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions 
> 
> 
> 
> 
> Midwest Internet Exchange 
> 
> 
> 
> The Brothers WISP 
> 
> 
>


-- 
I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
   ---maf

Re: QoS for Office365

[Warren Kumari]

On Mon, Jul 8, 2019 at 5:50 PM Mark Tinka  wrote:
>
>
>
> On 8/Jul/19 21:03, Robert Webb wrote:
> > I took the OP's request as for doing QoS at the edge of their network
> > and not necessarily the entire path.
>
> Indeed, but even then, you could be handing off the traffic to a
> downstream customer, and can't guarantee what they do to those ToS fields.

I disagree -- you *can* guarantee what someone else will do with your
ToS fields... they will A: ignore them and / or B: scribble all
over them.

At a previous employer (AOL, doing VoIP for customer service / call
centers, ~2004) we had a number of contractual agreements with
multiple providers to honor our QoS markings -- as far as I could tell
(marking test traffic under congestion events) only one of about seven
did anything at all with the marking, and that wasn't enough to make
any difference... I briefly toyed with the idea of asking for some
money back / trying to enforce the terms of the agreements, but
figured that there wasn't much point - expecting QoS to work in
someone else's network based upon your markings seems like a fool's
errand.

W

>
>
> >
> > As another person stated, the real answer is to add more bandwidth if
> > you are having to QoS to Office365 because it is affecting other
> > internet based services.
>
> Yes and no.
>
> More bandwidth never hurt anyone, but packet loss in the remote network
> toward the cloud will hurt you.
>
> Mark.



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: QoS for Office365

[Warren Kumari]

On Mon, Jul 8, 2019 at 2:59 PM Mark Tinka  wrote:
>
>
>
> On 8/Jul/19 20:50, Warren Kumari wrote:
>
> > Depends -- I'd note that the OP said "How can we mark the trafic while
> > keeping the security..." -- some people use the COS / DSCP bits to
> > annotate packets with security information, and use that to make
> > *security decisions* instead of using it to prioritize traffic. Now,
> > I'm not saying that this is why the OP is asking (or that I think it
> > is a good idea, because, well,  I don't think it is!), but it *is* a
> > practice worth knowing about.
>
> Assuming we are discussing such packets traversing the public Internet,
> a little tricky to expect IPP/DSCP values to remain intact in the life
> of an Internet packet.

Goodness no -- I've only ever seen this done within a single network
(including inside some tunnels); expecting this to work across the Big
I-internet is crazypants time. I personally think that the idea itself
is stupid, but, well, their network, their rules, and it "works" for
them.

W

>
> Mark.



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: QoS for Office365

[Warren Kumari]

On Mon, Jul 8, 2019 at 12:31 PM Jared Mauch  wrote:
>
>
>
> > On Jul 2, 2019, at 5:18 PM, Joe Yabuki  wrote:
> >
> > Hi all,
> >
> > How do you deal with QoS for Office365, since the IPs are subject to 
> > changes ?
> >
> > How can we mark the trafic while keeping the security (I fear the marking 
> > based on TCP/UDP Ports since they are not without an additional risk coming 
> > from worms/virus using those ports for example, and doing that directly on 
> > the PCs doesn't seem to be the best solution) ?
>
>
> Add bandwidth?
>
> QoS is a great tool when you’re constrained and must classify your critical 
> traffic, but it’s not a substitute of getting enough capacity to offices.

Depends -- I'd note that the OP said "How can we mark the trafic while
keeping the security..." -- some people use the COS / DSCP bits to
annotate packets with security information, and use that to make
*security decisions* instead of using it to prioritize traffic. Now,
I'm not saying that this is why the OP is asking (or that I think it
is a good idea, because, well,  I don't think it is!), but it *is* a
practice worth knowing about.

One enterprise I've seen does:
firewall {
family inet {
   filter Egress {
term allow {
from {
prefix-list {
TrustedSubnets;
}
dscp af42;
}
then accept;
}
term default {
then {
encapsulate CaptiveGarden;
}
}
}
  }
}

They have some shim thingie on corporate machines which tags
"approved" traffic with AF42 (and also mark on switches from other
devices which should have Internet access), and everyone else gets
bumped to a captive portal / logging / scrubbing firewall thingie.
This is remarkably bletcherous, but (because?) you can do 'iptables -t
mangle -A FORWARD -j dscp --set-dscp-class  AF42' to tag all
packets...

W

>
> I have only applied QoS to voice traffic to ensure it gets through, the rest 
> you need to budget for the bandwidth needs of the site.  The price of 
> bandwidth likely isn’t insane in your market, but your budget may be.. I’ve 
> found that most places won’t quote you a service for less than $1500 USD MRC. 
>  I know you can get the incumbents to often deliver 1G service for $2k/mo in 
> the US (and possibly cheaper).
>
> I’ve found a lot of people are still stuck in TDM mentality instead of just 
> getting a 1G/10G service.
>
> - Jared



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Protecting 1Gb Ethernet From Lightning Strikes

[Warren Kumari]

This probably won't fully solve your problem, but I run a bunch of
Ubiquiti access points and similar -- I suffered a number of lightning
related outages, and then started using their TOUGHcable -
https://www.ui.com/accessories/toughcable/
(don't forget to also get the special jacks / ends). Since changing to
this I've had no more issues. You should also look at
https://www.ui.com/accessories/ethernet-surge-protector/- I haven't
needed them, but...

W



On Tue, Aug 13, 2019 at 2:23 PM Javier J  wrote:
>
> I'm working with a client site that has been hit twice, very close by 
> lightening.
>
> I did lots of electrical work/upgrades/grounding but now I want to focus on 
> protecting Ethernet connections between core switching/other devices that 
> can't be migrated to fiber optic.
>
> I was looking for surge protection devices for Ethernet but have never 
> shopped for anything like this before. Was wondering if anyone has deployed a 
> solution?
> They don't have a large presence on site (I have been moving all of their 
> core stuff to AWS) but they still have core networking / connectivity and PoE 
> cameras / APs around the property.
> Since migrating their onsite servers/infra to the cloud, now their 
> connectivity is even more important.
>
> This is a small site, maybe about 200 switch ports, but I would only need to 
> protect maybe 12 core ones. but would be something I could use in the future 
> with larger deployments.
> it's just a 1Gbe network BTW.
>
> Hope someone with more experience can help make hardware recommendations?
>
> Thanks in advance.
>
> - Javier



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Google DNS Oddity

[Warren Kumari]

Yes, this is no longer occurring / is resolved.

Apologies,
W

On Mon, Sep 9, 2019 at 1:37 PM Florian Brandstetter via NANOG <
nanog@nanog.org> wrote:

> Unable to replicate this in London:
>
> ```
> ; <<>> DiG 9.11.5-P1-1ubuntu2.5-Ubuntu <<>> @ns1.google.com.
> www.google.com. 
> ; (2 servers found)
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61970
> ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
> ;; WARNING: recursion requested but not available
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 512
> ;; QUESTION SECTION:
> ;www.google.com.IN  
> ;; ANSWER SECTION:
> www.google.com. 300 IN  2a00:1450:4009:80d::2004
> ```
>
> going by the latency, ns1.google.com
> 
> travels to NL from our UK PoPs though:
>
> ```
> Host  Loss%   Snt   Last   Avg  Best  Wrst
> StDev
> 1. ???
> 2. ???
> 3. ae26-0.ebr01.lon3.uk.globalone  0.0%132.1   6.2   1.0  45.7
> 12.9
> 4. 2001:7f8:4::3b41:1  0.0%130.7   0.8   0.6   1.7
> 0.4
> 5. 2001:4860:0:1101::100.0%130.7   2.7   0.7  14.2
> 4.2
> 6. 2001:4860::c:4000:cf5b  0.0%131.8   2.1   1.5   4.0
> 0.7
> 7. 2001:4860::8:4000:d325  0.0%138.6   7.3   6.6   9.5
> 0.9
> 8. 2001:4860::22:4001:70b  0.0%136.4   9.5   6.4  36.9
> 8.3
> 9. 2001:4860:0:1::be7 23.1%137.3   7.5   7.3   7.7
> 0.1
> 10. ???
> 11. ???
> 12. ???
> 13. ???
> 14. ???
> 15. ???
> 16. ???
> 17. ???
> 18. ???
> 19. ns1.google.com  0.0%126.4   6.4   6.3   6.5
> 0.0
> ```
> On Sep. 6 2019, at 9:49 pm, Stephen Stuart  wrote:
>
> Do you see the same behavior when you execute your dig query without
> the trailing dot?
>
> Thanks,
> Stephen
>
> On Sep 6, 2019, at 3:11 PM, Chip Marshall via NANOG 
> wrote:
>
> Hello, I'm seeing an oddity when doing DNS lookups for www.google.com
> from our
> London datacenter, and I'm curious if other people are seeing the same
> behavior.
>
> It appears that when we ask for www.google.com. we sometimes get an answer
> that only contains records for www-anycast.google.com., which our resolver
> ignores as they don't match the query.
>
> As seen with dig:
>
> ```
> # dig @ns1.google.com. www.google.com. 
>
> ; <<>> DiG 9.10.3-P4-Ubuntu <<>> @ns1.google.com. www.google.com. 
> ; (2 servers found)
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42641
> ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
> ;; WARNING: recursion requested but not available
>
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 512
> ;; QUESTION SECTION:
> ;www.google.com. IN 
>
> ;; ANSWER SECTION:
> www-anycast.google.com. 300 IN  2001:4860:4802:34::75
> www-anycast.google.com. 300 IN  2001:4860:4802:38::75
> www-anycast.google.com. 300 IN  2001:4860:4802:36::75
> www-anycast.google.com. 300 IN  2001:4860:4802:32::75
>
> ;; Query time: 7 msec
> ;; SERVER: 216.239.32.10#53(216.239.32.10)
> ;; WHEN: Fri Sep 06 19:05:32 UTC 2019
> ;; MSG SIZE rcvd: 167
> ```
>
> So far I've observed this with A and  queries. It's my understanding
> that
> without a CNAME record in the answer, the resolver is doing the right
> thing by
> ignoring the answer, as there's no linkage between www and www-anycast.
>
> Is this broken, or is this just some weird DNS trick I've not come across
> before?
>
>
> You may want to post on dns-operations instead.
>
> Can you do a dig +trace www.google.com instead, that would be more
> instructive about whatт€™s happening at each layer o
>
> f the delegation.
>
>
> - Jared
>
> [image: Sent from Mailspring]



-- 
I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
   ---maf

Re: MX10003 rack size

[Warren Kumari]

On Tue, Aug 6, 2019 at 10:23 AM im  wrote:
>
> Hi,
>
> > Has anyone ever managed to fit a Juniper MX10003 in a 90cm deep rack? 
> > Without applying power tools to either the rack or the router ;)
>
> No.
>
> In my case, MX10003 needs 13cm gap between front-door and fron-panel,
> and needs 10cm gap between back-door and back-panel.
>
> You need at least 120cm depth.
>

... but... but the sales blurb says: "Space and Power Optimized
Provides advanced power-saving features in a small form factor to help
contain OpEx and ensure exceptional efficiency."

>
> you can install 90cm depth rack if without cabling, never power-on ;)

Ah. Yes, if you never power it on, it has awesome OpEx, and
exceptional efficiency... solved!

W

>
>
> thanks,
>
> --
> im 
>
>
> On Tue, 30 Jul 2019 14:32:23 +0200
> Sander Steffann  wrote:
>
> > Hi,
> >
> > Has anyone ever managed to fit a Juniper MX10003 in a 90cm deep rack? 
> > Without applying power tools to either the rack or the router ;)
> >
> > Cheers,
> > Sander
> >
>
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Recommendation: Good paging / alerting software ?

[Warren Kumari]

Hi there,

I'm looking for a recommendation for a good paging / alerting system
*for personal use*.

I'm monitoring a number of servers, VMs, routers / switches and such,
and currently get ~10 pages a week.

Things I've already tried:
I'm currently using OpsGenie, but don't really like the UI.
I briefly tried PagerTree, and have used PagerDuty in the past -- I
was happy with PagerDuty, but don't really want to be paying $10 per
month for this (it's just for personal use, and that seemed a bit
excessive).
I'm also a happy Pushover customer - this works well, but it's ability
to customise / close alerts seems to be missing. It works really well
for other types of notifications though.

Requirements:
1: Cheap!

2 : AlertManager integration - I mainly use Prometheus for monitoring,
and it sends alerts to AlertManager.

3: I'd like an iOS / Android app - having things come in over SMS /
messages makes it too easy to miss things. I also don't want to use
e.g Slack for this because it's too easy to miss them amongst other
messages.

4:  A web interface would be nice, but not 100% necessary.

5: "Alerts" - the ability to Ack / Close alerts. This signals back to
AlertManger.

6: Escalations would be nice - if I don't respond to an alert in N
minutes, send it again, possibly with a more grumpy noise.


Because this is just for personal use I really don't want to be
spending money on this...

Thanks in advance for any suggestions...
W


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: This DNS over HTTP thing

[Warren Kumari]

On Tue, Oct 1, 2019 at 3:42 PM K. Scott Helms  wrote:
>
> They almost have to change the default since there are (comparatively) very 
> few DoH providers compared to DNS providers.

>From the link that Damian sent (emphasis mine):
"More concretely, the experiment in Chrome 78 will **check if the
user’s current DNS provider** is among a list of DoH-compatible
providers, and upgrade to the equivalent DoH service **from the same
provider**. If the DNS provider isn’t in the list, Chrome will
**continue to operate as it does today.**"

W


>
> On Tue, Oct 1, 2019, 2:40 PM Damian Menscher via NANOG  
> wrote:
>>
>> On Tue, Oct 1, 2019 at 12:24 PM Jay R. Ashworth  wrote:
>>>
>>> - Original Message -
>>> > From: "Stephane Bortzmeyer" 
>>> > To: "Jeroen Massar" 
>>>
>>> >> While the 'connection to the recursor' is 'encrypted', the recursor
>>> >> is still in clear text... one just moves who can see what you are
>>> >> doing with this.
>>> >
>>> > As with any cryptographic protocol. Same thing with VPNs, SSH and
>>> > whatever: the remote end can see what you do. What's your point?
>>>
>>> I'm still assimilating this, but based on what I've read this half hour,
>>> his point is that "*it's none of Alphabet's damn business* where I go that
>>> isn't Google".
>>
>>
>> What's missing from this discussion are some basic facts, like "is Google 
>> going to change your DNS settings to 8.8.8.8?"
>>
>> The opening paragraph of 
>> https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html 
>> reads:
>>
>> "This experiment will be done in collaboration with DNS providers who 
>> already support DoH, with the goal of improving our mutual users’ security 
>> and privacy by upgrading them to the DoH version of their current DNS 
>> service. With our approach, the DNS service used will not change, only the 
>> protocol will. As a result, existing content controls of your current DNS 
>> provider, including any existing protections for children, will remain 
>> active."
>>
>> Could someone provide a reference of Google saying they'll change the 
>> default nameserver?  Without that, I think all of Jeroen's arguments fall 
>> apart?
>>
>> Damian



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: QoS for Office365

[Warren Kumari]

On Tue, Jul 9, 2019 at 10:02 AM Tom Beecher  wrote:
>>
>> At a previous employer (AOL, doing VoIP for customer service / call
>> centers, ~2004) we had a number of contractual agreements with
>> multiple providers to honor our QoS markings -- as far as I could tell
>> (marking test traffic under congestion events) only one of about seven
>> did anything at all with the marking, and that wasn't enough to make
>> any difference... I briefly toyed with the idea of asking for some
>> money back / trying to enforce the terms of the agreements, but
>> figured that there wasn't much point - expecting QoS to work in
>> someone else's network based upon your markings seems like a fool's
>> errand.
>
>
> Generally speaking, I agree that making QoS features work consistently on an 
> external network you do not control is a fool's errand.
>
> But if that language was inserted into the contracts, and you can 
> demonstrably prove it's not being done, enforcing contract terms should 
> always be done. Depending on the strength of the remedy, could have been a 
> lot of free service, enough financial incentive for them to MAKE it work 
> correctly, or leverage to open renegotiations for more favorable terms for 
> you.
>
> You know that in reverse they would have done the same to you. :)
>

Yeah, at that point in AOL's trajectory there were (at least from my
point of view!) much much bigger issues -- like, "Who's getting laid
off this week? and how do I remove their access? and who's going to do
whatever they were doing (if anything)?!" and trying to make enhanced
cRTP work over GRE over yet more GRE over IPSEC (because, well,
reasons).
Yes, in an ideal world we would have received some sort of credits --
but then again, in an ideal world, I wouldn't have been trying to run
VOIP over cRTP over GRE over GRE over IPSEC over DS3 to E3 converters
to India...

W


> On Mon, Jul 8, 2019 at 6:38 PM Warren Kumari  wrote:
>>
>> On Mon, Jul 8, 2019 at 5:50 PM Mark Tinka  wrote:
>> >
>> >
>> >
>> > On 8/Jul/19 21:03, Robert Webb wrote:
>> > > I took the OP's request as for doing QoS at the edge of their network
>> > > and not necessarily the entire path.
>> >
>> > Indeed, but even then, you could be handing off the traffic to a
>> > downstream customer, and can't guarantee what they do to those ToS fields.
>>
>> I disagree -- you *can* guarantee what someone else will do with your
>> ToS fields... they will A: ignore them and / or B: scribble all
>> over them.
>>
>> At a previous employer (AOL, doing VoIP for customer service / call
>> centers, ~2004) we had a number of contractual agreements with
>> multiple providers to honor our QoS markings -- as far as I could tell
>> (marking test traffic under congestion events) only one of about seven
>> did anything at all with the marking, and that wasn't enough to make
>> any difference... I briefly toyed with the idea of asking for some
>> money back / trying to enforce the terms of the agreements, but
>> figured that there wasn't much point - expecting QoS to work in
>> someone else's network based upon your markings seems like a fool's
>> errand.
>>
>> W
>>
>> >
>> >
>> > >
>> > > As another person stated, the real answer is to add more bandwidth if
>> > > you are having to QoS to Office365 because it is affecting other
>> > > internet based services.
>> >
>> > Yes and no.
>> >
>> > More bandwidth never hurt anyone, but packet loss in the remote network
>> > toward the cloud will hurt you.
>> >
>> > Mark.
>>
>>
>>
>> --
>> I don't think the execution is relevant when it was obviously a bad
>> idea in the first place.
>> This is like putting rabid weasels in your pants, and later expressing
>> regret at having chosen those particular rabid weasels and that pair
>> of pants.
>>---maf



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: SFP oraganizers / storage recommendations

[Warren Kumari]

If you buy your SFPs from fs.com, they come in a nice organizer -- and
if you buy less than a tray full, you still get a tray.
I keep spares in the trays, labeled on the outside -- I then put the
trays in a cheap toolbox / fishing tackle box, and list what's in each
one in a Google spreadsheet.

Whenever I'm actually at the cage / rack and have a few minutes I
compare the spreadsheet to reality, and update accordingly (SFPs, and
XFPs in particular evaporate over time...)

W

On Wed, Oct 30, 2019 at 9:36 AM Matthew Huff  wrote:
>
> Any recommendations to keep track of different SFP and keep them organized? 
> Any storage boxes / trays designed for SFPs?



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: ECN

[Warren Kumari]

On Thu, Nov 14, 2019 at 12:25 AM Matt Corallo  wrote:
>
> This sounds like a bug on Cloudflare’s end (cause trying to do anycast TCP 
> is... out of spec to say the least), not a bug in ECN/ECMP.

Err. I really don't think that there is any sort of spec that
covers that :-P

Using Anycast for TCP is incredibly common - the DNS root servers for
one obvious example.
More TCP centric well-known examples are Fastly and LinkedIn -
LinkedIn in particular did a really good podcast on their experience
with this.

There is also a good NANOG talk from the ~2000s (?) on people using
TCP anycast for long lived (serving ISO files, which were long-lived
in those days) flows, and how reliable it is - perhaps that's the talk
Todd mentioned?

W

>
> > On Nov 13, 2019, at 11:07, Toke Høiland-Jørgensen via NANOG 
> >  wrote:
> >
> > 
> >>
> >> Hello
> >>
> >> I have a customer that believes my network has a ECN problem. We do
> >> not, we just move packets. But how do I prove it?
> >>
> >> Is there a tool that checks for ECN trouble? Ideally something I could
> >> run on the NLNOG Ring network.
> >>
> >> I believe it likely that it is the destination that has the problem.
> >
> > Hi Baldur
> >
> > I believe I may be that customer :)
> >
> > First of all, thank you for looking into the issue! We've been having
> > great fun over on the ecn-sane mailing list trying to figure out what's
> > going on. I'll summarise below, but see this thread for the discussion
> > and debugging details:
> > https://lists.bufferbloat.net/pipermail/ecn-sane/2019-November/000527.html
> >
> > The short version is that the problem appears to come from a combination
> > of the ECMP routing in your network, and Cloudflare's heavy use of
> > anycast. Specifically, a router in your network appears to be doing ECMP
> > by hashing on the packet header, *including the ECN bits*. This breaks
> > TCP connections with ECN because the TCP SYN (with no ECN bits set) end
> > up taking a different path than the rest of the flow (which is marked as
> > ECT(0)). When the destination is anycasted, this means that the data
> > packets go to a different server than the SYN did. This second server
> > doesn't recognise the connection, and so replies with a TCP RST. To fix
> > this, simply exclude the ECN bits (or the whole TOS byte) from your
> > router's ECMP hash.
> >
> > For a longer exposition, see below. You should be able to verify this
> > from somewhere else in the network, but if there's anything else you
> > want me to test, do let me know. Also, would you mind sharing the router
> > make and model that does this? We're trying to collect real-world
> > examples of network problems caused by ECN and this is definitely an
> > interesting example.
> >
> > -Toke
> >
> >
> >
> > The long version:
> >
> > From my end I can see that I have two paths to Cloudflare; which is
> > taken appears to be based on a hash of the packet header, as can be seen
> > by varying the source port:
> >
> > $ traceroute -q 1 --sport=1 104.24.125.13
> > traceroute to 104.24.125.13 (104.24.125.13), 30 hops max, 60 byte packets
> > 1  _gateway (10.42.3.1)  0.357 ms
> > 2  albertslund-edge1-lo.net.gigabit.dk (185.24.171.254)  4.707 ms
> > 3  customer-185-24-168-46.ip4.gigabit.dk (185.24.168.46)  1.283 ms
> > 4  te0-1-1-5.rcr21.cph01.atlas.cogentco.com (149.6.137.49)  1.667 ms
> > 5  netnod-ix-cph-blue-9000.cloudflare.com (212.237.192.246)  1.406 ms
> > 6  104.24.125.13 (104.24.125.13)  1.322 ms
> >
> > $ traceroute -q 1 --sport=10001 104.24.125.13
> > traceroute to 104.24.125.13 (104.24.125.13), 30 hops max, 60 byte packets
> > 1  _gateway (10.42.3.1)  0.293 ms
> > 2  albertslund-edge1-lo.net.gigabit.dk (185.24.171.254)  3.430 ms
> > 3  customer-185-24-168-38.ip4.gigabit.dk (185.24.168.38)  1.194 ms
> > 4  10ge1-2.core1.cph1.he.net (216.66.83.101)  1.297 ms
> > 5  be2306.ccr42.ham01.atlas.cogentco.com (130.117.3.237)  6.805 ms
> > 6  149.6.142.130 (149.6.142.130)  6.925 ms
> > 7  104.24.125.13 (104.24.125.13)  1.501 ms
> >
> >
> > This is fine in itself. However, the problem stems from the fact that
> > the ECN bits in the IP header are also included in the ECMP hash (-t
> > sets the TOS byte; -t 1 ends up as ECT(0) on the wire and -t 2 is
> > ECT(1)):
> >
> > $ traceroute -q 1 --sport=1 104.24.125.13 -t 1
> > traceroute to 104.24.125.13 (104.24.125.13), 30 hops max, 60 byte packets
> > 1  _gateway (10.42.3.1)  0.336 ms
> > 2  albertslund-edge1-lo.net.gigabit.dk (185.24.171.254)  6.964 ms
> > 3  customer-185-24-168-46.ip4.gigabit.dk (185.24.168.46)  1.056 ms
> > 4  te0-1-1-5.rcr21.cph01.atlas.cogentco.com (149.6.137.49)  1.512 ms
> > 5  netnod-ix-cph-blue-9000.cloudflare.com (212.237.192.246)  1.313 ms
> > 6  104.24.125.13 (104.24.125.13)  1.210 ms
> >
> > $ traceroute -q 1 --sport=1 104.24.125.13 -t 2
> > traceroute to 104.24.125.13 (104.24.125.13), 30 hops max, 60 byte packets
> > 1  _gateway (10.42.3.1)  0.339 ms
> > 2  albertslund-edge1-lo.net.gigabit.dk 

Re: Question about normal ops - BGP Flaps nightly

[Warren Kumari]

On Fri, Nov 22, 2019 at 12:40 PM Christopher Morrow
 wrote:
>
> On Fri, Nov 22, 2019 at 12:32 PM Baldur Norddahl
>  wrote:
> >
> >
> >
> > On Fri, Nov 22, 2019 at 1:21 AM Christopher Morrow 
> >  wrote:
> >>
> >> On Fri, Nov 22, 2019 at 2:01 AM Saku Ytti  wrote:
> >> >
> >> > On Thu, 21 Nov 2019 at 19:44, Baldur Norddahl 
> >> >  wrote:
> >> >
> >> > > A BGP reset can cause routing trouble for as much as 15 minutes. Since 
> >> > > you have two sessions that mitigates the problem somewhat. But 
> >> > > nevertheless this will not be acceptable.
> >> >
> >> > As there are best path algorithms which consider route age, BGP reset
> >> > impact may be indefinite.
> >>
> >> fortunately we have a second actual provider... so this all isn't
> >> super impacting to us, just weird and unexpected on my part.
> >>
> >
> > No that is not helping. When the BGP session flaps your routes via that 
> > provider are withdrawn. Everyone out there that were using those routes 
> > will need to switch. But consider the following:
> >
> > ISP A has routes from both of your providers
> > ISP B has A as uplink
> >
> > BGP works so that ISP A is only announcing the route that he is actually 
> > using to ISP B. ISP B therefore does not have both of your routes. When the 
> > active route is withdrawn ISP B will momentary be without any route to your 
> > network. It can take some time after the withdraw before ISP A announces 
> > that he now is using the alternative route. This gets worse with longer 
> > chains. Also some ISPs are using route flap limiting techniques that can 
> > prolong this process.
> >
> > As I said, my experience is that you can expect as much as 15 minutes of 
> > flaky internet after a BGP reset. This is with multiple transit providers.
>
> Yup, I'm sensitive to flapping causing problems. This was why i
> started the thread, which really should have been:
>   "Is there a well known bug people are working around? or is this a
> new problem I should chase with the provider? or 'nah, everyone does
> this, you just aren't normally paying attention'"
>
> >
> > I can not say too much about why you have BGP resets, but I can say that 
> > you really want it fixed. It will affect your connectivity.
> >
>
> fortunately 3am local time is not prime-internet-use time :) phew!
> (not a great excuse though, of course)
>

The other saving grace / "meh" is that this is for a conference
network, and we are picking up sticks and leaving tomorrow... so, we
will let the provider know that there is something that should be
fixed, but a: our pain will have stopped :-P and b: we won't really
have a good way to know if they have fixed the issue (other than
perhaps watching for a spike of withdraws / reannouncements every 24
hours through this AS path)

W

> I'll be chasing up the provider to see what's up.
> thanks!
> -chris
>
> > Regards,
> >
> > Baldur
> >



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Short-circuited traceroutes on FIOS

[Warren Kumari]

On Fri, Dec 13, 2019 at 4:16 PM Javier J  wrote:
>
> Is anyone from Verizon on this list? They probably are but not allowed to 
> comment. I would love to know if there is an official comment on why they do 
> this.
>
> It annoyed me when they first implemented and I was trying to diagnose an 
> issue with a client.
>
> Regarding your edge device:
> Same here, I had ubiquity gear at my GW for a while and before that PFsense.
>
> When i saw 1ms responses to a ping one day I was confused.

Well, yes, but you *did* see a 1ms ping response.
I'm sure no ISPs would intentionally configure their networks to
artificially improve their latency measurements on certain automated
tools, so I don't know why that would be a useful outcome...

W

>
>
> - J
>
> On Thu, Dec 12, 2019 at 12:51 PM Peter Beckman  wrote:
>>
>> On Wed, 11 Dec 2019, Javier J wrote:
>>
>> > If you have static addressing (biz account) then possibly different from
>> > what I have.
>> >
>> > In North NJ, 3 different accounts I can verify have ICMP blocked as of
>> > sometime earlier this year or late last year so have to use udp to get a
>> > real traceroute.
>> >
>> > Could not be deployed in all areas the same way.
>>
>>   I noticed this about the same time I installed Ubiquiti gear at home,
>>   December 2018.
>>
>>   Until this thread, I thought there was something wrong with my gateway
>>   router config. I could do UDP/TCP traceroutes, but ICMP kept dying.
>>
>>   Glad to know it isn't my gateway, but frustrated as hell that Verizon
>>   decided that a few customers doing less-than-ideal things was enough to
>>   cut a standard network protocol off at the knees.
>>
>> ---
>> Peter Beckman  Internet Guy
>> beck...@angryox.com http://www.angryox.com/
>> ---



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: IPv6 Pain Experiment

[Warren Kumari]

On Fri, Oct 4, 2019 at 5:13 AM Masataka Ohta
 wrote:
>
> Doug Barton wrote:
>
> > And even
> > if you do need to change providers, once you have your addressing plan
> > in place all you have to change is the prefix.
>

This is the same as saying "If you need to change providers in IPv4,
once you have your addressing plan in place all you have to do is
change the prefix", or "To build the Eiffel Tower, all you have to do
is bolt bits of metal together" -- it's technically correct*, but
handwaves away the actual complexity and scale of work.
Yes, you (clearly) can renumber v6 networks, and it's *probably*
easier than renumbering v4, but "just change the prefix" oversells it.

> Your attempt to hype people that renumbering were easy has
> zero probability of success here.
>
> > Except that it's not failing,
>
> It failed from the beginning.

W
*: Yes, the best kind of correct.

>
> Masataka Ohta



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: "Using Cloud Resources to Dramatically Improve Internet Routing"

[Warren Kumari]

On Mon, Oct 7, 2019 at 4:45 PM Stephane Bortzmeyer  wrote:
>
> On Fri, Oct 04, 2019 at 03:52:26PM -0400,
>  Phil Pishioneri  wrote
>  a message of 9 lines which said:
>
> > Using Cloud Resources to Dramatically Improve Internet Routing
> > UMass Amherst researchers to use cloud-based ‘logically centralized
> > control’
>
> Executive summary: it's SDN for BGP. Centralizing Internet routing,
> what could go wrong? (As the authors say, "One reason is there is no
> single entity that has a big picture of what is going on, no
> manager". I wonder who will be Internet's manager.)
>
> Otherwise, an impressive amount of WTF. My favorite: "while
> communication by servers ___on the ground___ might take hundreds of
> milliseconds, in the cloud the same operation may take only one
> millisecond from one machine to another" I thought that universities
> were full of serious people, but university of Massachusets may be an
> exception?



I haven't found the actual work that is being referenced here, and I
*am* quite skeptical based upon the title / premise -- but, I suspect
(well, hope) that this is just another instance of complex technical
material being munged by marketing / reporters into something
unrecognizable -- note that "This article was originally published by
the UMass News Office."

Here is an abstract of one of Yang Song, Arun Venkataramani, Lixin
Gao's earlier papers:
"BGP is known to have many security vulnerabilities due to the very
nature of its underlying assumptions of trust among independently
operated networks. Most prior efforts have focused on attacks that can
be addressed using traditional cryptographic techniques to ensure
authentication or integrity, e.g., BGPSec and related works. Although
augmenting BGP with authentication and integrity mechanisms is
critical, they are, by design, far from sufficient to prevent attacks
based on manipulating the complex BGP protocol itself. In this paper,
we identify two serious attacks on two of the most fundamental goals
of BGP-to ensure reachability and to enable ASes to pick routes
available to them according to their routing policies-even in the
presence of BGPSec-like mechanisms. Our key contributions are to (1)
formalize a series of critical security properties, (2) experimentally
validate using commodity router implementations that BGP fails to
achieve those properties, (3) quantify the extent of these
vulnerabilities in the Internet's AS topology, and (4) propose simple
modifications to provably ensure that those properties are satisfied"

I'm assuming that it this were passed through many company /
university news / marketing orgs it would be translated into:
"The core protocol that makes all of the Internet, all e-commerce,
Internet banking and e-coin torrenting malware protection is
vulnerable to hackers stealing your identity. All existing efforts
have failed, because quantum computers can break cryptography. Our
researchers have identified simple attacks which bypass all Internet
security mechanisms and firewalls, and have demonstrated these
vulnerabilities in the wild. In order to protect Internet banking and
blockchain, and to ensure free elections, they have also developed a
simple and effective new system keep everyone secure. Contact us at
licens...@university.org to learn how to license this critical
technology. Click  to enroll in University, where you too can
learn to fix the Interwebs and earn lots of money."

W
-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Hi-Rise Building Fiber Suggestions

[Warren Kumari]

On Wed, Feb 26, 2020 at 11:20 AM Coy Hile  wrote:
>
> On 2020-02-26 11:14, Randy Bush wrote:
> >> We use plenty of multi-mode, but only in the data centre, between our
> >> own kit, for racks within the same cage.
> >
> > so you have to stock both single and multi?  hmmm
> >
> > randy
>
> I'd expect that from the ToR -> Servers would be MMF, but that other
> infrastructure cabling would be SMF.
> Even using aftermarket optics, putting single-mode transceivers in every
> server and access port would quickly become cost-prohibitive, would it
> not?

Cisco GLC-SX-MM Compatible 1000BASE-SX SFP 850nm 550m DOM Transceiver
Module - $6.00 - https://www.fs.com/products/11774.html
Cisco SFP-GE-L Compatible 1000BASE-LX/LH SFP 1310nm 10km DOM
Transceiver Module - $7.00 - https://www.fs.com/products/12622.html

Yup, it is $1.00 more for SM, and you need 2 per link, but unless you
are doing *lots* that's likely not cost-prohibitive. The delta on 10G
is a bit more ($21 vs $18), but still not crazy-pants territory...

Of course, sometimes you don't have the option of SM - you are
connecting some someone else than they only do MM, or you are
connecting to a piece of kit which doesn't have replaceable optics, or
you have legacy cabling which is MM, or... but, the cost of the optics
these days is not really the limiting factor.

>
> --
> Coy Hile
> coy.h...@coyhile.com



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: akamai yesterday - what in the world was that

[Warren Kumari]

On Fri, Jan 24, 2020 at 9:55 AM Aaron Gould  wrote:
>
> Thanks Jared, When I reminisce with my boss he reminds me that this telco/ISP 
> here initially started with a 56kbps internet uplink , lol

Oh, gods, what have you done?! This comment will bring everyone out of
the woodwork, reminiscing about the good ol' days

So, I grew up in South Africa, and one of the more fascinating /
cooler things I saw was a modem which would get you ~50bps (bps, not
Kbps) over a single strand of barbed wire -- you'd hammer a largish
nail into the ground, and clip one alligator[0] clip onto that, and
another alligator clip onto the barbed wire. Repeat the process on the
other side (up to ~5km away), plug the modems in, and bits would
flow... I only saw these used a few times, but always thought they
were cool

One of the first ISPs I worked at had an AGS+ as the "core" router
(and a pile of IGS's) -- the AGS had metal plate in the front to
access the "line cards", and a monster big squirrel fan - the squirrel
fan was strong enough that the vacuum / airflow would keep the metal
plate sucked on if you didn't do up the screw.. anyway, the device
also had a watchdog timer, which would reboot the box if IOS[1] locked
up, and the fan would slow down while this occurred -- so, for a long
time, our fastest / most reliable monitoring was an empty PC case
placed on the floor under the rack -- when the router locked up, the
fan would slow down, the front would fall off[2] and bounce on the PC
case, making an unholy racket - and alerting the NOC that something
bad was happening

Anyway, so I tied an onion to my belt, which was the style at the
time. Now, to take the ferry cost a nickel, and in those days, nickels
had pictures of bumblebees on 'em. Give me five bees for a quarter,
you'd say.
Now where were we? Oh yeah: the important thing was I had an onion on
my belt, which was the style at the time. They didn't have white
onions because of the war. The only thing you could get was those big
yellow ones...

W
[0]: In .za we called them crocodile clips -- true story.
[1]: For all you young whippersnappers, that's the Cisco IOS, not the
Apple iOS.. :-/
[2]: https://www.youtube.com/watch?v=3m5qxZm_JqM -- Unlike the rest of
this email, this is off-topic, but still great

>
> -Aaron
>
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: akamai yesterday - what in the world was that

[Warren Kumari]

On Thu, Jan 23, 2020 at 12:45 PM Hugo Slabbert  wrote:
>
> > This just follows the same rules as networks have always seemed to; If you 
> > build it, they will come, and you'll have to build more. :)
>
> https://en.m.wikipedia.org/wiki/Induced_demand
>

Yup, there is also (in networking at least) suppressed demand -- I'm
sure we've all seen capacity planning discussions along the lines of:
"My 1GE is running at 95% capacity - I'm replacing it with a 10GE and
it will be around 10% used... wait?! What?! It's now at 7Gbps?! How
the hell did that happen?!" scenarios. They usedto be funny, but these
days I just find it depressing...

W

> :-)
>
>
> On Thu., Jan. 23, 2020, 09:40 Tom Beecher  wrote:
>>>
>>> I think this is a tribute to how we’ve built and upgraded networks for 
>>> capacity and speed.
>>
>>
>> I think it's spot on.
>>
>> In years past it made more sense to distribute smaller , incremental 
>> patches. More work on the software side, but it was likely a better option 
>> than getting blasted on Twitter because "OMG I WANT TO PLAY AND MY DOWNLOAD 
>> IS TAKING 8 HOURS".
>>
>> This just follows the same rules as networks have always seemed to; If you 
>> build it, they will come, and you'll have to build more. :)
>>
>> On Thu, Jan 23, 2020 at 11:57 AM Jared Mauch  wrote:
>>>
>>>
>>>
>>> > On Jan 23, 2020, at 11:52 AM, Valdis Klētnieks  
>>> > wrote:
>>> >
>>> > On Thu, 23 Jan 2020 17:13:15 +0100, Bryan Holloway said:
>>> >
>>> >> Game releases are hardly a new thing, but these last two events seem to
>>> >> be almost an order of magnitude higher than what we're used to (at least
>>> >> on our predominantly eyeball network.)
>>> >>
>>> >> Any thoughts from the community? We're taking steps to accommodate, but
>>> >> from a capacity-planning perspective, this seems non-linear to me.
>>> >
>>> > Be prepared for an entire new world of hurt this holiday season. Sony has 
>>> > already
>>> > confirmed that PS5 releases will ship on 100Gbyte blu-ray disks.  Which 
>>> > means that
>>> > download sizes will be comparable…
>>>
>>> There’s also the “we will stream you all the data things” I keep hearing 
>>> about like the
>>> Consoles without discs or some other thing I can’t remember the name of.
>>>
>>> I think this is a tribute to how we’ve built and upgraded networks for 
>>> capacity and speed.
>>>
>>> - Jared
>>>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Flow based architecture in data centers(more specifically Telco Clouds)

[Warren Kumari]

On Sun, Feb 9, 2020 at 4:15 PM Christopher Morrow
 wrote:
>
>
>
> On Sun, Feb 9, 2020 at 1:06 PM Rod Beck  
> wrote:
>>
>> They don't have to be related.
>>
>
> makes a cogent conversation harder :)

Srsly?! Any conversation including Cogent is harder

W
(Sorry, couldn't resist. I tried, but failed...)


>
>>
>> I am curious about the distinction about the flow versus non-flow 
>> architecture for data centers and I am also fascinated by the separate issue 
>> of WAN architecture for these clouds.
>>
>
> WAN is probably: "least expensive option form A to B" plus some effort to 
> standardize across your deployment. Right?
>
> Akamai is probably a good example, from what I can tell they were 
> 'transit/peering only' until they realized their product was sending 'more 
> bits' between deployments than to customers (in some cases). So, pushing the 
> 'between our deployments' bits over dedicated links (be that dark, waves, 
> other L3 transport) made sense budget-wise.
>
> (again.. just a chemical engineer and not a peering engineer, but...)
>
>>
>> Regards,
>>
>> Roderick.
>>
>> 
>> From: Christopher Morrow 
>> Sent: Sunday, February 9, 2020 9:24 PM
>> To: Rod Beck 
>> Cc: Glen Kent ; nanog@nanog.org 
>> Subject: Re: Flow based architecture in data centers(more specifically Telco 
>> Clouds)
>>
>> (caution, I'm just a chemical engineer, but)
>>
>> You appear to ask one question: "What is the difference between flow
>> and non-flow architectures?"
>> then sideline in some discussion about fiber/waves vs
>> layer-3/transit/peering/x-connect
>>
>> I don't think the second part really relates to the first part of your 
>> message.
>> (I didn't put this content in-line because .. it's mostly trying to
>> clarify what you are asking Rod"
>>
>> On Sun, Feb 9, 2020 at 3:19 AM Rod Beck  
>> wrote:
>> >
>> > Please explain for us dumb sales guys the distinction between flow and 
>> > non-flow. My question is the fundamental architecture of these clouds. We 
>> > all know that Amazon is buying dark fiber and building a network based on 
>> > lighting 100 and 10 gig waves on IRU and titled fiber. Same for Microsoft 
>> > (I sold them in a past life some waves) and other large players.
>> >
>> > But there appear to be quite a few cloud players that rely heavily on 
>> > Layer 3 purchased from Level3 (CenturyLink) and other members of the 
>> > august Tier 1 club. And many CDN players are really transit + real estate 
>> > operations as was Akamai until recently.
>> >
>> > It seems the threshold for moving from purchased transit plus peering to a 
>> > Layer 1 and 2 network has risen over time. Many former Tier 2 ISPs pretty 
>> > much gutted their private line networks as transit prices continued 
>> > inexorable declines.
>> >
>> > Best,
>> >
>> > Roderick.
>> >
>> > 
>> > From: NANOG  on behalf of Glen Kent 
>> > 
>> > Sent: Sunday, February 9, 2020 11:02 AM
>> > To: nanog@nanog.org 
>> > Subject: Flow based architecture in data centers(more specifically Telco 
>> > Clouds)
>> >
>> > Hi,
>> >
>> > Are most of the Telco Cloud deployments envisioned to be modeled on a flow 
>> > based or a non flow based architecture? I am presuming that for deeper 
>> > insights into the traffic one would need a flow based architecture, but 
>> > that can have scale issues (# of flows, flow setup rates, etc) and was 
>> > hence checking.
>> >
>> > Thanks, Glen



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: help with diagnosing traffic blackhole to / from select akamai ranges?

[Warren Kumari]

On Thu, Jan 9, 2020 at 1:56 PM William McLendon  wrote:
>
> Good afternoon,
>
> we have a downstream customer originating a more specific /24 prefix, and 
> when they do so, traffic sourced from that /24 prefix to at least a subset of 
> akamai ranges (at minimum the 184.27.24.0/22 block at this time) are getting 
> blackholed somewhere along the path either to or from, but i’m not sure how 
> best to go about troubleshooting or getting assistance with diagnosing where 
> the problem may be.
>
> from a device in the offending IP block I cannot ping or curl to 
> www.akamai.com that resolves to 184.27.25.72, however from an IP outside the 
> /24 specific prefix but still in their supernet range I am able to ping and 
> get an HTTP 301 redirect response.  Some other akamai prefixes like 
> 23.73.0.0/20 seem to work without issue from what I can tell thus far.  If 
> the /24 prefix is removed, all works as expected via their covering 
> announcement (which does return via their primary provider, as we are 
> generally a backup path for their larger block).

The fact that when the more specific is announced through you, things
change *probably* implies that the route is being accepted (and it
isn't IRR filters, RPKI, etc). This sounds like it might be IP ACLs or
similar, but without much more detail (like the prefix, and your AS
number, etc) this is largely just shooting in the dark

W

>
> Any guidance the community can share as to how to go about trying to resolve 
> I would greatly appreciate — this is the first time i’ve had to trace down a 
> [seemingly random] reachability issue like this.  Connectivity to other 
> services seem ok from what I can gather so far, even to some other akamai 
> ranges.  from looking glass perspective it looks like the route is being 
> accepted properly by our upstreams and other large providers like NTT, etc.  
> I did send an email to n...@akamai.com but not sure that is the appropriate 
> way to reach out for assistance or not, since we nor our downstream customer 
> are direct customers or peers of theirs.
>
> Thanks,
>
> Will McLendon
> wimcl...@gmail.com
>
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: breakout

[Warren Kumari]

On Wed, Jan 8, 2020 at 2:20 PM Luke Guillory  wrote:
>
> You'd need something like this, which you can jumper over to the 10G port.
>
>
> https://www.fs.com/products/37016.html
>
> Cable to break it out.
>
> https://www.fs.com/products/68048.html
>
>

I believe that these (and the AOC option) require that the switch
understand / supports splitting the 40G interface into 4x10s -- now,
presumably Randy can figure that out for the Arcus, but it's
disappointing how many devices *dont* support this.

Actually, while we are discussing this, does anyone know of a current
NIC which has 4 or more 10G ports? The Mellanox switches can
channelize their ports (and they sell a cable for this), but the NICs
don't support it...

W

>
> Luke
>
>
>
> Ns
>
>
>
>
>
>
> -Original Message-
> From: NANOG  On Behalf Of Randy Bush
> Sent: Wednesday, January 08, 2020 1:10 PM
> To: North American Network Operators' Group 
> Subject: breakout
>
> *External Email: Use Caution*
>
> i am not a fiber/sfp/... geek, so clue bat please
>
> on my left, i have a delta 9020SL running arcos, female 40g qsfp
>
> on my right, i have incoming 10g 1310nm single mode from the seattle internet 
> exchange.  it is currently into a redstone 10g sfp
>
> NAMEVALUE
> -
> SwPort  1
> Status  PRESENT
> Valid   True
> Vendor  FiberStore
> Model   SFP-10GLR-31
> Serial-Number   G1804021292
> TypeSFP
> Module-Type 10G_BASE_SX
> Media-Type  FIBER
> Module-Capability   F_10G
> Length  255
> Length-Description
>
> which i am swapping out for the delta 9020
>
> so i am look at something such as https://www.fs.com/products/30900.html
> except i do not understand active/passive, AOC1M, etc
>
> thanks in advance
>
> randy



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Cost Recovery Surcharge & Va Personal Property Tax Recovery for IP Transit

[Warren Kumari]

On Mon, Jan 6, 2020 at 10:54 AM Christopher Morrow
 wrote:
>
> On Mon, Jan 6, 2020 at 10:30 AM William Herrin  wrote:
>
> > If it's not written in to your contract, it's a breach of contract. Either 
> > way it's a deceitfully imposed surcharge, not a state tax. Virginia does 
> > not tax the sale of services like transit and colo. More, the only personal 
> > property tax I've heard of in Virginia is on motor vehicles.
> >
>
> also, houses and ( I think ) boats. (personal property tax)
>

and mobile homes, and aircraft... oh, and, surprisingly, Flight
Simulators (a rate of $0.01 per $100 of assessed value). I guess that
this means that if I buy a joystick from amazon for $19.99 I owe the
country 0.002c...


> I could imagine this is: "Hey, have our customers pay our
> realty/property taxes for us!" plan... or that perhaps they are
> 'leasing you ground space" and passing on the %-age of their total
> space's tax footprint to you.
>
> not saying either of those sounds terrific though :)


Yup - this sounds like the "We will charge you a modem rental fee,
even if you don't, you know, actually rent a modem..." (like
https://arstechnica.com/information-technology/2019/07/frontier-customer-bought-his-own-router-but-has-to-pay-10-rental-fee-anyway/)

Warren "Waitin' for the servicefinder.se spam" Kumari.



--
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Requesting /24 from ARIN

[Warren Kumari]

On Sat, Dec 28, 2019 at 8:34 AM Terrance Devor  wrote:

> Hello Everyone,
>
> I know we are very late in the game however, I need the community's help.
> As our company continues to grow and establish long term relationships and
> bringing additional customers onto our infrastructure, we find ourselves
> desperately needing to reserve a /24.
>
> I understand that IPv4 addresses are getting depleted as of 2015, can
> someone on here please guide us on how to best secure /24?
>

Buying this on the auction market is probably the best (only realistic?)
way — e.g:
https://auctions.ipv4.global/

W


> Thank You in Advance,
>
> Terrance
>
-- 
I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
   ---maf

Re: [EXT] Shining a light on ambulance chasers - Noction

[Warren Kumari]

On Thu, Mar 26, 2020 at 9:50 AM Tom Beecher  wrote:
>
> Their device by itself did not leak anything, no. But it was the thing that 
> created the more specifics that were then leaked due to other errors.

"In order to further reduce the likelihood of these problems occurring
in the future, we will be adding a feature within Noction IRP to give
an option to tag all the more specific prefixes that it generates with
the BGP NO_EXPORT community. ***This will not be enabled by
default***, due to potential drawbacks; such as customers who use
multiple ASes or customers who have eBGP sessions with private ASes,
but it will be an option if a customer wants to use it. This way, even
if filters fail, more specific prefixes won’t be propagated to
external autonomous systems." -
https://www.noction.com/blog/route-optimizers (emphasis mine).

So, yes, "other errors" - but not tagging these by default with
NO_EXPORT is like shipping hand grenades with the pins removed. Yes,
some people might not know how to remove them - but these people
really shouldn't be touching them to begin with

W


>
>
> On Wed, Mar 25, 2020 at 7:50 PM Michel Py  wrote:
>>
>> > In recent months, I've been trying to bring your attention to BGP 
>> > optimization.
>>
>> Is that not the thing that leaked a massive amount of prefixes some time ago 
>> ?
>>
>> Michel.
>>
>> TSI Disclaimer:  This message and any files or text attached to it are 
>> intended only for the recipients named above and contain information that 
>> may be confidential or privileged. If you are not the intended recipient, 
>> you must not forward, copy, use or otherwise disclose this communication or 
>> the information contained herein. In the event you have received this 
>> message in error, please notify the sender immediately by replying to this 
>> message, and then delete all copies of it from your system. Thank you!...



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: LTE modem where I can control the MTU

[Warren Kumari]

On Fri, May 1, 2020 at 10:10 AM Dovid Bender  wrote:
>
> Hi,
>
> We have VZ wireless in the data center as a backup to our core 
> infrastructure. We have an issue where if packets have a large MTU they seem 
> to die. Does anyone know of a good 4G modem where I can set the MTU on the 
> cellular connection?

This (likely) isn't a function of the model, but rather whatever the
modem connects to -- set the MTU of the interface that the modem
connects to to be lower...

W

>
> TIA.
>
> Dovid
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: "Is BGP safe yet?" test

[Warren Kumari]

On Wed, Apr 22, 2020 at 11:45 AM Danny McPherson  wrote:
>
> On 2020-04-21 12:36, Rubens Kuhl wrote:
> > On Tue, Apr 21, 2020 at 1:10 PM Matt Corallo via NANOG
> >  wrote:
> >
> >> That’s an interesting idea. I’m not sure that LACNIC would want
> >> to issue a ROA for RIPE IP space after RIPE issues an AS0 ROA,
> >> though. And you’d at least need some kind of time delay to give
> >> other RIRs and operators and chance to discuss the matter before
> >> allowing RIPE to issue the AS0 ROA, eg in my example mitigation
> >> strategy.
> >
> > All 5 RIRs can issue ROAs for all the IP address spaces. They don't as
> > a matter of coordinated operations, but that doesn't prevent court
> > orders determining that to be done.
>
>
> Or a miscreant.  [insert-least-favorite-rir] is now part of your attack
> surface.

Or a slip of the keyboard / software ooops / mistake -- but, in spite
of this, I think that RPKI / ROAs / ROV is a good thing; as with
everything, this is an engineering trade off, and to me this feels
well worth it...

I do think that CloudFlare does some great things for the Internet -
they've moved DNSSEC forward immensely, significantly increased the
adoption of HTTPS/TLS, the OctoRPKI/GoRTR stuff is nice and easy,
their hosted RPKI cache, etc -- but their marketing pushes like this
feel overly aggressive.

W

>
>
> -danny



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Route aggregation w/o AS-Sets

[Warren Kumari]

On Tue, Apr 14, 2020 at 1:04 AM Alejandro Acosta
 wrote:
>
> Hello Lars,
>
>  As a comment there is a draft that proposes to deprecate AS_SET
> https://datatracker.ietf.org/doc/draft-ietf-idr-deprecate-as-set-confed-set/?include_text=1

Ta, thanks.
This completes the work started by RFC6472 - "Recommendation for Not
Using AS_SET and AS_CONFED_SET in BGP".

W



>
>
> Alejandro,
>
>
> On 4/11/20 7:09 AM, Lars Prehn wrote:
> > Hi everyone,
> >
> > how exactly do you aggregate routes? When do you add the AS_SET
> > attribute, when do you omit it? How does the latter interplay with RPKI?
> >
> > Best regards,
> >
> > Lars
> >
> >



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: Internet operations during pandemics

[Warren Kumari]

Many years ago (1990s) I worked for a startup in NYC. We had a
conference room called "Conference Room S", this was a semi-reserved
loft are in the Starbucks across the street, with an open tab[0].
The leads for each group had the project to design the DR / BCP plans
for the entire organization, and so we had a daily, 1 hour meeting in
Conf Room S.

Instead of actually working on the DR plan, we used the time to get
other work done - it was quiet, there was wifi, there were no
interruptions, there was free coffee...

After a few months the CTO asked us to finish up and give him the
plans... so, we wrote:
Step 1: Panic !!!
Step 2: Sell stock options (if any...)[1]
Step 3: Post resume on Monster.com
Step 4: ...

We printed up a bunch of copies of this, put it in an envelope,
labeled it as "DR plan - open in case of disaster" and gave it to the
CTO -- we fully expected him to open it, shout at us for a bit and /
or chuckle resignedly, and then demand we actually do something useful
- but, instead, something much much worse occurred... he thanked us,
and locked it, unopened, in his filing cabinet. As blinked and asked
him if he was going to read it, and he said "No, I trust you to have
done a good job..."

We felt *really* bad, and worked late over the next few weeks and
weekends to actually make a good BCP/DR plan, and then confessed our
sins. We also ran table-tops, distributed and tested the plans, etc..

I've always wondered whether the CTO somehow knew what we'd been up to
- because of our guilt, the quality / comprehensiveness of the plan
ended up much better than it would have otherwise...


W
[0]: We thought that we were super cool for this...
[1]: This was an ongoing joke - the company was always "almost ready"
to go public...

On Thu, Mar 19, 2020 at 8:35 AM Andrew Latham  wrote:
>
> In my past it has always benefited me to set use cases and plan accordingly. 
> For many it is difficult to imagine these less than awesome use cases. Having 
> working to get datacentres back online in 8.8 earthquakes and dealing with 
> fires in co-location sites it is hard.
>
> 1. Document
> 2. Generate use cases, DR plans, OOBM, document peers phone numbers offline
> 3. Implement, share, discuss
> 4. Profit
>
> While this sounds ideal and simple it is not a small effort. I have two talks 
> I must finish up where on is on *Organizations as Code* and how to survive 
> the worst.
>
>
>
> On Wed, Mar 18, 2020 at 5:25 PM Christopher Morrow  
> wrote:
>>
>> Did other folk on nanog-l see the nLnog-l note copied here?
>> I wonder how folk are planning for things (noted in the slides)
>>   o  supply chain for parts/equipment
>>  Wait, I can't get me a new shiny shipped because what??
>>
>>   o ongoing rollout of new equipment
>>  I'm deploying next week in KIX, I'm currently in LAX how do I get
>> there? equipment arrives.. in between...oops!
>>
>>   o noc/etc support staff
>> omg.. wait, I can't have my noc staff in the same room? our 'wfh'
>> solution is ... wait, where is that?
>> how do i get their phone queue sent to them? omg :( 
>>
>>   o services capacity crunches
>> I love my shiny new dns service.. .wait, why is there a smoking
>> hole where my dns servers were?
>>
>> I think some of this has been discussed (shifts in peaks, leveling of peaks)
>> Some hasn't really...  I expect that at least sharing some 'err, our
>> WFH changed now we do: X, Y , Z and use M to get N solved'
>> could be super cool to discuss/share and iterate for better solutions
>> for all of our users.
>>
>> thoughts? :)
>>
>> thanks!
>> -chris
>> (note all the hard work in this message is not mine... thanks Job!)
>>
>> -- Forwarded message -
>> From: Job Snijders 
>> Date: Wed, Mar 18, 2020 at 6:02 PM
>> Subject: Internet operations during pandemics
>> To: 
>>
>>
>> Dear all,
>>
>> I threw together a slidedeck today on the potential impact and second
>> order effects of COVID-19 on Internet network operations.
>>
>> http://instituut.net/~job/netops_during_pandemics.pdf
>>
>> I hope we together over time can add and extend projections in the deck
>> on what will happen and how we can mitigate the negative effects on
>> Internet operations.
>>
>> We have to answer questions such as:
>>
>> 1) what problems already exist today because of a few weeks of C19?
>> 2) What problems are still coming? Will those be localized or globally?
>> 3) What possible workarounds can we plan for those problems?
>>
>> I would appreciate feedback, comments, corrections or whatever you want
>> to tell me. None of us have been in this situation before, so my guess
>> is as good as yours.
>>
>> Kind regards,
>>
>> Job
>
>
>
> --
> - Andrew "lathama" Latham -



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: DHS letters for fuel and facility access

[Warren Kumari]

On Tue, Mar 17, 2020 at 1:21 PM Hiers, David  wrote:
>
> Good reminder to test, test, test...

Indeed -- and we had tested, multiple times. Unfortunately, the only
realistic way we would have found this would have been to kill power
to the building and run on generators for many hours, and then,
likely, we would only have discovered it when the gensets ran out of
power and fell over. IIRC, there is (or was) some noise and pollution
regulations in NYC where you could only run generators for short
periods of time (30min?) unless it was an actual emergency. I also
seem to remember something about having to test at night, probably
also for noise...

But, yes, regular testing is clearly a good practice - but so is
having a good BCP/DR plan (which you also test :-)
W


>
>
> -Original Message-
> From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Warren Kumari
> Sent: Tuesday, March 17, 2020 10:08 AM
> To: Paul Nash 
> Cc: Untitled 3 
> Subject: Re: DHS letters for fuel and facility access
>
> On Tue, Mar 17, 2020 at 12:44 PM Paul Nash  wrote:
> >
> > September 2001.  Just after the 9/11 attacks, all of lower Manhattan was 
> > shut down.  Out link (IIRC) was to a satellite farm on Staten island, 
> > across the bay to 60 Hudson.  Power went off, diesels kicked in, fuel 
> > trucks was not allowed in, and a few days later we lost all international 
> > connectivity.
>
> We had some interesting failures during 9/11 as well -- for some reason, the 
> UPS didn't kick in, so everything went down - and then came back a few 
> minutes later as the generators came online -- and then went down again ~2 
> hours later -- turns out that the genset air filters got clogged with dust, 
> and suffocated the diesel.
> This was "fixed" a few days later by brushing them off with brooms and 
> paintbrushes -- by this point they had completely discharged the 24V starter 
> batteries, and so someone (not me!) had to lug a pair of car batteries and 
> jumper cables. They restarted, and ran for a while, and then stopped again.
>
> It turns out that getting a permit to store lots of diesel on the roof is 
> hard (fair enough), and so there was only a small holding tank on the roof, 
> and the primary tanks were in the basement -- and the transfer pump from the 
> basement to roof storage was not, as we had been told, on generator power
>
> We had specified that the transfer pump be on the generator feed, there was a 
> schematic showing at is being on the generator feed, there was even a breaker 
> with a cable marked  "Transfer Pump (HP4,5)" --- but it turned out to just be 
> a ~3ft piece of cable stuffed into a conduit, and not actually, you know, 
> running all the way down to the basement and connected to the transfer pump.
>
> W
>
>
>
> >
> > Lots of important people lost power as well, so the feds decided to let the 
> > diesel tankers in after a few days’ deliberations.
> >
> > paul
> >
> > > On Mar 17, 2020, at 11:21 AM, Mark Tinka  wrote:
> > >
> > >
> > >
> > > On 17/Mar/20 17:15, Paul Nash wrote:
> > >
> > >> That same fuel shortage killed all Internet traffic to sub-Saharan 
> > >> Africa.  Took us a while to figure out what was wrong with the satellite 
> > >> link to the US.
> > >
> > > What year was that :-)?
> > >
> > > Mark.
> >
>
>
> --
> I don't think the execution is relevant when it was obviously a bad idea in 
> the first place.
> This is like putting rabid weasels in your pants, and later expressing regret 
> at having chosen those particular rabid weasels and that pair of pants.
>---maf
>
> --
> This message and any attachments are intended only for the use of the 
> addressee and may contain information that is privileged and confidential. If 
> the reader of the message is not the intended recipient or an authorized 
> representative of the intended recipient, you are hereby notified that any 
> dissemination of this communication is strictly prohibited. If you have 
> received this communication in error, notify the sender immediately by return 
> email and delete the message and any attachments from your system.



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: DHS letters for fuel and facility access

[Warren Kumari]

On Tue, Mar 17, 2020 at 12:44 PM Paul Nash  wrote:
>
> September 2001.  Just after the 9/11 attacks, all of lower Manhattan was shut 
> down.  Out link (IIRC) was to a satellite farm on Staten island, across the 
> bay to 60 Hudson.  Power went off, diesels kicked in, fuel trucks was not 
> allowed in, and a few days later we lost all international connectivity.

We had some interesting failures during 9/11 as well -- for some
reason, the UPS didn't kick in, so everything went down - and then
came back a few minutes later as the generators came online -- and
then went down again ~2 hours later -- turns out that the genset air
filters got clogged with dust, and suffocated the diesel.
This was "fixed" a few days later by brushing them off with brooms and
paintbrushes -- by this point they had completely discharged the 24V
starter batteries, and so someone (not me!) had to lug a pair of car
batteries and jumper cables. They restarted, and ran for a while, and
then stopped again.

It turns out that getting a permit to store lots of diesel on the roof
is hard (fair enough), and so there was only a small holding tank on
the roof, and the primary tanks were in the basement -- and the
transfer pump from the basement to roof storage was not, as we had
been told, on generator power

We had specified that the transfer pump be on the generator feed,
there was a schematic showing at is being on the generator feed, there
was even a breaker with a cable marked  "Transfer Pump (HP4,5)" ---
but it turned out to just be a ~3ft piece of cable stuffed into a
conduit, and not actually, you know, running all the way down to the
basement and connected to the transfer pump.

W



>
> Lots of important people lost power as well, so the feds decided to let the 
> diesel tankers in after a few days’ deliberations.
>
> paul
>
> > On Mar 17, 2020, at 11:21 AM, Mark Tinka  wrote:
> >
> >
> >
> > On 17/Mar/20 17:15, Paul Nash wrote:
> >
> >> That same fuel shortage killed all Internet traffic to sub-Saharan Africa. 
> >>  Took us a while to figure out what was wrong with the satellite link to 
> >> the US.
> >
> > What year was that :-)?
> >
> > Mark.
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

Re: South Africa On Lockdown - Coronavirus - Update!

[Warren Kumari]

On Mon, Mar 23, 2020 at 8:03 PM Owen DeLong  wrote:
>
>
>
> > On Mar 23, 2020, at 16:50 , Warren Kumari  wrote:
> >
> > On Mon, Mar 23, 2020 at 6:53 PM Sabri Berisha  wrote:
> >>
> >> Hi,
> >>
> >> In my experience, yubikeys are not very secure. I know of someone in my 
> >> team who would generate a few hundred tokens during a meeting and save the 
> >> output in a text file. Then they'd have a small python script which was 
> >> triggered by a hotkey on my macbook to push "keyboard" input. They did 
> >> this because the org they were working for would make you use yubikey auth 
> >> for pretty much everything, including updating a simple internal Jira 
> >> ticket.
> >
> > By that argument, SecureID (and other LCD tokens) are also really
> > insecure. When I worked at AOL we had to use them for almost
> > everything - a bunch of people got together and put their secureIDs in
> > a grid under a webcam. That way they didn't need  to carry them with
> > them - when they needed a token they would open the webcam page, and
> > know that theirs was third down, and fourth across….
>
> Not actually, no…
>
> SecurID and the others of its ilk have a safety feature in that the number 
> doesn’t change that often.
>
> It turns out to be awkward and time-consuming to do what is being done with 
> the UBIKEY.

Not if you run it in TOTP mode. Yubikeys support many options - if you
choose to use a weak solution, well that's your choice...
I guess you could ask them nicely to make a version without the
features you don't want to use - or you could just not *use* the
features you don't want to use


>
> I agree that this abuse of the UBI Key is more an issue of implementation 
> than the inherent nature of the
> UBIKEY, but the UBIKEY does allow this kind of abuse in ways that other 
> tokens don’t facilitate.

That's like saying that cars are worse than bicycles, because cars
allow you drive into things are a more dangerous speed. I mean, yes,
but 

W
>
> Owen
>
>


-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf