Re: COVID-19 vs. our Networks
On 03/18/20 09:29 -0500, Blake Hudson wrote: On 3/17/2020 1:54 PM, Dan White wrote: On 03/17/20 14:38 -0400, Rich Kulawiec wrote: On Tue, Mar 17, 2020 at 08:38:28AM -0700, Mike Bolitho wrote: That's the good news. Here's the bad news: in about 2-3 weeks, when our health care systems are stretched to the breaking point, there will be a window of opportunity for adversaries to maximize the damage. On a slightly tangential topic, we had a dictionary attack against customer voice accounts over night, presumably to implement toll fraud. We were in the middle of working out work-from-home plans and were quite distracted with other things. We managed to get on top of it quickly once someone noticed. Attackers taking advantage of this situation is a serious concern. Dan, we're aware of another telco that ran into a similar fraud situation last week. They stood up some more restrictive ACLs to combat the fraud, but broke VoIP RTP in the process. 'Hit em while they're occupied' type of attacks I guess should be expected right now. As my grandmother would say: an ounce of prevention is worth a pound of cure. Hey Blake, I appreciate that. We've got two tendencies going on here at the moment: 1) Man the ship of operations. Stay alert and fix the problems that arise. Be totally reactive, and be the "hero". 2) Increase visibility and focus on network design. Move planned upgrades up a few weeks/months. Be proactive. Option 2 is the better long term option, but the risk is that any change, while short staffed is going to run the risk of unintended consequences. Basically it's "If it's not broke, don't fix it." and "Be very paranoid about what you touch. -- Dan White Network Admin Lead
Re: COVID-19 vs. our Networks
On 03/17/20 14:38 -0400, Rich Kulawiec wrote: On Tue, Mar 17, 2020 at 08:38:28AM -0700, Mike Bolitho wrote: Anybody who works in the healthcare vertical will tell you just how bad medical devices are to work with from an IT perspective. Medical devices are appallingly bad to work with from an IT perspective. They're designed and built to work in idealized environments that don't exist, they make unduly optimistic assumptions, they completely fail to account for hostile actors, and whenever possible they are gratuitously incompatible to ensure vendor lock-in. That's the good news. Here's the bad news: in about 2-3 weeks, when our health care systems are stretched to the breaking point, there will be a window of opportunity for adversaries to maximize the damage. On a slightly tangential topic, we had a dictionary attack against customer voice accounts over night, presumably to implement toll fraud. We were in the middle of working out work-from-home plans and were quite distracted with other things. We managed to get on top of it quickly once someone noticed. Attackers taking advantage of this situation is a serious concern. -- Dan White Network Admin Lead
Re: AT is suspending broadband data caps for home internet customers due to coronavirus
On 03/17/20 19:25 +0100, Alexandre Petrescu wrote: Le 17/03/2020 à 19:17, Dan White a écrit : Things have been eerily quiet where we are (Oklahoma). We're an eyeball network and have had no noticeable changes in bandwidth usage that couldn't be explained by statistical noise. We keep game planning more and more contingency scenarios, waiting to jump when needed, but things have just been unexpectedly normal. Perhaps we're behind the game in impact. I'd be curious to hear about networks that are "ahead of us", and what the impact has been. I am not a sysadmin of a Network, but a few hours in advance. The bad news: I can ask you how many cases in Oklahoma? The good news: there is news about medication. By "ahead of us", I'm hoping to glean some operational experience from European, or networks in larger cities with a more impactful lock down. We seem to be going down the same lines of lock downs, and shelf clean outs, just a few days/weeks behind what I've been seeing in the news. I get nervous anytime I hear a school administrator or public official blast out "or binge-watch your favorite shows on Netflix, and of course, wash your hands a lot!" Fortunately the health impact has been minimal here. -- Dan White Network Admin Lead
Re: AT is suspending broadband data caps for home internet customers due to coronavirus
Things have been eerily quiet where we are (Oklahoma). We're an eyeball network and have had no noticeable changes in bandwidth usage that couldn't be explained by statistical noise. We keep game planning more and more contingency scenarios, waiting to jump when needed, but things have just been unexpectedly normal. Perhaps we're behind the game in impact. I'd be curious to hear about networks that are "ahead of us", and what the impact has been. On 03/15/20 02:30 +, John van Oppen wrote: We are seeing the peak spread out… we carry mostly pacific northwest residential networks… we are also seeing new, slightly higher evening peaks. From: NANOG On Behalf Of Rishi Singh Sent: Friday, March 13, 2020 8:25 AM To: Jared Mauch Cc: nanog@nanog.org Subject: Re: AT is suspending broadband data caps for home internet customers due to coronavirus Curious if anyone here (especially at CenturyLink / AT/ Comcast) has seen any graphs of network traffic over time and could share details (redacted of course due to the sensitivity). Would love to hear if/how capacity is constrained with more people working form home. On Thu, Mar 12, 2020 at 4:36 PM Jared Mauch mailto:ja...@puck.nether.net>> wrote: I do worry if the broadband networks have the capacity. WFH traffic is usually different from regular consumer traffic. My neighbors were telling me about the mandatory work from home they had today and how the VPN struggled to work. To those upgrading those things, keep at it. You will get there. Sent from my iCar On Mar 12, 2020, at 6:29 PM, Sean Donelan mailto:s...@donelan.com>> wrote: The first data cap waiver I've seen due to coronavirus. I expect other ISPs to quickly follow. https://www.vice.com/en_us/article/v74qzb/atandt-suspends-broadband-usage-caps-during-coronavirus-crisis AT is the first major ISP to confirm that it will be suspending all broadband usage caps as millions of Americans bunker down in a bid to slow the rate of COVID-19 expansion. Consumer groups and a coalition of Senators are now pressuring other ISPs to follow suit. -- Dan White Network Admin Lead
Re: End to End testing
We've used Accedian MetroNIDs to do this. On 12/12/19 14:53 +, Fawcett, Nick via NANOG wrote: Anyone have any suggestions on devices that I can put at two points in the network to test packet loss, latency, jitter etc. I was thinking of maybe engineering my own using a couple of pi's, but the downfall is they don't have SFP ports. I'm looking for something that's portable and easy to configure and drop in. Thanks. ~Nick -- Checked by SOPHOS http://www.sophos.com -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
Re: lots of traffic starting at 3 a.m. central time
Here's a graph of our eyeball network traffic (CDT): https://ibb.co/kJQYKMq This is an unprecedented increase in traffic for us, day or night, outside of DDOS traffic. Our traffic monitoring system identifies this a Akamai traffic. On 10/15/19 15:54 +, Luke Guillory wrote: That’s what I’m seeing as well, went from 2.2G around 2:50AM CST to a peak +of 16G. https://i.imgur.com/en89kyO.png Luke Guillory Vice President – Technology and Innovation From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Phil Lavin Sent: Tuesday, October 15, 2019 10:48 AM To: Aaron Gould; Nanog@nanog.org Subject: RE: lots of traffic starting at 3 a.m. central time > Anyone else see lots of traffic coming down starting at 3 a.m. central > time ? all of my internet connections showed strangely larger load for a > few early morning hours. > > Someone, on another list, mentioned a 70% increase in traffic to Akamai > which seems to correlate with a new Fortnite release -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
Re: BGP prefix filter list
On 05/15/19 13:58 +, Phil Lavin wrote: We're an eyeball network. We accept default routes from our transit providers so in theory there should be no impact on reachability. I'm pretty concerned about things that I don't know due to inefficient routing, e.g. customers hitting a public anycast DNS server in the wrong location resulting in Geolocation issues. Ah! Understood. The default route(s) was the bit I missed. Makes a lot of sense if you can't justify buying new routers. Have you seen issues with Anycast routing thus far? One would assume that routing would still be fairly efficient unless you're picking up transit from non-local providers over extended L2 links. We've had no issues so far but this was a recent change. There was no noticeable change to outbound traffic levels. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
Re: BGP prefix filter list
On 05/15/19 13:44 +, Phil Lavin wrote: We recently filtered out >=/24 prefixes since we're impacted by 768k day. What kind of network are you running? Doing such prefix filtering on an eyeball network strikes me as insane - you'd be cutting off customers from huge swathes of the Internet (including small companies like us) that don't have large IPv4 sequential allocations. We're an eyeball network. We accept default routes from our transit providers so in theory there should be no impact on reachability. I'm pretty concerned about things that I don't know due to inefficient routing, e.g. customers hitting a public anycast DNS server in the wrong location resulting in Geolocation issues. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
Re: BGP prefix filter list
We recently filtered out >=/24 prefixes since we're impacted by 768k day. I'm attaching our lightly researched list of exceptions. I'm interested in what others' operational experience is with filtering in this way. Filtering /24s cut our table down to around 315K. On 05/15/19 13:43 +0200, Baldur Norddahl wrote: Hello This morning we apparently had a problem with our routers not handling the full table. So I am looking into culling the least useful prefixes from our tables. I can hardly be the first one to take on that kind of project, and I am wondering if there is a ready made prefix list or similar? Or maybe we have a list of worst offenders? I am looking for ASN that announces a lot of unnecessary /24 prefixes and which happens to be far away from us? I would filter those to something like /20 and then just have a default route to catch all. Thanks, Baldur -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com ip prefix-list root-dns seq 1 permit 198.41.0.0/24 ip prefix-list root-dns seq 2 permit 199.9.14.0/24 ip prefix-list root-dns seq 3 permit 192.33.4.0/24 ip prefix-list root-dns seq 4 permit 199.7.91.0/24 ip prefix-list root-dns seq 5 permit 192.203.230.0/24 ip prefix-list root-dns seq 6 permit 192.5.5.0/24 ip prefix-list root-dns seq 7 permit 192.112.36.0/24 ip prefix-list root-dns seq 8 permit 198.97.190.0/24 ip prefix-list root-dns seq 9 permit 192.36.148.0/24 ip prefix-list root-dns seq 10 permit 192.58.128.0/24 ip prefix-list root-dns seq 11 permit 193.0.14.0/24 ip prefix-list root-dns seq 12 permit 199.7.83.0/24 ip prefix-list root-dns seq 13 permit 202.12.27.0/24 ip prefix-list arpa-dns seq 1 permit 199.180.182.0/24 ip prefix-list arpa-dns seq 2 permit 199.253.183.0/24 ip prefix-list arpa-dns seq 3 permit 196.216.169.0/24 ip prefix-list arpa-dns seq 4 permit 203.119.86.0/24 ip prefix-list arpa-dns seq 5 permit 193.0.9.0/24 ip prefix-list gtld-dns seq 1 permit 192.5.6.0/24 ip prefix-list gtld-dns seq 2 permit 192.33.14.0/24 ip prefix-list gtld-dns seq 3 permit 192.26.92.0/24 ip prefix-list gtld-dns seq 4 permit 192.31.80.0/24 ip prefix-list gtld-dns seq 5 permit 192.12.94.0/24 ip prefix-list gtld-dns seq 6 permit 192.35.51.0/24 ip prefix-list gtld-dns seq 7 permit 192.42.93.0/24 ip prefix-list gtld-dns seq 8 permit 192.54.112.0/24 ip prefix-list gtld-dns seq 9 permit 192.43.172.0/24 ip prefix-list gtld-dns seq 10 permit 192.48.79.0/24 ip prefix-list gtld-dns seq 11 permit 192.52.178.0/24 ip prefix-list gtld-dns seq 12 permit 192.41.162.0/24 ip prefix-list gtld-dns seq 13 permit 192.55.83.0/24 ip prefix-list common-public-dns seq 1 permit 8.8.8.0/24 ip prefix-list common-public-dns seq 2 permit 8.8.4.0/24 ip prefix-list common-public-dns seq 3 permit 199.85.126.0/24 ip prefix-list common-public-dns seq 4 permit 199.85.127.0/24 ip prefix-list common-public-dns seq 5 permit 208.67.222.0/24 ip prefix-list common-public-dns seq 6 permit 208.67.220.0/24 ip prefix-list common-public-dns seq 7 permit 8.26.56.0/24 ip prefix-list common-public-dns seq 8 permit 8.20.247.0/24 ip prefix-list common-public-dns seq 9 permit 64.6.64.0/24 ip prefix-list common-public-dns seq 10 permit 64.6.65.0/24 ip prefix-list common-public-dns seq 11 permit 1.1.1.0/24 ip prefix-list common-public-dns seq 12 permit 1.0.0.0/24 ! ARIN ip prefix-list critical-infrastructure seq 1 permit 149.112.112.0/24 ip prefix-list critical-infrastructure seq 2 permit 149.112.149.0/24 ip prefix-list critical-infrastructure seq 6 permit 192.30.45.0/24 ip prefix-list critical-infrastructure seq 9 permit 192.34.238.0/24 ip prefix-list critical-infrastructure seq 13 permit 192.42.173.0/24 ip prefix-list critical-infrastructure seq 14 permit 192.42.178.0/24 ip prefix-list critical-infrastructure seq 21 permit 192.68.130.0/24 ip prefix-list critical-infrastructure seq 22 permit 192.81.185.0/24 ip prefix-list critical-infrastructure seq 23 permit 192.82.133.0/24 ip prefix-list critical-infrastructure seq 24 permit 192.82.138.0/24 ip prefix-list critical-infrastructure seq 25 permit 192.149.62.0/24 ip prefix-list critical-infrastructure seq 26 permit 192.149.63.0/24 ip prefix-list critical-infrastructure seq 27 permit 192.149.64.0/24 ip prefix-list critical-infrastructure seq 28 permit 192.149.65.0/24 ip prefix-list critical-infrastructure seq 29 permit 192.149.66.0/24 ip prefix-list critical-infrastructure seq 30 permit 192.158.252.0/24 ip prefix-list critical-infrastructure seq 31 permit 192.228.21.0/24 ip prefix-list critical-infrastructure seq 32 permit 192.228.79.0/24 ip prefix-list critical-infrastructure seq 33 permit 192.228.92.0/24 ip prefix-list critical-infrastructure seq 34 permit 199.4.137.0/24 ip prefix-list critical-infrastructure seq 35 permit 199.4.138.0/24 ip prefix-list critical-infrastructure seq 36 permit 199.4.144.0/24 ip prefix-list critical-infrastructure seq 37 permit 199.5.26.0
oneamerica.com Contact
If there is a representative from OneAmerica here, please contact me off-list. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
Re: Proofpoint Mail Delivery Issues
On 01/10/19 08:13 -0800, Brian Kantor wrote: On Thu, Jan 10, 2019 at 10:01:07AM -0600, Mike Hammett wrote: There is a mailing list dedicated to email system operators. - Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP Would you have subscription information for that mailing list, please? - Brian https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
DirecTV Now Geolocation Contact
Are there any DirecTV Now contacts on-list? We are having geolocation trouble with a well established ip range. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
Re: Proving Gig Speed
We've found that running windows in safe mode produces better results with Ookla. And MACs usually do better as well. We've gotten >900mb/s with those two approaches. On 07/16/18 17:58 +, Chris Gross wrote: I'm curious what people here have found as a good standard for providing solid speedtest results to customers. All our techs have Dell laptops of various models, but we always hit 100% CPU when doing a Ookla speedtest for a server we have on site. So then if you have a customer paying for 600M or 1000M symmetric, they get mad and demand you prove it's full speed. At that point we have to roll out different people with JDSU's to test and prove it's functional where a Ookla result would substitute fine if we didn't have crummy laptops possibly. Even though from what I can see on some google results, we exceed the standards several providers call for. Most of these complaints come from the typical "power" internet user of course that never actually uses more than 50M sustained paying for a residential connection, so running a circuit test on each turn up is uncalled for. Anyone have any suggestions of the requirements (CPU/RAM/etc) for a laptop that can actually do symmetric gig, a rugged small inexpensive device we can roll with instead to prove, or any other weird solution involving ritual sacrifice that isn't too offensive to the eyes?
Re: CDN-provided caching platforms?
Valve/Steam. On 03/27/18 02:26 +, Russell Berg wrote: I work for a regional Midwestern US "Tier 2" ISP that provides both wholesale and enterprise Internet connectivity. We have caching platforms in place from the likes of Akamai, Google, Netflix, and Facebook; I was wondering if there are other CDN caching platforms out there we should be researching/deploying? PM me if more appropriate... -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@mybtc.com http://www.btcbroadband.com
Re: BGP Community Support WAS: Re: Cogent vs. HE ;-) WAS: Anyone using Cogent Ethernet
On 01/23/18 19:17 +, James Breeden wrote: I've seen the onestep list but the presentation is helpful. I guess my question was moreso to the people side of communities vs the technicalities of communities - what ones do people find themselves using most often and what do they wish was available from providers that really isn't otherwise out there? The most useful to me: Blackhole this prefix Only advertise this prefix to your customers Prepend my/your ASN to this prefix X number of times Do not readvertise this prefix to any content caches you host and I'd like to see more transits offer BFD. -- Dan White
TSYS Contact
If there are any personnel from TSYS here, please contact me off list. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Meraki Contact
Please contact me off list regarding the Meraki dashboard. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Re: Temperature monitoring
We use Asentria. On 07/13/17 22:33 -0400, Dovid Bender wrote: All, We had an issue with a DC where temps were elevated. The one bit of hardware that wasn't watched much was the one that sent out the initial alert. Looking for recommendations on hardware that I can mount/hang in each cabinet that is easy to set up and will alert us if temps go beyond a certain point. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Re: Looking for recommendations for a dedicated ping responder
We're being caught up in some sort of peering dispute between Level 3 and Google (in the Dallas area), and we've fielded several calls from larger customers complaining of 40-50% packet loss (to 8.8.8.8) when there appears to be no actual service impacting loss. We currently suggest customers use a Linux server to ping against, or another public host. Ideally we'd like to use a hardware based ICMP system for customer use - Accedian NIDs are good at this (exceptionally low jitter) accept they throttle at 500 pings per second. On 09/09/16 15:00 -0500, Josh Reynolds wrote: Can you elaborate? On Sep 9, 2016 2:54 PM, "Dan White" <dwh...@olp.net> wrote: Are there any products you're using which are dedicated to responding to customer facing pings? -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Looking for recommendations for a dedicated ping responder
Are there any products you're using which are dedicated to responding to customer facing pings? -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
GoDaddy Contact
Please contact me off list. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Re: ISP License in the USA?
Not familiar with the process, but look at E-rate if you want to provide service to schools, libraries and health providers. On 05/31/16 13:14 -0500, Lorell Hathcock wrote: NANOG: Our owner has hired a consultant who insists that we should have an ISP license to operate in the United States. (Like they have in other countries like Germany and in Africa where he has extensive personal experience.) I am asking him to tell me which license we should have because I don't know of a license that we are required to have to route IP traffic to end customers. I am familiar with CLEC status filed with our state. But it is not a requirement to pass traffic. He is suggesting COALS with which I am completely unfamiliar. Can anyone tell me if there is a Texas state and/or USA Federal license for a small operator to pass IP traffic from the internet to end users (commercial and/or residential). I am aware that there are some CALEA requirements of ISPs that seem to kick in once a CALEA request is made, but is that different from a license. -- Dan White BTC Broadband
Re: Internet DATA Center IP base utilization/Bandwidth Billing
We use Calix Flow Analyze. On 05/12/16 18:51 +0530, sathish kumar Ippani wrote: Hello All, We are looking for software/hardware which can monitor bandwidth usage of each IP address that enters Data center/Leave data center. Based on Bandwidth usage it will draw a graph or calculate Billing. We are running Datacenter and having 2000 and more clients. they are hosted their Network devices. -- With Regards, Sathish Ippani 9177166040 -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
HBO Go Contact
Please contact me off list regarding a geolocation error. -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Fw: new message
Hey! New message, please read <http://campingmeetingpoint.com/thoughts.php?j> Dan White
Fw: new message
Hey! New message, please read <http://ethanmichaelsalon.com/otherwise.php?zc> Dan White
Re: SNMP - monitoring large number of devices
On 09/29/15 22:20 +0200, Pavel Dimow wrote: recently I have been tasked with a NMS project. The idea is to pool about 20 OID's from 50k cable modems in less then 5 minutes (yes, I know it's a one million OID's). Before you say check out some very professional and expensive solutions I would like to know are there any alternatives like open source "snmp framework"? To be more descriptive many of you knows how big is the mess with snmp on cable modem. You always first perform snmp walk in order to discover interfaces and then read the values for those interfaces. As cable modem can bundle more DS channels, one time you can have one and other time you can have N+1 DS channels = interfaces. All in all I don't believe that there is something perfect out there when it comes to tracking huge number of cable modems so I would like to know is there any "snmp framework" that can be exteded and how did you (or would you) solve this problem. I've done about ~60,000 OID queries (over a few dozen devices) per 5 minutes using OpenNMS, which is Java based. At the scale you're looking at, disk I/O would be a major performance issue (if using rrdtool). Google for 'Tuning RRD' for some tips that can make a significant difference. -- Dan White
Re: Debian RWHOIS
On 07/08/15 19:38 +, Josh Moore wrote: Hello guys, What do you use for ARIN resource assignments? I am looking to setup a Debian-based RWHOIS server but don't see much information on it. As of a couple of years ago when I looked around, there were no recent packaged versions of rwhoisd for Debian. We run a compiled version. -- Dan White
Re: Inexpensive software bgp router that supports route tags?
On 07/01/15 15:47 -0400, David H wrote: Sorry I wasn't clear on that. Traditionally on a hardware, e.g. cisco/brocade, router performing the RTBH role, I'd add blackhole routes by way of static routes with a particular tag; one tag for block this source, one tag for block this destination. Redistribute static would let route maps operate against those tags to turn into bgp communities being applied to the announcements, and then the real routers can do what they need to do. When I tried out Quagga/Zebra as an alternative, it doesn't work this way, so while it was nice that it could pick up static routes from the OS, or have them added manually just like a hardware router, there was no concept of the route tag getting to Zebra for it to do the rest of the work on the BGP side. We're using Quagga to inject blackhole routes upstream, which can match routes on the OS's metric value: # IPv4 blackhole ~$ ip route add 203.0.113.42/32 dev lo metric 666 ! route-map map_bad_routes permit 10 match metric 666 set community x:yyy ... ! -- Dan White
Re: Department of Education contact (BOGON listing)?
On 10/28/14 09:38 -0500, Jack Bates wrote: Anyone have a good contact? I emailed the whois for the IP network but haven't heard back yet. Didn't find any contact info in whois for ed.gov. :( ed.gov. 86400 IN NS eduftcdnsp01.ed.gov. ed.gov. 86400 IN NS eduftcdnsp02.ed.gov. ed.gov. 86400 IN NS eduptcdnsp01.ed.gov. ed.gov. 86400 IN NS eduptcdnsp02.ed.gov. dig: couldn't get address for 'eduftcdnsp01.ed.gov' Unable to reach their nameservers from 104/8 networks. Other networks are fine. I also cannot retrieve glue records, but have them cached on my server: Fails (+trace implies +dnssec however): dig +short +trace ed.gov ns Works: ~$ dig @ns.olp.net ed.gov ns cut ;; ANSWER SECTION: ed.gov. 3478IN NS eduptcdnsp01.ed.gov. ed.gov. 3478IN NS eduftcdnsp02.ed.gov. ed.gov. 3478IN NS eduftcdnsp01.ed.gov. ed.gov. 3478IN NS eduptcdnsp02.ed.gov. ;; ADDITIONAL SECTION: eduftcdnsp01.ed.gov.26896 IN A 165.224.21.6 eduftcdnsp01.ed.gov.26896 IN 2610:e8:907f:1:cd99:4fb7:bb7:2213 eduftcdnsp02.ed.gov.26896 IN A 165.224.21.5 eduftcdnsp02.ed.gov.26896 IN 2610:e8:907f:1:d093:1118:adf3:7473 eduptcdnsp01.ed.gov.26896 IN A 165.224.212.6 eduptcdnsp01.ed.gov.26896 IN 2610:e8:9080:1:88dc:90ef:b68:490 eduptcdnsp02.ed.gov.26896 IN A 165.224.212.5 eduptcdnsp02.ed.gov.26896 IN 2610:e8:9080:1:8c49:4c6a:b596:f07b The servers are responding: ~$ dig @165.224.21.6 www.ed.gov cut ;; ANSWER SECTION: www.ed.gov. 3600IN CNAME www.ed.gov.adtihosting.com. edtihosting.com is not retrievable in whois either, but its dns is cached as well. Their webpage (www.adtihosting.com) advertises itself as Localweb.com, and their website lists this contact information: Departments Technical Support: For questions relating to setting up a new account or questions about existing accounts please contact our technical department at 1-800-525-0031 or via e-mail supp...@localweb.com -- Dan White
Re: send packets to different dhcp servers based on client options
On 10/21/14 12:52 +0200, Stephan Alz wrote: If my vendor class identifier contains lets say: motorola.fw0512.5112 string, send it to DHCP server 1 on ip 192.168.1.1 cisco.fw06411.111string, send it to DHCP server 2 on ip 172.16.15.44 The existent relay agents (isc-relay, dhcp-helper) send a copy of all the dhcp servers of the dhcpdiscover message. This is definitely not what I want. For ISC DHCP servers, turning off the 'authoritative' statement will prevent the server from issuing DHCPNAKs, and should essentially allow each server to ignore requests from unknown clients. See dhcpd.conf(5). -- Dan White
Re: About NetFlow/IPFIX and DPI
On 05/07/14 15:11 +0200, Antoine Meillet wrote: Hello, I'm currently writing a paper for school and I talk about net neutrality which brings the subject of NetFlow/IPFIX. Should those protocols be considered as tools to perform DPI ? That question can be taken a couple of ways. Netflow is useful for calculating information useful to providers and operators through sampling of data on high bandwidth links, where performing DPI is not feasible or desired. It is not a robust solution for DPI - or analysis of higher layer packet data, which is typically performed by a mirrored interface or an inline box/firewall that can perform high speed forwarding. -- Dan White
Re: ddos attacks
Can anyone recommend a vendor solution for DDOS mitigation? We are looking for a solution that detects DDOS attacks from sflow information and automatically announces BGP /32 blackhole routes to our upstream providers, or a similar solution. Thank You. On 08/05/13 21:09 +1000, Ahad Aboss wrote: Scott, Use a DDOS detection and mitigation system with DPI capabilities to deal with traditional DDOS attack and anomalous behaviour such as worm propagation, botnet attacks and malicious subscriber activity such as flooding and probing. There are only a few vendors who successfully play in this space who provide a self healing/self defending system. Cheers Ahad -Original Message- From: sgr...@airstreamcomm.net [mailto:sgr...@airstreamcomm.net] Sent: Friday, 2 August 2013 11:37 PM To: nanog@nanog.org Subject: ddos attacks I’m curious to know what other service providers are doing to alleviate/prevent ddos attacks from happening in your network. Are you completely reactive and block as many addresses as possible or null0 traffic to the effected host until it stops or do you block certain ports to prevent them. What’s the best way people are dealing with them? Scott -- Dan White
Re: BRAS
On 12/10/13 19:51 +0530, Nilesh Kahar wrote: Which is a good BRAS product, to handle 15000 subscribers sessions with full QoS other features? Juniper MX (480). -- Dan White
Re: BRAS
On 12/11/13 10:10 -0500, Clayton Zekelman wrote: At 09:30 AM 11/12/2013, Dan White wrote: On 12/10/13 19:51 +0530, Nilesh Kahar wrote: Which is a good BRAS product, to handle 15000 subscribers sessions with full QoS other features? Juniper MX (480). I heard there were some issues with the LAC/LNS functionality on the MX series vs. JUNOSe on the E series. Is that still the case? I have not used those features with the platform, so I can't confirm. The box has been very solid for us as a subscriber management platform for q-in-q termination. -- Dan White
Re: What is your favorite Network Tools Live CD / USB, which you could have running in remote offices?
On 08/22/13 12:06 -0500, Stefan wrote: I've been toying with Live distros (CD, then USB) for many years, in support of security toolsets, to which I kept adding my own stuff, or customizing existing components. I am now trying to build a network toolset LiveCD/USB, but this time with a completely different purpose: I would like to put it in the hands of all remote offices we have on our network, and use it to have local systems boot out of it, and help us then run troubleshooting tools, from the central office, by SSH/X-ing into the remote live system (e.g. iperf, hping3, httping, tcping, mtr, tcpdump, voip tools, some thin clients/apps, synthetic transactions scripted to run at diff time intervals, and report back to us the health seen form the remotes, etc.). Has anybody used a base network tools Live CD/USB that they would recommend, having used as basis for such a network probe functionality? NOTE: I assume *nix based (Linux or BSD flavors), not Windows ... live-build (Debian based) is what I've been using, and has the benefit of allowing you to pick and choose from Debian's vast repository. Here's my latest build script: http://web.olp.net/dwhite/lb.txt -- Dan White
Re: PRISM: NSA/FBI Internet data mining project
On 06/09/13 11:10 -0500, Dan White wrote: Let me put my gold tipped tinfoil hat on in response to your statement. http://www.guardian.co.uk/world/2013/jun/20/fisa-court-nsa-without-warrant If accurate, this is extremely concerning: Top secret documents submitted to the court that oversees surveillance by US intelligence agencies show the judges have signed off on broad orders which allow the NSA to make use of information inadvertently collected from domestic US communications without a warrant. The documents show that even under authorities governing the collection of foreign intelligence from foreign targets, US communications can still be collected, retained and used. ...However, alongside those provisions, the Fisa court-approved policies allow the NSA to: • Keep data that could potentially contain details of US persons for up to five years; Retain and make use of inadvertently acquired domestic communications if they contain usable intelligence, information on criminal activity, threat of harm to people or property, are encrypted, or are believed to contain any information relevant to cybersecurity; All protections afforded by the fourth amendment have essentially been thrown into the (rather large) bit bucket by the FISA court, when it comes to any bits which leave your premise. -- Dan White
Re: PRISM: NSA/FBI Internet data mining project
On 06/07/13 18:20 -0700, Owen DeLong wrote: While the government has no responsibility to protect my data, they do have a responsibility to respect my privacy. While you are correct in that proper personal security procedures to protect my data from random crackers would, in fact, also protect it from the government, that's a far cry from what is at issue here. The question here is whether or not it should be considered legitimate for the US Government to completely ignore the fourth and fifth amendments to the constitution and build out unprecedented surveillance capabilities capturing vast amounts of data without direct probable cause for that snooping. I'm not so much concerned about them gaining access to data I don't want them to access. I am far more disturbed by the trend which reflects a government which increasingly considers itself unrestrained by the laws it is in place to support and implement. Let me put my gold tipped tinfoil hat on in response to your statement. Suppose the following are true: * Meta data for emails sent to and from most US citizens can be captured on a government scale budget * Meta data for all phone calls and skype sessions can also * Cell phone location data - which cell towers your device associates with, over a long period of time - can be captured in log form or stored in a database * Social data can be analyzed to determine who your acquaintances are, and when you communicate with them over time. Now suppose that the NSA contracts with a private company to collect information about terrorist entities, who in turn privately contracts with the top X telecom providers and Y social media companies to obtain all available information that it can, via TAP ports or direct database access. That private organization, through analysis, knows a lot about you, such as every place you've physically been in the last 10 years, what your political leanings are, what criminals you have associated with in that time period, what the likelyhood is that you are a future criminal and of which crimes, how many guns you own, your browsing history and what you like to do in your free time, and insert your own creative idea here. Have your 4th Amendment rights been abridged in this scenario? If you think they have, how confident are you that the court system will agree with you? -- Dan White
Re: PRISM: NSA/FBI Internet data mining project
On 06/07/13 02:34 -0400, Rob McEwen wrote: The oh well, it happens, who cares, guess you need PGP comments on this thread are idiotic. Some of you would benefit from reading the text of the 4th Amendment: The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized OpenPGP and other end-to-end protocols protect against all nefarious actors, including state entities. I'll admit my first reaction yesterday after hearing this news was - so what? Network security by its nature presumes that an insecure channel is going to be attacked and compromised. The 4th Amendment is a layer-8 solution to a problem that is better solved lower in the stack. The Washington Post mentioned some safeguards... but those were pathetic. Why? They seemed to be similar to the following analogy: we'll keep that video camera in your home, recording your every move, and we promise we'll close our eyes when reviewing the tape whenever it shows you naked. THAT is essentially what they're saying. The access described by both the Washington Post and The Guardian is essentially unfettered/unmetered/unmonitored. Just as a doctors take the hippocratic oath to maintain decent standards which are to the benefit of modern civilization... shouldn't IT/Networking/Internet professionals (NANOG readers!!!) have standards that, hopefully, distinguishes us from... say... the State-run ISP of North Korea. And if these allegations are true... then... I have a difficult time believing that there was no quid pro quo involved. Especially since such companies risk a backlash and huge loss of customers if/when this gets out. So I don't think they'd do this without some kind of return in favor. Did they get special tax treatment? Tarp money of any kind (maybe to a parent company)? Easing of regulation enforcement? I assume these taps were put in place under the auspices of (by order of) homeland security or some such. If there were some financial incentive involved, I'd be surprise. -- Dan White
Re: PRISM: NSA/FBI Internet data mining project
On 06/07/13 11:11 -0400, Rob McEwen wrote: On 6/7/2013 9:50 AM, Dan White wrote: OpenPGP and other end-to-end protocols protect against all nefarious actors, including state entities. I'll admit my first reaction yesterday after hearing this news was - so what? Network security by its nature presumes that an insecure channel is going to be attacked and compromised. The 4th Amendment is a layer-8 solution to a problem that is better solved lower in the stack. That is JUST like saying... || now that the police can freely bust your door down and raid your house in a fishing expedition, without a search warrant, without court order, and without probable cause... the solution is for you to get a stronger metal door and hide all your stuff better.|| Hiding stuff better is generally good security practice, particularly in the absence of a search warrant. How effective those practices are is really what's important. From a data standpoint, those security procedures can be highly effective, even against law enforcement. But it's not law enforcement that I worry about the most (understandably, you may have a differing opinion); It's the random anonymous cracker who isn't beholden to any international laws or courts. I design my personal security procedures for him. That's why I don't, say, send passwords in emails. I don't trust state entities to protect the transmission of that data. I don't wish to place that burden on them. You're basically saying that it is OK for governments to defy their constitutions and trample over EVERYONE's rights, and that is OK since a TINY PERCENTAGE of experts will have exotic means to evade such trampling. But to hell with everyone else. They'll just have to become good little subjects to the State. If grandma can't do PGP, then she deserves it, right? I believe it's your responsibility to protect your own data, not the government's, and certainly not Facebook's. Yet... many people DIED to initiate/preserve/codify such human rights... but I guess others just give them away freely. What a shame. Ironically, many who think this is no big deal have themselves benefited immensely from centuries of freedom and prosperity that resulted from rule of law and the U.S. Constitution/Bill of Rights. Freedom is very important to me, as well as the laws that are in place to protect them. -- Dan White
Re: IP4 address conservation method
On 06/05/13 00:34 +0200, Mikael Abrahamsson wrote: I read: http://www.nanog.org/sites/default/files/tues.general.Papandreou.conservation.24.pdf I would like to point out RFC 3069. On most cisco equipment this is done using static routes and ip unnumbered. So my question is basically: What am I missing? Why can't data center guys not build their network the same way regular ETTH is done? Either one vlan per customer and sharing the IPv4 subnet between several vlans, or having several customers in the same vlan but use antispoofing etc (IETF SAVI-wg functionality) to handle the security stuff? VLAN-per-subscriber (1 customer per VLAN), can require more costly routing equipment, particularly if you're performing double tagging (outer tag for switch, inner tag for customer). Sharing an IPv4 subnet among customers is appropriate for residential and small business services, which is how we typically deliver service. But may be less appropriate for larger business customers (and I presume hosting customers) where the number of IPs is large enough that you're throwing away less addresses ratio-wise. Generally the simpler deployment model wins out in that type of scenario. Also, the 'ip unnumbered' approach may require some layer-3 security features. VLAN-per-service (1 customer sharing a VLAN) is problematic, and typically pushes a lot of IPv4 specific layer-3 security features (MACFF, DHCPv4 snooping, proxy arp, broadcast forwarding/split horizon) down into the access equipment, and that's rarely a perfect feature set. In my experience, IPv6 services lag behind on such equipment because those v4 security features break v6. One vlan per customer also works very well with IPv6. +1 -- Dan White
Re: IP4 address conservation method
On 06/05/13 18:57 +0200, Mikael Abrahamsson wrote: On Wed, 5 Jun 2013, William Herrin wrote: Nothing. The problem is that the arp source IP doesn't fall within the interface netmask at the receiver. Some receivers ignore that... after all, why do they care what the source IP is? They only care about the source MAC. Other receivers see a spoofed packet and drop it. Why wouldn't it be within the source IP mask? I would imagine local-proxy-arp would work exactly the same way as if a directly connected host with the IP the ARP request was for would have answered. I've seen two vendors get it wrong: 1) when originating an ARP request, the router uses a source IP that does not match the subnet of the ip being requested (happened when the interface on the router had secondary IPs); 2) when a customer had more than IP address assigned on an interface/VLAN, and one device ARPd the other, the router responded with its own MAC, creating a race condition where sometimes traffic between those two devices was forced up through the router. -- Dan White
Re: Dallas Peering Issues
On 05/13/13 05:59 +, Meshier, Brent wrote: Was alerted by our MSSP of connectivity issues, looks like something big is happening right now that's affecting Level3, Savvis, Cogent, Verizon and Century Link. Take a look at Internet Health Report, never seen it this bad. Anyone have more information? Does anyone have more information about this? We received a customer complaint yesterday, around this time period, with packet loss to 8.8.8.8 (our traceroutes to 8.8.8.8 typically route through Level 3 in Dallas). -- Dan White
McAfee Update failing for newly allocated IP block
http://update.nai.com/Products/CommonUpdater is returning an html generated error (Unable to forward this request at this time), when connecting via a recently allocated IP block. If someone from McAfee is monitoring this list, please contact me off-list. Thank You, -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Re: Global caches
On 02/04/13 14:03 +, Kyle Camilleri wrote: Some CDN providers such as Akamai and Google (often called Global Google Cache) are offering caches to ISPs. It is very convenient for small ISPs to alleviate bandwidth towards the provider, but also the CDN provider benefits by putting source of data closer to the user resulting in far better performance. Does anybody know of any other CDN providers that offer similar caches? Netflix does as well: https://signup.netflix.com/openconnect/hardware The last time I asked, they required 5GB/s of peak traffic to consider you. -- Dan White
Re: Global caches
On 02/04/13 08:33 -0600, Dan White wrote: On 02/04/13 14:03 +, Kyle Camilleri wrote: Some CDN providers such as Akamai and Google (often called Global Google Cache) are offering caches to ISPs. It is very convenient for small ISPs to alleviate bandwidth towards the provider, but also the CDN provider benefits by putting source of data closer to the user resulting in far better performance. Does anybody know of any other CDN providers that offer similar caches? Netflix does as well: https://signup.netflix.com/openconnect/hardware The last time I asked, they required 5GB/s of peak traffic to consider you. Make that: 5 gigabits/s of peak traffic. -- Dan White
Re: MPLS acceptable latency?
On 11/15/12 12:54 -0600, Mikeal Clark wrote: Hello! I have some ATT MPLS sites under a managed contract with latency averaging 75-85 ms without any load. These sites are only 45 minutes away. What is considered normal/acceptable? I recently had a scenario with some MPLS sites within the state nearly doubling in latency (below 50ms round trip). Typically I see round trip latency in the 20-35ms range, and those sites are within about a 90 minute drive from each other (Oklahoma, mostly T1s). When I asked an ATT tech to investigate, he did not see log entries to explain the increase or admit to any trouble, and stated that the service levels for these MPLS circuits allowed for 75-80ms and I don't recall if that was one way or round trip. He said that was to allow for coast to coast latency scenarios. Delay returned to typical levels about 4 days later, without explanation. -- Dan White
Re: Network scan tool/appliance horror stories
On 10/29/12 12:10 -0700, Pedersen, Sean wrote: We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok. http://www.tulsaworld.com/news/article.aspx?subjectid=334articleid=20121002_11_A1_CUTLIN325691 A layer 7 failure. Make sure all members of your organization are aware of your plans. -- Dan White
Re: Windstream outage in WI and MI?
A voice outage has been reported on the outages mailing list: https://puck.nether.net/pipermail/outages/2012-October/004619.html On 10/24/12 14:20 +, bernie listsub wrote: We're seeing multiple sites in our MPLS that are down, and got word from our Windstream sales rep that there was a fiber cut affecting WI and MI. The NOC is either ringing busy or giving a high call volume greeting. Anyone else affected or know more? -Bernie Nick Bernie Bernhardt nick.bernha...@landmark.coop We are a COOPERATIVE business dedicated to providing rural and urban customers with the HIGHEST QUALITY products and services. We will enhance our producers' profitability, EXCEED customer expectations and keep our cooperative financially STRONG. -- Dan White
Re: Issues encountered with assigning .0 and .255 as usable addresses?
On 10/22/12 17:18 -0500, Matt Buford wrote: On Mon, Oct 22, 2012 at 5:07 PM, Paul Zugnoni paul.zugn...@jivesoftware.com wrote: Any experience or recommendations? Besides replace the ISA proxy…. Since it's not mine to replace. Also curious whether there's an RFC recommending against the use of .0 or .255 addresses for this reason. Way back in the late 90's I tried this with a /23 dialup DHCP pool and quickly found that the .0 and .255 users couldn't get to some scattered web sites, though they seemed to be able to get to most of the Internet. However, a year or so ago I spun up an always-on Amazon ec2 instance with a static IP and was handed a .0 address. I still use this VM regularly and have not run into any problems with reachability for this address. I had a similar experience about 10 years ago, with DSL customers who had been assigned .0 or .255 addresses not able to reach some sites. -- Dan White
Re: best way to create entropy?
On 10/11/12 17:08 -0700, Jonathan Lassoff wrote: On Thu, Oct 11, 2012 at 5:01 PM, shawn wilson ag4ve...@gmail.com wrote: in the past, i've done many different things to create entropy - encode videos, watch youtube, tcpdump -vvv /dev/null, compiled a kernel. but, what is best? just whatever gets your cpu to peak or are some tasks better than others? Personally, I've used and recommend this USB stick: http://www.entropykey.co.uk/ Internally, it uses diodes that are reverse-biased just ever so close to the breakdown voltage such that they randomly flip state back and forth. +1. -- Dan White
Re: RFC becomes Visio
On 10/02/12 23:43 +0200, Michael Hallgren wrote: Le mardi 02 octobre 2012 à 23:25 +0200, Dan Luedtke a écrit : On Fri, 2012-09-28 at 19:31 +0100, Nick Hilliard wrote: Here's a visio diagram you can send them: http://www.foobar.org/~nick/bgp-network-diagram.vsd Is there a .png version of it somewhere? The whole thread made my day, I'm eager to see this diagram as well. I don't have this MS Visio thingy you all use to set up your Avian Carrier BGP sessions... Don't use ``MS Visio thingy'', prefer TeX with metapost, PGF/TikZ (or PSTRicks). The output is by far more beautiful, and maintaining the document much more slim. I'd love to use something like metapost, but have yet to find any network diagram oriented libraries. Do you have any examples that you could recommend? -- Dan White
Re: MTU mismatch on one link
On 08/31/12 09:30 -0400, Tom Taylor wrote: Has anyone run into a situation where the MTU at one end of a link was configured differently from the MTU at the other end? How did you catch it? In general, do you see any need for a debugging tool to be standardized to find such mismatches? Performing a ping with a large packet size '-s', and/or with packet fragmentation turned off '-M do' have been our primary tools for finding MTU (layer 2 and layer 3) mismatches. -- Dan White
Re: Fair Use Policy
On 08/23/12 10:51 +0430, Shahab Vahabzadeh wrote: Thanks about every ones speech in this topic but I think I can not describe my problem clearly, let me explain it some how more: You know I have two kind of ADSL services, Limited and Unlimited. Limited Like: 512Kb-4GB-3Month 1024Kb-4GB-3Month 2048Kb-6GB-3Month 4096Kb-8GB-3Month Unlimited Like: 128Kb-1Month 256Kb-1Month and etc. But when a customer is in our sales department they do not know he will download more or he will have a normal usage? Is he heavy peer-to-peer service downloader or not he is a doctor that he want to check his emails only, and he want this service always. We focus on customer education and tools. A bandwidth calculator on your website is recommended, both for your customers, and your sales department (mycricket.com has a great example). We also provide a way for customers to view their current usage for the month, in near real-time. We've implemented a monthly bandwidth quota, and have that discussion up front with new customers (and sent letters to existing customers) so that they can choose the appropriate tiered service, based on their expected usage patterns. -- Dan White
172.0.0.0/12 has been Allocated
172.0.0.0-172.15.255.255 was allocated on 2012-08-20 to ATT Internet Services. -- Dan White
Re: 172.0.0.0/12 has been Allocated
You can do a whois search at arin.net to see the allocation. 172.0.0.0/12 is often confused with the private 172.16.0.0/12 address space, which I would consider a 'scraping the bottom of the barrel' allocation. I also noticed a couple of subnets in that range showing up in the weekly Cidr reports, beginning in July. On 08/23/12 00:29 -0500, Otis L. Surratt, Jr. wrote: Dan, Can you provide a link to support this? If this is true, I wonder how this will work. Otis -Original Message- From: Dan White [mailto:dwh...@olp.net] Sent: Thursday, August 23, 2012 12:24 AM To: nanog@nanog.org Subject: 172.0.0.0/12 has been Allocated 172.0.0.0-172.15.255.255 was allocated on 2012-08-20 to ATT Internet Services. -- Dan White
Re: YouTube Video Streaming
On 05/18/12 10:37 +0800, Thames wrote: I would like to get some input for the following problem we face with YouTube video streaming. We are an ISP in Singapore and peer with Google at Equinix and SOX (Singapore Open Exchange), For about 2 weeks we have been facing choppy streaming or continuous buffering on various YouTube videos. These problem videos are streamed at HD or original quality. Our troubleshooting narrow down to those bad videos being streamed to us from outside Singapore. We contacted Google support, they are confused too, as why we are served from a cache server in Poland on one of the videos. The case has been escalated within Google, unfortunately no update from them since. Not all YouTube videos are bad through us, some 45 minutes videos can fully buffered within seconds on HD or original quality, of course the IP we streamed for these videos are through our local peering with Google. -Thames Thames, We went through something similar here about a year ago. It took around two weeks for Google engineers to resolve the trouble, however, the problem has popped up once or twice since then. Here is a post from the nanog archives that describes a workaround, where you relay youtube dns queries to another DNS resolver in your area that does not experience the problem: http://seclists.org/nanog/2011/May/21 -- Dan White
Re: The day SORBS goes away ...
On 04/09/12 09:50 -0700, Brian Keefer wrote: On Apr 7, 2012, at 4:41 PM, TR Shaw wrote: As for SORBS, most competent mail admins dropped its use a long time ago. I thought when Proofpoint took it over things would change (I actually thought they would dump the SORBS name because of bad karma) but it hasn't happened. Out of curiosity, has anyone other than the OP and one other gentlemen on the 4th had a serious issue? Do we know whether the issues from the 4th have been resolved? I'm wondering whether this is a chronic issue, or if folks are just extrapolating from one complaint. I looked back through the archives for the last year and the only other SORBS mentions were in July and August of last year. I've had nothing but sore issues with SORBS and getting removed from whatever black list was getting us blocked by participating mail systems. Our ARIN allocation is: 67.217.144.0/20 and SORBS had us listed within a larger black listed range, like the containing /12. It took us weeks to be removed from that range (or to have an exception added). This was probably a couple of years ago, or early last year. -- Dan White
Re: Customer Notification System.
We use Mailchimp to relay emails to our customers. They have the ability to maintain lists of customer addresses, and I believe they have an API for maintaining the list. On 02/21/12 17:58 -0500, James Wininger wrote: We are a smaller ISP in Indiana. We are growing quite rapidly (yeah for us). We have a need for a customer notification system. We have simply out grown the ability to send emails to our customers manually. We need to have a better way of notifying our customers of maintenance etc. We would need to send notifications out to say about 400 customers. Ideally the system would send an attached PDF. It would be great if this system were SQL based etc. Does anyone know of a system that is out there that does this? We have looked at a few applications (windows based) but integration with billing etc seems to be a caveat. I have thought of possibly using a mailing list type approach, but that gets us back to (almost) where we are today. Any pointers would be greatly appreciated. -- Jim Wininger jwinin...@ifncom.net -- Dan White
Re: Common operational misconceptions
On 02/15/12 14:47 -0600, John Kristoff wrote: Hi friends, As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct. For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing. I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with. I'd prefer replies off-list and can summarize back to the list if there is interest. I almost always see someone fill in the 'default gateway' field when they're configuring a temporary address on their computer to communicate with a device on the local network. On the topic of VLANs, it's common to think of 'trunking' and something that happens between switches. It's hard to get someone to ponder the fact that trunking isn't an all or nothing concept, and that a server can be configured to speak vlan. Confusing ftp, sftp, ftps. Or telnet, telnets, ssh. Packet loss at hop X in traceroute/mtr does not necessarily point to a problem at hop X. BGP does not magically load balance your ingress and egress traffic. Just because it's down for you, doesn't mean it's down for everyone. -- Dan White
Re: Dear RIPE: Please don't encourage phishing
The line gets crossed when you send an unsolicited message that includes a clickable change password link, that a phisher would find interesting to emulate. After the fact, if a phisher gets one of your customers to click on such a link, you'd like to tell them them in response, or preemptively, that your company never asks for sensitive information via email. With good policies in place, the customer should not be receiving clickable links via email that ask them for a password, except for a scenario where they've actively generated that email in response to an I forgot my password link on a website. On 02/10/12 09:18 -0800, Richard Barnes wrote: So because of phishing, nobody should send messages with URLs in them? On Fri, Feb 10, 2012 at 8:56 AM, Steven Bellovin s...@cs.columbia.edu wrote: I received the enclosed note, apparently from RIPE (and the headers check out). Why are you sending messages with clickable objects that I'm supposed to use to change my password? --- From: ripe_dbannou...@ripe.net Subject: Advisory notice on passwords in the RIPE Database Date: February 9, 2012 1:16:15 PM EST To: [Apologies for duplicate e-mails] Dear Colleagues, We are contacting you with some advice on the passwords used in the RIPE Database. There is no immediate concern and this notice is only advisory. At the request of the RIPE community, the RIPE NCC recently deployed an MD5 password hash change. Before this change was implemented, there was a lot of discussion on the Database Working Group mailing list about the vulnerabilities of MD5 passwords with public hashes. The hashes can now only be seen by the user of the MNTNER object. As a precaution, now that the hashes are hidden, we strongly recommend that you change all MD5 passwords used by your MNTNER objects in the RIPE Database at your earliest convenience. When choosing new passwords, make them as strong as possible. To make it easier for you to change your password(s) we have improved Webupdates. On the modify page there is an extra button after the auth: attribute field. Click this button for a pop up window that will encrypt a password and enter it directly into the auth: field. Webupdates: https://apps.db.ripe.net/webupdates/search.html There is a RIPE Labs article explaining details of the security changes and the new process to modify a MNTNER object in the RIPE Database: https://labs.ripe.net/Members/denis/securing-md5-hashes-in-the-ripe-database We are sending you this email because this address is referenced in the MNTNER objects in the RIPE Database listed below. If you have any concerns about your passwords or need further advice please contact our Customer Services team at ripe-...@ripe.net. (You cannot reply to this email.) Regards, Denis Walker Business Analyst RIPE NCC Database Group Referencing MNTNER objects in the RIPE Database: maint-rgnet -- Dan White BTC Broadband Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610email: dwh...@olp.net http://www.btcbroadband.com
Re: Environmental monitoring options
On 28/09/11 07:53 -0700, eric clark wrote: Thanks for all the replies everyone. Some good options, though I am surprised by how few options I'm finding that have a good centralized management system. I have to deploy monitoring to a bunch of sites spread around the world, centralized management is key. Thanks for all the suggestions. Someone else mentioned Assentria, which we also use for our contact closure alarms. We configure a north bound SNMP interface towards our central trap management system (Vaonet). -- Dan White
Re: Microsoft deems all DigiNotar certificates untrustworthy, releases updates
On 09/09/11 20:06 -0700, Michael DeMan wrote: Sorry for being ignorant here - I have not even been aware that it is possible to buy a '*.*.com' domain at all. I though wildcards were limited to having a domain off a TLD - like '*.mydomain.tld'. Is it true that the my browser on a windows, mac, or linux desktop may have listed as trusted authorities, an outfit that sells '*.*.tld' ? The issue is that a trusted third party's (Diginotar) trusted signing certificate was stolen, allowing the holder to create and sign whatever certificates he wished, which don't necessarily need to be wildcard certs to be effective. Certificate signers are not restricted to any domain hierarchy (a design feature of x.509 pki), which means that *any* trusted stolen signing certificate can wreak havok on the trusted nature of x.509. Even the hint that the claimed Diginotar cracker has gotten her hands on several other signing certificates may be significant motivation to find a replacement for the existing x.509 based pki. On Sep 9, 2011, at 2:54 PM, Paul wrote: On 09/09/2011 11:48 AM, Marcus Reid wrote: On Wed, Sep 07, 2011 at 09:17:10AM -0700, Network IP Dog wrote: FYI!!! http://seattletimes.nwsource.com/html/microsoftpri0/2016132391_microsoft_dee ms_all_diginotar_certificates_untrust.html Google and Mozilla have also updated their browsers to block all DigiNotar certificates, while Apple has been silent on the issue, a emblematic zombie response! Apple has sent out a notification saying that they are removing DigiNotar from their list of trusted root certs. I like this response; instant CA death penalty seems to put the incentives about where they need to be. Marcus Instant? This has been going on for over a week, and a lot of damage could have been done in that time, especially given certs for *.*.com were signed against Diginotar. Most cell phones are unable to update their certificates without an upgrade and you know how long it takes to get them through Cell Phone carriers. A number of alternative android builds are adding the ability to control accepted root certs to their builds in the interest of speeding this up. The CA system is fundamentally flawed. Paul -- Dan White
Re: VRF/MPLS on Linux
On 23/08/11 13:45 +, na...@rhemasound.org wrote: While I have found some information on a project called linux-mpls I am having a hard time finding any solid VRF framework for Linux. I have a monitoring system that needs check devices that sit in overlapping private ip space, and I was wondering if there is anyway I could use some kind or VRF type solution that would allow me to label the site the traffic is intended for. The upstream router supports VRF/MPLS, but I need to know how I can get the server to label the traffic. I would appreciate any input. Although I can't vouch for it, quagga seems to have the command set to function as an MPLS PE router (possibly in conjunction with linux-mpls) to pass vpnv4 routes and tags. That doesn't address how you're going to mux socket connections to the overloaded IP addresses in different VRFs, which would seem to require MPLS knowledge within your monitoring application to support (unless you're running multiple instances). You might consider a more straight forward approach, such as running a separate instance of your monitoring application within a VM, bridged to a separate VLAN towards your MPLS PE, or just running two hosts. -- Dan White
Re: network issue help
On 10/08/11 17:54 -0400, valdis.kletni...@vt.edu wrote: On Wed, 10 Aug 2011 23:37:04 +0200, Tim Vollebregt said: http://www.amazon.com/Networking-Dummies-Doug-Lowe/dp/0470534052 Here you go.. Oh, and he wants to read this helpful guide by Eric S. Raymond, too: http://www.catb.org/~esr/faqs/smart-questions.html Deric doesn't know he wants to.. but he *wants* to. *Right Now*. :) And along similar lines - How to Report Bugs Effectively: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html -- Dan White
Re: FTTH CPE landscape
On 04/08/11 14:32 -0700, Owen DeLong wrote: On Aug 4, 2011, at 2:08 PM, Jay Ashworth wrote: - Original Message - From: Owen DeLong o...@delong.com On Aug 4, 2011, at 8:35 AM, Jay Ashworth wrote: - Generic consumer grade NAT/Firewall Hobby horse: please make sure it support bridge mode? Those of us who want to put our own routers on the wire will hate you otherwise. Why? As long as it can be a transparent router, why would it need to be a bridge? Ask a Verizon FiOS customer who wants to run IPv4 VPNs. He didn't say IPv6 only, right? I have a couple of customers who can't get bridge mode on residence FiOS service, and therefore can't run their own routers to terminate IPsec. If they could get routed static IPv4 rather than bridge, why wouldn't they be able to terminate IPSec VPNs? Note I did say TRANSPARENT router. That would mean no NAT and routed static IPv4. For residential use, for users currently requesting one public address, that's a waste of a /30 block (sans routing tricks requiring higher end customer equipment). Multiply that by the number of residential customers you have and that's bordering on mismanagement of your address space. If you're dealing with business customers, then your usage versus wasted ratio is much higher and less of a concern, but what's the point? Are you trying to cut down on a large broadcast domain? -- Dan White
Re: How do you put a TV station on the Mbone? (was: Royal Wedding...)
On 29/04/11 14:04 -0400, valdis.kletni...@vt.edu wrote: On Fri, 29 Apr 2011 13:48:51 EDT, Jay Ashworth said: Will they not complain about having their equipment utilization go up with no recompense -- for something that is only of benefit to commercial customers of some other entity? Like their load didn't go up with no recompense this morning. For what it's worth, we didn't see much of bump this morning on our broadband network... maybe a 10-15% spike (and non-peak hours at that). What's the break-even point, the number of streams being sent at once where multicasting it starts taking less resources than N unicast streams? Video distribution is bound to continue to go in the direction of Netflix/Youtube where ISPs are going to be highly motivated to find cheaper ways to provide internet content to their end users. And directly peered, multicast agreements between CDNs and ISPs are going to be a real quick way to chop operational costs. Even if that doesn't apply to Netflix content today, it's bound to matter for content that consumers are going to want to consume in real time (sporting events). From the perspective of an ISP operating in a small market, we are seeing a big shift in usage toward Netflix and netflix-like services that is necessarily going to change the model of how we provide internet services. We have limited access to CDN or Content-Producer peering agreements (that would help to save costs) and, even if we did, we're in no position to demand ingress cash flow in those agreements (not enough eyeballs!). Since our users are the ones with the business arrangements with Netflix, and since their demand is shifting in that direction, I'd imagine we'd jump at a chance for private multicast agreements, even if demand didn't quite warrant it at this point. -- Dan White
Youtube Geolocation
We're experiencing very poor quality with You Tube, and it appears we're subject to a bad entry within a geolocation database somewhere. When we attempt to view videos, the contact comes back to us from IPs like: 208.117.226.21 (traceroute's through Frankfurt) 173.194.50.47 74.125.100.29 All of those IPs are 125ms away from us (67.217.144.0/20, and 216.14.144.0/20). However, we've never experienced redirection problems with Google before (we always land at www.google.com), so I'm not sure where to take our trouble. The page at: http://www.google.com/support/websearch/bin/request.py?contact_type=ip isn't of much help as it assumes the problem is google.com redirection. Are there any contacts at Youtube who could provide some assistance? Thanks, -- Dan White
Re: Youtube Geolocation
On 21/04/11 15:46 -0700, Carl Rosevear wrote: Quova, Maxmind, and others all return accurate results for everything of ours I have tested. Some of the IPs in question have been properly assigned or delegated to us for several years in whois. But yeah, thanks for the input... I actually hadn't checked Quova until now. Perhaps Google rolls their own... I'll have to re-echo your experience, which doesn't bode well for our chances. The link below returns accurate information for me too. Our blocks have been in use for years and we have not experienced even a hint of geolocation related problems before. Direct peering would be our ideal solution to this problem, but Google doesn't appear to play in our smaller market. On Thu, Apr 21, 2011 at 3:29 PM, Mike Schoenfeld mike.schoenf...@mediatek.com wrote: I don't know what Google uses but any company using F5 equipment is using Quova geolocation services. You can request updates and check your circuit here: http://www.quova.com/what/request-ip-update/ The problem is that the F5 devices don't update the database files automatically, they need to be manually updated. Unless I get a specific request at my company I don't bother updating on a regular basis.
Re: 0day Windows Network Interception Configuration Vulnerability
On 04/04/11 12:14 -0400, valdis.kletni...@vt.edu wrote: On Mon, 04 Apr 2011 08:46:22 PDT, andrew.wallace said: Someone has recently post to a mailing list: http://lists.grok.org.uk/pipermail/full-disclosure/2011-April/080096.html *yawn* No news, move along, nothing to see. RFC4862, section 6: The use of stateless address autoconfiguration and Duplicate Address Detection opens up the possibility of several denial-of-service attacks. For example, any node can respond to Neighbor Solicitations for a tentative address, causing the other node to reject the address as a duplicate. A separate document [RFC3756] discusses details about these attacks, which can be addressed with the Secure Neighbor Discovery protocol [RFC3971]. It should also be noted that [RFC3756] points out that the use of IP security is not always feasible depending on network environments. Note that similar text was present in RFC2462, all the way back in Dec 1998. So somebody's 13 years late to the party. For more information, see RFC 6104 for a comprehensive problem statement (rogue routers), and RFC 6105 for a proposed solution. -- Dan White
Re: The state-level attack on the SSL CA security model
On 24/03/11 10:09 -0400, Harald Koch wrote: On 3/23/2011 11:05 PM, Martin Millnert wrote: To my surprise, I did not see a mention in this community of the latest proof of the complete failure of the SSL CA model to actually do what it is supposed to: provide security, rather than a false sense of security. This story strikes me as a success - the certs were revoked immediately, and it took a surprisingly short amount of time for security fixes to appear all over the place. The point is that the 'short amount of time' should have been zero (from the time of the update of the CRL) which would have allowed an immediate announcement of the revocation to the public, with sufficient details for the public to make educated decisions about their internet usage. But because the CRL publication did not facilitate that, due to whatever deficiency there existed in the procotol or in browser implementations, announcement had to be delayed, providing a small group of attackers a larger window than necessary to compromise information. -- Dan White
Re: And so it ends...
On 03/02/11 10:38 -0500, Jeffrey Lyon wrote: On Thu, Feb 3, 2011 at 9:58 AM, Alex Rubenstein a...@corp.nac.net wrote: And we have yet to see what happens with backend transactions between private institutions that have large blocks laying around, and them realizing that they have a marketable and valuable thing. We may all say it won't happen, we may even say we don't want it to happen, or that it shouldn't be allowed - but I'm a realist. My theory is that IPv4 will continue to survive with companies becoming more and more conservative on the use of space. IPv6 adoption will happen more substantially as the cost of second hand IPv4 becomes more and more severe, approaching the apex of IPv4 cost vs. IPv6 adoption cost. That makes sense in a 'market' kind of way, however I expect v6 adoption to be much less of a cost curve and more of a flood gate as vendors start rolling out better support, or any support for v6. It's difficult for me to imagine any kind of v4 address market extending the life of the public v4-only internet more than a few months, any more than 2 or 3 legacy /8s getting returned to the global pool would. There's just too much growth and too few networks that would get those addresses to be significant in the larger picture. I do agree that v4 will continue to survive for quite some time though, but not at the expense of v6 adoption. -- Dan White
Re: Future of the IPv6 CPE survey on RIPE Labs - Your Input Needed
On 31/01/11 09:28 -0600, Jack Bates wrote: On 1/31/2011 9:23 AM, Chris Conn wrote: As for the DIR-615, it should, but it doesn't...At least, the E3/E4 revisions I had. I contacted D-LINK support and was able to get a beta build that seems promising. But DHCP-PD over PPPoE works relatively well, minus a couple of little features. I am hoping to have that hammered out soon, as the 615 is a capable little sub-50$ home CPE. But D-Link engineering seems receptive to my observations. My concern as an ISP is the fact that we provide our own CPE, but customers often buy off shelf CPE. This will lead to serious interoperability issues if the whole market doesn't get their act together. There's a fine line we're trying to hold with what we support. We want to establish a recommended list of residential grade routers for our customers (where appropriate), that they can purchase themselves off the shelf, without having to deal with the inevitable you sold me this router, so you need to make it work with my Wii and I don't feel that I should have to pay you type of headaches, if we were to actually sell the routers ourselves. That rules out 3rd party firmware like dd-wrt, since the customer is unlikely to get support when calling the vendor. At this point, I'd be happy with two good options (two different vendors) to recommend. So far, D-link is looking good. -- Dan White
Re: help needed - state of california needs a benchmark
On 29/01/11 10:00 -0800, Mike wrote: The rub is, that they want to legislate that web based 'speedtest.com' is the ONLY and MOST AUTHORITATIVE metric that trumps all other considerations and that the provider is %100 at fault and responsible for making fraudulent claims if speedtest.com doesn't agree. No discussion is allowed or permitted about sync rates, packet loss, internet congestion, provider route diversity, end user computer performance problems, far end congestion issues, far end server issues or cpu loading, latency/rtt, or the like. They are going to decide that the quality of any provider service, is solely and exclusively resting on the numbers returned from 'speedtest.com' alone, period. If you license the software with Ookla, you can install it on a local server and, with your permission, be listed on the speedtest.net site. When your customers visit speedtest.net, your server is, or is close to, the default server that your customers land at. You could try to convince the state that their metric is suboptimal and X is superior, but if your *customers* are anything like ours, it's even harder to educate them why remote speed tests aren't always an accurate measurement of the service you're providing. We've learned to pick our fights, and this isn't one of them. -- Dan White
Re: Future of the IPv6 CPE survey on RIPE Labs - Your Input Needed
On 27/01/11 08:17 -0600, Jack Bates wrote: On 1/27/2011 12:57 AM, Frank Bulk wrote: Have you looked at D-Link's DIR-825? It has most of the things you're looking for. The DIR-655 is a more affordable option. Haven't had the chance to look at that one. Will check it out. In regards to (2), is it even possible to do DHCPv6-PD on with a SLAAC WAN? It had better be, as IOS 12.2 SRE only supports SLAAC + DHCPv6-PD. Most of the Cisco documentation I've seen, says that is their beautiful layout. No more proxyarp/nd. Instead, assign a /64 to each subinterface, perform SLAAC, then hand out prefixes via DHCPv6-PD if someone needs a prefix. The DIR-825(Rev B) running firmware 2.05NA does. From the status screen: IPv6 Connection Type : Autoconfiguration (SLAAC/DHCPv6) Network Status : Connected WAN IPv6 Address : 2610:b8:0:234:218:e7ff:fef8:66dc/64 IPv6 Default Gateway : fe80::c67d:4fff:fed6:5401 LAN IPv6 Address : 2610:b8:100f:1:218:e7ff:fef8:66db/64 LAN IPv6 Link-Local Address : fe80::218:e7ff:fef8:66db/64 Primary IPv6 DNS Server : 2610:b8:0:3:215:c5ff:fef3:f9c8 Secondary IPv6 DNS Server :2610:b8:0:3:215:c5ff:feee:9448 DHCP-PD : Enabled IPv6 network assigned by DHCP-PD : 2610:b8:100f::/48 The latest firmware has fairly good support, but is lacking configurable v6 firewall settings. I haven't done any firewall testing yet, but I'd imagine all incoming v6 connections are blocked. The Emulator hasn't been updated yet to reflect the options in the new firmware, but this should give you an idea of what the configuration looks like: http://www.support.dlink.com/emulators/dir825_revB/203NA/adv_link_local.html The DIR-615 should have similar support, but I haven't upgraded it yet. We're also evaluating a Netgear NWR3500U, which also has v6 support. However, it has the problem that whatever subnet mask is assigned via DHCP-PD is assign on the LAN (so the LAN gets a /48 instead of a /64). Anybody found a work around for that, or have a contact at Netgear? -- Dan White
Re: wikileaks dns (was Re: Blocking International DNS)
On 03/12/10 00:52 -0500, Ken Chase wrote: On Fri, Dec 03, 2010 at 02:26:35PM +0900, Randy Bush said: so, if the site to which a dns entry points suffers a ddos, everydns will no longer serve the domain. i hope they apply this policy even handedly to all sufferers of ddos. if not, as a registrar, i guess i can no longer accept registrations where everydns is the ns delegatee. Let us know if they deviate from this isometric application of policy. I'll be happy to encourage people not to use them. Anyone have records of what wikileaks (RR, i assume) A record was? I should have queried my favourite open rDNS servers before they expired, assuming that the TTL was long enough (or modified to be long by a local cache policy). Quick, someone power up their hibernated laptop with the network unplugged and ping wikileaks (assuming you looked at it recently before hiberation, before it was pulled... :) Not sure that works in any windows (or other OS's for that matter) however. Their A records on Sunday were: #46.51.186.222 wikileaks.org #46.151.171.90 wikileaks.org -- Dan White
Re: RINA - scott whaps at the nanog hornets nest :-)
On 06/11/10 15:56 -0500, Jack Bates wrote: On 11/6/2010 3:36 PM, Richard A Steenbergen wrote: #2. The major vendors can't even agree on how they represent MTU sizes, so entering the same # into routers from two different vendors can easily result in incompatible MTUs. For example, on Juniper when you type mtu 9192, this is INCLUSIVE of the L2 header, but on Cisco the opposite is true. So to make a Cisco talk to a Juniper that is configured 9192, you would have to configure mtu 9178. Except it's not even that simple, because now if you start adding vlan tagging the L2 header size is growing. If you now configure vlan tagging on the interface, you've got to make the Cisco side 9174 to match the Juniper's 9192. And if you configure flexible-vlan-tagging so you can support q-in-q, you've now got to configure to Cisco side for 9170. I agree with the rest, but actually, I've found that juniper has a manual physical mtu with a separate logical mtu available, while cisco sets a logical mtu and autocalculates the physical mtu (or perhaps the physical is just hard set to maximum). It depends on the equipment in cisco, though. L3 and L2 interfaces treat mtu differently, especially noticeable when doing q-in-q on default switches without adjusting the mtu. Also noticeable in mtu setting methods on a c7600(l2 vs l3 methods) In practice, i think you can actually pop the physical mtu on the juniper much higher than necessary, so long as you set the family based logical mtu's at the appropriate value. Cisco calls this 'routing mtu' and 'jumbo mtu' on the platform we have to distinguish between layer 3 mtu (where packets which exceed that size get fragmented) and layer 2 mtu (where frames that exceed that size get dropped on the floor as 'giants'). We always set layer 2 mtu as high as we can on our switches (9000+), and strictly leave everything else (layer 3) at 1500 bytes. In my experience, setting two hosts to differing layer 3 MTUs will lead to fragmentation at some point along the routing path or within one of the hosts. With Path MTU Discovery moved to the end hosts in v6, the concept of a standardized MTU should go away, and open up much larger MTUs. However, that may not happen until dual stacked v4/v6 goes away. -- Dan White
Re: Only 5x IPv4 /8 remaining at IANA
On 21/10/10 16:07 +0100, Ben Butler wrote: Hi, Showing my ignorance here, but this is one of the things I have wondered, given that we run both v4 and v6 for a period of time on the Internet, presumably at one time or another a particular resource may only be able in v4 land, then v4 and v6, then finally v6 only. I have never been particularly clear how an end network that exists only in v4 or v6 address space is able to access a resource that only exists in the other. Is can sort of see some freaking huge NAT box type thing that summarizes v6 in a v4 address scope or contains the v4 address range at some point inside the v6 address space - but how can a v4 host get to a hot in v6 world that sits outside this without going through some form of proxy / nat gateway between the two. Or are the two simply not inter-communicable? I think that's the $64K question. Do you wait to roll out v6 until you start seeing v6-only hosts start popping up? From an accounting and cost recovery stand point, that probably makes sense in some environments. However, consider the fact that there will be v6 only hosts popping up after IANA/RIR/ISP exhaustion. There will be new entrants in the public internet space that cannot obtain v4 addresses and will be reachable via v6 only. That date is starting to become a bit more predictable too. Those v6 only sites won't be Google or Yahoo, but they will be entrepreneurs with good ideas and new services that your customers will be asking to get access to. We're pursuing a dual stacking model today because we anticipate that the dual-stacking process itself will take a while to deploy, and we want to anticipate customer demand for access to v6 only sites. We could hold off on that deployment, and then spend money on work at the moment of truth, but that approach is not very appealing to us. -- Dan White
Re: Only 5x IPv4 /8 remaining at IANA
On 21/10/10 14:53 -0400, Joe Maimon wrote: Dan White wrote: Or are the two simply not inter-communicable? I think that's the $64K question. Do you wait to roll out v6 until you start seeing v6-only hosts start popping up? When do you think that will happen and in what percentages of your target populations to matter? I could guess, but I'd probably be wrong. I'd like to be wrong in the right direction :) From an accounting and cost recovery stand point, that probably makes sense in some environments. However, consider the fact that there will be v6 only hosts popping up after IANA/RIR/ISP exhaustion. There is a phase you are missing between depletion and v6 only hosts. That would be continual and increasing difficulties of obtaining new v4 access and degradation of the quality of that service, hopefully along with a direct inverse effect on the quality and resultant value of v6 service. You're thinking in the big picture. I'm thinking of the specific scenario where my customers start calling me up because they can't get to *one* really important site that couldn't get v4 addresses. I view that as the drop dead date for implementing dual-stack for us. The time line and gradations of that phase are far less clear than depletion. That would explain why so many do not concern themselves with it at this time. Especially those who do not consider themselves to be the party initially responsible for resolving those issues. http://www.dilbert.com/fast/2006-07-30/ I understand the idea that there's going to be a sliding curve of adoption for those with the resources to purchase v4 transfer. I just don't buy into the idea that that's going to push back v6 adoption very much. That's going to be a game for the rich and the richer. In a way, it's kind of like the credit crunch of a year or two ago. The large banks and federal reserve colluded to make sure that credit kept flowing for small businesses and entrepreneurs, even though the current conditions of the market couldn't support it, because restricted access to credit by startups with good ideas would have been a rock to the head of the economy. I think the press are going to rip into the 'dinosaurs' and 'monopolies' who don't move quickly enough to support the nimble expansion of new services based around v6. -- Dan White
Re: Only 5x IPv4 /8 remaining at IANA
Step 1: On 21/10/10 18:34 -0700, Owen DeLong wrote: ROFL, Comcast is already telling their residential customers that if they want a static IPv4 address it will cost them an extra ~$60/month. (Delta between residential and business: ~$55/month, single static IPv4 address on business circuit: $5/month) Owen Step 2: http://lists.arin.net/pipermail/arin-issued/2010-October/000675.html ~$ whois 50.128.0.0 | grep 'NetRange\|OrgName' NetRange: 50.128.0.0 - 50.255.255.255 OrgName:Comcast Cable Communications Holdings, Inc Step 3: Profit! -- Dan White
Re: Definitive Guide to IPv6 adoption
On 18/10/10 19:24 -0700, Doug Barton wrote: On 10/18/2010 5:16 PM, Robert E. Seastrom wrote: sth...@nethelp.no writes: I still haven't seen any good argument for why residential users need /48s. No, I don't think that makes all the address assignments the same size is a particularly relevant or convincing argument. We're doing /56 for residential users, and have no plans to change this. If we were to give a /48 to every human on the face of the planet, we would use about .25 of the total available IPv6 address space. I'm confused. The hand out /48s everywhere crowd keeps saying that we need to do that because we haven't yet anticipated everything that end users might want to do with a /48 on their CPE. On the wider issue of we don't yet understand everything that can be done with the space I think we're in agreement. However my conclusion is that therefore we should be careful to preserve the maximum flexibility possible. After we have some operational experience with IPv6 we will be in a position to make better decisions; but we have to GET operational experience first. Grousing about lack of adherence to holy writ in that deployment doesn't help anybody. I agree with you, but have come to a different conclusion. I would fall under the /48s crowd, except that I'm not really interested in an attempt to standardize /48 deployments. But I still feel strongly that a /48 assignment model for residential customers is right for our environment. With v4 assignments, we have a different philosophy. When we received our v4 assignments from ARIN, is was natural for us to take a conservative approach when handing out addresses... by default we assign one dynamic address to each customer and provide one or more static addresses for a nominal fee to customers, not because we want to make money from it, but because we want to be good stewards of those addresses. That's our 'fail safe' approach to v4 distribution (1 per customer). With v6, our 'fail safe' approach, without strong operational experience, is to assign larger blocks rather than smaller. A cycle in our staff in 5 or 10 years is likely to appreciate that decision, and we can't really justify a /56-rather-than-/48 decision based on address constraints. We really do have the addresses to support /48 deployments for the foreseeable future, and would expect future staff to request more addresses when they're needed. -- Dan White
Re: 12 years ago today...
On 16/10/10 09:09 -0700, Rodney Joffe wrote: I'm not sure about a documentary, but a group of us are working on identifying all the different independent archives that have records from the early years with the idea of creating a Smithsonian/national archive collection at some point. We'll probably issue an rfc early next year. Hopefully someone remembers to call it the Postel Historical Institute[*]. [*] RFC 1607 -- Dan White
Re: Scam telemarketers spoofing our NOC phone number for callerid
On 06/10/10 10:29 -0400, Matthew Huff wrote: We have recently gotten complaints of harrassing and high pressure sales scams orginating from our NOC's phone number. Since the number is a virtual number on the PBX, it can't be used for outgoing calls. I assume the scammers choose the number from the whois db. Anyone else seen this happening? Any suggestions on whom we should contact? Could be Caller ID spoofing. If so, have a recipient of the call perform a trap and trace to find the originator of the call (doing so may require you to file a police report to find who's making the calls, depending on your jurisdiction). If your PBX is SIP based, you might be victim of a SIP registration hijack, which are on the rise, based on traffic we've been seeing in our network. -- Dan White
Re: Facebook down!! Alert!
On 06/10/10 17:05 -0400, david raistrick wrote: my point is that facebook has moved beyond being a pure content provider, and (much like, say, google) provide both content AND service. I have dependancies on facebook's (as do many many others who perhaps dont yet hire folks who even know what nanog is but someday will) services. without them, my teams can't work and my employeer loses signiicant figures of revenue per day. Why can't your teams work? Do they have email? I'm trying to imagine what operational scenarios are involved between the technical staff in a company that depend on Facebook being up, unless you're working for Facebook. Even if I were not email inclined, I'd set up a local XMPP server do to my communication. so facebook is very much operationally relevant for my network, and that these mixed content/service providers will be more and more relevant as time goes on and we as a community should figure out how to deal with their transition from pure content to perhaps some day pure service. How we deal with it is to create a viable distributed version of it. -- Dan White
Re: Lightly used IP addresses
On 16/08/10 09:47 -0700, John Springer wrote: On Sat, 14 Aug 2010, Frank Bulk wrote: This week I was told by my sales person at Red Condor that I'm the only one of his customers that is asking for IPv6. He sounded annoyed and it seemed like he was trying to make me feel bad for being the only oddball pushing the IPv6 feature requirement. FWIW, I asked the same question. My guy was polite, but w/o info. John Springer Hi Frank, I was actually told that there was some demand for it, and that they were targeting 2011 for support, which was acknowledged when I brought it up again in a difference conference call. I'll note that they just got bought out, which may change their priorities, for better or worse. -- Dan White
Re: Lightly used IP addresses
On 13/08/10 21:04 -, John Levine wrote: I've tried to deal with that a few times - mainly by writing up the first upstream AS. Usually they don't care (and every time I have noticed someone blatantly stealing space, it's been spammers). Has there ever been a case where ARIN has tried to take a block back from a party to whom they had allocated it and doesn't want to give it back? My impression is that stolen space is all swamp or legacy or abandoned, but I really don't know. In case it's not obvious, I'm not advocating that people thumb their noses at ARIN, but I don't see any obvious way to avoid my scenario. Make a public example of the situation. Assign such a block to an ARIN member with extensive legal resources who's willing to send some nasty letters out, and back it up with court action to establish legal precedence. Or ARIN could do so itself on the grounds of breach of contract. Of course, said block should clearly fall within ARIN's domain, backed up with a signed contract from the original party. -- Dan White
Re: Vyatta as a BRAS
On 14/07/10 02:18 +, Dobbins, Roland wrote: On Jul 14, 2010, at 3:26 AM, Tony Li wrote: The whole point about being DoS resistant is one of horsepower. To do DoS protection correctly, you need to be able to do packet examination at line rate. Right. And to date, such routers make use of ASICs - i.e., 'hardware-based' routers, in the vernacular. Routers which use only centralized, general-purpose processors can't handle even a fraction of 'line-rate' without tanking, as innumerable real-world examples of said behavior over the years have repeatedly and conclusively demonstrated. I'm not really all that opinionated on the subject, other than to say that the definition of a router, and what qualifies as a sufficient router for any given administrator's needs, greatly varies. However, to state something like as innumerable real-world examples of said behavior over the years have repeatedly and conclusively demonstrated. has the appearance of you struggling to hold on to an idea that may have been more true in the past, and less true today, as is evident based on the input from other list participants. -- Dan White
Re: Email over v6
On 08/07/10 19:04 +0200, Mikael Abrahamsson wrote: On Thu, 8 Jul 2010, Brielle Bruns wrote: By default, at least on Debian, TLS and IPv6 (if available, even if only using link local addresses) are on by default, so there's not too much that needs to be done to use TLS on the SMTP side. TLS wasn't enabled on my Debian using Postfix, so I guess it depends on more factors than just running Debian. IPv6 seems to be on by default, yes. I can confirm that STARTTLS was enabled out of the box on my Debian unstable system... using the snakeoil cert of course. IPv6 (port 25 incoming) was not enabled out of the box. I needed to add inet_protocols = ipv4, ipv6 to enable it. -- Dan White
Re: Inquiries to Acquire IPs
On 02/07/10 15:21 -0600, Michael Loftis wrote: Makes one wonder what dead:beef::/32 and c0ff:ee00::/32 will go for? :) Even more off topic: No match found for cafe:d00d:4:cafe:babe::/32 -- Dan White
Re: useful bgp example
On 19/05/10 13:37 -0500, Jeff Harper wrote: -Original Message- From: Jared Mauch [mailto:ja...@puck.nether.net] Sent: Wednesday, May 19, 2010 1:29 PM To: Jeff Harper Cc: Deric Kwok; nanog@nanog.org Subject: Re: useful bgp example Nice, but you don't show it as-path filtering your transits out. I frequently see people take something learned from transit A and sending it to transit B, and if it happens to be the backup path in-use for your customer, your transits will accept it and likely pick you as best-path and hairpin through your network. - Jared Yeah, I left out the actual prefix-list contents, in hindsight I should have added it, so here it is. Also, a typo in the network statement, lol. network 1.1.1.0 mask 255.255.0.0 ip prefix-list NETZ description The networks we advertise via BGP ip prefix-list NETZ seq 10 permit 1.1.1.0/16 ip prefix-list NETZ seq 1000 deny 0.0.0.0/0 le 32 You should be using 192.168.2.0 for documented examples,or at least private space. Configs like this tend to get cut and pasted into routers and get changed only when they don't work. I just had to change a router config a couple of months ago that a consult had set up using 11.0.0.0/24 and 12.0.0.0/24, for point to point links. -- Dan White
Re: Mail Submission Protocol
On 21/04/10 10:49 -0300, Claudio Lapidus wrote: Hello all, At our ISP operation, we are seeing increasing levels of traffic in our outgoing MTA's, presumably due to spammers abusing some of our subscribers' accounts. In fact, we are seeing connections from IPs outside of our network as many as ten times of that from inside IPs. Probably all of our customers are travelling abroad and sending back a lot of postcards, but just in case... ;-) So we are considering ways to further filter this traffic. We are evaluating implementation of MSA through port 587. However, we never did this and would like to know of others more knowledgeable of their experiences. The question is what best practices and stories do you guys have to share in this regard. Also please let me know if you need additional detail. Depending on what level of pain you want to inflict on your roaming users: 1) Require them to smtp auth to your server when sending mail 2) Require them to use the local SMTP of the server they are connected to, and do not allow remote relay at all. 3) Require them to send mail via a webmail interface when they are not on your local network I would not think that using port 587 is going to work in many cases, such as from Hotel wireless networks. -- Dan White
Re: China prefix hijack
On 08/04/10 13:27 -0400, Joe wrote: Just wondering if this was a Fat fingered mistake or intentional... If it was a mistake, I hope he fares a bit better than his counterparts in other Chinese industries... -- Dan White
Re: ARIN IP6 policy for those with legacy IP4 Space
On 08/04/10 17:17 +, bmann...@vacation.karoshi.com wrote: in the IPv4 space, it was common to have a min allocation size of a /20 ... or 4,096 addresses ... and yet this amnt of space was allocated to someone who only needed to address 3 servers... say six total out of a pool of four thousand ninty six. Granted, that may have been the case many years ago. However, this was not our experience when we obtained addresses, and the ARIN rules as I understand them would not allow such an allocation today. Thats a huge amnt of wasted space. If our wise and pragmatic leaders (drc, jc, et.al.) are correct, then IPv4 will be around for a very long time. What, if any, plan exists to improve the utilization density of the existant IPv4 pool? I believe your question is based on an outdated assumption. -- Dan White
Re: ARIN IP6 policy for those with legacy IP4 Space
On 08/04/10 18:00 +, bmann...@vacation.karoshi.com wrote: On Thu, Apr 08, 2010 at 12:50:26PM -0500, Dan White wrote: On 08/04/10 17:17 +, bmann...@vacation.karoshi.com wrote: in the IPv4 space, it was common to have a min allocation size of a /20 ... or 4,096 addresses ... and yet this amnt of space was allocated to someone who only needed to address 3 servers... say six total out of a pool of four thousand ninty six. Granted, that may have been the case many years ago. However, this was not our experience when we obtained addresses, and the ARIN rules as I understand them would not allow such an allocation today. i picked a fairly recent example - the min allocation size has fluctuated over time. still it is not the case that most folks will get -exactly- what they need - they will - in nearly every case - get more address space than they need - due to the min allocation rules We did, on our first allocation. We were well over 90% utilization and when we asked our upstream ISP for more addresses, we were informed they would not provide us a 17th /24. We scrambled to get our documentation together for ARIN. We had to show efficient use of those 16 /24s, and we had to document our immediate (12-24 month) need for addresses to get them. Thats a huge amnt of wasted space. If our wise and pragmatic leaders (drc, jc, et.al.) are correct, then IPv4 will be around for a very long time. What, if any, plan exists to improve the utilization density of the existant IPv4 pool? I believe your question is based on an outdated assumption. and that outdated assumption is? The assumption that ARIN allocations are based on anything other than 12-24 month need (with only a few exceptions). If there are a significant number of sparse allocations of IPv4 blocks in ARIN, then that's a good indication that allocation rules need to be updated. -- Dan White
Re: legacy /8
On 03/04/10 23:11 -0700, Vadim Antonov wrote: With all that bitching about IPv6 how come nobody wrote an RFC for a very simple solution to the IPv4 address exhaustion problem: +1 years. Step 1: specify an IP option for extra low order bits of source destination address. Add handling of these to the popular OSes. +5 years. Step 2: make NATs which directly connect extended addresses but also NAT them to non-extended external IPs. Step 3: leave backones unchanged. Gradually reduce size of allocated blocks forcing people to NAT as above. Never. Step 4: watch people migrating their apps to extended addresses to avoid dealing with NAT bogosity and resulting tech support calls costs. +10 years. Step 5: remove NATs. This is a good example of why patching v4 or trying to maintain backwards compatibility is not practical. -- Dan White
Re: legacy /8
On 03/04/10 00:09 +, bmann...@vacation.karoshi.com wrote: On Fri, Apr 02, 2010 at 03:13:16PM -0700, Owen DeLong wrote: Sigh... Guess you missed the last several go-arounds of Running out of IPv4 will create some hardships. That cannot be avoided. we won't run out, we won't exaust, we are -NOT- killing the last tuna. what we are doing is roughly what was anticipated in RFC 2050, we will get more efficent utilization of all the space. That statement becomes less truthy when more realistic definitions of 'we' are used. I'd suggest that attempts to stretch v4 addresses is going to fall over on its side, for large segments of we. Address exchanges on the free market, and RIR reclamation will certainly be sufficient for other large segments. However, there will be a point in the next 3-5 years in which neither of these methods will be able to keep up with the tide of technological advancement. Even if we were to reclaim the supposed unused legacy /8s, we'd still only extend the date of IPv4 runout by a few months. wrong analogy. there won't be green field space - but there will still be lots to go around... for legacy style use (e.g. the Internet as we know it today) -- want to do something different? then use IPv6. I already feel like a dinosaur sitting in front of my desktop, with a keyboard. The Internet as we know it today only has 2-3 years of v4 address supply left. The more we stretch address usage, the more we create something that does not resemble the Internet as we know it today. The amount of effort required to reclaim those few IPv4 addresses would vastly exceed the return on that effort. Far better for that effort to be directed towards the addition of IPv6 capabilities to existing IPv4 deployments so as to minimize the impact of IPv4 exhaustion. here we disagree. Im all in favor of demonstrating 85% utilization of the IPv4 address pool before handing out new address space. -- Dan White
100% want IPv6 - Was: New Linksys CPE, IPv6 ?
On 31/03/10 22:14 -0300, jim deleskie wrote: I'm a real life user, I know the difference and I could careless about v6. most anything I want I is on v4 and will still be there long after ( when ever it is) we run out of v4 addresses. If I'm on a From a content perspective, you may be right. Those with a quickly dwindling supply of v4 addresses will most likely use what they have left for business customers, and for content. However, there will be a time when a significant number of customers will not be able to access your content. content provider and I'm putting something new online I want everyone to see, they will find away for all of us with v4 and credit cards to see it, and not be so worried about developing countries or the sub 5% of people in developed countries for now. I'm sure @ some point v6 What percentage of sales are you willing to eat? will see the business need, but while I'm expect to have to deploy it for marketing reasons, I hope its someone else's problem but its a must have for real business. Are you willing to gamble your business on your expectations? Business models will develop that will take advantage of global addressing to end devices. The Next Big (Nth) Thing will. Do you feel that you have a perfect Crystal Ball, or do you want to start hedging your bets now? -- Dan White
Re: 100% want IPv6 - Was: New Linksys CPE, IPv6 ?
On 31/03/10 23:18 -0400, Patrick Giagnocavo wrote: Dan White wrote: From a content perspective, you may be right. Those with a quickly dwindling supply of v4 addresses will most likely use what they have left for business customers, and for content. However, there will be a time when a significant number of customers will not be able to access your content. ^^ Uncertainty . What percentage of sales are you willing to eat? ^^ Fear . Are you willing to gamble your business on your expectations? Business models will develop that will take advantage of global addressing to end devices. The Next Big (Nth) Thing will. Do you feel that you have a perfect Crystal Ball, or do you want to start hedging your bets now? ^^ Doubt. http://www.iana.org/assignments/ipv4-address-space/ -- Dan White
Re: 100% want IPv6 - Was: New Linksys CPE, IPv6 ?
On 31/03/10 23:52 -0400, Patrick Giagnocavo wrote: We have just (anecdotally, empirically) established earlier in this thread, that anything smaller than a mid-sized business, can't even *GET* IPv6 easily (at least in the USA); much less care about it. Talking about a crystal ball, in my view, is just a lot of hand-waving that means I don't have a real-world example to point to. Talking about the Next Big Thing means that somehow, the NBT will be present without any residential or small business broadband users partaking in it. Sounds like a pretty small piece of the pie for the NBT... For the record, I have no dog in this fight; I just think that the rhetoric / fanboi-ism / advocacy level is just a little too high - emotion rather than reason is taking over in the course of debate, which for me at least, is unwelcome. As a (small) service provider with very stiff competition from much larger providers where I work, we have to have a perfect Crystal Ball, or hedge our bets. Customer needs are constantly changing, and are a constantly moving target. Historically we have a good understanding of what they want. We were the first broadband provider in our footprint for several years, but we have lost customers to competition as well. Technology is most notable when it is disruptive, and is probably most devastating to a company like our's when it is. We will only survive if we are prepared, and that's the same advice I would offer anyone who has a penny to lose in this game. -- Dan White
Re: OpenLDAP and Active Directory
On 22/03/10 12:24 -0500, Andrews Carl 448 wrote: Please forgive me if this is an inappropriate place to make this requests. I need to setup an OpenLDAP server for proxy authentication to Microsoft Active Directory. From what I have been able to determine this is completely possible but I am missing something. I have the O'Reilly LDAP book but it was written several years prior to the current OpenLDAP version and there has been a major rewrite of the software. Depending on details, you might find assistance with these two lists: http://www.openldap.org/lists/mm/listinfo/openldap-software http://lists.andrew.cmu.edu/mailman/listinfo/cyrus-sasl If you're wanting to authenticate based on username/password (as apposed to client Kerberos credentials), include 'saslauthd' in your search. Can anyone help me or point me to somewhere I can get assistance? I have tried one consulting firm and that was a stellar failure. I've tried many different searches but a search for 'active directory, openldap, authentication, proxy, pass-through' either gives information for Squid or all go back to the same OpenLDAP Administrators guide from which I am missing something. -- Dan White
