Re: DoD IP Space

2021-01-20 Thread Brandon Martin

On 1/20/21 1:48 PM, Owen DeLong wrote:

Do you think this still holds true if DoD were to (e.g.) sell that space to 

I don’t have any knowledge of any events surrounding this space currently, but 
I do know that press releases and congress have discussed that possibility, so 
it cannot be ruled out.

This is a risk you take when using squad space of any form.  DoD space 
is somewhat uniquely "safe" in this regard but not immune to such things.

Honestly I'd be just about as worried as a potential legitimate non-DoD 
public Internet user of that space about reachability issues as I would 
as someone squatting on it internally within my network about it 
becoming a part of the common global routing table.

I also suspect your typical large corporate environment cares less about 
broad, global reachability than other Internet users in many cases.

Brandon Martin

Re: DoD IP Space

2021-01-20 Thread Brandon Martin
On 1/20/21 12:52 PM, John Curran wrote:
>   While route hijacking isn't necessarily an ARIN issue, I will note 
> that several US law enforcement agencies (FBI & NCIS Cybercrime units) are 
> quite interested in such events and do investigate them looking for criminal 
> activity.   
> (See 
>  for details.) 

I think the difference is semantic but a very important one nonetheless.

Announcing a netblock that isn't yours and that you don't have authorization to 
use to others under the same terms and assumptions as you announce those to 
which you do hold legitimate rights or otherwise purporting to be a legitimate 
user of them on what we know as the "public Internet", that is the Internet 
where numbers are managed by IANA and the relevant RIRs is a "big deal".

Using numbers in a manner contrary to how they are assigned on the "public 
Internet" within a more limited scope where everybody agrees that the use of 
such numbers may be contrary to IANA and relevant RIR assignments is more along 
the lines of "you operate your network however you want".

Other things would fall under the same purview.  For example "alternate root" 
DNS hierarchies with extra TLDs or even TLDs used in contrast to ICANN 
recommendations would have similar considerations.
Brandon Martin

Re: DoD IP Space

2021-01-20 Thread Brandon Martin

On 1/20/21 9:58 AM, j k wrote:
My question becomes, what level of risk are these companies taking on by 
using the DoD ranges on their internal networks? And have they 
quantified the costs of this outage against moving to IPv6?

Honestly I can't think of much unless maybe they're a defense contractor 
that would potentially end up with DoD ranges (non-isolated/classified 
networks) in their view of the global routing table.  Appropriately 
treating it like "my networks" and/or RFC1918 in your routing policies 
(not exporting it, not accepting routes for it, etc.) would be required 
to properly ensure network stability of course.

Some OSes treat RFC1918 space as inherently "special" (extra trusted, 
etc.) and wouldn't treat the DoD ranges as such, but those behaviors are 
typically undesirable or at least not relied on on a network of that 
scale, anyway.

Not that I'd recommend it.
Brandon Martin

Re: Hosting recommendations ... ?

2021-01-19 Thread Brandon Martin

On 1/19/21 12:56 PM, Bryan Holloway wrote:

I'm very curious about your assertion:

Is nested virtualization really a thing?

I mean, I'm not exactly trying to render Pixar's latest movie ... just 
trying to push some bits around (light web-sites, some e-mail ...)

It just seems inherently prone to issues.

Could you back this up with any white-papers or documentation on the 
subject? I'm genuinely interested ...

With KVM, if you have a recent kernel and qemu, it pretty much "just 
works" on supported hardware.  AFAIK Xen supports "Xen on Xen", too, but 
I haven't used it and don't know much about it.

The use case is pretty much exactly this.  You (the product consumer) 
are handed a product that amounts to a virtual machine on somebody 
else's $BIGBOX.  You want to deploy multiple virtual machines where you 
have direct control over their lifecycle, configuration, etc. and can 
bring in additional I/O resources, etc. at the hypervisor level 
(consider that, with KVM, the Linux kernel basically IS the hypervisor). 
 So, you run one or more VMs inside the top level VM that you're handed.

It's full of lots of little wiggles and can be a pain to maintain if you 
have visibility into both levels of the equation, but it does seem to 
work and is surprisingly performant.

See e.g.
Brandon Martin

Re: Hosting recommendations ... ?

2021-01-19 Thread Brandon Martin

On 1/19/21 1:50 PM, William Herrin wrote:

I haven't used Proxmox but from a 60 second glance through Google that
looks like you're asking for nested virtualization. If it works at
all, you'd take a double-hit on everything that wants to run in ring
0, a double-hit on virtualized I/O and a double-hit for OS overhead
making the result more than a little sluggish. Kinda has "bad idea"
written all over it.

KVM, at least, and I think Xen as well, have some features for 
"shunting" I/O and hypervisor calls through to the bare-metal hypervisor 
where possible and avoiding double processing and trampolining.  It's 
not nearly as bad as you might think in terms of performance as long as 
the hardware supports it (nested page tables being the big one).  The 
little I've played with it mostly has proven to be an administrative 
hassle rather than performance.

I would not recommend mixing and matching hypervisors (e.g. Xen on KVM 
or vice-versa), though.  I'm not even sure you can do so meaningfully, 
though I bet someone's working on it.

Brandon Martin

Re: Hosting recommendations ... ?

2021-01-19 Thread Brandon Martin

On 1/19/21 11:44 AM, William Herrin wrote:

Cloud = you get virtual servers with virtual storage, generally
adjustable to meet your needs. You manage the operating systems and
storage within the virtual environment. You DO NOT manage the host
operating systems or hypervisors.

It's worth pointing out that nested virtualization is a thing these 
days, and some providers might even support it!  That means you could 
buy one large instance and sub-divide it yourself into multiple VMs if 
you want to.

In practice, unless you need that flexibility to dynamically spin the 
VMs up and down with various specs AND don't want to or cannot use a 
provider's API for that, I'm not sure why you'd want to if you didn't 
have to for some crazy reason.

Brandon Martin

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

2021-01-06 Thread Brandon Martin

On 1/5/21 7:29 PM, Chris Adams wrote:

I don't know if an unsubscribed cell phone gets the emergency alerts (I
know you are supposed to be able to call 911 from any cell phone, even
if not carrying paid service).  If so, that'd be another cheap way to
get alerts.

They pretty much universally should as long as they still have a SIM in 
them (or are eSIM/ESN-only devices) to give them some idea of their 
preferred network to which to register.  Even if not, they're supposed 
to register with the "best network available" for emergency calling and 
alerting only, though I'm not sure how robust that is.

I have definitely received SMS-broadcast emergency alerts on an 
"inactive" phone that was on and not in airplane mode for various 
reasons, so it does seem to work.  It was a CDMA2000 ESN-only device, 
though, so still had provisioning info.  I usually keep such devices in 
airplane mode, if they're still in use, though, and obviously then the 
cell radio is inactive so no alerts.

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

2021-01-05 Thread Brandon Martin
On 1/4/21 9:07 PM, Masataka Ohta wrote:
> but harder to understand for people who lack precise
> knowledge on what computers and OSes are.

Hi, embedded developer here who spends a considerable amount of time writing 
firmware for "bare metal" systems with "no OS".

I have fairly high confidence that what Mike was referring to was "whatever 
base level software ships on the doohickey and provides the underlying 
infrastructure for the user-visible 'apps' that move media from the Internet to 
the monitor".  Mike, please correct me if I'm wrong.

In that context, it doesn't really matter what the box is running.  Could be 
Linux, could be Windows, could be QNX, could be a "while(1) scheduler" and some 
embedded IP stack.  Anything smart enough to fall under the purview of the 
discussion we're having regarding "streaming services" is going to have some 
sort of "OS-like" infrastructure that could, if desired, centrally handle these 
kinds of alerts so that every single streaming "app" doesn't have to concern 
itself with that.

I can't fathom somebody making a "media streaming device" these days without 
that kind of separation of duties internally regardless of what actually runs 
underneath the user-visible application.  It's not that you couldn't but rather 
that you wouldn't.
Brandon Martin

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

2021-01-04 Thread Brandon Martin

On 1/4/21 10:01 AM, Masataka Ohta wrote:

Any device with speaker should produce audible alert and any
device with display should produce visible alert.

I'm not sure typical North American residential construction could  
tolerate the sonic pressure generated by such a "goal"...

Streaming devices, sure.  They're in-line with conventional distribution  
channels for emergency alerts such as television, radio, etc.  That  
would include "smart speakers", TV streaming sources, etc.

But my refrigerator, washing machine, etc. does not need to do  
this...many could (they have Internet connectivity in some cases), but  
that's just overkill and honestly silly.

I'm also of the mindset that, in the end, the owner/operator of a given  
device should be given the authority to disable emergency alerts on that  
device. It's their device, after all. They may have other ways they  
prefer to receive such alerts, or they may want to die in a tornado. I'm  
not going to stop them. Conventional devices like TVs only alert when  
on, and things like weather radios can be placed into a non-alerting  
mode without losing other owner-relevant functionality.

I'll note that most mobile phones allow the user to turn off most  
(though usually not all) emergency alerts.  Non-OEM OS ROMs often go  

Brandon Martin

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

2021-01-03 Thread Brandon Martin

On 1/3/21 4:22 PM, Mark Delany wrote:

Creating quiescent sockets has certainly been discussed in the context of RSS 
where you
might want to server-notify a large number of long-held client connections very

While a kernel could quiesce a TCP socket down to maybe 100 bytes or so 
(endpoint tuples,
sequence numbers, window sizes and a few other odds and sods), a big residual 
cost is
application state - in particular TLS state.

Even with a participating application, quiescing in-memory state to something 
less than,
say, 1KB is probably hard but might be doable with a participating TLS library. 
If so, a
million quiescent connections could conceivably be stashed in a coupla GB of 
memory. And
of course if you're prepared to wear a disk read to recover quiescent state, 
in-memory cost could be less than 100 bytes allowing many millions of quiescent
connections per server.

Having said all that, as far as I understand it, none of the DNS-over-TCP 
systems imply
centralization, that's just how a few applications have chosen to deploy. We 
deploy DOH to
a private self-managed server pool which consume a measly 10-20 concurrent TCP 

I was thinking more in the original context of this thread w.r.t. 
potential distribution of emergency alerts.  That could, if 
semi-centralized, easily result in 100s of million connections to juggle 
across a single service just for the USA.  While it presumably wouldn't 
be quite that centralized, it's a sizable problem to manage.

Obviously you could distribute it out ala the CDN model that the content 
providers use, but then you're potentially devoting a sizable chunk of 
hardware resources at something that really doesn't otherwise require it.

The nice thing is that such emergency alerts don't require 
confidentiality and can relatively easily bear in-band, 
application-level authentication (in fact, that seems preferable to only 
using session-level authentication).  That means you could easily carry 
them over plain HTTP or similar which removes the TLS overhead you mention.

Several GB of RAM is nothing for a modern server, of course.  It sounds 
like you'd probably run into other scaling issues before you hit memory 
limitations needed to juggle legitimate TCP connection state.

Brandon Martin

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

2021-01-03 Thread Brandon Martin

On 1/3/21 3:11 PM, Jay R. Ashworth wrote:

Well, TCP means that the servers have to expect to have 100k's of open
connections; I remember that used to be a problem.

Out of curiosity, has anyone investigated if it's possible to hold open 
a low-traffic, long-lived TCP session without actually storing state 
using techniques similar to syncookies and do so in a compatible manner? 
 I suspect no since you don't have control over your peers sequence 
numbers, but then someone smarter than I came up with syncookies...

Brandon Martin

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

2021-01-02 Thread Brandon Martin

On 1/2/21 8:41 AM, Masataka Ohta wrote:

As streaming services are often offered from distant places
including foreign locations, generations of emergency alert
packets *MUST* be responsibility of *LOCAL* ISPs.

I mean, if you know where you are, it's trivial to ask various services 
that already exist (in most cases, in some form) if there are emergency 
alerts for your location.  It wouldn't be hard to make this a pubsub 
type system so that a device can just subscribe to it and be notified if 
it happens over a "NAT is everywhere" friendly long-term TCP session 
with TCP and occasionally application-level keepalives.

Media streaming devices could essentially do this now.  The governments 
which publish this information could help by running services that make 
this data more readily available in standard formats and with well-known 
locations and APIs.  IDK if they currently do that.

This is, IMO, how the Internet is supposed to work.  Somebody makes 
content available.  If you want it, ask them for it.  The network just 
moves the data.

ISPs are not typically in the business of flinging unsolicited traffic 
at endpoints.  We're not cable companies (or at least some of us are 
not). And, as you point out, unsolicited UDP traffic is almost 
guaranteed to get dropped even if you have end-to-end address 
transparency as stateful firewalls are quite common even then.

What the ISP can potentially help a lot with is having some 
easily-discovered service to provide the ISP's notion of "where am I 
(probably)?". I wouldn't expect E911 levels of granularity on this, or 
at least I don't think that's a reasonable request to make of most ISPs 
as that would require live data from DHCP, billing, etc. all to be put 
together in ways that could be difficult and cause security or privacy 

What I think IS feasible is something along the lines of a response that 
says "Well, the gear you're terminated on hosts customers within this 
city or zip code or whatever, so that's where you probably are."  This 
is largely static data that you can infer based on large IP swaths (many 
ISPs already essentially put it in their synthesized rDNS) for many 
common configurations and is sufficient for filtering most kinds of 
emergency alerts.

Devices which have GPS can obviously supplement/replace with their own 
location data.  Devices which have access to Wi-Fi/Bluetooth beacon 
location databases can largely do the same.  This is almost guaranteed 
to be more accurate AND more precise.

Brandon Martin

Re: [External] Re: 10g residential CPE

2020-12-27 Thread Brandon Martin

On 12/27/20 5:26 AM, Mark Tinka wrote:
I'm just not sure where all that Li-Ion will go after 15 - 20 years of 
use, though...

One European manufacturer (the one whose battery I bought) says that as 
of now, they can only recycle 20% of each battery they sell. To me, that 
sounds like just the metal case enclosure, and the plastic facia.

Ah well, maybe disposal tech. for Li-Ion storage will have improved by 2040. 

Interestingly, the Lithium content is the, in theory, valuable part of 
it. There's not actually much Li in a typical Li-Ion rechargeable 
battery (much less than a Li metal primary cell), but my understanding 
is that it's enough to have people interested considering that we're 
already basically consuming the world's Lithium supply just about as 
fast as we can economically mine and refine it.  However, that may 
account for the apparently low recyleable content of a given battery. 
By mass and volume, it's mostly electrodes, which are common metals, and 
paper separator which is worthless.

I would imagine that, as "dead" Li-Ion cells become more available and 
demand presumably continues to rise (absent a better battery tech), 
folks will get more serious about recycling the electrolyte.

Brandon Martin

Re: 10g residential CPE

2020-12-24 Thread Brandon Martin

On 12/24/20 7:13 PM, Steven Karp wrote:
Copper 2.5 Gbps Multi-gig uplinks on Wifi 6 gateways are coming out in 
2021 from most vendors.

I am using XGS PON in trials and have been impressed with the speed and 

Pretty much this.  XGS-PON seems to be "here now" and the costs on both 
the CO and CPE side have gotten down to where it's probably worth going 
straight to it (skipping GPON) in new deployments unless you think you 
can get away with just GPON for 5+ years.  I'm not sure if it's worth 
overlaying existing GPON deployments yet, but we're getting close, and 
offering "multi-gig" is, while still not very useful from a practical 
point of view for most customers, a potential marketing advantage.

I've been only recommending GPON for new, greenfield deployments in 
rural situations where expected speeds are low to begin with, density is 
low, and there may be a desire to push the optical link budget as it is 
a bit better than typical XGS-PON systems.  That's been the case for 
about a year, now.

Customer facing routers are not quite there, yet.  I think Asus has one, 
but I've seen mixed reviews.  And what's out now is still limited to 
2.5GBASE-T and often only on the WAN port (LAN ports are still 
1000BASE-T) meaning in practice customers can't get any more than 
gigabit speeds to a single endpoint (not that many endpoints can keep 
up, anyway) for that all-important speed test.

One of my router vendors has been teasing me with a "true 10Gb" router 
due out 1Q 2021.  I've been told to expect NBASE-T (1G, 2.5G, 5G, 10G) 
on both WAN and all LAN ports + 802.11ax "Wifi 6" with at least 5Gbps of 
real-world IPv4 throughput with NAT and essentially wire-speed IPv6 
without NAT or content inspection at a realistic price point.  I'll be 
interested to see what they actually deliver as that would make 
future-looking multi-gig deployments actually meaningful.

Of course, you can replace XGS-PON with 10G-EPON if that's your 
preference.  I actually kinda prefer the IEEE versions, but most of my 
vendors concentrate on the ITU/Bellcore stuff in North America, so 

Brandon Martin

Neteng field laptop/tablet

2020-11-20 Thread Brandon Martin
Sorry if this is perhaps a bit OT...

Can anyone recommend a smallish (10-13" display), relatively lightweight tablet 
or laptop (or convertible) with a native PCIe multi-gig Ethernet port, ideally 
both 10GBASE-T + 2.5GBASE-T (and 5GBASE-T perhaps).  I'm not seeing a lot out 
there, and it's hard to search for multi-gig in a laptop at this point.

Failing that, I'm sure someone has a recommendation for a similar device with 
native PCIe 1000BASE-T + a thunderbolt or similar port I can hang a dongle off 

Ruggedness is useful but not essential.  6-8 hour practical battery life is 
important but almost implied these days.  Must be able to nicely run Linux 
(distro is unimportant).
Brandon Martin

Re: Newbie Questions: How-to remove spurious IRR records (and keep them out for good)?

2020-11-02 Thread Brandon Martin
On 10/30/20 9:26 PM, Rubens Kuhl wrote:
> 1 - You should worry a little, but not much. Filters allowing unwanted
> announcements might be created using these erroneous IRR records, but
> they won't do any damage by themselves. An actual wrong BGP
> announcement is required for any damage to happen, and even without
> those IRR records, a wrong announcement will cause some havoc since
> not everyone builds filters based on IRR and not everyone runs RPKI
> validation.

I've had problems where people who build filters on IRR will build their 
filters SOLELY based on IRR.  That is, they are not permissive and will assume 
that, if there is an IRR object present for a prefix, that ONLY the 
announcements matching that object should be accepted.  This can lead to severe 
reachability issues if not corrected.
Brandon Martin

Re: att or sonic "residential" fiber service at a "nontraditional" residence.

2020-11-01 Thread Brandon Martin

On 11/1/20 8:20 PM, Mark Seiden wrote:

(would this violate some tariff?  could they refuse to install?)

AT's fiber service is not a tariffed service anywhere that I know of. 
 They absolutely could refuse to install it at what they deem a 
"business" location and likely would.  I know Comcast will only install 
"Business Class" service at what they deem "business" locations.  I 
assume this is, in both cases, due to the substantially higher margin on 
"business" services.

As to what Sonic would do, I have no idea.  Their market model is quite 
a bit different.  I also can't imagine they're actually overlaying 
AT's fiber-to-the-prem network as, to my knowledge, AT does not 
allow 3rd party access to it in any market.

Brandon Martin

Re: 100G over 100 km of dark fiber

2020-10-30 Thread Brandon Martin

On 10/30/20 10:27 AM, Vincentz Petzholtz wrote:

If it’s just a single 100G channel needed you could try 100GBASE-ZR4. Specified 
for 80km, 30db power budget they could actually reach more the 80km.
Dispersion should also be „no" problem in the 1310nm length. I have to say that 
I never tried this on 100km distance without coherent solutions.

If you end up marginal with the ZR4 at the receive end, you may be able 
to use a silicon optical preamp to make up the margin.

If you are at the noise floor already (ZR4 receivers are pretty good), 
you may just need to dump more power into the line.  I recently came 
across the PDFA (like an EDFA but the fiber is doped with Praseodymium 
instead of Erbium) which operate on the 1310 band and can push upwards 
of 20-23dBm of power out.  They are not cheap (low 5 figures), but they 
are cheaper than coherent and will still let you run on 1310 so you 
don't have chromatic dispersion issues.  With good (ER4 or ZR4) 
receivers good down to -20dBm or so, you've got 30-33dB of link budget 
which should just about make your 100km if it's decent fiber.  They're 
also usable as a pre-amp (different configuration optimized for gain and 
noise figure instead of power, similar to EDFA preamps) and have 
performance potentially better than silicon amps in that situation.

The 1550 PAM4 pizza box EDFA+Mux+DCM that others have mentioned may also 
work and end up even cheaper, but I normally see those for 40-60km or 
"up to" 80km with them really pushing the link budget at that point.

Honestly, I'd be tempted to just suck it up and do a coherent solution, 
though I admit it would probably be at least 2x the cost.  You can 
probably get a 200G carrier, though.

Brandon Martin

Re: cheap MPLS router recommendations

2020-10-21 Thread Brandon Martin

On 10/21/20 4:27 PM, wrote:

Just to clarify what cheap means, ideally  -$2000 to $4000 new

-new is preferred as buying used kit on second hand market one is at the 
mercy of the price fluctuations and availability.

Do you want SFP or BASE-T on the 1Gb ports?
Brandon Martin

Re: cheap MPLS router recommendations

2020-10-16 Thread Brandon Martin
Most Arista boxes can do pretty much full MPLS (with appropriate 
honor-system licensing) as long as you don't need full-table Internet PE 
capabilities.  At those bandwidths, you could easily get a used box off 
eBay and put it back under support (for more than you paid for the box) 
if you wanted to save some $$$.

Extreme SLX is essentially the same thing with a different badge and a 
different licensing structure.

An old Brocade (now Extreme) NetIron CER-4X (which is still supported 
and sold but nearing end of useful life for most providers) might even 
meet your needs.  You wouldn't need the enhanced route scale hardware 
which makes them cheap on the secondary market.  Avoid the CES even 
though it ostensibly does what you want.  The MPLS signaling on this 
platform is probably a bit more mature but also perhaps lacking some 
modern niceties.  Cost is probably not compelling if buying new, but IDK 
what they're actually selling them for these days.

Don't expect high-touch features like you might get from an ASR or MX, 
but if you just want to push/pop labels, signal L2/L3VPNs, participate 
in IGP with on-net routes, and move data around, they'll do the job with 
decent North-America facing sales and support facilities.

At those bandwidths and low port counts, you can also potentially use 
FRR or Quagga on Linux or *BSD on a suitably sized PC platform.  Linux 
has usable MPLS support these days, though documentation is a bit 
lacking.  One of the BSDs has had it longer and may be more thoroughly 

Brandon Martin

Re: Cogent Layer 2

2020-10-15 Thread Brandon Martin

On 10/15/20 6:15 PM, Robert Blayzor wrote:

On 10/14/20 1:56 PM, Shawn L via NANOG wrote:

When I last spoke to them, it sounded like they were using a bunch of
LAG groups based on ip address because they _really_ wanted to know how
many ip addresses we had and what kind of traffic we would be expecting
(eyeball networks, big data transport, etc).

Not why IP addresses are even an issue on an a "Layer 2" service. But
then again, this is Cogent we're talking about here.

Hashing on LAGs within their core.  Even otherwise fairly braindead 
Ethernet switches often hash on L3 to try and get more entropy.

I hope they hash on L2 MAC, as well, but a pretty common scenario for an 
L2 interconnect only has one MAC on each end of the link, so that 
doesn't help much.  They rallly don't want all your traffic ending 
up on one side of a LAG.

Brandon Martin

Re: Ingress filtering on transits, peers, and IX ports

2020-10-14 Thread Brandon Martin

On 10/13/20 9:40 PM, Nikolas Geyer wrote:

Tl;dr - definitely don’t accept your own prefix from the site it originated 
from, or other sites that have internal connectivity. But also don’t assume 
that an AS has a full-mesh of internal connectivity behind it and shouldn’t 
accept its own prefixes for any reason.

While I can understand some reasons why people don't do it, I believe 
the proper thing to do in this case is have multiple ASNs - one for each 

They obviously have distinct routing policy and thus qualify at least 
under ARIN policy AFAIK.  With AS4, we don't have any imminent shortage 
of ASNs and don't need to be particularly stingy about allocating them 
as long as a need is met.

Brandon Martin

Re: Hurricane Electric AS6939

2020-10-13 Thread Brandon Martin

On 10/13/20 7:29 PM, Aaron Gould wrote:

Do y’all like HE for Internet uplink?  I’m thinking about using them for 100gig 
in Texas.  It would be for my eyeballs ISP.  We currently have Spectrum, Telia 
and Cogent.

They're a good bulk/budget option in a blend.  A decent number of 
content hosts are keen to source traffic into them.  I would be loathe 
to be single-homed to them, but even then they're not the worst option 
in that department.

If you've already got Telia, Spectrum, and Cogent, 6939 is probably a 
great addition to your blend.

They'll probably want a 5yr contract out of you to get their best (and 
headline per-Mbps) rate.  Evaluate carefully what position that puts you 
in based on the continually dropping cost of wholesale transit and bulk 
interconnect opportunities.  You may prefer a shorter contract term at 
slightly higher rates.

As others said, if you don't need full routes from them, they have a 
VERY open peering policy, and that 100G port might be better suited to a 
local IX where you can pick them up along with a bunch of other content 

Brandon Martin

Re: Passive Wave Primer

2020-10-13 Thread Brandon Martin

On 10/13/20 4:01 PM, Mike Hammett wrote:
It seems incredibly simple to do, depending on the capabilities of your 

What am I missing?

If the span between the mux/demux pair is entirely passive, it's fairly 
straightforward.  That's going to limit distances to around 80km or so 
with conventional systems or maybe 120km with systems designed entirely 
around modern coherent optics.

If there are photonic devices in the span, you now have 
customer-supplied light being part of the rainbow that those photonics 
have to handle.  Balancing things at amplifiers requires careful 
coordination with the customer (or adding a separate managed/monitored 
VOA for each alien wave which somewhat defeats the point).  You end up 
with a scenario where a customer can do something screwy and potentially 
affect other waves on potentially multiple spans which your big-name 
carriers are obviously completely freaked out by.

It's obviously possible, but the operational headache seems large enough 
that the major mid-haul and long-haul carriers I've talked to (all North 
America and all midwest, for that matter), don't seem to want to sell it 
despite all the major optical transport platform vendors not just 
supporting it by heavily pushing it.

I really do hope it becomes a real product that I (as a smaller, local 
island operator) can buy, but it just doesn't seem to be there yet at 
least in my region.

Brandon Martin

Re: Passive Wave Primer

2020-10-13 Thread Brandon Martin

On 10/13/20 3:35 PM, Tony Wicks wrote:

We sell some wavelengths on passive CWDM/DWDM path's between Datacentres
(less than 80Km) to customers to spread the cost of leasing the dark fibre.
But yes, as far as long distance (apart from bespoke offerings) I'm yet to
see a productised alien wave service. If you are spending all that money on
OTN kit the extra cost of the transponders is not really significant I

It's in some providers' catalogs (amazingly) but they seem loathe to 
actually sell it when asked.  Even then, you're spot on with the 
description of "bespoke".  I saw Zayo's product description sheet for 
it, and, while it did seem to check the boxes I'd expect, it was also 
quite expectedly very complex (they also wouldn't actually sell it to me).

At less than 80km, you don't have to worry so much about balancing light 
levels, etc. as you can usually just throw things straight into a 
mux/demux on each end and rely on the power budget of the transceiver 
itself, so that makes sense for cheap DCI.

Brandon Martin

Re: Passive Wave Primer

2020-10-13 Thread Brandon Martin

On 10/13/20 8:27 AM, Rod Beck wrote:

Looking for a tutorial on passive waves. How it works. Pros and cons. .

If you're talking about what I think you are, the term the folks who 
make the transport gear seem to use is "spectrum" as in you (as service 
provider) sell your customer some portion of the WDM transport spectrum. 
 It typically comes in 50GHz increments or sometimes 100GHz 
corresponding to the standard ITU channel system but not always.

To the service provider, this is then an "alien wave" in that they have 
essentially no control or visibility into it other than light shows up, 
and they're responsible for getting it to the other end of the path with 
acceptable path characteristics.

I have yet to find a service provider that is actually willing to sell 
this even when they have it in their service offering catalog.  The 
difficulties of coordinating everything with the customer are so extreme 
that it seems to usually make sense to either lease the customer dark 
fiber or capitalize the transponders needed to carry it as a managed 
wave on the provider's transport system.  Might make sense if you 
literally want half the spectrum on a long-haul span or something.

Brandon Martin

CyrusOne Sales Contact?

2020-09-24 Thread Brandon Martin
I've been trying unsuccessfully for the past couple weeks to get in 
touch with the sales folks at CyrusOne.  E-mails and voicemails have 
gone unreturned.

Anyone have a usable contact there or able to matchmake?
Brandon Martin

Re: Ipv6 help

2020-08-29 Thread Brandon Martin

On 8/26/20 12:48 PM, JORDI PALET MARTINEZ via NANOG wrote:

I work and I'm in touch with many CPE vendors since long time ago ... many are on the way 
(I can remember about 12 on top of my head right now, but because contracts, can't name 
them). It takes time. However, in many cases, they just do for specific customers or 
specific models. I know other people that contacted the same vendors and they told them 
"we could do it for the model you use as well". In some cases, they require a 
minimum volume per year (less than what you could expect. I've seen cases that start with 
just 500 units per month).

But this only works if you contact them. The CPE vendors business model seems to be very 
"ISP" direct. I think the retail marked models, unfortunately, will take a bit 
more time.

A hint about some vendors: You may take a look at the co-authors in the RFC.

The whole point here is not that some vendors can support these features 
in a semi-custom firmware image that's specific to a particular ISP 
deployment - I know that's a possibility and have vendors willing to 
work with me on a much smaller volume than even what you've indicated - 
but rather what about customers who want to "upgrade" their router or, 
for whatever reason (and there are some very valid ones) want to provide 
their own.

Right now, it's essentially impossible for them to walk into a local 
retailer and walk out with a model that they know for sure will work 
with my network if I'm requiring e.g. 464XLAT for basic functionality. 
Even if they buy online and dive fairly deep into model documentation, 
it is still hard to come up with a model that they know will work, and 
the lack of documentation will limit their choice of models 
unnecessarily (i.e. there are probably some options that support the 
necessary functionality but don't advertise it in the slightest).

If someone wants me to provide them with a router, then of course I'll 
hand them one that I know works on my network.  That's easy since I have 
control over the selection of it (and have presumably done some testing) 
even if I don't have my own provider-customized firmware.

I'll point out that 500 new services (taking a new router) per month is 
not a particularly small provider.  It's not large, no, but it's also a 
lot bigger than many deployments are at least when they're new, and 
these are questions you have to essentially answer "up front" in many cases.

Brandon Martin

Re: Ipv6 help

2020-08-26 Thread Brandon Martin

On 8/26/20 2:48 AM, JORDI PALET MARTINEZ via NANOG wrote:

This is why we wrote RFC8585, so users can freely buy their own router ...

It's a great RFC.  Hopefully it continues to gain traction.

Do you know of a single router available in the US (or even broader 
North American) retail market that has "RFC8585" printed anywhere on the 
outside of the box or even in the documentation one might actually 
consult pre-purchase?  I would love to evaluate it (or, even better, a 
few of them) to ensure I'm compatible on the provider side with how the 
equipment makers have interpreted the RFC!  Then I can also have some 
specific models to direct people toward along with "Or just look for 
'RFC8585' on the box".

But, right now, I am aware of none.
Brandon Martin

Re: Ipv6 help

2020-08-26 Thread Brandon Martin

On 8/26/20 4:49 AM, Bjørn Mork wrote:

You aren't forcing anything if you allow the users to use any CPE and
document the features it must/should have.

  You want IPv4 access without DNS? Then you need CLAT
  You don't know what CLAT is?  Call your CPE vendor for support
  You don't care what CLAT is?  Use our CPE
  You want to call us for support?  Use our CPE

There is no force here.  It all is pretty similar to

  You want to connect the CPE to our ONT?  Then you need Ethernet
excluding all those TokenRing routers.

I don't force them to.  I was countering the other arguments up-thread 
from folks who do so, and I understand why they do.

I'd say easily 90% of my customers take my offered RG-CPE without any 
questions whatsoever - they in fact have come to expect that I provide one.

Of the remaining 10%, easily half or more of them are satisfied with the 
RG-CPE I provide after I explain a few things about it (and I have a 
cut-sheet for the support folks to do so where applicable).  They mostly 
want to know that it's not a braindead piece of crap presumably because 
they've had bad experiences in the past with SP-provided RG-CPEs (e.g. 
AT U-Verse).

It's the remainder I'm talking about.  It's almost all "power users". 
but even where they do have a fairly firm grasp of basic and even 
moderately advanced home/SMB networking, they're often unfamiliar with 
SP-side features that have cropped up and start to burden the routers 
such as IPv6-IPv4 transition features.  This is what I'd like to address 
in a more streamlined manner.

To wit:

It would be nice if the consumer router industry could get its
collective act together and at least come up with some easy-ish to
understand feature support table that customers can match up with
their service provider's list of needs.

If you create such a feature table as the service provider, and the
customer is unable to match it against their CPE documentation, then
that's a problem between the CPE vendor and the customer.

I can't understand why you want to make it your problem, as long as you
offer a CPE that just works.

I would LOVE to be able to just create such a required features table. 
The issue is that there are virtually no retail (even niche online 
channels) CPE devices that clearly document their capabilities to match 
up with my feature table.  That's what I'm whinging about that the 
retail RG-CPE industry really should address, IMO.

I can adopt best practices to make sure I provide configuration info via 
the proper DHCP(v6) options, attempt to delay making such features 
mandatory by providing e.g. NAT444 fallback, etc. as long as possible, 
etc., but eventually, people are going to have to migrate to equipment 
that supports these more modern features or be left behind, and, right 
now, it's very tough to even identify whether a device you're looking at 
in Best Buy or WalMart supports them or not (leaving aside the fact, for 
now, that the answer is often "it doesn't").

So, I'm left with the "do what works" option of attempting to enumerate 
models known to work.  Nobody likes this.  Customers feel like they're 
unnecessarily constrained, and service providers have to maintain that 
list and deal with questions about it for something that should be 100% 
a customer decision.

Or, I can just say "You have to use our RG-CPE otherwise you're on your 
own and I can't guarantee anything useful will even work.", and that's 
not a good way to sell service to the handful of generally outspoken 
customers who do want to do things their own way.

Brandon Martin

Re: Ipv6 help

2020-08-25 Thread Brandon Martin

On 8/25/20 3:38 PM, JORDI PALET MARTINEZ via NANOG wrote:
This is very common in many countries and not related to IPv6, but 
because many operators have special configs or features in the CPEs they 

I really, really hate to force users to use my network edge router (I 
provide the ONT, though, and I provide an edge router that works and 
most users do take it), but it can be tough to ensure users have 
something that supports all the right modern features and can be 
configured via standard means.

It would be nice if the consumer router industry could get its 
collective act together and at least come up with some easy-ish to 
understand feature support table that customers can match up with their 
service provider's list of needs.  The status quo of a list of devices 
that work right (which is of course often staggeringly short if you're 
doing any of these modern transition mechanisms) that needs constant 
updating and may not be easily available is not ideal.

Heck just having a real, complete list of supported features on the 
model support page on their website would be an improvement...

Brandon Martin

Re: Ipv6 help

2020-08-25 Thread Brandon Martin

On 8/25/20 12:28 PM, Ca By wrote:
I am aware of other big CPE makers too, but this is the public one 
providing product today. Also, anything based on OpenWRT works... which 
is increasingly the base vendors build on.

Last I asked SmartRG (Adtran), they were supporting 464XLAT with CLAT, 
though I haven't verified that it works.  They're at least acknowledging 
demand for it which is a nice step forward.

Brandon Martin

Re: Fiber Automatic Transfer Switch

2020-08-17 Thread Brandon Martin

On 8/17/20 3:33 PM, William Herrin wrote:

I want to dissuade you from using this architecture. When you have two
paths, it's important to keep them both lit so that you can detect
faults and correct them in a timely manner. The most intractable
problem with active/failover architectures is that the failover proves
inoperable when it's needed. The old backup tape that doesn't restore.
Keep your layer-1's active all the time and do fault handling at a
higher layer.

They have their uses - mostly protecting layer 1 products sold as such 
but also protecting things that can't be protected easily at higher 
levels such as RFoG.  However, if you're using them to sell protected 
layer 1 products (protected waves or OTN paths, basically), it's 
important to make sure you have at least some active traffic on the link 
at all times that can be monitored for exactly the reasons Bill alludes. 
 For a lot of networks, this can end up being just the OSC, but as 
that's often not subject to the full photonic path, I'd likewise advise 
against that being the case and to make sure you have at least some 
fully "in band" traffic that can be monitored along both legs.

Brandon Martin

Re: MAP-T in production

2020-07-24 Thread Brandon Martin

On 7/24/20 10:46 PM, Brian Johnson wrote:

OK Randy. How about a suggestion that is useful.

My approach thus far absent CPE support for transition mechanisms has 
been native IPv6 across the board + NAT444, but I use a VRF to 
regionalize the NAT444 routing and bring it to a semi-centralized 
gateway much like one would using DS-Lite, 464XLAT, MAP, LW4o6, etc.

No need to forklift anything, and you can deploy whatever suits you 
incrementally in regions where you need to.  It also works with all 
user-supplied CPE routers which is a bonus as I hate to require that 
users use my CPE router, and it's not yet easy for consumers to verify 
beforehand that the router they're buying at Best Buy will support any 
particular transition tech, though I'd really like to get that fixed.

I'm quite small, though.  I can imagine this would be a hassle compared 
to running a single stack access layer at scale, and of course the NAT 
is stateful no matter what you do with this technique.

Brandon Martin

Re: MAP-T in production

2020-07-22 Thread Brandon Martin
On 7/22/20 6:04 PM, JORDI PALET MARTINEZ wrote:
> The comparison between MAP-T and 464XLAT is not just state.
> With 464XLAT you can have more subscribers (almost unlimited) per IP address, 
> without a limitation on the number of ports, so you save a lot of money in 
> addresses.
> And of course, a limited number of ports in MAP-T means troubles for 
> customers, so help desk cost.

Indeed, the static nature of the carrier-side NAT64 translation, while removing 
state, also imposes restrictions that get more severe as you increase the 
degree of address overload.  I can certainly see why the cellular folks can't 
feasibly adopt it.

However, for strictly wireline folks, especially those just starting to run 
into address space exhaustion problems, even a 4:1 overload or so is quite 
useful and is likely, I think, to generate no more support load than fully 
stateful carrier-side NAT64 ala 464XLAT or NAT444 which is the punt solution, 
per se.

I think that both technologies have their strong use cases.  Folks needing lots 
of overload will probably prefer 464XLAT and just suck up the state handling 
for reasons you describe, while folks who figure they can essentially run out 
the clock on nearly-complete IPv6 transition with only modest overload may 
prefer MAP or LW4o6.

> I've been working a lot with CPE providers (see RFC8585), and I believe that 
> 464XLAT is getting more support.

This has also been my experience.  My preferred CPE vendor is claiming 464XLAT 
support now (though I've not tested it), but doesn't appear to even know what 
MAP or LW4o6 are and certainly has expressed no plans to support it at least at 
the sales engineer questionnaire level.
Brandon Martin

Re: questions asked during network engineer interview

2020-07-22 Thread Brandon Martin

On 7/22/20 6:55 PM, Łukasz Bromirski wrote:

And yes (to the main topic of this thread) - I have some certs.
I understand people without certs tend to discard them as
non-relevant or even toxic. Yes, I’ve met “paper” CCIEs,
but also JNCIEs and I can see the point being made. I’ve
met great minds (also on this list) without any networking
certificates. I believe that until you see real person on the
other side of table and not her/his cert(s), good chat and
questions will remove all doubts. Everyone has to start
somewhere and make those first errors, and being ‘expert’
doesn’t mean you’re not making them anymore.

The CCIE and JNCIE (and perhaps other vendor equivalents) are some of 
the few vendor certs I've found often (though not always) meaningful. 
If the candidate takes it upon themselves to really understand the why 
of what's going on in the prep and testing for those certs, it can be a 
very meaningful track.  Of course if they just answer cram, it's 
bordering on useless, but (and not having either I can't say this with 
certainty) those particular tests seem to try to make such cramming at 
minimum impractical if not impossible.

Of course, there's also plenty of folks out there without them or any 
certs at all that are just as useful in practice.  Getting those 
particular certifications does, however, seem to be a useful path to 
learning things that are actually of use in the "real world".  I look at 
such certificates similar to how I'd look at a 2- or 4-year degree in a 
related IT field and just a somewhat different, and perhaps more 
approachable for the self-coached or differently-learning, path.

At minimum, I think it's a useful jumping off point in an interview for 
a candidate who has paper experience but not a lot of real-world 
experience:  "So, tell me about a particularly dicey interoperability 
scenario you encountered while going for your CCIE?  What steps did you 
take to troubleshoot and either solve or work around it?" or similar.

Brandon Martin

Re: MAP-T in production

2020-07-22 Thread Brandon Martin

On 7/22/20 5:15 PM, Brian Johnson wrote:

Has anyone implemented a MAP-T solution in production? I am looking for 
feedback on this as a deployment strategy for an IPv6 only core design. My 
concern is MAP-T CE stability and overhead on the network. The BR will have to 
do overloaded NAT anyway to access IPv4 only resources. The idea being that 
when IPv4 is no longer needed, this will be a quicker “clean-up” project than a 
dual-stack solution.

I have reviewed Jordan Gotlieb’s presentation from Charter and would love to 
hear if this is still in use at Charter or if was ever fully implemented and 
the experiences)

I’d love any real life examples and success/failure stories.

I'd love to hear about this (or MAP-E, or lw4o6) as well especially with 
regard to CPE support.  My preferred CPE vendor keeps punting on it 
(though they do claim to support 464XLAT), and I'd really like something 
to point them to that will show them it's a "real thing".  Getting rid 
of state at the CGN as is (or can be, at least) necessary with 464XLAT 
seems like a real boon while placing minimal additional burden on the CPE.

Brandon Martin

Re: questions asked during network engineer interview

2020-07-21 Thread Brandon Martin

On 7/21/20 12:55 AM, Mark Tinka wrote:

Suffice it to say, to this day, we still don't know what SDN means to
us, hehe.

I think that's part of the problem.  It means too many different things 
to different people.  Much of what people refer to SDN is useful, albeit 
often pre-existing before the SDN just have to know what it is.

Reminds me of the early days of ".NET" at Microsoft.  Everything was 
".NET", and eventually it became an actual thing.

Brandon Martin

Re: questions asked during network engineer interview

2020-07-20 Thread Brandon Martin

On 7/20/20 4:02 PM, William Herrin wrote:

I find there's a strong INVERSE correlation between the quantity of
certificates on an applicant's resume and their ability to do the job.
I still have to laugh about the guy who let me know via his resume
that he was certified in setting up Kentrox CSU/DSUs.

Pass given to those who cram them into a "certificates" or "specifics" 
line or similar in order to get around HR filters, limit them to major 
certs (or ones your HR dept. specifically demanded), and don't really 
mention them otherwise.  Bear in mind as well that, even if your hiring 
process doesn't demand them, others' will, and many people have a 
standard-ish resume with application-specific cover letter.

Brandon Martin

Re: Anyone running C-Data OLTs?

2020-07-10 Thread Brandon Martin

On 7/10/20 6:22 PM, Alexander Neilson wrote:
I haven’t checked (on mobile) but those affected model numbers could 
confirm if it’s OLT, ONT, or both. Possibly the confusion could come 
from the bug affecting both.

All of the part numbers I was able to find a description of (after 
sifting through the numerous pages copying the vulnerability disclosure) 
appeared to be low-cost, low- to mid-density pizza-box EPON OLTs.  I 
didn't see any ONUs, but then I also didn't find data on everything.

I know a low of EPON deployments go for all-in-ones with the ONU, 
router, WLAN, etc. integrated into a single box presumably because it's 
cheaper for initial deployment than separate boxes for ONU and CPE 
router/AP.  No indication of those being affected in this notice, at 
least that I could find.

Brandon Martin

Re: Layer 3 Switches

2020-06-29 Thread Brandon Martin

On 6/26/20 10:53 PM, Nathaniel Wingard via NANOG wrote:
I’m looking to replace some access switches (Cisco Catalyst 3750 and 
3560G). I really just need L2 features (stacking, PoE+, VLAN). I’ve 
found a 2960X that I like, but Cisco is pushing their 9200 series. The 
only downside I see is that the 9200s look to all have Layer 3 features. 
I’ve always shied away from L3 switches when I don’t need the L3 
features, but I don’t have any solid reason not to just use the switches 
and turn off the L3 features I don’t need. I’m looking for thoughts on 
this approach.

While I can't speak for Cisco, L3 usually comes free (software licenses 
notwithstanding) from most vendors these days.  The off-the-shelf 
silicon generally handles it along with L2 switching.  I'm not sure if 
you can "turn off" the L3 features in IOS XE (which the 9200s run), but 
you can of course just not configure them if you don't need them.

Are you married to Cisco?  The 9200 is not a bad pizza box platform, but 
you can definitely get comparable features and bandwidth cheaper (or 
more bandwidth for the same price) from other folks.

Brandon Martin

Re: Router Suggestions

2020-06-15 Thread Brandon Martin

On 6/15/20 8:00 AM, Colton Conor wrote:
For around $11,000 right now, you can get a brand new Juniper MX204 
router. Alternatively, you can get a used MX240 / MX480 with quad power 
supplies, redundant quad core RE's, and 2 16X10G MIC cards for around 

My question, is there anything else worth looking at in this price range 
/ port configuration? Open to both new and used options. Looking to take 
full BGP routes.

Do you want high-touch or a packet pusher?  The MX204 is somewhere in 
the middle.

Extreme SLX9540 and Arista 7280SR will "take full tables" with some FIB 
compression and route caching.  YMMV if they'll actually work in your 
application, but my experience with the 9540 has been positive in a 
typical leaf edge application.  Street price is in the ballpark of what 
you're talking, and you get a few 100GbE ports to go with your 10GbE ports.

The SLX9640 or 7280R will apparently actually fit full routes in 
hardware, but the pricing seems to be a fair bit higher than you're talking.

All of these are pretty much packet pushers with minimal "touch".  In 
particular, traffic control capabilities are somewhat limited aside from 
applying them to the port itself, and they definitely won't do "BNG" 
type functionality with PPPoE or tag-per-customer with shared L2 
appearance at least not at any real scale.

Brandon Martin

Re: Outsourced NOC Solutions

2020-06-08 Thread Brandon Martin

On 6/8/20 3:01 PM, Matt Harris wrote:
Is that considered true by most leased dark fiber providers? If I'm 
leasing a dark fiber circuit from a provider, I generally expect that 
what I'm leasing is in fact one [or more] physical strands of fiber - 
not a somehow redundant connection. Since he mentioned that this would 
be a dark fiber network, I would tend to assume that's the product that 
he'd be offering. Indeed, this has also been my experience with other 
providers, including very large and relatively smaller ones - when 
leasing dark fiber, or subscribing to a DWDM-based service, I'm going to 
be tied to a single, specific path and physical disruptions to said path 
will impact my connectivity. That's always been my expectation and 
experience at least - am I wrong, or has this changed at some point?

Some carriers offer protected waves.  They're protected at layer 1/1.5 
using a combination of OTN wrappers and optical switches.  My experience 
has been that "wave" services are generally unprotected unless you 
request otherwise.  They're also one of the few "lit" services where 
grooming clauses are not just well-accepted but often standard or even 
implied in the service definition (the service is defined as traversing 
a specific path/paths).  Protection usually comes at a premium cost 
since you're essentially buying the same lambda (or ODU) along multiple 

Glass is glass.  If you want protection, find more glass.  I'm not even 
sure how you'd offer a protected "dark fiber" service without 
encroaching on the ability of the subscriber to light it to their pleasing.

Brandon Martin

Re: understanding IPv6

2020-06-07 Thread Brandon Martin

On 6/7/20 6:01 AM, Denys Fedoryshchenko wrote:
There are very interesting and unobvious moments on IPv4 vs IPv6, for 
example related to battery lifetime in embedded electronics. In ipv4, 
many devices are forced to send "keepalives" so that the NAT entry does 
not disappear, with IPv6 it is not required and bidirectional 
communications possible at any time. And in fact, it has a huge impact 
on the cost and battery life of IoT devices.
When I developed some IoT devices for clients, it turned out that if 
"IPv6-only" is possible, this significantly reduces the cost of the 
solution and simplify setup.

This is difficult to understate.  "People" are continually amazed when I 
show them that I can leave TCP sessions up for days at a time (with 
properly configured endpoints) with absolutely zero keepalive traffic 
being exchanged.

As amusingly useful as this may be, it pales in comparison to the 
ability to do the same on deeply embedded devices running off small 
primary cell batteries.  I've got an industrial sensor network product 
where the device poll interval is upwards of 10 minutes, and even then 
it only turns on its receiver.  The transmitter only gets lit up about 
once a day for a "yes I'm still here" notification unless it has 
something else to say.

In the end, we made it work across IPv4 by inserting an application 
level gateway.  We just couldn't get reliable, transparent IPv6 
full-prefix connectivity from any of the cellular telematics providers 
at the time.  I don't know if this has changed.  For our application, 
this was fine, but for mixed vendor "IoT" devices, it would probably not 
work out well.

Brandon Martin

Re: Integrated WIFI router and phone adapter

2020-05-18 Thread Brandon Martin

On 5/18/20 1:00 AM, K MEKKAOUI wrote:
Anyone knows about a good integrated WIFI router and phone adapter that 
can be used to provide home and business internet and phone service. We 
tried couple of them but we’ve seen some instability and reliability 
issues (i.e. wifi issues, phone issues, etc.). Also some of them are 
designed to work better over DSL but not over DOCSIS.

SmartRG (now part of Adtran) has some Ethernet-connected and therefore 
largely provider-tech agnostic models with FXS ports.  I've not used the 
voice functionality of them, but I've been quite happy with the Wi-Fi 
and NAT, and they generally "do the right thing" out of the box for most 

Brandon Martin

Re: How to manage Static IPs to customers

2020-05-08 Thread Brandon Martin

I'm curious...

Is it part of the DOCSIS spec that the CMTS terminates L3, or can they 
bridge to IEEE 802(.3) and delegate that to some other piece of gear? 
I'm unfortunately not familiar with the MSO world much at all aside from 
a little bit of L1.

Brandon Martin

Re: How to manage Static IPs to customers

2020-05-07 Thread Brandon Martin

On 5/7/20 4:49 PM, Javier Gutierrez Guerra wrote:

Just wanted to reach out and get an idea how is people managing customers with 
static Ips, more specifically on Docsis networks where the customer could be 
moved between cmts's when a node is split

Around here, Comcast seems to provision a GRE tunnel from the CPE back 
to some router within their network and run it over whatever IP address 
the CMTS hands out to the modem.  Not very efficient, and it mandates 
that you use their CPE (they won't give you the necessary info to set it 
up yourself).

AFAIK, such service is only available on their "business class" DOCSIS 
product and is upcharged even then.

Brandon Martin

Re: alternative to voip gateways

2020-05-07 Thread Brandon Martin
On 5/7/20 12:03 PM, Mel Beckman wrote:
> In the OP’s case however, the copper plant is private, and wholly owned and 
> already in operation. So surely in that situation fiber would be much more 
> expensive to dig and trench. 

Indeed, I was responding to Ohta's comments regarding copper vs. fiber.  In 
this case, using DSL over the existing plant seems like a slam dunk unless very 
high speeds are needed or the plant is in very poor condition.  Modern VDSL/2 
DSLAMs are relatively inexpensive and will push 100Mbps over surprising 
distances with essentially seamless fallback to ADSL2+ at ~24Mbps for 
long-reach situations.
Brandon Martin

Re: McAfee's certificate on akamai seems to be invalid

2020-05-07 Thread Brandon Martin

On 5/7/20 12:16 PM, Niels Bakker wrote:
It looks like you shouldn't attempt to access that site over HTTPS, just 
via plain HTTP.  Do you have any official bit of documentation 
that links to the HTTPS version?

Given the prevalence of opportunistic upgrades to TLS these days, I'd 
argue that having a misbehaving server listing on 443 (and accepting SNI 
for a name that works on plain HTTP, if applicable) at the same domain 
as a well-known, public HTTP server, especially from a "security" 
company, is a poor idea.

Brandon Martin

Re: alternative to voip gateways

2020-05-07 Thread Brandon Martin

On 5/7/20 5:13 AM, Masataka Ohta wrote:

Investment for FTTH is 10 times or more than that for plain DSL.

Only if you're comparing entirely new copper plant to existing copper  
plant (including drops), in my experience.  If you compare greenfield to  
greenfield, the cost of fiber to the prem is not much greater than  
copper (coax or twisted pair).

If you've already got access to existing copper plant, then reusing at  
least the drops is definitely worth looking into, yes.

In most of the USA, it's simply not cost-feasible to get access to that  
unless you either are the ILEC or are a well-established CLEC from a  
long time ago.  The ILEC mostly gets free reign to set the access costs,  
and they set them sufficiently high as to "discourage" competition from  
using it where they can get away with it.

Brandon Martin

Re: Arista Switches rebooting

2020-05-05 Thread Brandon Martin

On 5/5/20 2:09 AM, Saku Ytti wrote:

I2C is a pretty terrible bus, particularly if you try to actually hang
everything off of a single I2C bus, single misbehaving speaker and you
might get your power supplies offline. Hopefully we'll move to I3C or
10SPE Ethernet soon. Or maybe some sort of I2C switch where every
connection is on its own bus.

I was of the impression that, due to addressing, I2C bus switches are 
already typically used between each transceiver port and the system bus 
(hopefully one of many) that connects the micro to the ports (hopefully 
relatively dedicated to that purpose).

That's certainly been the case in all of the switches I've torn down 
and, last time I checked at least the SFP specification, there wasn't 
much of a way around it since there was each transceiver uses the same, 
fixed addresses for ID and DDM.

But yes, I2C, while very useful, is the devil.  Clock stretching is 
particularly annoying along with the requisite use of open-drain drivers 
to accomplish it.

I was not aware of 10SPE, though...looks very useful (for lots of 
purposes).  Physical multi-drop on low-cost cabling is quite useful.

Brandon Martin

Re: alternative to voip gateways

2020-05-04 Thread Brandon Martin

On 5/4/20 9:44 AM, Colton Conor wrote:
Adtran has a built in web interface too. I it slow, but it does work. I 
like CLI better.

Oh yeah, I forgot about that.  It does work for most day-to-day tasks, 
though there are some things you can't do from it and have to drop to 
the CLI for.  Overall, I prefer the CLI.

Brandon Martin

Re: alternative to voip gateways

2020-05-04 Thread Brandon Martin

On 5/4/20 6:11 AM, Nick Edwards wrote:

Thanks for suggestion, as per previous, how easy it to configure?
It needs to be understood by laymen if possible,  i'm no layman, but
im no carrier grade networking guru either, most my setups are 300 odd
users, where the gateway and dslam method is cost feasible, but not
when your looking at the numbers of this project - hence reason for my

I've not used Calix gear, but the Adtran TA5000 supports a Cisco-like 
CLI.  It can also be provisioned entirely using SNMP, if that's more to 
your liking, and in fact that is how Adtran's in-house provisioning 
suite works.  It also has some TL1 support if you really must...

The CLI has some necessary deviations from Cisco IOS, of course, as 
almost all "industry standard" CLIs do, but you can probably pick it up 
pretty quickly.

If you buy new/prime gear directly from an authorized Adtran disty you 
also get their provisioning and monitoring suite (AOE) "free" as long as 
you maintain a support contract (which isn't particularly expensive). 
It's kinda blah (and Flash-based, but I'm told that's changing by the 
end of the year...) but does work.

Brandon Martin

Re: CGNAT Solutions

2020-04-29 Thread Brandon Martin

On 4/29/20 10:12 PM, William Herrin wrote:

What allows them to work with v6 in such an efficient manner?

A piece of client software is installed on every phone that presents
an IPv4 address to the phone and then translates packets to IPv6 for
relay over the network. This works because T-Mobile has considerable
control over the phone.

FWIW, this software component (the CLAT) can also be on the CPE edge 
router which many ISPs either control outright these days or at least 
can influence.

Brandon Martin

Re: CGNAT Solutions

2020-04-29 Thread Brandon Martin

On 4/29/20 2:35 AM, Masataka Ohta wrote:

If you mean getting rid of logging, not necessarily. It is enough if
CPEs are statically allocated ranges of external port numbers.

Yes, you can get rid of the logging by statically allocating ranges of  
port numbers to a particular customer.

What I was referring to, though, was the programmatic state tracking of  
the {external IP, external port}-{internal IP, internal port} mappings.  
 You can't eliminate that unless the CPE also knows what internal port  
range it's mapped to so that it restricts what range it uses.  If you  
can do that, you can get rid of the programmatic state tracking entirely  
and just use static translations for TCP and UDP which, while nice, is  
impractical.  You're about 95% of the way to LW4o6 or MAP at that point.

Brandon Martin

Re: CGNAT Solutions

2020-04-28 Thread Brandon Martin

On 4/28/20 4:53 PM, William Herrin wrote:

How small is small? Up to a certain size regular NAT with enough
logging to trace back abusers will tend to work fine. if we're talking
single-digit gbps, it may not be worth the effort to consider the
wonderful world of CGNAT.

Depending on how many IPs you need to reclaim and what your target 
IP:subscriber ratio is, you may be able to eliminate the need for a lot 
of logging by assigning a range of TCP/UDP ports to a single inside IP 
so that the TCP/UDP port number implies a specific subscriber.

You can't get rid of all the state tracking without also having the CPE 
know which ports to use (in which case you might as well use LW4o6 or 
MAP), but at least you can get it down to where you really only need to 
log (or block and dole out public IPs as needed) port-less protocols.

Brandon Martin

Re: Are underground utility markers essential workers?

2020-04-21 Thread Brandon Martin

On 4/21/20 2:57 PM, Sean Donelan wrote:

Utility markers don't get the recognition they deserve.  If they aren't
essential workers, they should be and get hazard pay.

They help protect everyone's fiber and cables and pipes that go boom.

If underground construction is an essential activity (and it certainly 
is, at least repairs generally are - new construction perhaps could be 
argued), then underground utility marking is, too, since it's mandatory 
for safely performing underground construction.

Brandon Martin

Re: xplornet contact or any experience with their satellite service?

2020-04-21 Thread Brandon Martin
On 4/21/20 2:54 PM, Mel Beckman wrote:
> It’s not really oversold bandwidth. It’s just that the turnaround time for a 
> bolus of data is too long for two-way video conferencing to be smooth or 
> reliable. It’s like video conferencing using post cards :)

Are consumer satellite provider networks TDD or FDD?  I guess I had assumed the 
latter, but maybe not?

Assuming FDD, I don't see why the downstream needs to necessarily have problems 
with extremely long user-data-striping periods unless there's some factor I am 
just totally not considering.

The upstream is of course more of a problem.  I assume it's TDMA, and the 
terminals have imperfect clocks.
Brandon Martin

Re: xplornet contact or any experience with their satellite service?

2020-04-21 Thread Brandon Martin
On 4/21/20 2:35 PM, Brian J. Murrell wrote:
> Interesting.  So basically as Mel said, over-sold network.  :-(

This is pretty typical of consumer VSAT and such.  You can of course get better 
performance...if you're willing to pay for it.  If you find the right 
carrier/re-seller, you can perhaps find one whose "system" (to include ground 
station, terminals, and smarts on the bird) can respect DSCP and flag at least 
your voice traffic appropriately (probably EF) to perhaps lower the jitter and 
make conferencing more palatable.  Finding competent folks at a typical 
consumer provider's helpdesk to talk to about such things is probably the 
limiting factor.  The higher up the foodchain you go towards the folks who 
"own" the spectrum rather than re-sell will perhaps get you more luck on 
something like this though probably also at higher MRC.

Unfortunately, it's just what happens when you spread an already limited 
resource (transponder bandwidth) out over essentially an entire continent or at 
least substantial portions of it.  Imagine if you had a cable provider with a 
single node for an entire, say, US state.
Brandon Martin

Re: Constant Abuse Reports / Borderline Spamming from RiskIQ

2020-04-16 Thread Brandon Martin

On 4/15/20 11:33 PM, Ross Tajvar wrote:
Can you give some examples of the things you mention above? I'm not 
doing much in terms of customer filtering and would be interested to 
hear what others consider best practice.

My experience is that there's two groups of customers that are 
problematic from an abuse standpoint:

* Those who intend to abuse your network
* Those who enable others to abuse your network

The former are of course a little easier to detect up front and much, 
much easier to give the axe when they do commit AUP violations.  It 
looks like others have already given some hints as to how to detect 
these kinds of folks up-front.  I'd also recommend looking for 
references for any new customer who wants a very large amount of 
resources, explicitly wants to send email, is bringing their own IP 
space (especially if they are leasing it), etc.

The latter are far more problematic for legitimate operations.  I don't 
really run "hosting" providers as I'm mostly in the business of mid- and 
last-mile networks, but I always try to ask anyone who's either buying a 
plan that explicitly permits "hosting" or who is asking for personal-use 
exemptions to anti-hosting provisions in the AUP (which I do permit) 
what their intent is.  I don't really care so much what they're doing as 
long as they know what they're doing and that I get a vibe from them 
that they are competent.  "I want to host my wordpress blog" is an 
instant red flag since compromised wordpress instances are one of the 
biggest sources of snowshoe hosting in my experience.

Brandon Martin

Re: attribution

2020-04-13 Thread Brandon Martin

On 4/13/20 4:31 PM, Randy Bush wrote:

it seems a lot of folk think prepending acrually works.

I mean, there's prepending and then there's prepending 50+ times...  Has 
the latter EVER been useful in any way, shape, or form?

Brandon Martin

Re: Traffic destined for

2020-04-08 Thread Brandon Martin

On 4/8/20 2:42 PM, Drew Weaver wrote:


I’ve noticed over the past couple of weeks that some hosts on a network 
I manage appear to be trying to reach hosts in this network

It’s an IANA reserved block but I’m really not sure what it’s used for. 
I just notice it keeps coming up but it doesn’t have a route.

Has anyone else been seeing this?

This is part of the RFC6598 space for carrier-NAT deployments.  If 
you're seeing traffic inbound to your network to those addresses, 
someone's presumably got a default toward you and a hole in their 
internal routing table.  If you're seeing traffic outbound toward those 
addresses, some of your customers have somehow picked up a configuration 
expecting some sort of service there.

Inbound traffic toward your network from or to those addresses is 
effectively a bogon.  Outbound traffic from/to those addresses means you 
have a misconfiguration somewhere (presumably unintentional and perhaps 
some poorly behaved automatic config on a CPE).

Brandon Martin

Re: interesting troubleshooting

2020-03-24 Thread Brandon Martin
On 3/20/20 5:57 PM, Jared Mauch wrote:
> It’s the protocol 50 IPSEC VPNs.  They are very sensitive to path changes and 
> reordering as well.

Is there a reason these are so sensitive to re-ordering or path changes?  ESP 
should just encap whatever is underneath it on a packet-by-packet basis and be 
relatively stateless on its own unless folks are super strictly enforcing 
sequence numbering (maybe this is common?).  I can understand that some of the 
underlying protocols in use, especially LAN protocols like SMB/CIFS, might not 
really like re-ordering or public-Internet-like jitter and delay changes, but 
that's going to be the case with any transparent VPN and is one of SMB/CIFS 
many flaws.

For LAGs where both endpoints are on the same gear (either the same box/chassis 
or a multi-chassis virtual setup where both planes are geographically local) 
and all links traverse the same path i.e. the LAG is purely for capacity, I've 
always wondered by round-robin isn't more common.  That will re-order by at 
worst the number of links in the LAG, and if the links are much faster and well 
utilized compared to the sub-flows, I'd expect the re-ordering to be minimal 
even then though I haven't done the math to show it and might be wrong.

I'd argue that any remote access VPN product that can't handle minor packet 
re-ordering is sufficiently flawed as to be useless.  Systems designed for very 
controlled deployment on a long-term point-to-point basis are perhaps excepted, 
Brandon Martin

Re: Hi-Rise Building Fiber Suggestions

2020-02-26 Thread Brandon Martin

On 2/26/20 11:43 AM, Filip Hruska wrote:
Some NICs don't support SM optics, so even if you would like to run SM 
everywhere, it's not necessarily possible depending on the equipment.
For example, I had issues with some SolarFlare cards which happily take 
10G-SR MM but won't take 10G-LR SM.

Is this because they're dumb, or is there some serious reason?  AFAIK, 
the electrical side of the SFP+ is the same for all 10G-R PHYs.

My philosophy has generally been that all fixed infrastructure installed 
in this era might as well be single-mode.  If I'm just dropping a patch 
cord in a raceway or similar, I'll use multi-mode in many cases.  On the 
fixed side, I have enough trouble convincing folks that APC and UPC 
plugs are different let alone trying to explain why you can plug SM 
optics into MMF and it will (generally) work while the other way around 
does not beyond a few meters and, since SMF can't be gotten rid of 
entirely in fixed infrastructure, I'll take the normalization where I 
can get it.

Brandon Martin

Re: Hi-Rise Building Fiber Suggestions

2020-02-26 Thread Brandon Martin

On 2/25/20 10:48 PM, Abhi Devireddy wrote:
L2 rings IMHO seem pretty brittle. I know there are L2 ring products 
like Juniper BTI, which use ERPS and not strictly STP/RSTP to move 
blocking ports, and those seem a little better although it's mostly 
statically configured.

For a strict ring topology like this, I'd certainly consider E-RPS or 
similar over [R]STP if you were going to do this all at L2.  It's a lot 
more "predictable" in my experience insofar as it's harder to shoot 
yourself in the foot by mistuning some knob somewhere and getting 
behavior you don't expect.

That said, I'd be loathe to put 30+ switches on a ring even within the 
same building unless I had little other choice.

One thing that I haven't seen explored in this thread is the idea of 
doing things at L3.  If you've got 4 fibers, you can use Bi-Di optics to 
build a partial mesh/ladder/braid as you go up the building.  That can 
be a mess at L2 with (R)STP, but carving out an IGP area and doing it at 
L3 is often not nearly so ugly.  If you've got L3 switches (which are 
cheap, these days), it may be a good option, though it may also be 
annoying from an IP subnetting POV unless you overlay it with something 
like an IPv4-in-IPv6 core (MAP, 464XLAT, etc.) or an L2-in-IP overlay 
like VXLAN both of which substantially increase the conplexity of the 

Using CWDM or DWDM with 8-16 channels on either 2 or 4 trunks across 
your 4 fibers to do a more conventional home-run with multi-chassis LAG 
or similar is another reasonable option.

I'd avoid stacking 30+ switches even where you have stacking support 
over fiber especially if the switches aren't in a physical stack.  Too 
much opportunity for split-brain scenarios IMO.

I'd contribute to the "see really hard if you can just drop more fiber 
down the riser" echo chamber.

Brandon Martin

Re: QUIC traffic throttled on AT residential

2020-02-19 Thread Brandon Martin

On 2/19/20 2:54 PM, Fred Baker wrote:

The argument I have heard is that residential firewalls often block anything 
that is*not*  UDP or TCP. The question for the googlers was existential - can 
it work at all?

I'm not sure that they "block" it, per se, though some probably do have 
an explicit rule to that effect.  I would think the bigger issue is that 
they don't know how to 1:N NAT arbitrary L4s (and how would they), so 
the absolute best you might get is that the first device behind the NAT 
to establish a mapping sees all the relevant L4 traffic and everybody 
else is locked out.  I'd suspect the normal case is simply that they 
drop it on the floor unless there's a specified "DMZ" host.

Perhaps this is just a semantic difference, but I think it's actually an 
even more difficult issue to resolve.  If it were simply blocked, that's 
usually "easy" (either for the user, via a management interface, or for 
the vendor, via policy template) to fix.  Writing an entirely new L4 NAT 
helper is a different matter entirely.

IPv6 would of course render this moot, but we all know how well IPv6 
traffic gets treated...

Brandon Martin

Re: ATT Microcell in Austin, TX

2020-02-18 Thread Brandon Martin

On 2/18/20 2:25 PM, Matt Hoppes wrote:

Power goes out and the pole mounted nodes go out eventually.

Where "eventually" seems to vary a LOT.  I've observed hold up times as 
long as 8+ hours all the way down to "well, I guess there was a minor 
power glitch at the nearest power injection point because it dropped 
everything for 30 seconds while stuff came back up".  Lots of factors 
seem to play in this both in terms of design and maintenance.  For a 
while, some MSOs took their job seriously as they were offering 
relatively popular business-oriented voice products.  MSOs targeting 
only consumer service and still in the mindset of linear TV (or having 
not touched their plant since that was the major use case) often have no 
battery at all.

The same seems true of many FTTN deployments.  Hold-up time on nodes 
varies a lot.  The older the deployment, the more design hold-up time it 
seems to have, but of course maintenance varies a lot.  Newer 
deployments, especially fiber-to-the-curb often have essentially no 
hold-up at the local node unless it's back powered from the customer 
prem (in which case the customer can keep it up themselves).

Brandon Martin

Re: Why are IPsec SAs unidirectional

2020-02-16 Thread Brandon Martin

On 2/15/20 1:17 PM, Bart Hermans wrote:

Does someone know why these IPsec SAs are unidirectional?

My take on it:

* IP, on which IPSec is directly built, is not a bidirectional protocol. 
 It is unidirection and fire-and-forget.  There's no assumption made 
that the source address specified in a given packet is even reachable 
from the destination address (much to the chagrin of many network 
operators), though it's supposed to be the case that it is.  Making SAs 
bidirectional would therefore represent something of a layering 
inversion which the IP suite has been surprisingly careful to avoid.

* While many protocols built on top of IP, including ISAKMP are 
bidirectional, not all are, so having unidirectional SAs is potentially 
useful especially in the case of e.g. multicast as another poster 
pointed out.

* ISAKMP is not the only way to key IPSec SAs.  It's a fairly complex 
protocol and is separate from the base IPSec specifications.  Someone 
could come up with another, possibly better way to do it.  You can also 
key them manually.  Again, projecting the nature of ISAKMP onto IPSec 
would be a layering violation and might inhibit future use cases of the 

* An IPSec SA itself is quite simple.  Making it unidirectional is 
in-line with that notion and appears to have few consequences.

* An IPSec SPD is also unidirectional (one could argue that this is a 
mistake, but see all the above), and an SA follows directly from an SPD.

Brandon Martin

Re: akamai yesterday - what in the world was that

2020-02-14 Thread Brandon Martin

On 2/14/20 1:24 PM, Andy Ringsmuth wrote:

U, it is 2020, not 2010. 100M, 200M, 400M or 1G is increasingly common for 
home broadband. I’ve got 400M at home, could get 1G fiber for less than $100 if 
I wanted it, and I’m in your average, run-of-the-mill Midwest city.

And there are plenty of places in the USA with literally no wireline 
connectivity aside from POTS who rely on LTE or local WISPs to deliver 
bits, and they're doing well to give them 100M (capped or pay-by-the-GB 
on LTE, typically) and often much less.  There's also plenty of places 
where wireline speeds max out around 50-75Mbps in practice, even if the 
local MSO says you can get more.

The divide keeps getting bigger.
Brandon Martin

Re: akamai yesterday - what in the world was that

2020-02-13 Thread Brandon Martin

On 2/13/20 12:39 PM, Ahmed Borno wrote:
Strictly out of interest, I wanted to ask earlier if this irresponsible 
way of causing insane, instant, bandwidth demands is breaking anything 
on the ISP/CDN side or even the console owner ?! Or is it just an 
interesting phenomenon that is handled without a sweat. Does it break 
the buck in anyway?

A good service provider will plan for things like this.  They do happen 
from time to time and for reasons other than large game updates being 
dropped in the middle of the afternoon.  However, sudden large traffic 
surges can be unexpected, causing network congestion and poor 
performance for customers, and regardless they cost money to plan for.

These game updates have become very visible recently which I suspect is 
why they're getting attention.  They've also been generating traffic 
surges during or near prime time which compounds with typical streaming 
usage and is most likely to generate customer complaints.  If they were 
hitting at 4AM local, hence the discussion about consoles and such 
downloading them automatically at night, it wouldn't be nearly as big of 
a deal since network demand is typically low during those times on 
consumer-facing networks and congestion, if it occurs, is unlikely to 
generate complaint volume.

Brandon Martin

Re: akamai yesterday - what in the world was that

2020-02-12 Thread Brandon Martin

On 2/12/20 11:56 AM, Chris Adams wrote:

I think security is probably the sticking point for this.  Content
owners don't want anybody having direct access to their files, and as
more content is distributed over HTTPS, content distributors don't wany
anbody having access to their certificates.

Yeah, these were the "legal" aspects I was referring to above.  Not a 
technical problem, really.  I can't say I'm surprised, and I can think 
of some workarounds, but it's definitely a thing to consider.

Brandon Martin

Re: akamai yesterday - what in the world was that

2020-02-12 Thread Brandon Martin

On 2/12/20 11:22 AM, Seth Mattinen wrote:
My experience is that they want to see lots of traffic growth to stay 
interested. As companies get bigger the minimum bar to play keeps going 
up, and anyone below that bar is stuck relying on transit. Fall below 
the bar or don't show enough growth fast enough and they pull the 
resources away.

Which makes perfect sense for anything that involves physical 
infrastructure, talking to people, etc.  Resources have to be allocated 

I guess what I'm looking for is a more "standard product".  Load this 
VM, tell it your preference for upstream use vs. hit rate, let it 
announce some routes into your network, and you take what you get.  If 
you need more, presumably you have the volume to back it up.

I know somebody had done something like this for Steam at one point 
using a transparent HTTP proxy.  IDK if it still works, and I don't 
think it was actually supported by Valve.

Maybe I'm smoking something, here...
Brandon Martin

Re: akamai yesterday - what in the world was that

2020-02-12 Thread Brandon Martin

On 2/12/20 10:59 AM, Dave Bell wrote:

Night-time for you is daytime for someone else.

This is very true, though I am curious what the international 
demographics are like for COD in particular and many games in general. 
I suspect a lot of them are at least somewhat regional.

I agree that the folks pushing these massive data loads could be 
considerate of the networks they are running on. The steam preload thing 
is a great example. The content becomes available x weeks before launch 
and downloaded to your device at some point in that window. It then 
simply unlocked on launch day.

This works really, really well from what I can see.  I don't even see 
many major OS updates, new game drops, etc. in my stats.  One of my 
networks is too small and hence noisy to really tell, but the other is 
quite predictable.

I'm not sure how much incentive there is for people to implement this 
kind of system though. With the rise of CDN and global caching, people 
care less about the performance of their servers for content 
distribution as it just scales out.

It would be really nice if the major CDNs had virtual machines small 
network operators with very expensive regional transport costs could 
spin up.  Hit rate would be very low, of course, but the ability to grab 
some of these mass-market huge updates and serve them on the other end 
of the regional transport at essentially no extra cost would be great. 
I'm sure legal arrangements make that difficult, though.

Brandon Martin

Re: akamai yesterday - what in the world was that

2020-02-12 Thread Brandon Martin

On 2/11/20 6:41 PM, Tom Deligiannis wrote:
There is a major update that has released today, how's everything 
looking for everyone?

I run a couple distinct very small networks.  Both are transit-only with 
no direct peering or local caching and generally sub-gbps.

One set a new 1-min 95% record and did so by nearly 25% over its 
previous record.  The other matched its existing record.  The former 
thankfully has ample capacity, while the latter thankfully did so before 

These are definitely going to be harder to manage than the primetime 
hump.  It would be nice if things could drop overnight to hopefully 
spread things out during the daytime lull some.  I understand that some 
people won't pick it up until they get home and turn on devices (which 
is often after school hours for these type of game updates), but 
spreading things out would be really nice especially for those of us 
without local caching.

Brandon Martin

Re: Need NOC/IP admin contact for AS27506/Crown Castle/Sidera

2020-02-06 Thread Brandon Martin

On 2/6/20 8:03 PM, Brandon Martin wrote:
An RADB entry for IP range was recently entered by Crown 
Castle Fiber that appears to be in error.  Please contact me off-list to 
help resolve.  Thank you.

CCF claims it's been taken care of.  Thanks.
Brandon Martin

Need NOC/IP admin contact for AS27506/Crown Castle/Sidera

2020-02-06 Thread Brandon Martin
An RADB entry for IP range was recently entered by Crown 
Castle Fiber that appears to be in error.  Please contact me off-list to 
help resolve.  Thank you.

Brandon Martin

Re: Reminiscing our first internet connections (WAS) Re: akamai yesterday - what in the world was that

2020-01-27 Thread Brandon Martin
On 1/27/20 2:00 PM, Jamie Bowden via NANOG wrote:
> I don't know about the other ILECs out there, but I don't know if Verizon 
> will even provision a T1 anymore.  I know you can still get a PRI (that's how 
> our phone systems interface with the PSTN), but if we needed a CT1 instead, I 
> don't know that they'd be able (willing) to deliver it.  I know you can't get 
> a BRI.  We moved offices a few years ago and we basically lost the ability to 
> use our STEs for anything but voice as we couldn't get BRIs delivered to the 
> new space.

Last I checked (a couple years ago, I guess), you could absolutely still order 
a clear-channel point-to-point T1 in AT territory in the Indianapolis LATA 
including surrounding exchanges.  I don't know if AT themselves would put 
channelized/CAS voice on it, but I'm sure you could find someone they can 
terminate it to who would.  I assume this is the same in many places.  Now, 
presumably they'll deliver it on their existing TDM optical plant if you're in 
an on-net facility or SHDSL if you're not as has been the case for a couple 
decades at least, but it was still something you could order.

I got a quote on a channelized/split T1 with 4 voice channels and the remainder 
used for PPP IP service from Centurylink about a year back, too, for 
out-of-band and alarm use in a (non-Centurylink but about 3 blocks away from 
theirs) CO facility within part of their legacy Sprint/Embarq footprint.  They 
did give me a quote, so I assume they would have delivered it (somehow).  It 
was laughably expensive, but they did offer to do it.  IIRC they actually were 
going to do it as a PRI with 4 voice bearers, the 1 PRI signalling channel, and 
then the remaining 18 channels for the point-to-point data, so perhaps they 
were able to do that since it was a "PRI" product (which may have at least 
partially explained the exorbitant cost).  We ended up just ordering 3 POTS 
lines and  for OOB data.
Brandon Martin

Re: akamai yesterday - what in the world was that

2020-01-23 Thread Brandon Martin

On 1/23/20 11:13 AM, Bryan Holloway wrote:
This echoed events a month or so ago, and I'm curious as to what is 
making these releases more, uh, network-impacting.

My understanding is that, in addition to factors others have mentioned 
(games are larger, more network based delivery, etc.), that there's a 
move AWAY from differential patching, to the extent it was previously 
being used, toward simply delivering an entire new copy of the game, 
including assets that completely duplicate those that someone may 
already have.

Apparently the rationale is that this is easier on the publisher and 
those preparing the release, which allows them to get things out sooner, 
since they don't have to come up with a decent differential patcher and 
can just make use of the delivery mechanisms already present on the 
content platform the user is already using.

When you've got 100GB games with huge market penetration and each 
"patch" is an entirely new copy of said 100GB game, that's a lot of traffic.

Brandon Martin

Re: De-bogonising 2a10::/12

2020-01-10 Thread Brandon Martin

On 1/10/20 2:49 PM, Baldur Norddahl wrote:
The only way for me to send out traffic to bogons is if one my peers 
announces a bogon prefix. Even if I did null route bogons, manually or 
through the use of the Cymru service, a peer could still announce a more 
specific and override that.

The idea isn't necessarily that you explicitly null-route them but 
rather that you block/ignore announcements of them on the assumption 
that malfeasants may be attepmting to squat on them or otherwise use 
them for some form of, well, malfeasance.  As such, the filter you build 
isn't just e.g. "2a10::/12" (if indeed that range was to be considered a 
single bogon) but rather "2a10::/12 ge 12" which means you'd block 
more-specifics within that range, too.

Is there a way to use the RPKI system to ensure bogons are simply 
invalid? Seems much more effective to me. 

Someone like ICANN or IANA could publish an ROA to a reserved ASN (or to 
no ASN - is that possible?) for all unallocated space or something of 
the like, I suppose.

Brandon Martin

Re: Cost Recovery Surcharge & Va Personal Property Tax Recovery for IP Transit

2020-01-09 Thread Brandon Martin

On 1/8/20 11:30 AM, William Herrin wrote:

Did this tax change unpredictably, or am I still safe saying it's
deceitful to add a fee for it on top of the advertised and contracted
service price?

IMO, there's no reason to expect this tax to be directly passed on to an 
end customer just like there's no reason to expect the company's income 
tax to be directly passed on to the end customer.  It's a cost of doing 
business and needs to be included in the up-front negotiated rates.

Sales tax is somewhat special in that regard as it occurs right at the 
point of purchase and is, in some areas at least, REQUIRED to be 
itemized to the customer.

If it's not in the contract, I'd try to get out of it.
Brandon Martin

Re: Getting an ASN in ARIN

2020-01-06 Thread Brandon Martin

On 1/6/20 1:47 PM, William Herrin wrote:
If you're not multihomed, buy a $20 virtual server from Vultr or a 
comparable cloud service, read about the BGP service they provide (BGP 
via two providers is what will make you multihomed), set up a VPN from 
there back to your site and then see multihomed above.

"Distinct routing policy", while a bit less clear-cut than being 
multi-homed, is another valid reason to get an ASN and more flexible if 
you're in a situation where you might be concerned about "strictness". 
It's mostly applicable to ISPs rather than end-users, I would suppose. 
However, it generally suffices to say "I am a network operator and set 
my own routing policies, have my own PI space under those grounds, and 
my single upstream provider prefers that I have a public ASN if I want 
to run BGP with them which I wish to do as part of setting my network's 
routing policies."

As has been pointed out, there's not a shortage of ASNs like there is 
IPv4.  The policy surrounding assignment of ASNs is mostly to make sure 
that people who are getting one actually have a use for it and know what 
they intend to do with it.

Brandon Martin

Re: Third Party OLT Optics

2020-01-06 Thread Brandon Martin

On 1/6/20 11:05 AM, Mike Hammett wrote:
What is the vendor-lock scene like in the world of OLT optics? Is it as 
bad as Ethernet optics? If it exists, is there actually a good reason 
for it (other than money)?

My experience has been that it's arguably worse.  Some sort of vendor 
lock is downright common often with no way to override it.  Of course, 
you can always re-code the optics, but bleh.

The claims usually are that GPON (and PON in general) is more sensitive 
to timing and DDM characteristics of the optic than AE (which is, to a 
degree, true) and that OLT optics launch much higher power than typical 
AE optics (also true) and that therefore it's required that you use 1st 
party optics to maintain FDA laser compliance (which is, afaik, BS).

The way I know most of this (aside from the DDM interface specifics, 
potentially) is BS is that the same vendor(s) generally have 
surprisingly little to say about the passive optics.

I'm sure I'll get people saying that for something so important, one 
should only deploy first-party optics, regardless of the cost. Zhone's 
pricing isn't bad at all    but if there's effectively no 
difference, then you might as well buy the cheaper one.

My solution has been to grill my preferred OLT vendor on their 1st party 
optic cost.  I've generally been able to get them down to the point 
where, while still notably more expensive than white-box offerings, it's 
not patently outrageous, and then I don't get support hassles which is 
worth at least something (even if it's largely artificial) on a port 
with 32+ potential users on it if I do have an apparent issue that may 
be related to the optics.

My one complaint with optics has also been somewhat more variable 
launch power from unit to unit.  Though they are always within spec, 
they seem to vary a bit more than with bigger name modules especially 
within a lot.  That does matter a bit more with PON optics than with AE, 
so that is one thing to consider, too, though it's a minor complaint 
again since everything is always within spec.

Brandon Martin

Re: Cost Recovery Surcharge & Va Personal Property Tax Recovery for IP Transit

2020-01-06 Thread Brandon Martin

On 1/6/20 9:39 AM, Siyuan Miao wrote:

- Va Personal Property Tax Recovery (1.8%)
- Cost Recovery Surcharge (3%)?

These sound like your typical BS "authorized fees" many carriers at all 
levels love to levy.  They're basically attempting to itemize various 
regulatory (i.e. regulations they have to comply with just like any 
business would) overhead and pass it on to you directly.  Cable 
companies and incumbent telecoms love to do this since it's a way to 
make more money without raising base rates that they actually have to 
quote to people.

They were probably described deep in the bowels of your contract.  If might be able to get out of them.

Brandon Martin

Re: 5G roadblock: labor

2019-12-30 Thread Brandon Martin
On 12/30/19 6:31 PM, Ca By wrote:
> is is still a physics thing. Most purest will says 5G = new radio (NR). NR 
> can run in any band. And, the distance is a function of the band. Tmobile is 
> big on 600mhz NR, Sprint is big on 2500mhz NR and VZW has 28ghz NR. 

Is/are there defined standard(s) for this NR?  I was of the impression that we 
were basically just tweaking the LTE-A MAC and OFDM[A] PHY for wider bandwidth 
(mid-band and mmwave) or newly available low-band (600MHz) spectrum 
deployments.  AFAIK, there's no new PHY tricks going on for "5G" that aren't 
already being used for "4G" LTE-A deployments on existing mainstream spectrum.  
Aside from the new bands, what's the burden on the handset vs. base?  A lot of 
tricks (beamforming, multi-site MIMO, etc.) can be done without real changes to 
the handset, and we can thankfully mostly upgrade the OFDM PHY to support e.g. 
denser modulation constellations in a somewhat incremental manner.

I already see lots of outdoor small cells on fiber in dense retail, academic, 
business, etc. areas even in suburbs.  I assume they're quite common in urban 
areas, though I don't get to do much work in those environments.  Deployment of 
these started well before any 5G hype I was aware of even on the network 
operator side (think 6+ years ago).

All that is to say, what's the magic secret sauce that makes "5G" any real 
different from "modern 4G"?  I really don't want to go diving down the 3GPP 
document hole...
Brandon Martin

Re: 5G roadblock: labor

2019-12-30 Thread Brandon Martin

On 12/30/19 5:42 PM, Michael Thomas wrote:
Oh, I didn't know that. Seems like it's a relatively new thing. Seems 
like they went to a lot of trouble to essentially do what voip does. Or 
maybe not? I've been poking around trying figure out what's going on 
under the hood with wifi calling, and it seems like they're just 
tunneling PSTN bits over the internet. If true, that's certainly a quick 
and dirty hack. Maybe they're doing something similar for volte?

My understanding is that VoLTE is signaled using SIP.  I don't know how 
the media moves.  I think they tried to avoid re-inventing the wheel. 
Most of the "phone" guys are slinging a lot of inter-network calls via 
IP these days, anyway.

Mostly what I want in the future is a dollop of EF QoS bits and let me 
determine how to use them...

Believe it or not, several of the major wireless and even wireline 
carriers seem to do this to some degree, though my evidence is 
anecdotal.  They don't seem to drop when you exceed your dollop, though, 
but rather re-mark.

Brandon Martin

Re: 5G roadblock: labor

2019-12-30 Thread Brandon Martin

On 12/30/19 3:24 PM, Matthew Petach wrote:

If we solve the issue of endpoint identity on a connection
independent of the transport, so that your video stream
of the game doesn't have to stop and restart every time
you shift from one access point to the next, I could
definitely see wi-Fi beating 5G.

Otherwise, I think 5G will win, in terms of better
user experience when non-stationary.

In theory, this is what "Hotspot 2.0" is designed to solve.  You 
authenticate to the ESSID using your mobile carrier credentials, and the 
resulting connection backhauls over an Internet tunnel to your carrier 
who can handle the hand-off/roaming transparently for you.

It also includes some provisions for settlement so that Wi-Fi operator 
can get a kickback from the mobile carrier for offloading their traffic. 
 I'm sure that will be lucrative for mom and pop coffee shops...

Of course, this is also what Mobile IP was intended to solve, and we all 
know how widely that's deployed.

Brandon Martin

Re: 5G roadblock: labor

2019-12-30 Thread Brandon Martin

On 12/30/19 4:14 PM, Michael Thomas wrote:
The latency argument is what interests me. Supposedly 4G's latency and 
jitter are tough on voip. If that improves there is just no reason for 
TDM to phones which is a significant development because cell phones are 
probably the largest deployment of old style PSTN stuff these days as 
landlines wither and die. I would think that carriers would embrace that 
since it would be a cost-down, but I'm sure I'm wrong since that would 
be admit defeat to IP.

VoLTE is already essentially VoIP, including packet switched media, with 
some MAC layer QoS guarantees as I understand it.  Now, maybe those MAC 
layer guarantees essentially amount to a dedicated OFDMA sub-carrier 
during a voice call.  That I cannot speak to as I'm not intimately 
familiar with the LTE/LTE-A air interface.

I can say that plain ol' best-effort LTE data services are generally 
sufficient for VoIP in my experience if you have "good coverage".  That 
means what I'd generally consider "toll-grade" quality in terms of 
latency and, more inmportantly, jitter.  SSH is similarly quite usable 
generally.  If you're on the fringe of a cell or have a cell that's 
overloaded, YMMV.

Brandon Martin

Re: FCC proposes $10 Million fine for spoofed robocalls

2019-12-19 Thread Brandon Martin

On 12/19/19 12:09 PM, Andreas Ott wrote:

I have also been told that there is no equivalent of uRPF in the phone world.

This is the biggest issue, and unfortunately (and my knowledge of the 
PSTN is admittedly a bit lacking, here), there's likely no good way to 
add it.

Calls on the PSTN are routed essentially based on "who do I feel like 
handing this off to, today", and then that entity may do the same, and 
so on.  It's pretty routine for an outfit to have multiple contracts for 
termination that may not even be aware of the "legitimate" numbers from 
which their customers might "source" a call.

Further, it's entirely normal and perfectly legitimate (to varying 
degrees) for an outfit to purport in CID a number that is not directly 
assigned to them nor which will actually result in a callback being 
routed to them.

Think of caller ID more like reverse DNS.  It's largely advisory and, 
outside some situations where you deliberately want a higher degree of 
repuatation/identity verification and are willing to accept a 
potentially large number of false flags, there's no real reason to rely 
on it outside of human nicety.

The rough analogy to the source IP address is the ANI information that's 
not even passed to most end users.  That's "who should I bill this to?". 
 But even that can get overwritten sometimes during call routing, from 
what I gather.  It's also rarely a valid callback number for any 
non-trivial call source.  Or, at least, if you did call it, the person 
who (might) answer the phone will have no idea what prompted you to do so.

SHAKEN/STIR, the leading proposal to "fix" this, is more like RPKI in a 
way albeit very much re-envisioned based on circuit switching rather 
than packet switching.  Each intervening network can attest to what 
degree they are able to verify the CID (and maybe ANI?) information in 
the call.  Unfortunately, a perfectly valid attestation is "I cannot 
verify it", and indeed that's likely to be most of the attestations 
you'll see at least at first.  The best it really lets you do is figure 
out some networks at which to point fingers.

When "full attestation" is present, i.e. the network operator has been 
able to verify that the CID field represents a number authorized for use 
by the entity originating the call, it's maybe more like DKIM in that 
you can, with cryptographic certainty, know THE network at which to 
point fingers as they're the ones who admitted the call into the PSTN 
with authority that the CID field (among others) is "valid".

[And all the old PSTN folks will please forgive me if I'm inaccurate, 
here, though corrections are welcome]

Brandon Martin

Re: Fwd: urgent opening: Engineer-Transport - III

2019-12-18 Thread Brandon Martin

On 12/18/19 8:49 PM, Eric Kuhnke wrote:
The really scary and not uncommon thing now is for unethical recruiters 
to take your CV from somewhere, copy/paste it into their own word 
processing software, and start editing things in it (and removing your 
direct contact information) without permission from yourself, and send 
it onwards to their "clients".

Have seen this happen to at least five people I know.

And this is why I always tell people to:

1) Use a PDF (it makes it harder, they at least have to copy/paste text 
and do some re-formatting).

2) Ship or attach a PGP or other crypto signature (not that anybody 
verifies them).  I personally have been known to mention that the resume 
should be signed/have a signature available in a quick quip where I also 
mention experience with basic end-application crypto systems in the 
resume.  Recruiters often fail to remove that reference when they modify it.

3) Offer to "send a fresh copy" of the resume at the earliest 
opportunity in engaging a hiring manager.

4) Always bring multiple hard-copies as well as the PDF on portable 
media to interviews.

The hiring managers hate the recruiter-modified BS resumes as much as 
the interviewees do.  This goes double when it's been modified to the 
point that the qualifications stated bear little to no resemblance to 
the original ones stated.  That just wastes everyone's time.

Brandon Martin

Re: DDoS attack

2019-12-09 Thread Brandon Martin
On 12/9/19 3:32 PM, Florian Brandstetter via NANOG wrote:
> In any regard, <1 Gbps is pretty piss poor for an amplification attack too.

But, as others have pointed out, plenty to knock a single subscriber, shared 
access link (DOCSIS, wireless, or even well loaded GPON), or even a small 
regional PoP down.  Plenty of opportunity for mayhem even with just a couple 
100Mbps which is trivial to come up with these days as the spread of 
consumer-accessible speeds keeps growing.  Keeping it small makes it less 
likely to get noticed and, perhaps even more importantly for the perpetrator, 
harder for the networks responsible for the reflection/amplification to track 
down the problem using traffic analysis as well as coming in on the lower end 
of the "how much do I care?" part of the abuse team's line-up.
Brandon Martin

Re: Elephant in the room - Akamai

2019-12-08 Thread Brandon Martin

On 12/8/19 10:58 AM, Jared Mauch wrote:

Not all content is suitable in all locations based on the physical security or 
market situation. We have some content that can not be served, an example is 
locations where there are licensing requirements (eg: ICP for China).

You will see a different mix from our 20940 vs 16625 as well. Those have 
different requirements on the security side. If you treat your PKI data 
seriously you will appreciate what is done here.

In Marquette Michigan there will be different opportunities compared with 
Amsterdam or Ashburn as well.

Our customers and traffic mix makes it challenging to serve from a platform 
where you do capital planning for several year depreciation cycle. We have 
thousands of unique sites and that scale is quite different from serving on a 
few distinct IXPs and transit providers.

So yes you will see a difference and there are things we can do to improve it 
when there is a variance in the behavior.
I guess what I'm getting at is that it sounds like, if you cannot source 
the content locally to the peering link, there's not likely to be an 
internal connection to the same site from somewhere else within the 
Akamai network to deliver that content and, instead, the target network 
should expect it to come in over the "public Internet" via some other 
connection.  Is that accurate?

Thanks for the clarifications.
Brandon Martin

Re: Elephant in the room - Akamai

2019-12-08 Thread Brandon Martin

On 12/7/19 7:19 PM, Jared Mauch wrote:

Please see my email on Friday where I outlined a few of the dynamics at play.  
Akamai isn’t just one thing, it’s an entire basket of products that all have 
their own resulting behaviors.  This is why even though you may peer with us 
directly you may not see 100% of the traffic from that interconnection.  (Take 
SSL for example, it’s often not served via the clusters in an ISP due to the 
security requirements we place on those racks, and this is something we treat 
very seriously!)

Does this mean that, if you peer with Akamai at some location, only 
content locally available at that location will come over that peering 
session with the rest coming via other means?  Does Akamai not have 
private connectivity to their public peering points?

Brandon Martin

Re: RIPE our of IPv4

2019-12-03 Thread Brandon Martin

On 12/3/19 10:04 AM, Fernando Gont wrote:

Wwll, yeah.. you don't need IPv4 addresses if you are going to be using
somebody else's networks and services. Not that you should, though

OTOH, many many organizations, especially outside of service providers, 
in fact DO such a thing.  I'd suspect your average mid-size business 
these days really in fact does not "need" any IPv4 addresses to conduct 
their ordinary and even many extraordinary operations.

As long as you can make IPv4 HTTP/HTTPS destinations work to handle the 
long tail of non-IPv6 web destinations out there, I bet most people 
wouldn't even notice, and the only reason the IT folks would notice 
would be during testing/troubleshooting or the fact that their machine 
suspiciously has no RFC1918 nor public IPv4 address configured on it.

Most organizations do indeed outsource most of their IT functions in one 
way or another, and it's pretty easy these days to pick outsourcing 
partners for most common business needs that are indeed natively 
IPv6-enabled.  The remainder probably run over HTTP/HTTPS, anyway, and 
are easily translatable at the service provider level.

I'd certainly not (yet) say that that's a recommended configuration, but 
I suspect it would often work.  I certainly have IPv6-only testbeds. 
There's a few groaners usually, but a surprisingly large amount of stuff 
"just works".

Brandon Martin

Re: RIPE our of IPv4

2019-12-01 Thread Brandon Martin

On 12/1/19 8:56 PM, Mark Andrews wrote:

End Users
End users receive IP addresses for use in their internal networks only, and not 
for distribution to external users of their Internet services.

I guess it's possible that these networks would be considered end users, 
but I get the impression that they would probably be classified as ISPs, 
and then the fee would indeed be $500/yr for 2X-Small.  A bit ridiculous 
for IPv6-only, but still probably approaching noise in the budget for a 
service provider who has a legacy allocation unless they have remained 
tiny somehow (in which case, sell off some of that IP space you have and 
pay your $500/yr for the next decade or so).

FWIW, if you need it, you should also be immediately eligible for a /24 
for IPv6 deployment and transition tech at no additional cost since your 
legacy space wouldn't be considered by ARIN unless you specifically 
brought it under their purview AFAIK.  Even if you don't need the space, 
per se, there are often times where it's useful to have a disjoint /24 
e.g. for traffic engineering, anycast, DNS servers, etc.  All depends on 
how much legacy space you have, I guess.  I'm also somewhat hopeful 
that, as those allocations all come from a known block, the various 
content networks will recognize them as being likely to house the 
inevitable (eventually) CGN sources, but I won't hold my breath.

I guess you also get to vote in ARIN elections and comment on policy 
matters as a member, if that matters to you.

Brandon Martin

Re: RIPE our of IPv4

2019-11-30 Thread Brandon Martin
On 11/30/19 8:55 PM, Valdis Klētnieks wrote:
>> User apps prefer IPv6, Netflix stops, users complain
> And fallback to IPv4 fails to happen, why, exactly?

Inability to signal application-level failure on IPv6 and that fallback to IPv4 
would succeed.

Netflix definitely exhibits this.  I've also noticed that a lot of 
Cloudflare-hosted apps/sites block AS6939 outright which mostly only affects 
IPv6 as they don't actually originate much end-user-facing IPv4 space.  The 
result is an HTTP/403 which most browsers do not interpret as a response that 
could be remedied by "happy eyeballs" type behavior.  Whether banning an entire 
ASN like that in precisely a situation where this kind of thing is likely to 
occur is a good practice or not is left as an exercise to the reader.
Brandon Martin

Re: RIPE our of IPv4

2019-11-30 Thread Brandon Martin
On 11/30/19 12:18 PM, Justin Streiner wrote:
> Verizon is an interesting case.  While IPv6 penetration on the wireless side 
> is very high, the same is not true on the Fios/DSL side.  IPv6 deployment 
> there is nearly nonexistent.
> I've heard rumblings that some early Fios users will need to have their ONTs 
> replaced to be able to get v6, regardless if the router itself supports it.  
> I don't know if this is true, because I've never been able to get a 
> straight/authoritative answer from Verizon.

Does Verizon still own/manage ANY of their Fios territories?  I thought it was 
all sold off to Frontier at this point.  It certainly all is, along with all 
their legacy LEC territories not having FTTx and having some form of DSL, 
around here.  The latter DSL offerings are usually pretty hilarious.  It's 
mostly run out of the CO or existing RTUs from the POTS era.  I can't imagine 
they have a ton of subscribers outside of areas with literally no other 
wireline options, and I'm guessing a lot of that infrastructure hasn't been 
touched in a decade-plus.

Is Frontier-managed Fios included in those numbers regardless, or are they 
separate like I'd expect them to be?
Brandon Martin

Re: RIPE our of IPv4

2019-11-30 Thread Brandon Martin
On 11/30/19 4:48 PM, Matthew Kaufman wrote:
> See previous message about legacy IPv4 holders without budget for IPv6 blocks 

How slim are your margins to have been around long enough to have a legacy IPv4 
block but not be able to afford the ARIN fees to get a comparable/very usable 
(/48 to /52 for each IPv4) amount of IPv6?  And if you don't need a 
"comparable" amount of IPv6, presumably you aren't using all your legacy IPv4 
and can sell off part of its presumably huge allocation to get some funds.
Brandon Martin

Re: RIPE our of IPv4

2019-11-29 Thread Brandon Martin

On 11/29/19 11:29 AM, Brian Knight wrote:

0% of my IPv4-only customers have opened tickets saying they cannot reach some 
service that is only IPv6 accessible. So if they do care about IPv6 
connectivity, they haven’t communicated that to us.

I help admin a very small (<1k subs, but growing) municipal ISP.  We 
have had a couple requests from residential subscribers for IPv6 which 
is not yet enabled due to lack of support for a nice, but not strictly 
required, feature from one vendor in our stack (and they have opened a 
feature request and promised support in the next release - we'll see if 
they deliver).  If a SP with <1k subs has ANY registered interest, I'd 
say a non-trivial SP is going to have some even if it's not being 

Certainly, to me, if I have a choice of service providers, and one 
offers native IPv6 (or even well-supported 6rd) while the other offers 
no IPv6 and has no plans or is even downright IPv6-hostile (e.g. mucking 
with protocol 41), that's certainly going to factor into my decision. 
This is a pre-sales issue which, on standard consumer services, is 
unlikely to ever be communicated to the provider.  However, I'll admit 
that I'm not a normal customer.

Interestingly, albeit somewhat unsurprisingly, interest from enterprises 
on DIA services (where the aforementioned feature is irrelevant) has 
been not just nonexistent but outright hostile.  Every one of them has 
asked to have it explicitly disabled when prompted.  Enterprise is 
definitely the lagging factor, here.  I suspect it's at least partially 
because high-ratio NAT44 has been the norm for enterprise deployments 
for some time, and, among those who might otherwise be willing to 
support first-class dual stack, many enterprise IT folks lack the 
education to recognize the nuance between public addressing and 
unfiltered public reachability of a given host.  I suspect many of them 
are already using IPv6 for LAN traffic without even realizing it given 
Windows' penchant for doing so since Vista.

Brandon Martin

Re: RIPE our of IPv4

2019-11-25 Thread Brandon Martin

On 11/26/19 4:36 AM, Doug Barton wrote:
I get that some people still don't like it, but the answer is IPv6. Or, 
folks can keep playing NAT games, etc. But one wonders at what point 
rolling out IPv6 costs less than all the fun you get with [CG]NAT.

If it weren't for the ongoing need to continue to support IPv4 
reachability (i.e. if we'd flag-day'd several years ago), I think the 
(admittedly non-scientific) answer to that question is that we have 
already passed it.

However, in the face of continuing need for IPv4 reachability, I'm less 
sure.  I think that the incremental cost to deploy and support IPv6 is 
probably no more than the incremental savings of CGNAT headaches for 
service providers caused by offloading what traffic you can to native 
IPv6.  Those savings from not just from capacity savings (which can be 
extreme to totally trivial depending on your size) but also support for 
having 3rd party services properly treat an SP customer as an individual 
customer rather than the results of multiple SP customers being lumped 
onto a small CGNAT target pool.

That is, even if you are 100% committed to needing to run a functional 
CGNAT as a service provider and deal with everything that entails, I 
think it's probably STILL in your short-term economic best interest to 
deploy IPv6 simply due to the reduction in scope of "everything that 

Brandon Martin

Re: Hulu thinks all my IP addresses are "business class", how to reach them?

2019-11-20 Thread Brandon Martin

On 11/20/19 3:31 PM, Owen DeLong wrote:
As an ISP, there might be something there, but, you’d have to prove that 
you had a significant number of customers that left for that specific 
reason and you’d have to show the actual damages that resulted. Easy to 
estimate, very hard to prove.

Not only hard to prove, but the armchair lawyer in my has an inkling 
that you'd have to show that they did it intentionally or went beyond 
being dumb or knowledgeable about it and were somehow negligent.  The 
former seems even more difficult than proving actual damages, and the 
latter seems like it may not even apply or be possible.

What irks me most about these situations as an operator, and indeed 
something that may push back on my previous statement of intent or 
negligence not being possible/applicable, is that the services often 
make their geofencing/IP classification system failures out as being the 
fault of the user's telecommunications service provider when, in fact, 
the user's service provider often has absolutely no direct control over 
what happens and, even where they do have some form of direct control 
such as through a documented operations-appeals channel, are still at 
the mercy of the service doing the fencing/classification to correct the 
error.  At minimum, this could damage customer good will toward their 
service provider.

(And kudos where it's due to the providers who do NOT make such issues 
appear to be the fault of the user's telecommunications service provider 
and instead provide a real, useful means for the user to directly 
contact the content provider to resolve the issue)

Brandon Martin

  1   2   3   >