Re: Q: is RFC3531 still applicable?

[Adam Thompson]

Understood, yes, but I should have been more clear: I'm talking about 
statically allocating my own internal /64s out of the /56 I've reserved for my 
org's own use.  Is there any point in using a more complex scheme than just 
"next!" ?
-Adam

Get Outlook for Android<https://aka.ms/AAb9ysg>

From: Nicolas VUILLERMET 
Sent: Wednesday, May 15, 2024 7:31:31 AM
To: Mel Beckman ; Adam Thompson 
Cc: nanog 
Subject: Re: Q: is RFC3531 still applicable?


Hello,

The minimum addressable on a LAN is a /64. So you have to provide the customer 
with a larger subnet.

Public operators in France generally deliver a /60.

The RFC gives /56, however, as customers are mobile and there is a risk of 
disaggregating into PAs (or rather allowing the customer to keep his IPs, such 
as DID portability), we, as operators, supply /48s directly.

Talking about the number of IPs that can be assigned in IPv6 shows a lack of 
understanding of IPv6. It's time to get trained!

My 2 cents,

Nicolas VUILLERMET
Network Engineer... and IPv6 ready.

On 14/05/2024 22:12, Mel Beckman wrote:
I never could understand the motivation behind RFC3531. Just assign /64s. A 
single /64 subnet has 18,446,744,073,709,551,616  host addresses.  It is 
enough. Period.


 -mel

On May 14, 2024, at 12:54 PM, Adam Thompson 
<mailto:athomp...@merlin.mb.ca> wrote:



Not an IPv6 newbie by any stretch, but we still aren’t doing it “at scale” and 
some of you are, so…



For a very small & dense (on 128-bit scales, anyway) network, is RFC3531 still 
the last word in IPv6 allocation strategies?



Right now, we’re just approaching it as “pick the next /64 in the range”, as it 
all gets aggregated at the BGP border anyway, and internally if I really try 
hard, I might get to 200 subnets someday.



Is there any justification for the labour in doing something more complex like 
center-allocation in my situation?  Worrying about allocation strategies seems 
appropriate to me if you have 100,000 subnets, not 100.



Opinions wanted, please.

-Adam



Adam Thompson

Consultant, Infrastructure Services

MERLIN

100 - 135 Innovation Drive

Winnipeg, MB R3T 6A8

(204) 977-6824 or 1-800-430-6404 (MB only)

https://www.merlin.mb.ca<https://www.merlin.mb.ca/>

Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

RE: Q: is RFC3531 still applicable?

[Adam Thompson]

I may have mis-read it (I admit I didn’t read it all that carefully) but I 
think RFC3531 is talking about the strategy for assigning /64s out of a larger 
pool (a /56, say).
-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

From: Mel Beckman 
Sent: Tuesday, May 14, 2024 3:13 PM
To: Adam Thompson 
Cc: nanog 
Subject: Re: Q: is RFC3531 still applicable?

I never could understand the motivation behind RFC3531. Just assign /64s. A 
single /64 subnet has 18,446,744,073,709,551,616  host addresses.  It is 
enough. Period.




 -mel


On May 14, 2024, at 12:54 PM, Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:

Not an IPv6 newbie by any stretch, but we still aren’t doing it “at scale” and 
some of you are, so…

For a very small & dense (on 128-bit scales, anyway) network, is RFC3531 still 
the last word in IPv6 allocation strategies?

Right now, we’re just approaching it as “pick the next /64 in the range”, as it 
all gets aggregated at the BGP border anyway, and internally if I really try 
hard, I might get to 200 subnets someday.

Is there any justification for the labour in doing something more complex like 
center-allocation in my situation?  Worrying about allocation strategies seems 
appropriate to me if you have 100,000 subnets, not 100.

Opinions wanted, please.
-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

Q: is RFC3531 still applicable?

[Adam Thompson]

Not an IPv6 newbie by any stretch, but we still aren't doing it "at scale" and 
some of you are, so...

For a very small & dense (on 128-bit scales, anyway) network, is RFC3531 still 
the last word in IPv6 allocation strategies?

Right now, we're just approaching it as "pick the next /64 in the range", as it 
all gets aggregated at the BGP border anyway, and internally if I really try 
hard, I might get to 200 subnets someday.

Is there any justification for the labour in doing something more complex like 
center-allocation in my situation?  Worrying about allocation strategies seems 
appropriate to me if you have 100,000 subnets, not 100.

Opinions wanted, please.
-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

RE: TFTP over anycast

[Adam Thompson]

Others have addressed some of the issues, but one easy win for DHCP (which is 
otherwise a PITA to make redundany in *any* way) is to (a) not block ICMP 
anywhere, including on the client devices, and (b) have the DHCP ping before 
assignment.  That’s not always on by default, and it’ll eliminate ~90% of the 
conflicts you would otherwise encounter if the anycast node isn’t extremely 
stable.  If you become aware of a distributed DHCP server that actually works 
well in this environment, that’s worth a post to the list all by itself.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[cid:image001.png@01DA663E.AD5957D0]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01DA663E.AD5957D0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>
[https://res-h3.public.cdn.office.net/assets/bookwithme/misc/CalendarPerson20px.png]<https://outlook.office.com/bookwithme/user/a3e87142ccc14503bf5ec7ea59afd...@merlin.mb.ca?anonymous=bwmEmailSignature>
Book time to meet with 
me<https://outlook.office.com/bookwithme/user/a3e87142ccc14503bf5ec7ea59afd...@merlin.mb.ca?anonymous=bwmEmailSignature>


From: NANOG  On Behalf Of 
Javier Gutierrez
Sent: Thursday, February 22, 2024 12:47 PM
To: nanog@nanog.org
Subject: TFTP over anycast

Hi,
I'm working on some DR design and we want to not only have this site as a DR 
but also performing some active/active for some of the services we hosts and I 
was wondering if someone had some experience with using anycast for TFTP or 
DHCP services?
What are some of the pains/challenges you experienced and things we should 
lookout for?

Any input is greatly appreciated.


Kind regards,



Javier Gutierrez

Congestion/latency-aware routing for MPLS?

[Adam Thompson]

Using a mix of Juniper hardware...

Network provides VPLS to customer, over MPLS (obviously) in a 
dual-redundant-ring radio topology.  Each site is connected to one or more 
neighbors, generally with two radios, in two different bands, to *each* 
neighbor.  So an ordinary node might have 4 radios, 2 pointing in each 
direction.

Every single radio link has different bandwidth, different latency, and 
different interference characteristics.

These radio links do run at 100% capacity at least some of the time.

It's possible to set each link's relative cost in OSPF or IS-IS, of course, but 
I haven't found a way to make the router react to latency changes on one link 
or the other.  (Right now, I think costs are set equal so traffic will use both 
links.)  This means interference in one band invisibly diminishes the Ethernet 
bandwidth available and silently increases the latency on that link, sometimes 
dramatically.  This seems to do interestingly unpleasant things to the client's 
flows.

It's generally true that one band will be much more severely affected than the 
other, in any interference event.  Before anyone asks, I'm told the network is 
a mixture of licensed and unlicensed bands, that's not changing anytime soon.

In a perfect world, I'd like the routers to dynamically adjust traffic balance, 
but even just temporarily halting use of the impaired link would be helpful (or 
so I believe right now, at least).

Is this a pipe dream?  I'm not seeing anything in JunOS that could accomplish 
this...  I'm not even sure if a mesh protocol could handle dual active links 
like this?

Ideas, comments, etc. all appreciated.

Also, I'm not the direct operator of use network. I'm involved, but mostly just 
trying to help them find better solutions.  Nor am I an MPLS expert, as is 
obvious here.

Thanks,
-Adam


Adam Thompson

Consultant, Infrastructure Services

MERLIN

100 - 135 Innovation Drive

Winnipeg, MB R3T 6A8

(204) 977-6824 or 1-800-430-6404 (MB only)

https://www.merlin.mb.ca<https://www.merlin.mb.ca/>

Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

Re: FastNetMon Usage in the wild

[Adam Thompson]

Sorry for the late reply... Sightline *Insight* is the piece the sales team 
won't sell me, and TAC won't support me, for deployment in our private-cloud 
environment: it has to be hosted on one of 3 canned server configurations.

I am using Sightline/TMS virtually and it's fine there.

-Adam



Adam Thompson

Consultant, Infrastructure Services

MERLIN

100 - 135 Innovation Drive

Winnipeg, MB R3T 6A8

(204) 977-6824 or 1-800-430-6404 (MB only)

https://www.merlin.mb.ca<https://www.merlin.mb.ca/>

Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>


From: NANOG  on behalf of 
Dobbins, Roland via NANOG 
Sent: Tuesday, October 10, 2023 9:34:21 PM
To: nanog@nanog.org 
Subject: Re: FastNetMon Usage in the wild


On 11 Oct 2023, at 01:50, Adam Thompson  wrote:

you need to buy a moderately-expensive hardware server (they don’t let you 
virtualize it)

To clarify, Sightline has supported virtualization for many years, FYI.

 I’m not aware of any anti-DDoS products at ISP scale that aren’t SFlow + 
Flowspec, possibly including “scrubbing” (diverter box);

 I don’t know if it’s an in-band appliance, or a “scrubber”-on-a-stick

In addition to flow telemetry, D/RTBH, S/RTBH, and flowspec, Sightline/TMS 
supports intelligent DDoS mitigation directly in-line or via 
diversion/reinjection.

[Full disclosure:  I am an employee of NETSCOUT.]

RE: FastNetMon Usage in the wild

[Adam Thompson]

We use Arbor’s Sightline in an SFlow + Flowspec topology.  It… works.  It needs 
a lot of tuning.  It’s moderately expensive to deploy in this topology, unlike 
in-band which is holy-cow-expensive at our speeds.  If you want 
historical/forensic data, you need to buy a moderately-expensive hardware 
server (they don’t let you virtualize it) for their Insight module.  Arbor’s 
tech support is Quite Good Indeed, and their SE team is FANTASTIC.  Sales, 
however, not so much.  We don’t feel Sightline is doing all that much for us, 
but we also aren’t able to put the required amount of daily care and feeding 
into it that it needs, so YMMV.

My overall impression is that all the on-prem anti-DDOS products out there do 
the same thing, and work much the same way – thresholds, hopefully with 
auto-baselining.  The differentiating factors IMHO are whether the 
auto-baselining can take time-of-day, day-of-week, and month into account (e.g. 
business day, K-12 school year, etc.); we believe Sightline’s auto-baselining 
doesn’t do a great job here.  Beyond that, any product that uses an evolving 
statistical model (probably branded as “AI”, sigh) will have a slightly better 
chance of improving the successful hit ratio.

I’m not aware of any anti-DDoS products at ISP scale that aren’t SFlow + 
Flowspec, possibly including “scrubbing” (diverter box); having said that, I do 
know one of my upstreams has a large Sightline h/w appliance of some sort, I 
don’t know if it’s an in-band appliance, or a “scrubber”-on-a-stick, but it’s 
too expensive for them to upgrade and they’re apparently dropping it instead… 
once we stop telling them quite so loudly to NOT get rid of it , I guess??

AFAIK, FastNetMon is basically the same thing as Sightline, with a 
less-polished UI. (read: doesn’t make mgmt. as happy to look at it) and you 
need some external support bits to do the Flowspec.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[cid:image001.png@01D9FB80.2D14BDE0]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D9FB80.2D14BDE0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>


From: NANOG  On Behalf Of 
Javier Gutierrez
Sent: Friday, October 6, 2023 5:20 PM
To: nanog@nanog.org
Subject: FastNetMon Usage in the wild

Hi,
I wanted to drop a quick question as I would like to evaluate the FastNetMon 
solution to do DDoS protection and wanted to see what other companies are using 
it out there so I can have a base of how much should I recommend this.

Thanks in advance for your responses


Kind regards,


Javier Gutierrez,

RE: 10G CPE w/VXLAN - vendors?

[Adam Thompson]

The redundant links to the customer site that traverse independent underlay 
carriers, and in some cases, equal-cost paths that we want to load-balance 
across, are the hard part.  I’m not going to trust STP for that, and we aim for 
<3sec failover where we do have redundant paths.  ERPS can handle the failover, 
but not the load-balancing.  Any L2-over-L3 encapsulation protocol can handle 
the failover + ECMP features, but I need to do it at ~10G (~20G if ECMP) wire 
speed.

We provide IaaS services to our customers, which is why we’re stretching VLANs 
to them in the first place.  Viewed from the IaaS perspective, this is a bunch 
of DC-DC connections… but relative to the overall network, the customer-prem 
devices fall into the traditional “CPE” category.  (Most customers either just 
plug in bare fiber, or they connect to an intermediate carrier’s CPE.)

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN logo]]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D99ECB.C1CEDE50]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

From: Joe Freeman 
Sent: Wednesday, June 14, 2023 2:16 PM
To: Adam Thompson ; nanog 
Subject: Re: 10G CPE w/VXLAN - vendors?

I think you’re probably overthinking this a bit.

Why do you need to extend your vxlan/evpn to the customer premise? There are a 
number of 1G/10G even 100G CPE demarc devices out there that push/pop tags, 
even q-in-q, or 802.1ad. Assuming you have some type of aggregation node you 
bring these back to, tie those tags to the appropriate EVPN instance at the 
aggregation point. Don’t extend anything but a management tag and an S-tag 
essentially to the device at the customer premise.

You can even put that management tagged vlan in it’s own L3 segment, or a 
larger L3 network and impose security. This way you’re not exposing your whole 
service infrastructure to a bad actor that might unplug your cpe device and 
plug into your network directly.



From: NANOG 
mailto:nanog-bounces+joe=netbyjoe@nanog.org>>
 on behalf of Adam Thompson 
mailto:athomp...@merlin.mb.ca>>
Date: Wednesday, June 14, 2023 at 2:52 PM
To: nanog mailto:nanog@nanog.org>>
Subject: 10G CPE w/VXLAN - vendors?
Hello, all.
I’m having difficulty finding vendors, never mind products, that fit my need.

We have a small but growing number of L2 (bridged) customers that have diverse 
fiber paths available, and, naturally, want to make use of them.
We have a solution for this: we extend the edge of our EVPN VXLAN fabric right 
to the customer premise.  The customer-prem device needs 4x10G SFP+ cages (2 
redundant paths, plus LAG to customer), and the switches we currently use, 
Arista 7020Rs, are quite expensive if I’m deploying one one per customer.  
(Nice switches, but overkill here – I don’t need 40/100G, and I don’t need 24 
SFP+ ports.  And they still take forever to ship.)

We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode, which 
limits our choices somewhat.  I might be willing to entertain spinning up a 
separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and static VTEPs if it 
saves me a lot of pain.

However, I’m having trouble finding small & cheaper 1U (or even 
desktop/wallmount) devices that have 4 SFP+ cages, and can do VXLAN, in the 
first place.
Who even makes CPE gear with SFP+ ports?  (Other than Mikrotik CRS309-1G-8S+IN 
/ CRS317-1G-16S+RM, which are nice, but our policy requires vendor support 
contracts, so… no-go.)

Vendors?  Model#s, if you happen to know any?

Reply here or privately, whatever floats your boat – any pointers appreciated!

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN logo]]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D99EC2.B891B0A0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

10G CPE w/VXLAN - vendors?

[Adam Thompson]

Hello, all.
I’m having difficulty finding vendors, never mind products, that fit my need.

We have a small but growing number of L2 (bridged) customers that have diverse 
fiber paths available, and, naturally, want to make use of them.
We have a solution for this: we extend the edge of our EVPN VXLAN fabric right 
to the customer premise.  The customer-prem device needs 4x10G SFP+ cages (2 
redundant paths, plus LAG to customer), and the switches we currently use, 
Arista 7020Rs, are quite expensive if I’m deploying one one per customer.  
(Nice switches, but overkill here – I don’t need 40/100G, and I don’t need 24 
SFP+ ports.  And they still take forever to ship.)

We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode, which 
limits our choices somewhat.  I might be willing to entertain spinning up a 
separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and static VTEPs if it 
saves me a lot of pain.

However, I’m having trouble finding small & cheaper 1U (or even 
desktop/wallmount) devices that have 4 SFP+ cages, and can do VXLAN, in the 
first place.
Who even makes CPE gear with SFP+ ports?  (Other than Mikrotik CRS309-1G-8S+IN 
/ CRS317-1G-16S+RM, which are nice, but our policy requires vendor support 
contracts, so… no-go.)

Vendors?  Model#s, if you happen to know any?

Reply here or privately, whatever floats your boat – any pointers appreciated!

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN logo]]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D99EC2.B891B0A0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

Cisco Nexus VXLAN w/vlan-aware-bundle ?

[Adam Thompson]

Does anyone here use VXLAN on Cisco Nexus gear in the "vlan-aware-bundle" mode 
(RFC7432 §6.3), not the "vlan-based" mode (§6.1)?  If so, could you please 
contact me directly to explain how you did so?
Much appreciated,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[cid:image004.png@01D97DB7.68236EA0]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image003.png@01D97DB7.67506730]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

RE: Standard DC rack rail distance, front to back question

[Adam Thompson]

Fascinating.  I’ve never had an ASR-1001 come with two sets of ears, and I also 
note that the text of the instruction manual doesn’t reference the rear set at 
all.  I’ve never seen rear ears on any Cisco gear of my own, nor on anything 
the local ILEC has installed either.  I think the diagram is in error here.
However, the “optional” step 1 is a pretty solid hint (i.e. pretty much a 
clue-by-four upside the head, here!) that you really should use a shelf.  As in 
you REALLY SHOULD USE A SHELF of some kind.

It doesn’t even have to be a full shelf – any rail kit that relies on an 
“L”-shaped profile instead of interlocking sliding bits should support an 
ASR-1001 just fine,  e.g. Tripp-Lite’s 4POSTRAILKIT1U.  RackSolutions’ 
Universal Fixed Server Rack 
Rails<https://www.rack-solutions.ca/rack-rails.html> shows an example of a 
slightly different design that some prefer – it all works about the same way.

The other thing I’ve done is used a shallow cantilever shelf to support the 
rear end of equipment that only comes with ears, if it’s deep enough – 
something like StarTech’s CABSHELFV1U; the trick is finding a shelf that 
simultaneously doesn’t have the structural fold at the rear in the way AND 
doesn’t interfere with the device immediately below.  You’d think there’re only 
2 geometries of product to worry about, but there are actually more b/c there’s 
no standard – so test-fit first, or examine photos really carefully.  This is 
usually more of a hack than a permanent, supportable solution, but sometimes it 
can work very well and very cheaply.

Or, just make sure you’re installing the ASR immediately above something that 
does have proper 4-post mounting rails.  This is probably the single most 
common way to safely & securely mount “eared” devices in a 4-post rack that 
I’ve seen – that Dell PowerEdge server in the rack suddenly starts doing 
double-duty as a shelf!  (Or the UPS, or the KVM, or the ethernet switch, or…)

-Adam

Adam Thompson
Consultant, Infrastructure Services
[cid:image002.png@01D9790D.8F568C90]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image003.png@01D9790B.395F2C40]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>


From: NANOG  On Behalf Of Chuck 
Church
Sent: Thursday, April 27, 2023 10:36 AM
To: 'Mark Stevens' ; nanog@nanog.org
Subject: RE: Standard DC rack rail distance, front to back question

Hey all, sorry I did mean to say ASR1001 (an X model to be exact).  The 4 post 
mounting they show in a hardware mounting doc uses front and back ears, which 
I’ve never done:
https://www.cisco.com/c/en/us/td/docs/routers/asr1000/install/guide/asr1routers/asr-1000-series-hig/asr-hig-1001.html#task_1205646
see figure 16 slightly down from there.

I do see some generic rails from TrippLite that probably would work, as well as 
shelves.   I was hoping a standard depth that most vendors honored for 4 post 
existed, but it doesn’t seem likely.  We’ll have a variety of PaloAlto, Cisco, 
Checkpoint, and others co-habiting.

Chuck

From: NANOG 
mailto:nanog-bounces+chuckchurch=gmail@nanog.org>>
 On Behalf Of Mark Stevens
Sent: Thursday, April 27, 2023 11:17 AM
To: nanog@nanog.org<mailto:nanog@nanog.org>
Subject: Re: Standard DC rack rail distance, front to back question

Lucky you with a 19" data rack. All I have are 23" telco racks but I will say, 
the 23" extension ears from Cisco are serious and my router chassis' don't sag.

Mark

On 4/27/2023 10:04 AM, Chris Marget wrote:

On Thu, Apr 27, 2023 at 9:53 AM Chuck Church 
mailto:chuckchu...@gmail.com>> wrote:
for a Cisco ASA1001, there aren’t rails, but rather front and back ‘ears’ you 
use to hit both front and back posts.

Front *and* back ears? I'm not sure what an ASA 1001 is (ASR?) but my 
experience with these boxes is that they have a single pair of ears which can 
be mounted front OR back.

The heavier / deeper 1RU devices do tend to sag alarmingly.

 Is there a ‘standard’ distance between front and back rails that devices 
usually adhere to?

If you're thinking of setting the front/back distance to accommodate a specific 
device, table 2 might be of some interest:
https://i.dell.com/sites/doccontent/business/solutions/engineering-docs/en/Documents/rail-rack-matrix.pdf

RE: Can I do this in EVPN? (Multihome to more different CEs)

[Adam Thompson]

The solution we've deployed is to use a VXLAN termination device at each 
location requiring multi-path redundancy.
Run VXLAN over isolated L3 domains, let IS-IS or OSPF handle path selection, 
including ECMP if desired.
If multi-chassis redundancy is required, pick a platform that can do MLAG or 
similar.

So for example, I have two sites with multiple VLANs needing to be 
interconnected, and for whatever reason I can't just use a LAG (distance, lack 
of transparent L2 service, whatever).
We would put an Arista 7k-series pizzabox at each end, one end could be an MLAG 
pair.  Terminate two L2 or L3 services on the singleton box, terminate each 
service onto one half of the MLAG pair at the other site.  Run an IGP (ideally 
IS-IS with BFD, but YMNV) and ECMP and happens automatically, as does handling 
single-path failures.
This could equally be a MLAG-to-MLAG setup if you have too much money and need 
to use some up.
Cisco vPC does essentially the same thing, as does Juniper's VC.  Extreme has 
something similar, too.
STP does not get transported across the VXLAN transport, so you now avoid all 
the inherent problems with long-distance or multi-site STP bridging.

-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On
> Behalf Of Jason R. Rokeach via NANOG
> Sent: February 9, 2023 1:11 PM
> Cc: nanog@nanog.org
> Subject: Re: Can I do this in EVPN? (Multihome to more different CEs)
> 
> VPLS doesn't handle loop avoidance. At least, not apart from split
> horizon rules.
> 
> I assume that them properly connecting routers only and doing dynamic
> routing over your service is out of the question? (Even _just_ doing
> this doesn't completely solve the challenge though.)
> 
> It sounds to me like your customer is needing two separate services.
> One to provide connectivity to other sites at layer 2, and another to
> provide backup connectivity within their single campus at layer 2. I
> would suggest that you treat these as two separate services, because
> there's nothing in EVPN that's going to notice on the PE side of the
> equation that the customer has a break in the middle of their
> network.
> Maybe consider offering these two services in combination:
> 1) layer 2 VPN service (VPWS / single pseudowire) between the two
> sides of their campus. You would need to ensure L2CP transparency (or
> tunneling) for STP and they would need to run STP across the link to
> keep their campus whole
> 2) EVPN with ESI in single-active mode, as you had mentioned.
> 
> 
> 
> 
> --- Original Message ---
> On Thursday, February 9th, 2023 at 11:56 AM, Simon Lockhart
>  wrote:
> 
> 
> >
> 
> >
> 
> > On Thu Feb 09, 2023 at 11:54:28AM -0500, Shawn L wrote:
> >
> 
> > > You should be able to setup a VPLS between 3 (or more) devices.
> Something like this --
> >
> 
> >
> 
> > [snip]
> >
> 
> > Thanks - I'm not committed to EVPN, so VPLS could work too. Would
> VPLS
> > handle loop avoidance for me? (i.e. if I have two VPLS PE
> connections into
> > the same broadcast domain on the customer side)
> >
> 
> > Simon
> 
> ___
> Jason R. Rokeach
> m: 603.969.5549
> e: ja...@rokea.ch
> tg: jasonrokeach
> 
> 
> Sent with ProtonMail secure email. Get my PGP Public Key.

RE: BCP38 For BGP Customers

[Adam Thompson]

We’ve seen Juniper EX9ks implement uRPF in such a way that if I have two 
(load-balanced) BGP connections to the EX9k, and uRPF is turned on facing me, I 
immediately experience ~50% outbound packet loss.
Methinks the EX9ks apply uRPF a little too close to the hardware and ignore the 
RIB.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D8FE60.27B812C0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

From: NANOG  On Behalf Of Mike 
Hammett
Sent: November 8, 2022 2:29 PM
To: William Herrin 
Cc: nanog@nanog.org; Grant Taylor 
Subject: Re: BCP38 For BGP Customers

"Reverse path filtering literally says don't accept a packet from
somewhere that isn't currently the next hop for that packet's source
address."

FIB or RIB?

I knew of uRPF as available over an interface, per the routing table, not best 
path available. Or is that implementation dependent?



-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com


From: "William Herrin" mailto:b...@herrin.us>>
To: "Grant Taylor" 
mailto:gtay...@tnetconsulting.net>>
Cc: nanog@nanog.org<mailto:nanog@nanog.org>
Sent: Tuesday, November 8, 2022 2:01:49 PM
Subject: Re: BCP38 For BGP Customers

On Tue, Nov 8, 2022 at 8:40 AM Grant Taylor via NANOG 
mailto:nanog@nanog.org>> wrote:
> Maybe it's the lack of caffeine, but would someone please remind /
> enlighten me as to why uRPF is a bad idea on downstream interfaces?

Hi Grant,

Two words: asymmetric routing.

If the downstream network is architected in such a way that there's
more than one path in and out of their network then there's no way to
guarantee that any particular router believes the forward path to that
network goes to a particular next hop. It can currently map to any
next hop that goes in the direction of one of the valid paths. That
routing is completely independent of how next hops are selected in the
other direction. Packets can travel in one path and out another.

Reverse path filtering literally says don't accept a packet from
somewhere that isn't currently the next hop for that packet's source
address. When it's possible for the forward route to point a different
direction than the one from which valid packets are received, that is
the wrong thing to do. In fact, it breaks the network.

Useful automated reverse path filtering can ONLY be used when there is
exactly ONE valid path to which and from which packets can be
received. This is where strict mode uRPF actually works.


> N.B. Maybe I'm thinking more a varient of uRPF wherein if I have a route
> to the source from the interface in question.  Thus not uRPF-strict
> (active route) nor uRPF-loose (route on any interface).

Even if a particular router happens to have RIB entries for all the
valid paths to a host (a sketchy proposition at best), only one such
entry will be stored in the FIB where uRPF looks to make its filtering
decision.

As for loose mode, it's basically useless in a BCP38 discussion. Loose
mode only filters bogons. It doesn't prevent impersonation of any IP
address currently routed in the system and doesn't do anything at all
on a router with a default route.

Regards,
Bill Herrin




--
For hire. https://bill.herrin.us/resume/

RE: any dangers of filtering every /24 on full internet table to preserve FIB space ?

[Adam Thompson]

I can't believe that never occurred to me in all the time I was doing that, 
'way back when...  
Thanks for pointing that out!
-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On
> Behalf Of Brandon Martin
> Sent: October 21, 2022 4:30 PM
> To: nanog@nanog.org
> Subject: Re: any dangers of filtering every /24 on full internet
> table to preserve FIB space ?
> 
> On 10/20/22 17:50, Adam Thompson wrote:
> > Alternately, a valid technique is to have a default route AND a
> partial BGP feed (a filtered full feed is by definition a partial
> feed).  That helps optimize outbound routing a little bit, you still
> get the advantage - mostly - of multiple inbound carriers; but you
> still have to pick one carrier to do the heavy lifting for you.  And
> you are paying them to route for you, so that's not an unfair
> shifting of the routing burden, unlike relying on covering routes.
> Note that this approach does NOT provide any redundancy, unlike
> having full BGP feeds.
> 
> As a note, you can get redundancy (but still none of the best-path
> advantages of having multiple transits) by asking your transits to
> originate default in their BGP feed and then selectively accepting
> it.
> You can either ECMP it or pick priority with localpref.
> 
> You need multiple full-view transits for this to work, though.
> 
> --
> Brandon Martin

RE: jon postel

[Adam Thompson]

The book, being written by an actual credentialed historian, contains their 
complete sources as footnotes/endnotes.  That section was overwhelming, I 
mostly skipped it...

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On Behalf Of
> Carsten Bormann
> Sent: October 17, 2022 11:54 AM
> To: Grant Taylor 
> Cc: nanog@nanog.org
> Subject: Re: jon postel
> 
> On 2022-10-17, at 16:57, Grant Taylor via NANOG  wrote:
> >
> > In my not so humble opinion, Where Wizards Stay Up Late should be required
> reading for anyone wanting to learn about the history / development of the
> ARPAnet and the Internet.
> 
> That said, it would be a worthwhile project to collect the places in which
> this source can be supplemented with additional information (a.k.a. grains
> of salt).
> 
> Grüße, Carsten

RE: any dangers of filtering every /24 on full internet table to preserve FIB space ?

[Adam Thompson]

I can't find the original message, so replying to the wrong spot in the thread, 
but... no, filtering /24s is a bad idea if you want (more or less) all your 
packets to get to their destinations.

If you filter all /24s you will lose reachability to 4x /24s I publish that 
have no covering route because they are not contiguous and not part of any 
larger logical aggregate.  Then there's the 10-20 legacy /24s I *don't* 
currently publish - if I start advertising them, you won't be able to reach 
them, either, because they're in the same boat: discontiguous singletons.  
There are a LOT of legacy discontiguous IPv4 singletons assigned out of the old 
Class-C space to small/medium businesses, schools, etc. in the pre-ARIN days, 
and I would guess that the vast majority of them do not have a correct covering 
/23 or larger - certainly none of the ones I'm currently working with/aware of 
do.

I believe there's at least a couple of DNS servers running in my /24s, so you 
could potentially lose access to much more than those /24s.

Your packet will *probably* hit a next-hop carrier who happens to have the 
more-specific /24, and it will *probably* eventually reach me, but I thought 
everyone more-or-less agreed that internet router was already nondeterministic 
enough as it is?
IMHO, if you don't want all the /24s in your FIB (or even RIB!), just pick a 
carrier, set a default route, and stop worrying about all the headaches BGP 
provides.
Alternately, a valid technique is to have a default route AND a partial BGP 
feed (a filtered full feed is by definition a partial feed).  That helps 
optimize outbound routing a little bit, you still get the advantage - mostly - 
of multiple inbound carriers; but you still have to pick one carrier to do the 
heavy lifting for you.  And you are paying them to route for you, so that's not 
an unfair shifting of the routing burden, unlike relying on covering routes.  
Note that this approach does NOT provide any redundancy, unlike having full BGP 
feeds.

Separately, I don't know if Geoff has produced such a survey/article, but if 
not he can probably type it from memory by now :-).

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On Behalf Of
> Stephane Bortzmeyer
> Sent: October 10, 2022 10:21 AM
> To: Edvinas Kairys 
> Cc: NANOG Operators' Group 
> Subject: Re: any dangers of filtering every /24 on full internet table to
> preserve FIB space ?
> 
> On Mon, Oct 10, 2022 at 05:58:45PM +0300,
>  Edvinas Kairys  wrote
>  a message of 35 lines which said:
> 
> > But theoretically every filtered /24 could be routed via smaller
> > prefix /23 /22 /21 or etc.
> 
> I don't think this is true, even in theory, specially for legacy
> prefixes. There is probably somewhere a Geoff Huston survey on /24
> without a covering route.

RE: [External] Normal ARIN registration service fees for LRSA entrants after 31 Dec 2023 (was: Fwd: [arin-announce] Availability of the Legacy Fee Cap for New LRSA Entrants Ending as of 31 December 20

[Adam Thompson]

I believe that’s known as “saying the quiet part out loud”, Tom .  Without 
stating my own opinion, I merely observe that I know a great many people who 
agree with you, and a not-insignificant number who vehemently disagree with 
you.  Most of the conclusions to be drawn there are obvious, but humans can be 
infinitely surprising…

Whether ultimately good or bad, IANA decreeing “there’s no such thing as legacy 
any more” would absolutely simplify the future governance and administration of 
the RIRs, especially ARIN.  I don’t see it happening, albeit for political 
reasons instead of legal.  And I, as a Canadian citizen, have approximately 
zero influence over IANA and the organizations to which it is beholden, as 
they’re still, ultimately, mostly, arms of the U.S. government in one for or 
another.

I’d even be happy if ARIN were able to implement a validation/verification 
process for legacy assignments, as I’m aware of a decent number of abandoned 
legacy blocks currently being squatted on by [usually] WISPs who very 
definitely do NOT have the right to do so, but I cannot provide any proof of 
that so nothing can be done.
My own, VERY informal research suggests that while legacy blocks are around 34% 
of 0/0, between 1/5 and 1/3 of legacy blocks themselves (mostly the old Class-C 
blocks) are abandoned, and frequently being squatted-on.
Some of those blocks are assigned to my clients, most of whom would probably be 
happy to transfer, or (gasp) lease those IPs to the current, 
probably-illegitimate, users – public schools can always use a bit more cash!
Absent a clean-up effort, however, with appropriate policy supporting it, we’re 
stuck with the status-quo.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D8CAF7.9540E2A0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

From: NANOG  On Behalf Of Tom 
Beecher
Sent: September 17, 2022 10:19 AM
To: John Curran 
Cc: North American Network Operators' Group 
Subject: Re: [External] Normal ARIN registration service fees for LRSA entrants 
after 31 Dec 2023 (was: Fwd: [arin-announce] Availability of the Legacy Fee Cap 
for New LRSA Entrants Ending as of 31 December 2023)

I would honestly love it if IANA was able to say "As of X date, all LEGACY IPv4 
allocations are transferred to the RIRs . Assignees will not change, but will 
now need to comply with each RIRs policies."

Of course this will never happen, because it would just be a flood of billable 
hours, lawsuits, and injunctions, where companies will claim 'intellectual 
property' over something they didn't develop.

It's exhausting to watch this two tiered system where the legacy holders bleat 
about what the rules should be for the rest of us, while they can do whatever 
the heck they want, simply because they had the foresight to exist at the right 
time.

On Sat, Sep 17, 2022 at 10:41 AM John Curran 
mailto:jcur...@arin.net>> wrote:

On 16 Sep 2022, at 10:11 PM, John Gilmore mailto:g...@toad.com>> 
wrote:

John Curran mailto:jcur...@arin.net>> wrote:

... the long-term direction is to provide the same services to all
customers under the same agreement and fees – anything else wouldn’t
be equitable.

There are many "anything else"s that would indeed be equitable.  It is
equitable for businesses to sell yesterday's bread at a lower price than
today's bread.  Or to rent unused hotel rooms to late-night transients
for lower prices than those charged to people who want pre-booked
certainty about their overnight shelter.  ARIN could equitably charge
different prices to people in different situations; it already does.
And ARIN could equitably offer services to non-members, by charging them
transaction fees for services rendered, rather than trying to force them
into a disadvantageous long term contract.  Please don't confuse
"seeking equity" with "forcing everyone into the same procrustean bed".

John -

ARIN can most certainly charge different fees for different customers – we’re 
actually doing
exactly that today for all of the legacy resource holders who have entered an 
agreement
with ARIN already or who choose to do so in the coming year.  Rather than 
paying the
same registration service fees as everyone else, they have a cap on their total 
registry
maintenance fees (presently $150 per year, subject to an increase $25 per year) 
which
is a unique fee benefit that’s been provided only to the legacy resource 
holders.  The
announcement just made is that we will cease offering this fee cap for legacy 
resource
holders who sign an agreement after 31 Dec 2023; i.e. they will pay the same 
fees as
everyone else based on total resources held.

As others have already noted, this will move ARIN towards charging mor

Re: End of Cogent-Sprint peering wars?

[Adam Thompson]

1-800-FTC-HELP
(Doesn't exist but should)
-Adam

Get Outlook for Android<https://aka.ms/AAb9ysg>

From: Lou D 
Sent: Wednesday, September 7, 2022 5:49:25 PM
To: Adam Thompson 
Cc: Dave Taht ; Jawaid Bazyar ; 
NANOG ; Sean Donelan 
Subject: Re: End of Cogent-Sprint peering wars?

Is there a hotline for those of us who have Cogent and Sprint….. sighs

On Wed, Sep 7, 2022 at 6:17 PM Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
Well, yes, that goes hand-in-hand with "...expects hefty charge".
To me, this just says T-Mobile wants out of the POTS business at almost any 
cost.
All those poor people stuck with Cogent now, I feel sorry for them!

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>

> -Original Message-
> From: NANOG 
> mailto:merlin.mb...@nanog.org>>
>  On
> Behalf Of Jawaid Bazyar
> Sent: September 7, 2022 5:04 PM
> To: Dave Taht mailto:dave.t...@gmail.com>>; Sean Donelan 
> mailto:s...@donelan.com>>
> Cc: NANOG mailto:nanog@nanog.org>>
> Subject: Re: End of Cogent-Sprint peering wars?
>
> $1 deals usually come with an operation in the red, or assumption of
> significant debts.
>
> On 9/7/22, 2:55 PM, "NANOG on behalf of Dave Taht"  bounces+jbazyar=verobroadband@nanog.org<mailto:verobroadband@nanog.org>
>  on behalf of
> dave.t...@gmail.com<mailto:dave.t...@gmail.com>> wrote:
>
> On Wed, Sep 7, 2022 at 1:48 PM Sean Donelan 
> mailto:s...@donelan.com>>
> wrote:
> >
> >
> > Are Sprint AS1239 and Cogent AS174 finally going to settle
> their peering
> > disputes?
> >
> > T-Mobile sells legacy Sprint wireline business to Cogent for
> $1, expects
> > hefty charge
> > https://www.reuters.com/markets/deals/cogent-communications-
> acquire-t-mobiles-wireline-business-2022-09-07/
> >
> > 1,400 customers
> > 1,300 employees
> >
> > 19,000 long-haul route miles
> > 1,300 metro route miles
> > 16,800 leased route miles
>
> That's a dollar well spent. It also explains the layoffs.
> >
>
>
> --
> FQ World Domination pending:
> https://blog.cerowrt.org/post/state_of_fq_codel/
> Dave Täht CEO, TekLibre, LLC
>

RE: End of Cogent-Sprint peering wars?

[Adam Thompson]

Well, yes, that goes hand-in-hand with "...expects hefty charge".
To me, this just says T-Mobile wants out of the POTS business at almost any 
cost.
All those poor people stuck with Cogent now, I feel sorry for them!

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On
> Behalf Of Jawaid Bazyar
> Sent: September 7, 2022 5:04 PM
> To: Dave Taht ; Sean Donelan 
> Cc: NANOG 
> Subject: Re: End of Cogent-Sprint peering wars?
> 
> $1 deals usually come with an operation in the red, or assumption of
> significant debts.
> 
> On 9/7/22, 2:55 PM, "NANOG on behalf of Dave Taht"  bounces+jbazyar=verobroadband@nanog.org on behalf of
> dave.t...@gmail.com> wrote:
> 
> On Wed, Sep 7, 2022 at 1:48 PM Sean Donelan 
> wrote:
> >
> >
> > Are Sprint AS1239 and Cogent AS174 finally going to settle
> their peering
> > disputes?
> >
> > T-Mobile sells legacy Sprint wireline business to Cogent for
> $1, expects
> > hefty charge
> > https://www.reuters.com/markets/deals/cogent-communications-
> acquire-t-mobiles-wireline-business-2022-09-07/
> >
> > 1,400 customers
> > 1,300 employees
> >
> > 19,000 long-haul route miles
> > 1,300 metro route miles
> > 16,800 leased route miles
> 
> That's a dollar well spent. It also explains the layoffs.
> >
> 
> 
> --
> FQ World Domination pending:
> https://blog.cerowrt.org/post/state_of_fq_codel/
> Dave Täht CEO, TekLibre, LLC
> 

RE: Papers/analysis on network equipment pricing since pandemic/banning foreign competition

[Adam Thompson]

If you look at the NANOG archives, this was discussed just a month or so ago, 
somewhere in the middle of the Juniper MX204 EOL discussion, IIRC.

I’ve been seeing a number of analysts with models which show a brief reprieve 
in 2023 then a regression (i.e. back to where we are today, or even worse) from 
there out for another 4-5yrs, and no-one’s putting much credibility in models 
much past 5yrs right now.  Which is good, because some of those models say 
we’re screwed for the rest of our lifetimes.
I don’t know the details of why, although I do recall that some of it is based 
on queueing theory, so the ultra-short hyper-efficient JIT pipelines our 
society has spent the last 50yrs building likely play a part.

FWIW, I saw something about vehicle manufacturing where a factory rep was 
proudly proclaiming that they’d optimized the JIT process to where they needed 
some specific part – and engine, I think? – delivered from the part 
manufacturer twice daily... seemingly completely oblivious to the supply-chain 
risks involved.

-Adam

From: NANOG  On Behalf Of Drew 
Weaver
Sent: July 18, 2022 8:51 AM
To: 'Mel Beckman' ; 'Forrest Christian (List Account)' 

Cc: 'nanog list' 
Subject: RE: Papers/analysis on network equipment pricing since 
pandemic/banning foreign competition

Okay so I suppose how many years are we allocating mentally for them to take 
any action?



From: Mel Beckman mailto:m...@beckman.org>>
Sent: Saturday, July 16, 2022 8:37 PM
To: Forrest Christian (List Account) 
mailto:li...@packetflux.com>>
Cc: Drew Weaver mailto:drew.wea...@thenap.com>>; nanog 
list mailto:nanog@nanog.org>>
Subject: Re: Papers/analysis on network equipment pricing since 
pandemic/banning foreign competition

Drew,

The YouTube channel Asianometry has some good insights into the underlying 
supply chain problems:

https://youtu.be/YJrOuBkYCMQ

Deposits that the issue isn’t with leading as chips, as you might think, but 
with so-called “trailing edge“ chips: microprocessors and support circuits  
that make up the bulk of electronic components, including routers and switches.

Asianometry points out that trailing edge components account for 50% of sales 
in the chip market, and given their much lower price of just a few dollars 
each, they represent many times as many units. This makes them a much larger 
factor in the supply chain problem.

Everything is finally keyed to “just in time“ manufacturing, with very little 
component supply in the pipelines. When Covid hit, those pipelines dried up and 
everything collapsed.
 -mel beckman

On Jul 16, 2022, at 4:19 PM, Forrest Christian (List Account) 
mailto:li...@packetflux.com>> wrote:

The underlying problem is silicon Fab capacity.   It has nothing to do with the 
actual manufacturing of the products once all the components arrive.   If you 
don't have components for your product, a new order for components will take a 
year to arrive just because the factories that turn raw silicon wafers into 
computer chips are backlogged 12 to 18 months.

Note that all of the existing factories are running around the clock, and 
although additional facilities are being built it takes time for them to be 
completed.   I'm also assuming that there has been some question about whether 
this is short term or long term demand which would influence whether you spend 
a billion dollars or more building a new fab.

On Fri, Jul 15, 2022, 11:57 AM Drew Weaver 
mailto:drew.wea...@thenap.com>> wrote:
Has anyone seen any interesting write ups or analysis on what has been 
happening with network equipment pricing and availability in the United States 
over the last couple of years?

Everyone (or at least I did) knew by March 2020 that what manufacturers were 
doing wasn’t really going to work anymore going forward and yet it’s 2 1/3 
years later and we’re still looking at a 1+ year lead time on basic products.

Not to be glib but I am pretty sure I could’ve devised a process to build 
ethernet switches by hand with a soldering iron by now.

Not to mention how many additional supply chains could’ve been established 
since then.

Did Huawei actually serve an important purpose in the market?

Did we put too many eggs in the Broadcom basket?

Did the industry come together and “agree” that the time is nigh to charge 
$16,000 for a Dell S4148T but also that it would take 9 months to get it at 
that price?

I think it would be fascinating to learn more about what is happening in this 
market.

Please share any resources on or off-list.

Thanks and have a great weekend!
-Drew

RE: Re:

[Adam Thompson]

I run both OpenBSD + OpenBGPd + OpenBSD/OpenBGPd’s LG, and BIRD + 
xddxdd/bird-lg-go<https://github.com/xddxdd/bird-lg-go> (on two different 
servers, because I value my sanity) because they do a few things differently, 
and neither can show me everything I want.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D88572.EF7703F0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

From: NANOG  On Behalf Of 
Michel Blais
Sent: Monday, June 20, 2022 7:45 PM
To: North American Network Operators' Group 
Subject: Re:

Several seems to use OpenBSD with OpenBGP and BGPLG.

Le lun. 20 juin 2022 à 17:08, J. Hellenthal via NANOG 
mailto:nanog@nanog.org>> a écrit :
It's not about what you use as aposed more of where it's used from.
--
 J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.


On Jun 20, 2022, at 13:47, Josh Luthman 
mailto:j...@imaginenetworksllc.com>> wrote:

I use Cogent: https://www.cogentco.com/en/looking-glass and HE which is easier 
to remember: https://lg.he.net/

On Mon, Jun 20, 2022 at 9:56 AM Glenn Kelley  
wrote:
Good Monday Morning Everyone.

Quick Question:

What is everyone's favorite software for running a looking glass.

A friend asked me this over the weekend - and while there are others available 
on the internet to use - it would be helpful for them to run one within their 
own network.

It has been a while since i have played setting one up so figured might as well 
ask


Glenn S. Kelley, Connectivity.Engineer
Text and Voice Direct:  740-206-9624

Error! Filename not specified.

GoDaddy intermittent unreachability?

[Adam Thompson]

Does anyone from GoDaddy lurk here?
We’re getting multiple reports of GoDaddy-hosted websites being intermittently 
unreachable, but inconsistently – one IP can load a site, the next IP can’t, 
even two different outbound NAT addresses on the same link will have different 
results.  This smells like a hashing-involved problem to me, which will be 
(already is) difficult to troubleshoot.

(There’s my shot in the dark for today!)

Thanks,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D88172.DE1E4450]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

RE: Serious Juniper Hardware EoL Announcements

[Adam Thompson]

[Not specific to the Juniper EoLs...]

I sort of agree with Mark:

I've been sampling a fairly wide variety of sources in various parts of the 
global supply chain, and my synthesis of what they're saying is that we 
probably won't *consistently* have the ready availability of "stuff" (both 
electronic and not) we had pre-pandemic, for the rest of my career (10-15yrs), 
and maybe not in the lifetimes of anyone reading this today, either.

Whether those sources are accurate, their interpretation is accurate, my 
synthesis is accurate, whether I'm listening to the right people in the first 
place... all debatable.  I sure hope the above conclusion is wrong.

One possible upside: it might slow down the incessant upgrade hamster-wheel 
we're all running on?  Imagine having enough time to do your job thoroughly and 
properly...  Yes, I know I'm dreaming :-).


Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On Behalf
> Of Mark Tinka
> Sent: Tuesday, June 14, 2022 11:19 AM
> To: nanog@nanog.org
> Subject: Re: Serious Juniper Hardware EoL Announcements
> 
> 
> 
> On 6/14/22 18:06, JASON BOTHE via NANOG wrote:
> 
> > Saw this coming a mile away. With chips and technology progressing
> despite ability to manufacture, I’m certain many are going to do this.
> 
> All this will do is keep these boxes off the open market, which will
> simply bump up open market prices, with no incentive for the majority
> of
> folk to buy directly from the OEM.
> 
> I suspect supply chain will improve within the next 12 months, but
> then
> regress and hit a massive crunch from around Q4'23 onward. How long
> for,
> I can't say...
> 
> Mark.

Re: Upstream bandwidth usage

[Adam Thompson]

Ah, I did miss that, you're right.  We don't have very much GPON up where I am.
-Adam

Get Outlook for Android<https://aka.ms/AAb9ysg>

From: Mel Beckman 
Sent: Thursday, June 9, 2022 6:31:34 PM
To: Adam Thompson 
Cc: Michael Thomas ; nanog@nanog.org 
Subject: Re: Upstream bandwidth usage

Adam,

Your point on asymmetrical technologies is excellent. But you may not be aware 
that residential optical fiber is also asymmetrical. For example, GPON, the 
latest ITU specified PON standard, and the most widely deployed, calls for a 
2.4 Gbps downstream and a 1.25 Gbps upstream optical line rate.

 -mel

> On Jun 9, 2022, at 3:08 PM, Adam Thompson  wrote:
> However, if you're talking about fiber service, it's pretty much pure 
> marketing-dept-driven BS, combined with some vague justification of not 
> letting TOR nodes or copyright-ignoring seeders/Warez-providers/etc. 
> overwhelm the network in unexpected ways.

RE: Upstream bandwidth usage

[Adam Thompson]

On DOCSIS systems, upload/download ratios are frequency-mapped timeslot ratios 
that are not adjustable in real-time.
On xDSL systems, upload/download ratios are - VERY roughly speaking - a 
function of how much spectrum is allocated to each direction based on each 
individual line's characteristics, and also not adjustable in real-time.
Most fixed-wireless systems have similar limitations to one or the other above, 
although they vary in the details.

Anecdote: I used to maintain/sell/support a system that could automatically 
"tune" DSL service for the prevailing line conditions (as these change with 
age, weather, interference, etc.) and I recall learning from one customer that 
auto-tuning any more often than every 24hrs became *severely* 
counter-productive, because the connection had to drop and re-negotiate every 
time a change was made because of the way DSL modems work; our product had to 
incorporate a fall-back where we reverted to ADSL 1M rates if the line was 
still down an hour later, in case the remote modem just refused to renegotiate 
at what should have been a perfectly valid profile (speed) for some reason or 
other.


So the short answer is: because the harder limitation to solve is the last-mile 
technology, not the choke-points at the network edges where shaping happens.  
All that rate-shaping in the core is generally about preventing the downstream 
packet(s) that would overload the last-mile from ever reaching the last-mile 
device in the first place.


However, if you're talking about fiber service, it's pretty much pure 
marketing-dept-driven BS, combined with some vague justification of not letting 
TOR nodes or copyright-ignoring seeders/Warez-providers/etc. overwhelm the 
network in unexpected ways.


-Adam (who acknowledges he runs a very unusual SP network where rate-limiting 
is unheard of)

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On Behalf
> Of Michael Thomas
> Sent: Thursday, June 9, 2022 3:46 PM
> To: nanog@nanog.org
> Subject: Re: Upstream bandwidth usage
> 
> 
> On 6/9/22 1:26 PM, Mel Beckman wrote:
> > With 430 GB versus 32 GV average down versus up usage today,
> according
> > to your article, this is still not a case for symmetrical consumer
> > bandwidth. Yes, the upstream usage increased slightly more than the
> > downstream usage. But the ratio was still so big that it would take
> > decades for them to join. I doubt they ever will. Consumers just
> don’t
> > have that much days up to push yet, and probably never will.
> >
> > Also, a lot of that Usage can be explained by video conferencing
> > during Covid, which has dropped off significantly already.
> >
> >
> If it's so tiny, why shape it aggressively? Why shouldn't I be able to
> burst to whatever is available at the moment? I would think most users
> would be happy with that.
> 
> Mike

RE: BGP Javascript Map/Visualization

[Adam Thompson]

Neat.  Any idea who to ask questions of, regarding the incorrectness of the 
data?  I would have assumed Job, but he's long gone from NTT, is this 
abandonware or maintained?  Anyone know?

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca


> -Original Message-
> From: NANOG  On Behalf
> Of b...@uu3.net
> Sent: Thursday, May 26, 2022 3:50 PM
> To: nanog@nanog.org
> Subject: Re: BGP Javascript Map/Visualization
> 
> You were close... I think you mean this one?
> https://as2914.net/
> 
> -- Original message --
> 
> From: Brian Johnson 
> To: nanog@nanog.org
> Subject: BGP Javascript Map/Visualization
> Date: Thu, 26 May 2022 20:07:51 +
> 
> Hey all,
> 
> Sorry for the noise.  Years ago someone here built and shared a
> javascript
> visualization of what their routers saw for the state of BGP and paths
> to
> get to various ASs. One could use WSAD and other keys to fly around
> and
> examine various other ASs.  I thought it was as2814.net, but that
> seems to
> not be a thing anymore.  Can someone refresh my memory where that
> might be?
> Or did it get taken down?
> 
> Thanks in advance,
> - brian
> 
> 
> 
> 

Juniper MX204 allow oversubscription?

[Adam Thompson]

Hi all,
Hoping some Juniper-using folks know:
On the MX204, which comes with 4x100G + 8x10G ports, you can only use 3 of the 
4 100G ports if you want to use any of the 10G ports at all.
Supposedly this is to prevent oversubscription on what is a 400G-rated router.  
However, I’m perfectly fine with oversubscription with a 400G aggregate 
throughput cap.
Is there a way to stop the automatic “oh, I’ll disable these other ports for 
you so you don’t oversubscribe the box” behaviour and let all the front-panel 
ports be used at once?
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D8691C.50B29030]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

RE: 10 Do's + Don'ts for Visiting Québec + Register Now for N85!

[Adam Thompson]

I think I have actually heard « tire-toi une bûche » before!  But it was as a 
child, visiting our annual Fête du Voyageur historical re-enactment, and 
certainly not in any normal day-to-day setting.
I’m just happy that an American author (who quite likely has never been to 
Montreal), writing for an overwhelmingly-American audience, recognized that we 
have separate cultures in the first place.

Meanwhile, one thing I haven’t seen mentioned anywhere yet is ArriveCAN[1] 
and/or eTAs[2].
If you’re trying to enter Canada right now you must use the ArriveCAN system 
before you get to the border, or you’ll likely be denied entry.  That means 
doing it before you get on the plane, not after you land.
Anyone entering Canada using a Canadian or American passport should not need an 
eTA (Electronic Travel Authorization, sort of an e-visa), but pretty much 
everyone else does (e.g. passports from Mexico or anywhere in the Caribbean, in 
this case).

Absolutely check this out for yourself, links are below, I am not guaranteeing 
in any way the accuracy, nor the durability, of what I’ve written here.

References:
[1] COVID-19: Use ArriveCAN to enter Canada - 
Canada.ca<https://www.canada.ca/en/public-health/services/diseases/coronavirus-disease-covid-19/arrivecan.html>
[2] Electronic Travel Authorization (eTA) - 
Canada.ca<https://www.canada.ca/en/immigration-refugees-citizenship/services/visit-canada/eta.html>
General info: Visit Canada - 
Canada.ca<https://www.canada.ca/en/immigration-refugees-citizenship/services/visit-canada.html?outside>
Official landing page: Welcome / Bienvenue | Foreign Affairs, Trade and 
Development Canada / Affaires Étrangères, Commerce et Développement Canada 
(canadainternational.gc.ca)<https://www.canadainternational.gc.ca/>

I already wrote all of this up for a conference based in Ottawa, that sees a 
large qty. of int’l visitors from around the world: BSDCan 2022 - 
Travel<https://www.bsdcan.org/2022/travel.php#mozTocId145341>  Travelers from 
the US generally don’t have the kind of issues at customs I’ve described there.

-Adam


P.S. ArriveCAN is a pain to Canadians, too, I can’t just pop across the border 
for shopping trips whenever I want any more without planning for it in advance. 
 Yeah, I know, first-world problems…

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png@01D8614A.053D7EA0]Chat with me on 
Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>

From: NANOG  On Behalf Of J 
EMail
Sent: Friday, May 6, 2022 8:00 AM
To: Nanog News 
Cc: nanog@nanog.org; nanog-annou...@nanog.org
Subject: Re: 10 Do's + Don'ts for Visiting Québec + Register Now for N85!


On Thu, 5 May 2022 at 08:57, Nanog News mailto:n...@nanog.org>> 
wrote:
10 Do's + Don'ts for Visiting Québec
NANOG 85 Meeting Will Take Place Jun. 6 - 8 in Montréal

We are delighted to cross international borders in our mission to grow, inspire 
+ profoundly build the Internet of tomorrow!

Montréal is Canada's second-largest city and is known for its melting pot of 
diverse culture, established universities, enthralling art, food, history + 
festivals. It has been called one of the world's "happiest locations" as an 
estimated 45,000 immigrants relocate to the city every year.

For those who don't call Québec home, we have prepared a list of cultural "Do's 
and Don'ts" to help you quickly acclimate + thrive in this foreign destination.

READ MORE <https://www.nanog.org/stories/10-dos-dont-of-visting-québec/>


 I have lived ten minutes from Quebec and two hours from Montreal for a long 
time, I have never encountered either item 1 or item 2. Of course, there might 
be a place that won't take a credit card, but your credit card company will 
charge you a fee and be happy to use a terrible exchange rate as will 
restaurants if you pay in US cash.

Consider getting cash from your bank account at an ATM at a bank once you land 
although there are fees there as well. If you are a TD bank customer, TD is a 
Canadian Bank and that will eliminate a fee.

As for culture, smoked meat, bagels (they are different) and poutine should be 
on this list.

RE: Announcement of Experiments

[Adam Thompson]

I am not claiming any of this is official MERLIN position on the matter, these 
are merely my thoughts so far based on the incomplete knowledge & data I have:

IMHO, it's somewhat the same as if I made public statements that started with 
"Well, I talked to Randy Bush and he said ".  I'm clearly the one 
articulating that sentence, but I'm nonetheless attributing to you something 
that is (presumably) false.
This will, I think, taint historical time-series data (e.g. RIPEStat) for any 
ASNs the experimenters use, and I could easily see in my organization being 
called upon to ask "Why were we transiting x.x.x.x/y in May 2022?" and not 
having any answer.
The operational impact will probably be somewhere between zero and negligible, 
assuming the experiment is run correctly, but operational impacts aren't the 
only impacts: reputational risks are very important to some organizations.

In addition to people not fully understanding AS_PATH, which even here will be 
a non-zero number, there will also be a number of people (myself included in 
this number) who have no idea what the PEERING testbed is, nor how it works, 
nor the effects it can produce.  I'm in alignment with several other commenters 
in that I should not have to go spend time to learn about Yet Another Piece of 
Technology just to assess the risks, operational and reputational, I now face.

>From my limited understanding of the experiment, I agree that opt-in would 
>kind of defeat the purpose, but at the same time, the opt-out email bordered 
>on insulting/careless: "hey, we're going to simulate a crime scene with your 
>fingerprints unless you tell us not to within a week" wouldn't fly most 
>places.  If they had run their experiment without telling anyone, possibly 5 
>or 10 people/orgs worldwide would have noticed, assumed someone was doing 
>something naughty (or incompetent), and gone on with their lives.  But no 
>notice would arguably have been even more wrong than the notice we did get 
>here.

Is it possible to run such an experiment ethically without tainting the data in 
advance by announcing it?  I don't know.


Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca
Chat with me on Teams: athomp...@merlin.mb.ca

> -Original Message-
> From: NANOG  On
> Behalf Of Randy Bush
> Sent: Monday, May 2, 2022 3:50 PM
> To: Alexandros Milolidakis 
> Cc: nanog@nanog.org
> Subject: Re: Announcement of Experiments
> 
> > We are a group of researchers from the KTH Royal Institute of Technology
> > (Sweden).
> >
> > Starting from May 9 until May 31, we plan to conduct a research study
> > involving AS-PATH poisoning to measure how reliable route collectors
> > are to report BGP poisoned routes.
> >
> > We will use the PEERING Testbed [1] to announce the following two
> > prefixes:
> >
> >  - 184.164.236.0/24
> >
> >  - 184.164.237.0/24
> >
> > for our AS-path poisoning experiments.
> >
> > The above experimental prefixes do not host any production services,
> > hence user traffic will *not* be affected.
> >
> > Furthermore, we will always start the AS-PATH with the correct ASN as the
> > origin.
> >
> > Lastly, to keep the AS-PATH short, we will announce no more than four
> > Poisoned ASNs per announcement. The frequency of the announcements
> > will not exceed four per hour.
> 
> seems quite harmless.  though i am sure folk who do not really
> understand AS_PATH will get their nickers in a twist.
> 
> randy

RE: fs.com Ethernet switches

[Adam Thompson]

One of my clients deployed S3900s, both 24- and 48-port copper models, across 
half a dozen sites, and I did 99% of the config.
Theoretically the 5800s are just a faster/beefier version but haven’t seen them 
in person.
They… work, more or less.  Some of the hardcoded limits are just stupid, like 
max 32 DHCP-relayed devices per L3 interface/VLAN and the 33rd client just 
doesn’t get DHCP.
Either the intermediate carrier out there is stripping VLAN tags, or there’s 
something really weird with their trunking, not sure which yet.
Both the GUI and CLI are required to configure a switch in practice – perhaps 
you can use the CLI exclusively if you’re an expert, but holy cow some of the 
config language is radically unintuitive.
OTOH, even basic models have some advanced features like EAPS/ERPS in the base 
system.
It’s very clear to me from the capabilities and language used in the original 
OS release that this model, at least, was originally targeting ILECs almost 
exclusively (e.g. console port == craft interface).  Newer software releases 
have made them a little less obscure or difficult to work work.
I can’t quite say “don’t buy them” but I sure wouldn’t recommend them, either.
Broadly put: you get what you pay for!
-Adam


Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of 
Richard Angeletti
Sent: Wednesday, April 13, 2022 2:11 PM
To: nanog@nanog.org
Subject: fs.com Ethernet switches

Wondering if anyone on the list has any experiences with fs.com<http://fs.com> 
Ethernet switches that they are willing to share (good or bad)?

We're looking for some cost effective L2 only 10Gb-T switches and their S58XX 
switches have come up as a potential option.

Thanks,
Rich

RE: Fwd: Fw: HOST IRR Retirement

[Adam Thompson]

>From https://www.irr.net/docs/list.html

Registry Name (Source): HOST
IP address or DNS name: rr.host.net
Ftp site:   ftp://ftp.host.net/host/dbase
Databases Mirrored: ALTDB, AOLTW, APNIC, ARIN, EPOCH, LEVEL3, 
PANIX, RADB, REACH, RIPE
Mirror Port and Info:   rr.host.net, port 43
Whois Location: rr.host.net
Type of Primary Data:   Host.net customers and general Internet 
community
Contact Info:   db-ad...@host.net
NOC Info:   n...@host.net

The better question is, how do you authenticate Ross' email, and how do you 
authenticate the... not the contents, but the authenticity & accuracy of the 
meaning of the email?
I suppose you don't have to - at some point they'll just turn it off and then 
we'll all know the email was legit :-).

As to Ross' question: IIRC there exists, or existed at one time, a private(?) 
mailing list for IRR operators.  I would start with r...@merit.edu, if that 
address still works.  I can also reach out to the folks running the CANARIE IRR 
to see if they know, if necessary.

-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

> -Original Message-
> From: NANOG  On
> Behalf Of babydr DBA James W. Laferriere
> Sent: Monday, April 11, 2022 3:50 PM
> To: Ross Tajvar 
> Cc: North American Network Operators' Group 
> Subject: Re: Fwd: Fw: HOST IRR Retirement
> 
>   Hello Ross ,  I do not see a host name or IP4or6 in the below .
>   Hth ,  JimL
> 
> On Mon, 11 Apr 2022, Ross Tajvar wrote:
> > I tried sending the below message from my work account, but it's not a
> > nanog subscriber, so the email was rejected. If anyone doubts the
> > authenticity, feel free to reach out to me at rtaj...@365datacenters.com.
> >
> >
> > --
> > *From:* Ross Tajvar
> > *Sent:* Monday, April 11, 2022 3:53 PM
> > *To:* nanog@nanog.org 
> > *Subject:* HOST IRR Retirement
> >
> > Hi all,
> >
> > We (365 Datacenters) inherited the HOST IRR. We have removed all stale
> > objects from it, and moved all valid objects to other IRRs. We will
> > eventually (hopefully soon) turn it off altogether. Please, if you are
> > mirroring it, stop doing that. If you maintain documentation that lists
> > IRRs, please update it to reflect that HOST is no longer in use.
> >
> > Thanks!
> >
> > P.S. If anyone thinks I should also announce this somewhere else, please
> > let me know.
> >
> > *Ross Tajvar*
> >
> > Network Engineer
> >
> > Office: (571)-341-8899
> >
> > Support: (866)-365-6246
> >
> 
> --
> +-+
> | James   W.   Laferriere| SystemTechniques | Give me VMS |
> | Network & System Engineer  | 3237 Holden Road |  Give me Linux  |
> | j...@system-techniques.com | Fairbanks, AK. 99709 |   only  on  AXP |
> +-+

RE: More product suggestions: small/cheap IS-IS or VXLAN devices?

[Adam Thompson]

Probably not an option for us, but thank you – I wasn’t aware VyOS included 
VXLAN.
Merci,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: Pierre LANCASTRE 
Sent: Tuesday, April 5, 2022 11:23 AM
To: Adam Thompson 
Cc: nanog 
Subject: Re: More product suggestions: small/cheap IS-IS or VXLAN devices?


Le mer. 23 févr. 2022 à 00:44, Adam Thompson 
mailto:athomp...@merlin.mb.ca>> a écrit :
At the risk of sounding like a broken record, I’m asking for product 
suggestions yet again:

We’re wondering if anything small & cheap (think CPE-grade) exists that 
supports either IS-IS or VXLAN?

If IS-IS, total route count it would have to carry would be small, probably in 
the ~500 range.
If VXLAN, it needs to interoperate with Arista.
If both… yay!

Hi Adam,

A very late answer on that topic. You could look at VyOS devices compatible, 
VXLAN is working on the 1.4 rolling release. No idea about the timeline for 
this version to become the main train.

We've shared an article on our lab tests of VyOS/Arista/Cumulus VX interop in a 
EVPN VXLAN bgp/bgp fabric, it could help you in your research : 
https://wiki.nesevo.com/index.php/VyosAristaCVX-VXLAN-lab

When I say CPE-grade, I mean under C$1k (~US$800, ~€700), and can be emplaced 
at a customer site without any unusual infrastructure (e.g. no -48VDC power, or 
DIN rail mounting, non-business-office-typical).

You should find something which fits your needs here : 
https://vyos.io/hcl/?vendor=edgecore-networks=lanner-electronics-inc=dell=aaeon

BR,

Pierre

RE: Opinions on Arista for BGP?

[Adam Thompson]

TL;DR: Yes, go ahead, they’re good, we like them.

I won’t say they’re perfect, but we’re using them at the edge (two of them in a 
hybrid core/edge model right now, even!) and I would happily endorse them for 
edge routers.  Their BGP stack hasn’t put up any major roadblocks for us so far 
(at least, that weren’t, ahem, self-inflicted).  We’ve had 1 incident in the 
last ~2 years where a stuck route on one router needed a full reboot to clear 
out, following a partial outage - that’s the worst thing I can remember right 
now.

Don’t know if you know this already or not, so making it clear:  the one thing 
to beware of IMO, compared to e.g. a high-end Juniper MX960-style system where 
you can turn every single feature on without caring, is that the Aristas can do 
almost anything you can dream of… but not necessarily all at the same time on 
the same box, no matter which model you’re looking at.
So if you use it as an edge router?  Fine.  As a VXLAN gateway?  Fine.  As a 
core router or switch with every kind of accounting turned on?  Fine.  All of 
those things simultaneously?  Maybe.  It’ll be decision time for which 
specific, individual sub-features you can live without.  But you’re paying 
1/10th (probably less!) of what you would for an MX960, so there you go.

If this helps, they’re similar to the Cisco Nexus platform in this regard, e.g. 
if you enable and use every single “Feature” on the fixed-configuration Nexuses 
you’ll start running out of hardware configuration resources to enable them 
long before you can finish configuring or using all those features.

This is something your Arista SE can go through with you in excruciating detail 
(keyword: “TCAM Profile”), if you think you might be veering into that 
territory.  After lots of iterations, and a new software release or two, our 
all-in-one boxes (7280SR2K) do more or less everything we want them to.  
(Apparently we aren’t typical Arista customers.  Go figure.)  If you want to do 
BGP and MLAG at the same time on the same box, get your SE involved from the 
start.

For anyone not trying to overload the platform or do too much “weird” stuff, it 
should be a quick and easy deployment producing much happiness.

-Adam


Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of David 
Hubbard
Sent: Thursday, March 31, 2022 8:10 AM
To: nanog@nanog.org
Subject: Opinions on Arista for BGP?

Hi all, would love to get any current opinions (on or off list) on the 
stability of Arista’s BGP implementation these days.  Been many years since I 
last looked into it and wasn’t ready for a change yet.  Past many years have 
been IOS XR on NCS5500 platform and Arista everywhere but the edge.  I’ve been 
really happy with them in the other roles, so am thinking about edge now.  I do 
like and use XR’s RPL, and prefix/as/community/object sets, but we can live 
without via our own config management if there aren’t easy equivalents.  No 
fancy needs at all, just small web server networks, so just need reliable eBGP 
and internal OSPF/OSPFv3.

Thanks,

David

RE: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times

[Adam Thompson]

> -Original Message-
> From: NANOG  On
> Behalf Of Joe Maimon
> Sent: Thursday, March 31, 2022 6:20 PM
> Subject: Re: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255
> times
>
> [...]
> I think more and perhaps different knobs were and still are needed.

YES.  YESYESYES.

Having said that, I am currently able to express my local traffic policies 
(roughly 1/3 due to one particular upstream hamstrung by their parent company, 
1/3 due to NREN, 1/3 due to local IXP) using prepends of no more than +2 (both 
outbound and inbound, which I gather might be somewhat unusual).  That includes 
dealing with a "special" issue because both my local NREN RAN and I are both 
connected to the local IXP, and to a mutual downstream, which produces an 
interesting double-triangle topology, and this topology produces surprising 
emergent behaviours that I haven't seen described anywhere.

Prior to some reorganization (and several compromises) I needed +5 on both 
inbound and outbound (usually not at the same time) to adequately steer normal 
traffic.


Prepending is, IMHO literally, the abuse of a mechanism not initially designed 
to express complex policies.  It's a 1-dimensional tool in a space that really 
needs a 2- or 3-dimensional tool.

LOCAL_PREF is not useful for me and my upstreams, downstreams and peers: I have 
not yet found a scenario where it fixes any of my problems without creating 
even greater ones.  My traffic-steering issues ~all exist n>3 hops away, not 
directly with my BGP peers.

So prepending remains the only useful, usable knob I have.  And it's not a 
great knob for this, which I think might be the one thing everyone here can 
agree on!


> through nowadays, how about a long call back to each AS in the path

I'm not really loving that solution more than prepends, but at least it's 
something different?


> Would be nice to be able to publish your community scheme as simply
> conforming with RFCX and the to configure peers with process-rfcX
> statement and be mostly done.

That's a LONG way off!  Not that any other solution isn't equally far off, 
so...   Gotta start somewhere, I guess!

I don't have any better suggestions, other than I wish the LOCAL_PREF crowd 
could understand that it doesn't solve all the world's problems.  (Neither of 
my commercial upstreams currently admit to supporting communities that control 
it, anyway!)


Frustrated at the state of the world today,
-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

RE: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times

[Adam Thompson]

e 
located?  Or the 519 at 11?  Remember, those are already the de-prepended 
paths.  I don’t want to have to police the RIB that tightly, deciding which 
routes I will and won’t accept and adjusting the limit periodically.

Unless your intent is to eliminate prepend-based traffic engineering from the 
internet altogether, which case 10 is a perfectly reasonable choice, but in the 
absence of any other globally-usable tool/knob, that’s a hill I WILL die on.

If there’s broad consensus that a path length limit is a good thing, I would 
suggest a value of no less than 32, based on the data I’ve got in my RIB right 
now.  I think, based only on random sampling of longer AS paths in my RIB, that 
32 would still give operators the latitude to perform AS-path-based traffic 
engineering at origin, during transit, and locally upon receipt, without routes 
getting inexplicably blackholed anywhere.

I’ve heard tonight that a path length limit of 32 is already commonly 
implemented.  Regardless of whether I think that’s a good idea, the spectre of 
the stack-breakingly-long path seems to be already mitigated in many places, 
but perhaps not widely?

To sum up, Tom’s conclusions as expressed in his email (below) may be 
qualitatively correct, but they are quantitatively wrong, at least on the 
matter of the numeric threshold.  And… I don’t really want to be the next 
Sprint(?) in BGP history just to protect myself from newbies on Mikrotiks[1], 
do you?

-Adam


[1] and others, yes, I know it’s not purely a Mikrotik issue.


Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: Tom Beecher 
Sent: Saturday, March 26, 2022 11:35 AM
To: Adam Thompson 
Cc: Paschal Masha ; nanog 
Subject: Re: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times

Mostly what Matt said. ( I should have also said 'ride the 0/0 train INTO the 
DFZ, my mistake.)

Essentially, if ASN X is announcing a prefix with an excessive number of 
prepends, they are saying to the world 'This path exists , but it is hot 
garbage.' I'm more than happy to oblige those instructions and just drop that 
announcement completely. If ASN X announces that prefix with a reasonable 
number of prepends, happy to take it and use it.

If I get a prefix with an as-path longer than 10, (regardless of the state of 
prepends), I interpret that as :

1. They have made a mistake.
2. Someone else made a mistake.
3. They think that's a good idea, and have some things yet to learn. ( The old 
clue by four as Matt put it.)
4. It's malicious.
5. They actually are somehow more than 10 ASNs away from me. ( I don't even 
know if this is possible anymore without extreme effort. )

In any of those scenarios , I don't really care about optimized routing to that 
destination. Perfectly happy to just follow 0/0 to a DFZ upstream and let it go 
how it's going to go, or not. If there is a traffic impact such that an 
exception HAS to be made, that can be addressed as needed, but I can't think of 
a single circumstance I have hit where the correction involved accepting an 
obnoxiously long as_path announcement. ( I don't mean to imply none exist ; I'm 
sure someone has had to work though that. )

Maybe a length of 10 doesn't work for some environments and use cases, but I 
still am a firm believer that at a certain point, there is a length that 
becomes straight 'really don't care'.  I'd rather not discover a BGP 
implementation problem or acid trip memory pointer party by accepting 
announcements that are useless.







On Fri, Mar 25, 2022 at 5:56 PM Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
Tom, how exactly does someone “ride the 0/0” train in the DFZ?

I’m connected to both commercial internet and NREN, and unfortunately-long 
paths are not uncommon in this scenario, in order to do traffic steering.  If 
there’s another solution that affects global inbound traffic distributions, I’d 
love to hear about it (and so would a lot of my peers in edu).

If there were a usable way to “dump” the excessively-long path only as long as 
a better path was already known by at least one edge router, that might be 
workable, but you’d have to keep track of it somewhere to reinstall it if the 
primary route went away… at which point you may as well have not dropped it in 
the first place.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG 
mailto:merlin.mb...@nanog.org>> 
On Behalf Of Tom Beecher
Sent: Friday, March 25, 2022 4:13 PM
To: Paschal Masha 
mailto:paschal.ma...@ke.wananchi.com>>
Cc: nanog mailto:nanog@nanog.org>>

RE: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times

[Adam Thompson]

Tom, how exactly does someone “ride the 0/0” train in the DFZ?

I’m connected to both commercial internet and NREN, and unfortunately-long 
paths are not uncommon in this scenario, in order to do traffic steering.  If 
there’s another solution that affects global inbound traffic distributions, I’d 
love to hear about it (and so would a lot of my peers in edu).

If there were a usable way to “dump” the excessively-long path only as long as 
a better path was already known by at least one edge router, that might be 
workable, but you’d have to keep track of it somewhere to reinstall it if the 
primary route went away… at which point you may as well have not dropped it in 
the first place.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of Tom 
Beecher
Sent: Friday, March 25, 2022 4:13 PM
To: Paschal Masha 
Cc: nanog 
Subject: Re: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times

The best practice with regards to as_path length is to have an edge filter that 
dumps any prefix with a length longer than say 10. Depending on the situation, 
might even be able to go smaller.

At a certain point, keeping that route around does nothing for you, just shoot 
it and ride the 0/0 train.

On Fri, Mar 25, 2022 at 4:09 AM Paschal Masha 
mailto:paschal.ma...@ke.wananchi.com>> wrote:
:) probably the longest prepend in the world.

A thought though, is it breaking any standard or best practice procedures?

Regards
Paschal Masha | Engineering
Skype ID: paschal.masha

- Original Message -
From: "Erik Sundberg" mailto:esundb...@nitelusa.com>>
To: "nanog" mailto:nanog@nanog.org>>
Sent: Friday, March 25, 2022 6:43:38 AM
Subject: DMARC ViolationAS21299 - 46.42.196.0/24<http://46.42.196.0/24> ASN 
prepending 255 times

If anyone from AS21299 is lurking on Nanog. Please reduce your AS prepends for 
46.42.196.0/24<http://46.42.196.0/24> from 255 prepends to a more reasonable 
number of prepends let's say 20. Thanks!

This is a Kazakhstan register IP Block and ASN



Network Next Hop Metric LocPrf Weight Path

*> 46.42.196.0/24<http://46.42.196.0/24> x.x.x.x 0 100 0 2914 174 3216 3216 
35168 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21 299 21299 21299 21299 21299 
21299 21299 21299 21299 21299 21299 21299 21299 21299 i








CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or 
previous e-mail messages attached to it may contain confidential information 
that is legally privileged. If you are not the intended recipient, or a person 
responsible for delivering it to the intended recipient, you are hereby 
notified that any disclosure, copying, distribution or use of any of the 
information contained in or attached to this transmission is STRICTLY 
PROHIBITED. If you have received this transmission in error please notify the 
sender immediately by replying to this e-mail. You must destroy the original 
transmission and its attachments without reading or saving in any manner.
Thank you.

More product suggestions: small/cheap IS-IS or VXLAN devices?

[Adam Thompson]

At the risk of sounding like a broken record, I’m asking for product 
suggestions yet again:

We’re wondering if anything small & cheap (think CPE-grade) exists that 
supports either IS-IS or VXLAN?

If IS-IS, total route count it would have to carry would be small, probably in 
the ~500 range.
If VXLAN, it needs to interoperate with Arista.
If both… yay!

When I say CPE-grade, I mean under C$1k (~US$800, ~€700), and can be emplaced 
at a customer site without any unusual infrastructure (e.g. no -48VDC power, or 
DIN rail mounting, non-business-office-typical).

Thanks in advance, everyone.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

Proofpoint woes?

[Adam Thompson]

I realize this is a more networking-focused forum, but I'm wondering if anyone 
has contacts inside Proofpoint?  They're an anti-spam service provider who has 
suddenly begun rejecting metric s***tons of email for (seemingly) bogus DMARC 
failures within the last week or so.
-Adam

Get Outlook for Android

Cloudflare human?

[Adam Thompson]

If anyone from Cloudflare lurks here, could you please reach out to me 
directly?  My messages to peering@ about one specific long-standing issue (and 
only this issue) are going unanswered with no explanation why.
Thanks,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

RE: Simplified BGP peering solution

[Adam Thompson]

4 upstream ISP connections (2 commercial, 1 NREN, 1 partial feed from HE)
1 local IX (MBIX), so 5 more ASNs there (including HE, above) plus all the 
open-peering ASes accessible through the IX route servers
3 downstream customers with their own ASNs

The two commercial ISPs are because end-user internet access here is the usual 
telco/cableco near-total-duopoly, and those two %$#@!!s only peer ~30ms away 
from me.  (Interestingly, I have one ILEC but two cablecos, geographically 
separated, within my service area.)
The NREN should be self-evident… that’s usually over 1/3 of my bandwidth, in 
fact, thanks to CANARIE’s CDN network.
MBIX is to cover all the non-duopoly ISPs in the province.  (Guess who doesn’t 
show up at the IX… first infinity guesses don’t count.)

So… I actually am an end-user trying to load balance.  But I’m also looking for 
the best paths.  And I have three POPs now where I meet other carriers 
delivering L2 circuits.  Finally, I’m an ISP but not for the general public.

I don’t think you can categorize the customer type effectively based on either 
the peering counts OR the number of “ISPs” they deal with.  (Not to mention, 
you need to define “ISP” in order for this question to make sense in the first 
place.)

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of Josh 
Saul
Sent: Friday, February 4, 2022 8:02 PM
To: nanog@nanog.org
Subject: Simplified BGP peering solution

How many active ISPs are most of the people on this list dealing with?

1-2 - I'm an end user just trying to load balance
3-5 - I'm aggressively looking for the best paths for my "customer" traffic
6-20 - I have a meet-me POP room or a specific business need for so many 
connections
21+  - I'm an ISP

Thank you!

RE: Long hops on international paths

[Adam Thompson]

Peering connection, I think, can explain this.
With some notable exceptions (all of whom participate here, I think), carriers 
still don’t throw around 100G+ peering routers around like sprinkles on a 
donut.  (Even those big networks don’t, it just looks like they do because 
they’re freaking huge.)
I suspect what you may be seeing is NOT “international carriers all concentrate 
on a single router” at all, but rather a) “large carriers tend to interconnect 
at only a few points” and b) “the best path from North America to Europe will 
therefore tend to always go through the same small set of inter-AS peering 
routers on this side”.
What you’ve described, purely in the public-visible layer 3 internet, is normal 
in my experience.  I would fully expect upwards of 90% of my daily traffic 
crosses one of 3 peering routers in my network, no surprise there, but ALSO I 
estimate at least 80% of it crosses one of no more than 5 or 6 routers even as 
I get 5, 6, 7, 8 hops away from me.  The internet isn’t as diversely-pathed as 
it once was.
In the MPLS carrier case, I’m aware of a couple that use MPLS to tunnel peering 
traffic from their edge back to a centralized “core” router that speaks BGP.  
Not sure how common this is, I have a very small sample set.  But in this 
example, no matter how diverse the carrier is at L1/L2, your L3 investigation 
will always hit the same much-smaller set of routers.

To directly answer your question, the cost/benefit is driven by the fact that 
running BGP is (relatively) complicated, error-prone and expensive, compared to 
not running BGP.  And those routers running BGP are (broadly speaking) the 
routers controlling inter-carrier traffic.  So the “chokepoints” naturally 
occur as an emergent property of each carrier controlling their own operational 
and financial risk.  Very much depends on the carrier and their operational 
philosophy.  I know nearly nothing about Telia, but Zayo doesn’t (didn’t? did 
they get bought?) have a lot of peering routers – they were historically more 
of an L2 and/or Private-network operator, as I understand it.  (I was never a 
customer, so that’s hearsay.)
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of PAUL 
R BARFORD
Sent: Tuesday, January 18, 2022 8:49 AM
To: davidbass...@gmail.com
Cc: nanog@nanog.org
Subject: Re: Long hops on international paths

Hello David,

Understanding the physical topology of the network is not​ our objective.  What 
we're trying to understand is the logical topology revealed by traceroute (we 
are well-aware of traceroute limitations) and why a relatively small set of 
routers in different countries tend to have the majority of the international 
connections.  Our expectation was that layer 3 connectivity revealed in 
traceroute to be relatively evenly spread out along coasts and near submarine 
landing points.  We're not seeing that.  So, the question is what is the 
cost/benefit to providers to configure/maintain routes (that include long MPLS 
tunnels) that tend to concentrate international connectivity at a relatively 
small number of routers?

Regards, PB

From: davidbass...@gmail.com<mailto:davidbass...@gmail.com> 
mailto:davidbass...@gmail.com>>
Sent: Tuesday, January 18, 2022 8:22 AM
To: PAUL R BARFORD mailto:p...@cs.wisc.edu>>
Cc: morrowc.li...@gmail.com<mailto:morrowc.li...@gmail.com> 
mailto:morrowc.li...@gmail.com>>; 
nanog@nanog.org<mailto:nanog@nanog.org> 
mailto:nanog@nanog.org>>
Subject: Re: Long hops on international paths

I think a large part of your problem is that you’re using trace route to try 
and determine the full topology of a large complex network.  It won’t show the 
full topology.

On Mon, Jan 17, 2022 at 7:43 PM PAUL R BARFORD 
mailto:p...@cs.wisc.edu>> wrote:
What we're considering specifically are consecutive (layer 3) hops as 
identified by traceroute.  Thus, TTL is decremented by 1 and no more than 1 
(i.e., we have to get full information (not *) from consecutive hops to 
consider the link).  I have asked my colleague to put together a set of 
examples.  We assume that there are multiple layer 1 and 2 links, and possibly 
layer 3 hops masked from traceroute by MPLS.  But what we're seeing in terms of 
hops exposed by traceroute make it look like a single (TTL decremented by 1) 
hop.

I'll post the examples when I get them.

PB

From: morrowc.li...@gmail.com<mailto:morrowc.li...@gmail.com> 
mailto:morrowc.li...@gmail.com>>
Sent: Monday, January 17, 2022 5:13 PM
To: PAUL R BARFORD mailto:p...@cs.wisc.edu>>
Cc: Pengxiong Zhu mailto:pzhu...@ucr.edu>>; 
nanog@nanog.org<mailto:nanog@nanog.org> 
mailt

Useful ping targets for end-users?

[Adam Thompson]

Before you start reading, yes, I fully understand how silly this question is.  
But I need to give _something_ to a customer who has the ability to run 
ping/traceroute but nothing else.  (And they have an intermittent latency 
problem that we haven’t been able to isolate yet.)

Does anyone curate a list of “useful” ICMP responders that are at least 
kinda-sorta reliable/expected to continue responding?  For example, all the 
major anycast DNS cloud providers respond to ICMP, but I don’t really want to 
tell my customer to ping an anycast IP address because the RTT results will be 
useless data (for comparative purposes).
I’m also not excited about providing random router IP address for what should 
be obvious reasons.  There are some IPs that my routing paths that should be 
stable, but between routing changes and control-plane policing, those aren’t 
awesome.  I’m looking for IPs I can suggest that are well outside my network.

Restatement: yes, there are much better ways to diagnose problems, but my 
customer can only run ping & traceroute (and pathping, I suppose) and is 
capable enough to run those tools and self-assess before calling me.

It sounds foolish to even ask, but maybe there’s a resource out there I don’t 
know about…
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

RE: SRv6 Capable NOS and Devices

[Adam Thompson]

My question is, why do you think you need Segment Routing at all?  Is your 
network so enormously large and/or complex that IS-IS (and/or MPLS-TE) isn't 
capable of handling it?
So far, SR looks like a solution in search of a problem, at least to me.
I'm not saying you don't have a need for it, but I am questioning whether you 
do, or whether you're just being sucked in by all the latest sizzle (i.e. sales 
& marketing materials).  (After all, that's what the sizzle is *designed* to 
do!)
-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

> -Original Message-
> From: NANOG  On
> Behalf Of Colton Conor
> Sent: Tuesday, January 11, 2022 9:17 AM
> To: NANOG 
> Subject: SRv6 Capable NOS and Devices
> 
> I know the SRv6 is a fairly new technology. I am wondering which
> vendors and network operating systems fully support SRv6 today? Has
> anyone deployed this new technology?
> 
> If building a greenfield regional ISP network, would SRv6 be a requirement?
> 
> My understanding is that because it's using IPv6 in the dataplane, not
> all devices have to have SRv6 enabled. The in-between core devices
> just have to support IPv6, but not necessarily support SRv6. This is
> much different than traditional MPLS networks today where all devices
> have to support MPLS/LDP correct?

RE: BGP Route Monitoring

[Adam Thompson]

Most monitoring products allow you to monitor custom SNMP OIDs, and your entire 
BGP RIB is – usually – exposed via SNMP.
Most monitoring products also treat “missing” OIDs specially, and can alert on 
that fact.
At least, that’s how I would start doing it.
We use Observium here, and it can do what you want, albeit with a little bit of 
futzing around in the Custom OID and Alerts sections.

Cisco does weird things with getting SNMP data from VRFs, though, so… YMMV.  I 
know there used to be a Cisco-proprietary way to select which VRF you were 
polling common OIDs from, but don’t remember the details.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[MERLIN]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of 
Sandoiu Mihai
Sent: Thursday, January 6, 2022 4:35 AM
To: nanog@nanog.org
Subject: BGP Route Monitoring

Hi

I am looking for a route monitoring product that does the following:
-checks if a specific bgp route from a specific neighbor is present the BGP 
table (in some vrf, not necessarily internet routed vrf) of an ASR9K running 
IOS XR
-sends a syslog message or an alarm if the route goes missing

The use case is the following: we are receiving same routes over 2 or more bgp 
peerings, due to best route we cannot really see at the moment if one of the 
routes ceased to be received over a certain peering.

Alternative approach: a product that measures the number of bgp received 
prefixes from a certain peer.

Do you know of such product that is readily available and does not require ssh 
sessions to the routers and parsing the outputs?
I am trying to find a solution that does not require much scripting or 
customization.

Many thanks.

Regards
Mihai

RE: IP tracking system

[Adam Thompson]

> This may have been asked and answered, but I couldn’t find the answer.

The answer changes every year, anyway.

> What are people recommending these days for IP tracking systems? I’m
> looking for something to track the used/available IP addresses in my new
> lab.

FWIW, we use TeemIP/iTop, although I can't say I would recommend it, it does 
work.

We've looked at NetBox, which is quite nice, but getting our data *out* of 
TeemIP/iTop proved to be much more difficult than expected, so we haven't 
migrated yet.  CANARIE (Canadian NREN) and all the subsidiary RANs here are 
adopting NetBox, AFAIK.

-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

RE: anyone use fbtracert successfully?

[Adam Thompson]

Thank you!!  Some of those tools are proving much more useful for me than 
fbtracert.  (In particular, traceflow has been updated recently enough that it 
“just works” in common environments that have Python3.  And while it may not be 
perfect, it’s good enough to show what I need.)
-Adam
(who apparently has lost the skills needed to Google usefully, in his 
decrepitude)

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: Hugo Slabbert 
Sent: Thursday, November 25, 2021 10:39 AM
To: Thomas Scott 
Cc: Adam Thompson ; nanog 
Subject: Re: anyone use fbtracert successfully?

What about some other options?

https://paris-traceroute.net/
https://dublin-traceroute.net/
https://github.com/rucarrol/traceflow

--
Hugo Slabbert


On Wed, Nov 24, 2021 at 9:54 AM Thomas Scott 
mailto:mr.thomas.sc...@gmail.com>> wrote:
Ha, my apologies, I thought I was writing this for a Linux User Group, not a 
NOG. Ignore my simplistic explanations.
- Thomas Scott | mr.thomas.sc...@gmail.com<mailto:mr.thomas.sc...@gmail.com>


On Wed, Nov 24, 2021 at 12:47 PM Thomas Scott 
mailto:mr.thomas.sc...@gmail.com>> wrote:
I have used it successfully in a test environment that I was using ECMP in. 
Most of the public networks that I've worked with don't use ECMP as often as 
other methods for steering traffic (LAGs, BGP MEDs, etc).

What I have seen it fantastically useful for was troubleshooting a transit 
provider, or for when they were congested or had a flapping core link. Granted 
I think it's still subject to ICMP deprioritization (most SP's use it 
prodigiously), and most MPLS cores don't decrement TTL, but it was still useful 
to be able to show them "no, at this IP, I always drop traffic, when..."

- Thomas Scott | mr.thomas.sc...@gmail.com<mailto:mr.thomas.sc...@gmail.com>


On Wed, Nov 24, 2021 at 12:23 PM Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
The tool fbtracert (http://github.com/facebookarchive/fbtracert) was mentioned 
here recently as a way to get visibility into multi-pathing.
Has anyone here ever used this tool successfully?

Supposedly Facebook uses this tool internally, but… that doesn’t help much.

I’ve tried it on 4 different platforms/OSes (WSL Ubuntu; RedHat; Debian; 
OpenBSD), and versions of Go (v1.10 through v1.16), in three very different 
environments (on-prem public IP; on-prem NAT’d; cloud public IP), and I’ve yet 
to see it produce any meaningful output – each run/iteration/thread only 
detects one, single, hop out of the entire chain of routers, making it less 
than useful.  Granted, that’s not a full regression test by any means, but if 
anyone here has ever used it successfully, could you please let me know what 
sort of environment you ran it in/on?

Thanks,
-Adam

Adam Thompson
Consultant, Infrastructure Services

100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

anyone use fbtracert successfully?

[Adam Thompson]

The tool fbtracert (http://github.com/facebookarchive/fbtracert) was mentioned 
here recently as a way to get visibility into multi-pathing.
Has anyone here ever used this tool successfully?

Supposedly Facebook uses this tool internally, but… that doesn’t help much.

I’ve tried it on 4 different platforms/OSes (WSL Ubuntu; RedHat; Debian; 
OpenBSD), and versions of Go (v1.10 through v1.16), in three very different 
environments (on-prem public IP; on-prem NAT’d; cloud public IP), and I’ve yet 
to see it produce any meaningful output – each run/iteration/thread only 
detects one, single, hop out of the entire chain of routers, making it less 
than useful.  Granted, that’s not a full regression test by any means, but if 
anyone here has ever used it successfully, could you please let me know what 
sort of environment you ran it in/on?

Thanks,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

Re: Validating multi-path in production?

[Adam Thompson]

The problem I'm looking to solve is the logical opposite, I think: I want to 
demonstrate that no links are malfunctioning in such a way that packets on a 
certain path are getting silently dropped.  Which has some "proving a negative" 
aspects to it, unfortunately.
I think the only way I can demonstrate it is to determine that every single 
multi-path/hashed-member link is working, which is... hard.  Especially if I 
need to deal with the combinatoric explosion - I *think* I can skip that part.
-Adam

Get Outlook for Android<https://aka.ms/AAb9ysg>

From: James Bensley 
Sent: Sunday, November 14, 2021 5:29:25 AM
To: Adam Thompson ; nanog 
Subject: Re: Validating multi-path in production?

On Fri, 12 Nov 2021 at 16:54, Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
The best I've come up with so far is to have two test systems (typically VMs) 
that use adjacent IP addresses and adjacent MAC addresses, and test both 
inbound and outbound to/from those, blindly trusting/hoping that hashing 
algorithms will probably exercise both paths.

If the goal is to test that traffic *is* being distributed across multiple 
links based on traffic headers, then you can definable roll your own. I think 
the problem is orchestrating it (feeding your topology data into the tool, 
running the tool, getting the results out, and interpreting the results etc).

A coupe of public examples:
https://github.com/facebookarchive/UdpPinger
https://www.youtube.com/watch?v=PN-4JKjCAT0

If you do roll your own, you need to taylor the tests to your topology and your 
equipment. For example, you can have two VMs as you mentioned, each at opposite 
ends of the network. Then, if your network uses a 5-tuple for ECMP inside the 
core for example, you could send many flows between the two VMs, rotating the 
sauce port for example, to ensure all links in a LAG or all ECMP paths are used.

It's tricky to know the hashing algo for every type of device you have in your 
network, and for each traffic type for each device type, if you have a multi 
vendor network. Also, if your network carries a mix of IPv4, IPv6, PPP, MPLS L3 
VPNs, MPLS L2 VPNs, GRE, GTP, IPSEC, etc. The number of permutations of tests 
you need to run and the result sets you need to parse, grows very rapidly.

Cheers,
James.

Validating multi-path in production?

[Adam Thompson]

Hello all.
Over time, we've run into occurrences of both bugs and human error, both in our 
own gear and in our partner networks' gear, specifically affecting multi-path 
forwarding, at pretty much all layers: Multi-chassis LAG, ECMP, and BGP MP.  
(Yes, I am a corner-case magnet.  Lucky me.)

Some of these issues were fairly obvious when they happened, but some were 
really hard to pin down.

We've found that typical network monitoring tools (Observium & Smokeping, not 
to mention plain old ping and traceroute) can't really detect a hashing-related 
or multi-path-related problem: either the packets get through or they don't.

Can anyone recommend either tools or techniques to validate that multi-path 
forwarding either is, or isn't, working correctly in a production network?  I'm 
looking to add something to our test suite for when we make changes to critical 
network gear.  Almost all the scenarios I want to test only involve two paths, 
if that helps.

The best I've come up with so far is to have two test systems (typically VMs) 
that use adjacent IP addresses and adjacent MAC addresses, and test both 
inbound and outbound to/from those, blindly trusting/hoping that hashing 
algorithms will probably exercise both paths.

Some of the problems we've seen show that merely looking at interface counters 
is insufficient, so I'm trying to find an explicit proof, not implicit.

Any suggestions?  Surely other vendors and/or admins have screwed this up in 
subtle ways enough times that this knowledge exists?  (My Google-fu is usually 
pretty good, but I'm striking out - maybe I'm using the wrong terms.)

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

Re: PCH Peering Survey 2021

[Adam Thompson]

Question: if I have a written contract with a peer that covers the link and IP 
service in general, but that contract does not specifically discuss BGP or 
peering, is that a Yes or No?
Also, how should I indicate "unknown" , particularly for the Written Contract 
field?
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG on behalf of Bill Woodcock
Sent: Friday, October 29, 2021 06:47
To: NANOG
Subject: PCH Peering Survey 2021

Background:

Five and ten years ago PCH conducted comprehensive global surveys 
characterizing Internet peering agreements. They are the only ones of their 
kind, and are relied upon by legislators and regulators throughout the world to 
understand the Internet interconnection landscape.

Our write-ups of the prior surveys can be found here:

https://www.pch.net/resources/Papers/peering-survey/PCH-Peering-Survey-2011.pdf

https://www.pch.net/resources/Papers/peering-survey/PCH-Peering-Survey-2016/PCH-Peering-Survey-2016.pdf

…and video of the NANOG presentation of the 2016 results is here:

https://www.youtube.com/watch?v=lPuoBmxyXMc

At the time of the 2011 survey, we committed to repeating the survey every five 
years, to provide time-series data about the direction peering trends take. 
We’re now conducting the third iteration of the survey.

Among other things, the surveys have helped establish a better understanding of 
trends in:

• The increasingly uniform global norms of interconnection
• National preferences within the network operator community for country of 
governing law
• The long tail of small networks which don’t yet support IPv6 routing
• The significance of multilateral peering agreements in the density of the 
interconnection mesh

These findings, particularly the first, have been invaluable in giving 
regulators in the vast majority of the world’s countries a data-driven basis 
for refraining from prescriptively regulating Internet interconnection. They 
have demonstrated in objective terms that the Internet self-regulates in a way 
that’s more globally uniform and closely harmonized than any coordination of 
national regulatory bodies could accomplish.

Participation:

The survey is global in scope, and our goal is to reflect the diversity of 
peering agreements in the world. Your participation ensures that your norms and 
ways of doing business are represented accurately and proportionately in the 
dataset. If you don’t participate, the way you do business will be less 
well-represented in the data, and will seem less normal to regulators and 
policy-makers. We’re interested in large ISPs and small ISPs, ISPs in 
Afghanistan and in Zimbabwe, bilateral agreements and multilateral, private and 
public. Our intent is to be as comprehensive as possible. In 2011, the 
responses we received represented 4,331 networks in 96 countries, or 86% of the 
world’s ISPs at that time. In 2016, responses represented 10,794 networks in 
148 countries, or 60% of the world’s ISPs in 2016. Our aim is to be equally 
inclusive this year.

Since our principal method of soliciting participation is via NOG mailing 
lists, I’m afraid many of you will see this message several times, on different 
lists, for which we apologize.

Privacy:

In 2011 and 2016, we promised to collect the smallest set of data necessary to 
answer the questions, to perform the analysis immediately, and not to retain 
the data after the analysis was accomplished. In that way, we ensured that the 
privacy of respondents was fully protected. We did as we said, no data was 
leaked, and the whole community benefited from the trust that was extended to 
us. We ask for your trust again now as we make the same commitment to protect 
the privacy of all respondents, using the same process as was successfully 
employed the last two times. We are asking for no more data than is absolutely 
necessary. We will perform the analysis immediately upon receiving all of the 
data. We will delete the data once the analysis has been performed.

The Survey:

We would like to know your Autonomous System Number, and the following five 
pieces of information relative to each other AS you peer with:

• Your peer’s ASN (peers only, not upstream transit providers or downstream 
customers)
• Whether a written and signed peering agreement exists (the alternative being 
a less formal arrangement, such as a "handshake agreement")
• Whether the terms are roughly symmetric (the alternative being that they 
describe an agreement with different terms for each of the two parties, such as 
one compensating the other, or one receiving more or fewer than full customer 
routes)
• Whether a jurisdiction of governing law is defined
• Whether IPv6 routes are being exchange

Re: Increase bandwidth usage in partial-mesh network?

[Adam Thompson]

Hah, no not your client .  Their existing network is actually surprisingly 
stable, but it is bandwidth-constrained.  As well as the various other replies 
I've seen here and off-list (THANKS!), the only commercial product I've found 
so far that might have a hope of handling this is HPE/Aruba's Silverpeak line.  
We'll see what else comes out of the woodwork, though - if nothing else, it's a 
very interesting exercise!

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: Fletcher Kittredge 
Sent: October 13, 2021 12:59
To: Adam Thompson 
Cc: nanog 
Subject: Re: Increase bandwidth usage in partial-mesh network?


Hey! From the description it must be one of our clients!

Just beware if you go this route, a network that is probably already unstable 
and unreliable will become at least an order of magnitude worse. You can't fix 
ten lbs of stuff into a 4 lb stuff bag. The internet protocols do not tolerate 
congestion well.


On Wed, Oct 13, 2021 at 1:31 PM Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
Looking for recommendtions or suggestions...

I've got a downstream customer asking for help;  they have a private internal 
network that I've taken to calling the "partial-mesh network from hell": it's 
got two partially-overlapping radio networks, mixed with islands of isolated 
fiber connectivity.
Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select 
the _best_ path, they won't spread the load unless all paths are equal - and 
they are very unequal in this network, ECMP would likely fail horribly.
The network is becoming bandwidth-limited, so they're wanting to make use of 
all available paths, not just the single "best" path.  It's also remote and 
spread out, so adding new links or upgrading existing links is difficult and 
expensive.
Oh, and their routers are overdue for a refresh, so acquiring replacement h/w 
is now possible.

Has anyone come across any product or technology that can handle the 
multi-path-ness and the private-network-ness like a regular router, but also 
provides the intelligent per-flow path steering based on e.g. latency, like an 
SD-WAN device (and/or some firewalls)?

Here's hoping,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


--
Fletcher Kittredge
GWI
207-602-1134
www.gwi.net<http://www.gwi.net>

Increase bandwidth usage in partial-mesh network?

[Adam Thompson]

Looking for recommendtions or suggestions...

I've got a downstream customer asking for help;  they have a private internal 
network that I've taken to calling the "partial-mesh network from hell": it's 
got two partially-overlapping radio networks, mixed with islands of isolated 
fiber connectivity.
Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select 
the _best_ path, they won't spread the load unless all paths are equal - and 
they are very unequal in this network, ECMP would likely fail horribly.
The network is becoming bandwidth-limited, so they're wanting to make use of 
all available paths, not just the single "best" path.  It's also remote and 
spread out, so adding new links or upgrading existing links is difficult and 
expensive.
Oh, and their routers are overdue for a refresh, so acquiring replacement h/w 
is now possible.

Has anyone come across any product or technology that can handle the 
multi-path-ness and the private-network-ness like a regular router, but also 
provides the intelligent per-flow path steering based on e.g. latency, like an 
SD-WAN device (and/or some firewalls)?

Here's hoping,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

Re: [External] Re: uPRF strict more

[Adam Thompson]

IMHO, no, it's not worth it... at least, not unless you have a larger budget 
than mine, a larger department than mine, and possibly more skilled operators 
than I am.

I don't even grok how this is supposed to work - the only place I "peer" in the 
classical sense is my local IX; all my other "peers" are ALSO either downstream 
or upstream networks for me.  (e.g. my NREN regional affiliate, who is a 
lateral peer for many prefixes, but also an upstream access network to reach 
the national, and then global, REN[s])
If a router doesn't have a default route, and also doesn't have full tables, I 
can't use it for downstream customers even if they're BGP peers; they're 
expecting me to either provide full tables, or act as a default gateway.

Do people in other parts of the world have access (both physical and logical) 
to enough bilateral peering (and budgets...) that it makes sense to deploy a 
router per peer?

For the NREN case, it's not full tables, and it's not default routes, but it's 
still a pretty big table.   And they're the location of the "triangle" routing 
where I have several downstream clients who also peer directly with them.  I'm 
both a lateral "peer" AND a downstream customer to them... so they tried to 
turn on uRPF on the L3 interfaces towards me and, well, bad things happened to 
our mutual customers' traffic.  Admittedly this was triggered by the downstream 
customer doing questionable things with different-length prefixes, but the fact 
remains uRPF causes (sometimes partial) outages anywhere you have multi-path 
"downstream" clients.
And based on the topology, I cannot conceive of any set of ACLs that I could 
feasibly apply to inbound traffic on the peering link with my NREN affiliate, 
which makes it... more difficult to be BCP/MARNS-compliant.  Commercial traffic 
regularly transits R unexpectedly, and vice-versa: path asymmetry is common 
here.

-Adam


P.S. the topology in question was as simple as this.  Cust. advertised /N to 
me, but /N+1 to the R side, so traffic started taking an unanticipated detour 
via the R side.  IRR/RPKI does not solve this: it was a legitimate 
advertisement.
​ ┌─┐ ┌───┐
   ┌►│MRnet├►│R│
   │ └─┘ └───┘
   │▲
   ││
 ┌─┴───┐ ┌──┴───┐┌──┐
 │Cust.├►│MERLIN├───►│Commercial│
 └─┘ └──┘└──┘


Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of Randy 
Bush 
Sent: October 1, 2021 12:28
To: Mark Tinka 
Cc: North American Network Operators' Group 
Subject: Re: [External] Re: uPRF strict more

> A partial table with no default is perfectly fine for a peering router.
>
> As long as your peering router knows how to get to your prefixes and
> those of your customers, as well as the prefixes of the networks you
> peer with, you're good to go.

in fact, this seems to be the modern conservative style for some years.
i sometimes wonder if it is worth the config pain.

randy

Re: uPRF strict more

[Adam Thompson]

Yeah, but loose mode is inherently useless on any router carrying full tables.  
(Ok, it can spot bogons, but that's a side effect and I have other ways to 
catch those.)
Point being that MANRS implementation in the "simple" case is (or, at least, 
CAN be) almost trivially easy, but in the "complex" case is quite difficult - 
I'm still not even sure I know how to do it 100% correctly with multi-homed 
downstreams clients.  "Just turn on RPF"  is starting to feel more like an 
article of faith rather than genuine technical guidance.  :-(
-Adam

Get Outlook for Android<https://aka.ms/AAb9ysg>

From: Brian Johnson 
Sent: Friday, October 1, 2021 8:31:15 AM
To: Adam Thompson 
Cc: Amir Herzberg ; Randy Bush ; North 
American Network Operators' Group 
Subject: Re: uPRF strict more

For strict-mode... Completely agree.

As has been previously said, this is a tool that all players involved need to 
understand. This is no different than everyone correctly using BGP in their 
application for their outcomes.

On Sep 29, 2021, at 12:07 PM, Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:

We just ran into a typical case where uRPF caused a partial outage for one of 
my customers: the customer is multi-homed, with another provider that I'm also​ 
connected to.  Customer advertised a longer-prefix to the other guy, so I 
started sending traffic destined for Customer to the Other Provider... who then 
promptly dropped it because they had uRPF enabled on the peering link, and they 
were seeing random source IPs that weren't mine.  Well... yeah, that can happen 
(semi-legitimately) anytime you have a topological triangle in peering.

I've concluded over the last 2 years that uRPF is only​ useful on interfaces 
pointing directly at non-multi-homed customers, and actively dangerous anywhere 
else.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG 
mailto:nanog-bounces+athompson=merlin.mb...@nanog.org>>
 on behalf of Amir Herzberg mailto:amir.li...@gmail.com>>
Sent: September 28, 2021 20:06
To: Randy Bush mailto:ra...@psg.com>>
Cc: North American Network Operators' Group 
mailto:nanog@nanog.org>>
Subject: Re: uPRF strict more

Randy, great question. I'm teaching that it's very rarely, if ever, used (due 
to high potential for benign loss); it's always great to be either confirmed or 
corrected...

So if anyone replies just to Randy - pls cc me too (or, Randy, if you could sum 
up and send to list or me - thanks!)

Amir
--
Amir Herzberg

Comcast professor of Security Innovations, Computer Science and Engineering, 
University of Connecticut
Homepage: https://sites.google.com/site/amirherzberg/home
`Applied Introduction to Cryptography' textbook and lectures: 
https://sites.google.com/site/amirherzberg/applied-crypto-textbook<https://sites.google.com/site/amirherzberg/applied-crypto-textbook>




On Tue, Sep 28, 2021 at 8:50 PM Randy Bush 
mailto:ra...@psg.com>> wrote:
do folk use uPRF strict mode?  i always worried about the multi-homed
customer sending packets out the other way which loop back to me;  see
RFC 8704 §2.2

do vendors implement the complexity of 8704; and, if so, do operators
use it?

clue bat please

randy

Re: uPRF strict more

[Adam Thompson]

We just ran into a typical case where uRPF caused a partial outage for one of 
my customers: the customer is multi-homed, with another provider that I'm also​ 
connected to.  Customer advertised a longer-prefix to the other guy, so I 
started sending traffic destined for Customer to the Other Provider... who then 
promptly dropped it because they had uRPF enabled on the peering link, and they 
were seeing random source IPs that weren't mine.  Well... yeah, that can happen 
(semi-legitimately) anytime you have a topological triangle in peering.

I've concluded over the last 2 years that uRPF is only​ useful on interfaces 
pointing directly at non-multi-homed customers, and actively dangerous anywhere 
else.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of Amir 
Herzberg 
Sent: September 28, 2021 20:06
To: Randy Bush 
Cc: North American Network Operators' Group 
Subject: Re: uPRF strict more

Randy, great question. I'm teaching that it's very rarely, if ever, used (due 
to high potential for benign loss); it's always great to be either confirmed or 
corrected...

So if anyone replies just to Randy - pls cc me too (or, Randy, if you could sum 
up and send to list or me - thanks!)

Amir
--
Amir Herzberg

Comcast professor of Security Innovations, Computer Science and Engineering, 
University of Connecticut
Homepage: https://sites.google.com/site/amirherzberg/home
`Applied Introduction to Cryptography' textbook and lectures: 
https://sites.google.com/site/amirherzberg/applied-crypto-textbook<https://sites.google.com/site/amirherzberg/applied-crypto-textbook>




On Tue, Sep 28, 2021 at 8:50 PM Randy Bush 
mailto:ra...@psg.com>> wrote:
do folk use uPRF strict mode?  i always worried about the multi-homed
customer sending packets out the other way which loop back to me;  see
RFC 8704 §2.2

do vendors implement the complexity of 8704; and, if so, do operators
use it?

clue bat please

randy

Re: Never push the Big Red Button (New York City subway failure)

[Adam Thompson]

Now I'm curious... in all of the DCs and COs I've worked in - to the best of my 
knowledge, I haven't personally tested this! - the EPO button does not​ switch 
to emergency power.  It turns off ALL equipment power in the space - no lights, 
no klaxons, nothing.  In simpler setups, the EPO is connected to the UPS so 
anything plugged in to the UPS does dark instantly.  In one DC I'm familiar 
with, the EPO switch kills all the UPS output and​ uses several relays to kill 
commercial power at the same time.
In some, the room lights were not covered by the EPO switch, in some they were. 
 Emergency exit lamps will continue to be lit, as they have internal batteries, 
and are required by building/fire code.

Is it (somewhat) common for an EPO switch to only disconnect commercial power 
and leave local redundant power live?  What sort of facilities would have this?

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of Jay 
R. Ashworth 
Sent: September 11, 2021 22:23
To: nanog 
Subject: Re: Never push the Big Red Button (New York City subway failure)

- Original Message -
> From: "Sean Donelan" 

> NEW YORK CITY TRANSIT RAIL CONTROL CENTER POWER
> OUTAGE ISSUE ON AUGUST 29, 2021
> Key Findings
> September 8, 2021
>
> https://www.governor.ny.gov/sites/default/files/2021-09/WSP_Key_Findings_Summary-for_release.pdf
>
> Key Findings
> [...]
>
> 3. Based on the electrical equipment log readings and the manufacturer’s
> official assessment, it was determined that the most likely cause of RCC
> shutdown was the “Emergency Power Off” button being manually activated.

I don't even *do* datacenter for a living, and I know that when you hit the
Molly button,

1) A Klaxon goes off in the Data Center -- one that sounds *different* from
the Halon Klaxon, in both cadence and tone (just for a couple bursts), and

2) Yellow rotating beacons turn on, and stay on while you're on Emergency Power.

Yes, real honest-to-ghod *rotating mechanical beacons*, none of this flashing 
LED
crap.

Clearly, it's important that the use of Emergency Power be annoyingly 
noticeable.

Cheers,
-- jra
--
Jay R. Ashworth  Baylink   j...@baylink.com
Designer The Things I Think   RFC 2100
Ashworth & Associates   http://www.bcp38.info  2000 Land Rover DII
St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647 1274

Re: Newbie Questions: How-to monitor/control unauthorized uses of our IPs and DNS zones?

[Adam Thompson]

I just had a conversation with John Curran (of ARIN) about this, in fact...

You don't own IP addresses.  But you also don't rent IP addresses, either.

IP addresses are not a thing, good, or object, not even an intangible good.  
They are an address, or an index, if you will.  (You might think of an IP 
address as the index on a giant, internet-wide, shared array... that we call 
"the routing table".)
Your annual fee purchases registration services, specifically, the service of 
ARIN entering your IP addresses into their master copy of a database that other 
people use.  (And some ancillary services that ARIN provides to you.)  That's 
it.

The closest analogy I have are either phone numbers or street addresses.  You 
don't own either of those things, nor do you rent them.  In the case of phone 
numbers, the phone company isn't renting you the phone#, they're renting you 
the POTS service that gives you the ability to make outgoing, and answer 
incoming, calls.  Your ILEC also typically adds your name and # into a phone 
book, as part of the service.  (Yeah, VoIP providers have mangled this analogy 
beyond recognition.)  They can (and have) changed your phone number at will.  
At least ARIN doesn't do that.

In the case of a street address, you own the property.  The address is just an 
index to a giant, irregular 2D array called "the streets in your city".  Again, 
when you buy or rent the property, you aren't buying or renting the address 
itself from anyone, much less the city.  But there are all sorts of directories 
("databases") you can register your business in so that people know who 
occupies such-and-such a property, and marketing folks do this all the time 
(even in 2021).  When you pay those companies money, you aren't renting the 
property from them, you're registering​ your property with them.

Here's the problematic part: there's absolutely nothing saying you have to 
register your addreses with an RIR to get them into the global routing table.  
You could probably find an ISP somewhere willing to overlook all the rules and 
conventions and advertise new address space that just happens to overlap with 
someone else's registered addresses, or maybe you found some that aren't 
currently advertised.  In fact, I'd say it's 100% possible to do so.

However, nearly everyone agrees to play by a common set of rules, in order that 
the Internet, well... works.  As expected.  Almost 100% of the time, taken as a 
whole.  Those rules include requiring you to register with an RIR, to ensure 
there are no overlaps, and law enforcement can find you if necessary.

Again, you aren't buying or renting IP addresses - you're paying an admission 
fee of sorts, in order to play in the global routing table.  The fact your RIR 
assigned you a block of addresses is part of good internet governance, and is 
not actually the commercial aspect of the transaction (even though we all think 
of it that way anyway, including me).

Ultimately, almost everyone thinks of it the way you do, but it's technically 
quite wrong.  (My statements may not be correct in jurisdictions deriving from 
systems other than English common law.)

Beyond this, this is a discussion for ARIN-DISCUSS not NANOG-L.  Or perhaps in 
your case, whatever discussion list APNIC runs, since ARIN rules don't apply in 
Thailand.  But I expect APNIC will tell you almost the same thing as I just did.

-Adam

P.S. If you feel this is B.S. and it shouldn't work this way, most of the RIRs 
are always looking for participants in their policy process - I know ARIN is.  
Well, I don't know what's up with AfriNIC, that unfortunately seems to be a 
rolling dumpster fire, but I suppose they'll need new people to put the pieces 
all back together, too.

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of 
Pirawat WATANAPONGSE via NANOG 
Sent: August 19, 2021 13:32
To: nanog@nanog.org 
Subject: Re: Newbie Questions: How-to monitor/control unauthorized uses of our 
IPs and DNS zones?

Huh.
And I thought that I did lay down information (and questions) pretty clearly, 
but as you correctly pointed out, I didn't.
So, here goes the second version:

Background Information Section (v2):
We are a Registrant and already registered a zone/domain with a Registry, we 
are also a LIR and have been allocated an IP block straight from RIR.
[What I meant to say is that they all keep saying that we don’t “own” those 
resources and we also have to pay the annual fee so, even though we are a 
Registrant and a LIR, it’s still practically a form of rent anyway.]
We DNSsec-sign and host both forward and reverse zones ourselves, with NSEC3 to 
prevent zone enumeration.
We register our IP

Re: PeerinDB refuses to register certain networks [was: Setting sensible max-prefix limits]

[Adam Thompson]

I have an example locally: BellMTS (ASNs 684, 7122, 4398), the local ILEC.
To the best of my knowledge, they only peer with downstream customers 
(including myself) and their sole upstream, Bell Canada (AS577).  Meanwhile 
that's a ~700k eyeball network (with some hosting, sure), roughly ~400Gbps 
upstream connectivity, and no public peering whatsoever.  In fact, one might 
describe their peering model as "feudal", where they're subjugate to their 
corporate overlord (Bell Canada).
It's unfortunate, I know there are some smart people working there, but either 
they don't understand the value of sub-1ms access to root nameservers (*cough* 
MBIX *cough*), or they're prevented from doing anything about it.

[Disclaimer: I'm on the MBIX board.  But I also used to work for MTS, and tried 
to setup the first peering relationship but got shot down for "marketing" 
reasons, something about "legitimizing the competition".  Very monopolistic 
thinking, IMO.]

Meanwhile, MTS still has a PeeringDB  record, even though it documents quite 
nicely the fact that perhaps that record shouldn't exist, or at least doesn't 
need to.

FWIW, their upstream, Bell Canada, is a very different story.  And also mostly 
~8msec away.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of Eric 
Kuhnke 
Sent: August 19, 2021 10:32
To: Ben Maddison ; nanog@nanog.org list 

Subject: Re: PeerinDB refuses to register certain networks [was: Setting 
sensible max-prefix limits]

I agree with you in the utility of that, but sort of as a side topic...

I wonder how many ASes are out there that have any significant volume of 
traffic/multi-site presences, but are exclusively 100% transit customers, do 
not have any PNIs at major carrier hotels, and are not members of any IX.

What would be a good example of such an AS and how big of a network would it 
be? Undoubtedly there are some enterprise end user type customers set up like 
that, but I can't imagine they receive a very large volume of unsolicited 
peering requests.

On Thu, Aug 19, 2021 at 6:32 AM Ben Maddison via NANOG 
mailto:nanog@nanog.org>> wrote:
Hi Patrick,

On 08/18, Patrick W. Gilmore wrote:
> > Of course! Including headers to show authenticity. I was very amused by the
> > explanation of the "chicken and egg" problem. Who's creating that? The 
> > networks
> > who refuse to peer with non-peeringdb registered ASNs, or peeringdb who 
> > won't
> > recognize ASNs that are not peering with anyone because nobody wants to peer
> > with them because they are not registered in peeringdb because nobody wants 
> > to
> > peer with them? You get the idea.
>
> First, most networks do not require a PDB record to peer. (Silly of
> them, I know, but still true.)
>
> Second, you do not need to have a PDB record to get a link to an IXP.
> Even membership in a free IXP is sufficient for an account in PDB, as
> Grizz points out below.
>
> Third, if you have an agreement, even just an email, saying a network
> will peer with you once you have a record, that may well suffice. Have
> you asked any network to peer? Private peering (because you are not on
> an IXP) is usually reserved for networks with more than a modicum of
> traffic. If your network is large enough to qualify for private
> peering, I have trouble believing you cannot get another network to
> agree to peer so you can get a record.
>
> I guess you are right, the _Peering_DB does not register “certain”
> networks. Those networks would be ones that do not peer. Which seems
> pretty obvious to me - it is literally in the name.
>
A PDB record for an Internet-connected ASN, listing no IXPs or
facilities, but with a note saying approximately "We only use transit,
and don't peer" has some utility: it saves prospective peers from
finding contacts to ask and sending emails, etc.

I'd argue this is in scope for PDB. But perhaps there was additional
context to the original decision that I'm missing?

Cheers,

Ben

Re: "Tactical" /24 announcements

[Adam Thompson]

Yes, it is bad practice.  Yes, it's polluting the route table.
If the # of /24s involved is not ridiculously large (say, <64?) them I would go 
ahead, as long as IRR and/or RPKI are also updated.
Obviously if everyone did it (i.e. advertising /24s exclusively) then our FIBs 
would collectively balloon to a grotesquely un-manageable size, at least on 
platforms that can't auto-aggregate that back down.  Thankfully, everyone isn't 
doing it.
I, too, would vastly prefer no-one did this, but I have two customers that 
demand it from time to time... and we've even done it for our own allocation 
sometimes, and there's no robust, never mind bullet-proof, technical argument 
why I can't do that for them (or for ourselves).  OTOH robust arguments exist 
for why it's a good thing to do - sometimes, and temporarily.
¯\_(ツ)_/¯
-Adam


Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of Billy 
Croan 
Sent: August 9, 2021 10:47
To: nanog list 
Subject: "Tactical" /24 announcements

How does the community feel about using /24 originations in BGP as a
tactical advantage against potential bgp hijackers?

All of our allocations are larger and those prefixes we announce for
clients as well usually are.  But we had a request recently to
originate everything as distinct /24 prefixes, to reduce the effect of
a potential bgp hijack.  It seemed a little bit like a tragedy of the
commons situation.

Is this seen as route table pollution, or a necessary evil in today's world?
How many routers out there today would be affected if everyone did this?
Are there any big networks that drop or penalize announcements like this?

Re: Anycast but for egress

[Adam Thompson]

Without any sarcasm: to make it harder to block.
If, say, Google, always crawled your site from 8.8.1.2 (random made-up example) 
then you would see a not-insignificant number of hosts and networks 
null-routing that IP.  I have no idea why someone would do so, but I've seen it 
done many times.  Mostly by people who don't understand how un-special they are 
on the internet.  Also it would trigger IDS/IPS systems all over the place, 
having gobs and gobs of connections coming from a single IP.

That's setting aside the technical issues involved; routing is often 
asymmetric, i.e. the return packet takes a different path than the inbound 
packet.  So it would, as Owen implied, be nearly impossible to ensure the reply 
packets got back to the correct TCP stack.  As an example, I'm multi-homed and 
use path-prepending, so if a packet claiming to be from 8.8.8.8 arrived on one 
of my commercial links, I would send the reply out the cheapest link, which in 
my case is a flat-rate R network (that has a path to Google), thus ensuring 
the reply does not get to the originating anycast node.

When my clients make connections outbound to anycast addresses, the destination 
is more-or-less stable, and the replies come back to the client's unique IP, so 
anycast works in that direction.  The guarantees are not present in the reverse 
direction.

The logical extremity of this is that it would be nearly impossible for two 
anycast addresses to establish a TCP connection to each other.  (In general.  
There will be lots of local cases where it does happen to work, by coincidence.)

You'll find that even anycast nodes do not make connections outbound using 
their anycast address, pretty much for these reasons.

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of Vimal 

Sent: July 27, 2021 12:54
To: nanog@nanog.org 
Subject: Anycast but for egress

(Unsure if this is the right forum to ask this question, but here goes:)

>From what I understand, IP Anycast can be used to steer traffic into a server 
>that's close to the client.

I am curious if anyone here has/encountered a setup where they use anycast IP 
on their gateways... to have a predictable egress IP for their traffic, 
regardless of where they are located?

For example, a search engine crawler could in principle have the same IP 
advertised all over the world, but it looks like they don't...  I wonder why?

--
Vimal

Re: 1G/10G BaseT switch recommendation

[Adam Thompson]

While acknowledging that some people love Rucks for legitimate reasons, our 
experience with them can be summed up as "never again".  YMMV.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of Jörg 
Kost 
Sent: July 22, 2021 14:39
To: Drew Weaver 
Cc: nanog@nanog.org 
Subject: Re: 1G/10G BaseT switch recommendation

Ruckus ICX 7650-48ZP has

- 24x 1GB RJ45
- 24x 2.5/5/10G RJ45
- stacking
- uplink 100G | 40G | 10G uplink module
- BGP, OSPF, ACL

https://de.commscope.com/product-type/enterprise-networking/ethernet-switches/icx7650


On 22 Jul 2021, at 21:29, Adam Thompson wrote:

> If you've already looked at Cisco, Juniper, Extreme, Juniper, and
> Arista, that's the big ones.  Everything else is increasingly niche
> vendors.
> -Adam
>
>
> Adam Thompson

RE: 1G/10G BaseT switch recommendation

[Adam Thompson]

True.  I forget carrier space often, these days.

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: Saku Ytti 
Sent: Thursday, July 22, 2021 2:34 PM
To: Adam Thompson 
Cc: Drew Weaver ; nanog@nanog.org
Subject: Re: 1G/10G BaseT switch recommendation

On Thu, 22 Jul 2021 at 22:32, Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote

If you've already looked at Cisco, Juniper, Extreme, Juniper, and Arista, 
that's the big ones.  Everything else is increasingly niche vendors.

Extreme is a mom and pop shop compared to Nokia and Huawei, and I guess quite 
selection of names.

--
  ++ytti

Re: 1G/10G BaseT switch recommendation

[Adam Thompson]

If you've already looked at Cisco, Juniper, Extreme, Juniper, and Arista, 
that's the big ones.  Everything else is increasingly niche vendors.
-Adam


Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  on behalf of Drew 
Weaver 
Sent: July 22, 2021 13:46
To: 'nanog@nanog.org' 
Subject: 1G/10G BaseT switch recommendation


Hello everyone,



I’m looking for recommendations from the community on 48x10G RJ45/4-6 SFP28 
(uplink ports) switches that people actually like working with.



Features are VPC or non-vendor specific equivalent, L2/L3 BGP/OSPFv3, ACLs, 
functional CoPP and some sort of API to manage them. [the CLI would work, my 
lib can handle most Networking OS CLIs anyway]



My problem point is coming from the RJ45 requirement, most vendors have one 
switch that they sell that is RJ45 at 10G or at the most one in each line 
(enterprise/datacenter) and they seem to be almost an afterthought. [probably 
because SFP28 is better in every way if you are already using fiber at the 
endpoint] sadly, we are not.



I just want to make sure I am not excluding any vendors from my research.



I appreciate any suggestions or recommendations. Can even keep it off-list if 
you want.



Thanks,

-Drew

Re: A survey on BGP MRAI timer values in practice

[Adam Thompson]

My question at this point is, what slow global convergence?  When I (or any of 
my downstreams) adjusts a prefix, I nearly always see global propagation in 
well under 60 seconds.  Among networks where I can check, at least.
I understand it could be technically possible to see near-instantaneous global 
convergence in more like <5sec, but... on a global scale, neither I nor my 
customers care about the difference between 5s and 60s.  Do other people need 
<60s propagation?
-Adam

From: NANOG  on behalf of Mark 
Tinka 
Sent: June 10, 2021 02:36
To: nanog@nanog.org 
Subject: Re: A survey on BGP MRAI timer values in practice



On 6/10/21 08:26, Saku Ytti wrote:

> I don't understand the question, but the way I read the question it
> may be unanswerable even if I did understand it. As the reader would
> self-define negligible and well acceptable and answer yes/no based on
> the definition they used, which might be different to the definition
> writer intended.

It's possible we've become accustom to a slow, global BGP, due to a
perception of fragility (and complexity), which favours stability over
speed.

I suppose the size of the current BGP and the nature of the FSM it lends
itself to does some to account for those perceptions.

At a per-ISP level, it is not impossible to speed up (i)BGP convergence.
On a global scale, taking the least common denominator to allow for all
manner of network we don't know about, allowing the ship a wide turn in
BGP waters, at least on a perceptive level, seems like an unsigned
social agreement amongst autonomous systems.

Ultimately, I feel we aren't talking enough about this, and hopefully,
this thread gets us to that point.

Mark.

Re: A survey on BGP MRAI timer values in practice

[Adam Thompson]

+1 to Saku's concerns - I simply ignored the survey because I wasn't sure what 
MRAI was, and I wasn't sure what my values would be.  But I have time to be 
interested right now, so a-spelunking I go...

The term "MRAI" does not appear anywhere in Arista's or Extreme's 
documentation.  Nor does this timer interval appear in any BGP-related "show" 
output, on any of my platforms, that I can see.

I've found out that "out-delay" (not "delay out") is synonymous with MRAI and 
seems widely used.

I found "out-delay" in an Arista technote, and now I know how to override it on 
Aristas, and the default is zero (0).  Unfortunately, "show" commands on EOS 
will only show you the current out-delay if it is non-zero, which makes 
reporting it a bit difficult.

Extreme's MLXe platform doesn't appear to support an out-delay/MRAI knob at 
all, at least as far as I can tell.  I know there are several other current and 
former MLXe operators here, maybe one of them will know?

Based on my limited history with NANOG-L, I guess your initial email might have 
been seen by perhaps 20 people who immediately knew what you were talking 
about, and 2000 who didn't.  (I don't actually know subscriber numbers, that's 
totally a WAG.  And maybe more people have touched out-delay knobs than I 
think.  Dunno.)

 This illustrates the gap between academia 
and industry - academics can research a narrow topic, and come at issues from a 
theoretic standpoint using unfamiliar terminology.  As a practitioner, I get 
told to add carrier X as a peer by end-of-day Friday, and "just make it work".  
I wasn't even able to understand your initial question, because I haven't spent 
a semester understanding the intricacies of BGP propagation - I just know it 
usually works, knobs exist that I shouldn't fiddle with, and that's good enough 
for my job. 

If your work results in actionable recommendations such as "don't use BGP 
out-delay timers to mitigate XYZ in circumstance LMNO, do ABC instead", that's 
fantastic.  Please keep us advised, and do post aggregated survey results here 
once you close the survey.

I am specifically interested in the answer to "Have you ever had to adjust BGP 
out-delay with any of your peers, and why?"  It would be great if we could 
derive that answer from the survey results, but anecdotal replies here would 
also be helpful.  All you larger(-than-me) network operators out there: when 
would I need to use out-delay?  Why?  What does it accomplish?

Good luck in reformulating your survey to get better engagement,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of Saku 
Ytti 
Sent: June 8, 2021 01:06
To: shahr...@cs.umass.edu 
Cc: nanog list ; Arun Venkataramani 
Subject: Re: A survey on BGP MRAI timer values in practice

On Mon, 7 Jun 2021 at 19:32,  wrote:

> We often read that the Internet (i.e. BGP) has a long convergence delay.
> But why is it so slow? And can we (researchers) do anything about it?

Create business incentives to improve it. This is a non-technical
problem, we've long had technical tools to make it fast, there just
isn't incentive to make it fast. Customers are not asking operators
for better convergence speeds.

> Please help us out to find out by answering our short anonymous survey
> (<10 minutes).

Can you tell me what have you done so far? What are the default MRAI
values for each AFI/SAFI for IOS, IOS-XR, Junos, SROS, VRP and EOS?
Then people responding don't have to check what their NOS does, they
can refer to your table and tell the default value, since this is what
>99% will be using.

Now your survey has built-in selection bias, people who answer it are
people who know what it is and who are concerned about it and have
changed it, this is not a representative group and you will start your
work with very bad data.



--
  ++ytti

Re: MPLS/MEF Switches and NIDs

[Adam Thompson]

EXOS is a perfectly good OS that bears absolutely no resemblance to anything 
else you've ever used in your career.  If you start from scratch without 
training courses, you're looking at wasting 6 months (maybe more) just learning 
the OS well enough to figure out how to configure your desired deployment.  
Then you get to the usual month or so of fine-tuning required that every 
product needs.

Given how many companies will pay for training nowadays, it's a relevant 
concern IMHO.

Now that I'm used to EXOS, I like it.  But I would never recommend an EXOS 
newbie start a project with a product that runs EXOS, without some jump-start 
training.  It really is/was that painful.  It's like giving a 100% Windows 
admin a UNIX box to get the new service running on, with no training.  (Or 
vice-versa.)

NOTE: Extreme's goal (supposedly targeting 2021) was to ship one common 
hardware platform that could run any of their 3 OSes.  I don't know if they're 
achieved it, but generally speaking, for any EXOS box, there's two more 
products, one running the Nortel/Avaya OS and one IronWare (Foundry/Broadcom), 
both of which are fairly "normal".

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: Colton Conor 
Sent: May 31, 2021 15:30
To: Adam Thompson 
Cc: NANOG 
Subject: Re: MPLS/MEF Switches and NIDs

Adam.

When you say "Beware using any EXOS-based product (anything that starts with 
"X") unless you're already familiar with EXOS!" Are you saying stay away from 
this line completely, or what do you mean by this statement. I have heard good 
things about Extreme for deploying service provider G.8032 and MPLS functions.

Yes, I was aware of https://www.mef.net/certify/technology-registry/ and have 
gone through pretty much every vendor looking at their solutions. Extreme for 
example is not listed at all, so I guess they didn't want to pay those fees! 
There are quite a few Chinese vendors we can't use.


On Mon, May 31, 2021 at 12:44 PM Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
Extreme has excellent MEF implementations.  I've never used their MPLS 
implementations, but it's definitely there on, I think, all their products.  I 
only have the X620 model in my network, which may or may not work for you.  
Beware using any EXOS-based product (anything that starts with "X") unless 
you're already familiar with EXOS!  I cannot emphasise this enough!
Extreme's other product lines come from Nortel/Avaya and Broadcom heritage, and 
also have good MEF implementations (and more-or-less-sane OSes).  They have 
MPLS support, but again, no experience with it.
I can't give much advice on pricing as I get both edu & gov discounts, but they 
are competitive with Arista and Cisco when we go to RFP.

Also, Juniper's MX (and maybe PTX?) families support MEF if that's a hard 
requirement.  I know some but not all EX switches have had both MEF and MPLS, 
too.  Beware many EX models have pretty minimalist MPLS implementations (e.g. 
no VPLS).  Agreed on their pricing, though, which is why I don't have any .  
But for 4x10G the MX104 is a very nice box - if you can afford it.

Lastly, have you seen https://www.mef.net/certify/technology-registry/ ?

-Adam


Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG 
mailto:merlin.mb...@nanog.org>> 
on behalf of Colton Conor 
mailto:colton.co...@gmail.com>>
Sent: May 26, 2021 11:39
To: NANOG mailto:nanog@nanog.org>>
Subject: MPLS/MEF Switches and NIDs

For MPLS and MEF switches, I know Juniper, Cisco, and Nokia are commonly talked 
about on this list. However, I was wondering if anyone has evaluated other 
brands? We are not interested in looking at chinese based vendors, so ZTE and 
Huawei are not an option. Anyone else worth looking into?

We have used Juniper's ACX line primarily, but there is a big gap in their 
product line. The ACX2200 has only two 10G ports. The next jump up from there 
is the ACX710 with 24 10G ports. They have nothing in between that has 4-12 10G 
ports. Not to mention, Juniper is very proud price wise. We are looking for 
cost efficient 10G NIDs with at least 4 10G ports on them and aggregation boxes 
with at least 12 10G ports on them with 25g/100G uplinks.

Ciena seems to have multiple options available with Segment Routing, MPLS, and 
streaming telemetry support. I am probably most interested in what Ciena has to 
offer. Has anyone deployed the 3000 or 5000 product line o

Re: MPLS/MEF Switches and NIDs

[Adam Thompson]

Extreme has excellent MEF implementations.  I've never used their MPLS 
implementations, but it's definitely there on, I think, all their products.  I 
only have the X620 model in my network, which may or may not work for you.  
Beware using any EXOS-based product (anything that starts with "X") unless 
you're already familiar with EXOS!  I cannot emphasise this enough!
Extreme's other product lines come from Nortel/Avaya and Broadcom heritage, and 
also have good MEF implementations (and more-or-less-sane OSes).  They have 
MPLS support, but again, no experience with it.
I can't give much advice on pricing as I get both edu & gov discounts, but they 
are competitive with Arista and Cisco when we go to RFP.

Also, Juniper's MX (and maybe PTX?) families support MEF if that's a hard 
requirement.  I know some but not all EX switches have had both MEF and MPLS, 
too.  Beware many EX models have pretty minimalist MPLS implementations (e.g. 
no VPLS).  Agreed on their pricing, though, which is why I don't have any .  
But for 4x10G the MX104 is a very nice box - if you can afford it.

Lastly, have you seen https://www.mef.net/certify/technology-registry/ ?

-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of 
Colton Conor 
Sent: May 26, 2021 11:39
To: NANOG 
Subject: MPLS/MEF Switches and NIDs

For MPLS and MEF switches, I know Juniper, Cisco, and Nokia are commonly talked 
about on this list. However, I was wondering if anyone has evaluated other 
brands? We are not interested in looking at chinese based vendors, so ZTE and 
Huawei are not an option. Anyone else worth looking into?

We have used Juniper's ACX line primarily, but there is a big gap in their 
product line. The ACX2200 has only two 10G ports. The next jump up from there 
is the ACX710 with 24 10G ports. They have nothing in between that has 4-12 10G 
ports. Not to mention, Juniper is very proud price wise. We are looking for 
cost efficient 10G NIDs with at least 4 10G ports on them and aggregation boxes 
with at least 12 10G ports on them with 25g/100G uplinks.

Ciena seems to have multiple options available with Segment Routing, MPLS, and 
streaming telemetry support. I am probably most interested in what Ciena has to 
offer. Has anyone deployed the 3000 or 5000 product line of Ciena? How does it 
compare to Juniper? The Ciena 3924 is sub $1000 for example, and has 4 10G 
ports on it.

Adva has quite a few options as well, but I don't think their routing stack is 
as strong as Ciena's.

Tejas was an unknown player to me, but they seem to have a couple of options 
that fit the bill. Price wise, I have heard the run circles around everyone.

RAD has some options, but their pricing looks much higher than Ciena.

Accedian looked interesting, but it seems they don't make aggregation switches, 
only NIDs.

ECI Telecom / Ribbon seems to have some options, but I have not talked to them.

What does Nokia and Cisco have in this space, and price wise is it going to 
compare to these less known vendors?

RE: Juniper hardware recommendation

[Adam Thompson]

At least it isn’t Arista, where SVI egress counters are disabled by default, 
and once enabled count everything UNLESS the packet egresses via a LAG!  Talk 
about being “impactful”, we’re having to buy new routers to insert behind them, 
just to count packets so we can bill accurately, and for that matter, have 
traffic graphs that work at all.  :-(

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of 
Michael Fiumano
Sent: Friday, May 14, 2021 12:06 PM
To: nanog@nanog.org
Subject: RE: Juniper hardware recommendation

If accurate interface stats are important to you, MX’s don’t support accurate 
SNMP Interface Utilization, ie they don’t comply with RFC2665/3635, which seems 
like a fairly basic thing to do but they decided not to, and has been impactful 
to me in the past.  So, any SNMP monitoring of an interface will always show 
less utilization than what is actually occurring, possibly leading to a false 
sense of security, or delay in augmentation.  Would also affect usage based 
billing, if you do that.

https://www.juniper.net/documentation/us/en/software/junos/network-mgmt/topics/topic-map/snmp-mibs-and-traps-supported-by-junos-os.html

For M Series, T Series, and MX Series, the SNMP counters do not count the 
Ethernet header and frame check sequence (FCS). Therefore, the Ethernet header 
bytes and the FCS bytes are not included in the following four tables:

ifInOctets

ifOutOctets

ifHCInOctets

ifHCOutOctets


Thanks,
Michael Fiumano

From: NANOG On Behalf Of Mark Tinka
Sent: Monday, May 10, 2021 10:25 AM
To: nanog@nanog.org<mailto:nanog@nanog.org>
Subject: Re: Juniper hardware recommendation


On 5/10/21 16:19, aar...@gvtc.com<mailto:aar...@gvtc.com> wrote:
I prefer MX204 over the ACX5048.  The ACX5048 can’t add L3 interface to an mpls 
layer 2 type of service.  There are other limitations to the ACX5048 that cause 
me to want to possibly replace them with MX204’s.  But in defense of the 
ACX5048, we have gotten some good mileage (a few years now) of good resi/busi 
bb over vrf’s and also carrier ethernet for businesses and lots of cell 
backhaul… so they are good for that.  I’ve heard the ACX5448 was even better.

Trio will always provide better features, but come with the price tag to boot.


I’m looking at the MX240 for the SCB3E MPC10E hefty with 100 gig ports

You might want to look at the MX10003, in that case, as well. We are deploying 
those for 100Gbps service (customer-facing). Works out cheaper than offering 
100Gbps service on the MX240/480/960 for the same task.

Mark.

RE: Juniper hardware recommendation

[Adam Thompson]

I did not know such a thing existed!  Cool!  Holy murdering your port density, 
though.  Ouch$$$.

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

> -Original Message-
> From: Nick Hilliard 
> Sent: Friday, May 14, 2021 9:40 AM
> To: Adam Thompson 
> Cc: nanog@nanog.org
> Subject: Re: Juniper hardware recommendation
> 
> Adam Thompson wrote on 14/05/2021 14:30:
> > However, the MX 10k family still only shows as being compatible with
> > two QSFP cards.  And yes, you can get a QSFP-SFP+ breakout cable, but
> > those don't let you use SFP+ CWDM/DWDM transceivers.
> you can also get QSA adapters to convert from a QSFP form factor port to a
> SFP+ port.  This should allow SFP+ WDM transceivers.
> 
> Nick

RE: Juniper hardware recommendation

[Adam Thompson]

OK, enough people have pointed it out :-).

Clearly I was wrong about the MX 2K family, I missed the SFP+ MIC completely.  
That is good to know.

However, the MX 10k family still only shows as being compatible with two QSFP 
cards.  And yes, you can get a QSFP-SFP+ breakout cable, but those don't let 
you use SFP+ CWDM/DWDM transceivers.

-Adam

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

> -Original Message-
> From: Bjørn Mork 
> Sent: Saturday, May 8, 2021 6:32 AM
> To: Adam Thompson 
> Cc: Javier Gutierrez Guerra ; nanog@nanog.org
> Subject: Re: Juniper hardware recommendation
> 
> Adam Thompson  writes:
> 
> 
> >   * Skip the MX 2k/10k series – they don’t support SFP+ interfaces!
> 
> https://apps.juniper.net/hct/model/?component=MX2K-MPC6E
> https://apps.juniper.net/hct/model/?component=MIC6-10G
> 
> 
> Bjørn

RE: Juniper hardware recommendation

[Adam Thompson]

Hi, Javier!
MX series: Full-featured – sings, dances, walks the cat, etc. But painful 
racking (as you noted).  Very nice and comprehensive boxes otherwise.  
Interfaces are more expensive, but often modular and wider variety.
EX/QFX series: Nice switches, OK L3 routers.  Lots of limitations in MPLS and 
various other corner-case limitations.

My personal opinion:

  *   Skip the MX480 (and up), it’s just too expensive.  Consider an EX9200 
instead, which can do 90% of the same functions.  (If you can afford an MX480 
or MX960, by all means, get one!)
  *   MX240 is reasonable, but dated.  A pair of MX204s in HA would make more 
sense, to me.
  *   Skip the MX 2k/10k series – they don’t support SFP+ interfaces!  (“No 10G 
WDM for you!”)  Also no 1G, you need a separate step-down switch for that.  I 
don’t know what SP Juniper thinks they’re targeting with these.
  *   1U/2U EX/QFX are reasonable edge devices as long as you’ve verified they 
can do what you need.  Not core-router class IMHO.
  *   If you don’t already know that you want a PTX, then you don’t want a PTX. 
 The product is fine, but niche, and has the same interface limitations as 
MX10k.
  *   ACX: MEF-compliant mini-MX, basically.  Edge device only, pairs well with 
an MX480 (IIRC).  Top-end are exceptions: ACX5k/7k might work, depending on 
what you need it to do.  Not normally deployed as a core router.

My experience is that you never fill up an EX9208 or MX480 chassis, but the 
MX240 is too small.  YMMV.  MX480 line cards are stupid expensive compared to, 
well, everything else.

I’m probably out-of-date on some (or much) of my knowledge, let’s see what 
everyone else here has to say!

-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of 
Javier Gutierrez Guerra
Sent: Friday, May 7, 2021 3:55 PM
To: nanog@nanog.org
Subject: Juniper hardware recommendation

Hi,
Just out of curiosity, what would you recommend using for a core router/switch 
from Juniper?
MX208,480,10K
Datasheets show them all as very nice and powerful devices (although they do 
use a lot of rack space and side to side airflow is painful) but I’m just 
wondering here what most people use and how good or bad of an experience you 
have with it 
Thanks,

Javier Gutierrez Guerra
Network Analyst
CCNA R, JNCIA
Westman Communications Group
Phone: 204-717-2827
Email: guer...@westmancom.com<mailto:guer...@westmancom.com>
[WCG_Corp_Logo_horiz_cFullcolorHR]<http://westmancom.com/personal/>

[cisco-certified-network-associate-routing-and-switching-ccna-routing-and-switching]

RE: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

[Adam Thompson]

LOLOLOL.
“%VXLAN-4-IPV6_UNDERLAY_UNSUPPORTED: VXLAN encapsulation using IPv6 VTEP 
addresses is not supported on this platform”
Guess it’s going to be a non-issue for me, at this time, since VxLAN was the 
main reason for this entire setup…
Thanks for all the responses!
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of Adam 
Thompson
Sent: Tuesday, May 4, 2021 10:29 AM
To: Saku Ytti 
Cc: nanog 
Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

I don't believe APIPA and Link-Local are precisely equivalent, but I agree it's 
the closest thing IPv4 has.  IS-IS/IPv4 would presumably use APIPA addresses if 
nothing else were assigned to the interface, based on my reading of the RFC.  
I'm unsure what the RFC authors think should happen in a HELLO packet when the 
interface has multiple IPv4 addresses, but none of that is my problem here.

I don't like LLAs because they are - intrinsically - meaningless.  In the 
context of my L3 core, I know that for any subnet, .1/::1 is such-and-such a 
router, .2/::2 is that one, .3/::3, is the other one, etc., etc.  (Yes, I have 
a very small & topologically simple L3 core.  Let's not talk about L2!)  When I 
look at my IPv4 routing table, I know which next-hop is which just by looking 
at it, and I can spot anomalies very easily.

When I look at my IPv6 routing table, the next-hops are all... well... 
gibberish, at least to me.  My experience is that LLAs are not durable, so 
memorizing them is not IMHO a useful task.  Figuring out an (IS-IS) IPv6 route 
currently involves a couple of extra steps to locate the LLA's interface route, 
find the MAC address of that LLA on that link, and then identify the router 
from its MAC address.

Am I missing something obvious?

Thanks!
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: Saku Ytti mailto:s...@ytti.fi>>
Sent: May 4, 2021 10:20
To: Adam Thompson mailto:athomp...@merlin.mb.ca>>
Cc: Mark Tinka mailto:mark@tinka.africa>>; nanog 
mailto:nanog@nanog.org>>
Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

On Tue, 4 May 2021 at 18:15, Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:

Hey Adam,

> I don't see any rationale in RFC 5308 for why the HELLO packet may only 
> contain the LLA - does anyone know/remember why?  (I'm hoping that 
> understanding the rationale will make this an easier pill to swallow.)  
> Obviously this behaviour/requirement is net-new to the IPv6 TLVs, as there's 
> no LLA-cognate in IPv4 (APIPA doesn't count).  There is in OSI, I think, but 
> I'm still too sane to read those docs.

IPv4 link local is 169.254/16, you may have seen them in Windows.

> It makes sense that you would not want LLAs in LSPs, only GUAs, but does that 
> imply that you must use either ULAs or GUAs in order to establish IPv6 routes 
> in IS-IS, in an IPv6 environment?  That makes about as much sense to me as 
> forcing LLAs for next-hops.

The list may benefit from the context you have, it is not obvious to
me why you'd want anything but LLA.

--
  ++ytti

Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

[Adam Thompson]

I don't believe APIPA and Link-Local are precisely equivalent, but I agree it's 
the closest thing IPv4 has.  IS-IS/IPv4 would presumably use APIPA addresses if 
nothing else were assigned to the interface, based on my reading of the RFC.  
I'm unsure what the RFC authors think should happen in a HELLO packet when the 
interface has multiple IPv4 addresses, but none of that is my problem here.

I don't like LLAs because they are - intrinsically - meaningless.  In the 
context of my L3 core, I know that for any subnet, .1/::1 is such-and-such a 
router, .2/::2 is that one, .3/::3, is the other one, etc., etc.  (Yes, I have 
a very small & topologically simple L3 core.  Let's not talk about L2!)  When I 
look at my IPv4 routing table, I know which next-hop is which just by looking 
at it, and I can spot anomalies very easily.

When I look at my IPv6 routing table, the next-hops are all... well... 
gibberish, at least to me.  My experience is that LLAs are not durable, so 
memorizing them is not IMHO a useful task.  Figuring out an (IS-IS) IPv6 route 
currently involves a couple of extra steps to locate the LLA's interface route, 
find the MAC address of that LLA on that link, and then identify the router 
from its MAC address.

Am I missing something obvious?

Thanks!
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: Saku Ytti 
Sent: May 4, 2021 10:20
To: Adam Thompson 
Cc: Mark Tinka ; nanog 
Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

On Tue, 4 May 2021 at 18:15, Adam Thompson  wrote:

Hey Adam,

> I don't see any rationale in RFC 5308 for why the HELLO packet may only 
> contain the LLA - does anyone know/remember why?  (I'm hoping that 
> understanding the rationale will make this an easier pill to swallow.)  
> Obviously this behaviour/requirement is net-new to the IPv6 TLVs, as there's 
> no LLA-cognate in IPv4 (APIPA doesn't count).  There is in OSI, I think, but 
> I'm still too sane to read those docs.

IPv4 link local is 169.254/16, you may have seen them in Windows.

> It makes sense that you would not want LLAs in LSPs, only GUAs, but does that 
> imply that you must use either ULAs or GUAs in order to establish IPv6 routes 
> in IS-IS, in an IPv6 environment?  That makes about as much sense to me as 
> forcing LLAs for next-hops.

The list may benefit from the context you have, it is not obvious to
me why you'd want anything but LLA.

--
  ++ytti

Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

[Adam Thompson]

Thank you, both!

...that kinda sucks, though.

I don't see any rationale in RFC 5308 for why the HELLO packet may only contain 
the LLA - does anyone know/remember why?  (I'm hoping that understanding the 
rationale will make this an easier pill to swallow.)  Obviously this 
behaviour/requirement is net-new to the IPv6 TLVs, as there's no LLA-cognate in 
IPv4 (APIPA doesn't count).  There is in OSI, I think, but I'm still too sane 
to read those docs.

It makes sense that you would not want LLAs in LSPs, only GUAs, but does that 
imply that you must​ use either ULAs or GUAs in order to establish IPv6 routes 
in IS-IS, in an IPv6 environment?  That makes about as much sense to me as 
forcing LLAs for next-hops.

-Adam


Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of Saku 
Ytti 
Sent: May 4, 2021 01:44
To: Mark Tinka 
Cc: nanog list 
Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

On Tue, 4 May 2021 at 07:24, Mark Tinka  wrote:

> Junos:
>> 2c0f:feb0::1/128   *[IS-IS/18] 02:43:49, metric 5870
>to fe80::1205:caff:fe86:4ac3 via et-4/0/2.0
>to fe80::5287:89ff:fef3:25c3 via et-4/0/2.0
>to fe80::1205:caff:fe86:4b10 via et-5/0/2.0
> >  to fe80::5287:89ff:fef3:2610 via et-5/0/2.0
>
> IOS XE:
> I2  2C0F:FEB0::1/128 [115/6410]
>  via FE80::1205:CAFF:FE86:4AC3, TenGigabitEthernet1/0/0
>  via FE80::1205:CAFF:FE86:4B10, TenGigabitEthernet0/0/0
>  via FE80::5287:89FF:FEF3:25C3, TenGigabitEthernet1/0/0
>  via FE80::5287:89FF:FEF3:2610, TenGigabitEthernet0/0/0
>
> IOS XR:
> i L2 2c0f:feb0::1/128
>   [115/5870] via fe80::1205:caff:fe86:4b10, 02:45:22, HundredGigE0/3/0/0 
> (!)
>   [115/5810] via fe80::f60f:1bff:feb0:75c4, 02:45:22, HundredGigE0/2/0/1
>

SROS:
2001:218:0:1000::1/128Remote  ISIS  48d22h13m  18
   fe80::42de:adff:fe98:87e4-"lag1" 25301


https://tools.ietf.org/html/rfc5308

For Hello PDUs, the "Interface Address" TLV MUST
   contain only the link-local IPv6 addresses assigned to the interface
   that is sending the Hello.  For LSPs, the "Interface Address" TLVs
   MUST contain only the non-link-local IPv6 addresses assigned to the
   IS.


These are hello derived:
A:y...@a04.chcgil09.us.bb# show router isis adjacency
r22.chcgil09.us.bb-re0 detail |match Neigh
IPv6 Neighbor : fe80::42de:adff:fe98:87e4
IPv4 Neighbor : 129.250.3.205

Vendors do not have the option to use GUA while being RFC5308 compliant.


--
  ++ytti

IS-IS and IPv6 LLA next-hop - just Arista, or everyone?

[Adam Thompson]

Hey, just checking as I don’t have any Cisco or Extreme or Juniper gear running 
IS-IS to verify myself…

On current Arista (7280SR2K) and older Brocade (MLXe) routers, the IPv6 
next-hop address in IS-IS seems to always be the link-local address of the 
neighbour, instead of any manually-assigned address on the interface. I believe 
this is *not* the case with Cisco, in particular.  Not sure about other 
vendors, I don’t have any handy that can run IS-IS.  I don’t appear to have any 
knobs to control this behaviour, and haven’t found anything related in the docs.

Can anyone confirm that Cisco or other vendors does/do not do this (prefer LLA 
over GUA)?

Thanks,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

Re: Zayo or HE for IP transit

[Adam Thompson]

IMHO:

  *   Zayo = worse coverage/connectivity, adequate connectivity to other 
Tier1/Tier2, massive fiber network
  *   HE = less redundancy built into their network, but best-connected to 
leaves/edges of the internet (and still good connectivity to core)

Others may have had different experiences, but that's what I see from where I 
sit.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of James 
Lumby 
Sent: April 19, 2021 16:30
To: nanog@nanog.org 
Subject: Zayo or HE for IP transit


What is the current experience with Zayo or HE?  I’m looking at possibly adding 
one of them into a mix of cogent and a mix from my datacenter.  Would be using 
BGP full routes.  Any experiences would be appreciated.



Sincerely,

James

MS Teams orig/term carrier?

[Adam Thompson]

Despite having worked for an ILEC in network ops for 10yrs, I can't figure out 
how to answer this without an SS7 sniffer (which I obviously no longer have, 
nor access to an SSP or SCP).

What 3rd-party carrier(s), if any, does Microsoft use to originate & terminate 
MS Teams-to-PSTN voice calls in Canada?

Pointers to rabbit holes welcome, if you think there’s an actual answer at the 
end ;-).

Thanks,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

Re: Petition for Board of Trustees adoption consideration of ARIN-2020-2

[Adam Thompson]

Frankly, at this point, I'll support whatever lets me never have to hear about 
the subject again.
-Adam

(Yes, I know Outlook has filters, thank you.  In fact, I think I'll go create 
one right now.)

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: ARIN-PPML  on behalf of Jon Bachtold 

Sent: January 12, 2021 05:56
To: arin-p...@arin.net 
Subject: [arin-ppml] Petition for Board of Trustees adoption consideration of 
ARIN-2020-2

I continue to support Reinstatement ie, ARIN-2020-2,  and

I support this petition to move the policy consideration to the Board of 
Trustees.

Jon Bachtold
CIRBN

ARIN-2020-2: Reinstatement of Organizations Removed from Waitlist by 
Implementation of ARIN-2019-16



ARIN-2020-2 reverted to Draft Policy on 6 January following a Last Call period. 
Per ARIN's Policy Development Process (PDP), all Recommended Draft Policies not 
sent to the ARIN Board of Trustees for adoption consideration within 60 days of 
Last Call completion will revert to Draft Policy status.




Jon Bachtold​
Chief Technology Officer
CIRBN, LLC
Tel:309-820-7321
Email:  jcb...@cirbn.org<mailto:jcb...@cirbn.org>
Web:www.cirbn.org<https://www.cirbn.org/>
[CIRBN]<http://www.cirbn.org/>
200 W Front St, Ste 500A
Bloomington,IL  61701
[Facebook]<https://www.facebook.com/cirbn>
[LinkedIn]<https://www.linkedin.com/company/3729509/>
[CIRBN Twitter]<https://twitter.com/CIRBN_LLC>

RE: favorite network troubleshooting tools (online)

[Adam Thompson]

I see NLNOG’s IRRexplorer has been mentioned, but what about the NLNOG 
RING<https://ring.nlnog.net/> ?  There’s a publicly-reachable LG 
(lg.ring.nlnog.net) but you have to sign up for access to the rest.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of Matt 
Harris
Sent: Thursday, July 16, 2020 9:49 AM
To: Mehmet Akcin 
Cc: nanog 
Subject: Re: favorite network troubleshooting tools (online)

Not one specific tool, but I often use various large SP looking glasses to 
determine what their routing tables look like when necessary.

RIPE Atlas is another cool project that has also yet to be mentioned.

Cloudflare's RPKI tools may also be of use.


[https://netfire.net/logo_sig_gen2.png]

Matt Harris​

|


Infrastructure Lead Engineer



816‑256‑5446

|


Direct







Looking for something?

Helpdesk Portal<https://help.netfire.net/>

|


Email Support<mailto:h...@netfire.net>

|


Billing Portal<https://my.netfire.net/>



[https://netfire.net/Flag-United-States-of-America.jpg]

We build and deliver end‑to‑end IT solutions.







On Thu, Jul 16, 2020 at 9:08 AM Marcos Manoni 
mailto:marcosman...@gmail.com>> wrote:
https://stat.ripe.net/ has lots of widgets https://stat.ripe.net/widget/list
http://irrexplorer.nlnog.net/ for detailed IRR info
(https://bgp.he.net also have some)

El mié., 15 jul. 2020 a las 14:41, Mehmet Akcin 
(mailto:meh...@akcin.net>>) escribió:
>
> hey there,
>
> I recently have come across this http://ping.pe/ website, I have no 
> association with this but it's pretty awesome. This made me wonder what other 
> tools out there which I do not know about it.
>
> what are your favorite network troubleshooting tools?
>
> In addition to ping.pe<http://ping.pe>, I like https://bgp.he.net but would 
> love to hear your thought about other tool recommendations as especially the 
> ones that are distributed.
>
> Mehmet

Re: L2VPN/L2transport, Cumulus Linux & hardware suggestion

[Adam Thompson]

I do run the 7280SR2-48YC6, but I don't do VPLS or pseudowires on them right 
now so I can't help directly with that.


Based on my experience with Arista so far, it'll be perfectly-well documented, 
just for a different platform, and in a blog post instead of in the user 
manual.  :-(


(Note to anyone from Arista lurking on the list: your User Manual sucks rocks 
because it's wildly incomplete.  Please put some of the effort that goes into 
those EOS Central blog posts, into the manual instead.)


As to the Juniper, I'm a client on a Juniper-based VPLS system, and the only 
thing it consistently intercepts is LLDP... which I'm actually OK with, mostly. 
 Other BPDUs and other Ethernet protocols get passed through (that we've 
tested, so far).  We have heard of some feature limitations on the EX4650, no 
CCC is unfortunate.  I don't have any experience with the QFX series as an 
operator or customer so can't comment.


Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of 
Jürgen Jaritsch 
Sent: Tuesday, July 7, 2020 5:05:03 PM
To: nanog@nanog.org
Subject: AW: L2VPN/L2transport, Cumulus Linux & hardware suggestion

Dear Adam,

yeah, forget about LACP - the bigger problem is all the LLDP and STP stuff,
that gets interpreted at the UNI port. LACP is a bad example - but there are
many other frames and protocols, which must work. Could be that a customer
wants to run MPLS+LDP on his VLL (for whatever reason ...).

> For your requirements, although I hesitate to recommend them for
enterprise/carrier use, Miktotik's EoIP protocol does a much better job of
this than most "carrier-grade" implementations.

Not at wirespeed ... and not without causing other issues (single thread
load, etc).

> Juniper has the EX4650 that matches your h/w specs,...  Not 100% sure the
Juniper EX does 25G, now that I think of it.

Yeah, EX4650 it does: 48x 1/10/25G + 6x 100G + MPLS
It also supports Ethernet over MPLS (at least they say here:
https://www.juniper.net/documentation/en_US/junos/topics/topic-map/mpls-over
view.html#id-mpls-feature-support-on-qfx-series-and-ex4600-switches) but at
some of their sites they mention, that MPLS-based CCC are not support:
https://www.juniper.net/documentation/en_US/junos/topics/topic-map/mpls-over
view.html#jd0e2531

" ... MPLS-based circuit cross-connects (CCC) are not supported—only
circuit-based pseudowires are supported. ..."

There is also the QFX5120-48Y - 48x 1/10/25G + 8x 100G + MPLS
In the past QFX wasn't the best idea for MPLS topics ... has this changed?

> and Arista has, oh, at least half a dozen boxes of various spec that
comply, too.

Yeah, I already know them (do have some older 7050S). The call it "VXLAN P2P
Pseudowire", but there is absolutely nothing in there CLI documentation :(.
Looks like the feature is only support on the 7280 platform.

Possible options:
7280SR2-48YC6

Do you have any experience with what they call "VXLAN P2P Pseudowire"? I
can't even find a config example on the net :(


thanks & best regards
Jürgen






-Ursprüngliche Nachricht-
Von: Adam Thompson [mailto:athomp...@merlin.mb.ca]
Gesendet: Dienstag, 7. Juli 2020 23:09
An: Jürgen Jaritsch ; nanog@nanog.org
Betreff: RE: L2VPN/L2transport, Cumulus Linux & hardware suggestion

Good luck with tunnelling LACP, no matter what boxes you have - LACP has (de
facto) hard jitter requirements of under 1msec, or you'll be getting TCP
resets coming out your ears due to mis-ordered packets.

For your requirements, although I hesitate to recommend them for
enterprise/carrier use, Miktotik's EoIP protocol does a much better job of
this than most "carrier-grade" implementations.

Otherwise, Juniper and Arista both come to mind, Juniper has the EX4650 that
matches your h/w specs, and Arista has, oh, at least half a dozen boxes of
various spec that comply, too.  Not 100% sure the Juniper EX does 25G, now
that I think of it.

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only) mailto:athomp...@merlin.mb.ca
http://www.merlin.mb.ca

> -Original Message-
> From: NANOG <mailto:nanog-bounces+athompson=merlin.mb...@nanog.org> On
Behalf
> Of Jürgen Jaritsch
> Sent: Tuesday, July 7, 2020 3:15 PM
> To: mailto:nanog@nanog.org
> Subject: L2VPN/L2transport, Cumulus Linux & hardware suggestion
>
> Dear folks,
>
> have anyone already tried to run VXLAN/EVPN + “Bridge Layer 2 Protocol
> Tunneling” on Cumulus Linux as an replacement for classic MPLS
> L2VPN/VPWS (“xconnect”, l2circuit, VLL) ?
>
> I need to provide transparent Et

Re: L2VPN/L2transport, Cumulus Linux & hardware suggestion

[Adam Thompson]

If jitter were defined anywhere vis-à-vis LACP, it would be _de jure_, not _de 
facto_ as I said.

Yes, if you have *guaranteed* that TCP sessions hash uniquely to a single link 
in your network, you might be able to successfully tunnel LACP (or 
EtherChannel, or any other L1 link-bonding technique).  The last time I 
attempted to do this on my network, I discovered that guarantee wasn't nearly 
as ironclad as I expected.  I don't remember the gory details, at this remove, 
sorry.  Maybe it wasn't TCP?  Maybe it wasn't the default hashing algorithm? 
Dunno.

-Adam

On Jul. 8, 2020 00:48, Saku Ytti  wrote:
Hey Adam,

On Wed, 8 Jul 2020 at 00:11, Adam Thompson  wrote:

> Good luck with tunnelling LACP, no matter what boxes you have - LACP has (de 
> facto) hard jitter requirements of under 1msec, or you'll be getting TCP 
> resets coming out your ears due to mis-ordered packets.

Can you elaborate on this? Where is LACP jitter defined and for what
purpose? We push packets around the globe in sub 200us jitter on any
given day, so 1000us isn't for us a particularly hard goal.

Only reason why I could imagine someone would care about jitter here
is if protocol measures delay (LACP doesn't) and relies on delay to
remain static and then balances per-packet or per-byte or otherwise
between multiple links.
However we of course put all packets from given TCP session to always
same LACP interface, so from TCP session POV, each LACP is exactly a 1
interface. Per-packet balancing on LACP is possible via a special
configuration, but anyone who does it, doesn't care about reordering,
no matter of jitter, because even in very stable jitter, the paths may
be unequal length and cause reordering.

LACP hellos are sent every 1s when in fast mode with 3s keepalive,
which also isn't particularly tight. We do have customers running LACP
over MPLS pseudowires over great distances.

--
  ++ytti

RE: L2VPN/L2transport, Cumulus Linux & hardware suggestion

[Adam Thompson]

Good luck with tunnelling LACP, no matter what boxes you have - LACP has (de 
facto) hard jitter requirements of under 1msec, or you'll be getting TCP resets 
coming out your ears due to mis-ordered packets.

For your requirements, although I hesitate to recommend them for 
enterprise/carrier use, Miktotik's EoIP protocol does a much better job of this 
than most "carrier-grade" implementations.

Otherwise, Juniper and Arista both come to mind, Juniper has the EX4650 that 
matches your h/w specs, and Arista has, oh, at least half a dozen boxes of 
various spec that comply, too.  Not 100% sure the Juniper EX does 25G, now that 
I think of it.

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

> -Original Message-
> From: NANOG  On Behalf Of
> Jürgen Jaritsch
> Sent: Tuesday, July 7, 2020 3:15 PM
> To: nanog@nanog.org
> Subject: L2VPN/L2transport, Cumulus Linux & hardware suggestion
> 
> Dear folks,
> 
> have anyone already tried to run VXLAN/EVPN + “Bridge Layer 2 Protocol
> Tunneling” on Cumulus Linux as an replacement for classic MPLS L2VPN/VPWS
> (“xconnect”, l2circuit, VLL) ?
> 
> I need to provide transparent Ethernet P2P virtual leased lines to my
> customers and these have to support stuff like LLDP, STP, LACP, etc. The
> transport L2 network is not THAT big: max hops between VTEP is 4.
> 
> Anyone have suggestions for the below hardware request?
> #) 1-3U L2/L3 box
> #) 48x SFP28 / 1/10/25G
> #) 6x QSFP28 / 100G
> #) VXLAN/EVPN with L2 tunneling support
> or
> #) MPLS VPWS/l2circuit
> #) Dual PSU
> 
> 
> thanks & best regards
> Jürgen
> 

Re: Ensuring RPKI ROAs match your routing intent

[Adam Thompson]

So in the ARIN world, Krill only works with "delegated" RPKI, not "hosted" RPKI 
- do I understand that correctly?

If so, are there any plans to allow Krill's analytics and rules to monitor ARIN 
Hosted RPKI ROAs?

-Adam


Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of Alex 
Band 
Sent: Thursday, June 25, 2020 8:31:52 AM
To: Nanog
Subject: Ensuring RPKI ROAs match your routing intent

Hi everyone,

Over the last two years NLnet Labs has been working on free, open source RPKI 
software and research for the community, supported by the RIPE NCC Community 
Projects Fund, Brazilian NIR NIC.br and Asia Pacific RIR APNIC. I have an 
update that we’d like to share.

When creating a ROA in RPKI, it can have an effect on one or more BGP 
announcements, making them either Valid, Invalid or NotFound. Understanding 
what exactly determines these three states is not immediately obvious, 
especially in the beginning.

At times, this can make creating ROAs a bit of a shot in the dark. I’ve seen 
several examples in the past where an operator created a ROA in their RIR 
Portal, waited for it to be published and then checked in services like BGPMon 
or the HE BGP Toolkit to see if everything turned out as expected.

This is why, during my time at the RIPE NCC, we put a lot of work into making 
it immediately obvious what the effect of a ROA is going to be on the BGP 
announcements with your address space. Several RIRs have followed in these 
footsteps since.

I presented on this journey at NANOG 63 in 2015:
https://archive.nanog.org/meetings/abstract?id=2500

Now, in my new adventure at NLnet Labs, we’ve gotten the same team together to 
make simple, intuitive ROA management for Delegated RPKI available for 
everyone, seamlessly across RIR regions.

With Krill 0.7.1 ‘Sobremesa’ you can easily create and maintain ROAs in a user 
interface that incorporates all of the best practices and lessons learned over 
the last 10 years and monitor them in ways never before possible, such as 
through Prometheus.

Blog post with details:
http://link.medium.com/1SsTJSAvB7

All the best,

Alex

Re: Switch for SFP+

[Adam Thompson]

Have you actually looked at Mikrotik switches?  I don't like the OS, but the 
hardware does what you want it to.


https://mikrotik.com/products/group/switches?filter=c={%22sfp_plus_interface%22:{%22s%22:%223%22,%22e%22:%2224%22}}#!

If necessary, buy your SFP modules from FS.com and get them coded as Mikrotik 
modules at the factory - that's what we do for Cisco, Brocade, Juniper, 
Extreme, etc.

Even the top-of-the-line Mikrotik only costs US$899.


-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of Mauro Gasparini 

Sent: Thursday, May 14, 2020 8:46:21 AM
To: Mehmet Akcin
Cc: nanog
Subject: Re: Switch for SFP+

Thank you. The problem is that to get a price lower than U$D 3000 I have to 
resort to a used device.

El 14/5/20 a las 01:08, Mehmet Akcin escribió:
Used Juniper QFX5100-48T will do it. Probably overkill but you can grab one 
cheap @ebay

On Wed, May 13, 2020 at 16:36 Mauro Gasparini 
mailto:mjgaspar...@gmail.com>> wrote:
Good afternoon.

I'm looking for a switch with the following capabilities:
. transport for more than 20 gbps
. link aggregation LACP
. slots for SFP+
. seamlessly when trunking vlans through the link aggregation.

And essentially that doesn't exceed US$D 2000 and is compatible with
10GBASE-ER and/or 10GBASE-ZR modules that are not from the vendor itself
(e.g. SPFs: Huawei, Mikrotik, Sumitomo, OEMs).

If any of you have a good experience with a device that meets these
requirements (which are minimal with the exception of price and
compatibility) ?

Regards.
Mauro Gasparini
--
Mehmet
+1-424-298-1903

Re: xplornet contact or any experience with their satellite service?

[Adam Thompson]

Although I don't know of a way to solve this for videoconferencing, 
historically one way to mitigate the radio/vsat "batchiness" issue and its 
effect on end-to-end latency was to use a caching proxy server connected to a, 
er, "real" network somewhere, preferably as near as possible to the 
headend/uplink station.  The modern web's move to TLS also means this technique 
is becoming pointless even for HTTP/S (although MITMing remains a way around 
that - many a HOWTO abounds, describing how to do this with Squid).


FWIW, some very old radio systems behaved similarly... albeit not with 600+msec 
latency :-/.  Some of the really old asymmetric TV systems (dial-up for uplink, 
CATV for downlink) exhibited similar characteristics and were similarly 
difficult to mitigate.


Good luck!


Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


From: NANOG  on behalf of Mel Beckman 

Sent: Tuesday, April 21, 2020 9:05:20 AM
To: Brian J. Murrell
Cc: nanog@nanog.org
Subject: Re: xplornet contact or any experience with their satellite service?

Brian,

Satellite services are shared bandwidth broadcast systems. The behavior you’re 
seeing is pretty common at times when you’re competing for access with other 
users. Just like the regular Internet, there are times of day when people tend 
to move more data, and because of latency and other limitations on 
bidirectional traffic, packets get delivered in batches. It’s not possible to 
interleave bytes, or even packets themselves.

So when you see low or no throughput, it’s because the transponder is 
addressing packets to other users.

 -mel beckman

> On Apr 21, 2020, at 6:53 AM, Brian J. Murrell  wrote:
>
> A friend of mine just recently got Xplornet satellite service at his
> rural home.  I'm well aware of the latency issues with satellite
> although frankly his latency is much better than I had feared it would
> be and is around 600-700ms.
>
> But what seems to be worse than the latency is the "burstiness" of the
> traffic and I am just wondering if that is normal/expected for
> satellite service in general, and/or expected from Xplornet's service,
> or if what I am seeing is not expected at all (i.e. not an artifact of
> the satellite signal but rather a network management issue).
>
> Here's iperf3 for 30 seconds sending data (i.e. upload speed):
>
> [ ID] Interval   Transfer Bitrate
> [  5]   0.00-1.21   sec  12.9 KBytes  87.4 Kbits/sec
> [  5]   1.21-2.00   sec  6.47 KBytes  67.2 Kbits/sec
> [  5]   2.00-3.00   sec  22.0 KBytes   180 Kbits/sec
> [  5]   3.00-4.00   sec  41.4 KBytes   339 Kbits/sec
> [  5]   4.00-5.00   sec  41.4 KBytes   339 Kbits/sec
> [  5]   5.00-6.00   sec  55.6 KBytes   456 Kbits/sec
> [  5]   6.00-7.00   sec  69.9 KBytes   572 Kbits/sec
> [  5]   7.00-8.00   sec  89.3 KBytes   731 Kbits/sec
> [  5]   8.00-9.00   sec   120 KBytes   986 Kbits/sec
> [  5]   9.00-10.00  sec  86.7 KBytes   710 Kbits/sec
> [  5]  10.00-11.00  sec   133 KBytes  1.09 Mbits/sec
> [  5]  11.00-12.00  sec   184 KBytes  1.51 Mbits/sec
> [  5]  12.00-13.00  sec   186 KBytes  1.53 Mbits/sec
> [  5]  13.00-14.00  sec   159 KBytes  1.30 Mbits/sec
> [  5]  14.00-15.00  sec  0.00 Bytes  0.00 bits/sec
> [  5]  15.00-16.00  sec  0.00 Bytes  0.00 bits/sec
> [  5]  16.00-17.00  sec  93.2 KBytes   763 Kbits/sec
> [  5]  17.00-18.00  sec   264 KBytes  2.16 Mbits/sec
> [  5]  18.00-19.00  sec   124 KBytes  1.02 Mbits/sec
> [  5]  19.00-20.00  sec   157 KBytes  1.28 Mbits/sec
> [  5]  20.00-21.00  sec   120 KBytes   986 Kbits/sec
> [  5]  21.00-22.00  sec  86.7 KBytes   710 Kbits/sec
> [  5]  22.00-23.00  sec   369 KBytes  3.02 Mbits/sec
> [  5]  23.00-24.00  sec   197 KBytes  1.61 Mbits/sec
> [  5]  24.00-25.00  sec  90.6 KBytes   741 Kbits/sec
> [  5]  25.00-26.00  sec   193 KBytes  1.58 Mbits/sec
> [  5]  26.00-27.00  sec   192 KBytes  1.57 Mbits/sec
> [  5]  27.00-28.00  sec   189 KBytes  1.55 Mbits/sec
> [  5]  28.00-29.00  sec   193 KBytes  1.58 Mbits/sec
> [  5]  29.00-30.00  sec   179 KBytes  1.46 Mbits/sec
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval   Transfer Bitrate Retr
> [  5]   0.00-32.20  sec  4.41 MBytes  1.15 Mbits/sec  388 sender
> [  5]   0.00-30.00  sec  3.57 MBytes   998 Kbits/sec  receiver
>
> which averaged the overall prescribed "upload" speed, but notice that
> it's not 1Mb/s in any kind of a steady stream but rather bursts of
> higher than 1Mb/s speed followed by low/no speed.  At one point it was
> 2 seconds with no transfer at

Practical guide to predicting latency effects?

[Adam Thompson]

I’m looking for a practical guide – i.e. specifically NOT an academic paper, 
thanks anyway – to predicting the effect of increased (or decreased) latency on 
my user’s applications.

Specifically, I want to estimate how much improvement there will be in 
{bandwidth, application XYZ responsiveness, protocol ABC goodput, whatever} if 
I decrease the RTT between the user and the server by 10msec, or by 20msec, or 
by 40msec.

My googling has come up with lots of research articles discussing theoretical 
frameworks for figuring this out, but nothing concrete in terms of a calculator 
or even a rule-of-thumb.

Ultimately, this goes into MY calculator – we have the usual north-american 
duopoly on last-mile consumer internet here; I’m connected directly to only one 
of the two.  There’s a cost $X to improve connectivity so I’m peered with both, 
how do I tell if it will be worthwhile?

Anyone got anything at all that might help me?

Thanks in advance,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

RE: COVID-19 vs. peering wars

[Adam Thompson]

Worldwide, I don’t know.

In Canada, peering is pretty messed up, i.e. it simply doesn’t happen at scale. 
 At all.  Even where it should.  The overwhelmingly vast majority of Canadian 
traffic, even when nominally in-country, still transits the USA somewhere.

If we had “ideal” full-mesh peering (i.e. setting aside all commercial 
considerations) at, say, regional IXes, including various popular CDNs, then 
service would take a giant step for the better for everyone who isn’t a big-4 
(Bell, Telus, Shaw or Rogers) customer.  Which admittedly would be an 
improvement for “only” about 30%-40% of the population… negligible, really, 
we’re only a country of 10M after all :-/.

FYI, we have 4 big ISPs because none of them cover the entire country: they 
all* descend from local/regional monopolies or duopolies.   *Mostly, that’s an 
approximation.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: Matthew Petach 
Sent: Friday, March 20, 2020 2:31 PM
To: Adam Thompson 
Cc: Sadiq Saif ; nanog@nanog.org
Subject: Re: COVID-19 vs. peering wars



I'm curious;
would people say that fixing peering inefficiencies could have
a bigger impact on service performance than asking that
Netflix, Amazon Prime, Youtube, Hulu, and other video
streaming services cut their bit rates down?

https://www.bbc.com/news/technology-51968302
https://arstechnica.com/tech-policy/2020/03/netflix-and-youtube-cut-streaming-quality-in-europe-to-handle-pandemic/

It seems that perhaps the fingers, and the regulatory
hammer, are being pointed in the wrong direction at
the moment.  ^_^;

Matt
staying safely under the saran-wrap blanket for the next few weeks




On Fri, Mar 20, 2020 at 9:31 AM Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
Every large ISP does this (or rather, doesn't) at every IX in Canada.  Bell 
isn't unique by any stretch.

It's not in their economic interest to peer at a local IX, because from their 
perspective, the IX takes away business (Managed L2 point-to-point circuits, at 
the very least) from them.

Don't expect the dominant wireline ISP(s) in any region to join local IXes 
anytime soon, sadly, no matter how much it would benefit their customers.  
After all, the customer is always free to purchase service to the IX and join 
the IX, right???  *grumble*

In my local case, if BellMTS joined MBIX, un-cached DNS resolution times could 
potentially drop by 15msec.  That's HUGE.  But the end-user experience is not 
their primary goal.  Their primary goal is profit, as always.

-Adam Thompson
 Founding member, MBIX (once upon a time)

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca>

> -Original Message-
> From: NANOG mailto:nanog-boun...@nanog.org>> On 
> Behalf Of Sadiq Saif
> Sent: Friday, March 20, 2020 9:38 AM
> To: nanog@nanog.org<mailto:nanog@nanog.org>
> Subject: Re: COVID-19 vs. peering wars
>
> On Fri, 20 Mar 2020, at 10:31, Steve Mikulasik via NANOG wrote:
> >
> > In Canada the CRTC really needs to get on Canadian ISPs about peering
> > very liberally at IXs in each province. I know of one major
> > institution right now that would have a major work from home issue
> > resolved if one big ISP would peer with one big tier 1 in the IX they
> > are both located at in the same province. Instead traffic needs to
> > flow across the country or to the USA to get back to the same city.
>
> **cough** Bell Canada **cough**.
>
> --
>   Sadiq Saif
>   https://sadiqsaif.com/

RE: COVID-19 vs. peering wars

[Adam Thompson]

Every large ISP does this (or rather, doesn't) at every IX in Canada.  Bell 
isn't unique by any stretch.

It's not in their economic interest to peer at a local IX, because from their 
perspective, the IX takes away business (Managed L2 point-to-point circuits, at 
the very least) from them.

Don't expect the dominant wireline ISP(s) in any region to join local IXes 
anytime soon, sadly, no matter how much it would benefit their customers.  
After all, the customer is always free to purchase service to the IX and join 
the IX, right???  *grumble*

In my local case, if BellMTS joined MBIX, un-cached DNS resolution times could 
potentially drop by 15msec.  That's HUGE.  But the end-user experience is not 
their primary goal.  Their primary goal is profit, as always.

-Adam Thompson
 Founding member, MBIX (once upon a time)

Adam Thompson
Consultant, Infrastructure Services
MERLIN
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca

> -Original Message-
> From: NANOG  On Behalf Of Sadiq Saif
> Sent: Friday, March 20, 2020 9:38 AM
> To: nanog@nanog.org
> Subject: Re: COVID-19 vs. peering wars
> 
> On Fri, 20 Mar 2020, at 10:31, Steve Mikulasik via NANOG wrote:
> >
> > In Canada the CRTC really needs to get on Canadian ISPs about peering
> > very liberally at IXs in each province. I know of one major
> > institution right now that would have a major work from home issue
> > resolved if one big ISP would peer with one big tier 1 in the IX they
> > are both located at in the same province. Instead traffic needs to
> > flow across the country or to the USA to get back to the same city.
> 
> **cough** Bell Canada **cough**.
> 
> --
>   Sadiq Saif
>   https://sadiqsaif.com/

RE: WIKI documentation Software?

[Adam Thompson]

[Disclaimer: former Atlassian Reseller and Certified Confluence Administrator 
here.]
Atlassian Confluence.
It’s not cheap, and it certainly has its flaws, but it incorporates one feature 
that most (all?) other wikis don’t – hierarchy.  You can organize information 
(pages) hierarchically like a directory structure.
The key here is that some people think hierarchically, and some people don’t.  
For the hierarchical thinkers, a free-form wiki (i.e. most of them) is absolute 
hell to navigate, and you can still cross-link pages and use tags and 
categories, so the non-hierarchical thinkers are still just as much at home as 
with other products.
Another plus, despite the cost, is you can host it on-site or in the cloud, 
depending on your needs.
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: NANOG  On Behalf Of Craig
Sent: Saturday, March 14, 2020 7:08 AM
To: nanog group 
Subject: WIKI documentation Software?

Wanted to ask what WIKI software teams are using to save documentation to / how 
to's for staff, etc.

pro's
con's

We have an older wiki bare-metal wiki server, that I want to get replaced 
before it kicks the bucket and was looking into various ones.

thanks;

CPV

sflow -> aggregated aspath visualization?

[Adam Thompson]

I’m looking for product recommendations:

We’ve noticed that about 20% of our traffic here lately has decamped from the 
free (or, at least, flat-rate) connection to CANARIE (our R network) and its 
various connected content-delivery networks, and onto our commercial provider.
While this is presumptively a legitimate shift, we’d like to better understand 
these changes when they occur, in a way that our executive can understand at a 
glance.
We do have sFlow (et al.) going to an Arbor PeakFlow box for analysis, but it’s 
lacklustre at best at understanding changes like this.
I want:

  *   Top #n ASNs by traffic volume, per router/interface, stacked chart
  *   Some way to visualize large jumps in that dataset, e.g. if Cloudflare 
ditched their CANARIE connection and now that traffic all goes commercial, I 
don’t know what sort of graphic would be useful, maybe a stacked polar chart so 
you could see when an AS jumped from one sector to another?  Even stacked bar 
charts could be useful.

If anyone knows of tools capable of generating easy-to-understand reports, 
dashboards, including historical “what changed this week”-type data, please let 
me know.

For that matter, if you have a technique of collecting this data and using 
Excel to do the reporting, that would work too.

(Yes, I could theoretically build this off of existing open source tools… 
eventually)

Thanks,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>