Re: Increase bandwidth usage in partial-mesh network?

[Arie Vayner]

Maybe something like this (if you can break it into different bgp ASNs by
network area):

"draft-mohanty-bess-ebgp-dmz-03"
https://datatracker.ietf.org/doc/html/draft-mohanty-bess-ebgp-dmz-03

On Wed, Oct 13, 2021, 10:30 Adam Thompson  wrote:

> Looking for recommendtions or suggestions...
>
> I've got a downstream customer asking for help;  they have a private
> internal network that I've taken to calling the "partial-mesh network from
> hell": it's got two partially-overlapping radio networks, mixed with
> islands of isolated fiber connectivity.
> Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only
> select the _best_ path, they won't spread the load unless all paths are
> equal - and they are very unequal in this network, ECMP would likely fail
> horribly.
> The network is becoming bandwidth-limited, so they're wanting to make use
> of all available paths, not just the single "best" path.  It's also remote
> and spread out, so adding new links or upgrading existing links is
> difficult and expensive.
> Oh, and their routers are overdue for a refresh, so acquiring replacement
> h/w is now possible.
>
> Has anyone come across any product or technology that can handle the
> multi-path-ness and the private-network-ness like a regular router, but
> also provides the intelligent per-flow path steering based on e.g. latency,
> like an SD-WAN device (and/or some firewalls)?
>
> Here's hoping,
> -Adam
>
> *Adam Thompson*
> Consultant, Infrastructure Services
> [image: 1593169877849]
> 100 - 135 Innovation Drive
> Winnipeg, MB, R3T 6A8
> (204) 977-6824 or 1-800-430-6404 (MB only)
> athomp...@merlin.mb.ca
> www.merlin.mb.ca
>

RE: Increase bandwidth usage in partial-mesh network?

[Brian Turnbow via NANOG]

Has anyone come across any product or technology that can handle the 
multi-path-ness and the private-network-ness like a regular router, but also 
provides the intelligent per-flow path steering based on e.g. latency, like an 
SD-WAN device (and/or some firewalls)?

Maybe add a little bit of linear optimization on top of 
faucet/openvswitch/openflow to calculate best paths based upon bandwidth, 
paths, and fill-factors.  There is a presentation where Google uses that 
technique to obtain high utilization on their links (not necessarily those 
tools though).

Raymond Burkholder

This is what a large Italian wisp has done, here are a couple of presentations 
made at our ITNOG sessions.
I’m not sure if they have open sourced anything though.
https://www.itnog.it/itnog4/files/14-Traffic%20Engineering%20-%20the%20EOLO%20way%20of%20life.pdf
https://www.itnog.it/itnog3/files/ITNOG3-EOLO.pdf
Brian

Re: Increase bandwidth usage in partial-mesh network?

[Raymond Burkholder]

On 10/13/21 11:29 AM, Adam Thompson wrote:
I've got a downstream customer asking for help;  they have a private 
internal network that I've taken to calling the "partial-mesh network 
from hell": it's got two partially-overlapping radio networks, mixed 
with islands of isolated fiber connectivity.
Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will 
only select the _best_ path, they won't spread the load unless all 
paths are equal - and they are very unequal in this network, ECMP 
would likely fail horribly.
The network is becoming bandwidth-limited, so they're wanting to make 
use of all available paths, not just the single "best" path.  It's 
also remote and spread out, so adding new links or upgrading existing 
links is difficult and expensive.
Oh, and their routers are overdue for a refresh, so acquiring 
replacement h/w is now possible.


Has anyone come across any product or technology that can handle the 
multi-path-ness and the private-network-ness like a regular router, 
but also provides the intelligent per-flow path steering based on e.g. 
latency, like an SD-WAN device (and/or some firewalls)?


Maybe add a little bit of linear optimization on top of 
faucet/openvswitch/openflow to calculate best paths based upon 
bandwidth, paths, and fill-factors.  There is a presentation where 
Google uses that technique to obtain high utilization on their links 
(not necessarily those tools though).


Raymond Burkholder

Re: Increase bandwidth usage in partial-mesh network?

[Karsten Thomann via NANOG]

  I would agree.Not sure if other vendors have something similar, but in Juniper land you could use traffic engineering with container lsp to go a step further than just plain rsvp-te.Kind regards Karsten Von: nanog@nanog.orgGesendet: 14. Oktober 2021 03:06An: athomp...@merlin.mb.caAntworten: mrodrig...@fletnet.comCc: nanog@nanog.orgBetreff: Re: Increase bandwidth usage in partial-mesh network?  Assuming that the reasons for the low bandwidth and use of radio is due to physical constraints - distances, inhospitable terrain between nodes, etc.  In this case, some good 'ol MPLS traffic engineering using LSP's with bandwidth reservations may be the way to influence how traffic is routed.  Then, they may need some platform to provide observability and potentially dynamic re-routing of LSP's based on actual or predicted congestion situations.  If traffic patterns and utilization are not ideally deterministic, then skip the bandwidth reservation and ensure that the automation is in place to reroute traffic when necessary.I know, adding complexity, but if you just can't build the links you would want, this may be a way to work with what you've got.Best Regards,Mauricio RodriguezFounder / OwnerFletnet Network Engineering (www.fletnet.com)Follow us on LinkedInmauricio.rodrig...@fletnet.comOffice: +1 786-309-1082Direct: +1 786-309-5493On Wed, Oct 13, 2021 at 1:33 PM Adam Thompson <athomp...@merlin.mb.ca> wrote:






Looking for recommendtions or suggestions...




I've got a downstream customer asking for help;  they have a private internal network that I've taken to calling the "partial-mesh network from hell": it's got two partially-overlapping radio networks, mixed with islands of isolated fiber connectivity.

Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select the _best_ path, they won't spread the load unless all paths are equal - and they are very unequal in this network, ECMP would likely fail horribly.


The network is becoming bandwidth-limited, so they're wanting to make use of all available paths, not just the single "best" path.  It's also remote and spread out, so adding new links or upgrading existing links is difficult and expensive.

Oh, and their routers are overdue for a refresh, so acquiring replacement h/w is now possible.





Has anyone come across any product or technology that can handle the multi-path-ness and the private-network-ness like a regular router, but also provides the intelligent per-flow path steering based on e.g. latency, like an SD-WAN device (and/or some firewalls)?




Here's hoping,


-Adam








Adam Thompson
Consultant, Infrastructure Services

100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca









This message (and any associated files) may contain confidential and/or privileged information. If you are not the intended recipient or authorized to receive this for the intended recipient, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by sending a reply e-mail and delete this message. Thank you for your cooperation.

Re: Increase bandwidth usage in partial-mesh network?

[Mauricio Rodriguez via NANOG]

Assuming that the reasons for the low bandwidth and use of radio is due to
physical constraints - distances, inhospitable terrain between nodes, etc.
In this case, some good 'ol MPLS traffic engineering using LSP's with
bandwidth reservations may be the way to influence how traffic is routed.
Then, they may need some platform to provide observability and potentially
dynamic re-routing of LSP's based on actual or predicted congestion
situations.  If traffic patterns and utilization are not ideally
deterministic, then skip the bandwidth reservation and ensure that the
automation is in place to reroute traffic when necessary.

I know, adding complexity, but if you just can't build the links you would
want, this may be a way to work with what you've got.

Best Regards,

Mauricio Rodriguez

Founder / Owner

Fletnet Network Engineering (www.fletnet.com)
*Follow us* on LinkedIn 

mauricio.rodrig...@fletnet.com

Office: +1 786-309-1082

Direct: +1 786-309-5493



On Wed, Oct 13, 2021 at 1:33 PM Adam Thompson 
wrote:

> Looking for recommendtions or suggestions...
>
> I've got a downstream customer asking for help;  they have a private
> internal network that I've taken to calling the "partial-mesh network from
> hell": it's got two partially-overlapping radio networks, mixed with
> islands of isolated fiber connectivity.
> Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only
> select the _best_ path, they won't spread the load unless all paths are
> equal - and they are very unequal in this network, ECMP would likely fail
> horribly.
> The network is becoming bandwidth-limited, so they're wanting to make use
> of all available paths, not just the single "best" path.  It's also remote
> and spread out, so adding new links or upgrading existing links is
> difficult and expensive.
> Oh, and their routers are overdue for a refresh, so acquiring replacement
> h/w is now possible.
>
> Has anyone come across any product or technology that can handle the
> multi-path-ness and the private-network-ness like a regular router, but
> also provides the intelligent per-flow path steering based on e.g. latency,
> like an SD-WAN device (and/or some firewalls)?
>
> Here's hoping,
> -Adam
>
> *Adam Thompson*
> Consultant, Infrastructure Services
> [image: 1593169877849]
> 100 - 135 Innovation Drive
> Winnipeg, MB, R3T 6A8
> (204) 977-6824 or 1-800-430-6404 (MB only)
> athomp...@merlin.mb.ca
> www.merlin.mb.ca
>

-- 
This message (and any associated files) may contain confidential and/or 
privileged information. If you are not the intended recipient or authorized 
to receive this for the intended recipient, you must not use, copy, 
disclose or take any action based on this message or any information 
herein. If you have received this message in error, please advise the 
sender immediately by sending a reply e-mail and delete this message. Thank 
you for your cooperation.

Re: Increase bandwidth usage in partial-mesh network?

[Adam Thompson]

Hah, no not your client .  Their existing network is actually surprisingly 
stable, but it is bandwidth-constrained.  As well as the various other replies 
I've seen here and off-list (THANKS!), the only commercial product I've found 
so far that might have a hope of handling this is HPE/Aruba's Silverpeak line.  
We'll see what else comes out of the woodwork, though - if nothing else, it's a 
very interesting exercise!

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>

From: Fletcher Kittredge 
Sent: October 13, 2021 12:59
To: Adam Thompson 
Cc: nanog 
Subject: Re: Increase bandwidth usage in partial-mesh network?


Hey! From the description it must be one of our clients!

Just beware if you go this route, a network that is probably already unstable 
and unreliable will become at least an order of magnitude worse. You can't fix 
ten lbs of stuff into a 4 lb stuff bag. The internet protocols do not tolerate 
congestion well.


On Wed, Oct 13, 2021 at 1:31 PM Adam Thompson 
mailto:athomp...@merlin.mb.ca>> wrote:
Looking for recommendtions or suggestions...

I've got a downstream customer asking for help;  they have a private internal 
network that I've taken to calling the "partial-mesh network from hell": it's 
got two partially-overlapping radio networks, mixed with islands of isolated 
fiber connectivity.
Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select 
the _best_ path, they won't spread the load unless all paths are equal - and 
they are very unequal in this network, ECMP would likely fail horribly.
The network is becoming bandwidth-limited, so they're wanting to make use of 
all available paths, not just the single "best" path.  It's also remote and 
spread out, so adding new links or upgrading existing links is difficult and 
expensive.
Oh, and their routers are overdue for a refresh, so acquiring replacement h/w 
is now possible.

Has anyone come across any product or technology that can handle the 
multi-path-ness and the private-network-ness like a regular router, but also 
provides the intelligent per-flow path steering based on e.g. latency, like an 
SD-WAN device (and/or some firewalls)?

Here's hoping,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>


--
Fletcher Kittredge
GWI
207-602-1134
www.gwi.net<http://www.gwi.net>

Re: Increase bandwidth usage in partial-mesh network?

[William Herrin]

On Wed, Oct 13, 2021 at 10:30 AM Adam Thompson  wrote:
> Has anyone come across any product or technology that can handle the 
> multi-path-ness and the private-network-ness like a regular router, but also 
> provides the intelligent per-flow path steering based on e.g. latency, like 
> an SD-WAN device (and/or some firewalls)?

The babel protocol does some of this.
https://datatracker.ietf.org/doc/html/rfc6126

Regards,
Bill Herrin

-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: Increase bandwidth usage in partial-mesh network?

[Mark Tinka]

On 10/13/21 19:59, Fletcher Kittredge wrote:



Hey! From the description it must be one of our clients!

Just beware if you go this route, a network that is probably already 
unstable and unreliable will become at least an order of magnitude 
worse. You can't fix ten lbs of stuff into a 4 lb stuff bag. The 
internet protocols do not tolerate congestion well.


It sounds like they need to get back to the basics first.

Simplification, in lieu of added complexity, seems to be the appealing 
approach.


Mark.

Re: Increase bandwidth usage in partial-mesh network?

[Fletcher Kittredge]

Hey! From the description it must be one of our clients!

Just beware if you go this route, a network that is probably already
unstable and unreliable will become at least an order of magnitude worse.
You can't fix ten lbs of stuff into a 4 lb stuff bag. The internet
protocols do not tolerate congestion well.


On Wed, Oct 13, 2021 at 1:31 PM Adam Thompson 
wrote:

> Looking for recommendtions or suggestions...
>
> I've got a downstream customer asking for help;  they have a private
> internal network that I've taken to calling the "partial-mesh network from
> hell": it's got two partially-overlapping radio networks, mixed with
> islands of isolated fiber connectivity.
> Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only
> select the _best_ path, they won't spread the load unless all paths are
> equal - and they are very unequal in this network, ECMP would likely fail
> horribly.
> The network is becoming bandwidth-limited, so they're wanting to make use
> of all available paths, not just the single "best" path.  It's also remote
> and spread out, so adding new links or upgrading existing links is
> difficult and expensive.
> Oh, and their routers are overdue for a refresh, so acquiring replacement
> h/w is now possible.
>
> Has anyone come across any product or technology that can handle the
> multi-path-ness and the private-network-ness like a regular router, but
> also provides the intelligent per-flow path steering based on e.g. latency,
> like an SD-WAN device (and/or some firewalls)?
>
> Here's hoping,
> -Adam
>
> *Adam Thompson*
> Consultant, Infrastructure Services
> [image: 1593169877849]
> 100 - 135 Innovation Drive
> Winnipeg, MB, R3T 6A8
> (204) 977-6824 or 1-800-430-6404 (MB only)
> athomp...@merlin.mb.ca
> www.merlin.mb.ca
>


-- 
Fletcher Kittredge
GWI
207-602-1134
www.gwi.net

Increase bandwidth usage in partial-mesh network?

[Adam Thompson]

Looking for recommendtions or suggestions...

I've got a downstream customer asking for help;  they have a private internal 
network that I've taken to calling the "partial-mesh network from hell": it's 
got two partially-overlapping radio networks, mixed with islands of isolated 
fiber connectivity.
Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select 
the _best_ path, they won't spread the load unless all paths are equal - and 
they are very unequal in this network, ECMP would likely fail horribly.
The network is becoming bandwidth-limited, so they're wanting to make use of 
all available paths, not just the single "best" path.  It's also remote and 
spread out, so adding new links or upgrading existing links is difficult and 
expensive.
Oh, and their routers are overdue for a refresh, so acquiring replacement h/w 
is now possible.

Has anyone come across any product or technology that can handle the 
multi-path-ness and the private-network-ness like a regular router, but also 
provides the intelligent per-flow path steering based on e.g. latency, like an 
SD-WAN device (and/or some firewalls)?

Here's hoping,
-Adam

Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca
www.merlin.mb.ca