[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Björn Tillenius]

** Changed in: maas
Milestone: next => None

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dimitri John Ledkov]

https://www.freedesktop.org/software/systemd/man/systemd.network.html#VRF=

VRF= The name of the VRF to add the link to. See systemd.netdev(5).

vrf A Virtual Routing and Forwarding (VRF) interface to create separate
routing and forwarding domains.


[VRF] Section Options
The [VRF] section only applies for netdevs of kind "vrf" and accepts the 
following key:

Table=
The numeric routing table identifier. This setting is compulsory.

Example 15. /etc/systemd/network/25-vrf.netdev

Create a VRF interface with table 42.

[NetDev]
Name=vrf-test
Kind=vrf

[VRF]
Table=42


So there is backend support in networkd to create tables / vrf, and add link to 
a given vrf. Not sure about if routes are hooked up in networkd too.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dimitri John Ledkov]

** Also affects: netplan.io (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: netplan.io (Ubuntu)
   Status: New => Confirmed

** Changed in: netplan.io (Ubuntu)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Björn Tillenius]

On Tue, May 05, 2020 at 06:04:28PM -, Billy Olsen wrote:
> @bjornt Are you going to copy it over to the MAAS discourse feature set
> then?

We would prefer that  one of the stakeholders actually would add it to
discourse. That will ensure that the stakeholders are still in the loop,
if there are questions about the feature.

Otherwise it might be that the MAAS team will have a discussion with
itself :)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Billy Olsen]

@bjornt Are you going to copy it over to the MAAS discourse feature set
then?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Björn Tillenius]

This sounds great, but it's not a bug report. It's a feature request.
For MAAS, we track feature requests at
https://discourse.maas.io/c/features, so I'm going to mark this bug
report as Invalid for MAAS.

** Changed in: maas
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Ante Karamatić]

** Tags added: sts

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dmitrii Shcherbakov]

Sandor,

Not on the VRF usage side but there is a feature in MAAS 2.6 to have a
better way to work in multi-homed environments (for bionic+ machines):

https://docs.maas.io/2.6/en/intro-new
"Networking - Multiple default gateways"

It relies on "routing policy database" (RPDB) functionality
https://paste.ubuntu.com/p/xg6vFm8Hx7/ (netplan config, routing-policy sections 
are defined only for subnets that have a gateway configured in MAAS)

At the target machine you will see something like this:

# ip rule
0:  from all lookup local 
0:  from 10.232.24.0/21 to 10.232.24.0/21 lookup main 
0:  from 10.232.40.0/21 to 10.232.40.0/21 lookup main 
100:from 10.232.24.0/21 lookup 2 
100:from 10.232.40.0/21 lookup 1 
32766:  from all lookup main 
32767:  from all lookup default 

# ip route show table 1
default via 10.232.40.1 dev b-enp4s0f0-2730 proto static 

# ip route show table 2
default via 10.232.24.1 dev b-enp4s0f0-2731 proto static 

This works well for TCP when responding to traffic (even when software
listens on 0.0.0.0). For UDP a frequent server use-case is DNS servers
and bind9 binds its UDP sockets to interface addresses directly as
opposed to using 0.0.0.0 (some other DNS servers do the same, e.g.
PowerDNS - they even have a post about it
https://blog.powerdns.com/2012/10/08/on-binding-datagram-udp-sockets-to-
the-any-addresses/).

For sending, the policy rules will also kick in provided that a client
socket (TCP or UDP) is bound to a specific address (so that the source
IP is not automatically selected). This requires that the target
software supports binding client sockets to specific addresses
unfortunately.

So far using static routes to summarized prefixes has been a solution
for east-west traffic (because we control nodes managed by MAAS) and
using the approach above for client responses to arbitrary networks (via
https://jaas.ai/u/canonical-bootstack/policy-routing).

After juju starts supporting this new MAAS feature
https://bugs.launchpad.net/juju/+bug/1829150 we can stop using charm-
policy-routing.

I hope that helps while VRF functionality is not implemented.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Sandor Zeestraten]

Any news or progress on these issues now that we are a year and a half
into this bug?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[james beedy]

BUMP! +1000 for this feature. We need vrf support in MAAS to be able to
use it with in bgp/vrf stack. @dmitriis thank you for the thorough write
up and detailed explanation of the problem/solution here!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dmitrii Shcherbakov]

John,

Interfaces of a host carry enough information to be used to make routing
decisions - that's the core idea of host and router-side VRF
implementations. Network spaces as of now do not help you to solve
routing problems in any way unless you have one big L2 network and
"routing" is done without routers: via ARP/NDP in a single broadcast
domain.

Static routes are not flexible enough and are a workaround for the lack
of VRF support. They require many additional steps from a deployer's
perspective to worry about: one should just take a set of VLANs and
subnets to configure in MAAS and assign them to a network space. With a
default gateway per subnet there is always a next hop to delegate a
routing decision to for a given network space from a host's perspective.
Charms and potentially applications do need to be VRF-aware (discussed
above on how).

BGP on a host, while feasible in some scenarios, is not always doable in
practice: not every network and/or security department will give you an
ability to deploy something and set up peering with their BGP-enabled
routers.

I'd be happy to discuss scenarios in-depth here or out of band but the
idea is that Network Spaces need to learn how to assist with Routing and
Forwarding parts - currently they solve only end-end discovery via
relation data.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[John A Meinel]

It would be good to have a clearer discussion of what issues you are
running into with routes. There are several ways that we *could* tackle the
issue. Static Routes was the mechanism that we started modeling because
that was the ask from the field (because, as-I-understand, that was the
solution they were using manually).
There have been discussions about enabling things like BGP. It would be
possible to push heavier on the modeling aspect, and give ways to model
routing as part of the network and space model, and then have Juju tracking
and updating those (eg, route traffic from this space to that space via
these gateways).



On Fri, Jan 26, 2018 at 7:42 PM, Sandor Zeestraten 
wrote:

> I'd like to chime in and add that this is something that we've been
> missing in our Juju and MAAS deployment of OpenStack.
>
> One of our problem area for example is properly routing management,
> storage and public traffic for our OpenStack deployment without complex
> static rules and a lot of annoying workarounds.
>
> I hope that both the Juju and MAAS team take a proper look at supporting
> these use cases.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1737428
>
> Title:
>   VRF support to solve routing problems associated with multi-homing
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions
>

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Sandor Zeestraten]

I'd like to chime in and add that this is something that we've been
missing in our Juju and MAAS deployment of OpenStack.

One of our problem area for example is properly routing management,
storage and public traffic for our OpenStack deployment without complex
static rules and a lot of annoying workarounds.

I hope that both the Juju and MAAS team take a proper look at supporting
these use cases.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Anastasia]

Marking as Incomplete Wishlist as per 'maas'

** Changed in: juju
   Status: New => Incomplete

** Changed in: juju
   Importance: Undecided => Wishlist

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dmitrii Shcherbakov]

Andres,

I'm not going to be at the sprint but the problems described need a
proper solution in MAAS and Juju at least from the end host perspective.
Similar to how VLANs are supported natively in MAAS & Juju, L3
virtualization technologies like VRF should be as well. I hope the
information I will give here will be enough to understand the use-cases
and past experience in this field.

The concept is very similar to VLANs but for L3 which is probably less
familiar and spans many hosts and routers/L3 switches within a single
organization instead of being tied to a given switch fabric and either
the same process or a group of processes on a host need to (1) receive &
respond and (2) send data using different L3 topologies. Instead of
virtual broadcast domains you get virtual paths because of per-
virtual-L3 routing topologies. Good L2 analogies are Multiple Spanning
Tree Protocol (MSTP) or PVST+ that were created to avoid blocking of
switchports depending on logical L2 topologies related to a VLAN or
group of VLANs (this is hidden on L2 though - no end host modifications
required).

The use-cases I am talking about are not new - they were not used as
much in data center networks until a certain point. They were used in
service provider networks for multi-site L3 VPN for many years
(https://tools.ietf.org/html/rfc4364). There are still many deployments
which rely on large L2 domains where those problems do not occur as much
because routing is done trivially via using directly connected routes
and ARP broadcasts (there is never a hop between a source and
destination host in most cases).

I may be wrong but it seems to me that Network Spaces were originally
designed with multi-homing in mind but with limited support for multi-L2
and routing in mind (I don't judge, VRFs are fairly new to the Linux
kernel). They are not that far from supporting that though because of
the recent upstream kernel work.

With leaf-spine you are building a complex L3 network with different
virtual topologies for different purposes and different SLAs for various
kinds of traffic (IOW, a multi-tenant network). This is a typical
service provider scenario with different customers on a shared
infrastructure. You need to build many parallel dedicated communication
lines but since infrastructure is shared it is not possible physically,
however, you still need to do load-sharing across links, use distinct
paths for different kinds of traffic and other optimizations to make
sure your physical links are utilized and clients get certain quality of
service and are separated from each other. In this case L3 VPNs are
built not for clients (companies "x" and "y") but for different
purposes: general purpose data, storage access or replication,
management, public API traffic (originally, this was done for
voice/video/data, see the first two paragraphs in the "background"
section https://www.google.ch/patents/US8457117).

I can describe this in many ways, i.e. we need:

* multi-point L3VPN between racks to simulate L3 virtual circuits/pseudowires 
for different types of traffic;
* virtual routing domains (VRFs);
* traffic and routing separation for multi-L2 segment networks;
* L3 network multi-tenancy.

This is definitely not new, the service provider concepts may be less
familiar though:

1) Static routes + VLSM - DIY routing - doesn't scale and difficult to manage 
when a deployment grows beyond the original VLSM design;
2) VRF-lite (VRF without MPLS) - separate address spaces and routing tables for 
different traffic on routers and, potentially, hosts, interface-based selection 
of a VRF on a given network device;
3) MPLS - this is like VXLAN for virtual L3 networks. In a service provider 
network two MPLS labels are used: one for VRF identification and another one 
for next-hop router identification (in a data center network think of an 
internal or public API label, storage access label, storage replication label 
etc.).

This has been used for years to separate out traffic of different
customers or, for example, general purpose data, voice and video for a
single customer. Containers do not solve this problem with a separate
network namespace because the same process or a group of processes need
to use a different routing table "per-purpose".

What I am asking for is not that difficult because we are only concerned
with end hosts (unless MAAS resides on a ToR or a leaf and we control
the switch OS). I need building blocks to use either VRF-lite or full
VRFs with MPLS in a sane way while keeping routing complexity (BGP, MPLS
etc.) in a data center provider network managed by other people.

Terminology-wise, I think changes are needed as well:
https://github.com/CanonicalLtd/maas-docs/issues/737 - Routing Domain,
L3VPN or VRF are common names for what we refer to as a Network Space,
and what is actually a virtual L3 network with its own complete address
space, routing table copies and dedicated host/router physical or
logical interfaces.

Examples:

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dmitrii Shcherbakov]

** Description changed:

  Problem description:
  
  * a host is multi-homed if it has multiple network interfaces with L3
  addresses configured (physical or virtual interfaces, natural to
  OpenStack regardless of IPv4/IPv6 and IPv6 in general);
+ 
+ (see 3.3.4  Local Multihoming
+ https://tools.ietf.org/html/rfc1122#page-60 and 3.3.4.2  Multihoming
+ Requirements)
  
  * if all hosts that need to participate in L3 communication are located
  on the same L2 network there is no need for a routing device to be
  present. ARP/NDP and auto-created directly connected routes are enough;
  
  * multi-homing with hosts located on different L2 networks requires more 
intelligent routing:
    - "directly connected" routes are no longer enough to talk to all relevant 
hosts in the same network space;
    - a default gateway in the main routing table may not be the correct 
routing device that knows where to forward traffic (management network traffic 
goes to a management switch and router, other traffic goes to L3 ToR switch but 
may go via different bonds);
    - even if a default gateway knows where to forward traffic, it may not be 
the intended physical path (storage replication traffic must go through a 
specific outgoing interface, not the same interface as storage access traffic 
although both interfaces are connected to the same ToR);
    - there is no longer a single "default gateway" as applications need either 
per-logical-direction routers or to become routers themselves (if destination 
== X, forward to next-hop Y). Leaf-spine architecture is a good example of how 
multiple L2 networks force you to use spaces that have VLANs in different 
switch fabrics => one or more hops between hosts with interfaces associated 
with the same network space;
    - while network spaces implicitly require L3 reachability between each host 
that has a NIC associated with a network space, the current definition does not 
mention routing infrastructure required for that. For a single L2 this problem 
is hidden by directly connected routes, for multi-L2, no solution is provided 
or discussed;
  
  * existing solutions to multi-homing require routing table management on
  a given host: complex static routing rules, dynamic routing (e.g.
  running an OSPF or BGP daemon on a host);
  
  * using static routes is rigid and requires network planning (i.e.
  working with network engineers which may have varying degrees of
  experience, doing VLSM planning etc.);
  
  * using dynamic routing requires a broader integration into an
  organization's L3 network infrastructure. Routing can be implemented
  differently across different organizations and it is a security and
  operational burden to integrate with a company's routing infrastructure.
  
  Summary: a mechanism is needed to associate an interface with a
  forwarding table (FIB) which has its own default gateway and make an
  application with a listen(2)ing socket(2) return connected sockets
  associated with different FIBs. In other words, applications need to
  implicitly get source/destination-based routing capabilities without the
  need to use static routing schemes or dynamic routing and with minimum
  or no modifications to the applications themselves.
  
  Goals:
  
  * avoid turning individual hosts into routers;
  * avoid complex static rules;
  * better support multi-fabric deployments with minimum effort (Juju, charms, 
MAAS, applications, network infrastructure);
  * reduce operational complexity (custom L3 infrastructure integration for 
each deployment);
  * reduce delivery risks (L3 infrastructure, L3 department responsiveness 
varies);
  * avoid any form of L2 stretching at the infrastructure level - this is 
inefficient for various reasons.
  
  NOTE: https://cumulusnetworks.com/blog/vrf-for-linux/ - I recommend to
  read this post to understand suggestions below.
  
  How to solve it?
  
  What does it mean for Juju to support VRF devices?
  
  * enslave certain devices on provisioning based on network space information 
(physical NICs, VLAN devices, bonds AND bridges created for containers must be 
considered) - VRF devices logically enslave devices similar to bridges but work 
differently (on L3, not L2);
  * the above is per network namespace so it will work equally well in a LXD 
container;
  
  Conceptually:
  
  # echo 'net.ipv4.tcp_l3mdev_accept = 1' >> /etc/sysctl.conf
  # echo 'net.ipv4.udp_l3mdev_accept = 1' >> /etc/sysctl.conf
  # sysctl -p
  
  # # create additional routing tables
  # cat >> /etc/iproute2/rt_tables.d/vrf.conf 

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Joseph Salisbury]

** Tags added: kernel-da-key

** Changed in: linux (Ubuntu)
   Importance: Undecided => Wishlist

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Andres Rodriguez]

Hi Dimitrii,

This request seems like something we have never seen before. This
doesn't seem to be quite a complex thing to do. Right now, this is and
never has been in our plans to implement. This is also something that
has never come up before.

I'm going to mark this as a wishlist, and incomplete, until we
understand what are the needs and the impact on the project. I would
suggest you raise this during the sprint.

** Changed in: maas
   Status: New => Incomplete

** Changed in: maas
   Importance: Undecided => Wishlist

** Changed in: maas
Milestone: None => next

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dmitrii Shcherbakov]

** Description changed:

  Problem description:
  
  * a host is multi-homed if it has multiple network interfaces with L3
  addresses configured (physical or virtual interfaces, natural to
  OpenStack regardless of IPv4/IPv6 and IPv6 in general);
  
  * if all hosts that need to participate in L3 communication are located
  on the same L2 network there is no need for a routing device to be
  present. ARP/NDP and auto-created directly connected routes are enough;
  
  * multi-homing with hosts located on different L2 networks requires more 
intelligent routing:
-   - "directly connected" routes are no longer enough to talk to all relevant 
hosts in the same network space;
-   - a default gateway in the main routing table may not be the correct 
routing device that knows where to forward traffic (management network traffic 
goes to a management switch and router, other traffic goes to L3 ToR switch but 
may go via different bonds);
-   - even if a default gateway knows where to forward traffic, it may not be 
the intended physical path (storage replication traffic must go through a 
specific outgoing interface, not the same interface as storage access traffic 
although both interfaces are connected to the same ToR);
-   - there is no longer a single "default gateway" as applications need either 
per-logical-direction routers or to become routers themselves (if destination 
== X, forward to next-hop Y). Leaf-spine architecture is a good example of how 
multiple L2 networks force you to use spaces that have VLANs in different 
switch fabrics => one or more hops between hosts with interfaces associated 
with the same network space;
-   - while network spaces implicitly require L3 reachability between each host 
that has a NIC associated with a network space, the current definition does not 
mention routing infrastructure required for that. For a single L2 this problem 
is hidden by directly connected routes, for multi-L2, no solution is provided 
or discussed;
+   - "directly connected" routes are no longer enough to talk to all relevant 
hosts in the same network space;
+   - a default gateway in the main routing table may not be the correct 
routing device that knows where to forward traffic (management network traffic 
goes to a management switch and router, other traffic goes to L3 ToR switch but 
may go via different bonds);
+   - even if a default gateway knows where to forward traffic, it may not be 
the intended physical path (storage replication traffic must go through a 
specific outgoing interface, not the same interface as storage access traffic 
although both interfaces are connected to the same ToR);
+   - there is no longer a single "default gateway" as applications need either 
per-logical-direction routers or to become routers themselves (if destination 
== X, forward to next-hop Y). Leaf-spine architecture is a good example of how 
multiple L2 networks force you to use spaces that have VLANs in different 
switch fabrics => one or more hops between hosts with interfaces associated 
with the same network space;
+   - while network spaces implicitly require L3 reachability between each host 
that has a NIC associated with a network space, the current definition does not 
mention routing infrastructure required for that. For a single L2 this problem 
is hidden by directly connected routes, for multi-L2, no solution is provided 
or discussed;
  
  * existing solutions to multi-homing require routing table management on
  a given host: complex static routing rules, dynamic routing (e.g.
  running an OSPF or BGP daemon on a host);
  
  * using static routes is rigid and requires network planning (i.e.
  working with network engineers which may have varying degrees of
  experience, doing VLSM planning etc.);
  
  * using dynamic routing requires a broader integration into an
  organization's L3 network infrastructure. Routing can be implemented
  differently across different organizations and it is a security and
  operational burden to integrate with a company's routing infrastructure.
  
  Summary: a mechanism is needed to associate an interface with a
  forwarding table (FIB) which has its own default gateway and make an
  application with a listen(2)ing socket(2) return connected sockets
  associated with different FIBs. In other words, applications need to
  implicitly get source/destination-based routing capabilities without the
  need to use static routing schemes or dynamic routing and with minimum
  or no modifications to the applications themselves.
  
  Goals:
  
  * avoid turning individual hosts into routers;
  * avoid complex static rules;
  * better support multi-fabric deployments with minimum effort (Juju, charms, 
MAAS, applications, network infrastructure);
  * reduce operational complexity (custom L3 infrastructure integration for 
each deployment);
  * reduce delivery risks (L3 infrastructure, L3 department responsiveness 
varies);
  * avoid any form of L2 stretching at the 

[Bug 1737428] Re: VRF support to solve routing problems associated with multi-homing

[Dmitrii Shcherbakov]

For Ubuntu kernel this is a backport request.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1737428

Title:
  VRF support to solve routing problems associated with multi-homing

To manage notifications about this bug go to:
https://bugs.launchpad.net/juju/+bug/1737428/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs