On 9/28/26 11:45 AM, Alexandra Rukomoinikova via dev wrote: > Several ovn-controller instances may run with the same chassis name and > the same encapsulation IP, an anycast address that is reachable over the > underlay through ECMP. OVN then sees the whole group of nodes as a > single chassis: a single Chassis record, a single Encap record and a > single binding for every port placed on it, while the underlay decides > which node of the group terminates a given tunnel flow. > > There is no coordination and no leader election between the nodes of the > group while writing to SBDB: every ``ovn-controller`` independently does the > full bookkeeping for that one chassis, and it is the idempotence of those > writes that keeps them from fighting each other. > > The flip side is that the nodes must be configured identically in > chassis-related params, and that nothing which relies on conntrack can be > used, because the group shares no runtime state. > > Document all of this, and add a multinode test that deploys two > gateway nodes as one replicated gateway, gives a hypervisor and a node that > does not run OVN an underlay ECMP route to the anycast encapsulation IP, > checks the north-south connectivity through each of the two nodes and > checks that the controllers do not fight over the shared records. > > Signed-off-by: Alexandra Rukomoinikova <[email protected]> > ---
Hi Alexandra, I have a few minor comments on this patch, please see below. I was thinking of taking care of them myself and squash a small fixup patch in when pushing yours to main: https://github.com/dceara/ovn/commit/42b64d5 I didn't test this locally but CI is running in my fork: https://github.com/dceara/ovn/actions/runs/42295260684 Assuming that passes would you be OK with this incremental change? Regards, Dumitru > Documentation/automake.mk | 1 + > Documentation/topics/index.rst | 1 + > Documentation/topics/replicated-gateway.rst | 171 +++++++++++++++++ > NEWS | 5 + > controller/ovn-controller.8.xml | 9 +- > tests/multinode-macros.at | 22 ++- > tests/multinode.at | 197 ++++++++++++++++++++ > 7 files changed, 399 insertions(+), 7 deletions(-) > create mode 100644 Documentation/topics/replicated-gateway.rst > > diff --git a/Documentation/automake.mk b/Documentation/automake.mk > index 685ba9588..b64a0193d 100644 > --- a/Documentation/automake.mk > +++ b/Documentation/automake.mk > @@ -25,6 +25,7 @@ DOC_SOURCE = \ > Documentation/topics/testing.rst \ > Documentation/topics/test-development.rst \ > Documentation/topics/high-availability.rst \ > + Documentation/topics/replicated-gateway.rst \ > Documentation/topics/dynamic-routing/architecture.rst \ > Documentation/topics/dynamic-routing/configuration-examples.rst \ > Documentation/topics/dynamic-routing/index.rst \ > diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst > index 0b676a98b..d5210e03a 100644 > --- a/Documentation/topics/index.rst > +++ b/Documentation/topics/index.rst > @@ -40,6 +40,7 @@ OVN > dynamic-routing/index > incremental-processing/index > high-availability > + replicated-gateway > role-based-access-control > vif-plug-providers/index > testing > diff --git a/Documentation/topics/replicated-gateway.rst > b/Documentation/topics/replicated-gateway.rst > new file mode 100644 > index 000000000..c3e43da03 > --- /dev/null > +++ b/Documentation/topics/replicated-gateway.rst > @@ -0,0 +1,171 @@ > +.. > + Licensed under the Apache License, Version 2.0 (the "License"); you may > + not use this file except in compliance with the License. You may obtain > + a copy of the License at > + > + http://www.apache.org/licenses/LICENSE-2.0 > + > + Unless required by applicable law or agreed to in writing, software > + distributed under the License is distributed on an "AS IS" BASIS, > WITHOUT > + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See > the > + License for the specific language governing permissions and limitations > + under the License. > + > + Convention for heading levels in OVN documentation: > + > + ======= Heading 0 (reserved for the title in a document) > + ------- Heading 1 > + ~~~~~~~ Heading 2 > + +++++++ Heading 3 > + ''''''' Heading 4 > + > + Avoid deeper levels because they do not render well. > + > +=================== > +Replicated Gateway > +=================== Nit: one = too many. > + > +A *replicated gateway* is a group of two or more physical nodes that present > +themselves to OVN as a single chassis. Every node of the group runs its own > +``ovn-controller``, but all of them are configured with the same chassis name > +(``external_ids:system-id``) and with the same encapsulation IP > +(``external_ids:ovn-encap-ip``). The encapsulation IP is an anycast address, > +it is announced into the underlay (statically or by a routing protocol) by > all > +the nodes of the group, and the underlay load-balances the tunnel traffic > +between them with ECMP. > + > +:: > + > + underlay with ECMP to 10.0.0.1 > + > + +-------------------+ +----------------------+ > + | | nexthop via gw-1 | gateway node 1 | > + | hypervisor +---------------------->| system-id: vgw | > + | | | ovn-encap-ip: | > + | ovn-encap-ip: | | 10.0.0.1 | > + | 192.168.0.10 | +----------+-----------+ > + | | | > + | | nexthop via gw-2 +----------+-----------+ > + | +---------------------->| gateway node 2 | > + | | | system-id: vgw | > + +-------------------+ | ovn-encap-ip: | > + | 10.0.0.1 | > + +----------+-----------+ > + | > + physical network | > + -------------------------------------------------------+------------ > + > +From the point of view of the OVN databases there is a single chassis: a > single > +``Chassis`` record, a single ``Encap`` record, and a single binding for every > +port that is placed on the replicated gateway (for example the > chassisredirect > +port of a distributed gateway port). ``ovn-northd`` and the other > +``ovn-controller`` instances do not know, and do not need to know, how many > +physical nodes are behind that chassis. A hypervisor keeps a single tunnel to > +the anycast encapsulation IP, and the underlay decides which node of the > group > +terminates a given flow. > + > +There is no coordination and no leader election between the nodes of the > +group while writing to SBDB: every ``ovn-controller`` independently does the > +full bookkeeping for that one chassis, and it is the idempotence of those > +writes that keeps them from fighting each other. > + > +Two consequences follow from this. First, the columns that OVN derives from > +the local node rather than from the configuration must be forced to the same > +value on every node. More detailed information on this is provided in the > +``Identical OVS and OVN configuration and features`` section below. Second, > +the nodes do not share any runtime state, so nothing that relies on conntrack > +can be used on a replicated gateway, see the ``Only stateless traffic is > +supported`` section below. We shouldn't break in the middle of the verbatim section, that renders weird. > + > +Use case > +-------- > +This is useful when the capacity or the availability of a single gateway node > +is not enough: the group scales horizontally, and losing one node only means > +that the underlay stops using one of the ECMP nexthops, without any change in > +the OVN databases and without the gateway having to be rescheduled to another > +chassis. > + > +A typical deployment looks as follows: > + > +* Two (or more) gateway nodes are installed with the same OVN and OVS > + versions and with an identical ``ovn-controller`` configuration. Each of > + them configures the anycast encapsulation IP locally, e.g. on the loopback > + interface, so that the kernel accepts the tunnel packets destined to it:: > + > + $ ovs-vsctl set open . external-ids:system-id=vgw > + $ ovs-vsctl set open . external-ids:ovn-encap-ip=10.0.0.1 > + $ ovs-vsctl set open . external-ids:ovn-encap-type=geneve > + $ ip address add 10.0.0.1/32 dev lo > + > +* The anycast address is reachable in the underlay through the physical > + addresses of all the nodes of the group. On the hypervisors, and on any > + other node that has to reach the replicated gateway, this is an ECMP > route:: > + > + $ ip route replace 10.0.0.1/32 \ > + nexthop via 192.168.0.1 dev eth0 \ > + nexthop via 192.168.0.2 dev eth0 > + > + In a real deployment the route is normally learned from the fabric, with > the > + gateway nodes advertising the anycast prefix over BGP. > + > +* The logical topology is configured as usual, referring to the chassis by > its > + single name, e.g.:: > + > + $ ovn-nbctl lrp-set-gateway-chassis lr0-public vgw 10 > + > +Requirements and restrictions > +----------------------------- > + > +The nodes of a replicated gateway are not independent chassis, they are one > +chassis that happens to run on several machines. Everything that OVN stores > +per chassis is therefore shared, and everything that OVN computes per chassis > +must produce the same result on every node of the group. This leads to the > +following requirements. > + > +Identical OVS and OVN configuration and features > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +All the nodes of the group must run the same OVS and OVN versions and must > +expose the same set of features. ``ovn-controller`` reports the capabilities > +of the local OVS instance in the ``Chassis`` record (``other_config``, for > +example the supported datapath interface types, and the various > +``ovn-chassis-feature`` entries), and ``ovn-northd`` generates the logical > +flows for the chassis based on those values. If the nodes disagree, the > +``Chassis`` record will keep flapping between the values reported by each > node, > +``ovn-northd`` will keep recomputing, and the node whose capabilities do not > +match the ones currently published may not be able to process the flows it > +receives. > + > +For the same reason, every chassis scoped configuration option has to be > +identical on all the nodes, including, for example: > ``external_ids:hostname``, > +``external_ids:ovn-bridge-mappings`` and all others. > + > +Only stateless traffic is supported > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The nodes of a replicated gateway do not share any runtime state, and in > +particular they do not share conntrack. The underlay picks one of them per > +flow, and nothing guarantees that the two directions of a connection, or the > +packets of a connection before and after a change of the ECMP nexthops (a > node > +being added, removed, rebooted or simply an ECMP rehash), are handled by the > +same node. Any feature that needs the connection state to be on the node that > +processes the packet is therefore not supported on a replicated gateway: > + > +* load balancers, > +* stateful NAT, that is ``snat`` and ``dnat_and_snat`` (unless configured as > + stateless), and NAT with ``ct-commit-all`` enabled, > +* stateful ACLs, and anything else that relies on conntrack, for example > + load balancer hairpinning or ECMP routes with > + ``options:ecmp_symmetric_reply`` set. > + > +Only stateless forwarding, e.g. plain routing and switching, stateless NAT > and > +stateless ACLs, behaves correctly, because every packet can be handled > +independently by any node of the group. > + > +Example > +------- > + > +The ``ovn multinode replicated gateway`` test in ``tests/multinode.at`` sets > up > +a complete example of a two node replicated gateway with an underlay ECMP > route > +to the anycast encapsulation IP, a hypervisor with a VM behind the gateway > +and an external node that does not run OVN at all. > diff --git a/NEWS b/NEWS > index f1c56dd14..7bc3f2113 100644 > --- a/NEWS > +++ b/NEWS > @@ -129,6 +129,11 @@ OVN v26.09.0 - xxx xx xxxx > represent a logical router port (e.g. ovn-ic transit switch LSPs). > Such ports are treated like type=router, including omission from > _MC_flood_l2. > + - Documented the "replicated gateway" setup, in which several > ovn-controller > + instances share one chassis name and one anycast encapsulation IP > + reachable over the underlay through ECMP, so that they appear to OVN as > + a single chassis. See Documentation/topics/replicated-gateway.rst for > the > + requirements and the restrictions. > > OVN v26.03.0 - xxx xx xxxx > -------------------------- > diff --git a/controller/ovn-controller.8.xml b/controller/ovn-controller.8.xml > index 3c33654ff..37c64ebf3 100644 > --- a/controller/ovn-controller.8.xml > +++ b/controller/ovn-controller.8.xml > @@ -100,7 +100,14 @@ > local OVN "etc" directory or via the <code>-n</code> command-line > option. > The following precedence is used: first, the command-line option is > read; > if not present, the <code>system-id-override</code> file is read; if > not > - present, then the name configured in the database is used.</dd> > + present, then the name configured in the database is used. > + <p> > + Several <code>ovn-controller</code> instances running on different > + nodes may share the same <code>system-id</code>, so that they appear > + to OVN as a single chassis backed by an anycast encapsulation IP. > + See the "Replicated Gateway" document for the requirements and the > + restrictions of such a setup. > + </p></dd> > > <dt><code>external_ids:hostname</code></dt> > <dd>The hostname to use in the Chassis table.</dd> > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at > index 1c85f79f8..81ab2a5b8 100644 > --- a/tests/multinode-macros.at > +++ b/tests/multinode-macros.at > @@ -296,23 +296,33 @@ multinode_setup_northd() { > m_as $c ovn-sbctl set-connection p${REMOTE_PROT}:6642 > } > > -# multinode_setup_controller NODE ENCAP_IP REMOTE_IP [ENCAP_TYPE] > -# > -# Sets up controller on specified node. > +# multinode_setup_controller NODE SYSTEM_ID ENCAP_IP REMOTE_IP > +# [ENCAP_TYPE] [START_CONTROLLER] > +# > +# Sets up controller on specified node. SYSTEM_ID defaults to NODE; passing a > +# different value (or the same value on more than one node) allows a test to > +# control the chassis name the node registers with. START_CONTROLLER > defaults > +# to "yes"; with "no" only OVS is set up, so that a test can finish the node > +# configuration before starting ovn-controller itself. > multinode_setup_controller() { > c=$1 > + system_id=${2:-$1} > encap_ip=$3 > remote_ip=$4 > encap_type=${5:-"geneve"} > + start_controller=${6:-"yes"} > > # Cleanup existing one > m_as $c /usr/share/openvswitch/scripts/ovs-ctl stop > m_as $c /usr/share/ovn/scripts/ovn-ctl stop_controller > m_as $c sh -c "rm -f /etc/openvswitch/*.db" > > - m_as $c /usr/share/openvswitch/scripts/ovs-ctl start --system-id=$c > - echo "Using ${CONTROLLER_SSL_ARGS} for ovn-controller". > - m_as $c /usr/share/ovn/scripts/ovn-ctl start_controller > ${CONTROLLER_SSL_ARGS} > + m_as $c /usr/share/openvswitch/scripts/ovs-ctl start > --system-id=$system_id > + if [[ "$start_controller" = "yes" ]]; then > + echo "Using ${CONTROLLER_SSL_ARGS} for ovn-controller". > + m_as $c /usr/share/ovn/scripts/ovn-ctl start_controller \ > + ${CONTROLLER_SSL_ARGS} > + fi I think if we move the part that starts ovn-controller to the end of this function we don't need the kind of ugly start_controller argument. We can just set everything we need and then start ovn-controller. > > m_as $c ovs-vsctl set open . external_ids:ovn-encap-ip=$encap_ip > m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=$encap_type > diff --git a/tests/multinode.at b/tests/multinode.at > index 0c277f5e8..16cfa804f 100644 > --- a/tests/multinode.at > +++ b/tests/multinode.at > @@ -5554,3 +5554,200 @@ AT_CHECK([cat ch2_eth2.tcpdump], [0], [dnl > ]) > > AT_CLEANUP > + > +AT_SETUP([ovn multinode replicated gateway]) > + > +check_fake_multinode_setup > +cleanup_multinode_resources > + > +# Two gateway nodes are deployed as one and the same "replicated" gateway: > both > +# ovn-controllers register with the same chassis name and with the same > anycast > +# encap IP. The encap IP is reachable over the underlay via ECMP through the > +# physical addresses of both gateway nodes, so a geneve tunnel to the > replicated > +# gateway may land on either of them. > +# > +# underlay (eth1, 170.168.0.0/16) > +# ┌───────────────────┐ > ┌──────────────────┐ > +# │ ovn-chassis-1 │ 1.1.1.1/32 │ ovn-gw-1 > │ > +# │ (compute node) ├──── nexthop via gw-1 eth1 ────────┤ > ovn-replicated-gw │ > +# │ │ │ encap 1.1.1.1 > │ > +# │ sw0p1 10.0.0.3 ├──── nexthop via gw-2 eth1 ───┐ > └────────┬─────────┘ > +# └───────────────────┘ │ │ > +# │ > ┌────────┴─────────┐ > +# ┌───────────────────┐ └────┤ ovn-gw-2 > │ > +# │ ovn-chassis-2 │ 1.1.1.1/32 │ > ovn-replicated-gw │ > +# │ (external client) ├──── nexthop via gw-1 eth1 ────────┤ encap 1.1.1.1 > │ > +# │ ├──── nexthop via gw-2 eth1 ────────┤ > │ > +# │ │ > └────────┬─────────┘ > +# │ eth2 172.20.0.50 ├───────── provider network (eth2, br-ex) ────┘ > +# └───────────────────┘ lr0-public 172.20.0.100 > +# > +# On ovn-chassis-2 (aka client) we have client port in public switch for > checking > +# internal connectivity, public switch is connected to dgp port that claimed > by > +# replicated chassis. Compute and client node both have underlay route for > replicated gw ip (1.1.1.1) > + > +vgw1=ovn-gw-1 > +vgw2=ovn-gw-2 > +compute=ovn-chassis-1 > +client=ovn-chassis-2 > +vgw_name=ovn-replicated-gw > +vgw_encap_ip=1.1.1.1 > + > +m_underlay_ip() { > + m_as $1 ip -4 addr show eth1 | grep inet | awk '{print $2}' | cut -d'/' > -f1 > +} > + > +central_ip=$(m_as ovn-central-az1 ip -4 addr show eth1 | grep inet | awk > '{print $2}' | cut -d'/' -f1) This should now be: central_ip=$( (m_central_as ip -4 addr show eth1 2>/dev/null || m_central_as ip -4 addr show eth2) | grep inet | awk '{print $2}' | cut -d'/' -f1) Since we merged 44d3bf28e4b7 ("multinode: Add support for INLINE_BR_EX configuration of multinode.") > +vgw1_ip=$(m_underlay_ip $vgw1) > +vgw2_ip=$(m_underlay_ip $vgw2) > +compute_ip=$(m_underlay_ip $compute) > + > +m_as $compute ip link del sw0p1-p > +m_as $client ip link del sw0p2-p > +m_as $client ip link del sw1p1-p > + > +# Bring up both gateway nodes as the very same chassis: same chassis name and > +# same encap IP, which is not part of the underlay address space. > +for c in $vgw1 $vgw2; do > + # Bring the node back to its original chassis name and encap IP. > + on_exit "multinode_setup_controller $c $c $(m_underlay_ip $c) > $central_ip" > + multinode_setup_controller $c $vgw_name $vgw_encap_ip $central_ip geneve > no > + on_exit "m_as $c ip addr del $vgw_encap_ip/32 dev lo" > + check m_as $c ip addr add $vgw_encap_ip/32 dev lo > + check m_as $c ovs-vsctl set open . external-ids:hostname=$vgw_name > + check m_as $c ovs-vsctl set open . > external-ids:ovn-bridge-mappings=public:br-ex I don't think your test uses the ovn-bridge mapping in any way. So then we might as well add hostname as an optional argument to multinode_setup_controller(). > +done > + > +for c in $vgw1 $vgw2; do > + m_as $c /usr/share/ovn/scripts/ovn-ctl start_controller > ${CONTROLLER_SSL_ARGS} > +done If we change multinode_setup_controller() as I mentioned above, we don't need to manually start it here. > + > +# Both gateway nodes must end up sharing a single chassis record with a > single > +# encap record. > +m_wait_row_count Chassis 1 name=$vgw_name > +m_wait_row_count Encap 1 chassis_name=$vgw_name > +m_check_column "$vgw_encap_ip" Encap ip chassis_name=$vgw_name > + > +# ECMP route to the replicated gateway encap IP in the underlay, both on the > +# compute node and on the external client. > +for c in $compute $client; do > + on_exit "m_as $c ip route del $vgw_encap_ip/32" > + check m_as $c ip route replace $vgw_encap_ip/32 \ > + nexthop via $vgw1_ip dev eth1 \ > + nexthop via $vgw2_ip dev eth1 > +done > + > + Nit: one empty line too many. > +# The anycast encap IP is reachable over the underlay from both nodes. > +AT_CHECK([m_as $compute ping -q -c 3 -i 0.3 -w 2 $vgw_encap_ip | > FORMAT_PING], [0], [dnl > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > +]) > +AT_CHECK([m_as $client ping -q -c 3 -i 0.3 -w 2 $vgw_encap_ip | > FORMAT_PING], [0], [dnl > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > +]) > + > +cr_lport_log_count() { > + m_as $1 sh -c "cat /var/log/ovn/ovn-controller.log 2>/dev/null | \ > + grep -ac 'Claiming lport cr-lr0-public'" > +} > + > +vgw1_cr_logs=$(cr_lport_log_count $vgw1) > +vgw2_cr_logs=$(cr_lport_log_count $vgw2) > + > +check multinode_nbctl ls-add sw0 > +check multinode_nbctl lsp-add sw0 sw0-port1 > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 > 10.0.0.3" > + > +check multinode_nbctl lr-add lr0 > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > +check multinode_nbctl lsp-add-router-port sw0 sw0-lr0 lr0-sw0 > + > +check multinode_nbctl ls-add public > +check multinode_nbctl lsp-add public external_client > +check multinode_nbctl lsp-set-addresses external_client "50:55:00:00:00:03 > 172.20.0.99" > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 > 172.20.0.100/24 > +check multinode_nbctl lsp-add-router-port public public-lr0 lr0-public > +check multinode_nbctl lrp-set-gateway-chassis lr0-public $vgw_name 10 > + > +m_as $compute /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 1342 > 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a > +m_as $client /data/create_fake_vm.sh external_client external > 50:55:00:00:00:03 1342 172.20.0.99 24 172.20.0.100 1001::3/64 1001::a > + > +check multinode_nbctl --wait=hv sync > +m_wait_for_ports_up sw0-port1 > +m_wait_for_ports_up external_client > + > +# The distributed gateway port is claimed by the replicated gateway chassis. > +m_wait_row_count Port_Binding 1 logical_port=cr-lr0-public > +vgw_uuid=$(m_central_as ovn-sbctl --bare --columns _uuid find Chassis > name=$vgw_name) > +m_wait_column "$vgw_uuid" Port_Binding chassis logical_port=cr-lr0-public > + > +# The two controllers share a single Chassis record and a single binding for > +# the chassisredirect port. Since both nodes are configured identically and > +# both resolve to the same chassis, each of them finds the records already > +# holding the values it was about to write and its transaction is a no-op, so > +# once the setup has settled no transaction touching those tables is > expected. > +# If the nodes fought over a record, they would add one transaction per main > +# loop iteration to the southbound database log. > +sb_txn_count() { > + m_central_as grep -ac "\"$1\":" /etc/ovn/ovnsb_db.db > +} > + > +chassis_txns=$(sb_txn_count Chassis) > +pb_txns=$(sb_txn_count Port_Binding) > +sleep 5 > +check test $(($(sb_txn_count Chassis) - chassis_txns)) -le 1 > +check test $(($(sb_txn_count Port_Binding) - pb_txns)) -le 1 > + > +cr_logs=$(($(cr_lport_log_count $vgw1) + $(cr_lport_log_count $vgw2))) > +# Claim on both > +check test $((cr_logs - vgw1_cr_logs - vgw2_cr_logs)) -le 2 > + > +check multinode_nbctl --wait=hv sync > + > +# North-south connectivity through the replicated gateway, with the underlay > ECMP > +# route in place on both ends. > +check_ns_connectivity() { > + M_NS_CHECK_EXEC([$compute], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > 172.20.0.99 | FORMAT_PING], \ > +[0], [dnl > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > +]) > + > + M_NS_CHECK_EXEC([$compute], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > 172.20.0.100 | FORMAT_PING], \ > +[0], [dnl > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > +]) > + > + M_NS_CHECK_EXEC([$client], [external], [ping -q -c 3 -i 0.3 -w 2 > 10.0.0.3 | FORMAT_PING], \ > +[0], [dnl > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > +]) > +} > + > +# The traffic towards the replicated gateway must be tunneled to the anycast > encap IP. > +M_START_TCPDUMPS([$compute], [-c 2 -neei eth1 host $vgw_encap_ip and udp > port 6081], [ch1_underlay]) > + > +check_ns_connectivity > + > +OVS_WAIT_UNTIL([test -s ch1_underlay.tcpdump]) > +AT_CHECK([grep -q "$vgw_encap_ip" ch1_underlay.tcpdump]) > + > +# Each of the two replicated gateway nodes on its own must be able to carry > the > +# traffic: pin the underlay route to a single nexthop at a time. > +for nexthop in $vgw1_ip $vgw2_ip; do > + for c in $compute $client; do > + check m_as $c ip route replace $vgw_encap_ip/32 via $nexthop dev eth1 > + done > + > + check_ns_connectivity > +done > + > +# Back to ECMP over both gateway nodes. > +for c in $compute $client; do > + check m_as $c ip route replace $vgw_encap_ip/32 \ > + nexthop via $vgw1_ip dev eth1 \ > + nexthop via $vgw2_ip dev eth1 > +done > + > +check_ns_connectivity > + > +AT_CLEANUP _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
