Re: [ovs-dev] [PATCH v4 ovn] Replace chassis mac with router port mac on destination chassis
Hi Numan, Thanks for calling out the failure. Fixed the same in v5. Did not realize that we have removed OVS tests from repo, hence based on test number, I ruled it out as an unrelated tests. Regards, Ankur From: Numan Siddique Sent: Tuesday, September 10, 2019 12:51 AM To: Ankur Sharma Cc: ovs-dev@openvswitch.org Subject: Re: [ovs-dev] [PATCH v4 ovn] Replace chassis mac with router port mac on destination chassis On Mon, Sep 9, 2019 at 1:58 AM Ankur Sharma mailto:ankur.sha...@nutanix.com>> wrote: During E-W routing for vlan backed networks, we replace router port mac with chassis mac, when packet leaves the source hypervisor. As a result, the destination VM (on remote hypervisor) will see chassis mac as source mac in received packet. Although, functionality wise this does not cause any issue, however chassis mac being see as source on VM, will lead to following: a. INCONSISTENT SOURCE MAC: If the destination VM moves to same hypervisor as sender, then it will see router port mac as source mac. Whereas, on a remote hypervisor, source mac will be the sender chassis mac. This will cause inconsistency in packet headers for the same flow and could be confusing for someone looking at packet captures inside the vm. b. SYSTEM MAC BEING EXPOSED TO VM: Chassis mac is a CMS provided mac, i.e it is an infrastructure mac. It is not a good practice to expose such values to VM, which should not be seeing them in first place. In order to replace chassis mac with router port mac, we will do following. a. Create conjunction for each chassis mac and router port vlan id combination. For example, for a 2 node chassis setup, where we have a logical router, connected to 4 logical switches with vlan ids: 2000, 1000, 0 and 24, the conjunction flow will look like following: cookie=0x0, duration=9094.608s, table=0, n_packets=0, n_bytes=0, idle_age=9094, priority=180,dl_src=aa:bb:cc:dd:ee:22 actions=conjunction(100,1/2) cookie=0x0, duration=9094.608s, table=0, n_packets=0, n_bytes=0, idle_age=9094, priority=180,dl_src=aa:bb:cc:dd:ff:ee actions=conjunction(100,1/2) cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0, idle_age=9094, priority=180,dl_vlan=2000 actions=conjunction(100,2/2) cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0, idle_age=9094, priority=180,dl_vlan=1000 actions=conjunction(100,2/2) cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0, idle_age=9094, priority=180,vlan_tci=0x/0x1fff actions=conjunction(100,2/2) cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0, idle_age=9094, priority=180,dl_vlan=24 actions=conjunction(100,2/2) b. Using this conjunction as match, we can identify if packet entering destination hypervisor was routed at the source or not. This will be done in table=0 (Physical to logical) at priority=180. For example: cookie=0x0, duration=9795.957s, table=0, n_packets=1391, n_bytes=141882, idle_age=8396, priority=180,conj_id=100,in_port=146,dl_vlan=1000 actions=.,mod_dl_src:00:00:01:01:02:03,... c. We use conjunction, as it will ensure that we do not end up having lot of flows as we scale up. If we do not use conjunction, then we will have N (number of chassis macs) X M (number of router vlans) number of ovs flows. Conjunction converts it to N + M. Consider a setup, with 500 Chassis and 500 routed vlans. Without conjunction we will need 25000 (500 * 500) flows, whereas with conjunction that number comes down to 1000 (500 + 500). Signed-off-by: Ankur Sharma mailto:ankur.sha...@nutanix.com>> Hi Ankur, Thanks for v4. The below test case is failing ## --- ## 116: ovn -- 2 HVs, 2 lports/HV, localnet ports, DVR N-S Ping FAILED (ovn.at:15773 [ovn.at]<https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at-3A15773=DwMFaQ=s883GpUCOChKOHiocYtGcg=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY=mSPIZIN9J6N8ibOG1VZbwAPCcyGl5w2dX3HIc407lc8=MtiEiCBeIkE5PnqLVt87SPAVbZwQl_KEuWC2xagdlYo=>) ## - ## ## Test results. ## ## - ## Numan --- controller/chassis.c| 2 +- controller/chassis.h| 3 + controller/ovn-controller.c | 5 + controller/physical.c | 222 ++-- controller/physical.h | 1 + ovn-architecture.7.xml | 10 +- tests/ovn.at [ovn.at]<https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at=DwMFaQ=s883GpUCOChKOHiocYtGcg=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY=mSPIZIN9J6N8ibOG1VZbwAPCcyGl5w2dX3HIc407lc8=s8f0ZsvzCYQCkNxo-BC_hmZqrygAXjeM-CJR1tRspx8=> | 12 ++- 7 files changed, 243 insertions(+), 12 deletions(-) diff --git a/controller/chassis.c b/controller/chassis.c index 937c557..699b662 100644 --- a/controller/chassis.c +++ b/controller/chassis.c @@ -144,7 +144,7 @@ get_bridge_mappings(const struct smap *ext_ids) return smap_get_def(ext_i
Re: [ovs-dev] [PATCH v4 ovn] Replace chassis mac with router port mac on destination chassis
On Mon, Sep 9, 2019 at 1:58 AM Ankur Sharma
wrote:
> During E-W routing for vlan backed networks, we replace router port
> mac with chassis mac, when packet leaves the source hypervisor.
>
> As a result, the destination VM (on remote hypervisor) will see
> chassis mac as source mac in received packet.
>
> Although, functionality wise this does not cause any issue,
> however chassis mac being see as source on VM, will
> lead to following:
> a. INCONSISTENT SOURCE MAC:
>If the destination VM moves to same hypervisor as sender,
>then it will see router port mac as source mac. Whereas, on
>a remote hypervisor, source mac will be the sender chassis mac.
>
>This will cause inconsistency in packet headers for the same
>flow and could be confusing for someone looking at packet
>captures inside the vm.
>
> b. SYSTEM MAC BEING EXPOSED TO VM:
>Chassis mac is a CMS provided mac, i.e it is an infrastructure
>mac. It is not a good practice to expose such values to VM,
>which should not be seeing them in first place.
>
> In order to replace chassis mac with router port mac, we will
> do following.
>
> a. Create conjunction for each chassis mac and router port vlan
>id combination. For example, for a 2 node chassis setup, where
>we have a logical router, connected to 4 logical switches with
>vlan ids: 2000, 1000, 0 and 24, the conjunction flow will look
>like following:
>
>cookie=0x0, duration=9094.608s, table=0, n_packets=0, n_bytes=0,
> idle_age=9094, priority=180,dl_src=aa:bb:cc:dd:ee:22
> actions=conjunction(100,1/2)
>cookie=0x0, duration=9094.608s, table=0, n_packets=0, n_bytes=0,
> idle_age=9094, priority=180,dl_src=aa:bb:cc:dd:ff:ee
> actions=conjunction(100,1/2)
>
>cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
> idle_age=9094, priority=180,dl_vlan=2000 actions=conjunction(100,2/2)
>cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
> idle_age=9094, priority=180,dl_vlan=1000 actions=conjunction(100,2/2)
>cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
> idle_age=9094, priority=180,vlan_tci=0x/0x1fff
> actions=conjunction(100,2/2)
>cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
> idle_age=9094, priority=180,dl_vlan=24 actions=conjunction(100,2/2)
>
> b. Using this conjunction as match, we can identify if packet entering
> destination
>hypervisor was routed at the source or not. This will be done in
> table=0 (Physical to logical)
>at priority=180.
>For example:
>cookie=0x0, duration=9795.957s, table=0, n_packets=1391,
> n_bytes=141882, idle_age=8396,
> priority=180,conj_id=100,in_port=146,dl_vlan=1000
> actions=.,mod_dl_src:00:00:01:01:02:03,...
>
> c. We use conjunction, as it will ensure that we do not end up having lot
> of flows
>as we scale up. If we do not use conjunction, then we will have
>N (number of chassis macs) X M (number of router vlans) number of ovs
> flows.
>Conjunction converts it to N + M.
>Consider a setup, with 500 Chassis and 500 routed vlans.
>Without conjunction we will need 25000 (500 * 500) flows,
>whereas with conjunction that number comes down to 1000 (500 + 500).
>
> Signed-off-by: Ankur Sharma
>
Hi Ankur,
Thanks for v4.
The below test case is failing
## --- ##
116: ovn -- 2 HVs, 2 lports/HV, localnet ports, DVR N-S Ping FAILED (
ovn.at:15773)
## - ##
## Test results. ##
## - ##
Numan
---
> controller/chassis.c| 2 +-
> controller/chassis.h| 3 +
> controller/ovn-controller.c | 5 +
> controller/physical.c | 222
> ++--
> controller/physical.h | 1 +
> ovn-architecture.7.xml | 10 +-
> tests/ovn.at| 12 ++-
> 7 files changed, 243 insertions(+), 12 deletions(-)
>
> diff --git a/controller/chassis.c b/controller/chassis.c
> index 937c557..699b662 100644
> --- a/controller/chassis.c
> +++ b/controller/chassis.c
> @@ -144,7 +144,7 @@ get_bridge_mappings(const struct smap *ext_ids)
> return smap_get_def(ext_ids, "ovn-bridge-mappings", "");
> }
>
> -static const char *
> +const char *
> get_chassis_mac_mappings(const struct smap *ext_ids)
> {
> return smap_get_def(ext_ids, "ovn-chassis-mac-mappings", "");
> diff --git a/controller/chassis.h b/controller/chassis.h
> index eb46ca3..178d295 100644
> --- a/controller/chassis.h
> +++ b/controller/chassis.h
> @@ -27,6 +27,7 @@ struct sbrec_chassis;
> struct sbrec_chassis_table;
> struct sset;
> struct eth_addr;
> +struct smap;
>
> void chassis_register_ovs_idl(struct ovsdb_idl *);
> const struct sbrec_chassis *chassis_run(
> @@ -42,5 +43,7 @@ bool chassis_get_mac(const struct sbrec_chassis *chassis,
> const char *bridge_mapping,
> struct eth_addr *chassis_mac);
> const char *chassis_get_id(void);
> +const char *
[ovs-dev] [PATCH v4 ovn] Replace chassis mac with router port mac on destination chassis
During E-W routing for vlan backed networks, we replace router port
mac with chassis mac, when packet leaves the source hypervisor.
As a result, the destination VM (on remote hypervisor) will see
chassis mac as source mac in received packet.
Although, functionality wise this does not cause any issue,
however chassis mac being see as source on VM, will
lead to following:
a. INCONSISTENT SOURCE MAC:
If the destination VM moves to same hypervisor as sender,
then it will see router port mac as source mac. Whereas, on
a remote hypervisor, source mac will be the sender chassis mac.
This will cause inconsistency in packet headers for the same
flow and could be confusing for someone looking at packet
captures inside the vm.
b. SYSTEM MAC BEING EXPOSED TO VM:
Chassis mac is a CMS provided mac, i.e it is an infrastructure
mac. It is not a good practice to expose such values to VM,
which should not be seeing them in first place.
In order to replace chassis mac with router port mac, we will
do following.
a. Create conjunction for each chassis mac and router port vlan
id combination. For example, for a 2 node chassis setup, where
we have a logical router, connected to 4 logical switches with
vlan ids: 2000, 1000, 0 and 24, the conjunction flow will look
like following:
cookie=0x0, duration=9094.608s, table=0, n_packets=0, n_bytes=0,
idle_age=9094, priority=180,dl_src=aa:bb:cc:dd:ee:22
actions=conjunction(100,1/2)
cookie=0x0, duration=9094.608s, table=0, n_packets=0, n_bytes=0,
idle_age=9094, priority=180,dl_src=aa:bb:cc:dd:ff:ee
actions=conjunction(100,1/2)
cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
idle_age=9094, priority=180,dl_vlan=2000 actions=conjunction(100,2/2)
cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
idle_age=9094, priority=180,dl_vlan=1000 actions=conjunction(100,2/2)
cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
idle_age=9094, priority=180,vlan_tci=0x/0x1fff actions=conjunction(100,2/2)
cookie=0x0, duration=9094.552s, table=0, n_packets=0, n_bytes=0,
idle_age=9094, priority=180,dl_vlan=24 actions=conjunction(100,2/2)
b. Using this conjunction as match, we can identify if packet entering
destination
hypervisor was routed at the source or not. This will be done in table=0
(Physical to logical)
at priority=180.
For example:
cookie=0x0, duration=9795.957s, table=0, n_packets=1391, n_bytes=141882,
idle_age=8396, priority=180,conj_id=100,in_port=146,dl_vlan=1000
actions=.,mod_dl_src:00:00:01:01:02:03,...
c. We use conjunction, as it will ensure that we do not end up having lot of
flows
as we scale up. If we do not use conjunction, then we will have
N (number of chassis macs) X M (number of router vlans) number of ovs flows.
Conjunction converts it to N + M.
Consider a setup, with 500 Chassis and 500 routed vlans.
Without conjunction we will need 25000 (500 * 500) flows,
whereas with conjunction that number comes down to 1000 (500 + 500).
Signed-off-by: Ankur Sharma
---
controller/chassis.c| 2 +-
controller/chassis.h| 3 +
controller/ovn-controller.c | 5 +
controller/physical.c | 222 ++--
controller/physical.h | 1 +
ovn-architecture.7.xml | 10 +-
tests/ovn.at| 12 ++-
7 files changed, 243 insertions(+), 12 deletions(-)
diff --git a/controller/chassis.c b/controller/chassis.c
index 937c557..699b662 100644
--- a/controller/chassis.c
+++ b/controller/chassis.c
@@ -144,7 +144,7 @@ get_bridge_mappings(const struct smap *ext_ids)
return smap_get_def(ext_ids, "ovn-bridge-mappings", "");
}
-static const char *
+const char *
get_chassis_mac_mappings(const struct smap *ext_ids)
{
return smap_get_def(ext_ids, "ovn-chassis-mac-mappings", "");
diff --git a/controller/chassis.h b/controller/chassis.h
index eb46ca3..178d295 100644
--- a/controller/chassis.h
+++ b/controller/chassis.h
@@ -27,6 +27,7 @@ struct sbrec_chassis;
struct sbrec_chassis_table;
struct sset;
struct eth_addr;
+struct smap;
void chassis_register_ovs_idl(struct ovsdb_idl *);
const struct sbrec_chassis *chassis_run(
@@ -42,5 +43,7 @@ bool chassis_get_mac(const struct sbrec_chassis *chassis,
const char *bridge_mapping,
struct eth_addr *chassis_mac);
const char *chassis_get_id(void);
+const char * get_chassis_mac_mappings(const struct smap *ext_ids);
+
#endif /* controller/chassis.h */
diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
index 86f29ac..2a4001b 100644
--- a/controller/ovn-controller.c
+++ b/controller/ovn-controller.c
@@ -1268,9 +1268,14 @@ en_flow_output_run(struct engine_node *node)
(struct sbrec_port_binding_table *)EN_OVSDB_GET(
engine_get_input("SB_port_binding", node));
+struct sbrec_chassis_table *chassis_table =
+(struct
