Hi Numan,
Yes, it is the openstack setup with networking-ovn.
"ovn-nbctl list logical_router_port | grep redirect-chassis" has null,
and I had followed your command to set the redirect-chassis, but before
that the patch ports for localnet are
created successfully because I am recreating the environment again.
Last time the patch ports are deleted by not clear reason when I am
doing ofproto/trace. I am trying to do it again because the vm
dnat_and_snat doesn't work in my openstack
and ovn environment.
ovn: 2.8.1, networking-ovn: master(with most latest patches)
VM: | dcb35bb8-12d5-444b-96ba-22b60c70f950 | A | ACTIVE | - |
Running | private=20.0.0.2, 172.16.0.131 |
[root@node-1 ~]# ovn-nbctl list logical_router_port
_uuid : e0c7e73e-42a7-469c-9b01-86dd9cc5563e
enabled : []
external_ids : {}
gateway_chassis : []
mac : "fa:16:3e:b5:99:71"
name : "lrp-82f21142-a15a-4cbc-9c59-c4cbf88e2348"
networks : ["20.0.0.1/24"]
options :
{redirect-chassis="f49d19ac-d140-44d4-9f1c-3601a23d67d1"}
peer : []
_uuid : 6d67d962-38a5-4a29-86b5-067dc26f78d4
enabled : []
external_ids : {}
gateway_chassis : [22ab4c4e-3bbc-4b41-8f65-09ddc060efb0]
mac : "fa:16:3e:2e:ea:e9"
name : "lrp-640d0475-ff83-47b7-8a4d-9ea0e770fb24"
networks : ["172.16.0.130/16"]
options : {}
peer : []
[root@node-1 ~]# ovn-nbctl show
switch 9ac587cd-becc-4584-99c9-24282f8707e2
(neutron-668bf1b8-cfec-44d1-b659-30a42c83e29f) (aka public)
port provnet-668bf1b8-cfec-44d1-b659-30a42c83e29f
type: localnet
addresses: ["unknown"]
port 9572b4f3-0e02-4803-acab-1bf1d0b716cb (aka internal_gw)
addresses: [""]
port 640d0475-ff83-47b7-8a4d-9ea0e770fb24
type: router
router-port: lrp-640d0475-ff83-47b7-8a4d-9ea0e770fb24
switch 1a4ccee6-86ff-4ec4-9e42-3fffa1709d59
(neutron-599de5c1-200d-4962-a3ae-ca3e0434cc0a) (aka private)
port 6fd717e1-572e-4bad-8e74-cdda95d1ce49 (aka nic_1510210943.93)
addresses: ["fa:16:3e:b9:c2:dd 20.0.0.2"]
port 82f21142-a15a-4cbc-9c59-c4cbf88e2348
type: router
router-port: lrp-82f21142-a15a-4cbc-9c59-c4cbf88e2348
router e2acff1f-b71e-4c74-83cf-0481b66a004b
(neutron-18bb9c8c-30f7-4ada-bd64-c02dfa198e75) (aka shared_router)
port lrp-640d0475-ff83-47b7-8a4d-9ea0e770fb24
mac: "fa:16:3e:2e:ea:e9"
networks: ["172.16.0.130/16"]
port lrp-82f21142-a15a-4cbc-9c59-c4cbf88e2348
mac: "fa:16:3e:b5:99:71"
networks: ["20.0.0.1/24"]
nat 78d9f764-ff99-4718-a8e2-9c769a26b675
external ip: "172.16.0.130"
logical ip: "20.0.0.0/24"
type: "snat"
nat c3a60413-daef-4ce1-b56a-7db3c812729d
external ip: "172.16.0.131"
logical ip: "20.0.0.2"
type: "dnat_and_snat"
Now VM floatingip 172.16.0.131 is not reachable, the dump-conntrack
showed below rules related with 172.16.0.131, does it enough to work?
icmp,orig=(src=172.16.0.2,dst=172.16.0.131,id=64371,type=8,code=0),reply=(src=20.0.0.2,dst=172.16.0.2,id=64371,type=0,code=0),zone=4
icmp,orig=(src=172.16.0.2,dst=172.16.0.131,id=64371,type=8,code=0),reply=(src=172.16.0.131,dst=172.16.0.2,id=64371,type=0,code=0)
Also, seems I am playing wrong on ovn-trace, with below test, it
always showed no match, drop. my intention is to ping the VM
fixedip/floatingip from external network what should be the proper
ovn-trace command here?
[root@node-1 ~]# ovn-trace --detail
e2acff1f-b71e-4c74-83cf-0481b66a004b 'inport ==
"lrp-82f21142-a15a-4cbc-9c59-c4cbf88e2348" && eth.src == fa:16:3e:b5:99:71
&& ip4.src == 20.0.0.1 && ip4.dst == 20.0.0.2 && ip.ttl == 32'
#
ip,reg14=0x3,vlan_tci=0x0000,dl_src=fa:16:3e:b5:99:71,dl_dst=00:00:00:00:00:00,nw_src=20.0.0.1,nw_dst=20.0.0.2,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=32
ingress(dp="shared_router", inport="lrp-82f211")
------------------------------------------------
0. lr_in_admission: no match (implicit drop)
[root@node-1 ~]# ovn-trace --detail e2acff1f-b71e-4c74-83cf-0481b66a004b
'inport == "lrp-82f21142-a15a-4cbc-9c59-c4cbf88e2348" && eth.src ==
fa:16:3e:b5:99:71 && ip4.src == 20.0.0.1 && eth.dst == fa:16:3e:b9:c2:dd &&
ip4.dst == 20.0.0.2 && ip.ttl == 32'
#
ip,reg14=0x3,vlan_tci=0x0000,dl_src=fa:16:3e:b5:99:71,dl_dst=fa:16:3e:b9:c2:dd,nw_src=20.0.0.1,nw_dst=20.0.0.2,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=32
ingress(dp="shared_router", inport="lrp-82f211")
------------------------------------------------
0. lr_in_admission: no match (implicit drop)
Below is the simple picture of my test environment, basically expect VM
can go dnat_and_snat(not from local hypervisor due to physical limitation)
from another dedicated chassis having set
ovn-bridge-mappings="physnet1:br-ex" and can be schedlued the gateway
router port on it.
What else useful way can be help to debug vm floatingip not reachable
from outside, thanks advance.
[image: Inline image 1]
Best Regards.
Hui.
On Thu, Nov 9, 2017 at 4:05 AM, Numan Siddique <[email protected]> wrote:
> Is this openstack setup with networking-ovn ?
>
> Can you run "ovn-nbctl list logical_router_port | grep redirect-chassis"
> and make sure that networking-ovn has scheduled the gateway router port to
> the desired chassis ?
>
> May be the gateway router port is scheduled to some other chassis.
>
> You can can run "ovn-nbctl set logical_router_port
> options:redirect-chassis="CHASSIS_UUID_WHERE_YOU_WANT_TO_SCHEDULE" to
> schedule it to your desired chassis.
>
> Thanks
> Numan
>
>
>
> On Wed, Nov 8, 2017 at 8:41 AM, Ben Pfaff <[email protected]> wrote:
>
>> Is update_sb_monitors() in ovn-controller.c not doing the right thing to
>> ensure that localnet ports are present? That is the first place to
>> look, I think.
>>
>> On Tue, Nov 07, 2017 at 04:36:12PM +0800, Hui Xiang wrote:
>> > Hi folks,
>> >
>> > When I am running ovn in one of my node having the gateway port
>> connected
>> > external network via localnet, the patch port can't be created between
>> > br-ex(set by ovn-bridge-mappings) with br-int, after gdb, it seems the
>> > result get from SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl)
>> > doesn't include 'localnet' binding type, however it does exist from
>> > ovn-sbctl list port_binding, either I am missing any configuration to
>> make
>> > it work or this is a bug.
>> >
>> > Please have a look and thank much.
>> >
>> > external_ids : {hostname="node-1.domain.tld",
>> > ovn-bridge-mappings="physnet1:br-ex", ovn-encap-ip="168.254.101.10",
>> > ovn-encap-type=geneve, ovn-remote="tcp:192.168.0.2:6642",
>> > rundir="/var/run/openvswitch",
>> > system-id="88596f9f-e326-4e15-ae91-8cc014e7be86"}
>> > iface_types : [geneve, gre, internal, lisp, patch, stt, system,
>> > tap, vxlan]
>> >
>> > (gdb) n
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb) display binding->logical_port
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > (gdb) n
>> > 183 } else if (!strcmp(binding->type, "l2gateway")) {
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb)
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb)
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 5: binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 183 } else if (!strcmp(binding->type, "l2gateway")) {
>> > 5: binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189628d0 "patch"
>> > (gdb)
>> > 183 } else if (!strcmp(binding->type, "l2gateway")) {
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189628d0 "patch"
>> > (gdb)
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189628d0 "patch"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > (gdb) n
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb) display binding->logical_port
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > (gdb) n 183 } else if (!strcmp(binding->type, "l2gateway"))
>> {
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb)
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e718960650
>> > "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189608b0 "patch"
>> > (gdb)
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 5: binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 183 } else if (!strcmp(binding->type, "l2gateway")) { 5:
>> > binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e7189622d0
>> > "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718962380 "patch"
>> > (gdb)
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189628d0 "patch"
>> > (gdb)
>> > 183 } else if (!strcmp(binding->type, "l2gateway")) {
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189628d0 "patch"
>> > (gdb)
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189628d0 "patch"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e718962820
>> > "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e7189628d0 "patch"
>> > (gdb)
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 5: binding->logical_port = 0x55e718961100
>> > "3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718961370 "patch"
>> > (gdb)
>> > 183 } else if (!strcmp(binding->type, "l2gateway")) {
>> > 5: binding->logical_port = 0x55e718961100
>> > "3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e718961100
>> > "3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718961370 "patch"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e718961100
>> > "3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > 4: binding->type = 0x55e718961370 "patch"
>> > (gdb)
>> > 181 if (!strcmp(binding->type, "localnet")) {
>> > 5: binding->logical_port = 0x55e718961b70
>> > "cr-lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e718961c60 "chassisredirect"
>> > (gdb)
>> > 183 } else if (!strcmp(binding->type, "l2gateway")) {
>> > (gdb)
>> > 193 continue;
>> > 5: binding->logical_port = 0x55e718961b70
>> > "cr-lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e718961c60 "chassisredirect"
>> > (gdb)
>> > 179 SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
>> > 5: binding->logical_port = 0x55e718961b70
>> > "cr-lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > 4: binding->type = 0x55e718961c60 "chassisredirect"
>> > (gdb)
>> > 222 shash_destroy(&bridge_mappings);
>> >
>> > [root@node-1 ~]# ovn-sbctl list port_binding
>> > _uuid : a72d3348-ed6f-4e98-9d79-1515f7653ec3
>> > chassis : []
>> > datapath : 47447f44-5c5f-4501-9802-0e7822031f32
>> > gateway_chassis : []
>> > logical_port : "b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > mac : [router]
>> > nat_addresses : []
>> > options : {peer="lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 2
>> > type : patch
>> >
>> > gateway_chassis : []
>> > logical_port : "d94cb413-f53a-4943-9590-c75e60e63568"
>> > mac : [""]
>> > nat_addresses : []
>> > options : {}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 3
>> > type : ""
>> >
>> > _uuid : 45d6d9ac-c432-406e-9e23-bf0abf0add3c
>> > chassis : []
>> > datapath : 47447f44-5c5f-4501-9802-0e7822031f32
>> > external_ids : {}
>> > gateway_chassis : []
>> > logical_port : "provnet-c6f21507-53c8-459e-9690-1257d389246f"
>> > mac : [unknown]
>> > nat_addresses : []
>> > options : {network_name="physnet1"}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 1
>> > type : *localnet *
>> >
>> > _uuid : 60f823bd-ca39-48f1-bd04-6669d9a48e3d
>> > gateway_chassis : []
>> > logical_port : "18ea6e6a-76f3-4a10-9907-1d1dda038988"
>> > mac : ["fa:16:3e:c1:f5:95 192.168.10.2"]
>> > nat_addresses : []
>> > options : {}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 1
>> > type : ""
>> >
>> > _uuid : c56b8f65-a11b-471f-ad8a-935602a1816c
>> > chassis : []
>> > datapath : ddee5a44-0cdc-440a-ab1b-c0d9a6b78ef9
>> > external_ids : {}
>> > gateway_chassis : []
>> > logical_port : "lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > mac : []
>> > nat_addresses : []
>> > options : {peer="3a938edc-8809-4b79-b1a6-8145066e4fe3"}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 3
>> > type : patch
>> >
>> > _uuid : 4b603b6e-3420-41de-9382-b9b3fb4d5b35
>> > gateway_chassis : []
>> > logical_port : "3a938edc-8809-4b79-b1a6-8145066e4fe3"
>> > mac : [router]
>> > nat_addresses : []
>> > options : {peer="lrp-3a938edc-8809-4b79-b1a6-8145066e4fe3"}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 2
>> > type : patch
>> >
>> > _uuid : 371de3de-20d4-4d76-8c0a-dbddd2ff2632
>> > chassis : []
>> > gateway_chassis : []
>> > logical_port : "lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > mac : []
>> > nat_addresses : []
>> > options : {peer="b3edbc9a-3248-43e5-b84e-01689a9c83e2"}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 1
>> > type : patch
>> >
>> > _uuid : f2a93fee-7ec0-4302-9db3-64d13037fa7b
>> > chassis : 391e99d0-fcef-4453-867a-875d54ddc316
>> > datapath : 6d042f46-86be-49ff-84f9-e923bb4b903a
>> > external_ids : {name="nic_1509949090.74", "neutron:cidrs"="
>> > 192.168.10.3/24",
>> > "neutron:device_id"="58f85bf0-e737-423a-ac05-f1a8d223eeab",
>> > "neutron:port_name"="nic_1509949090.74",
>> > "neutron:project_id"="0a87bdf15bcd4124944fde70f787cc49"}
>> > gateway_chassis : []
>> > logical_port : "989a951d-a8fc-4231-8494-c0ede61da7c2"
>> > mac : ["fa:16:3e:e9:0e:15 192.168.10.3"]
>> > nat_addresses : []
>> > options : {}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 3
>> > type : ""
>> >
>> > _uuid : 6d396447-0ac4-4af1-bfaa-6746bfa6063b
>> > chassis : []
>> > datapath : ddee5a44-0cdc-440a-ab1b-c0d9a6b78ef9
>> > external_ids : {}
>> > gateway_chassis : [889addfe-e29b-4c08-a193-e33da211e3e6,
>> > d9de02b0-df2c-4000-b91c-fed0cdc7168d]
>> > logical_port : "cr-lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"
>> > mac : []
>> > nat_addresses : []
>> > options :
>> > {distributed-port="lrp-b3edbc9a-3248-43e5-b84e-01689a9c83e2"}
>> > parent_port : []
>> > tag : []
>> > tunnel_key : 2
>> > type : chassisredirect
>>
>> > _______________________________________________
>> > discuss mailing list
>> > [email protected]
>> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>
>> _______________________________________________
>> discuss mailing list
>> [email protected]
>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>
>
>
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
