Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
> From: "Devin Acosta" <de...@pabstatencio.com> > To: "Lance Richardson" <lrich...@redhat.com> > Cc: "Marcin Mirecki" <mmire...@redhat.com>, "users" <Users@ovirt.org> > Sent: Tuesday, December 6, 2016 12:07:31 PM > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances > between nodes. > > Lance, > > It appears that firewalld was my issue, can you just confirm with me what > Ports should be opened for Geneve and OVN to work properly? > Hi Devin, That's good to hear! On the ovn-northd node, you need TCP ports 6641 and 6642 open (as mentioned in Marcin's blog), on the ovn-controller nodes you need to allow packets with destination UDP port 6081 (Geneve tunnels). Lance ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
Lance, It appears that firewalld was my issue, can you just confirm with me what Ports should be opened for Geneve and OVN to work properly? On Tue, Dec 6, 2016 at 8:43 AM, Lance Richardson <lrich...@redhat.com> wrote: > > From: "Devin Acosta" <de...@pabstatencio.com> > > To: "Lance Richardson" <lrich...@redhat.com> > > Cc: "Marcin Mirecki" <mmire...@redhat.com>, "users" <Users@ovirt.org> > > Sent: Tuesday, December 6, 2016 10:49:59 AM > > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of > instances between nodes. > > > > Lance, > > > > I have attached the output of each into different files. I really > > appreciate your help very much. > > > > Based on asking around about the "dropping duplicate flow:", it's a > known issue that is harmless (other than the noise). I'll try > to find out if someone has a fix in the works. > > It seems your node1 has no port bindings... is that expected? > > From the counters, it looks like node2 and node3 have attempted > to send packets on the geneve tunnels, but neither has received > anything. > > Could you verify that node2 and node3 have connectivity on the > IPs used for the tunnels, e.g. by trying to ping 172.10.10.75 > and 172.10.10.73 from node2? > > If that works, the issue might be iptables rules dropping > geneve packets, the simplest way around that would be to > "systemctl stop firewalld" if that's running (ok for a lab > environment anyway). > > Thanks, > >Lance > -- Devin Acosta Red Hat Certified Architect, LinuxStack 602-354-1220 || de...@linuxguru.co ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
> From: "Devin Acosta" <de...@pabstatencio.com> > To: "Lance Richardson" <lrich...@redhat.com> > Cc: "Marcin Mirecki" <mmire...@redhat.com>, "users" <Users@ovirt.org> > Sent: Tuesday, December 6, 2016 10:49:59 AM > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances > between nodes. > > Lance, > > I have attached the output of each into different files. I really > appreciate your help very much. > Based on asking around about the "dropping duplicate flow:", it's a known issue that is harmless (other than the noise). I'll try to find out if someone has a fix in the works. It seems your node1 has no port bindings... is that expected? >From the counters, it looks like node2 and node3 have attempted to send packets on the geneve tunnels, but neither has received anything. Could you verify that node2 and node3 have connectivity on the IPs used for the tunnels, e.g. by trying to ping 172.10.10.75 and 172.10.10.73 from node2? If that works, the issue might be iptables rules dropping geneve packets, the simplest way around that would be to "systemctl stop firewalld" if that's running (ok for a lab environment anyway). Thanks, Lance ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
Lance,
I have attached the output of each into different files. I really
appreciate your help very much.
--
Devin Acosta
Red Hat Certified Architect, LinuxStack
de...@linuxguru.co
On Tue, Dec 6, 2016 at 7:36 AM, Lance Richardson <lrich...@redhat.com>
wrote:
> > From: "Devin Acosta" <de...@pabstatencio.com>
> > To: "Lance Richardson" <lrich...@redhat.com>
> > Cc: "Marcin Mirecki" <mmire...@redhat.com>, "users" <Users@ovirt.org>
> > Sent: Monday, December 5, 2016 11:28:17 PM
> > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of
> instances between nodes.
> >
> > Lance,
> >
> > Well I installed the new kernel module and it cleared up a lot of the
> > errors I was seeing in the log, but what I notice is that I still can't
> > ping instances between hosts. I'm starting to wonder am I missing
> > something fundamental here? I don't see anything in the ovs-vswitchd.log
> to
> > show tunnel?
> >
>
> Hi Devin,
>
> OK, some small progress then. I think the best next step would be to look
> at the current state of your system. Could you send the output of the
> following commands?
>
> On the nodes running ovn-controller:
>
>ps -fwwC ovn-controller
>ovs-vsctl show
>ovs-dpctl show -s
>ovs-ofctl -O OpenFlow13 dump-flows br-int
>
> On the node running ovn-northd:
>
>ovn-sbctl show
>ovn-sbctl dump-flows
>
> Thanks,
>
>Lance
>
[root@dev001-022-002 ~]# ovn-sbctl show
Chassis "c0dc0909-4274-400d-826e-82348fee47e9"
hostname: "las01-902-001.dev.mosaic451.com"
Encap geneve
ip: "172.10.10.73"
options: {csum="true"}
Chassis "252778b5-bc63-486d-8d82-c3e4350ee650"
hostname: "las01-902-003.dev.mosaic451.com"
Encap geneve
ip: "172.10.10.75"
options: {csum="true"}
Port_Binding "34fd5a0d-28c6-47a8-8230-bb0c398af271"
Port_Binding "937595fd-c8db-43ff-babb-ebf566e407f6"
Chassis "45694909-7882-4d7a-948a-bdf34e6472cb"
hostname: "las01-902-002.dev.mosaic451.com"
Encap geneve
ip: "172.10.10.74"
options: {csum="true"}
Port_Binding "56432d2b-a96d-4ac7-b0e9-3450a006e1d4"
[root@dev001-022-002 ~]# ovn-sbctl dump-flows
Datapath: "devins-net" (605b5285-46cd-42d6-b611-94ad4bb5516d) Pipeline:
ingress
table=0 (ls_in_port_sec_l2 ), priority=100 , match=(eth.src[40]),
action=(drop;)
table=0 (ls_in_port_sec_l2 ), priority=100 , match=(vlan.present),
action=(drop;)
table=0 (ls_in_port_sec_l2 ), priority=50 , match=(inport ==
"6b289418-8b8e-42b4-8334-c71584afcd3e"), action=(next;)
table=0 (ls_in_port_sec_l2 ), priority=50 , match=(inport ==
"6cc29f8b-2746-4701-bbaa-10409f6ae6e2"), action=(next;)
table=0 (ls_in_port_sec_l2 ), priority=50 , match=(inport ==
"71ef81f1-7c20-4c68-b536-d274703f7541"), action=(next;)
table=0 (ls_in_port_sec_l2 ), priority=50 , match=(inport ==
"91d4f4f5-4b9f-42c0-aa2c-8a101474bb84"), action=(next;)
table=1 (ls_in_port_sec_ip ), priority=0, match=(1), action=(next;)
table=2 (ls_in_port_sec_nd ), priority=0, match=(1), action=(next;)
table=3 (ls_in_pre_acl ), priority=0, match=(1), action=(next;)
table=4 (ls_in_pre_lb ), priority=0, match=(1), action=(next;)
table=5 (ls_in_pre_stateful ), priority=100 , match=(reg0[0] == 1),
action=(ct_next;)
table=5 (ls_in_pre_stateful ), priority=0, match=(1), action=(next;)
table=6 (ls_in_acl ), priority=0, match=(1), action=(next;)
table=7 (ls_in_qos_mark ), priority=0, match=(1), action=(next;)
table=8 (ls_in_lb ), priority=0, match=(1), action=(next;)
table=9 (ls_in_stateful ), priority=100 , match=(reg0[1] == 1),
action=(ct_commit(ct_label=0/1); next;)
table=9 (ls_in_stateful ), priority=100 , match=(reg0[2] == 1),
action=(ct_lb;)
table=9 (ls_in_stateful ), priority=0, match=(1), action=(next;)
table=10(ls_in_arp_rsp ), priority=0, match=(1), action=(next;)
table=11(ls_in_dhcp_options ), priority=100 , match=(inport ==
"6b289418-8b8e-42b4-8334-c71584afcd3e" && eth.src == 00:1a:4a:16:01:5c &&
ip4.src == 0.0.0.0 && ip4.dst == 255.255.255.255 && udp.src == 68 && udp.dst ==
67), action=(reg0[3] = put_dhcp_opts(offerip = 10.10.10.4, netmask =
255.255.255.0, router = 10.10.10.1, server_id = 10.10.10.0, lease_time =
4294967295); next;)
table=11(ls_in_dhcp_options ), priority=100 , match=(inport ==
"6cc29f8b-2
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
> From: "Devin Acosta" <de...@pabstatencio.com> > To: "Lance Richardson" <lrich...@redhat.com> > Cc: "Marcin Mirecki" <mmire...@redhat.com>, "users" <Users@ovirt.org> > Sent: Monday, December 5, 2016 11:28:17 PM > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances > between nodes. > > Lance, > > Well I installed the new kernel module and it cleared up a lot of the > errors I was seeing in the log, but what I notice is that I still can't > ping instances between hosts. I'm starting to wonder am I missing > something fundamental here? I don't see anything in the ovs-vswitchd.log to > show tunnel? > Hi Devin, OK, some small progress then. I think the best next step would be to look at the current state of your system. Could you send the output of the following commands? On the nodes running ovn-controller: ps -fwwC ovn-controller ovs-vsctl show ovs-dpctl show -s ovs-ofctl -O OpenFlow13 dump-flows br-int On the node running ovn-northd: ovn-sbctl show ovn-sbctl dump-flows Thanks, Lance ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
low: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:22:37.408Z|00017|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:22:42.408Z|00018|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:22:47.409Z|00019|ofctrl|INFO|Dropped 1 log messages in last 5 seconds (most recently, 5 seconds ago) due to excessive rate 2016-12-06T04:22:47.409Z|00020|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:22:57.411Z|00021|ofctrl|INFO|Dropped 3 log messages in last 10 seconds (most recently, 5 seconds ago) due to excessive rate 2016-12-06T04:22:57.411Z|00022|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:23:12.413Z|00023|ofctrl|INFO|Dropped 4 log messages in last 10 seconds (most recently, 5 seconds ago) due to excessive rate 2016-12-06T04:23:12.413Z|00024|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:23:22.415Z|00025|ofctrl|INFO|Dropped 3 log messages in last 10 seconds (most recently, 5 seconds ago) due to excessive rate 2016-12-06T04:23:22.415Z|00026|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:23:37.417Z|00027|ofctrl|INFO|Dropped 5 log messages in last 10 seconds (most recently, 5 seconds ago) due to excessive rate 2016-12-06T04:23:37.417Z|00028|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:23:47.419Z|00029|ofctrl|INFO|Dropped 3 log messages in last 10 seconds (most recently, 5 seconds ago) due to excessive rate 2016-12-06T04:23:47.419Z|00030|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) 2016-12-06T04:23:57.421Z|00031|ofctrl|INFO|Dropped 3 log messages in last 10 seconds (most recently, 5 seconds ago) due to excessive rate 2016-12-06T04:23:57.421Z|00032|ofctrl|INFO|dropping duplicate flow: table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) [root@ovirt-node2 openvswitch]# brctl show bridge name bridge id STP enabled interfaces ;vdsmdummy; 8000. no DEV-NOC 8000.0cc47a1ef306 no bond0 DEV-VM-NET 8000.0cc47a1ef306 no bond0.700 ovirtmgmt 8000.0cc47a08b3c2 no enp7s0f0 -- Devin Acosta Red Hat Certified Architect, LinuxStack de...@linuxguru.co On Mon, Dec 5, 2016 at 2:34 PM, Lance Richardson <lrich...@redhat.com> wrote: > > From: "Devin Acosta" <de...@pabstatencio.com> > > To: "Lance Richardson" <lrich...@redhat.com> > > Cc: "Marcin Mirecki" <mmire...@redhat.com>, "users" <Users@ovirt.org> > > Sent: Monday, December 5, 2016 4:17:35 PM > > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of > instances between nodes. > > > > Lance, > > > > I found some interesting logs, we have (3) oVIRT nodes. > > > > We are running: > > CentOS Linux release 7.2.1511 (Core) > > Linux hostname 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC > > 2016 x86_64 x86_64 x86_64 GNU/Linux > > > > > > > 2016-12-05T20:47:56.774Z|00021|ofctrl|INFO|OpenFlow error: OFPT_ERROR > > (OF1.3) (xid=0x17): OFPBAC_BAD_TYPE > > This (generally unintelligible message usually indicates that the kernel > openvswitch module doesn't support conntrack. > > > > > > > 2016-12-05T20:35:04.345Z|1|vlog|INFO|opened log file > > /var/log/openvswitch/ovs-vswitchd.log > > 2016-12-05T20:35:04.347Z|2|ovs_numa|INFO|Discovered 16 CPU cores on > > NUMA node 0 > > 2016-12-05T20:35:04.347Z|3|ovs_numa|INFO|Discovered 16 CPU cores on > > NUMA node 1 > > 2016-12-05T20:35:04.347Z|4|ovs_numa|INFO|Discovered 2 NUMA nodes > and 32 > > CPU cores > > 2016-12-05T20:35:04.348Z|5|reconnect|INFO|unix:/ > var/run/openvswitch/db.sock: > > connecting... > > 2016-12-05T20:35:04.348Z|6|reconnect|INFO|unix:/ > var/run/openvswitch/db.sock: > > connected > > 2016-12-05T20:35:04.350Z|7|ofproto_dpif|INFO|system@ovs-system: > > Datapath supports recirculation > > 2016-12-05T20:35:04.350Z|8|ofproto_dpif|INFO|system@ovs-system: MPLS > > label stack length probed as 1 > > 2016-12-05T20:35:04.350Z|9|ofproto_dpif|INFO|system@ovs-system: > > Datapath does not support truncate action > > 2016-12-05T20:35:04.350Z|00010|ofproto_dpif|INFO|system@ovs-system: > > Datapath supports unique flow ids > > 2016-12-05T20:35:04.350Z|00011|ofproto_dpif|INFO|system@ovs-system: > > Datapath does not support ct_state > > 2016-12-05T20:35:04.350Z|0
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
> From: "Devin Acosta" <de...@pabstatencio.com> > To: "Lance Richardson" <lrich...@redhat.com> > Cc: "Marcin Mirecki" <mmire...@redhat.com>, "users" <Users@ovirt.org> > Sent: Monday, December 5, 2016 4:17:35 PM > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances > between nodes. > > Lance, > > I found some interesting logs, we have (3) oVIRT nodes. > > We are running: > CentOS Linux release 7.2.1511 (Core) > Linux hostname 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC > 2016 x86_64 x86_64 x86_64 GNU/Linux > > 2016-12-05T20:47:56.774Z|00021|ofctrl|INFO|OpenFlow error: OFPT_ERROR > (OF1.3) (xid=0x17): OFPBAC_BAD_TYPE This (generally unintelligible message usually indicates that the kernel openvswitch module doesn't support conntrack. > > 2016-12-05T20:35:04.345Z|1|vlog|INFO|opened log file > /var/log/openvswitch/ovs-vswitchd.log > 2016-12-05T20:35:04.347Z|2|ovs_numa|INFO|Discovered 16 CPU cores on > NUMA node 0 > 2016-12-05T20:35:04.347Z|3|ovs_numa|INFO|Discovered 16 CPU cores on > NUMA node 1 > 2016-12-05T20:35:04.347Z|4|ovs_numa|INFO|Discovered 2 NUMA nodes and 32 > CPU cores > 2016-12-05T20:35:04.348Z|5|reconnect|INFO|unix:/var/run/openvswitch/db.sock: > connecting... > 2016-12-05T20:35:04.348Z|6|reconnect|INFO|unix:/var/run/openvswitch/db.sock: > connected > 2016-12-05T20:35:04.350Z|7|ofproto_dpif|INFO|system@ovs-system: > Datapath supports recirculation > 2016-12-05T20:35:04.350Z|8|ofproto_dpif|INFO|system@ovs-system: MPLS > label stack length probed as 1 > 2016-12-05T20:35:04.350Z|9|ofproto_dpif|INFO|system@ovs-system: > Datapath does not support truncate action > 2016-12-05T20:35:04.350Z|00010|ofproto_dpif|INFO|system@ovs-system: > Datapath supports unique flow ids > 2016-12-05T20:35:04.350Z|00011|ofproto_dpif|INFO|system@ovs-system: > Datapath does not support ct_state > 2016-12-05T20:35:04.350Z|00012|ofproto_dpif|INFO|system@ovs-system: > Datapath does not support ct_zone > 2016-12-05T20:35:04.350Z|00013|ofproto_dpif|INFO|system@ovs-system: > Datapath does not support ct_mark > 2016-12-05T20:35:04.350Z|00014|ofproto_dpif|INFO|system@ovs-system: > Datapath does not support ct_label > 2016-12-05T20:35:04.350Z|00015|ofproto_dpif|INFO|system@ovs-system: > Datapath does not support ct_state_nat OK, "Datapath does not support ct_*" confirms that the kernel openvswitch module doesn't support the conntrack features needed by OVN. Most likely the loaded module is the stock CentOS one, you can build the out-of-tree kernel module RPM from the same source tree where you built the other OVS/OVN RPMs via: make rpm-fedora-kmod This should leave an RPM named something like: openvswitch-kmod-2.6.90-1.el7.centos.x86_64.rpm Install that and reboot and things should be working better. Regards, Lance > > Your help is greatly appreciated! > > Devin > > On Mon, Dec 5, 2016 at 12:31 PM, Lance Richardson <lrich...@redhat.com> > wrote: > > > > From: "Devin Acosta" <de...@pabstatencio.com> > > > To: "Marcin Mirecki" <mmire...@redhat.com> > > > Cc: "users" <Users@ovirt.org> > > > Sent: Monday, December 5, 2016 12:11:46 PM > > > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of > > instances between nodes. > > > > > > Marcin, > > > > > > Also I noticed in your original post it mentions: > > > > > > ip link - the result should include a link called genev_sys_ ... > > > > > > I noticed that on my hosts I don't see any links with name: genev_sys_ ?? > > > Could this be a problem? > > > > > > lo: > > > enp4s0f0: > > > enp4s0f1: > > > enp7s0f0: > > > enp7s0f1: > > > bond0: > > > DEV-NOC: > > > ovirtmgmt: > > > bond0.700@bond0: > > > DEV-VM-NET: > > > bond0.705@bond0: > > > ;vdsmdummy;: > > > vnet0: > > > vnet1: > > > vnet2: > > > vnet3: > > > vnet4: > > > ovs-system: > > > br-int: > > > vnet5: > > > vnet6: > > > > > > > Hi Devin, > > > > What distribution and kernel version are you using? > > > > One thing you could check is whether the vport_geneve kernel module > > is being loaded, e.g. you should see something like: > > > > $ lsmod | grep vport > > vport_geneve 12560 1 > > openvswitch 246755 5 vport_geneve > > > > If vport_geneve is not loaded, you could "sudo modprobe vport_geneve" > > to make sure it's available and can be loaded. > > > > The first 100 lines or so of ovs-vswitchd.log might have some useful > > information about where things are going wrong. > > > > It does sound as though there is some issue with geneve tunnels, > > which would certainly explain issues with inter-node traffic. > > > > Regards, > > > > Lance > > > > > > -- > > Devin Acosta > Red Hat Certified Architect, LinuxStack > 602-354-1220 || de...@linuxguru.co > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
id=0x31): (***truncated to 64 bytes from 136***) 04 0e 00 88 00 00 00 31-00 00 00 00 00 00 00 00 |...1| 0010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |6..d| 0020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 || 0030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."| 2016-12-05T21:15:47.902Z|00026|ofctrl|INFO|OpenFlow error: OFPT_ERROR (OF1.3) (xid=0x3d): OFPBAC_BAD_TYPE OFPT_FLOW_MOD (OF1.3) (xid=0x3d): (***truncated to 64 bytes from 160***) 04 0e 00 a0 00 00 00 3d-00 00 00 00 00 00 00 00 |...=| 0010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...d| 0020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 || 0030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."| 2016-12-05T21:15:47.902Z|00027|ofctrl|INFO|OpenFlow error: OFPT_ERROR (OF1.3) (xid=0x3f): OFPBAC_BAD_TYPE OFPT_FLOW_MOD (OF1.3) (xid=0x3f): (***truncated to 64 bytes from 120***) 04 0e 00 78 00 00 00 3f-00 00 00 00 00 00 00 00 |...x...?| 0010 00 00 00 00 00 00 00 00-15 00 00 00 00 00 00 64 |...d| 0020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 || 0030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."| 2016-12-05T21:15:47.902Z|00028|ofctrl|INFO|OpenFlow error: OFPT_ERROR (OF1.3) (xid=0x40): OFPBAC_BAD_TYPE OFPT_FLOW_MOD (OF1.3) (xid=0x40): (***truncated to 64 bytes from 160***) 04 0e 00 a0 00 00 00 40-00 00 00 00 00 00 00 00 |...@| 0010 00 00 00 00 00 00 00 00-36 00 00 00 00 00 00 64 |6..d| 0020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 || 0030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."| 2016-12-05T21:15:47.902Z|00029|ofctrl|INFO|OpenFlow error: OFPT_ERROR (OF1.3) (xid=0x46): OFPBAC_BAD_TYPE OFPT_FLOW_MOD (OF1.3) (xid=0x46): (***truncated to 64 bytes from 136***) 04 0e 00 88 00 00 00 46-00 00 00 00 00 00 00 00 |...F| 0010 00 00 00 00 00 00 00 00-19 00 00 00 00 00 00 64 |...d| 0020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 || 0030 00 01 00 22 80 00 0a 02-08 00 00 01 01 08 00 00 |..."| 2016-12-05T21:15:47.902Z|00030|ofctrl|INFO|OpenFlow error: OFPT_ERROR (OF1.3) (xid=0x54): OFPBAC_BAD_TYPE OFPT_FLOW_MOD (OF1.3) (xid=0x54): (***truncated to 64 bytes from 120***) 04 0e 00 78 00 00 00 54-00 00 00 00 00 00 00 00 |...x...T| 0010 00 00 00 00 00 00 00 00-15 00 00 00 00 00 00 64 |...d| 0020 ff ff ff ff ff ff ff ff-ff ff ff ff 00 00 00 00 || 0030 00 01 00 22 80 00 0a 02-86 dd 00 01 01 08 00 00 |..."........| Your help is greatly appreciated! Devin On Mon, Dec 5, 2016 at 12:31 PM, Lance Richardson <lrich...@redhat.com> wrote: > > From: "Devin Acosta" <de...@pabstatencio.com> > > To: "Marcin Mirecki" <mmire...@redhat.com> > > Cc: "users" <Users@ovirt.org> > > Sent: Monday, December 5, 2016 12:11:46 PM > > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of > instances between nodes. > > > > Marcin, > > > > Also I noticed in your original post it mentions: > > > > ip link - the result should include a link called genev_sys_ ... > > > > I noticed that on my hosts I don't see any links with name: genev_sys_ ?? > > Could this be a problem? > > > > lo: > > enp4s0f0: > > enp4s0f1: > > enp7s0f0: > > enp7s0f1: > > bond0: > > DEV-NOC: > > ovirtmgmt: > > bond0.700@bond0: > > DEV-VM-NET: > > bond0.705@bond0: > > ;vdsmdummy;: > > vnet0: > > vnet1: > > vnet2: > > vnet3: > > vnet4: > > ovs-system: > > br-int: > > vnet5: > > vnet6: > > > > Hi Devin, > > What distribution and kernel version are you using? > > One thing you could check is whether the vport_geneve kernel module > is being loaded, e.g. you should see something like: > > $ lsmod | grep vport > vport_geneve 12560 1 > openvswitch 246755 5 vport_geneve > > If vport_geneve is not loaded, you could "sudo modprobe vport_geneve" > to make sure it's available and can be loaded. > > The first 100 lines or so of ovs-vswitchd.log might have some useful > information about where things are going wrong. > > It does sound as though there is some issue with geneve tunnels, > which would certainly explain issues with inter-node traffic. > > Regards, > > Lance > -- Devin Acosta Red Hat Certified Architect, LinuxStack 602-354-1220 || de...@linuxguru.co ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
> From: "Devin Acosta" <de...@pabstatencio.com> > To: "Marcin Mirecki" <mmire...@redhat.com> > Cc: "users" <Users@ovirt.org> > Sent: Monday, December 5, 2016 12:11:46 PM > Subject: Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances > between nodes. > > Marcin, > > Also I noticed in your original post it mentions: > > ip link - the result should include a link called genev_sys_ ... > > I noticed that on my hosts I don't see any links with name: genev_sys_ ?? > Could this be a problem? > > lo: > enp4s0f0: > enp4s0f1: > enp7s0f0: > enp7s0f1: > bond0: > DEV-NOC: > ovirtmgmt: > bond0.700@bond0: > DEV-VM-NET: > bond0.705@bond0: > ;vdsmdummy;: > vnet0: > vnet1: > vnet2: > vnet3: > vnet4: > ovs-system: > br-int: > vnet5: > vnet6: > Hi Devin, What distribution and kernel version are you using? One thing you could check is whether the vport_geneve kernel module is being loaded, e.g. you should see something like: $ lsmod | grep vport vport_geneve 12560 1 openvswitch 246755 5 vport_geneve If vport_geneve is not loaded, you could "sudo modprobe vport_geneve" to make sure it's available and can be loaded. The first 100 lines or so of ovs-vswitchd.log might have some useful information about where things are going wrong. It does sound as though there is some issue with geneve tunnels, which would certainly explain issues with inter-node traffic. Regards, Lance ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
Marcin,
Also I noticed in your original post it mentions:
ip link - the result should include a link called genev_sys_ ...
I noticed that on my hosts I don't see any links with name: genev_sys_ ??
Could this be a problem?
lo:
enp4s0f0:
enp4s0f1:
enp7s0f0:
enp7s0f1:
bond0:
DEV-NOC:
ovirtmgmt:
bond0.700@bond0:
DEV-VM-NET:
bond0.705@bond0:
;vdsmdummy;:
vnet0:
vnet1:
vnet2:
vnet3:
vnet4:
ovs-system:
br-int:
vnet5:
vnet6:
However, the br-int appears to have been configured:
[root@las01-902-001 ~]# ovs-vsctl show
4c817c66-9842-471d-b53a-963e27e3364f
Bridge br-int
fail_mode: secure
Port "vnet6"
Interface "vnet6"
Port "vnet5"
Interface "vnet5"
Port "ovn-456949-0"
Interface "ovn-456949-0"
type: geneve
options: {csum="true", key=flow, remote_ip="172.10.10.74"}
Port "ovn-252778-0"
Interface "ovn-252778-0"
type: geneve
options: {csum="true", key=flow, remote_ip="172.10.10.75"}
Port br-int
Interface br-int
type: internal
ovs_version: "2.6.90"
However there is no traffic showing:
[root@las01-902-001 ~]# ifconfig br-int
br-int: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether 2e:c4:a6:fa:0c:40 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
On Mon, Dec 5, 2016 at 8:05 AM, Devin Acosta <de...@pabstatencio.com> wrote:
>
> Marcin,
>
> For OVN to work properly does the port that the traffic flows over need to
> be a bridge, or OVS port? Right now it's just going over the ovirtmgmt
> network which is just a standard port. I know like in Neutron you have to
> configure (br-ex) and then it would need to be using the OVS protocol, and
> then all the nodes would need to be an OVS port. I presume OVN tries to
> simplify this setup?
> I also seen that there is (openvswitch-ovn-vtep), would this need to be
> configured in any way?
>
>
> On Mon, Dec 5, 2016 at 1:43 AM, Marcin Mirecki <mmire...@redhat.com>
> wrote:
>
>> Devin,
>>
>> Please not the OVN-controller is not the central part where OVN northd is
>> running.
>> OVN-controllers are the OVN processes deployed on the hosts.
>> The correct usage of the 'vdsm-tool ovn-config'.
>> - the IP of the OVN-central (not to be confused with OVN-controllers,
>> which is the part of OVN running on the hosts)
>> - the local host IP to be used for tunneling to other OVN hosts
>> for example, if the OVN-central IP should be 10.10.10.1, and the IP of
>> the local host used for tunneling: 10.10.10.101:
>> vdsm-tool ovn-config 10.10.10.1 10.10.10.101
>>
>> Looking at the output of 'ovs-vsctl' the tunnels have been created.
>>
>> The OVN log saying 'dropping duplicate flow' is worrying, let me forward
>> this to
>> the OVN team to take a look at it.
>>
>> Marcin
>>
>>
>>
>> - Original Message -
>> > From: "Devin Acosta" <de...@pabstatencio.com>
>> > To: "users" <Users@ovirt.org>
>> > Sent: Saturday, December 3, 2016 12:24:21 AM
>> > Subject: [ovirt-users] oVIRT 4 / OVN / Communication issues of
>> instances between nodes.
>> >
>> >
>> > Note: When I configured vdsm-tool ovn-config, I passed it the IP
>> address of
>> > the OVN-Controller which is using the ovirtmgmt network, which is just
>> one
>> > of the NIC's on the nodes.
>> >
>> > I am opening up new thread as this I feel differs a bit from my original
>> > request. I have OVN which I believe is deployed correctly. I have
>> noticed
>> > that if instances get spun up on the same oVIRT node they can all talk
>> > without issues to one another, however if one instance gets spun up on
>> > another node even if it has the same (OVN network/subnet), it can't
>> ping or
>> > reach other instances in the subnet. I noticed that the OVN-Controller
>> of
>> > the instance that can't talk is logging:
>> >
>> > 2016-12-02T22:50:54.907Z|00181|pinctrl|INFO|DHCPOFFER 00:1a:4a:16:01:5c
>> > 10.10.10.4
>> > 2016-12-02T22:50:54.908Z|00182|pinctrl|INFO|DHCPACK 00:1a:4a:16:01:5c
>> > 10.10.10.4
>> > 2016-12-02T22:50:55.695Z|00183|ofctrl|INFO|Dropped 7 log messages in
>> last 10
>> > seconds (most recently, 0 seconds ago) due to excessive rat
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
Lance, We have a problem with communication between different hosts in OVN. Could you please take a look at the log below? The part with "dropping duplicate flow" sounds worrying. Thanks, Marcin - Original Message - > From: "Devin Acosta" <de...@pabstatencio.com> > To: "users" <Users@ovirt.org> > Sent: Saturday, December 3, 2016 12:24:21 AM > Subject: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances > between nodes. > > > Note: When I configured vdsm-tool ovn-config, I passed it the IP address of > the OVN-Controller which is using the ovirtmgmt network, which is just one > of the NIC's on the nodes. > > I am opening up new thread as this I feel differs a bit from my original > request. I have OVN which I believe is deployed correctly. I have noticed > that if instances get spun up on the same oVIRT node they can all talk > without issues to one another, however if one instance gets spun up on > another node even if it has the same (OVN network/subnet), it can't ping or > reach other instances in the subnet. I noticed that the OVN-Controller of > the instance that can't talk is logging: > > 2016-12-02T22:50:54.907Z|00181|pinctrl|INFO|DHCPOFFER 00:1a:4a:16:01:5c > 10.10.10.4 > 2016-12-02T22:50:54.908Z|00182|pinctrl|INFO|DHCPACK 00:1a:4a:16:01:5c > 10.10.10.4 > 2016-12-02T22:50:55.695Z|00183|ofctrl|INFO|Dropped 7 log messages in last 10 > seconds (most recently, 0 seconds ago) due to excessive rate > 2016-12-02T22:50:55.695Z|00184|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:10.705Z|00185|ofctrl|INFO|Dropped 6 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:10.705Z|00186|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:20.710Z|00187|ofctrl|INFO|Dropped 4 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:20.710Z|00188|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:35.718Z|00189|ofctrl|INFO|Dropped 5 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:35.718Z|00190|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:45.724Z|00191|ofctrl|INFO|Dropped 3 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:45.724Z|00192|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:55.730Z|00193|ofctrl|INFO|Dropped 5 log messages in last 10 > seconds (most recently, 0 seconds ago) due to excessive rate > 2016-12-02T22:51:55.730Z|00194|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:10.738Z|00195|ofctrl|INFO|Dropped 5 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:10.739Z|00196|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:20.744Z|00197|ofctrl|INFO|Dropped 3 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:20.744Z|00198|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:35.752Z|00199|ofctrl|INFO|Dropped 5 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:35.752Z|00200|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:45.758Z|00201|ofctrl|INFO|Dropped 4 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:45.758Z|00202|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > > From the OVN-Controller: > > [root@dev001-022-002 ~]# ovn-nbctl show > switch ddb3b92f-b359-4b59-a41a-ebae6df7fe9a (devins-net) > port 6b289418-8b8e-42b4-8334-c71584afcd3e > addresses: ["00:1a:4a:16:01:5c dynamic"] > port 71ef81f1-7c20-4c68-b536-d274703f7541 > addresses: ["00:1a:4a:16:01:61 dynamic"] > port 91d4f4f5-4b9f-42c0-aa2c-8a101474bb84 > addresses: ["00:1a:4a:16:01:5e dynamic"] > > Do I need to do something special in order to allow communication between > nodes of instances on same OVN network? > > Output of ovs-vsctl show from node3: > > 61af799c-a621-445e-81
Re: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
Devin, Please not the OVN-controller is not the central part where OVN northd is running. OVN-controllers are the OVN processes deployed on the hosts. The correct usage of the 'vdsm-tool ovn-config'. - the IP of the OVN-central (not to be confused with OVN-controllers, which is the part of OVN running on the hosts) - the local host IP to be used for tunneling to other OVN hosts for example, if the OVN-central IP should be 10.10.10.1, and the IP of the local host used for tunneling: 10.10.10.101: vdsm-tool ovn-config 10.10.10.1 10.10.10.101 Looking at the output of 'ovs-vsctl' the tunnels have been created. The OVN log saying 'dropping duplicate flow' is worrying, let me forward this to the OVN team to take a look at it. Marcin - Original Message - > From: "Devin Acosta" <de...@pabstatencio.com> > To: "users" <Users@ovirt.org> > Sent: Saturday, December 3, 2016 12:24:21 AM > Subject: [ovirt-users] oVIRT 4 / OVN / Communication issues of instances > between nodes. > > > Note: When I configured vdsm-tool ovn-config, I passed it the IP address of > the OVN-Controller which is using the ovirtmgmt network, which is just one > of the NIC's on the nodes. > > I am opening up new thread as this I feel differs a bit from my original > request. I have OVN which I believe is deployed correctly. I have noticed > that if instances get spun up on the same oVIRT node they can all talk > without issues to one another, however if one instance gets spun up on > another node even if it has the same (OVN network/subnet), it can't ping or > reach other instances in the subnet. I noticed that the OVN-Controller of > the instance that can't talk is logging: > > 2016-12-02T22:50:54.907Z|00181|pinctrl|INFO|DHCPOFFER 00:1a:4a:16:01:5c > 10.10.10.4 > 2016-12-02T22:50:54.908Z|00182|pinctrl|INFO|DHCPACK 00:1a:4a:16:01:5c > 10.10.10.4 > 2016-12-02T22:50:55.695Z|00183|ofctrl|INFO|Dropped 7 log messages in last 10 > seconds (most recently, 0 seconds ago) due to excessive rate > 2016-12-02T22:50:55.695Z|00184|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:10.705Z|00185|ofctrl|INFO|Dropped 6 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:10.705Z|00186|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:20.710Z|00187|ofctrl|INFO|Dropped 4 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:20.710Z|00188|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:35.718Z|00189|ofctrl|INFO|Dropped 5 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:35.718Z|00190|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:45.724Z|00191|ofctrl|INFO|Dropped 3 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:51:45.724Z|00192|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:51:55.730Z|00193|ofctrl|INFO|Dropped 5 log messages in last 10 > seconds (most recently, 0 seconds ago) due to excessive rate > 2016-12-02T22:51:55.730Z|00194|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:10.738Z|00195|ofctrl|INFO|Dropped 5 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:10.739Z|00196|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:20.744Z|00197|ofctrl|INFO|Dropped 3 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:20.744Z|00198|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:35.752Z|00199|ofctrl|INFO|Dropped 5 log messages in last 15 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:35.752Z|00200|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > 2016-12-02T22:52:45.758Z|00201|ofctrl|INFO|Dropped 4 log messages in last 10 > seconds (most recently, 5 seconds ago) due to excessive rate > 2016-12-02T22:52:45.758Z|00202|ofctrl|INFO|dropping duplicate flow: > table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33) > > From the OVN-Controller: > > [root@dev001-022-002 ~]# ovn-nbctl show > switch ddb3b92f-b359-4b59
[ovirt-users] oVIRT 4 / OVN / Communication issues of instances between nodes.
Note: When I configured vdsm-tool ovn-config, I passed it the IP address of
the OVN-Controller which is using the ovirtmgmt network, which is just one
of the NIC's on the nodes.
I am opening up new thread as this I feel differs a bit from my original
request. I have OVN which I believe is deployed correctly. I have noticed
that if instances get spun up on the same oVIRT node they can all talk
without issues to one another, however if one instance gets spun up on
another node even if it has the same (OVN network/subnet), it can't ping or
reach other instances in the subnet. I noticed that the OVN-Controller of
the instance that can't talk is logging:
2016-12-02T22:50:54.907Z|00181|pinctrl|INFO|DHCPOFFER 00:1a:4a:16:01:5c
10.10.10.4
2016-12-02T22:50:54.908Z|00182|pinctrl|INFO|DHCPACK 00:1a:4a:16:01:5c
10.10.10.4
2016-12-02T22:50:55.695Z|00183|ofctrl|INFO|Dropped 7 log messages in last
10 seconds (most recently, 0 seconds ago) due to excessive rate
2016-12-02T22:50:55.695Z|00184|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:51:10.705Z|00185|ofctrl|INFO|Dropped 6 log messages in last
15 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:51:10.705Z|00186|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:51:20.710Z|00187|ofctrl|INFO|Dropped 4 log messages in last
10 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:51:20.710Z|00188|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:51:35.718Z|00189|ofctrl|INFO|Dropped 5 log messages in last
15 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:51:35.718Z|00190|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:51:45.724Z|00191|ofctrl|INFO|Dropped 3 log messages in last
10 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:51:45.724Z|00192|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:51:55.730Z|00193|ofctrl|INFO|Dropped 5 log messages in last
10 seconds (most recently, 0 seconds ago) due to excessive rate
2016-12-02T22:51:55.730Z|00194|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:52:10.738Z|00195|ofctrl|INFO|Dropped 5 log messages in last
15 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:52:10.739Z|00196|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:52:20.744Z|00197|ofctrl|INFO|Dropped 3 log messages in last
10 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:52:20.744Z|00198|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:52:35.752Z|00199|ofctrl|INFO|Dropped 5 log messages in last
15 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:52:35.752Z|00200|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
2016-12-02T22:52:45.758Z|00201|ofctrl|INFO|Dropped 4 log messages in last
10 seconds (most recently, 5 seconds ago) due to excessive rate
2016-12-02T22:52:45.758Z|00202|ofctrl|INFO|dropping duplicate flow:
table_id=32, priority=150, reg10=0x2/0x2, actions=resubmit(,33)
>From the OVN-Controller:
[root@dev001-022-002 ~]# ovn-nbctl show
switch ddb3b92f-b359-4b59-a41a-ebae6df7fe9a (devins-net)
port 6b289418-8b8e-42b4-8334-c71584afcd3e
addresses: ["00:1a:4a:16:01:5c dynamic"]
port 71ef81f1-7c20-4c68-b536-d274703f7541
addresses: ["00:1a:4a:16:01:61 dynamic"]
port 91d4f4f5-4b9f-42c0-aa2c-8a101474bb84
addresses: ["00:1a:4a:16:01:5e dynamic"]
Do I need to do something special in order to allow communication between
nodes of instances on same OVN network?
Output of ovs-vsctl show from node3:
61af799c-a621-445e-8183-23dcb38ea3cc
Bridge br-int
fail_mode: secure
Port "ovn-456949-0"
Interface "ovn-456949-0"
type: geneve
options: {csum="true", key=flow, remote_ip="172.10.10.74"}
Port "ovn-c0dc09-0"
Interface "ovn-c0dc09-0"
type: geneve
options: {csum="true", key=flow, remote_ip="172.10.10.73"}
Port br-int
Interface br-int
type: internal
ovs_version: "2.6.90"
--
Devin Acosta
Red Hat Certified Architect, LinuxStack
602-354-1220 || de...@linuxguru.co
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
