[ovs-discuss] OVN scale

2020-07-20 Thread Tony Liu
Hi folks, ​This is my first email here. Please let me know if there is any rule or convention I need to follow. Don't want to break it. I started with OpenStack Ussuri and OVN 20.03.0 recently and currently running some scaling test. Searched around for scaling info and noticed some improvements

Re: [ovs-discuss] OVN scale

2020-07-20 Thread Tony Liu
nt OVN. Any comment is welcome. Thanks! Tony From: Cory Hawkless Sent: July 20, 2020 10:04 PM To: Tony Liu ; ovs-discuss@openvswitch.org Subject: RE: OVN scale I would expect to see 100% cpu utilisation on anything involved in the process of creating 4000 networks

[ovs-discuss] OVN router interface status

2020-07-21 Thread Tony Liu
Hi, The setup is OpenStack Ussuri and OVN 20.03.0. After adding bunch of networks to a router via Neutron API, some of those router interfaces are "Active" and some of them are "Down". Is this expected? [cid:bfcd4781-c418-4695-afbc-83b677a5a6c2] Thanks! Tony ___

Re: [ovs-discuss] OVN scale

2020-07-21 Thread Tony Liu
el Alvarez Sent: July 21, 2020 12:06 AM To: Tony Liu Cc: Cory Hawkless ; ovs-discuss@openvswitch.org ; Dumitru Ceara Subject: Re: [ovs-discuss] OVN scale Hi Tony, all On 21 Jul 2020, at 07:53, Tony Liu wrote:  Hi Cory, With 4000 networks all connecting to one router with external GW, all net

Re: [ovs-discuss] OVN scale

2020-07-21 Thread Tony Liu
lcome. Thanks! Tony ____ From: Tony Liu Sent: July 21, 2020 10:22 AM To: Daniel Alvarez Cc: Cory Hawkless ; ovs-discuss@openvswitch.org ; Dumitru Ceara Subject: Re: [ovs-discuss] OVN scale Hi Daniel, all 4000 networks and 50 routers, 200 networks on each route

Re: [ovs-discuss] OVN scale

2020-07-21 Thread Tony Liu
sync. Is there any way to force ovn-northd sync them? Thanks! Tony From: Tony Liu Sent: July 21, 2020 08:39 PM To: Daniel Alvarez Cc: Cory Hawkless ; ovs-discuss@openvswitch.org ; Dumitru Ceara Subject: Re: [ovs-discuss] OVN scale When create a network (and subnet) on O

Re: [ovs-discuss] OVN nb-db and sb-db out of sync

2020-07-22 Thread Tony Liu
elete it? Thanks! Tony From: dev on behalf of Tony Liu Sent: July 22, 2020 11:33 AM To: ovs-dev Subject: [ovs-dev] OVN nb-db and sb-db out of sync Hi, During a scaling test where 4000 networks are created from OpenStack, I see that nb-db and sb-db are out of sync.

[ovs-discuss] OVN: Two datapath-bindings are created for the same logical-switch

2020-07-23 Thread Tony Liu
eader; trying another server Thanks! Tony From: Numan Siddique Sent: July 23, 2020 03:15 AM To: Tony Liu Cc: ovs-dev ; ovs-discuss@openvswitch.org Subject: Re: [ovs-discuss] OVN nb-db and sb-db out of sync On Thu, Jul 23, 2020 at 8:22 AM Tony Liu mailto:to

Re: [ovs-discuss] OVN nb-db and sb-db out of sync

2020-07-23 Thread Tony Liu
t;ovn-sbctl destroy" to remove the record, but onv-sbctl is stuck there forever. Thanks! Tony From: Numan Siddique Sent: July 23, 2020 03:15 AM To: Tony Liu Cc: ovs-dev ; ovs-discuss@openvswitch.org Subject: Re: [ovs-discuss] OVN nb-db and sb-db out of

[ovs-discuss] OVN Controller Incremental Processing

2020-07-23 Thread Tony Liu
Hi, Is this implemented and released? https://www.slideshare.net/hanzhou1978/ovn-controller-incremental-processing Could anyone share an update on this? Thanks! Tony ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailma

Re: [ovs-discuss] OVN nb-db and sb-db out of sync

2020-07-23 Thread Tony Liu
ode, that "stale data" is caused by some index mismatch. Any clues? Thanks! Tony From: Numan Siddique Sent: July 23, 2020 11:54 AM To: Tony Liu Cc: ovs-dev ; ovs-discuss@openvswitch.org Subject: Re: [ovs-discuss] OVN nb-db and sb-db out of sync On

[ovs-discuss] OVN: ovn-sbctl backoff

2020-07-23 Thread Tony Liu
Hi, In case of ovsdb cluster, when I run ovn-sbctl, it connects to the unix socket of local sb-db. If local sb-db is not the leader, ovn-sbctl tries another server to look for the leader. How does ovn-sbctl connect to another server? By which connection? How does ovn-sbctl know the connection? O

Re: [ovs-discuss] OVN: ovn-sbctl backoff

2020-07-23 Thread Tony Liu
Appreciate it! You connected dots for me. Tony From: Han Zhou Sent: July 23, 2020 05:10 PM To: Tony Liu Cc: Han Zhou ; ovs-discuss Subject: Re: [ovs-discuss] OVN: ovn-sbctl backoff On Thu, Jul 23, 2020 at 4:34 PM mailto:tonyliu0...@hotmail.com>>

Re: [ovs-discuss] [ovs-dev] OVN Controller Incremental Processing

2020-07-23 Thread Tony Liu
. How can I trigger ovn-northd to finish the translation for all the rest logical switches? Thanks! Tony From: Han Zhou Sent: July 23, 2020 04:19 PM To: Tony Liu Cc: Han Zhou ; ovs-dev ; ovs-discuss Subject: Re: [ovs-dev] OVN Controller Incremental Processing

Re: [ovs-discuss] [ovs-dev] OVN: Two datapath-bindings are created for the same logical-switch

2020-07-23 Thread Tony Liu
. Thanks again! Tony From: Han Zhou Sent: July 23, 2020 06:53 PM To: Tony Liu ; Ben Pfaff Cc: Numan Siddique ; ovs-dev ; ovs-discuss@openvswitch.org Subject: Re: [ovs-dev] OVN: Two datapath-bindings are created for the same logical-switch On Thu, Jul 23, 2020

Re: [ovs-discuss] OVN scale

2020-07-27 Thread Tony Liu
), 4K routers (one for each network), 50 routers (one for 80 networks), 1 router (for all 4K networks)... All networks are isolated by ACL on the logical router. Which option should work better? Any comment is appreciated. Thanks! Tony ____ From: discuss on behal

[ovs-discuss] OVN nb-db and sb-db election timer

2020-07-27 Thread Tony Liu
Hi, During scaling test, when sb-db is busy, followers believe the leader is dead and started election request. Some inconsistency happens during such leader switch. Two datapath bindings are created for the same logical switch. To avoid such case, I was recommended to increase election timer x

Re: [ovs-discuss] Display OpenFlow port number, interface name and bridge name in single OvS CLI command

2020-07-28 Thread Tony Liu
for p in $(ovs-vsctl list-ports br-int); do \ ovs-vsctl -f table --columns=ofport,name list interface $p; \ done Tony From: discuss on behalf of Matteo Olivi Sent: July 28, 2020 11:14 AM To: ovs-discuss@openvswitch.org Subject: [ovs-discuss] Display Op

[ovs-discuss] OVN: resync nb-db to sb-db

2020-07-28 Thread Tony Liu
Hi, When I run a script to create bunch of networks and routers from OpenStack, for whatever reason, nb-db is fully updated, but sb-db is only partially updated. For example, there are 500 logical routers in nb-db, but only 218 datapath bindings in sb-db. In this case, is there any way to resyn

[ovs-discuss] OVN: configuration in Neutron DB?

2020-07-28 Thread Tony Liu
Hi, In case of integration with OpenStack, for example, when a client requests to create a network, is this network configuration saved in both Neutron DB and OVN DB, or OVN DB only? Also, when a client gets a network from Neutron API, is the configuration read from Neutron DB or OVN DB? Other

Re: [ovs-discuss] OVN: configuration in Neutron DB?

2020-07-28 Thread Tony Liu
et OK response of creation request. Is this expected? Thanks! Tony From: dev on behalf of Tony Liu Sent: July 28, 2020 10:37 PM To: ovs-discuss@openvswitch.org ; ovs-...@openvswitch.org Subject: [ovs-dev] OVN: configuration in Neutron DB? Hi, In ca

Re: [ovs-discuss] OVN: configuration in Neutron DB?

2020-07-28 Thread Tony Liu
Thanks Numan! Tony From: Numan Siddique Sent: July 28, 2020 11:41 PM To: Tony Liu Cc: ovs-discuss@openvswitch.org ; ovs-...@openvswitch.org ; Daniel Alvarez Sanchez ; Lucas Alvares Gomes Martins Subject: Re: [ovs-discuss] OVN: configuration in Neutron DB

Re: [ovs-discuss] [ovs-dev] OVN: configuration in Neutron DB?

2020-07-29 Thread Tony Liu
Alvares Gomes<mailto:lucasago...@gmail.com> Sent: Wednesday, July 29, 2020 2:22 AM To: Numan Siddique<mailto:num...@ovn.org> Cc: Tony Liu<mailto:tonyliu0...@hotmail.com>; ovs-...@openvswitch.org<mailto:ovs-...@openvswitch.org>; Lucas Alvares Gomes Martins<mailto:lmart...@red

[ovs-discuss] [OVN] DB backup and restore

2020-07-29 Thread Tony Liu
Hi, There is any guidance to backup and restore OVN nb-db and sb-db? Is /var/lib/openvswitch/ovn-[ns]b/ovn[ns]b.db the only database file? For 3-node DB cluster, is replication 3 (the data is replicated onto All 3 nodes)? Are DB files on 3 nodes identical? If I stop a DB follower and empty the

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-30 Thread Tony Liu
option. Any thoughts here? Thanks! Tony From: Han Zhou<mailto:hz...@ovn.org> Sent: Wednesday, July 29, 2020 11:38 PM To: Tony Liu<mailto:tonyliu0...@hotmail.com> Cc: ovs-discuss<mailto:ovs-discuss@openvswitch.org>; ovs-dev<mailto:ovs-...@openvswitch.org> Subject: Re: [o

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-30 Thread Tony Liu
is no configuration update from OpenStack side while running such snapshot and restore process. Thanks! Tony From: Han Zhou<mailto:hz...@ovn.org> Sent: Thursday, July 30, 2020 12:23 PM To: Tony Liu<mailto:tonyliu0...@hotmail.com> Cc: Han Zhou<mailto:hz...@ovn.org>; ovs-dis

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-30 Thread Tony Liu
. ovsdb-client: ovsdb error: /dev/stdin: cannot identify file type The file was created by "backup" command. Thanks! Tony From: Tony Liu<mailto:tonyliu0...@hotmail.com> Sent: Thursday, July 30, 2020 3:41 PM To: Han Zhou<mailto:hz...@ovn.org> Cc: Han Zhou<mailto:hz...@ovn

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-30 Thread Tony Liu
? Thanks! Tony From: Tony Liu<mailto:tonyliu0...@hotmail.com> Sent: Thursday, July 30, 2020 4:38 PM To: Han Zhou<mailto:hz...@ovn.org> Cc: Han Zhou<mailto:hz...@ovn.org>; ovs-dev<mailto:ovs-...@openvswitch.org>; ovs-discuss<mailto:ovs-discuss@openvswitch.org> Subje

Re: [ovs-discuss] [OVN] Where is DB cluster info stored?

2020-07-30 Thread Tony Liu
Sorry for bothering, please discard this question. Server ID is in DB file as well. Tony From: Tony Liu<mailto:tonyliu0...@hotmail.com> Sent: Thursday, July 30, 2020 5:22 PM To: ovs-discuss<mailto:ovs-discuss@openvswitch.org> Subject: [OVN] Where is DB cluster info stored? Hi, W

[ovs-discuss] [OVN] Where is DB cluster info stored?

2020-07-30 Thread Tony Liu
Hi, Where is RAFT DB cluster info stored? I see cluster ID in the DB file, but where is server ID stored? Thanks! Tony ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-30 Thread Tony Liu
perienced such issue? To Numan, it would great if you could share the details to use Neutron-ovn-sync-util. Thanks! Tony From: Tony Liu<mailto:tonyliu0...@hotmail.com> Sent: Thursday, July 30, 2020 4:51 PM To: Numan Siddique<mailto:nusid...@redhat.com>; Han Zhou<mailto:hz...@ov

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-30 Thread Tony Liu
Mmm... nb-db rolled back, but sb-db is not re-synced, ovn-northd complaints "clustered database server has stale data; trying another server". Any way to workaround it or I need to snapshot and rollback sb-db as well? Thanks! Tony From: Tony Liu<mailto:tonyliu0...@hotmail.com>

Re: [ovs-discuss] [ovs-dev] OVN: Two datapath-bindings are created for the same logical-switch

2020-07-30 Thread Tony Liu
supported for ovn-northd to delete a configuration that is being referenced. Is that right? If yes, should we fix it or it's the intention? Thanks! Tony From: Tony Liu<mailto:tonyliu0...@hotmail.com> Sent: Thursday, July 23, 2020 7:51 PM To: Han Zhou<mailto:zhou...@gmail.com>; B

Re: [ovs-discuss] [ovs-dev] OVN: Two datapath-bindings are created for the same logical-switch

2020-07-30 Thread Tony Liu
I agree, that will stop the duplication from being created. Thanks! Tony From: Han Zhou<mailto:zhou...@gmail.com> Sent: Thursday, July 30, 2020 7:30 PM To: Tony Liu<mailto:tonyliu0...@hotmail.com> Cc: Ben Pfaff<mailto:b...@ovn.org>; ovs-dev<mailto:ovs-...@openvswit

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-30 Thread Tony Liu
restore. http://www.openvswitch.org/support/dist-docs/ovsdb-client.1.txt Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Thursday, July 30, 2020 7:19 PM > To: Tony Liu > Cc: Numan Siddique ; Han Zhou ; ovs- > dev ; ovs-discuss > Subject: Re: [ovs-discuss

Re: [ovs-discuss] [OVN] DB backup and restore

2020-07-31 Thread Tony Liu
Thanks Han! It's clear! Tony > -Original Message- > From: Han Zhou > Sent: Friday, July 31, 2020 10:11 AM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; ovs- > dev ; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] DB backup and restore > > > >

[ovs-discuss] [OVN] ovn-northd takes much CPU when no configuration update

2020-07-31 Thread Tony Liu
Hi, I see the active ovn-northd takes much CPU (30% - 100%) when there is no configuration from OpenStack, nothing happening on all chassis nodes either. Is this expected? What is it busy with? 2020-07-31T23:08:09.511Z|04267|poll_loop|DBG|wakeup due to [POLLIN] on fd 8 (10.6.20.84:4435

[ovs-discuss] [OVN] ovn-northd HA

2020-08-01 Thread Tony Liu
Hi, I have a few questions about ovn-northd HA. Does the lock for active ovn-northd have to be acquired from the leader of sb-db? If ovn-northd didn't acquire the lock, it becomes standby. Does it keep trying to acquire the lock, or wait for notification, or monitor the active ovn-northd? If it

Re: [ovs-discuss] [OVN] ovn-northd HA

2020-08-01 Thread Tony Liu
does that sb-db leader switch affect such sync? Thanks! Tony > -Original Message- > From: dev On Behalf Of Tony Liu > Sent: Saturday, August 1, 2020 5:26 PM > To: ovs-discuss ; ovs-dev d...@openvswitch.org> > Subject: [ovs-dev] [OVN] ovn-northd HA > > Hi, >

[ovs-discuss] [OVN] constraint violation error from Neutron OVN ML2 driver

2020-08-03 Thread Tony Liu
Hi, Any clues about this error? It is reproduceable but not consistently. There are 3 Neutron nodes and 3 OVN DB nodes (RAFT cluster). It happened when connecting network to router by OpenStack cli. === 2020-08-03 12:30:17.054 22 ERROR ovsdbapp.backend.ovs_idl.tran

[ovs-discuss] [OVN] no response to inactivity probe

2020-08-03 Thread Tony Liu
Hi, Neutron OVN ML2 driver was disconnected by ovn-nb-db. There are many error messages from ovn-nb-db leader. 2020-08-04T02:31:39.751Z|03138|reconnect|ERR|tcp:10.6.20.81:58620: no response to inactivity probe after 5 seconds, disconnecting 2020-08-04T02:31:42.484Z|03139|reconnect|ERR|tc

Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no configuration update

2020-08-03 Thread Tony Liu
e any event library? Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Saturday, August 1, 2020 9:11 PM > To: Tony Liu > Cc: ovs-discuss ; ovs-dev d...@openvswitch.org> > Subject: Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no > configura

Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no configuration update

2020-08-03 Thread Tony Liu
age- > From: Han Zhou > Sent: Monday, August 3, 2020 8:22 PM > To: Tony Liu > Cc: Han Zhou ; ovs-discuss ; > ovs-dev > Subject: Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no > configuration update > > Sorry that I didn't make it clear enough. Th

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-03 Thread Tony Liu
side doesn't respond the probe, if it's bi-directional? I don't see any activities from logging, other than connect/drop and reconnect... BTW, please let me know if this is not the right place to discuss Neutron OVN ML2 driver. Thanks! Tony > -Original Message- > Fr

Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no configuration update

2020-08-04 Thread Tony Liu
Sent: Tuesday, August 4, 2020 12:38 AM > To: Tony Liu > Cc: Han Zhou ; ovs-dev ; ovs- > discuss > Subject: Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no > configuration update > > > > On Tue, Aug 4, 2020 at 9:02 AM Tony Liu <mailto:tonyliu0...@hotmail.

[ovs-discuss] [OVN] stale data complained by ovn-controller after db restore

2020-08-04 Thread Tony Liu
Hi, Here is how I restore OVN DB. * Stop all ovn-nb-db, ovn-sb-db and ovn-northd services. * Clean up all DB files. * Start all DB services. Fresh ovn-nb-db and ovn-sb-db clusters are up and running. * Set DB election timer to 10s. * Restore DB to ovn-nb-db by ovsdb-client. * Start all ovn-north

Re: [ovs-discuss] [ovs-dev] [OVN] stale data complained by ovn-controller after db restore

2020-08-04 Thread Tony Liu
would restoring to a fresh cluster be better? Is it necessary to stop or restart ovn-northd during DB restore? Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Tuesday, August 4, 2020 11:13 AM > To: Tony Liu > Cc: ovs-discuss ; ovs-dev d...@openvswitch.org> > S

Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no configuration update

2020-08-04 Thread Tony Liu
Inline... Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Tuesday, August 4, 2020 11:01 AM > To: Numan Siddique ; Ben Pfaff ; Leonid > Ryzhyk > Cc: Tony Liu ; Han Zhou ; ovs- > dev ; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] ovn-north

Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no configuration update

2020-08-04 Thread Tony Liu
-Original Message- > From: Han Zhou > Sent: Tuesday, August 4, 2020 12:34 PM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; Ben Pfaff > ; Leonid Ryzhyk ; ovs-dev d...@openvswitch.org>; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-04 Thread Tony Liu
; -Original Message- > From: Numan Siddique > Sent: Tuesday, August 4, 2020 12:36 AM > To: Tony Liu > Cc: ovs-discuss ; ovs-dev d...@openvswitch.org> > Subject: Re: [ovs-discuss] [OVN] no response to inactivity probe > > > > On Tue, Aug 4, 2020 at 9:12

Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no configuration update

2020-08-04 Thread Tony Liu
: Tuesday, August 4, 2020 4:02 PM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; Ben Pfaff > ; Leonid Ryzhyk ; ovs-dev d...@openvswitch.org>; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] ovn-northd takes much CPU when no > configuration update > > Hi Tony, > > I

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-04 Thread Tony Liu
In that case, I can use set-connection to set one row. Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Tuesday, August 4, 2020 4:44 PM > To: Tony Liu > Cc: Numan Siddique ; Han Zhou ; ovs- > discuss ; ovs-dev > Subject: Re: [ovs-discuss] [OVN] no r

Re: [ovs-discuss] OVN scale

2020-08-04 Thread Tony Liu
434d-b95b-017a77969b55' with network name 'physnet1' First of all, this kind of scale should work fine, right? Any advices how to look into it? Thanks! Tony > -----Original Message- > From: dev On Behalf Of Tony Liu > Sent: Monday, July 27, 2020 10:16 A

[ovs-discuss] packet drop

2020-08-05 Thread Tony Liu
Hi, I am running ping from external to VM via OVN gateway. On the compute node, ICMP request packet is consistently coming into interface "ovn-gatewa-1". But there is about 10 out of 25 packet loss on tap interface. It's like the switch pauses 10s after every 15s. Has anyone experiences such issu

Re: [ovs-discuss] packet drop

2020-08-05 Thread Tony Liu
by ovn-controller via ovs-vswitchd? Thanks! Tony > -Original Message- > From: discuss On Behalf Of Tony > Liu > Sent: Wednesday, August 5, 2020 2:48 PM > To: ovs-discuss@openvswitch.org; ovs-...@openvswitch.org > Subject: [ovs-discuss] packet drop > > Hi, >

Re: [ovs-discuss] OVN Scale with RAFT: how to make raft cluster clients to balanced state again

2020-08-05 Thread Tony Liu
Sorry for hijacking this thread, I'd like to get some clarifications. How is the initial balanced state established, say 100 ovn-controllers connecting to 3 ovn-sb-db? The ovn-controller doesn't have to connect to the leader of ovn-sb-db, does it? In case it connects to the follower, the write re

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-05 Thread Tony Liu
ginal Message- > From: Han Zhou > Sent: Tuesday, August 4, 2020 4:44 PM > To: Tony Liu > Cc: Numan Siddique ; Han Zhou ; ovs- > discuss ; ovs-dev > Subject: Re: [ovs-discuss] [OVN] no response to inactivity probe > > > > On Tue, Aug 4, 2020 at 2:50 PM Tony Li

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-05 Thread Tony Liu
to probe ovn-controller? Thanks! Tony > -Original Message- > From: discuss On Behalf Of Tony > Liu > Sent: Wednesday, August 5, 2020 4:29 PM > To: Han Zhou > Cc: ovs-dev ; ovs-discuss disc...@openvswitch.org> > Subject: Re: [ovs-discuss] [OVN] no response to i

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-06 Thread Tony Liu
Inline... (please read with monospaced font:)) Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Wednesday, August 5, 2020 11:48 PM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; ovs-dev > ; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] no respons

Re: [ovs-discuss] [ovs-dev] packet drop

2020-08-06 Thread Tony Liu
Inline... Thanks! Tony > -Original Message- > From: Numan Siddique > Sent: Thursday, August 6, 2020 10:03 AM > To: Tony Liu > Cc: ovs-discuss@openvswitch.org; ovs-...@openvswitch.org > Subject: Re: [ovs-dev] packet drop > > > > On Thu, A

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-06 Thread Tony Liu
Inline... Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Thursday, August 6, 2020 11:37 AM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; ovs-dev > ; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] no response to inactivity probe > > > >

Re: [ovs-discuss] [ovs-dev] packet drop

2020-08-06 Thread Tony Liu
Inline... Thanks! Tony > -Original Message- > From: Numan Siddique > Sent: Thursday, August 6, 2020 11:49 AM > To: Tony Liu > Cc: ovs-...@openvswitch.org; ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] [ovs-dev] packet drop > > On Fri, Aug 7, 2020 at

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-06 Thread Tony Liu
I think I have everything clarified here. I will make necessary tunes and rerun the test. Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Thursday, August 6, 2020 12:42 PM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; ovs-dev > ; ovs-discuss > Subj

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-06 Thread Tony Liu
ght? Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Thursday, August 6, 2020 12:42 PM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; ovs-dev > ; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] no response to inactivity probe > > > > On Th

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-06 Thread Tony Liu
tivity probe after 8.74 seconds, disconnecting That tcp:127.0.0.1:6640 is the connection between ovn-controller and local ovsdb-server. Any settings I missed? Thanks! Tony > -Original Message- > From: dev On Behalf Of Tony Liu > Sent: Thursday, August 6, 2020 7:45

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-06 Thread Tony Liu
ting... 2020-08-07T03:55:40.771Z|04003|reconnect|INFO|tcp:127.0.0.1:6640: connected Thanks! Tony > -Original Message- > From: discuss On Behalf Of Tony > Liu > Sent: Thursday, August 6, 2020 8:23 PM > To: Han Zhou ; Numan Siddique > Cc: ovs-dev ; ovs-d

[ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
Hi, Here are some logging snippets from ovn-controller. 2020-08-07T16:38:04.020Z|29250|timeval|WARN|Unreasonably long 8954ms poll interval (8895ms user, 0ms system) What's that mean? Is it harmless? 2020-08-07T16:38:04.021Z|29251|timeval|WARN|context switches: 0 volunt

Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
=== Could that stuck loop be the cause of high cpu usage? What is it polling in? Why is it stuck, waiting for message from sb-db? Isn't it supposed to release the cpu while waiting? Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Friday, August 7, 2020 10:32 A

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-07 Thread Tony Liu
:15:27.771Z|02743|reconnect|INFO|tcp:10.6.20.84:6644: connected Which configuration is for that probe interval? Thanks! Tony > -Original Message- > From: dev On Behalf Of Tony Liu > Sent: Thursday, August 6, 2020 7:45 PM > To: Han Zhou ; Numan Siddique > Cc

Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
Inline... Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Friday, August 7, 2020 12:29 PM > To: Tony Liu > Cc: ovs-discuss ; ovs-dev d...@openvswitch.org> > Subject: Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no > changes in sb-db

Re: [ovs-discuss] [OVN] no response to inactivity probe

2020-08-07 Thread Tony Liu
Good to know, thanks! Tony > -Original Message- > From: Han Zhou > Sent: Friday, August 7, 2020 12:36 PM > To: Tony Liu > Cc: Han Zhou ; Numan Siddique ; ovs-dev > ; ovs-discuss > Subject: Re: [ovs-discuss] [OVN] no response to inactivity probe > > The ra

Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
-controller on gateway node is taking 100% cpu. It's probably related to the ports on the external network? Any specific messages I need to check? Any hint to look into it is appreciated! Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Friday, August 7, 2020 12:39 PM

Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
total: 1196 cmap_expand0.0/sec 0.000/sec0./sec total: 3 82 events never hit Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Friday, August 7, 2020 1:09 PM > To: Tony Liu > Cc: ovs-discuss ; ovs-dev d...@

Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
48a0-a3d1-2973c0b823cc":{"modify":{"external_ids":["map",[["neutron:liveness_check_at","2020-08-07T19:44:07.130233+00:00"]]]] Nailed it... https://bugs.launchpad.net/neutron/+bug/1883554 Tony > -Original Message- >

Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
Is the changes made by Neutron liveness-check in SB properly handled by inc_proc_eng? Given my testing scale, even the frequency is lowered by that fix, ovn-controller still takes quite lot of CPU and time to compute SB. Thanks! Tony > -Original Message- > From: Tony Liu

[ovs-discuss] [OVN] log file rotation and time zone

2020-08-07 Thread Tony Liu
Hi, Is log file rotation and compression supported for all OVN services? How is the time stamp time zone determined by logging? Is it /etc/localtime, hardcoded UTC, or anything else? Thanks! Tony ___ discuss mailing list disc...@openvswitch.org http

Re: [ovs-discuss] [OVN] ovn-controller takes 100% cpu while no changes in sb-db

2020-08-07 Thread Tony Liu
Good to know. Yes, I am using 20.03. Will try to upgrade. Thanks! Tony > -Original Message- > From: Han Zhou > Sent: Friday, August 7, 2020 3:39 PM > To: Tony Liu > Cc: ovs-dev ; ovs-discuss disc...@openvswitch.org> > Subject: Re: [ovs-discuss] [OVN] ovn-cont

[ovs-discuss] [OVN] Kolla Ansible deployment

2020-08-07 Thread Tony Liu
> Sent: Thursday, July 23, 2020 3:27 AM > To: Tony Liu > Cc: Daniel Alvarez ; ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] OVN scale > > Hi Tony, > > I’m the core reviewer/developer behind OVN implementation in Kolla- > Ansible. > Just to make some cl

Re: [ovs-discuss] ovn-k8s scale: how to make new ovn-controller process keep the previous Open Flow in br-int

2020-08-08 Thread Tony Liu
Hi, I wonder if we can have a solution combining all your excellent ideas? * Don't clean up flows until recompute is done. * Use nb_cfg to determine if update is required. * Compute and apply the diff only. * Save ovn-controller version in ovsdb. Two scenarios to cover, restart and upgrade. If w

[ovs-discuss] [OVN] not-equal in ACL

2020-08-10 Thread Tony Liu
Hi Numan, Create a new thread here to follow up ACL questions. > > > I think this is a big problem here. We should not use "!=" in > > > logical flows, although OVN allows. > > > > Is this a generic recommendation or for certain cases? > > Is it OK to add an ACL with "!=", like below? > > > > ovn

[ovs-discuss] [OVN] Dependency to OVS

2020-08-11 Thread Tony Liu
Hi, Is there any version dependency between OVN and OVS, like which OVS has to be used for certain OVN release? Is such info shared anywhere? Thanks! Tony ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/

[ovs-discuss] [OVN] ToR as the gateway

2020-08-11 Thread Tony Liu
Hi, In http://www.openvswitch.org/support/dist-docs/ovn-architecture.7.html, I see this. For connecting to gateways, in addition to Geneve and STT, OVN supports VXLAN, because only VXLAN support is common on top-of-rack (ToR) switches. Does anyone have experiences in using ph

Re: [ovs-discuss] [OVN] not-equal in ACL

2020-08-12 Thread Tony Liu
> -Original Message- > From: Numan Siddique > Sent: Wednesday, August 12, 2020 2:17 AM > To: Tony Liu > Cc: ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] [OVN] not-equal in ACL > > On Mon, Aug 10, 2020 at 11:11 PM Tony Liu > wrote: > > &

Re: [ovs-discuss] [OVN] not-equal in ACL

2020-08-12 Thread Tony Liu
> -Original Message- > From: Numan Siddique > Sent: Wednesday, August 12, 2020 10:25 AM > To: Tony Liu > Cc: ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] [OVN] not-equal in ACL > > On Wed, Aug 12, 2020 at 10:41 PM Tony Liu > wrote: >

Re: [ovs-discuss] [OVN] not-equal in ACL

2020-08-13 Thread Tony Liu
> -Original Message- > From: discuss On Behalf Of Tony > Liu > Sent: Monday, August 10, 2020 10:41 AM > To: Numan Siddique > Cc: ovs-discuss@openvswitch.org > Subject: [ovs-discuss] [OVN] not-equal in ACL > > Hi Numan, > > Create a new threa

[ovs-discuss] ovsdb-server unix socket permission

2020-08-21 Thread Tony Liu
Hi, The ovsdb-server UNIX socket permission is 0750. It works fine for OVS services, like ovs-vswitchd and ovn-controller who run as root. When integrate with OpenStack, neutron-ovn-metadata-agent running as user "neutron" needs to connect to ovsdb-server. TCP connection works fine. But, since it

Re: [ovs-discuss] ovsdb-server unix socket permission

2020-08-22 Thread Tony Liu
> -Original Message- > From: Matthew Booth > Sent: Saturday, August 22, 2020 3:12 PM > To: Tony Liu > Cc: ovs-discuss@openvswitch.org; ovs-dev > Subject: Re: [ovs-discuss] ovsdb-server unix socket permission > > On Fri, 21 Aug 2020 at 20:40, Tony Liu wrote:

Re: [ovs-discuss] How to restart raft cluster after a complete shutdown?

2020-08-25 Thread Tony Liu
Start the first node to create the cluster. https://github.com/ovn-org/ovn/blob/master/utilities/ovn-ctl#L228 https://github.com/openvswitch/ovs/blob/master/utilities/ovs-lib.in#L478 Start the rest nodes to join the cluster. https://github.com/ovn-org/ovn/blob/master/utilities/ovn-ctl#L226 https:/

[ovs-discuss] [OVN] Gateway router scale

2020-08-30 Thread Tony Liu
Hi, Could anyone share experiences in gateway routers scaling? How many gateway routers on a chassis have be tested, hundreds, thousands? What may be the bottleneck for gateway router scaling, resources like memory, CPU, or ovn-controller, OVSDB? BTW, the link in ovn-architecture.7 is not valid a

[ovs-discuss] [OVN] failed to connect 5K networks to 5K routers

2020-09-09 Thread Tony Liu
Hi, Here is what I did. #1 Create 5K networks, 5K routers and 1 provider/physical network. All resources are created in NB and translated to SB properly. #2 Connect 5K routers to provider network. All resources are created and updated in NB properly, but SB is not fully updated. I did 3 snapsho

[ovs-discuss] [ovn] MAC in table acl and lb

2020-09-22 Thread Tony Liu
Hi, When I look at a datapath ingress pipeline, = table=3 (ls_in_pre_acl ), priority=110 , match=(eth.dst == 1a:d2:77:6e:42:98), action=(next;) table=3 (ls_in_pre_acl ), priority=0, match=(1), action=(next;) table=4 (ls_in_pre_lb ), priority=110

Re: [ovs-discuss] [ovn] MAC in table acl and lb

2020-09-22 Thread Tony Liu
That MAC is also in table 19. == table=19(ls_in_l2_lkup ), priority=110 , match=(eth.dst == 1a:d2:77:6e:42:98), action=(handle_svc_check(inport);) == Thanks! Tony > -Original Message- > From: Tony Liu > Sent: Tuesday, Sep

[ovs-discuss] [ovn] distributed router port and distributed SNAT

2020-09-23 Thread Tony Liu
Hi, I read through this long discussion [1]. Here is what I am doing. +--+ |external logical switch | +-+-++-+ | || +--+--+ +--+--+

Re: [ovs-discuss] [ovn] distributed router port and distributed SNAT

2020-09-24 Thread Tony Liu
> -Original Message- > From: Numan Siddique > Sent: Wednesday, September 23, 2020 11:12 PM > To: Tony Liu > Cc: ovs-discuss > Subject: Re: [ovs-discuss] [ovn] distributed router port and distributed > SNAT > > > > On Thu, Sep 24, 2020 at 10

[ovs-discuss] [ovn] gateway router vs. distributed gateway port

2020-09-24 Thread Tony Liu
Hi, Given "Distributed Gateway Ports" in [1], and my test, central SNAT, central FIP and distributed FIP are all supported by distributed gateway port. I wonder in which case gateway router has to be used? One case is mentioned in [2], DR is used in K8S to support node service and avoid central

[ovs-discuss] [ovn] no tunnel from GW to compute

2020-10-16 Thread Tony Liu
Hi, I am seeing an interesting issue today. When ping a FIP from external, request arrives on GW, but no tunnel from GW to compute. When ping from VM to external, egress works fine, request goes through tunnel from compute to GW, then to external. Reply arrives at GW, no tunnel from GW back to com

[ovs-discuss] [ovn] does not match prerequisite

2020-10-20 Thread Tony Liu
Hi, >From ovnsb log, I see many of the following messages. What does it mean? Is that a concern? 2020-10-20T18:52:50.483Z|00093|raft|INFO|current entry eid 2ab3eff8-87e1-4e19-9a1f-d359ad56a9ad does not match prerequisite c6ffd854-6f6e-4533-a6d8-b297acb542e0 in execute_command_request Thanks!

Re: [ovs-discuss] [ovn] no tunnel from GW to compute

2020-10-27 Thread Tony Liu
Saw the same problem again. Recreate network, attach to router and launch VM, problem is gone. Probably some glitch happened during the early deployment. Any hints how to look into it? Thanks! Tony > -Original Message- > From: discuss On Behalf Of Tony > Liu > Sent: Friday

Re: [ovs-discuss] [ovn] no tunnel from GW to compute

2020-10-27 Thread Tony Liu
w_src=172.16.160.1,nw_dst=10.10.0.10,nw_tos=0,nw_ecn=0,nw_ttl=124,icmp_type=8,icmp_code=0 icmp_csum:6011 == Thanks! Tony > -Original Message- > From: dev On Behalf Of Tony Liu > Sent: Tuesday, October 27, 2020 2:23 PM > To: ovs-dis

Re: [ovs-discuss] [ovs-dev] [ovn] no tunnel from GW to compute

2020-10-28 Thread Tony Liu
nal Message- > From: Numan Siddique > Sent: Wednesday, October 28, 2020 1:56 AM > To: Tony Liu > Cc: ovs-discuss ; ovs-dev d...@openvswitch.org> > Subject: Re: [ovs-dev] [ovn] no tunnel from GW to compute > > On Wed, Oct 28, 2020 at 11:29 AM Tony Liu > wrote: &g

  1   2   >