Hi，Ilya Maximetsthanks for your replay
I am running ovn large scale test，with 1000 sandbox (that is 1000
ovn-controller)，3 clustered nb ，3 nb-relay， 3 clustered sb，20 sb-relay
configure flows：neutron-server <----> nb-relay <----> nb <----> northd <---->
sb <----> sb-relay <----> ovn-controller
default 5 seconds probe interval will cause connection flapping： large
transaction handing，db log compression，...
ovsdb relay server has two kinds of connections：active connection and passive
connection， active connection ,as ovsdb client，connect to clustered ovsdb
server，and passive connection listening other client connect to itself
I config this two kinds of connections in nb: active connection:
"tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641" passive connection:
“ptcp:6641:0.0.0.0”
it cannot relay server share the same connection configurations with clustered
ovsdb-server？
it is not a good way to have another small database with a relay configuration.
an example： ovn-northd has no database，probe interval read from NB，config
northd probe interval like this：ovn-nbctl set NB_Global .
options:northd_probe_interval=60000,can relay sever read probe interval from
NB or SB? if probe interval of relay server cannot read from NB or SB,
appctl command can be as consider，because it can reconfig without restart
Best regards, Wentao Jia
the follow msg is configuration of my test：
clustered ovsdb server
ovsdb-server -vconsole:info -vsyslog:off -vfile:off
--log-file=/var/log/ovn/ovsdb-server-nb.log
--remote=punix:/var/run/ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid
--unixctl=/var/run/ovn/ovnnb_db.ctl
--remote=db:OVN_Northbound,NB_Global,connections
--private-key=db:OVN_Northbound,SSL,private_key
--certificate=db:OVN_Northbound,SSL,certificate
--ca-cert=db:OVN_Northbound,SSL,ca_cert
--ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols
--ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers /etc/ovn/ovnnb_db.db
ovsdb relay server：
ovsdb-server --remote=db:OVN_Northbound,NB_Global,connections -vconsole:info
-vsyslog:off -vfile:off --log-file=/var/log/ovn/ovsdb-server-nb.log
relay:OVN_Northbound:tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641
connecion configuration：one active connection and one passive connection
()[root@ovn-busybox-0 /]# ovn-nbctl list connection
_uuid : 5ddab5a4-a267-42b4-9dd4-76d55855a109
external_ids : {}
inactivity_probe : 120000
is_connected : true
max_backoff : []
other_config : {}
status : {sec_since_connect="143208", state=ACTIVE}
target : "tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641"
_uuid : 351b99bb-dd6a-4ba3-9c30-c0b4cff183e7
external_ids : {}
inactivity_probe : 0
is_connected : true
max_backoff : []
other_config : {}
status : {bound_port="6641", sec_since_connect="0",
sec_since_disconnect="0"}
target : "ptcp:6641:0.0.0.0"
发件人：Ilya Maximets <[email protected]>
发送日期：2021-08-26 02:38:58
收件人：[email protected],"贾文涛" <[email protected]>
抄送人：[email protected]
主题：Re: [ovs-discuss] ovsdb relay server active connection probe interval do not
work>> hi，all
>>
>>
>> the default inactivity probe interval of ovsdb relay server to nb/sb ovsdb
>> server is 5000ms.
>> I set an active connection as follow，set inactivity probe interval to
>> 120000ms ：
>> _uuid : 5ddab5a4-a267-42b4-9dd4-76d55855a109
>> external_ids : {}
>> inactivity_probe : 120000
>> is_connected : true
>> max_backoff : []
>> other_config : {}
>> status : {sec_since_connect="0", state=ACTIVE}
>> target : "tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641"
>
>
>Hmm. How exactly did you configure that?
>
>>
>> ovn-ovsdb-nb.openstack.svc.cluster.local is a vip
>> but the inactivity probe is still 5000>
>> 2021-08-24T12:34:17.313Z|04924|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
>> idle 120225 ms, sending inactivity probe
>> 2021-08-24T12:36:17.759Z|05854|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
>> idle 120446 ms, sending inactivity probe
>> 2021-08-24T12:37:06.326Z|06145|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
>> idle 6853 ms, sending inactivity probe
>> 2021-08-24T12:37:11.330Z|06155|reconnect|DBG|tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641:
>> idle 5004 ms, sending inactivity probe
>
>This looks like you have 2 different connections. One with 5000 and
>one with 120000 inactivity probe interval.
>
>I suspect that relay server is started something like this:
>
>ovsdb-server ... --remote=db:OVN_Northbound,NB_Global,connections \
> relay:OVN_Northbound:tcp:ovn-ovsdb-nb.openstack.svc.cluster.local:6641
>
>And the connection showed above is configured in this 'connections' row, right?
>
>Connections configured with '--remote' are not the same as 'relay' connections.
>So, in this case ovsdb-server will create a relay with the remote specified
>in a 'relay:' part and with a default inactivity probe interval. And it will
>open a connection to what is specified in a database row pointed by '--remote'
>with a configured values for that connection. It will expect a client on the
>other side of the connection. So, this connection will connect main server
>with relay, but they both will just wait database queries from each other.
>
>Configuring things this way you will also, probably, have a self-connection
>from the main server to itself, right?
>
>
>In general, currently, there is no way to configure inactivity probe interval
>for "relay" --> "main server" connection, you can only configure it in the
>opposite direction.
>Does default inactivity interval cause problems for your setup?
>
>I have a plan to implement that though. There are several options how to do
>that:
>
>1. Add a simple cmdline argument like '--relay-inactivity-probe=N' that will
> affect all the relay databases on this ovsdb-server process.
>
> Pros: Simple
> Cons: Affects all relay databases of this process, change requires restart,
> configuration applied to a single process.
>
>2. appctl command that can be executed against relay server, e.g.
> ovs-appctl ovsdb-server/relay-set-inactivity-probe OVN_Northbound 120000
>
> Pros: Simple
> Cons: Doesn't survive restart, configuration applied to a single process.
>
>3. Add more configuration options to the 'relay:' syntax, e.g.:
> relay:inactivity-probe=120000:OVN_Northbound:tcp:127.0.0.1:6641
>
> Pros: Simple
> Cons: Doesn't look like a good API.
>
>4. Have a separate small database with a relay configuration, e.g.
>
> ovsdb-server ... relay:db:OVSDB_Relay,Relay,relays relay.db
>
> And a small tool to interact with this local database:
>
> ovs-relayctl add-relay OVN_Northbound \
> tcp:127.0.0.1:6641 inactivity-probe=120000
>
> This will add a new relay configuration to the OVSDB_Relay database
> and ovsdb-server will start relaying it.
>
> Pros: Lots of things can be configured including inactivity probes and
> backoff. Can be extended with relay specific configs in the future.
> Survives restart. relay.db can be relayed from a separate
> ovsdb-server,
> if needed, so there is no need to configure each relay separately.
> Cons: A bit more complex implementation.
>
> Example of a complex setup would be:
> # start a main database server
> a. ovsdb-server --remote=db:OVN_Nortbound:NB_Global,connections ovnnb.db
>
> # start a small database server that only holds relay.db
> b. ovsdb-server --remote=pssl:6647:server relay.db
> ovs-relayctl add-relay OVN_Northbound tcp:your-server:6641
> inactivity-probe=120000
>
> # start a relay server that relays OVSDB_Relay and relays everything
> # that configured in this db. If OVSDB_Relay db has configured
> # OVN_Northbound db, start accepting connections on remotes configured
> there.
> c. ovsdb-server --remote=db:OVN_Nortbound:NB_Global,connections \
> relay:db:OVSDB_Relay,Relay,relays \
> relay:OVSDB_Relay:ssl:server:6647
>
> Once this started, server 'c' will connect to server 'b' and get the
> OVSDB_Relay
> database, then it will look at the content of OVSDB_Relay database and
> connect to
> the server 'a' to start relaying the OVN_Northbound database.
>
> Additional cons: Configuration looks insanely complex at a first glance :)
> even
> if it's really not.
>
> Simpler setup without relaying the OVSDB_Relay database:
>
> a. ovsdb-server --remote=db:OVN_Nortbound:NB_Global,connections ovnnb.db
>
> b. ovsdb-server --remote=db:OVN_Nortbound:NB_Global,connections \
> relay:db:OVSDB_Relay,Relay,relays relay.db
> ovs-relayctl add-relay OVN_Northbound tcp:your-server:6641
> inactivity-probe=120000
>
> Here relay.db is local for a particular relay, but this means that
> configuration
> (ovs-relayctl) command should be issued separately for each relay server.
>
>Hope this makes some sense. :)
>What do you think? Which option looks better in your opinion?
>
>Best regards, Ilya Maximets.
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
