I don't see any ovn-northd.log log, I only see those when I'm running
OVN outside the k8s cluster.
Before I start the Satefulset on my k8 nodes I run:
ovn-ctl stop_northd
ovn-ctl stop_ovsdb
rm -rf /usr/etc/ovn/*.db
The only logs I see are ovn-controller.log (I'm running ovn controller
on the K8 nodes) ovsdb-server-nb.log ovsdb-server-sb.log
These logs look normal
k8 node 1 logs
============
2020-08-05T13:44:51.958Z|00001|vlog|INFO|opened log file
/var/log/ovn/ovsdb-server-nb.log
2020-08-05T13:44:51.961Z|00002|ovsdb_server|INFO|ovsdb-server (Open
vSwitch) 2.13.90
2020-08-05T13:44:51.961Z|00003|reconnect|INFO|tcp:[253.255.0.34]:6643:
connecting...
2020-08-05T13:44:51.961Z|00004|reconnect|INFO|tcp:[253.255.0.34]:6643:
connected
2020-08-05T13:44:51.962Z|00005|raft_rpc|INFO|learned cluster ID 6e05
2020-08-05T13:44:51.962Z|00006|raft|INFO|tcp:[253.255.0.34]:6643:
learned server ID c8a5
2020-08-05T13:44:51.962Z|00007|raft|INFO|server c8a5 is leader for term 1
2020-08-05T13:44:51.962Z|00008|raft|INFO|rejecting append_request
because previous entry 1,5 not in local log (mismatch past end of log)
2020-08-05T13:44:51.964Z|00009|raft|INFO|server c8a5 added to configuration
2020-08-05T13:44:51.966Z|00010|raft|INFO|server 0d3d added to configuration
2020-08-05T13:44:51.966Z|00011|raft|INFO|tcp:253.255.0.34:50744: learned
server ID c8a5
2020-08-05T13:44:51.966Z|00012|raft|INFO|tcp:253.255.0.34:50744: learned
remote address tcp:[253.255.0.34]:6643
2020-08-05T13:44:52.015Z|00013|raft|INFO|tcp:253.255.0.35:51780: learned
server ID 941e
2020-08-05T13:44:52.015Z|00014|raft|INFO|tcp:253.255.0.35:51780: learned
remote address tcp:[253.255.0.35]:6643
2020-08-05T13:44:52.015Z|00015|raft|INFO|adding 941e (941e at
tcp:[253.255.0.35]:6643) to cluster 6e05 failed (not leader)
2020-08-05T13:44:52.015Z|00016|raft|INFO|server 941e added to configuration
2020-08-05T13:44:52.016Z|00017|reconnect|INFO|tcp:[253.255.0.35]:6643:
connecting...
2020-08-05T13:44:52.016Z|00018|reconnect|INFO|tcp:[253.255.0.35]:6643:
connected
2020-08-05T13:45:01.962Z|00019|memory|INFO|8424 kB peak resident set
size after 10.0 seconds
2020-08-05T13:45:01.962Z|00020|memory|INFO|cells:40 monitors:0
raft-connections:4
2020-08-05T14:10:03.593Z|00021|reconnect|INFO|tcp:[253.255.0.34]:6643:
connection closed by peer
2020-08-05T14:10:03.595Z|00022|raft|INFO|server 941e is leader for term 2
2020-08-05T14:10:04.594Z|00023|reconnect|INFO|tcp:[253.255.0.34]:6643:
connecting...
2020-08-05T14:10:04.594Z|00024|reconnect|INFO|tcp:[253.255.0.34]:6643:
connection attempt failed (Connection refused)
2020-08-05T14:10:04.594Z|00025|reconnect|INFO|tcp:[253.255.0.34]:6643:
waiting 2 seconds before reconnect
2020-08-05T14:10:06.594Z|00026|reconnect|INFO|tcp:[253.255.0.34]:6643:
connecting...
2020-08-05T14:10:06.594Z|00027|reconnect|INFO|tcp:[253.255.0.34]:6643:
connection attempt failed (Connection refused)
2020-08-05T14:10:06.594Z|00028|reconnect|INFO|tcp:[253.255.0.34]:6643:
waiting 4 seconds before reconnect
2020-08-05T14:10:08.453Z|00029|raft|INFO|received leadership transfer
from 941e in term 2
2020-08-05T14:10:08.453Z|00030|raft|INFO|term 3: starting election
2020-08-05T14:10:08.453Z|00031|reconnect|INFO|tcp:[253.255.0.35]:6643:
connection closed by peer
2020-08-05T13:44:48.971Z|00001|vlog|INFO|opened log file
/var/log/ovn/ovsdb-server-sb.log
2020-08-05T13:44:48.973Z|00002|ovsdb_server|INFO|ovsdb-server (Open
vSwitch) 2.13.90
2020-08-05T13:44:48.973Z|00003|reconnect|INFO|tcp:[253.255.0.34]:6644:
connecting...
2020-08-05T13:44:48.973Z|00004|reconnect|INFO|tcp:[253.255.0.34]:6644:
connected
2020-08-05T13:44:48.973Z|00005|raft_rpc|INFO|learned cluster ID 281c
2020-08-05T13:44:48.973Z|00006|raft|INFO|tcp:[253.255.0.34]:6644:
learned server ID eeed
2020-08-05T13:44:48.974Z|00007|raft|INFO|server eeed is leader for term 1
2020-08-05T13:44:48.974Z|00008|raft|INFO|rejecting append_request
because previous entry 1,4 not in local log (mismatch past end of log)
2020-08-05T13:44:48.976Z|00009|raft|INFO|server eeed added to configuration
2020-08-05T13:44:48.977Z|00010|raft|INFO|server 5098 added to configuration
2020-08-05T13:44:48.977Z|00011|raft|INFO|tcp:253.255.0.34:50628: learned
server ID eeed
2020-08-05T13:44:48.977Z|00012|raft|INFO|tcp:253.255.0.34:50628: learned
remote address tcp:[253.255.0.34]:6644
2020-08-05T13:44:49.044Z|00013|raft|INFO|tcp:253.255.0.35:49594: learned
server ID b9de
2020-08-05T13:44:49.044Z|00014|raft|INFO|tcp:253.255.0.35:49594: learned
remote address tcp:[253.255.0.35]:6644
2020-08-05T13:44:49.044Z|00015|raft|INFO|adding b9de (b9de at
tcp:[253.255.0.35]:6644) to cluster 281c failed (not leader)
2020-08-05T13:44:49.044Z|00016|raft|INFO|server b9de added to configuration
2020-08-05T13:44:49.044Z|00017|reconnect|INFO|tcp:[253.255.0.35]:6644:
connecting...
2020-08-05T13:44:49.045Z|00018|reconnect|INFO|tcp:[253.255.0.35]:6644:
connected
2020-08-05T13:44:58.975Z|00019|memory|INFO|8408 kB peak resident set
size after 10.0 seconds
2020-08-05T13:44:58.975Z|00020|memory|INFO|cells:39 monitors:0
raft-connections:4
2020-08-05T14:10:03.593Z|00021|reconnect|INFO|tcp:[253.255.0.34]:6644:
connection closed by peer
2020-08-05T14:10:03.595Z|00022|raft|INFO|server b9de is leader for term 2
2020-08-05T14:10:04.595Z|00023|reconnect|INFO|tcp:[253.255.0.34]:6644:
connecting...
2020-08-05T14:10:04.595Z|00024|reconnect|INFO|tcp:[253.255.0.34]:6644:
connection attempt failed (Connection refused)
2020-08-05T14:10:04.595Z|00025|reconnect|INFO|tcp:[253.255.0.34]:6644:
waiting 2 seconds before reconnect
2020-08-05T13:47:27.231Z|00001|vlog|INFO|opened log file
/usr/var/log/ovn/ovn-controller.log
2020-08-05T13:47:27.233Z|00002|reconnect|INFO|unix:/usr/var/run/openvswitch/db.sock:
connecting...
2020-08-05T13:47:27.233Z|00003|reconnect|INFO|unix:/usr/var/run/openvswitch/db.sock:
connected
2020-08-05T13:47:27.235Z|00004|main|INFO|OVS IDL reconnected, force
recompute.
2020-08-05T13:47:27.236Z|00005|reconnect|INFO|tcp:253.255.0.35:6642:
connecting...
2020-08-05T13:47:27.236Z|00006|main|INFO|OVNSB IDL reconnected, force
recompute.
2020-08-05T13:47:27.236Z|00007|reconnect|INFO|tcp:253.255.0.35:6642:
connected
2020-08-05T13:47:27.237Z|00008|ofctrl|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting to switch
2020-08-05T13:47:27.237Z|00009|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:47:27.237Z|00010|rconn|WARN|unix:/usr/var/run/openvswitch/br-int.mgmt:
connection failed (No such file or directory)
2020-08-05T13:47:27.237Z|00011|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
waiting 1 seconds before reconnect
2020-08-05T13:47:28.239Z|00012|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:47:28.239Z|00013|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connected
2020-08-05T13:47:28.240Z|00001|pinctrl(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting to switch
2020-08-05T13:47:28.240Z|00002|rconn(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:47:28.240Z|00003|rconn(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connected
2020-08-05T13:47:28.436Z|00014|dpif_netlink|INFO|The kernel module does
not support meters.
2020-08-05T14:10:05.486Z|00015|main|INFO|OVNSB commit failed, force
recompute next time.
2020-08-05T14:10:05.486Z|00016|jsonrpc|WARN|tcp:253.255.0.35:6642:
receive error: Connection reset by peer
2020-08-05T14:10:05.486Z|00017|reconnect|WARN|tcp:253.255.0.35:6642:
connection dropped (Connection reset by peer)
2020-08-05T14:10:05.486Z|00018|reconnect|INFO|tcp:253.255.0.33:6642:
connecting...
2020-08-05T14:10:05.486Z|00019|reconnect|INFO|tcp:253.255.0.33:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:05.487Z|00020|reconnect|INFO|tcp:253.255.0.34:6642:
connecting...
2020-08-05T14:10:05.487Z|00021|reconnect|INFO|tcp:253.255.0.34:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:06.488Z|00022|reconnect|INFO|tcp:253.255.0.35:6642:
connecting...
2020-08-05T14:10:06.488Z|00023|reconnect|INFO|tcp:253.255.0.35:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:06.488Z|00024|reconnect|INFO|tcp:253.255.0.35:6642:
waiting 2 seconds before reconnect
2020-08-05T14:10:08.490Z|00025|reconnect|INFO|tcp:253.255.0.33:6642:
connecting...
2020-08-05T14:10:08.490Z|00026|reconnect|INFO|tcp:253.255.0.33:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:08.490Z|00027|reconnect|INFO|tcp:253.255.0.33:6642:
waiting 4 seconds before reconnect
2020-08-05T14:10:12.492Z|00028|reconnect|INFO|tcp:253.255.0.34:6642:
connecting...
2020-08-05T14:10:12.492Z|00029|reconnect|INFO|tcp:253.255.0.34:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:12.492Z|00030|reconnect|INFO|tcp:253.255.0.34:6642:
continuing to reconnect in the background but suppressing further logging
K8 node 2
---------------
2020-08-05T13:43:11.094Z|00001|vlog|INFO|opened log file
/var/log/ovn/ovsdb-server-nb.log
2020-08-05T13:43:11.095Z|00002|raft|INFO|term 1: 40130328487 ms timeout
expired, starting election
2020-08-05T13:43:11.095Z|00003|raft|INFO|term 1: elected leader by 1+ of
1 servers
2020-08-05T13:43:11.098Z|00004|ovsdb_server|INFO|ovsdb-server (Open
vSwitch) 2.13.90
2020-08-05T13:43:12.074Z|00005|jsonrpc|WARN|unix#4: send error: Broken pipe
2020-08-05T13:43:12.074Z|00006|reconnect|WARN|unix#4: connection dropped
(Broken pipe)
2020-08-05T13:43:15.942Z|00007|jsonrpc|WARN|tcp:253.255.0.35:60448:
receive error: Connection reset by peer
2020-08-05T13:43:15.942Z|00008|reconnect|WARN|tcp:253.255.0.35:60448:
connection dropped (Connection reset by peer)
2020-08-05T13:43:15.947Z|00009|raft|INFO|tcp:253.255.0.33:46572: learned
server ID 0d3d
2020-08-05T13:43:15.947Z|00010|raft|INFO|tcp:253.255.0.33:46572: learned
remote address tcp:[253.255.0.33]:6643
2020-08-05T13:43:15.947Z|00011|raft|INFO|starting to add server 0d3d
(0d3d at tcp:[253.255.0.33]:6643) to cluster 6e05
2020-08-05T13:43:15.950Z|00012|raft|INFO|cluster 6e05: installed
snapshot on server 0d3d up to 1:1
2020-08-05T13:43:15.951Z|00013|reconnect|INFO|tcp:[253.255.0.33]:6643:
connecting...
2020-08-05T13:43:15.952Z|00014|reconnect|INFO|tcp:[253.255.0.33]:6643:
connected
2020-08-05T13:43:15.952Z|00015|raft|INFO|adding 0d3d (0d3d at
tcp:[253.255.0.33]:6643) to cluster 6e05 succeeded (completed)
2020-08-05T13:43:15.997Z|00016|raft|INFO|tcp:253.255.0.35:60500: learned
server ID 941e
2020-08-05T13:43:15.997Z|00017|raft|INFO|tcp:253.255.0.35:60500: learned
remote address tcp:[253.255.0.35]:6643
2020-08-05T13:43:15.997Z|00018|raft|INFO|starting to add server 941e
(941e at tcp:[253.255.0.35]:6643) to cluster 6e05
2020-08-05T13:43:16.000Z|00019|raft|INFO|cluster 6e05: installed
snapshot on server 941e up to 1:1
2020-08-05T13:43:16.001Z|00020|reconnect|INFO|tcp:[253.255.0.35]:6643:
connecting...
2020-08-05T13:43:16.001Z|00021|reconnect|INFO|tcp:[253.255.0.35]:6643:
connected
2020-08-05T13:43:16.002Z|00022|raft|INFO|adding 941e (941e at
tcp:[253.255.0.35]:6643) to cluster 6e05 succeeded (completed)
2020-08-05T13:43:21.101Z|00023|memory|INFO|8452 kB peak resident set
size after 10.0 seconds
2020-08-05T13:43:21.101Z|00024|memory|INFO|cells:40 monitors:0
raft-connections:4
2020-08-05T13:43:11.105Z|00001|vlog|INFO|opened log file
/var/log/ovn/ovsdb-server-sb.log
2020-08-05T13:43:11.106Z|00002|raft|INFO|term 1: 40130328497 ms timeout
expired, starting election
2020-08-05T13:43:11.106Z|00003|raft|INFO|term 1: elected leader by 1+ of
1 servers
2020-08-05T13:43:11.108Z|00004|ovsdb_server|INFO|ovsdb-server (Open
vSwitch) 2.13.90
2020-08-05T13:43:12.959Z|00005|raft|INFO|tcp:253.255.0.33:46140: learned
server ID 5098
2020-08-05T13:43:12.959Z|00006|raft|INFO|tcp:253.255.0.33:46140: learned
remote address tcp:[253.255.0.33]:6644
2020-08-05T13:43:12.959Z|00007|raft|INFO|starting to add server 5098
(5098 at tcp:[253.255.0.33]:6644) to cluster 281c
2020-08-05T13:43:12.962Z|00008|raft|INFO|cluster 281c: installed
snapshot on server 5098 up to 1:1
2020-08-05T13:43:12.963Z|00009|reconnect|INFO|tcp:[253.255.0.33]:6644:
connecting...
2020-08-05T13:43:12.963Z|00010|reconnect|INFO|tcp:[253.255.0.33]:6644:
connected
2020-08-05T13:43:12.963Z|00011|raft|INFO|adding 5098 (5098 at
tcp:[253.255.0.33]:6644) to cluster 281c succeeded (completed)
2020-08-05T13:43:12.972Z|00012|jsonrpc|WARN|tcp:253.255.0.35:40706:
receive error: Connection reset by peer
2020-08-05T13:43:12.972Z|00013|reconnect|WARN|tcp:253.255.0.35:40706:
connection dropped (Connection reset by peer)
2020-08-05T13:43:13.027Z|00014|raft|INFO|tcp:253.255.0.35:44172: learned
server ID b9de
2020-08-05T13:43:13.027Z|00015|raft|INFO|tcp:253.255.0.35:44172: learned
remote address tcp:[253.255.0.35]:6644
2020-08-05T13:43:13.027Z|00016|raft|INFO|starting to add server b9de
(b9de at tcp:[253.255.0.35]:6644) to cluster 281c
2020-08-05T13:43:13.029Z|00017|raft|INFO|cluster 281c: installed
snapshot on server b9de up to 1:1
2020-08-05T13:43:13.030Z|00018|reconnect|INFO|tcp:[253.255.0.35]:6644:
connecting...
2020-08-05T13:43:13.030Z|00019|reconnect|INFO|tcp:[253.255.0.35]:6644:
connected
2020-08-05T13:43:13.031Z|00020|raft|INFO|adding b9de (b9de at
tcp:[253.255.0.35]:6644) to cluster 281c succeeded (completed)
2020-08-05T13:43:21.109Z|00021|memory|INFO|8604 kB peak resident set
size after 10.0 seconds
2020-08-05T13:43:21.109Z|00022|memory|INFO|cells:39 monitors:0
raft-connections:4
2020-08-05T13:45:52.338Z|00001|vlog|INFO|opened log file
/usr/var/log/ovn/ovn-controller.log
2020-08-05T13:45:52.340Z|00002|reconnect|INFO|unix:/usr/var/run/openvswitch/db.sock:
connecting...
2020-08-05T13:45:52.340Z|00003|reconnect|INFO|unix:/usr/var/run/openvswitch/db.sock:
connected
2020-08-05T13:45:52.342Z|00004|main|INFO|OVS IDL reconnected, force
recompute.
2020-08-05T13:45:52.342Z|00005|reconnect|INFO|tcp:253.255.0.35:6642:
connecting...
2020-08-05T13:45:52.342Z|00006|main|INFO|OVNSB IDL reconnected, force
recompute.
2020-08-05T13:45:52.342Z|00007|reconnect|INFO|tcp:253.255.0.35:6642:
connected
2020-08-05T13:45:52.344Z|00008|ofctrl|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting to switch
2020-08-05T13:45:52.344Z|00009|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:45:52.344Z|00010|rconn|WARN|unix:/usr/var/run/openvswitch/br-int.mgmt:
connection failed (No such file or directory)
2020-08-05T13:45:52.344Z|00011|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
waiting 1 seconds before reconnect
2020-08-05T13:45:53.344Z|00012|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:45:53.345Z|00013|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connected
2020-08-05T13:45:53.346Z|00014|dpif_netlink|INFO|The kernel module does
not support meters.
2020-08-05T13:45:53.348Z|00001|pinctrl(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting to switch
2020-08-05T13:45:53.348Z|00002|rconn(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:45:53.348Z|00003|rconn(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connected
2020-08-05T14:08:29.432Z|00015|main|INFO|OVNSB commit failed, force
recompute next time.
2020-08-05T14:08:29.432Z|00016|jsonrpc|WARN|tcp:253.255.0.35:6642:
receive error: Connection reset by peer
2020-08-05T14:08:29.432Z|00017|reconnect|WARN|tcp:253.255.0.35:6642:
connection dropped (Connection reset by peer)
2020-08-05T14:08:29.432Z|00018|reconnect|INFO|tcp:253.255.0.33:6642:
connecting...
2020-08-05T14:08:29.432Z|00019|reconnect|INFO|tcp:253.255.0.33:6642:
connection attempt failed (Connection refused)
2020-08-05T14:08:29.433Z|00020|reconnect|INFO|tcp:253.255.0.34:6642:
connecting...
2020-08-05T14:08:29.433Z|00021|reconnect|INFO|tcp:253.255.0.34:6642:
connection attempt failed (Connection refused)
2020-08-05T14:08:30.434Z|00022|reconnect|INFO|tcp:253.255.0.35:6642:
connecting...
2020-08-05T14:08:30.434Z|00023|reconnect|INFO|tcp:253.255.0.35:6642:
connection attempt failed (Connection refused)
2020-08-05T14:08:30.434Z|00024|reconnect|INFO|tcp:253.255.0.35:6642:
waiting 2 seconds before reconnect
2020-08-05T14:08:32.436Z|00025|reconnect|INFO|tcp:253.255.0.33:6642:
connecting...
2020-08-05T14:08:32.436Z|00026|reconnect|INFO|tcp:253.255.0.33:6642:
connection attempt failed (Connection refused)
2020-08-05T14:08:32.436Z|00027|reconnect|INFO|tcp:253.255.0.33:6642:
waiting 4 seconds before reconnect
2020-08-05T14:08:36.439Z|00028|reconnect|INFO|tcp:253.255.0.34:6642:
connecting...
2020-08-05T14:08:36.439Z|00029|reconnect|INFO|tcp:253.255.0.34:6642:
connection attempt failed (Connection refused)
2020-08-05T14:08:36.439Z|00030|reconnect|INFO|tcp:253.255.0.34:6642:
continuing to reconnect in the background but suppressing further logging
k8 node 3
--------------
2020-08-05T13:44:52.007Z|00001|vlog|INFO|opened log file
/var/log/ovn/ovsdb-server-nb.log
2020-08-05T13:44:52.009Z|00002|ovsdb_server|INFO|ovsdb-server (Open
vSwitch) 2.13.90
2020-08-05T13:44:52.010Z|00003|reconnect|INFO|tcp:[253.255.0.34]:6643:
connecting...
2020-08-05T13:44:52.010Z|00004|reconnect|INFO|tcp:[253.255.0.34]:6643:
connected
2020-08-05T13:44:52.010Z|00005|raft_rpc|INFO|learned cluster ID 6e05
2020-08-05T13:44:52.010Z|00006|raft|INFO|tcp:[253.255.0.34]:6643:
learned server ID c8a5
2020-08-05T13:44:52.011Z|00007|raft|INFO|server c8a5 is leader for term 1
2020-08-05T13:44:52.011Z|00008|raft|INFO|rejecting append_request
because previous entry 1,6 not in local log (mismatch past end of log)
2020-08-05T13:44:52.013Z|00009|raft|INFO|server c8a5 added to configuration
2020-08-05T13:44:52.014Z|00010|raft|INFO|server 0d3d added to configuration
2020-08-05T13:44:52.014Z|00011|reconnect|INFO|tcp:[253.255.0.33]:6643:
connecting...
2020-08-05T13:44:52.014Z|00012|reconnect|INFO|tcp:[253.255.0.33]:6643:
connected
2020-08-05T13:44:52.015Z|00013|raft|INFO|server 941e added to configuration
2020-08-05T13:44:52.015Z|00014|raft|INFO|tcp:253.255.0.34:52448: learned
server ID c8a5
2020-08-05T13:44:52.015Z|00015|raft|INFO|tcp:253.255.0.34:52448: learned
remote address tcp:[253.255.0.34]:6643
2020-08-05T13:44:52.015Z|00016|raft|INFO|tcp:253.255.0.33:56762: learned
server ID 0d3d
2020-08-05T13:44:52.015Z|00017|raft|INFO|tcp:253.255.0.33:56762: learned
remote address tcp:[253.255.0.33]:6643
2020-08-05T13:45:02.010Z|00018|memory|INFO|8884 kB peak resident set
size after 10.0 seconds
2020-08-05T13:45:02.010Z|00019|memory|INFO|cells:40 monitors:0
raft-connections:4
2020-08-05T14:10:03.593Z|00020|raft|INFO|received leadership transfer
from c8a5 in term 1
2020-08-05T14:10:03.593Z|00021|raft|INFO|term 2: starting election
2020-08-05T14:10:03.593Z|00022|reconnect|INFO|tcp:[253.255.0.34]:6643:
connection closed by peer
2020-08-05T14:10:03.594Z|00023|raft|INFO|term 2: elected leader by 2+ of
3 servers
2020-08-05T14:10:04.594Z|00024|reconnect|INFO|tcp:[253.255.0.34]:6643:
connecting...
2020-08-05T14:10:04.594Z|00025|reconnect|INFO|tcp:[253.255.0.34]:6643:
connection attempt failed (Connection refused)
2020-08-05T14:10:04.594Z|00026|reconnect|INFO|tcp:[253.255.0.34]:6643:
waiting 2 seconds before reconnect
2020-08-05T14:10:06.595Z|00027|reconnect|INFO|tcp:[253.255.0.34]:6643:
connecting...
2020-08-05T14:10:06.595Z|00028|reconnect|INFO|tcp:[253.255.0.34]:6643:
connection attempt failed (Connection refused)
2020-08-05T14:10:06.595Z|00029|reconnect|INFO|tcp:[253.255.0.34]:6643:
waiting 4 seconds before reconnect
cat /usr/var/log/ovn//backups/ovsdb-server-sb.13
2020-08-05T13:44:49.037Z|00001|vlog|INFO|opened log file
/var/log/ovn/ovsdb-server-sb.log
2020-08-05T13:44:49.039Z|00002|ovsdb_server|INFO|ovsdb-server (Open
vSwitch) 2.13.90
2020-08-05T13:44:49.040Z|00003|reconnect|INFO|tcp:[253.255.0.34]:6644:
connecting...
2020-08-05T13:44:49.040Z|00004|reconnect|INFO|tcp:[253.255.0.34]:6644:
connected
2020-08-05T13:44:49.040Z|00005|raft_rpc|INFO|learned cluster ID 281c
2020-08-05T13:44:49.040Z|00006|raft|INFO|tcp:[253.255.0.34]:6644:
learned server ID eeed
2020-08-05T13:44:49.040Z|00007|raft|INFO|server eeed is leader for term 1
2020-08-05T13:44:49.040Z|00008|raft|INFO|rejecting append_request
because previous entry 1,5 not in local log (mismatch past end of log)
2020-08-05T13:44:49.043Z|00009|raft|INFO|server eeed added to configuration
2020-08-05T13:44:49.043Z|00010|raft|INFO|server 5098 added to configuration
2020-08-05T13:44:49.043Z|00011|reconnect|INFO|tcp:[253.255.0.33]:6644:
connecting...
2020-08-05T13:44:49.043Z|00012|reconnect|INFO|tcp:[253.255.0.33]:6644:
connected
2020-08-05T13:44:49.044Z|00013|raft|INFO|server b9de added to configuration
2020-08-05T13:44:49.044Z|00014|raft|INFO|tcp:253.255.0.34:33550: learned
server ID eeed
2020-08-05T13:44:49.044Z|00015|raft|INFO|tcp:253.255.0.34:33550: learned
remote address tcp:[253.255.0.34]:6644
2020-08-05T13:44:49.044Z|00016|raft|INFO|tcp:253.255.0.33:58002: learned
server ID 5098
2020-08-05T13:44:49.044Z|00017|raft|INFO|tcp:253.255.0.33:58002: learned
remote address tcp:[253.255.0.33]:6644
2020-08-05T13:44:59.041Z|00018|memory|INFO|8652 kB peak resident set
size after 10.0 seconds
2020-08-05T13:44:59.041Z|00019|memory|INFO|cells:39 monitors:0
raft-connections:4
2020-08-05T14:10:03.593Z|00020|raft|INFO|received leadership transfer
from eeed in term 1
2020-08-05T14:10:03.593Z|00021|raft|INFO|term 2: starting election
2020-08-05T14:10:03.593Z|00022|reconnect|INFO|tcp:[253.255.0.34]:6644:
connection closed by peer
2020-08-05T14:10:03.594Z|00023|raft|INFO|term 2: elected leader by 2+ of
3 servers
2020-08-05T14:10:04.594Z|00024|reconnect|INFO|tcp:[253.255.0.34]:6644:
connecting...
2020-08-05T14:10:04.594Z|00025|reconnect|INFO|tcp:[253.255.0.34]:6644:
connection attempt failed (Connection refused)
2020-08-05T14:10:04.594Z|00026|reconnect|INFO|tcp:[253.255.0.34]:6644:
waiting 2 seconds before reconnect
2020-08-05T14:10:05.378Z|00027|reconnect|INFO|tcp:[253.255.0.33]:6644:
connection closed by peer
2020-08-05T13:47:29.329Z|00001|vlog|INFO|opened log file
/usr/var/log/ovn/ovn-controller.log
2020-08-05T13:47:29.331Z|00002|reconnect|INFO|unix:/usr/var/run/openvswitch/db.sock:
connecting...
2020-08-05T13:47:29.331Z|00003|reconnect|INFO|unix:/usr/var/run/openvswitch/db.sock:
connected
2020-08-05T13:47:29.334Z|00004|main|INFO|OVS IDL reconnected, force
recompute.
2020-08-05T13:47:29.334Z|00005|reconnect|INFO|tcp:253.255.0.34:6642:
connecting...
2020-08-05T13:47:29.334Z|00006|main|INFO|OVNSB IDL reconnected, force
recompute.
2020-08-05T13:47:29.334Z|00007|reconnect|INFO|tcp:253.255.0.34:6642:
connected
2020-08-05T13:47:29.336Z|00008|ofctrl|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting to switch
2020-08-05T13:47:29.336Z|00009|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:47:29.336Z|00010|rconn|WARN|unix:/usr/var/run/openvswitch/br-int.mgmt:
connection failed (No such file or directory)
2020-08-05T13:47:29.336Z|00011|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
waiting 1 seconds before reconnect
2020-08-05T13:47:30.337Z|00012|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:47:30.338Z|00013|rconn|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connected
2020-08-05T13:47:30.340Z|00014|dpif_netlink|INFO|The kernel module does
not support meters.
2020-08-05T13:47:30.341Z|00001|pinctrl(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting to switch
2020-08-05T13:47:30.341Z|00002|rconn(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connecting...
2020-08-05T13:47:30.341Z|00003|rconn(ovn_pinctrl0)|INFO|unix:/usr/var/run/openvswitch/br-int.mgmt:
connected
2020-08-05T14:10:03.592Z|00015|main|INFO|OVNSB commit failed, force
recompute next time.
2020-08-05T14:10:03.593Z|00016|jsonrpc|WARN|tcp:253.255.0.34:6642:
receive error: Connection reset by peer
2020-08-05T14:10:03.593Z|00017|reconnect|WARN|tcp:253.255.0.34:6642:
connection dropped (Connection reset by peer)
2020-08-05T14:10:03.593Z|00018|reconnect|INFO|tcp:253.255.0.33:6642:
connecting...
2020-08-05T14:10:03.593Z|00019|reconnect|INFO|tcp:253.255.0.33:6642:
connected
2020-08-05T14:10:05.378Z|00020|main|INFO|OVNSB commit failed, force
recompute next time.
2020-08-05T14:10:05.378Z|00021|jsonrpc|WARN|tcp:253.255.0.33:6642:
receive error: Connection reset by peer
2020-08-05T14:10:05.378Z|00022|reconnect|WARN|tcp:253.255.0.33:6642:
connection dropped (Connection reset by peer)
2020-08-05T14:10:05.379Z|00023|reconnect|INFO|tcp:253.255.0.35:6642:
connecting...
2020-08-05T14:10:05.379Z|00024|reconnect|INFO|tcp:253.255.0.35:6642:
connected
2020-08-05T14:10:05.485Z|00025|main|INFO|OVNSB commit failed, force
recompute next time.
2020-08-05T14:10:05.485Z|00026|jsonrpc|WARN|tcp:253.255.0.35:6642:
receive error: Connection reset by peer
2020-08-05T14:10:05.485Z|00027|reconnect|WARN|tcp:253.255.0.35:6642:
connection dropped (Connection reset by peer)
2020-08-05T14:10:06.486Z|00028|reconnect|INFO|tcp:253.255.0.34:6642:
connecting...
2020-08-05T14:10:06.487Z|00029|reconnect|INFO|tcp:253.255.0.34:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:06.487Z|00030|reconnect|INFO|tcp:253.255.0.34:6642:
waiting 2 seconds before reconnect
2020-08-05T14:10:08.488Z|00031|reconnect|INFO|tcp:253.255.0.33:6642:
connecting...
2020-08-05T14:10:08.488Z|00032|reconnect|INFO|tcp:253.255.0.33:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:08.488Z|00033|reconnect|INFO|tcp:253.255.0.33:6642:
waiting 4 seconds before reconnect
2020-08-05T14:10:12.492Z|00034|reconnect|INFO|tcp:253.255.0.35:6642:
connecting...
2020-08-05T14:10:12.493Z|00035|reconnect|INFO|tcp:253.255.0.35:6642:
connection attempt failed (Connection refused)
2020-08-05T14:10:12.493Z|00036|reconnect|INFO|tcp:253.255.0.35:6642:
continuing to reconnect in the background but suppressing further logging
On 05/08/2020 20:54, Dumitru Ceara wrote:
On 8/5/20 5:14 PM, Brendan Doyle wrote:
Folks,
I'm stumped here, I have the k8 ovnkube-db-raft Stateful set up and
running.
But when I create a simple network, no SB flows are generated.
ovn-nbctl show shows my network. ovn-sbctl show shows the physicals
systems in my network.
But I can't ping between any hosts because ovn-sbctl lflow-list is
empty, and there are no
errors or warnings in the logs. The ovn cluster says it is up and healthy.
Anybody got any ideas why this might be?
Hi Brendan,
Maybe I missed it but is ovn-northd running anywhere? If so, could you
please share its logs?
Thanks,
Dumitru
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
