Hi, Everything all works with the suggestions below. I have a question though I noticed when I brought the interface down on the leader node, a new leader was eventually elected, but in my 3 node cluster, the South and North leaders were put on different nodes which was a surprise to me, I would have thought they would migrate to the same node. Is this expected behavior? Everything still works, but was just surprised by this.
Brendan On 29/06/2020 15:21, Numan Siddique wrote:
On Mon, Jun 29, 2020 at 7:42 PM Brendan Doyle <[email protected] <mailto:[email protected]>> wrote:Hi, I've a question about running OVN "central" in a cluster. So I have (I think) successfully created a cluster on nodes ca-rain01 (leader) , ca-rain03 and ca-rain03 (See cluster status dumps) below. But It seems I can only run ovn-nbctl/ovn-sbctl on the leader node? Is this how it is supposed to work, what's the point of the cluster if so? Either you can run as: - ovn-nbctl --no-leader-only <command> or- ovn-nbctl --db="tcp:CLUSTER_IP1:6641,tcp:CLUSTER_IP2:6641,tcp:CLUSTER_IP3:6641" <command>Also assuming the cluster is up and running, what way should the ovn_controller external_ids be set on ovn chassis: Stand alone ------------------ ovn-ctl stop_controller ovs-vsctl set open . external-ids:ovn-bridge=br-int ovs-vsctl set open . external_ids:ovn-remote="tcp:<ovn central IP>:6642" ovs-vsctl set open . external_ids:ovn-nb="tcp:<ovn_central_IP>:6641" ovs-vsctl set open . external_ids:ovn-encap-ip=<ovn_central_IP> ovs-vsctl set open . external_ids:ovn-encap-type=geneve ovn-ctl start_controller Clustered ??? ------------------ ovn-ctl stop_controller ovs-vsctl set open . external-ids:ovn-bridge=br-int ovs-vsctl set open . external_ids:ovn-remote="tcp:<ovn central-1 IP>:6642,tcp:<ovn central-2 IP>:6642,tcp:<ovn central-3 IP>:6642" ovs-vsctl set open . external_ids:ovn-nb="tcp:<ovn_central-1 IP>:6641, tcp:<ovn_central-2 IP>:6641,tcp:<ovn_central-3 IP>:6641 " ovs-vsctl set open . external_ids:ovn-encap-ip="<ovn_central-1 IP, ovn_central-2 IP, ovn_central-3 IP>" ovs-vsctl set open . external_ids:ovn-encap-type=geneve ovn-ctl start_controller OR is it intended that the Clustered OVN be fronted by a floating IP/VIP and the ovn controllers pointed to that? What you're doing here is correct - i.e pass all the IPs in the remote. Thanks Numan ca-rain01 (leader) ----------------------- ovs-appctl -t /usr/var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound c5cb Name: OVN_Northbound Cluster ID: 3124 (312468f5-b4a6-432e-9a7f-25c84cdeb8a6) Server ID: c5cb (c5cbcb32-bc25-4744-b394-383f23a6e42b) Address: tcp:253.255.0.33:6643 <http://253.255.0.33:6643> Status: cluster member Role: leader Term: 1 Leader: self Vote: self Election timer: 1000 Log: [2, 5] Entries not yet committed: 0 Entries not yet applied: 0 Connections: <-a03a ->a03a <-1111 ->1111 Servers: c5cb (c5cb at tcp:253.255.0.33:6643 <http://253.255.0.33:6643>) (self) next_index=2 match_index=4 1111 (1111 at tcp:253.255.0.34:6643 <http://253.255.0.34:6643>) next_index=5 match_index=4 a03a (a03a at tcp:253.255.0.35:6643 <http://253.255.0.35:6643>) next_index=5 match_index=4 #ovs-appctl -t /usr/var/run/ovn/ovnsb_db.ctl cluster/status OVN_Southbound 6bc8 Name: OVN_Southbound Cluster ID: a548 (a548b761-aa92-476a-b992-80a7e0c86859) Server ID: 6bc8 (6bc80fbf-4c0f-42e2-af77-23a47291779b) Address: tcp:253.255.0.33:6644 <http://253.255.0.33:6644> Status: cluster member Role: leader Term: 1 Leader: self Vote: self Election timer: 1000 Log: [2, 6] Entries not yet committed: 0 Entries not yet applied: 0 Connections: ->de6e <-49e0 ->49e0 <-c91b Servers: c91b (c91b at tcp:253.255.0.34:6644 <http://253.255.0.34:6644>) next_index=6 match_index=5 de6e (de6e at tcp:253.255.0.34:6644 <http://253.255.0.34:6644>) next_index=6 match_index=4 49e0 (49e0 at tcp:253.255.0.35:6644 <http://253.255.0.35:6644>) next_index=6 match_index=5 6bc8 (6bc8 at tcp:253.255.0.33:6644 <http://253.255.0.33:6644>) (self) next_index=2 match_index=5 ca-rain02 (follower) -------------------------- ovs-appctl -t /usr/var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound 1111 Name: OVN_Northbound Cluster ID: 3124 (312468f5-b4a6-432e-9a7f-25c84cdeb8a6) Server ID: 1111 (1111cdaf-554c-44eb-aad2-b5868e4f02a9) Address: tcp:253.255.0.34:6643 <http://253.255.0.34:6643> Status: cluster member Role: follower Term: 1 Leader: c5cb Vote: unknown Election timer: 1000 Log: [2, 5] Entries not yet committed: 0 Entries not yet applied: 0 Connections: ->0000 ->a03a <-c5cb <-a03a Servers: c5cb (c5cb at tcp:253.255.0.33:6643 <http://253.255.0.33:6643>) 1111 (1111 at tcp:253.255.0.34:6643 <http://253.255.0.34:6643>) (self) a03a (a03a at tcp:253.255.0.35:6643 <http://253.255.0.35:6643>) ovs-appctl -t /usr/var/run/ovn/ovnsb_db.ctl cluster/status OVN_Southbound c91b Name: OVN_Southbound Cluster ID: a548 (a548b761-aa92-476a-b992-80a7e0c86859) Server ID: c91b (c91b0e2e-76a7-4b16-8af2-aafcaecd1d4f) Address: tcp:253.255.0.34:6644 <http://253.255.0.34:6644> Status: cluster member Role: follower Term: 1 Leader: 6bc8 Vote: unknown Election timer: 1000 Log: [2, 6] Entries not yet committed: 0 Entries not yet applied: 0 Connections: ->0000 ->49e0 <-0000 <-0000 Servers: c91b (c91b at tcp:253.255.0.34:6644 <http://253.255.0.34:6644>) (self) de6e (de6e at tcp:253.255.0.34:6644 <http://253.255.0.34:6644>) 49e0 (49e0 at tcp:253.255.0.35:6644 <http://253.255.0.35:6644>) 6bc8 (6bc8 at tcp:253.255.0.33:6644 <http://253.255.0.33:6644>) ca-rain03 (follower) -------------------------- #ovs-appctl -t /usr/var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound a03a Name: OVN_Northbound Cluster ID: 3124 (312468f5-b4a6-432e-9a7f-25c84cdeb8a6) Server ID: a03a (a03a8a83-b9d7-4538-a340-9ad8eba440c6) Address: tcp:253.255.0.35:6643 <http://253.255.0.35:6643> Status: cluster member Role: follower Term: 1 Leader: c5cb Vote: unknown Election timer: 1000 Log: [2, 5] Entries not yet committed: 0 Entries not yet applied: 0 Connections: ->0000 <-c5cb <-1111 ->1111 Servers: c5cb (c5cb at tcp:253.255.0.33:6643 <http://253.255.0.33:6643>) 1111 (1111 at tcp:253.255.0.34:6643 <http://253.255.0.34:6643>) a03a (a03a at tcp:253.255.0.35:6643 <http://253.255.0.35:6643>) (self) #ovs-appctl -t /usr/var/run/ovn/ovnsb_db.ctl cluster/status OVN_Southbound 49e0 Name: OVN_Southbound Cluster ID: a548 (a548b761-aa92-476a-b992-80a7e0c86859) Server ID: 49e0 (49e02f6f-4aa0-46af-8a74-90e30f121e58) Address: tcp:253.255.0.35:6644 <http://253.255.0.35:6644> Status: cluster member Role: follower Term: 1 Leader: 6bc8 Vote: unknown Election timer: 1000 Log: [2, 6] Entries not yet committed: 0 Entries not yet applied: 0 Connections: ->0000 ->de6e <-6bc8 <-c91b Servers: c91b (c91b at tcp:253.255.0.34:6644 <http://253.255.0.34:6644>) de6e (de6e at tcp:253.255.0.34:6644 <http://253.255.0.34:6644>) 49e0 (49e0 at tcp:253.255.0.35:6644 <http://253.255.0.35:6644>) (self) 6bc8 (6bc8 at tcp:253.255.0.33:6644 <http://253.255.0.33:6644>) Thanks Brendan _______________________________________________ discuss mailing list [email protected] <mailto:[email protected]> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
