Hi Numan, Sorry for delay reply but I figured it out how to setup cluster basically I have to remove everything from /var/lib/ovn directory on each new nodes before I join them to cluster.
Now I have other issue where I messed up my cluster using leave command and don’t know how to rejoin it. So basically I destroy cluster and rebuild it but in that process i lost my ovsdb database and now I don’t know how to restore previous data? I found some backup files inside /var/lib/ovn/ directories and I did restore using ovsdb-client utility but they didn’t help. I love ovn design but missing documents specially related cluster operation and whenever I google something all I found people asking question but not good answer. It would be great to have nice cluster operation and management doc. (I’m planning to write blog once I get everything working in lab) Sent from my iPhone > On May 21, 2021, at 4:35 PM, Numan Siddique <num...@ovn.org> wrote: > > On Fri, May 21, 2021 at 9:53 AM Satish Patel <satish....@gmail.com> wrote: >> >> Folks, >> >> I have 3 controller nodes and am trying to setup clustering for OVN >> deployment, but lack enough documentation and have some issues and >> questions regarding it. >> >> I found this document but again its little confusing and very old - >> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-March/046470.html >> >> # controller 1 >> >> /usr/share/ovn/scripts/ovn-ctl --db-nb-addr=172.30.40.93 \ >> --db-nb-create-insecure-remote=yes \ >> --db-sb-addr=172.30.40.93 \ >> --db-sb-create-insecure-remote=yes \ >> --db-nb-cluster-local-addr=172.30.40.93 \ >> --db-sb-cluster-local-addr=172.30.40.93 \ >> --ovn-northd-nb-db=tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641 >> \ >> --ovn-northd-sb-db=tcp:172.30.40.93:6642,tcp:172.30.40.25:6642,tcp:172.30.40.177:6642 >> \ >> start_northd >> >> >> # controller 2 >> >> /usr/share/ovn/scripts/ovn-ctl --db-nb-addr=172.30.40.25 \ >> --db-nb-create-insecure-remote=yes \ >> --db-sb-addr=172.30.40.25 \ >> --db-sb-create-insecure-remote=yes \ >> --db-nb-cluster-local-addr=172.30.40.25 \ >> --db-sb-cluster-local-addr=172.30.40.25 \ >> --db-nb-cluster-remote-addr=172.30.40.93 \ >> --db-sb-cluster-remote-addr=172.30.40.93 \ >> --ovn-northd-nb-db=tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641 >> \ >> --ovn-northd-sb-db=tcp:172.30.40.93:6642,tcp:172.30.40.25:6642,tcp:172.30.40.177:6642 >> \ >> start_northd >> >> >> # controller 3 >> >> /usr/share/ovn/scripts/ovn-ctl --db-nb-addr=172.30.40.177 \ >> --db-nb-create-insecure-remote=yes \ >> --db-nb-cluster-local-addr=172.30.40.177 \ >> --db-sb-addr=172.30.40.177 \ >> --db-sb-create-insecure-remote=yes \ >> --db-sb-cluster-local-addr=172.30.40.177 \ >> --db-nb-cluster-remote-addr=172.30.40.93 \ >> --db-sb-cluster-remote-addr=172.30.40.93 \ >> --ovn-northd-nb-db=tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641 >> \ >> --ovn-northd-sb-db=tcp:172.30.40.93:6642,tcp:172.30.40.25:6642,tcp:172.30.40.177:6642 >> \ >> start_northd >> >> ## Validation steps >> >> controller-2# export\ >> remote="tcp:172.30.40.93:6641,tcp:172.30.40.25:6641,tcp:172.30.40.177:6641" >> >> controller-2# ovn-nbctl --db=$remote show >> controller-2# >> >> In the above command i am seeing output only when it hit controller-1 >> node, but for node-2 and note-3 giving me empty output that means data >> replication doesn't work. what is the command to verify >> synchronization working between all 3 nodes? >> >> Do I need to restart any other services? > > Hi Satish, > > Can you check out this script and try it - > https://github.com/ovn-org/ovn-fake-multinode/blob/master/ovn_cluster.sh#L353 > > Probably there is no need for you to start northd on all three nodes. > If you want to start northd on 3 nodes, > then they should be configured to connect to the clustered databases. > Probably in your case, ovn-norths > are connecting to the local unix sockets of ovsdb servers. > > Thanks > Numan > >> _______________________________________________ >> discuss mailing list >> disc...@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >> _______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
