Re: [ovs-discuss] raft ovsdb clustering

Cool! Yup Makes sense for sandbox northd also to point to clustered nb/sb dbs. On Wed, Apr 4, 2018 at 4:01 PM, Ben Pfaff wrote: > Oh, I see, from reading further in the thread, that this was indeed a > misunderstanding. Well, in any case that new option to ovs-sandbox can > be

Re: [ovs-discuss] raft ovsdb clustering

Oh, I see, from reading further in the thread, that this was indeed a misunderstanding. Well, in any case that new option to ovs-sandbox can be useful. On Wed, Apr 04, 2018 at 04:00:20PM -0700, Ben Pfaff wrote: > I would like to support cluster-wide locks. They require extra work and > they

Re: [ovs-discuss] raft ovsdb clustering

I would like to support cluster-wide locks. They require extra work and they require new OVSDB JSON-RPC protocol design (because locks are currently per-server, not per-database). I do not currently have a schedule for designing and implementing them. However, I am surprised that this is an

Re: [ovs-discuss] raft ovsdb clustering

Sure: #Node1 /usr/share/openvswitch/scripts/ovn-ctl --db-nb-addr=192.168.220.101 --db-nb-port=6641 --db-nb-cluster-local-addr=tcp:192.168.220.101:6645 --db-nb-create-insecure-remote=yes start_nb_ovsdb /usr/share/openvswitch/scripts/ovn-ctl --db-sb-addr=192.168.220.101 --db-sb-port=6642

Re: [ovs-discuss] raft ovsdb clustering

Thanks Aliasgar, I am still facing the same issue. Can you also share the (ovn-ctl) commands you used to start/join the ovsdb-server clusters in your nodes ? Thanks Numan On Tue, Mar 27, 2018 at 11:04 PM, aginwala wrote: > Hu Numan: > > You need to use --db as you are now

Re: [ovs-discuss] raft ovsdb clustering

Hu Numan: You need to use --db as you are now running db in cluster, you can access data from any of the three dbs. So if the leader crashes, it re-elects from the other two. Below is the e.g. command: # export remote="tcp:192.168.220.103:6641,tcp:192.168.220.102:6641,tcp: 192.168.220.101:6641"

Re: [ovs-discuss] raft ovsdb clustering

Hi Aliasgar, In your setup, if you kill the leader what is the behaviour ? Are you still able to create or delete any resources ? Is a new leader elected ? In my setup, the command "ovn-nbctl ls-add" for example blocks until I restart the ovsdb-server in node 1. And I don't see any other

Re: [ovs-discuss] raft ovsdb clustering

Sounds good. Just checked the patch, by default the C IDL has "leader_only" as true, which ensures that connection is to leader only. This is the case for northd. So the lock works for northd hot active-standby purpose if all the ovsdb endpoints of a cluster are specified to northd, since all

Re: [ovs-discuss] raft ovsdb clustering

Hi : Just sorted out the correct settings and northd also works in ha in raft. There were 2 issues in the setup: 1. I had started nb db without --db-nb-create-insecure-remote 2. I also started northd locally on all 3 without remote which is like all three northd trying to lock the ovsdb locally.

Re: [ovs-discuss] raft ovsdb clustering

Ali, sorry if I misunderstand what you are saying, but pacemaker here is for northd HA. pacemaker itself won't point to any ovsdb cluster node. All northds can point to a LB VIP for the ovsdb cluster, so if a member of ovsdb cluster is down it won't have impact to northd. Without clustering

Re: [ovs-discuss] raft ovsdb clustering

:) The only thing is while using pacemaker, if the node that pacemaker if pointing to is down, all the active/standby northd nodes have to be updated to new node from the cluster. But will dig in more to see what else I can find. @Ben: Any suggestions further? Regards, On Wed, Mar 21, 2018 at

Re: [ovs-discuss] raft ovsdb clustering

Thanks Numan: Yup agree with the locking part. For now; yes I am running northd on one node. I might right a script to monitor northd in cluster so that if the node where it's running goes down, script can spin up northd on one other active nodes as a dirty hack. Sure, will await for the inputs

Re: [ovs-discuss] raft ovsdb clustering

Hi Aliasgar, ovsdb-server maintains locks per each connection and not across the db. A workaround for you now would be to configure all the ovn-northd instances to connect to one ovsdb-server if you want to have active/standy. Probably Ben can answer if there is a plan to support ovsdb locks

Re: [ovs-discuss] raft ovsdb clustering

Hi Numan: Just figured out that ovn-northd is running as active on all 3 nodes instead of one active instance as I continued to test further which results in db errors as per logs. # on node 3, I run ovn-nbctl ls-add ls2 ; it populates below logs in ovn-north

Re: [ovs-discuss] raft ovsdb clustering

That's great Numan On Thu, Mar 15, 2018 at 2:57 AM, aginwala wrote: > Hi Numan: > > I tried on new nodes (kernel : 4.4.0-104-generic , Ubuntu 16.04)with fresh > installation and it worked super fine for both sb and nb dbs. Seems like > some kernel issue on the previous nodes

Re: [ovs-discuss] raft ovsdb clustering

Hi Numan: I tried on new nodes (kernel : 4.4.0-104-generic , Ubuntu 16.04)with fresh installation and it worked super fine for both sb and nb dbs. Seems like some kernel issue on the previous nodes when I re-installed raft patch as I was running different ovs version on those nodes before. For

Re: [ovs-discuss] raft ovsdb clustering

Sure. To add on , I also ran for nb db too using different port and Node2 crashes with same error : # Node 2 /usr/share/openvswitch/scripts/ovn-ctl --db-nb-addr=10.99.152.138 --db-nb-port=6641 --db-nb-cluster-remote-addr="tcp:10.99.152.148:6645"

Re: [ovs-discuss] raft ovsdb clustering

On Tue, Mar 13, 2018 at 9:46 PM, aginwala wrote: > Thanks Numan for the response. > > There is no command start_cluster_sb_ovsdb in the source code too. Is > that in a separate commit somewhere? Hence, I used start_sb_ovsdb which I > think would not be a right choice? > Sorry,

Re: [ovs-discuss] raft ovsdb clustering

Thanks Numan for the response. There is no command start_cluster_sb_ovsdb in the source code too. Is that in a separate commit somewhere? Hence, I used start_sb_ovsdb which I think would not be a right choice? # Node1 came up as expected. ovn-ctl --db-sb-addr=10.99.152.148 --db-sb-port=6642

Re: [ovs-discuss] raft ovsdb clustering

Hi Aliasgar, On Tue, Mar 13, 2018 at 7:11 AM, aginwala wrote: > Hi Ben/Noman: > > I am trying to setup 3 node southbound db cluster using raft10 > in review. > > # Node 1 create-cluster > ovsdb-tool create-cluster