Hello Brendan, After you run the './daemonset.sh` script, there will be two DB related yaml files in `dist/yaml` folder. The ovnkube-db.yaml brings up standalone OVN DBs, whilst the ovnkube-db-raft brings up the OVN Clustered DBs. Please do `kubectl apply -f $HOME/work/src/ github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db-raft.yaml <http://github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db.yaml>`.
Furthermore, if you read that YAML file the node selector is set to nodes with label `k8s.ovn.org/ovnkube-db=true`. So, you will need to annotate at least 3 nodes with that label. HTH Regards, ~Girish On Mon, Jul 6, 2020 at 8:37 AM Brendan Doyle <[email protected]> wrote: > So I've tried the steps in > https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d > : > > cd $HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/images > ./daemonset.sh --image=docker.io/ovnkube/ovn-daemonset-u:latest \ > --net-cidr=192.168.0.0/16 --svc-cidr=172.16.1.0/24 \ > --gateway-mode="local" \ > --k8s-apiserver=https://$MASTER_IP:6443 > > > # Create OVN namespace, service accounts, ovnkube-db headless service, > configmap, and policies > kubectl create -f > $HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovn-setup.yaml > > # Run ovnkube-db deployment. > kubectl create -f > $HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db.yaml > > # Run ovnkube-master deployment. > kubectl create -f > $HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-master.yaml > > # Run ovnkube daemonset for nodes > kubectl create -f > $HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-node.yaml > > > And I see nothing in my k8 cluster, seems like it does nothing > > [root@ca-rain01 yaml]# kubectl create -f ovnkube-master.yaml > deployment "ovnkube-master" created > [root@ca-rain01 yaml]# kubectl delete deployment ovnkube-master > Error from server (NotFound): deployments.extensions "ovnkube-master" not > found > > Has anybody got this working, or used any other means to deploy an OVN > cluster as a K8 Statefulset? > > > Brendan > > > > > On 06/07/2020 12:33, Brendan Doyle wrote: > > Hi, > > So I'm really confused by what you have pointed me to here. As stated I do > NOT > want to use OVN as a CNI. I have a k8s cluster that use flannel as the > CNI. I simply > want to create an OVN "central" cluster as a Stateful set in my *existing* > K8 > config. > > This repo: > > https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d > > Seems to be for setting up a K8s cluster to use OVN as the CNI?? > Have you tried this? > What IP do the ovn-controllers use to reach the OVN "central cluster? > It seems to use an OVN docker image from docker.io, I want to use my own > OVN src > Do I use/modify the dist/images/Dockerfile in this repo? that has loads of > references to CNI > like I said I don't want to use OVN as the CNI?? > > > The instructions here > https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst > seem more promising, if not a little confusing: > > IN the section "Starting OVN Central services in containers" > > Export following variables in .env and place it under project root: > > $ OVN_BRANCH=<BRANCH> > $ OVN_VERSION=<VERSION> > $ DISTRO=<LINUX_DISTRO> > $ KERNEL_VERSION=<LINUX_KERNEL_VERSION> > $ GITHUB_SRC=<GITHUB_URL> > $ DOCKER_REPO=<REPO_TO_PUSH_IMAGE> > > > Does it mean create a file called ".env" and place it in the toplevel dir > of the cloned ovn repo? > Or does it mean just add these to you shell environment (i.e put them in > .bashrc)? > > Then we have: > > 1) > > Start OVN containers using below command: > > $ docker run -itd --net=host --name=ovn-nb \ > <docker_repo>:<tag> ovn-nb-tcp > > $ docker run -itd --net=host --name=ovn-sb \ > <docker_repo>:<tag> ovn-sb-tcp > > $ docker run -itd --net=host --name=ovn-northd \ > <docker_repo>:<tag> ovn-northd-tcp > > followed by > > 2) > > $ docker run -e "host_ip=<host_ip>" -e "nb_db_port=<port>" -itd \ > --name=ovn-nb-raft --net=host --privileged <docker_repo>:<tag> \ > ovn-nb-cluster-create > > $ docker run -e "host_ip=<host_ip>" -e "sb_db_port=<port>" -itd \ > --name=ovn-sb-raft --net=host --privileged <docker_repo>:<tag> \ > ovn-sb-cluster-create > > $ docker run -e "OVN_NB_DB=tcp:<node1>:6641,tcp:<node2>:6641,\ > tcp:<node3>:6641" -e "OVN_SB_DB=tcp:<node1>:6642,tcp:<node2>:6642,\ > tcp:<node3>:6642" -itd --name=ovn-northd-raft <docker_repo>:<tag> \ > ovn-northd-cluster > > Does it mean do 1), then 2) or does it mean do 1) for non HA OVN central > *OR* 2) > for HA/clustered OVN Central? > > It's not clear > > Thanks > > > > > > > On 25/06/2020 17:36, aginwala wrote: > > Hi: > > There are a couple of options as I have been exploring this too: > > 1. Upstream ovn-k8s patches ( > https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d) > uses statefulset and headless service for starting ovn central raft cluster > with 3 replicas. Cluster startup code and pod specs are pretty neat that > addresses most of the doubts. > > OVN components have been containerized too to start them in pods. You can > also refer to > https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst > for the same and use them to make it work in pod specs too. > > > 2. Write a new ovn operator similar to etcd operator > https://github.com/coreos/etcd-operator which just takes the count of > raft replicas and does the job in the background. > > I also added ovn-k8s group so they can comment on any other ideas too. > Hope it helps. > > > > On Thu, Jun 25, 2020 at 7:15 AM Brendan Doyle <[email protected]> > wrote: > >> Hi, >> >> So I'm trying to find information on setting up an OVN "Central/Master" >> high availability (HA) >> Not as Active-Backup with Pacemaker, but as a cluster. But I want to >> deploy and manage that >> cluster as a Kubernetes service . >> >> I can find lots of stuff on "ovn-kube" but this seems to be using OVN as >> a kubernetes CNI instead of >> Flannel etc. But this is not what I want to do, I have a kubernetes >> cluster using Flannel as the CNI, >> now I want to deploy a HA OVN "Central" as a kubernetes service. Kind >> of like how you can deploy >> a MySQL cluster in kubernetes using a SatefulSet deployment. >> >> I have found this: >> https://github.com/ovn-org/ovn-kubernetes#readme >> >> But it is not clear to me if this is how to setup OVN as a kubernetes >> CNI or it's how to setup a HA >> OVN central as kubernetes service. >> >> Can anybody comment, has anyone done this? >> >> >> I guess I could run an OVN central as standalone and use a kubernetes >> deployment with 3 >> replica sets and "export" as a NodePort service. And have a >> floating/VIP on my kubernetes >> nodes. And direct ovn-controllers to the VIP. So only the pod that holds >> the VIP would service >> requests. This would work and give HA, but you don't get the performance >> of an OVN >> clustered Database Model, where each OVN central could service requests. >> >> >> >> >> Thanks >> >> >> Rdgs >> Brendan >> >> _______________________________________________ >> discuss mailing list >> [email protected] >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >> > > > _______________________________________________ > discuss mailing > [email protected]https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > _______________________________________________ > discuss mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
