OK so an update, and thanks for the help todate. These steps get my OVN
cluster up and running:
kubectl create -f ovn-setup.yaml kubectl apply -f ovnkube-db-raft.yaml
However I see warning messages of the like:
2020-07-07T15:15:12Z|00001|ovsdb_idl|WARN|OVN_Northbound database lacks
Forwarding_Group table (database needs upgrade?)
I suspect because the OVN docker image is not the same as the OVN bits
I'm running on my chassis, or is not the latest.
I guess now its a question of figuring out how to use my own OVN bits
and not those that are in the
http://docker.io/ovnkube/ovn-daemonset:latest
Thanks
Brendan
On 07/07/2020 11:33, Brendan Doyle wrote:
On 06/07/2020 21:29, Girish Moodalbail wrote:
Hello Brendan,
After you run the './daemonset.sh` script, there will be two DB
related yaml files in `dist/yaml` folder. The ovnkube-db.yaml brings
up standalone OVN DBs, whilst the ovnkube-db-raft brings up the OVN
Clustered DBs. Please do `kubectl apply -f
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db-raft.yaml
<http://github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db.yaml>`.
Humm "kubectl apply -f" or "kubectl create-f" as per
https://github.com/ovn-org/ovn-kubernetes/ instructions. And what
needs to be run before that?
create -f ovn-setup.yaml apply -f ovnkube-db-raft.yaml
What about ovnkube-master.yaml, I think not?
Furthermore, if you read that YAML file the node selector is set to
nodes with label `k8s.ovn.org/ovnkube-db=true`
<http://k8s.ovn.org/ovnkube-db=true`>. So, you will need to annotate
at least 3 nodes with that label.
Yes, would be good to have that in a README.
Thanks I will try again with these.
HTH
Regards,
~Girish
On Mon, Jul 6, 2020 at 8:37 AM Brendan Doyle
<[email protected] <mailto:[email protected]>> wrote:
So I've tried the steps in
https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d:
|cd $HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/images
<http://github.com/ovn-org/ovn-kubernetes/dist/images>
./daemonset.sh --image=docker.io/ovnkube/ovn-daemonset-u:latest
<http://docker.io/ovnkube/ovn-daemonset-u:latest> \
--net-cidr=192.168.0.0/16 <http://192.168.0.0/16>
--svc-cidr=172.16.1.0/24 <http://172.16.1.0/24> \
--gateway-mode="local" \ --k8s-apiserver=https://$MASTER_IP:6443|
|# Create OVN namespace, service accounts, ovnkube-db headless
service, configmap, and policies kubectl create -f
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovn-setup.yaml
<http://github.com/ovn-org/ovn-kubernetes/dist/yaml/ovn-setup.yaml>
# Run ovnkube-db deployment. kubectl create -f
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db.yaml
<http://github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-db.yaml>
# Run ovnkube-master deployment. kubectl create -f
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-master.yaml
<http://github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-master.yaml>
# Run ovnkube daemonset for nodes kubectl create -f
$HOME/work/src/github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-node.yaml
<http://github.com/ovn-org/ovn-kubernetes/dist/yaml/ovnkube-node.yaml>
And I see nothing in my k8 cluster, seems like it does nothing
[root@ca-rain01 yaml]# kubectl create -f ovnkube-master.yaml
deployment "ovnkube-master" created [root@ca-rain01 yaml]#
kubectl delete deployment ovnkube-master Error from server
(NotFound): deployments.extensions "ovnkube-master" not found Has
anybody got this working, or used any other means to deploy an
OVN cluster as a K8 Statefulset? Brendan |
On 06/07/2020 12:33, Brendan Doyle wrote:
Hi,
So I'm really confused by what you have pointed me to here. As
stated I do NOT
want to use OVN as a CNI. I have a k8s cluster that use flannel
as the CNI. I simply
want to create an OVN "central" cluster as a Stateful set in my
*existing* K8
config.
This repo:
https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d
Seems to be for setting up a K8s cluster to use OVN as the CNI??
Have you tried this?
What IP do the ovn-controllers use to reach the OVN "central
cluster?
It seems to use an OVN docker image from docker.io
<http://docker.io>, I want to use my own OVN src
Do I use/modify the dist/images/Dockerfile in this repo? that
has loads of references to CNI
like I said I don't want to use OVN as the CNI??
The instructions here
https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst
seem more promising, if not a little confusing:
IN the section "Starting OVN Central services in containers"
Export following variables in .env and place it under project root:
$ OVN_BRANCH=<BRANCH>
$ OVN_VERSION=<VERSION>
$ DISTRO=<LINUX_DISTRO>
$ KERNEL_VERSION=<LINUX_KERNEL_VERSION>
$ GITHUB_SRC=<GITHUB_URL>
$ DOCKER_REPO=<REPO_TO_PUSH_IMAGE>
Does it mean create a file called ".env" and place it in the
toplevel dir of the cloned ovn repo?
Or does it mean just add these to you shell environment (i.e put
them in .bashrc)?
Then we have:
1)
Start OVN containers using below command:
$ docker run -itd --net=host --name=ovn-nb \
<docker_repo>:<tag> ovn-nb-tcp
$ docker run -itd --net=host --name=ovn-sb \
<docker_repo>:<tag> ovn-sb-tcp
$ docker run -itd --net=host --name=ovn-northd \
<docker_repo>:<tag> ovn-northd-tcp
followed by
2)
$ docker run -e "host_ip=<host_ip>" -e "nb_db_port=<port>" -itd \
--name=ovn-nb-raft --net=host --privileged <docker_repo>:<tag> \
ovn-nb-cluster-create
$ docker run -e "host_ip=<host_ip>" -e "sb_db_port=<port>" -itd \
--name=ovn-sb-raft --net=host --privileged <docker_repo>:<tag> \
ovn-sb-cluster-create
$ docker run -e "OVN_NB_DB=tcp:<node1>:6641,tcp:<node2>:6641,\
tcp:<node3>:6641" -e "OVN_SB_DB=tcp:<node1>:6642,tcp:<node2>:6642,\
tcp:<node3>:6642" -itd --name=ovn-northd-raft <docker_repo>:<tag> \
ovn-northd-cluster
Does it mean do 1), then 2) or does it mean do 1) for non HA OVN
central *OR* 2)
for HA/clustered OVN Central?
It's not clear
Thanks
On 25/06/2020 17:36, aginwala wrote:
Hi:
There are a couple of options as I have been exploring this too:
1. Upstream ovn-k8s patches
(https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d)
uses statefulset and headless service for starting ovn central
raft cluster with 3 replicas. Cluster startup code and pod
specs are pretty neat that addresses most of the doubts.
OVN components have been containerized too to start them in
pods. You can also refer to
https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst
for the same and use them to make it work in pod specs too.
2. Write a new ovn operator similar to etcd operator
https://github.com/coreos/etcd-operator which just takes the
count of raft replicas and does the job in the background.
I also added ovn-k8s group so they can comment on any other
ideas too. Hope it helps.
On Thu, Jun 25, 2020 at 7:15 AM Brendan Doyle
<[email protected] <mailto:[email protected]>> wrote:
Hi,
So I'm trying to find information on setting up an OVN
"Central/Master"
high availability (HA)
Not as Active-Backup with Pacemaker, but as a cluster. But
I want to
deploy and manage that
cluster as a Kubernetes service .
I can find lots of stuff on "ovn-kube" but this seems to be
using OVN as
a kubernetes CNI instead of
Flannel etc. But this is not what I want to do, I have a
kubernetes
cluster using Flannel as the CNI,
now I want to deploy a HA OVN "Central" as a kubernetes
service. Kind
of like how you can deploy
a MySQL cluster in kubernetes using a SatefulSet deployment.
I have found this:
https://github.com/ovn-org/ovn-kubernetes#readme
But it is not clear to me if this is how to setup OVN as a
kubernetes
CNI or it's how to setup a HA
OVN central as kubernetes service.
Can anybody comment, has anyone done this?
I guess I could run an OVN central as standalone and use a
kubernetes
deployment with 3
replica sets and "export" as a NodePort service. And have a
floating/VIP on my kubernetes
nodes. And direct ovn-controllers to the VIP. So only the
pod that holds
the VIP would service
requests. This would work and give HA, but you don't get
the performance
of an OVN
clustered Database Model, where each OVN central could
service requests.
Thanks
Rdgs
Brendan
_______________________________________________
discuss mailing list
[email protected] <mailto:[email protected]>
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
[email protected] <mailto:[email protected]>
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
[email protected] <mailto:[email protected]>
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
