On 06/07/2020 21:10, aginwala wrote:
On Mon, Jul 6, 2020 at 4:33 AM Brendan Doyle <[email protected]
<mailto:[email protected]>> wrote:
Hi,
So I'm really confused by what you have pointed me to here. As
stated I do NOT
want to use OVN as a CNI. I have a k8s cluster that use flannel as
the CNI. I simply
want to create an OVN "central" cluster as a Stateful set in my
*existing* K8
config.
This repo:
https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d
Seems to be for setting up a K8s cluster to use OVN as the CNI??
Still wondering about this? the above repo seems to be for creating an
OVN CNI for kubernetes
Is this correct ???
But it also seems to include yamls for creating an OVN cluster as a k8s
service via a StatefulSet
Is it necessary to create the OVN CNI in order to use
ovnkube-db-raft.yam ???
As I have said I have an existing k8s cluster using a flannel CNI, I
just want to deploy an OVN
central as a StatefulSet to that.
Have you tried this?
What IP do the ovn-controllers use to reach the OVN "central cluster?
It seems to use an OVN docker image from docker.io
<http://docker.io>, I want to use my own OVN src
Do I use/modify the dist/images/Dockerfile in this repo? that has
loads of references to CNI
like I said I don't want to use OVN as the CNI??
A pre-req for running ovn central as a k8s app is containerize ovn
central components. Hence, you need to start your own containers using
docker.
Either you follow the approach from ovn-k8s repo as to how to build
ovn images or refer to the docker instructions in ovn repo. Since this
app (ovn central) will run behind a k8s service, ovn-controller should
point to the service ip of ovn central k8s app. k8s folks can comment
on how to build image that is in k8s pod specs e.g
http://docker.io/ovnkube/ovn-daemonset:latest
Yes the Docker image claims to be built using dist/images/Dockerfile,
which installs more than just the OVN ovn central components
The instructions here
https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst
seem more promising, if not a little confusing:
1)
Start OVN containers using below command:
$ docker run -itd --net=host --name=ovn-nb \
<docker_repo>:<tag> ovn-nb-tcp
$ docker run -itd --net=host --name=ovn-sb \
<docker_repo>:<tag> ovn-sb-tcp
$ docker run -itd --net=host --name=ovn-northd \
<docker_repo>:<tag> ovn-northd-tcp
followed by
2)
$ docker run -e "host_ip=<host_ip>" -e "nb_db_port=<port>" -itd \
--name=ovn-nb-raft --net=host --privileged <docker_repo>:<tag> \
ovn-nb-cluster-create
$ docker run -e "host_ip=<host_ip>" -e "sb_db_port=<port>" -itd \
--name=ovn-sb-raft --net=host --privileged <docker_repo>:<tag> \
ovn-sb-cluster-create
$ docker run -e "OVN_NB_DB=tcp:<node1>:6641,tcp:<node2>:6641,\
tcp:<node3>:6641" -e "OVN_SB_DB=tcp:<node1>:6642,tcp:<node2>:6642,\
tcp:<node3>:6642" -itd --name=ovn-northd-raft <docker_repo>:<tag> \
ovn-northd-cluster
Does it mean do 1), then 2) or does it mean do 1) for non HA OVN
central *OR* 2)
for HA/clustered OVN Central?
Doc says Start OVN containers in cluster mode using below command on
node2 and node3 to make them join the peer using below command:.
Hence, you can even play with just docker on 3 nodes where you run
step1 on node1 that creates cluster
Ok, is that the 1) above, surely 2) above creates the cluster
"ovn-nb-cluster-create" ???
and do the join-cluster on rest two nodes to give you a clear idea
before moving to pod in k8s. Not sure if you need more details to
update doc. We can always improvise. Upstream ovn-k8s does the same
for pods where e.g. ovn-kube0 pod creates a cluster and rest two pods
joins
It's not clear
The docs are not clear, it seems to me the docs intend to say..
"
OVN containers can then be started either as a Stand Alone Database or as a
Clustered Database.
To start OVN containers as a Stand Alone Database use the commands below
$ docker run -itd --net=host --name=ovn-nb \ <docker_repo>:<tag> ovn-nb-tcp
etc..
To start OVN containers in cluster mode for a 3 node cluster using below
command on node1:
$ docker run -e "host_ip=<host_ip>" -e "nb_db_port=<port>" -itd \
--name=ovn-nb-raft --net=host --privileged <docker_repo>:<tag> \
ovn-nb-cluster-create Then Start OVN containers in cluster mode using
below command on node2 and node3 to make them join the peer using below
command:
$ docker run -e "host_ip=<host_ip>" -e "remote_host=<remote_host_ip>" \
-e "nb_db_port=<port>" -itd --name=ovn-nb-raft --net=host \ --privileged
<docker_repo>:<tag> ovn-nb-cluster-join etc..
Brendan.
Thanks
On 25/06/2020 17:36, aginwala wrote:
Hi:
There are a couple of options as I have been exploring this too:
1. Upstream ovn-k8s patches
(https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d)
uses statefulset and headless service for starting ovn central
raft cluster with 3 replicas. Cluster startup code and pod specs
are pretty neat that addresses most of the doubts.
OVN components have been containerized too to start them in pods.
You can also refer to
https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst
for the same and use them to make it work in pod specs too.
2. Write a new ovn operator similar to etcd operator
https://github.com/coreos/etcd-operator which just takes the
count of raft replicas and does the job in the background.
I also added ovn-k8s group so they can comment on any other ideas
too. Hope it helps.
On Thu, Jun 25, 2020 at 7:15 AM Brendan Doyle
<[email protected] <mailto:[email protected]>> wrote:
Hi,
So I'm trying to find information on setting up an OVN
"Central/Master"
high availability (HA)
Not as Active-Backup with Pacemaker, but as a cluster. But I
want to
deploy and manage that
cluster as a Kubernetes service .
I can find lots of stuff on "ovn-kube" but this seems to be
using OVN as
a kubernetes CNI instead of
Flannel etc. But this is not what I want to do, I have a
kubernetes
cluster using Flannel as the CNI,
now I want to deploy a HA OVN "Central" as a kubernetes
service. Kind
of like how you can deploy
a MySQL cluster in kubernetes using a SatefulSet deployment.
I have found this:
https://github.com/ovn-org/ovn-kubernetes#readme
But it is not clear to me if this is how to setup OVN as a
kubernetes
CNI or it's how to setup a HA
OVN central as kubernetes service.
Can anybody comment, has anyone done this?
I guess I could run an OVN central as standalone and use a
kubernetes
deployment with 3
replica sets and "export" as a NodePort service. And have a
floating/VIP on my kubernetes
nodes. And direct ovn-controllers to the VIP. So only the pod
that holds
the VIP would service
requests. This would work and give HA, but you don't get the
performance
of an OVN
clustered Database Model, where each OVN central could
service requests.
Thanks
Rdgs
Brendan
_______________________________________________
discuss mailing list
[email protected] <mailto:[email protected]>
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
