rabbah closed pull request #85: enable active-active controller replicas URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/85
This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/kubernetes/controller/README.md b/kubernetes/controller/README.md index a3faed5..5976d4f 100644 --- a/kubernetes/controller/README.md +++ b/kubernetes/controller/README.md @@ -15,15 +15,15 @@ kubectl apply -f controller.yml # Controller Deployment Changes ## Increase Controller Count -If you want to increase the number of controllers deployed, -you will need to update a number of properties, for Kafka and Nginx. +If you want to change the number of controllers deployed, +you will also need to update properties for Kafka. * Kafka: Look at the Kafka [README](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/kafka/README.md) * Controller: You will need to update the replication count for the - Controllers [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L10) + Controllers [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L25) + and the value of CONTROLLER_INSTANCES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L80) + and the value of AKKA_CLUSTER_SEED_NODES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L110) and redeploy. -* Nginx: Take a look at the Nginx [README](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/nginx/README.md#increase-controller-count) - [StatefulSet]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ diff --git a/kubernetes/controller/controller.yml b/kubernetes/controller/controller.yml index 1ecbaa8..7e5e1b3 100644 --- a/kubernetes/controller/controller.yml +++ b/kubernetes/controller/controller.yml @@ -105,11 +105,13 @@ spec: - name: "CONTROLLER_BLACKBOXFRACTION" value: "0.10" - name: "CONTROLLER_LOCALBOOKKEEPING" - value: "TRUE" - - name: "CONTROLLER_HA" value: "FALSE" + - name: "CONTROLLER_HA" + value: "TRUE" - name: "AKKA_CLUSTER_SEED_NODES" value: "controller-0.controller.openwhisk controller-1.controller.openwhisk" + - name: "AKKA_ACTOR_PROVIDER" + value: "cluster" # properties for DB connection - name: "DB_USERNAME" diff --git a/kubernetes/nginx/README.md b/kubernetes/nginx/README.md index ff0dae4..db1dd0d 100644 --- a/kubernetes/nginx/README.md +++ b/kubernetes/nginx/README.md @@ -94,14 +94,6 @@ Kubernetes will then go through an update any deployed Nginx instances. Updating all of the keys defined in the nginx Secrets. -## Increase Controller Count - -If you are updating the number of controllers being deployed with OpenWhiks -from the default 2, you will need to make a few changes. The Nginx conf -file has routes for Controller [StatefulSet][StatefulSet] addresses. -Specifically [these lines](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/nginx/nginx.conf#L15-L20). -will need to be updated with a list of all available routes. - # Create Nginx Docker Image To build the Nginx docker image for Kubernetes on OpenWhisk, @@ -119,4 +111,3 @@ tmp directory, builds the Blackbox image and copies it into the Docker image. Then, each of the published WSK CLIs are download into the Docker image so that users are able to download them as usual. -[StatefulSet]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ diff --git a/kubernetes/nginx/nginx.conf b/kubernetes/nginx/nginx.conf index c76c40d..1f586fa 100644 --- a/kubernetes/nginx/nginx.conf +++ b/kubernetes/nginx/nginx.conf @@ -11,15 +11,6 @@ http { '$http_referer $http_user_agent $upstream_addr'; access_log /logs/nginx_access.log combined-upstream; - upstream controllers { - # These addresses are for the controller. If there are more - # than two invoker instances deployed then you will need to - # add additional controller addresses. - - server controller-0.controller.openwhisk:8080 fail_timeout=60s; - server controller-1.controller.openwhisk:8080 backup; - } - server { listen 443 default ssl; @@ -38,12 +29,17 @@ http { ssl_prefer_server_ciphers on; proxy_ssl_session_reuse off; + # Hack to convince nginx to dynamically resolve the dns entry. + # Required to get HA for controller to work. See issue #77. + resolver kube-dns.kube-system; + set $controller controller.openwhisk.svc.cluster.local; + # proxy to the web action path location / { if ($namespace) { rewrite /(.*) /api/v1/web/${namespace}/$1 break; } - proxy_pass http://controllers; + proxy_pass http://$controller:8080; proxy_read_timeout 70s; # 60+10 additional seconds to allow controller to terminate request } @@ -52,7 +48,7 @@ http { if ($namespace) { rewrite ^ /api/v1/web/${namespace}/public/index.html break; } - proxy_pass http://controllers; + proxy_pass http://$controller:8080; proxy_read_timeout 70s; # 60+10 additional seconds to allow controller to terminate request } ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
