rabbah closed pull request #133: fixes for apigateway; restructure deployment
to eliminate cycles
URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/133
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/README.md b/README.md
index ddf2c80..0dd241c 100644
--- a/README.md
+++ b/README.md
@@ -38,9 +38,17 @@ You can also provision a Kubernetes cluster from a cloud
provider, subject to th
# Configuring OpenWhisk
+The first time you deploy OpenWhisk on Kubernetes, we recommend
+following the steps below manually so you can inspect the results and
+debug your setup. After you are confident that OpenWhisk deploys
+smoothly on your cluster, you might find it useful to drive your
+deployments using the script [build.sh](tools/travis/build.sh) that we
+use to deploy OpenWhisk on Kubernetes for our Travis CI testing.
+
## Initial Cluster Configuration
* Follow the steps for initial [Cluster
Setup](kubernetes/cluster-setup/README.md)
+* Configure your [Ingresses](kubernetes/ingress/README.md), including
configuring the wsk CLI.
## Configure or Deploy CouchDB
@@ -62,11 +70,13 @@ directory tree. Follow the instructions for each step in
order.
* Deploy [Zookeeper](kubernetes/zookeeper/README.md)
* Deploy [Kafka](kubernetes/kafka/README.md)
* Deploy [Controller](kubernetes/controller/README.md)
-* Deploy [Nginx](kubernetes/nginx/README.md)
-* Deploy [Ingress](kubernetes/ingress/README.md), including configuring the
wsk CLI.
* Deploy [Invoker](kubernetes/invoker/README.md)
-* Deploy [RouteMgmt](kubernetes/routemgmt/README.md)
-* Deploy [Package Catalog](kubernetes/openwhisk-catalog/README.md)
+* Deploy [Nginx](kubernetes/nginx/README.md)
+
+## Install system actions and the openwhisk catalog
+
+* Install [RouteMgmt](kubernetes/routemgmt/README.md)
+* Install [Package Catalog](kubernetes/openwhisk-catalog/README.md)
## Verify
diff --git a/kubernetes/apigateway/README.md b/kubernetes/apigateway/README.md
index 91db8e2..672016a 100644
--- a/kubernetes/apigateway/README.md
+++ b/kubernetes/apigateway/README.md
@@ -8,3 +8,10 @@ To deploy the ApiGateway, you only need to run the following
command:
```
kubectl apply -f apigateway.yml
```
+
+Note: The URL returned from `wsk api create` may contain a spurious
+:8080 due to its assumption about the meaning of PUBLIC_MANAGEDURL_HOST.
+Working on a fix to the upstream incubator-openwhisk-apigateway project
+to weaken the assumption that the API URL is constructed by concatenating
+PUBLIC_MANAGEDURL_HOST:PUBLIC_MANAGEDURL_PORT as this is not always
+appropriate for kube-based deployments.
diff --git a/kubernetes/apigateway/apigateway.yml
b/kubernetes/apigateway/apigateway.yml
index 6d1fa3c..98d8d3b 100644
--- a/kubernetes/apigateway/apigateway.yml
+++ b/kubernetes/apigateway/apigateway.yml
@@ -1,22 +1,3 @@
----
-apiVersion: v1
-kind: Service
-metadata:
- name: apigateway
- namespace: openwhisk
- labels:
- name: apigateway
-spec:
- selector:
- name: apigateway
- ports:
- - port: 8080
- targetPort: 8080
- name: mgmt
- - port: 9000
- targetPort: 9000
- name: api
-
---
apiVersion: extensions/v1beta1
kind: Deployment
@@ -39,7 +20,7 @@ spec:
imagePullPolicy: IfNotPresent
image: redis:3.2
- name: apigateway
- imagePullPolicy: IfNotPresent
+ imagePullPolicy: Always
image: openwhisk/apigateway
ports:
- name: mgmt
@@ -51,3 +32,8 @@ spec:
value: "127.0.0.1"
- name: "REDIS_PORT"
value: "6379"
+ - name: "PUBLIC_GATEWAY_URL"
+ valueFrom:
+ configMapKeyRef:
+ name: whisk.ingress
+ key: apigw_url
diff --git a/kubernetes/cluster-setup/README.md
b/kubernetes/cluster-setup/README.md
index b86758b..d44adbf 100644
--- a/kubernetes/cluster-setup/README.md
+++ b/kubernetes/cluster-setup/README.md
@@ -14,6 +14,12 @@ Perform the following steps to prepare your cluster for
OpenWhisk.
kubectl apply -f namespace.yml
```
+### Create services
+
+```
+kubectl apply -f services.yml
+```
+
### Create authorization secrets
The command below installs the default guest and system authorization
diff --git a/kubernetes/cluster-setup/services.yml
b/kubernetes/cluster-setup/services.yml
new file mode 100644
index 0000000..0e02ff1
--- /dev/null
+++ b/kubernetes/cluster-setup/services.yml
@@ -0,0 +1,100 @@
+########
+# First define services that are exposed via Ingresses to the outside world
+########
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: nginx
+ namespace: openwhisk
+ labels:
+ name: nginx
+spec:
+ type: NodePort
+ selector:
+ name: nginx
+ ports:
+ - port: 80
+ targetPort: 80
+ name: http
+ - port: 443
+ targetPort: 443
+ name: https-api
+ - port: 8443
+ targetPort: 8443
+ name: https-admin
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: apigateway
+ namespace: openwhisk
+ labels:
+ name: apigateway
+spec:
+ type: NodePort
+ selector:
+ name: apigateway
+ ports:
+ - port: 8080
+ targetPort: 8080
+ name: mgmt
+ - port: 9000
+ targetPort: 9000
+ name: api
+
+########
+# Second define internal services used by the OpenWhisk implementation
+########
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: controller
+ namespace: openwhisk
+ labels:
+ name: controller
+spec:
+ selector:
+ name: controller
+ clusterIP: None
+ ports:
+ - port: 8080
+ targetPort: 8080
+ name: http
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: kafka
+ namespace: openwhisk
+ labels:
+ name: kafka
+spec:
+ selector:
+ name: kafka
+ ports:
+ - port: 9092
+ targetPort: 9092
+ name: kafka
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: zookeeper
+ namespace: openwhisk
+ labels:
+ name: zookeeper
+spec:
+ selector:
+ name: zookeeper
+ ports:
+ - port: 2181
+ targetPort: 2181
+ name: zookeeper
+ - port: 2888
+ targetPort: 2888
+ name: server
+ - port: 3888
+ targetPort: 3888
+ name: leader-election
diff --git a/kubernetes/controller/README.md b/kubernetes/controller/README.md
index 755c154..db0505c 100644
--- a/kubernetes/controller/README.md
+++ b/kubernetes/controller/README.md
@@ -15,10 +15,10 @@ kubectl apply -f controller.yml
# Controller Deployment Changes
## Increase Controller Count
-You will need to update the replication count for the
-Controllers
[here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L26)
-and the value of CONTROLLER_INSTANCES
[here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L82)
-and the value of AKKA_CLUSTER_SEED_NODES
[here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L112)
+You will need to update the number of replicas for the
+Controllers
[here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L10)
+and the value of CONTROLLER_INSTANCES
[here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L60)
+and the value of AKKA_CLUSTER_SEED_NODES
[here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L88)
and redeploy.
[StatefulSet]:
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
diff --git a/kubernetes/controller/controller.yml
b/kubernetes/controller/controller.yml
index 4eb90d7..993caf4 100644
--- a/kubernetes/controller/controller.yml
+++ b/kubernetes/controller/controller.yml
@@ -1,20 +1,4 @@
---
-apiVersion: v1
-kind: Service
-metadata:
- name: controller
- namespace: openwhisk
- labels:
- name: controller
-spec:
- selector:
- name: controller
- clusterIP: None
- ports:
- - port: 8080
- targetPort: 8080
- name: http
----
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
diff --git a/kubernetes/couchdb/README.md b/kubernetes/couchdb/README.md
index 642a76d..3ce9c6a 100644
--- a/kubernetes/couchdb/README.md
+++ b/kubernetes/couchdb/README.md
@@ -20,8 +20,7 @@ This is because it might not yet be configured. To check if
the
DB has been setup, you can look at the Pod logs with
```
- export COUCH_DB_POD=$(kubectl -n openwhisk get pods -o wide --show-all |
grep "couchdb" | awk '{print $1}')
- kubectl -n openwhisk logs $COUCH_DB_POD
+kubectl -n openwhisk logs -lname=couchdb
```
In the logs, you should see the line:
@@ -36,7 +35,7 @@ This indicates that the CouchDB instancs is up and running.
## Usernames and Passwords
To configure custom usernames and passwords, you can edit
-the CouchDB [setup
pod](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/couchdb/couchdb.yml#L46-L49).
+the CouchDB [setup
pod](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/couchdb/couchdb.yml#L48-L51).
**NOTE** If the CouchDB username and password properties
are updated, then you will need to update the Controller
diff --git a/kubernetes/couchdb/couchdb.yml b/kubernetes/couchdb/couchdb.yml
index 5bd67e9..30a277f 100644
--- a/kubernetes/couchdb/couchdb.yml
+++ b/kubernetes/couchdb/couchdb.yml
@@ -1,3 +1,5 @@
+# The couchdb service is defined here instead of in cluster-config/services
+# because some deployments will not put the database within the kube cluster.
---
apiVersion: v1
kind: Service
diff --git a/kubernetes/ingress/README.md b/kubernetes/ingress/README.md
index 4c7892f..8cc8cc6 100644
--- a/kubernetes/ingress/README.md
+++ b/kubernetes/ingress/README.md
@@ -16,10 +16,13 @@ vary across cloud providers. The detailed instructions
configurations. We welcome contributions from the community to
describe how to configure Ingress for additional cloud providers.
-2. Record the value of API_HOST in a Kubernetes configmap for later
-use within the OpenWhisk deployment:
+2. Record the value of API_HOST and APIGW_URL in a Kubernetes configmap
+for later use within the OpenWhisk deployment. Note that API_HOST is
+expected to be either a host or host:port pair, but APIGW_URL is
+expected to be a URL, including protocol (http or https depending on
+your ingress):
```
-kubectl -n openwhisk create configmap whisk.ingress
--from-literal=api_host=API_HOST
+kubectl -n openwhisk create configmap whisk.ingress
--from-literal=api_host=API_HOST --from-literal=apigw_url=APIGW_URL
```
3. Configure the OpenWhisk CLI, wsk, by setting the auth and apihost
@@ -36,26 +39,33 @@ wsk property set --auth `cat ../cluster-setup/auth.guest`
--apihost API_HOST
## NodePort
-When it was deployed, the nginx service was configured to expose
-itself via a NodePort
[see](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/nginx/nginx.yml#L10)
+When it was deployed, the apigateway and nginx services were
+configured to expose themselves via a NodePort
+[see](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/cluster-setup/services.yml#L13)
By determining the IP address of a worker node and the exposed port
-number, you can determine your API_HOST. There are no additional files
-to apply. TLS termination is handled by the nginx service.
+numbers, you can determine your API_HOST and APIGW_URL. There are no
+additional files to apply. TLS termination is handled by the nginx
+service.
1. Obtain the IP address of the Kubernetes nodes.
-
```
kubectl get nodes
```
2. Obtain the public port for https port of the openwhisk.nginx Service
-
```
kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort|
awk '{print $3}' | cut -d'/' -f1
```
Use IP_ADDR:PUBLIC_PORT as your API_HOST
+3. Obtain the public port for https port of the openwhisk.apigateway Service
+ ```
+kubectl -n openwhisk describe service apigateway | grep mgmt | grep NodePort|
awk '{print $3}' | cut -d'/' -f1
+ ```
+
+Use http://IP_ADDR:PUBLIC_PORT as your APIGW_URL
+
## Simple Service Ingress
@@ -93,6 +103,14 @@ kubectl -n openwhisk describe service nginx | grep
https-api | grep NodePort| aw
```
Use PublicIP:PORT as your API_HOST
+3. Obtain the public port for https port of the openwhisk.apigateway Service
+
+ ```
+kubectl -n openwhisk describe service apigateway | grep mgmt | grep NodePort|
awk '{print $3}' | cut -d'/' -f1
+ ```
+
+Use http://IP_ADDR:PUBLIC_PORT as your APIGW_URL
+
### IBM Cloud standard cluster
A template file ingress-ibm.yml is provided. You will need to edit
@@ -126,8 +144,8 @@ After editing the template file, deploy it.
kubectl apply -f ingress-ibm.yml
```
-Your OpenWhisk API_HOST will be <ibmdomain>/openwhisk
-
+Your API_HOST will be <ibmdomain>/openwhisk
+Your APIGW_URL will be https://<ibmdomain>/apigateway
## Other cloud providers
diff --git a/kubernetes/ingress/ingress-ibm.yml
b/kubernetes/ingress/ingress-ibm.yml
index 505004f..0947c34 100644
--- a/kubernetes/ingress/ingress-ibm.yml
+++ b/kubernetes/ingress/ingress-ibm.yml
@@ -4,7 +4,7 @@ metadata:
name: ow-ingress
namespace: openwhisk
annotations:
- ingress.bluemix.net/rewrite-path: "serviceName=nginx rewrite=/"
+ ingress.bluemix.net/rewrite-path: "serviceName=nginx
rewrite=/;serviceName=apigateway rewrite=/"
spec:
tls:
- hosts:
@@ -17,4 +17,8 @@ spec:
- path: /openwhisk/
backend:
serviceName: nginx
- servicePort: 80
+ servicePort: http
+ - path: /apigateway/
+ backend:
+ serviceName: apigateway
+ servicePort: mgmt
diff --git a/kubernetes/ingress/ingress-simple.yml
b/kubernetes/ingress/ingress-simple.yml
index 90ad2a7..fba89ea 100644
--- a/kubernetes/ingress/ingress-simple.yml
+++ b/kubernetes/ingress/ingress-simple.yml
@@ -6,4 +6,7 @@ metadata:
spec:
backend:
serviceName: nginx
- servicePort: 443
+ servicePort: https-api
+ backend:
+ serviceName: apigateway
+ servicePort: mgmt
diff --git a/kubernetes/kafka/kafka.yml b/kubernetes/kafka/kafka.yml
index 19817cf..9f67ece 100644
--- a/kubernetes/kafka/kafka.yml
+++ b/kubernetes/kafka/kafka.yml
@@ -1,19 +1,3 @@
----
-apiVersion: v1
-kind: Service
-metadata:
- name: kafka
- namespace: openwhisk
- labels:
- name: kafka
-spec:
- selector:
- name: kafka
- ports:
- - port: 9092
- targetPort: 9092
- name: kafka
-
---
apiVersion: extensions/v1beta1
kind: Deployment
diff --git a/kubernetes/nginx/nginx.yml b/kubernetes/nginx/nginx.yml
index f9659a4..486a66d 100644
--- a/kubernetes/nginx/nginx.yml
+++ b/kubernetes/nginx/nginx.yml
@@ -1,26 +1,3 @@
----
-apiVersion: v1
-kind: Service
-metadata:
- name: nginx
- namespace: openwhisk
- labels:
- name: nginx
-spec:
- type: NodePort
- selector:
- name: nginx
- ports:
- - port: 80
- targetPort: 80
- name: http
- - port: 443
- targetPort: 443
- name: https-api
- - port: 8443
- targetPort: 8443
- name: https-admin
-
---
apiVersion: extensions/v1beta1
kind: Deployment
diff --git a/kubernetes/zookeeper/zookeeper.yml
b/kubernetes/zookeeper/zookeeper.yml
index 50123ed..cd292c5 100644
--- a/kubernetes/zookeeper/zookeeper.yml
+++ b/kubernetes/zookeeper/zookeeper.yml
@@ -1,25 +1,3 @@
----
-apiVersion: v1
-kind: Service
-metadata:
- name: zookeeper
- namespace: openwhisk
- labels:
- name: zookeeper
-spec:
- selector:
- name: zookeeper
- ports:
- - port: 2181
- targetPort: 2181
- name: zookeeper
- - port: 2888
- targetPort: 2888
- name: server
- - port: 3888
- targetPort: 3888
- name: leader-election
-
---
apiVersion: extensions/v1beta1
kind: Deployment
diff --git a/tools/travis/build.sh b/tools/travis/build.sh
index 87b7066..4fbed1a 100755
--- a/tools/travis/build.sh
+++ b/tools/travis/build.sh
@@ -146,9 +146,20 @@ kubectl describe nodes
echo "Performing steps from cluster-setup"
pushd kubernetes/cluster-setup
kubectl apply -f namespace.yml
+ kubectl apply -f services.yml
kubectl -n openwhisk create secret generic whisk.auth
--from-file=system=auth.whisk.system --from-file=guest=auth.guest
popd
+# configure Ingress and wsk CLI
+# We use the NodePorts for nginx and apigateway services for Travis CI testing
+pushd kubernetes/ingress
+ WSK_PORT=$(kubectl -n openwhisk describe service nginx | grep https-api |
grep NodePort| awk '{print $3}' | cut -d'/' -f1)
+ APIGW_PORT=$(kubectl -n openwhisk describe service apigateway | grep mgmt |
grep NodePort| awk '{print $3}' | cut -d'/' -f1)
+ WSK_HOST=$(kubectl describe nodes | grep Hostname: | awk '{print $2}')
+ kubectl -n openwhisk create configmap whisk.ingress
--from-literal=api_host=$WSK_HOST:$WSK_PORT
--from-literal=apigw_url=http://$WSK_HOST:$APIGW_PORT
+ wsk property set --auth `cat ../cluster-setup/auth.guest` --apihost
$WSK_HOST:$WSK_PORT
+popd
+
# setup couchdb
echo "Deploying couchdb"
pushd kubernetes/couchdb
@@ -189,6 +200,15 @@ pushd kubernetes/controller
statefulsetHealthCheck "controller"
popd
+# setup the invoker
+echo "Deploying invoker"
+pushd kubernetes/invoker
+ kubectl apply -f invoker.yml
+
+ # wait until the invoker is ready
+ deploymentHealthCheck "invoker"
+popd
+
# setup nginx
echo "Deploying nginx"
pushd kubernetes/nginx
@@ -206,24 +226,6 @@ pushd kubernetes/nginx
deploymentHealthCheck "nginx"
popd
-# configure Ingress and wsk CLI
-# We use a NodePort for Travis CI testing
-pushd kubernetes/ingress
- WSK_PORT=$(kubectl -n openwhisk describe service nginx | grep https-api |
grep NodePort| awk '{print $3}' | cut -d'/' -f1)
- WSK_HOST=$(kubectl describe nodes | grep Hostname: | awk '{print $2}')
- kubectl -n openwhisk create configmap whisk.ingress
--from-literal=api_host=$WSK_HOST:$WSK_PORT
- wsk property set --auth `cat ../cluster-setup/auth.guest` --apihost
$WSK_HOST:$WSK_PORT
-popd
-
-# setup the invoker
-echo "Deploying invoker"
-pushd kubernetes/invoker
- kubectl apply -f invoker.yml
-
- # wait until the invoker is ready
- deploymentHealthCheck "invoker"
-popd
-
# install routemgmt
echo "Installing routemgmt"
pushd kubernetes/routemgmt
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
