dgrove-oss closed pull request #177: Test on multiple Kubernetes versions URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/177
This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/.travis.yml b/.travis.yml index 584f4bb..7b38ed4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,13 +2,13 @@ sudo: required env: global: - - TRAVIS_KUBE_VERSION=v1.7.4 - - TRAVIS_MINIKUBE_VERSION=v0.23.0 - secure: d7CuMXbhT83W2x78qiLwgogX1+3aPicd1PlTwwNNDN6QSkImbxareyKThnsqlHIiNj3o5l5DBuiYjy7wrF/xD1g8BQMmTwm99DRx5q3CI3Im3VCi/ZK8SaNjuOy24d7cf5k2tB/87Gk7zmKsMDYm+fpCl+GpgUmIEeIwthiAxuXSDWZ8eQPIptmxj56DeFRNouvXG+dEUtBfWiwN27UPxNKExCixFnegmdtffLbz6hhst7BHr5Ry9acbycre98PCwWZcu9lxFs+SJ1kvnzX2iue4otmDkF1WkJjxaOFPJVs/D3YItg+neLCSxjwBskPed+Fct8bOjcM/uVROJPNIq5icBmaPX2isH0lvtxOeVw/dmioWYXYPN9ygBOe4eO/vtPllN0bcAUo5xl9jXev8ciAozYrYpHVh9Fplfd81rcYTeYzALmRJBdoiWoc3KQGzwGc9sB1ffmy+KWgG9T0zbnS4fALSR4PSzyNlKSLXw9vuvdNP0OBYtO+6yTJXavIxqmDoj64Lb93n+uGPatnaIGPTKEEBMJTSjsgYVEfxzzZuxUT9Ldkf2lzqvN2PCllGoMqxkgsdb8i4v4QgRaMWBDbKa5Va4k0O4dnhRmtdbJavOSKN6fECJbyfoJlV1VvJGxk5znVLRznBmUPBKbNccyPZJULugKD3QIh4q8Q5jBU= - secure: CJtnU94HTDqd4A6uvhFl8IpnmU+wTdlzb8bPBFUl/lI/VKXiRrYpgJdKUro5xEoxFKuqMprLhbyf66niyWLTIeogjUAEu/h/o2dBVeGgSGKoqC0hQgqvnxKFeGlzFJ0XuEs3vbStJGRnQszGsfnnDrscJtR0x9X+1w4aBKI7iPyyuFtVkDD1UsmBbSi+M8FTeq7G7A0reMDaey7uog3CFCpIMl4geshcohQEcKEGbnXQZoLPFpb7cBOE83VXBJ7Y7Dgf/U4keiLovvnuJThGKZm/SVV2KlELmBmtmbx3rMT6Vb5k9ChSdRWapromNnnzmJBIQ5Scc2mwV3A93/SMha1F3IlYpDKs5djfTw8jZfVnuiou7HhTaRjHkmmcwP12/k30gLe2kw0Vezg1TCY4zgtOpcmCxc8RHEy0ceA74rKvRi8LbexTCwX+iAMQFn/pSrh/OqAq/50JbLyczcoO1zXWS38txUQNLW8i+XllhCg9pjkjyfBeGjOOcWiVIz9rWJd2XufjSXDcj6xoZHtkh1XDt1CnVkpsYKtyyZucQnhUM9ebmaWqbSW2+bpqC/2hI+G+kOyyCesGdB1q+VmN1augMMs6RgWjk4yw5dyLAshATSoUlE8KH2cDcJL19r4ECaQ99PSLwxoB89yfPoJiNc42vwxRdsLmB1BMNyPa81Y= matrix: - - OW_CONTAINER_FACTORY=docker - - OW_CONTAINER_FACTORY=kube + - OW_CONTAINER_FACTORY=docker TRAVIS_KUBE_VERSION=v1.7.4 TRAVIS_MINIKUBE_VERSION=v0.23.0 + - OW_CONTAINER_FACTORY=docker TRAVIS_KUBE_VERSION=v1.8.0 TRAVIS_MINIKUBE_VERSION=v0.25.2 + - OW_CONTAINER_FACTORY=docker TRAVIS_KUBE_VERSION=v1.9.0 TRAVIS_MINIKUBE_VERSION=v0.25.2 + - OW_CONTAINER_FACTORY=kube TRAVIS_KUBE_VERSION=v1.8.0 TRAVIS_MINIKUBE_VERSION=v0.25.2 services: - docker diff --git a/README.md b/README.md index cc701e5..a9f4501 100644 --- a/README.md +++ b/README.md @@ -17,20 +17,28 @@ This repository can be used to deploy OpenWhisk to a Kubernetes cluster. Several requirements must be met for OpenWhisk to deploy on Kubernetes. **Kubernetes** -* [Kubernetes](https://github.com/kubernetes/kubernetes) version 1.6+. However, avoid Kubernetes 1.6.3 due to an [issue with volume mount subpaths](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.6.md#known-issues-for-v163). Our Travis CI testing uses Kubernetes version 1.7.4. +* [Kubernetes](https://github.com/kubernetes/kubernetes) version 1.6+. However, multiple minor releases of Kubernetes, including 1.6.3, 1.7.14, 1.8.9 and 1.9.4 will not work for OpenWhisk due to bugs with volume mount subpaths (see[[1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.6.md#known-issues-for-v163), [2](https://github.com/kubernetes/kubernetes/issues/61076)]. This bug will surface as a failure when deploying the nginx container. * The ability to create Ingresses to make a Kubernetes service available outside of the cluster so you can actually use OpenWhisk. -* Endpoints of Kubernetes services must be able to loopback to themselves ("hairpin mode"). - -**OpenWhisk** -* Docker version 1.12+ +* Endpoints of Kubernetes services must be able to loopback to themselves (the kubelet's `hairpin-mode` must not be `none`). # Setting up Kubernetes ## Using Minikube -For local development and testing, we recommend using Minikube version 0.23+ -with the docker network in promiscuous mode. Our Travis CI testing uses Minikube 0.23.0. -Take a look at these [instructions](/docs/setting_up_minikube/README.md). +For local development and testing, we recommend using Minikube with +the docker network in promiscuous mode. Not all combinations of +Minikube and Kubernetes versions will work for running OpenWhisk. +Although other combinations may work, we recommend at least initially +using a combination from the table below that is verified by our +Travis CI testing. + +| Kubernetes Version | Minikube Version | +--- | --- | +1.7.4 | 0.23.0 | +1.8.0 | 0.25.2 | +1.9.0 | 0.25.2 | + +For details on setting up Minikube, see these [instructions](/docs/setting_up_minikube/README.md). ## Using a Kubernetes cluster from a cloud provider diff --git a/docs/setting_up_minikube/README.md b/docs/setting_up_minikube/README.md index 9ba6b0a..805ea8c 100644 --- a/docs/setting_up_minikube/README.md +++ b/docs/setting_up_minikube/README.md @@ -45,3 +45,7 @@ minikube ssh -- sudo ip link set docker0 promisc on ``` Your Minikube cluster should now be ready to deploy OpenWhisk. + +# Troubleshooting + +For some combinations of Minikube and Kubernetes versions, you may need to workaround a [Minikube DNS issue](https://github.com/kubernetes/minikube/issues/2240#issuecomment-348319371). A common symptom of this issue is that the OpenWhisk couchdb pod will fail to start with the error that it is unable to resolve `github.com` when cloning the openwhisk git repo. A work around is to delete the minikube cluster, issue the command `minikube config set bootstrapper kubeadm` and then redo the `minikube start` command above. diff --git a/tools/travis/build.sh b/tools/travis/build.sh index fbd65e1..77e4309 100755 --- a/tools/travis/build.sh +++ b/tools/travis/build.sh @@ -7,7 +7,7 @@ couchdbHealthCheck () { # wait for the pod to be created before getting the job name sleep 5 - POD_NAME=$(kubectl -n openwhisk get pods -o wide --show-all | grep "couchdb" | awk '{print $1}') + POD_NAME=$(kubectl -n openwhisk get pods -l name=couchdb -o wide --show-all | grep "couchdb" | awk '{print $1}') PASSED=false TIMEOUT=0 @@ -40,7 +40,7 @@ deploymentHealthCheck () { PASSED=false TIMEOUT=0 until $PASSED || [ $TIMEOUT -eq 60 ]; do - KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -o wide | grep "$1" | awk '{print $3}') + KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -l name="$1" -o wide | grep "$1" | awk '{print $3}') if [ "$KUBE_DEPLOY_STATUS" == "Running" ]; then PASSED=true break @@ -55,7 +55,7 @@ deploymentHealthCheck () { if [ "$PASSED" = false ]; then echo "Failed to finish deploying $1" - kubectl -n openwhisk logs $(kubectl -n openwhisk get pods -o wide | grep "$1" | awk '{print $1}') + kubectl -n openwhisk logs $(kubectl -n openwhisk get pods -l name="$1" -o wide | grep "$1" | awk '{print $1}') exit 1 fi @@ -71,7 +71,7 @@ statefulsetHealthCheck () { PASSED=false TIMEOUT=0 until $PASSED || [ $TIMEOUT -eq 60 ]; do - KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -o wide | grep "$1"-0 | awk '{print $3}') + KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -l name="$1" -o wide | grep "$1"-0 | awk '{print $3}') if [ "$KUBE_DEPLOY_STATUS" == "Running" ]; then PASSED=true break @@ -135,6 +135,9 @@ set -x SCRIPTDIR=$(cd $(dirname "$0") && pwd) ROOTDIR="$SCRIPTDIR/../../" +# Default to docker container factory if not specified +OW_CONTAINER_FACTORY=${OW_CONTAINER_FACTORY:="docker"} + cd $ROOTDIR # Label invoker nodes (needed for DockerContainerFactory-based invoker deployment) @@ -220,18 +223,19 @@ pushd kubernetes/invoker echo "Deploying invoker using DockerContainerFactory" kubectl -n openwhisk create cm invoker.config --from-env-file=invoker-dcf.env kubectl apply -f invoker-dcf.yml - deploymentHealthCheck "invoker" elif [ "$OW_CONTAINER_FACTORY" = "kube" ]; then echo "Deploying invoker using KubernetesContainerFactory" kubectl -n openwhisk create cm invoker.config --from-env-file=invoker-k8scf.env kubectl apply -f invoker-agent.yml deploymentHealthCheck "invoker-agent" kubectl apply -f invoker-k8scf.yml - deploymentHealthCheck "invoker-0" else echo "Unknown container factory $OW_CONTAINER_FACTORY" exit 1 fi + + # wait until invoker is ready + deploymentHealthCheck "invoker" popd # setup nginx ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
