rabbah closed pull request #249: refactor of kube-deploy docker images
URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/249
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docker/README.md b/docker/README.md
index 4af5406..c0915bc 100644
--- a/docker/README.md
+++ b/docker/README.md
@@ -24,16 +24,12 @@ These images are built automatically and published
to DockerHub under the openwhisk userid. Docker images are
published on all successful Travis CI builds of the master branch.
The built images are:
+ * whisk-ansible-runner - An alpine-based utility image for running
+ ansible playbooks.
+ * whisk-script-runner - An alpine-based utility image for running
+ simple bash scripts that need the `wsk` cli available to them.
* couchdb - creates and initializes a CouchDB instance for
dev/testing of OpenWhisk. This image is not intended for
production usage.
- * docker-pull - performs a 'docker pull' for action runtimes
- specified in runtimesManifest format -- used to prefetch
- action runtime images for invoker nodes
* invoker-agent - worker node invoker agent -- used to implement
suspend/resume and log consolidation ops for a remote invoker
- * openwhisk-catalog - installs the catalog from the project
- incubator-openwhisk-calalog to the system namespace of the
- OpenWhisk deployment.
- * routemgmt - installs OpenWhisk's route management package
- in the system namespace of the OpenWhisk deployment.
diff --git a/docker/kafkapkg-installer/Dockerfile
b/docker/kafkapkg-installer/Dockerfile
deleted file mode 100644
index 88a0c50..0000000
--- a/docker/kafkapkg-installer/Dockerfile
+++ /dev/null
@@ -1,14 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
-
-from ubuntu:latest
-
-RUN apt-get -y update && apt-get -y install \
- git \
- wget \
- zip
-
-COPY init.sh /init.sh
-RUN chmod +x /init.sh
-
-CMD ["/init.sh"]
diff --git a/docker/openwhisk-catalog/Dockerfile
b/docker/openwhisk-catalog/Dockerfile
deleted file mode 100644
index 0d7f59d..0000000
--- a/docker/openwhisk-catalog/Dockerfile
+++ /dev/null
@@ -1,13 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
-
-from ubuntu:latest
-
-RUN apt-get -y update && apt-get -y install \
- git \
- wget
-
-COPY init.sh /init.sh
-RUN chmod +X /init.sh
-
-CMD ["/init.sh"]
diff --git a/docker/openwhisk-catalog/init.sh b/docker/openwhisk-catalog/init.sh
deleted file mode 100755
index e9ee73c..0000000
--- a/docker/openwhisk-catalog/init.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/bash
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
-
-set -ex
-
-# Clone openwhisk-catalog
-# TODO: when openwhisk-catalog has releases, download release instead!
-git clone https://github.com/apache/incubator-openwhisk-catalog
openwhisk-catalog
-
-# TODO: installCatalog.sh wants OPENWHISK_HOME set, but doesn't actually need
-# it for anything. Fix upstream and then remove this.
-export OPENWHISK_HOME=/openwhisk
-mkdir -p $OPENWHISK_HOME/bin
-
-# Download and install openwhisk cli
-pushd $OPENWHISK_HOME/bin
- wget -q
https://github.com/apache/incubator-openwhisk-cli/releases/download/$WHISK_CLI_VERSION/OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
- tar xzf OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
-popd
-
-# Run installCatalog.sh
-pushd openwhisk-catalog/packages
- ./installCatalog.sh $WHISK_AUTH $WHISK_API_HOST_NAME $OPENWHISK_HOME/bin/wsk
-popd
diff --git a/docker/routemgmt/Dockerfile b/docker/routemgmt/Dockerfile
deleted file mode 100644
index c3560f1..0000000
--- a/docker/routemgmt/Dockerfile
+++ /dev/null
@@ -1,14 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
-
-from ubuntu:latest
-
-RUN apt-get -y update && apt-get -y install \
- git \
- wget \
- zip
-
-COPY init.sh /init.sh
-RUN chmod +X /init.sh
-
-CMD ["/init.sh"]
diff --git a/docker/docker-pull/Dockerfile
b/docker/whisk-ansible-runner/Dockerfile
similarity index 66%
rename from docker/docker-pull/Dockerfile
rename to docker/whisk-ansible-runner/Dockerfile
index b81d304..938b5c3 100644
--- a/docker/docker-pull/Dockerfile
+++ b/docker/whisk-ansible-runner/Dockerfile
@@ -1,13 +1,22 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
# license agreements; and to You under the Apache License, Version 2.0.
-from ubuntu:latest
+from alpine
ENV DOCKER_VERSION 1.12.0
-RUN apt-get -y update && apt-get -y install \
- wget \
- python-pip
+RUN apk --no-cache add \
+ python \
+ py-pip \
+ libffi \
+ openssl
+
+RUN apk --no-cache add --virtual build-dependencies \
+ python-dev \
+ build-base \
+ libffi-dev \
+ openssl-dev \
+ wget
# Install docker client
RUN wget --no-verbose
https://get.docker.com/builds/Linux/x86_64/docker-${DOCKER_VERSION}.tgz && \
@@ -19,6 +28,6 @@ RUN pip install --upgrade setuptools
RUN pip install ansible==2.5.2
RUN pip install jinja2==2.9.6
-COPY pull_images.yml /pull_images.yml
+RUN apk del build-dependencies
-CMD ["/usr/local/bin/ansible-playbook", "/pull_images.yml"]
+CMD ["/usr/bin/ansible-playbook", "/task/playbook.yml"]
diff --git a/docker/whisk-script-runner/Dockerfile
b/docker/whisk-script-runner/Dockerfile
new file mode 100644
index 0000000..b7801d1
--- /dev/null
+++ b/docker/whisk-script-runner/Dockerfile
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+
+from alpine
+
+RUN apk --no-cache add \
+ bash \
+ git \
+ libc6-compat \
+ wget \
+ zip
+
+# Install `wsk` cli in /usr/local/bin
+ENV WHISK_CLI_VERSION latest
+RUN wget -q
https://github.com/apache/incubator-openwhisk-cli/releases/download/$WHISK_CLI_VERSION/OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
&& \
+ tar xzf OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz -C /usr/local/bin
wsk && \
+ rm OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
+
+COPY init.sh /init.sh
+RUN chmod +X /init.sh
+
+CMD ["/init.sh"]
diff --git a/docker/whisk-script-runner/init.sh
b/docker/whisk-script-runner/init.sh
new file mode 100755
index 0000000..e76c2ca
--- /dev/null
+++ b/docker/whisk-script-runner/init.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+
+set -ex
+
+# Run the custom task mounted into /task by kube/docker
+if [ -f /task/myTask.sh ]; then
+ . /task/myTask.sh
+else
+ echo "I was not given a task to execute. Exiting with error"
+ exit 1
+fi
diff --git a/helm/openwhisk/configMapFiles/README.md
b/helm/openwhisk/configMapFiles/README.md
new file mode 100644
index 0000000..9fafff4
--- /dev/null
+++ b/helm/openwhisk/configMapFiles/README.md
@@ -0,0 +1,23 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+This directory contains files that are used to create ConfigMaps
+that are volume-mounted into containers. By keeping them as separate
+files, we avoid needing to escape {{ or }} and can use natural indentation.
+
diff --git a/docker/docker-pull/pull_images.yml
b/helm/openwhisk/configMapFiles/dockerPullRuntimes/playbook.yml
similarity index 100%
rename from docker/docker-pull/pull_images.yml
rename to helm/openwhisk/configMapFiles/dockerPullRuntimes/playbook.yml
diff --git a/helm/openwhisk/configMapFiles/installCatalog/myTask.sh
b/helm/openwhisk/configMapFiles/installCatalog/myTask.sh
new file mode 100644
index 0000000..61c6293
--- /dev/null
+++ b/helm/openwhisk/configMapFiles/installCatalog/myTask.sh
@@ -0,0 +1,13 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+
+# TODO: fix upstream: installCatalog.sh requires OPENWHISK_HOME set, but
doesn't actually need it to be valid
+export OPENWHISK_HOME=/openwhisk
+
+# Clone openwhisk-catalog
+git clone https://github.com/apache/incubator-openwhisk-catalog
openwhisk-catalog
+
+# Run installCatalog.sh
+pushd openwhisk-catalog/packages
+ ./installCatalog.sh $WHISK_AUTH $WHISK_API_HOST_NAME /usr/local/bin/wsk
+popd
diff --git a/docker/routemgmt/init.sh
b/helm/openwhisk/configMapFiles/installRouteMgmt/myTask.sh
old mode 100755
new mode 100644
similarity index 55%
rename from docker/routemgmt/init.sh
rename to helm/openwhisk/configMapFiles/installRouteMgmt/myTask.sh
index 6c860c5..533fa1c
--- a/docker/routemgmt/init.sh
+++ b/helm/openwhisk/configMapFiles/installRouteMgmt/myTask.sh
@@ -1,23 +1,11 @@
-#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
# license agreements; and to You under the Apache License, Version 2.0.
-set -ex
-
export OPENWHISK_HOME=/openwhisk
-# Clone openwhisk repo to get latest installRouteMgmt.sh and core/routemgmt
-# TODO: when OpenWhisk has releases, download release artifacts instead!
+# Clone openwhisk repo to get installRouteMgmt.sh and core/routemgmt
git clone https://github.com/apache/incubator-openwhisk openwhisk
-cd $OPENWHISK_HOME
-
-# Download and install openwhisk cli
-pushd bin
- wget -q
https://github.com/apache/incubator-openwhisk-cli/releases/download/$WHISK_CLI_VERSION/OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
- tar xzf OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
-popd
-
# Setup env for installRouteMgmt.sh
if [ "$WHISK_API_GATEWAY_USER" ]; then
export GW_USER=$WHISK_API_GATEWAY_USER
@@ -37,6 +25,6 @@ else
fi
# Run installRouteMgmt.sh
-pushd ansible/roles/routemgmt/files
- ./installRouteMgmt.sh $WHISK_AUTH $WHISK_API_HOST_NAME $WHISK_NAMESPACE
$OPENWHISK_HOME/bin/wsk
+pushd $OPENWHISK_HOME/ansible/roles/routemgmt/files
+ ./installRouteMgmt.sh $WHISK_AUTH $WHISK_API_HOST_NAME $WHISK_NAMESPACE
/usr/local/bin/wsk
popd
diff --git a/helm/openwhisk/templates/_invokerHelpers.tpl
b/helm/openwhisk/templates/_invokerHelpers.tpl
index 35ac441..216a440 100644
--- a/helm/openwhisk/templates/_invokerHelpers.tpl
+++ b/helm/openwhisk/templates/_invokerHelpers.tpl
@@ -34,6 +34,9 @@
volumeMounts:
- name: dockersock
mountPath: "/var/run/docker.sock"
+ - name: task-dir
+ mountPath: "/task/playbook.yml"
+ subPath: "playbook.yml"
env:
# action runtimes
- name: "RUNTIMES_MANIFEST"
diff --git a/helm/openwhisk/templates/installCatalogJob.yaml
b/helm/openwhisk/templates/installCatalogJob.yaml
index 7dfc05b..41d41ab 100644
--- a/helm/openwhisk/templates/installCatalogJob.yaml
+++ b/helm/openwhisk/templates/installCatalogJob.yaml
@@ -13,18 +13,21 @@ spec:
name: install-catalog
spec:
restartPolicy: Never
+ volumes:
+ - name: task-dir
+ configMap:
+ name: install-catalog
initContainers:
# Wait for a controller to be up so we can perfom our CRUD actions with
the CLI
{{ include "readiness.waitForController" . | indent 6 }}
containers:
- name: catalog
- image: openwhisk/kube-openwhisk-catalog
+ image: openwhisk/kube-whisk-script-runner
+ volumeMounts:
+ - name: task-dir
+ mountPath: "/task/myTask.sh"
+ subPath: "myTask.sh"
env:
- - name: "WHISK_CLI_VERSION"
- valueFrom:
- configMapKeyRef:
- name: whisk.config
- key: whisk_cli_version_tag
- name: "WHISK_AUTH"
valueFrom:
secretKeyRef:
@@ -36,3 +39,12 @@ spec:
configMapKeyRef:
name: whisk.config
key: whisk_api_host_nameAndPort
+
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: install-catalog
+ namespace: {{ .Release.Namespace | quote }}
+data:
+{{ (.Files.Glob "configMapFiles/installCatalog/myTask.sh").AsConfig | indent 2
}}
diff --git a/helm/openwhisk/templates/installRouteMgmtJob.yaml
b/helm/openwhisk/templates/installRouteMgmtJob.yaml
index 51a014f..3d14ddb 100644
--- a/helm/openwhisk/templates/installRouteMgmtJob.yaml
+++ b/helm/openwhisk/templates/installRouteMgmtJob.yaml
@@ -13,18 +13,21 @@ spec:
name: install-routemgmt
spec:
restartPolicy: Never
+ volumes:
+ - name: task-dir
+ configMap:
+ name: install-routemgmt
initContainers:
# Wait for a controller to be up so we can perfom our CRUD actions with
the CLI
{{ include "readiness.waitForController" . | indent 6 }}
containers:
- name: routemgmt
- image: openwhisk/kube-routemgmt
+ image: openwhisk/kube-whisk-script-runner
+ volumeMounts:
+ - name: task-dir
+ mountPath: "/task/myTask.sh"
+ subPath: "myTask.sh"
env:
- - name: "WHISK_CLI_VERSION"
- valueFrom:
- configMapKeyRef:
- name: whisk.config
- key: whisk_cli_version_tag
- name: "WHISK_AUTH"
valueFrom:
secretKeyRef:
@@ -43,3 +46,12 @@ spec:
key: whisk_system_namespace
- name: "WHISK_API_GATEWAY_HOST_V2"
value:
"http://$(APIGATEWAY_SERVICE_HOST):$(APIGATEWAY_SERVICE_PORT_API)/v2"
+
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: install-routemgmt
+ namespace: {{ .Release.Namespace | quote }}
+data:
+{{ (.Files.Glob "configMapFiles/installRouteMgmt/myTask.sh").AsConfig | indent
2 }}
diff --git a/helm/openwhisk/templates/invoker.yaml
b/helm/openwhisk/templates/invoker.yaml
index 9a3ab11..68f5a4d 100644
--- a/helm/openwhisk/templates/invoker.yaml
+++ b/helm/openwhisk/templates/invoker.yaml
@@ -38,6 +38,9 @@ spec:
{{- if eq .Values.invoker.containerFactory.impl "docker" }}
volumes:
{{ include "docker_volumes" . | indent 6 }}
+ - name: task-dir
+ configMap:
+ name: docker-pull-runtimes
{{- end }}
initContainers:
@@ -175,6 +178,9 @@ spec:
{{ include "docker_volumes" . | indent 6 }}
- name: userlogs
emptyDir: {}
+ - name: task-dir
+ configMap:
+ name: docker-pull-runtimes
initContainers:
# Pull images for all default runtimes before starting invoker
@@ -194,3 +200,13 @@ spec:
- name: userlogs
mountPath: "/action-logs"
{{- end }}
+
+
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: docker-pull-runtimes
+ namespace: {{ .Release.Namespace | quote }}
+data:
+{{ (.Files.Glob "configMapFiles/dockerPullRuntimes/playbook.yml").AsConfig |
indent 2 }}
diff --git a/helm/openwhisk/values.yaml b/helm/openwhisk/values.yaml
index 5a71a4f..028f8b5 100644
--- a/helm/openwhisk/values.yaml
+++ b/helm/openwhisk/values.yaml
@@ -153,7 +153,7 @@ invoker:
imagePullPolicy: "IfNotPresent"
restartPolicy: "Always"
port: 8080
- pullRuntimesImage: "openwhisk/kube-docker-pull:latest"
+ pullRuntimesImage: "openwhisk/kube-whisk-ansible-runner:latest"
options: ""
jvmHeapMB: "512"
jvmOptions: ""
diff --git a/docker/kafkapkg-installer/init.sh
b/helm/providers/charts/kafka/configMapFiles/myTask.sh
similarity index 52%
rename from docker/kafkapkg-installer/init.sh
rename to helm/providers/charts/kafka/configMapFiles/myTask.sh
index 1da13ef..0d9e8c0 100644
--- a/docker/kafkapkg-installer/init.sh
+++ b/helm/providers/charts/kafka/configMapFiles/myTask.sh
@@ -1,23 +1,12 @@
-#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
# license agreements; and to You under the Apache License, Version 2.0.
-set -ex
+# installCatalog.sh expects the wsk cli to be $OPENWHISK_HOME/bin/wsk
+export OPENWHISK_HOME=/usr/local
git clone https://github.com/apache/incubator-openwhisk-package-kafka.git
-# TODO: installxxxCatalog.sh wants OPENWHISK_HOME set, but doesn't actually
need
-# it for anything. Fix upstream and then remove this.
-export OPENWHISK_HOME=/openwhisk
export DB_URL=http://$DB_USERNAME:$DB_PASSWORD@$DB_HOST:$DB_PORT
-mkdir -p $OPENWHISK_HOME/bin
-
-# Download and install openwhisk cli
-pushd $OPENWHISK_HOME/bin
- wget -q
https://github.com/apache/incubator-openwhisk-cli/releases/download/latest/OpenWhisk_CLI-latest-linux-amd64.tgz
- tar xzf OpenWhisk_CLI-latest-linux-amd64.tgz
-popd
-
pushd /incubator-openwhisk-package-kafka
./installKafka.sh $AUTH $APIHOST $DB_URL $DB_PREFIX $APIHOST
./installCatalog.sh $AUTH $APIHOST $DB_URL $DB_PREFIX $APIHOST
diff --git a/helm/providers/charts/kafka/templates/installPkgKafkaJob.yaml
b/helm/providers/charts/kafka/templates/installPkgKafkaJob.yaml
index 5a742b9..a30c887 100644
--- a/helm/providers/charts/kafka/templates/installPkgKafkaJob.yaml
+++ b/helm/providers/charts/kafka/templates/installPkgKafkaJob.yaml
@@ -13,9 +13,17 @@ spec:
name: install-package-kafka
spec:
restartPolicy: Never
+ volumes:
+ - name: task-dir
+ configMap:
+ name: install-package-kafka
containers:
- name: kafkapkginstaller
- image: openwhisk/kube-kafkapkginstaller
+ image: openwhisk/kube-whisk-script-runner
+ volumeMounts:
+ - name: task-dir
+ mountPath: "/task/myTask.sh"
+ subPath: "myTask.sh"
env:
- name: "AUTH"
valueFrom:
@@ -51,3 +59,11 @@ spec:
key: db_password
- name: "DB_PREFIX"
value: "{{- .Values.kafkaprovider.dbPrefix -}}"
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: install-package-kafka
+ namespace: {{ .Release.Namespace | quote }}
+data:
+{{ (.Files.Glob "configMapFiles/myTask.sh").AsConfig | indent 2 }}
diff --git a/tools/travis/deploy.sh b/tools/travis/deploy.sh
index 4eac1b2..1eb9436 100755
--- a/tools/travis/deploy.sh
+++ b/tools/travis/deploy.sh
@@ -9,20 +9,14 @@ ROOTDIR="$SCRIPTDIR/../../"
cd $ROOTDIR
+echo "Publishing kube-whisk-script-runner image"
+./tools/travis/publish.sh openwhisk kube-whisk-script-runner latest
docker/whisk-script-runner
+
+echo "Publishing kube-whisk-ansible-runner image"
+./tools/travis/publish.sh openwhisk kube-whisk-ansible-runner latest
docker/whisk-ansible-runner
+
echo "Publishing kube-couchdb image"
./tools/travis/publish.sh openwhisk kube-couchdb latest docker/couchdb
-echo "Publishing kube-docker-pull image"
-./tools/travis/publish.sh openwhisk kube-docker-pull latest docker/docker-pull
-
echo "Publishing kube-invoker-agent image"
./tools/travis/publish.sh openwhisk kube-invoker-agent latest
docker/invoker-agent
-
-echo "Publishing kube-openwhisk-catalog image"
-./tools/travis/publish.sh openwhisk kube-openwhisk-catalog latest
docker/openwhisk-catalog
-
-echo "Publishing kube-routemgmt image"
-./tools/travis/publish.sh openwhisk kube-routemgmt latest docker/routemgmt
-
-echo "Publishing kube-kafkapkginstaller image"
-./tools/travis/publish.sh openwhisk kube-kafkapkginstaller latest
docker/kafkapkg-installer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
