This is an automated email from the ASF dual-hosted git repository.
csantanapr pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk.git
The following commit(s) were added to refs/heads/master by this push:
new 06892e6 Remove unsupported distributed playbooks. (#4245)
06892e6 is described below
commit 06892e6f79300f86670cad1b5a5e2eb75b567f99
Author: rodric rabbah <[email protected]>
AuthorDate: Wed Feb 6 16:58:04 2019 -0500
Remove unsupported distributed playbooks. (#4245)
---
ansible/README_DISTRIBUTED.md | 102 -----------------
ansible/boot_instances_dist.yml | 71 ------------
.../distributed/files/openstack/README_OS.md | 41 -------
.../distributed/files/openstack/openstack.env | 20 ----
ansible/environments/distributed/group_vars/all | 82 --------------
ansible/environments/distributed/hosts | 32 ------
ansible/prereq_build.yml | 21 ----
ansible/provision_env_dist.yml | 66 -----------
ansible/registry.yml | 22 ----
ansible/roles/couchdb/tasks/deploy.yml | 12 --
ansible/roles/docker/tasks/clean.yml | 11 --
ansible/roles/docker/tasks/deploy.yml | 37 ------
ansible/roles/docker/tasks/main.yml | 12 --
ansible/roles/docker/templates/docker.j2 | 13 ---
ansible/roles/registry/tasks/clean.yml | 14 ---
ansible/roles/registry/tasks/deploy.yml | 70 ------------
ansible/roles/registry/tasks/gencert.yml | 7 --
ansible/roles/registry/tasks/main.yml | 13 ---
ansible/roles/registry/tasks/publish.yml | 28 -----
.../registry/templates/recycle-registry.py.j2 | 126 ---------------------
ansible/roles/whiskhost/tasks/addUser.yml | 27 -----
ansible/roles/whiskhost/tasks/clean.yml | 7 --
ansible/roles/whiskhost/tasks/deploy.yml | 9 --
ansible/roles/whiskhost/tasks/main.yml | 12 --
ansible/roles/whiskhost/tasks/patchKernel.yml | 43 -------
tools/ubuntu-setup/README.md | 6 +-
26 files changed, 3 insertions(+), 901 deletions(-)
diff --git a/ansible/README_DISTRIBUTED.md b/ansible/README_DISTRIBUTED.md
deleted file mode 100644
index 3b859ad..0000000
--- a/ansible/README_DISTRIBUTED.md
+++ /dev/null
@@ -1,102 +0,0 @@
-<!--
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
--->
-
-Deploying Distributed OpenWhisk using Ansible
-=========
-
-**Caveat:** All Ansible commands are meant to be executed from the `ansible`
directory.
-This is important because that's where `ansible.cfg` is located which contains
generic settings that are needed for the remaining steps.
-
-By default, if you omit the `-i` parameter in Ansible commands, the `local`
environment will be used.
-
-In all instructions, replace `<openwhisk_home>` with the base directory of
your OpenWhisk source tree. e.g., `openwhisk`.
-
-
-#### Setup and provision OpenWhisk component VMs
-A set of Ubuntu 14.04 machines will need to be provisioned in the targeted
IaaS (Infrastructure as a Service platform). These VMs will need to provisioned
manually in most IaaS providers, but we have added some scripts to automate
VM/disk provisioning against Openstack CPIs. These scripts are not being
actively maintained at the moment, but PRs to enhance the scripts and add
support for other IaaS offerings (AWS, GCE, etc) are certainly encouraged. Once
the VMs are up and reachable by th [...]
-
-If using Openstack, please follow the README at
[environments/distributed/files/openstack/README_OS.md](environments/distributed/files/openstack/README_OS.md)
to manage the required VMs. Otherwise, provision each VM manually.
-
-Login to your bootstrapper VM. Your local machine can act as the bootstrapping
machine as well, as long as it can connect to the VMs deployed in your IaaS.
-
-Add the remote_user and private_key_file values to the defaults section of the
`ansible.cfg` file. The remote_user value sets the default ssh user. The
private_key_file is required when using a private key that is not in the
default `~/.ssh` folder.
-
-```
-[defaults]
-remote_user = ubuntu
-private_key_file=/path/to/file.pem
-```
-
-Ensure that the Ansible VM can authenticate to the OpenWhisk VMs via SSH using
the following command.
-
-```
-ansible all -i environments/distributed -m ping
-```
-
-Generate config files
-
-```
-ansible-playbook -i environments/distributed setup.yml
-```
-
-Install prerequisites on OpenWhisk nodes.
-
-```
-ansible-playbook -i environments/distributed prereq_build.yml
-```
-
-#### Build and deploy OpenWhisk
-
-Deploy registry.
-
-```
-ansible-playbook -i environments/distributed registry.yml
-```
-
-Build and distribute OpenWhisk docker images. Must be executed with root
privileges. The IP for the registry VM can be found in the
[hosts](environments/distributed/hosts) file.
-
-```
-cd ..
-./gradlew distDocker -PdockerHost=<registry_vm_ip>:4243
-PdockerRegistry=<registry_vm_ip>:5000
-```
-Now run the following steps. These are equivalent to a [single VM](README.md)
deployment.
-
-Deploy CouchDB and configure OpenWhisk deployment.
-
-```
-cd ansible
-ansible-playbook -i environments/<environment> couchdb.yml
-ansible-playbook -i environments/<environment> initdb.yml
-ansible-playbook -i environments/<environment> wipe.yml
-ansible-playbook -i environments/<environment> openwhisk.yml
-
-# installs a catalog of public packages and actions
-ansible-playbook -i environments/<environment> postdeploy.yml
-
-# to use the API gateway
-ansible-playbook -i environments/<environment> apigateway.yml
-ansible-playbook -i environments/<environment> routemgmt.yml
-```
-
-Setup your CLI and verify that OpenWhisk is working.
-
-```
-../bin/wsk property set --auth $(cat files/auth.whisk.system) --apihost
<edge_url>
-../bin/wsk -i -v action invoke /whisk.system/samples/helloWorld --blocking
--result
-```
diff --git a/ansible/boot_instances_dist.yml b/ansible/boot_instances_dist.yml
deleted file mode 100644
index c8cb643..0000000
--- a/ansible/boot_instances_dist.yml
+++ /dev/null
@@ -1,71 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# boot_instances.yml
-# TODO, in "with_instance" loops, the "instance" variable that is passed in
gets unset, look up the proper way to do nested loops
-- name: set vars
- set_fact:
- name: "{{instance.name}}"
- flavor: "{{instance.flavor | default (default_flavor, true) }}"
- volume_env: "{{ lookup('env', 'OS_WSK_' + instance.name|upper +
'_VOLUME')}}"
- volume_dict: "{{instance.volume | default({})}}"
- # instance: "{{instances | selectattr('name', 'equalto', 'db') | list |
first}}"
-
-- name: create volume
- os_volume:
- state: present
- size: "{{volume_env | default( volume_dict.size )}}"
- display_name: "{{name}}-{{item}}_volume"
- wait: yes
- with_sequence: count={{instance.num_instances}}
- when: (volume_env and volume_env != "") or (volume_dict and volume_dict.size)
-
-- name: launch an instance
- os_server:
- validate_certs: False
- name: "{{name}}-{{item}}"
- state: present
- auth: "{{auth}}"
- nics:
- - net-id: "{{net_id}}"
- #- net-name: "{{net_name}}"
- image: "{{image}}"
- flavor: "{{item.flavor | default (flavor) }}"
- key_name: "{{key_name}}"
- security_groups: "{{security_groups}}"
- auto_ip: no
- wait: yes
- register: instance_info
- with_sequence: count={{instance.num_instances}}
-
-- name: attach volume to host
- os_server_volume:
- server: "{{name}}-{{item}}"
- state: present
- volume: "{{name}}-{{item}}_volume"
- wait: yes
- # register: volume_info
- with_sequence: count={{instance.num_instances}}
- when: (volume_env and volume_env != "") or (volume_dict and volume_dict.size)
-
-# - add_host: ansible_ssh_host={{ item.server.private_v4 }} groups={{name}}
-# with_items: instance_info.results
-
-# - local_action: add_host ansible_ssh_host={{ item.server.private_v4 }}
groupname={{name}}
-# with_items: instance_info.results
-
-- shell: "grep -e {{name}} {{hosts_dir}}/hosts"
- ignore_errors: True
- register: hosts_file
-
-# TODO, should render the hosts file from a template by using the add_host
module commented below with a template
-- shell: "echo [{{name}}] >> {{hosts_dir}}/hosts"
- when: "{{hosts_file.rc}} != 0"
-
-- lineinfile: insertafter="^{{name}}"
line="{{item.server.networks.itervalues().next()[0]}}
block_device={{item.server.volumes[0].device }}" dest="{{hosts_dir}}/hosts"
- with_items: instance_info.results
- when: (volume_env and volume_env != "") or (volume_dict and volume_dict.size)
-
-- lineinfile: insertafter="^{{name}}"
line="{{item.server.networks.itervalues().next()[0]}}"
dest="{{hosts_dir}}/hosts"
- with_items: instance_info.results
- when: (not volume_env or volume_env == "" ) and (not volume_dict or not
volume_dict.size)
diff --git a/ansible/environments/distributed/files/openstack/README_OS.md
b/ansible/environments/distributed/files/openstack/README_OS.md
deleted file mode 100644
index 39feb2b..0000000
--- a/ansible/environments/distributed/files/openstack/README_OS.md
+++ /dev/null
@@ -1,41 +0,0 @@
-<!--
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
--->
-#### Distributed Deployment using OpenStack as IaaS
-
-To communicate with the Openstack APIs, the bootstapper will need a few
additional dependencies to be installed.
-
-Install prerequisites
-```
-sudo apt-get -y install python-setuptools python-dev libssl-dev
build-essential libssl-dev libffi-dev python-dev python-novaclient
-sudo pip install shade pytz positional appdirs monotonic rfc3986 pyparsing
stevedore debtcollector netaddr oslo.config futures warlock six
-```
-
-Populate the [OpenStack .env file](openstack.env) with valid
credentials/endpoint. Please note that OS_WSK_DB_VOLUME is optional. If not
specified, local disk will be used instead of persistent disk for CouchDB.
-
-By default, 2 invokers are created. To adjust this value, simply change the
num_instances value in the
[environments/distributed/group_vars/all](environments/distributed/group_vars/all:67)
file.
-
-Once the group_vars and .env files have been populated, run the following
playbook to boot instances and generate the respective hosts file.
-```
-ansible-playbook -i environments/distributed provision_env_dist.yml
-```
-
-Ensure the VMs are up and that the bootstrapper can reach them
-```
-ansible all -i environments/distributed -m ping
-```
diff --git a/ansible/environments/distributed/files/openstack/openstack.env
b/ansible/environments/distributed/files/openstack/openstack.env
deleted file mode 100644
index 0b3637e..0000000
--- a/ansible/environments/distributed/files/openstack/openstack.env
+++ /dev/null
@@ -1,20 +0,0 @@
-export OS_FLAVOR=m1.medium
-export OS_IMAGE=Ubuntu14.04-1Nic
-export OS_KEY_NAME=key_name
-export OS_NET_NAME=network_name
-export OS_NET_ID=e489dcf2-4601-4809-a459-e3821a95d23a
-export OS_USERNAME=abcxyz
-export OS_PASSWORD=*******
-export OS_PROJECT_NAME=OpenWhisk
-export OS_SECURITY_GROUPS=sec_group
-export OS_WSK_DB_VOLUME=15
-
-## Keystone v2
-export OS_AUTH_URL=https://OpenStack_URL:5000/v2.0
-export OS_TENANT_NAME="OpenWhisk"
-export OS_TENANT_ID=a9e6a61ab914455cb4329592d5733325
-
-## Keystone v3
-export OS_AUTH_URL=https://OpenStack_URL:5000/v3
-export OS_PROJECT_ID=a9e6a61ab914455cb4329592d5733325
-export OS_USER_DOMAIN_NAME="domain"
diff --git a/ansible/environments/distributed/group_vars/all
b/ansible/environments/distributed/group_vars/all
deleted file mode 100755
index 2056dec..0000000
--- a/ansible/environments/distributed/group_vars/all
+++ /dev/null
@@ -1,82 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
-
----
-
-environment_type: "distributed"
-
-db_provider: CouchDB
-db_port: 5984
-db_protocol: http
-db_username: couch_user
-db_password: couch_password
-db_host: "{{ groups['db'] | first }}"
-db_prefix: whisk_distributed_
-
-config_root_dir: /tmp/wskconf
-whisk_logs_dir: /tmp/wsklogs
-registry_storage_dir: "/"
-docker_registry: "{{ groups['registry'] | first }}:5000/"
-docker_dns: ""
-
-env_hosts_dir: "{{ playbook_dir }}/environments/distributed"
-
-registry:
- version: 2.3.1
- port: 5000
- gencert: false
- recycle:
- threshold: 80
- keeptags: 5
-
-# openstack vars
-# TODO, should move these to a role and create a designated "defaults.yml" file
-rc_file:
-#net_name: "{{ lookup('env','OS_NET_NAME') }}"
-net_id: "{{ lookup('env','OS_NET_ID') }}"
-image: "{{ lookup('env','OS_IMAGE') }}"
-default_flavor: "{{ lookup('env','OS_FLAVOR') | default('m1.small') }}"
-auth:
- auth_url: "{{ lookup('env','OS_AUTH_URL') }}"
- username: "{{ lookup('env','OS_USERNAME') }}"
- password: "{{ lookup('env','OS_PASSWORD') }}"
- project_name: "{{ lookup('env','OS_PROJECT_NAME') }}"
-key_name: "{{ lookup('env','OS_KEY_NAME') }}"
-security_groups: "{{ lookup('env','OS_SECURITY_GROUPS') }}"
-
-# list of instances.
-# TODO, move each object to designated group_vars file
-instances:
- - name: registry
- num_instances: 1
- flavor:
-
- - name: edge
- num_instances: 1
- flavor:
-
- - name: controllers
- num_instances: 1
- flavor:
-
- - name: kafka
- num_instances: 1
- flavor:
-
- - name: invokers
- num_instances: 2
- flavor:
-
- - name: db
- num_instances: 1
- flavor:
- volume:
- name:
- size:
- fstype:
- fsmount:
-
-# API GW connection configuration
-apigw_auth_user: ""
-apigw_auth_pwd: ""
-apigw_host_v2: "http://{{ groups['apigateway']|first
}}:{{apigateway.port.api}}/v2"
diff --git a/ansible/environments/distributed/hosts
b/ansible/environments/distributed/hosts
deleted file mode 100755
index 9c9045b..0000000
--- a/ansible/environments/distributed/hosts
+++ /dev/null
@@ -1,32 +0,0 @@
-; the first parameter in a host is the inventory_hostname
-
-; used for local actions only
-ansible ansible_connection=local
-
-[registry]
-10.3.2.153 ansible_host=10.3.2.153
-
-[edge]
-10.3.2.154 ansible_host=10.3.2.154
-
-[apigateway:children]
-edge
-
-[redis:children]
-edge
-
-[controllers]
-10.3.2.155 ansible_host=10.3.2.155
-
-[kafkas]
-10.3.2.156 ansible_host=10.3.2.156
-
-[zookeepers:children]
-kafkas
-
-[invokers]
-10.3.2.158 ansible_host=10.3.2.158
-10.3.2.159 ansible_host=10.3.2.159
-
-[db]
-10.3.2.16 ansible_host=10.3.2.16
diff --git a/ansible/prereq_build.yml b/ansible/prereq_build.yml
deleted file mode 100644
index c5de2b7..0000000
--- a/ansible/prereq_build.yml
+++ /dev/null
@@ -1,21 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This playbook prepares all target hosts for Openwhisk installation.
-# It will create the docker user and establish ssh keys.
-# It will install all necessary packages to run Openwhisk playbooks.
-# In addition it will also install docker on the target hosts
-
-# If the docker user and ansible ssh user differ, use the following
-# Set -e ansible_ssh_pass=xxx -e ansible_become_pass=xxx -e ansible_user=yyy
when executing this playbook.
-# xxx == passwd of a user that can become root or root passwd
-# yyy == user that can become root or root itself
-# Note that the user and passwd needs to be the same on all systems.
-# Also you need to have sshpass installed.
-# (apt-get for ubuntu, for mac: brew install
https://raw.githubusercontent.com/kadwanev/bigboybrew/master/Library/Formula/sshpass.rb)
-
-- hosts: all:!ansible
- roles:
- - whiskhost
- - docker
- - prereq
diff --git a/ansible/provision_env_dist.yml b/ansible/provision_env_dist.yml
deleted file mode 100644
index 1ec3229..0000000
--- a/ansible/provision_env_dist.yml
+++ /dev/null
@@ -1,66 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# provision_env.yml
-- name: create instances and hosts file
- hosts: ansible
- tasks:
- - name: add header to hosts file
- copy:
- content: "; the first parameter in a host is the
inventory_hostname\n\nansible ansible_connection=local\n"
- dest: "{{hosts_dir}}/hosts"
-
- - name: Create instances
- import_tasks: boot_instances_dist.yml instance={{item}}
- with_items: instances
-
- - meta: refresh_inventory
-
-# TODO, add support to loop through instances, configure all vms that have
cinder disks (kafka, elk, etc). Hardcoding to db for now
-- name: Configure couch db disks
- hosts: db
- tasks:
- - name: Ensure VM is up and running before starting disk configuration
- wait_for: host={{ansible_default_ipv4.address}} port=22
- - name: Get list of configured disks
- shell: df -h
- register: disks
- - name: Check to see if volume size has been set via env or group_vars
- set_fact:
- volume_env: "{{lookup('env', 'OS_WSK_DB_VOLUME')}}"
- instance: "{{instances | selectattr('name', 'equalto', 'db') | list |
first}}"
- - name: Set boolean flag to determine if volume is to be configured or
not. Skip if volume value is not set in both env and group_vars, or if it has
already been configured
- set_fact:
- configure_volume: "{{(volume_env is defined and volume_env != '') or
- ((instance.volume) and (instance.volume.fsmount
and instance.volume.fsmount not in disks.stdout))}}"
- - name: Get fsmount/fstype params
- set_fact:
- fsmount: "{{ instance.volume.fsmount | default('/mnt/' +
group_names|first, true ) }}"
- fstype: "{{ instance.volume.fstype | default('ext4', true) }}"
- - name: Create mount folder
- shell: mkdir -p {{ fsmount }}
- become: true
- when: configure_volume and (fsmount not in disks.stdout)
- - name: Partition persistent disk
- shell: mkfs.{{ fstype }} {{ block_device }} 2>/dev/null
- become: true
- when: configure_volume and (fsmount not in disks.stdout)
- - name: Mount persistent disk
- shell: mount -t {{ fstype }} {{ block_device }} {{ fsmount }}
- # - mount: name={{fsmount}} src={{device}} fstype={{fstype}}
state=present # TODO, mount module not working as expected
- become: true
- when: configure_volume and (fsmount not in disks.stdout)
-
-- name: append registry url to hosts file
- hosts: all:!ansible
- tasks:
- - lineinfile: line="{{ groups['registry'] | first }} whisk-registry"
dest=/etc/hosts
- become: true
-
-- name: Install git on registry
- hosts: registry
- tasks:
- - apt: update_cache=yes
- become: true
- - package: name=git
- become: true
diff --git a/ansible/registry.yml b/ansible/registry.yml
deleted file mode 100644
index d44750a..0000000
--- a/ansible/registry.yml
+++ /dev/null
@@ -1,22 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This playbook deploys a docker registry using a self-signed cert.
-
-# Generate cert if needed. Not used currently.
-- hosts: ansible
- tasks:
- - import_tasks: roles/registry/tasks/gencert.yml
- when: mode == "deploy" and registry.gencert == true
-
-# Deploy registry
-- hosts: registry
- roles:
- - registry
-
-# publish cert on all hosts
-- hosts: all:!ansible
- become: true
- tasks:
- - import_tasks: roles/registry/tasks/publish.yml
- when: mode == "deploy" and registry.gencert
diff --git a/ansible/roles/couchdb/tasks/deploy.yml
b/ansible/roles/couchdb/tasks/deploy.yml
index b71156b..8cad011 100644
--- a/ansible/roles/couchdb/tasks/deploy.yml
+++ b/ansible/roles/couchdb/tasks/deploy.yml
@@ -15,18 +15,6 @@
fail: msg="The db provider in your {{ hosts_dir }}/group_vars/all is {{
db.provider }}, it has to be CouchDB, pls double check"
when: db.provider != "CouchDB"
-- name: check for persistent disk
- shell: df -h
- register: disk_status
- when: block_device is defined
-
-- name: "set the volume_dir"
- vars:
- instance: "{{instances | selectattr('name', 'equalto', 'db') | list |
first}}"
- set_fact:
- volumes: "{{ volumes }} + [ '{{ instance.volume.fsmount | default( '/mnt/'
+ group_names|first, true ) }}:/usr/local/var/lib/couchdb' ]"
- when: (block_device is defined) and (block_device in disk_status.stdout)
-
- include_tasks: gen_erl_cookie.yml
when: (db.instances|int >= 2)
diff --git a/ansible/roles/docker/tasks/clean.yml
b/ansible/roles/docker/tasks/clean.yml
deleted file mode 100644
index b34086d..0000000
--- a/ansible/roles/docker/tasks/clean.yml
+++ /dev/null
@@ -1,11 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# Remove docker
-
-- name: install docker
- apt:
- name: docker-engine
- state: absent
- force: yes
- become: true
diff --git a/ansible/roles/docker/tasks/deploy.yml
b/ansible/roles/docker/tasks/deploy.yml
deleted file mode 100644
index c731974..0000000
--- a/ansible/roles/docker/tasks/deploy.yml
+++ /dev/null
@@ -1,37 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This role will install Docker using apt.
-
-- name: add docker repo
- apt_repository:
- repo: deb https://apt.dockerproject.org/repo ubuntu-trusty main
- update_cache: yes
- state: present
- become: true
-
-- name: install docker
- apt:
- name: "docker-engine={{ docker.version }}"
- state: present
- force: yes
- become: true
-
-- name: configure docker
- template:
- src: docker.j2
- dest: "/etc/default/docker"
- become: true
-
-- name: add user to docker group
- user:
- name: "{{docker.user|default(ansible_user_id)}}"
- groups: docker
- append: true
- become: true
-
-- name: restart docker service
- service:
- name: docker
- state: restarted
- become: true
diff --git a/ansible/roles/docker/tasks/main.yml
b/ansible/roles/docker/tasks/main.yml
deleted file mode 100644
index 1878ef1..0000000
--- a/ansible/roles/docker/tasks/main.yml
+++ /dev/null
@@ -1,12 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This role will install docker on target VMs. This should only be run on
ubuntu targets.
-# In deploy mode it will install docker using apt.
-# In clean mode it will uninstall docker.
-
-- import_tasks: deploy.yml
- when: mode == "deploy"
-
-- import_tasks: clean.yml
- when: mode == "clean"
diff --git a/ansible/roles/docker/templates/docker.j2
b/ansible/roles/docker/templates/docker.j2
deleted file mode 100644
index 4498b81..0000000
--- a/ansible/roles/docker/templates/docker.j2
+++ /dev/null
@@ -1,13 +0,0 @@
-# Docker Upstart and SysVinit configuration file.
-
-# Customize location of Docker binary (especially for development testing).
-#DOCKER="/usr/local/bin/docker"
-
-# Use DOCKER_OPTS to modify the daemon startup options.
-DOCKER_OPTS="-H tcp://0.0.0.0:{{ docker.port }} -H unix:///var/run/docker.sock
--storage-driver={{ docker.storagedriver }} --insecure-registry {{
groups['registry']|first }}:5000"
-
-# If you need Docker to use an HTTP proxy, it can also be specified here.
-#export http_proxy="http://127.0.0.1:3128/";
-
-# This is also a handy place to tweak where Docker's temporary files go.
-#export TMPDIR="/mnt/bigdrive/docker-tmp"
\ No newline at end of file
diff --git a/ansible/roles/registry/tasks/clean.yml
b/ansible/roles/registry/tasks/clean.yml
deleted file mode 100644
index d4270ec..0000000
--- a/ansible/roles/registry/tasks/clean.yml
+++ /dev/null
@@ -1,14 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# Remove registry and volumes.
-
-- name: remove registry and volumes
- shell: "docker rm -fv whisk_docker_registry"
- ignore_errors: True
-
-- name: remove cron job for registry recycle
- file:
- path: "/etc/cron.daily/recycle-registry.py"
- state: absent
- become: true
diff --git a/ansible/roles/registry/tasks/deploy.yml
b/ansible/roles/registry/tasks/deploy.yml
deleted file mode 100644
index ca40964..0000000
--- a/ansible/roles/registry/tasks/deploy.yml
+++ /dev/null
@@ -1,70 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This role will install a docker registry. If the registry existed, it will
try to recycle its images.
-
-- name: "pull the {{ registry.version }} image of registry"
- shell: "docker pull registry:{{ registry.version }}"
- register: result
- until: (result.rc == 0)
- retries: "{{ docker.pull.retries }}"
- delay: "{{ docker.pull.delay }}"
-
-- name: "create {{ registry.confdir }} on registry {{ ansible_host }}"
- file:
- path: "{{ registry.confdir }}/certs"
- state: directory
- when: registry.gencert
-
-- name: "copy cert to registry"
- copy:
- src: "files/whisk_docker_registry.crt"
- dest: "{{ registry.confdir }}/certs"
- when: registry.gencert
-
-- name: "copy key to registry"
- copy:
- src: "files/whisk_docker_registry.key"
- dest: "{{ registry.confdir }}/certs"
- when: registry.gencert
-
-- name: (re)start registry with ssl enabled
- docker_container:
- name: whisk_docker_registry
- image: "registry:{{ registry.version }}"
- state: started
- restart_policy: "{{ docker.restart.policy }}"
- hostname: registry
- env:
- "REGISTRY_HTTP_TLS_CERTIFICATE": "/certs/whisk_docker_registry.crt"
- "REGISTRY_HTTP_TLS_KEY": "/certs/whisk_docker_registry.key"
- ports:
- - "{{ registry.port }}:5000"
- volumes:
- - "{{ registry.confdir }}/certs:/certs"
- when: registry.gencert
-
-- name: (re)start registry with insecure ssl
- docker_container:
- name: whisk_docker_registry
- image: "registry:{{ registry.version }}"
- state: started
- restart_policy: "{{ docker.restart.policy }}"
- hostname: registry
- ports:
- - "{{ registry.port }}:5000"
- when: not registry.gencert
-
-- name: wait until registry in this host is up and running
- wait_for:
- delay: 2
- host: "{{ ansible_host }}"
- port: "{{ registry.port }}"
- timeout: 60
-
-- name: make cron job for registry recycle
- template:
- src: recycle-registry.py.j2
- dest: "/etc/cron.daily/recycle-registry"
- mode: 0755
- become: true
diff --git a/ansible/roles/registry/tasks/gencert.yml
b/ansible/roles/registry/tasks/gencert.yml
deleted file mode 100644
index c989101..0000000
--- a/ansible/roles/registry/tasks/gencert.yml
+++ /dev/null
@@ -1,7 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# Generate a new self-signed certificate for the registry. To be used later.
-
-- name: generate new cert
- shell: "openssl req -nodes -x509 -newkey rsa:2048 -keyout
files/whisk_docker_registry.key -out files/whisk_docker_registry.crt -days 356
-subj '/C=US/ST=NY/L={{location | default('New York')}}/O=WHISK/CN={{
groups['registry']|first }}'"
diff --git a/ansible/roles/registry/tasks/main.yml
b/ansible/roles/registry/tasks/main.yml
deleted file mode 100644
index e4e611e..0000000
--- a/ansible/roles/registry/tasks/main.yml
+++ /dev/null
@@ -1,13 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This role will install a docker registry using self-signed certs.
-# In deploy mode it will deploy the registry. If it pre-exists, it will be
recycled.
-# Recycling means retaining the last few images of all repos and importing
them into the new instance.
-# In clean mode it will remove the registry, deleting all images.
-
-- import_tasks: deploy.yml
- when: mode == "deploy"
-
-- import_tasks: clean.yml
- when: mode == "clean"
diff --git a/ansible/roles/registry/tasks/publish.yml
b/ansible/roles/registry/tasks/publish.yml
deleted file mode 100644
index 8087f20..0000000
--- a/ansible/roles/registry/tasks/publish.yml
+++ /dev/null
@@ -1,28 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# Publish certificate to all hosts. Also add registry to /etc/hosts if no fqdn
provided.
-
-- name: "make registry dir in /etc/docker/certs.d/"
- file:
- path: "/etc/docker/certs.d/{{ groups['registry']|first }}:{{ registry.port
}}"
- state: directory
-
-- name: "copy cert to registry dir"
- copy:
- src: "files/whisk_docker_registry.crt"
- dest: "/etc/docker/certs.d/{{ groups['registry']|first }}:{{ registry.port
}}"
-
-- name: "remove old entry from /etc/hosts if needed"
- lineinfile:
- dest: /etc/hosts
- regexp: "{{ groups['registry']|first }}"
- state: absent
- when: hostvars[groups['registry']|first]['ansible_host'] is defined
-
-- name: "add registry to /etc/hosts if needed"
- lineinfile:
- dest: /etc/hosts
- line: "{{ hostvars[groups['registry']|first]['ansible_host'] }} {{
groups['registry']|first }}"
- state: present
- when: hostvars[groups['registry']|first]['ansible_host'] is defined
diff --git a/ansible/roles/registry/templates/recycle-registry.py.j2
b/ansible/roles/registry/templates/recycle-registry.py.j2
deleted file mode 100644
index b7f3c18..0000000
--- a/ansible/roles/registry/templates/recycle-registry.py.j2
+++ /dev/null
@@ -1,126 +0,0 @@
-#!/usr/bin/env python
-#
-# Maintenance script that is supposed to run as a cron-daily job.
-# Makes sure the registry never uses more than a certain percentage of local
disk space.
-# Tries to keep the last versions of every image when recycling.
-#
-
-import sys
-import os
-import json
-import subprocess
-import re
-import time
-import syslog
-
-REGISTRY = "{{ groups['registry']|first }}:{{ registry.port }}"
-REGISTRY_URL = "https://%s"; % REGISTRY
-REGISTRY_NAME = "whisk_docker_registry"
-REGISTRY_VERSION = "registry:{{ registry.version }}"
-THRESHOLD = {{ registry.recycle.threshold }}
-STORAGE = "{{ registry_storage_dir }}"
-TAGS_TO_KEEP = {{ registry.recycle.keeptags }}
-
-def getDiskUsage(dir):
- cmd=['df', '--output=pcent', dir]
- output = subprocess.check_output(cmd)
- pattern = "[0-9]+"
- result = re.search(pattern, output).group()
- return result
-
-def getImageRepos():
- syslog.syslog("Getting image repositories from URL %s" % REGISTRY_URL)
- cmd=['curl', '-s', '-k', REGISTRY_URL + "/v2/_catalog"]
- output = subprocess.check_output(cmd)
- repos = json.loads(output)["repositories"]
- return repos
-
-def getImageTags(repo):
- syslog.syslog("Getting tags for repo %s" % repo)
- cmd=['curl', '-s', '-k', REGISTRY_URL + "/v2/" + repo + "/tags/list"]
- output = subprocess.check_output(cmd)
- tags = json.loads(output)["tags"]
- tags_desc = sorted(tags, reverse=True)
- return tags_desc
-
-def pullImage(repo, tag):
- syslog.syslog("Pulling image %s:%s" % (repo,tag))
- cmd=['docker', 'pull', REGISTRY + "/" + repo + ":" + tag]
- output = subprocess.check_output(cmd)
- return output
-
-def pushImage(repo, tag):
- syslog.syslog("Pushing image %s:%s" % (repo,tag))
- cmd=['docker', 'push', REGISTRY + "/" + repo + ":" + tag]
- output = subprocess.check_output(cmd)
- return output
-
-def deleteImage(repo, tag):
- syslog.syslog("Deleting image %s:%s" % (repo,tag))
- cmd=['docker', 'rmi', '-f', REGISTRY + "/" + repo + ":" + tag]
- output = subprocess.check_output(cmd)
- return output
-
-def killRegistry():
- syslog.syslog("Killing registry %s" % REGISTRY_NAME)
- cmd=['docker', 'rm', '-f', '-v', REGISTRY_NAME]
- output = subprocess.check_output(cmd)
- return output
-
-def startRegistry():
- syslog.syslog("Starting registry %s" % REGISTRY_NAME)
- cmd=['docker', 'run', '-d', '-p', '{{ registry.port }}:5000', '--name',
REGISTRY_NAME, '--restart={{ docker.restart.policy }}', '-v', '{{
registry.confdir }}/certs:/certs', '-e',
- 'REGISTRY_HTTP_TLS_CERTIFICATE=/certs/whisk_docker_registry.crt', '-e',
'REGISTRY_HTTP_TLS_KEY=/certs/whisk_docker_registry.key', REGISTRY_VERSION]
- output = subprocess.check_output(cmd)
- return output
-
-############################################
-# BEGIN MAIN PROGRAM
-############################################
-
-syslog.syslog("BEGIN REGISTRY CHECK")
-
-# Step 1. Get disk space usage
-USAGE=getDiskUsage(STORAGE)
-syslog.syslog("Usage on %s is %s%%" % (STORAGE, USAGE))
-
-if int(USAGE) < THRESHOLD:
- syslog.syslog("Usage below %s%%. Exiting." % THRESHOLD)
- syslog.syslog("REGISTRY CHECK END")
- exit(0)
-
-syslog.syslog("Usage at least %s%%. Cleaning registry." % THRESHOLD)
-REPO_TAG = {}
-
-# Step 2. Get all image repos
-REPOSITORIES=getImageRepos()
-
-# Step 3. For each repo get tags
-for repo in REPOSITORIES:
- REPO_TAG[repo] = []
- TAGS = getImageTags(repo)
- # Step 4. Pull last TAGS_TO_KEEP tags
- for i in range(TAGS_TO_KEEP):
- if len(TAGS) > i:
- tag = TAGS[i]
- pullImage(repo, tag)
- REPO_TAG[repo].append(tag)
-
-# Step 5. Recycle registry
-killRegistry()
-startRegistry()
-
-syslog.syslog("Waiting a couple of seconds for the registry to come up...")
-time.sleep(5)
-
-# Step 6. Push saved images back to new registry
-for repo in REPOSITORIES:
- for tag in REPO_TAG[repo]:
- pushImage(repo, tag)
-
-# Step 7. Delete pushed images locally
-for repo in REPOSITORIES:
- for tag in REPO_TAG[repo]:
- deleteImage(repo, tag)
-
-syslog.syslog("REGISTRY CHECK END")
diff --git a/ansible/roles/whiskhost/tasks/addUser.yml
b/ansible/roles/whiskhost/tasks/addUser.yml
deleted file mode 100644
index f72cec3..0000000
--- a/ansible/roles/whiskhost/tasks/addUser.yml
+++ /dev/null
@@ -1,27 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# Add docker user to the target host.
-# Set -e ansible_ssh_pass=xxx -e ansible_become_pass=xxx -e ansible_user=yyy
when executing this playbook.
-# Also you need to have sshpass installed.
-# (apt-get for ubuntu, for mac: brew install
https://raw.githubusercontent.com/kadwanev/bigboybrew/master/Library/Formula/sshpass.rb)
-
-- name: add {{docker.user}} group
- group: name={{docker.user}}
- become: true
-
-- name: add {{docker.user}} user
- user: name={{docker.user}} shell=/bin/bash generate_ssh_key=yes
group={{docker.user}} groups=sudo append=yes
- become: true
-
-- name: remove user passwd
- shell: passwd --delete {{docker.user}}
- become: true
-
-- name: make {{docker.user}} sudoer
- shell: echo "{{docker.user}} ALL=(ALL) NOPASSWD:ALL" >
/etc/sudoers.d/{{docker.user}}
- become: true
-
-- name: add {{docker.user}} ssh key to authorized_keys
- authorized_key: user={{docker.user}} key="{{ lookup('file', '{{ creds_home
}}/ssh/id_rsa_{{docker.user}}.pub') }}"
- become: true
diff --git a/ansible/roles/whiskhost/tasks/clean.yml
b/ansible/roles/whiskhost/tasks/clean.yml
deleted file mode 100644
index defffa8..0000000
--- a/ansible/roles/whiskhost/tasks/clean.yml
+++ /dev/null
@@ -1,7 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# Do nothing.
-
-- name: do nothing
- command: "true"
diff --git a/ansible/roles/whiskhost/tasks/deploy.yml
b/ansible/roles/whiskhost/tasks/deploy.yml
deleted file mode 100644
index c921a43..0000000
--- a/ansible/roles/whiskhost/tasks/deploy.yml
+++ /dev/null
@@ -1,9 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# Add user and patch kernel.
-
-- import_tasks: addUser.yml
- when: docker.user is defined
-
-- import_tasks: patchKernel.yml
diff --git a/ansible/roles/whiskhost/tasks/main.yml
b/ansible/roles/whiskhost/tasks/main.yml
deleted file mode 100644
index 23bbefa..0000000
--- a/ansible/roles/whiskhost/tasks/main.yml
+++ /dev/null
@@ -1,12 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This role will prepare target VMs with the right kernel and other system
settings. This should only be run on ubuntu targets.
-# In deploy mode it will patch the system with the right kernel.
-# In clean mode it will do nothing atm.
-
-- import_tasks: deploy.yml
- when: mode == "deploy"
-
-- import_tasks: clean.yml
- when: mode == "clean"
diff --git a/ansible/roles/whiskhost/tasks/patchKernel.yml
b/ansible/roles/whiskhost/tasks/patchKernel.yml
deleted file mode 100644
index e40c2b8..0000000
--- a/ansible/roles/whiskhost/tasks/patchKernel.yml
+++ /dev/null
@@ -1,43 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
-# license agreements; and to You under the Apache License, Version 2.0.
----
-# This role will patch the kernel to the version defined in linux.version.
-
-- name: "install headers for kernel {{ linux.version }}"
- apt:
- name: "linux-headers-{{ linux.version }}"
- update_cache: yes
- state: present
- force: yes
- become: true
-
-- name: "install generic headers for kernel {{ linux.version }}"
- apt:
- name: "linux-headers-{{ linux.version }}-generic"
- update_cache: yes
- state: present
- force: yes
- become: true
-
-- name: "install kernel {{ linux.version }}"
- apt:
- name: "linux-image-{{ linux.version }}-generic"
- update_cache: yes
- state: present
- force: yes
- become: true
- register: patchKernel_apt
-
-- name: restart host to load new kernel
- shell: shutdown -r now
- when: patchKernel_apt.changed
- ignore_errors: true
- become: true
-
-- name: wait until host comes back up, timeout after 10 minutes
- local_action: wait_for host={{ ansible_host }} port=22 state=started
delay=10 timeout=600
-
-# can not use apt module because of
https://github.com/ansible/ansible-modules-core/issues/4029
-- name: remove old kernels
- shell: apt-get -y autoremove --purge
- become: true
diff --git a/tools/ubuntu-setup/README.md b/tools/ubuntu-setup/README.md
index ceabcd7..9446751 100644
--- a/tools/ubuntu-setup/README.md
+++ b/tools/ubuntu-setup/README.md
@@ -21,7 +21,9 @@
The following are verified to work on Ubuntu 14 LTS. You may need `sudo` or
root access to install required software depending on your system setup.
-The commands below should be executed on the host machine for single VM/server
deployments of OpenWhisk. For a distributed deployment spanning multiple VMs,
the commands should be executed on a machine with network connectivity to all
the VMs in the deployment - this is called the `bootstrapper` and it is ideally
an Ubuntu 14.04 VM that is provisioned in an IaaS (infrastructure as a service
platform e.g., OpenStack). Your local machine can act as the bootstrapper as
well if it can conne [...]
+The commands below should be executed on the host machine for single VM/server
deployments of OpenWhisk.
+For a distributed deployment spanning multiple VMs, the commands should be
executed on a machine with network connectivity to all the VMs in the
deployment - this is called the `bootstrapper` and it is ideally an Ubuntu
14.04 VM that is provisioned in an IaaS (infrastructure as a service platform
e.g., OpenStack).
+Your local machine can act as the bootstrapper as well if it can connect to
the VMs deployed in your IaaS.
```
# Install git if it is not installed
@@ -48,8 +50,6 @@ If you choose to install Oracle JDK 8 instead of Open JDK 8,
please run the foll
(cd tools/ubuntu-setup && ./all.sh oracle)
```
-If you are deploying OpenWhisk in a distributed environment across multiple
VMs, then follow the instructions in
[ansible/README_DISTRIBUTED.md](../../ansible/README_DISTRIBUTED.md) to
complete the deployment. Otherwise, continue with the instructions below.
-
### Select a data store
Follow instructions [tools/db/README.md](../db/README.md) on how to configure
a data store for OpenWhisk.
