Repository: airavata Updated Branches: refs/heads/develop 59142dd7d -> 7f983b886
Implement Ansible playbook for OpenStack resource provisioning Project: http://git-wip-us.apache.org/repos/asf/airavata/repo Commit: http://git-wip-us.apache.org/repos/asf/airavata/commit/7f983b88 Tree: http://git-wip-us.apache.org/repos/asf/airavata/tree/7f983b88 Diff: http://git-wip-us.apache.org/repos/asf/airavata/diff/7f983b88 Branch: refs/heads/develop Commit: 7f983b8868aa0b7f726a9d6c130f48e5649f7464 Parents: 59142dd Author: Gourav Shenoy <[email protected]> Authored: Fri Sep 30 14:14:35 2016 -0400 Committer: Gourav Shenoy <[email protected]> Committed: Fri Sep 30 14:14:35 2016 -0400 ---------------------------------------------------------------------- modules/cloud/ansible-playbooks/README.md | 61 ++++++++++++++++---- .../roles/openstack/tasks/main.yml | 28 +++++++++ .../roles/openstack/vars/main.yml | 10 ++++ .../openstack/vars/openstack-credentials.yml | 10 ++++ modules/cloud/ansible-playbooks/site.yml | 5 ++ 5 files changed, 102 insertions(+), 12 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/airavata/blob/7f983b88/modules/cloud/ansible-playbooks/README.md ---------------------------------------------------------------------- diff --git a/modules/cloud/ansible-playbooks/README.md b/modules/cloud/ansible-playbooks/README.md index 9ce4e47..b716670 100644 --- a/modules/cloud/ansible-playbooks/README.md +++ b/modules/cloud/ansible-playbooks/README.md @@ -5,21 +5,55 @@ If you are working with new set of hosts which you haven't ssh login to those ho ## Configurations -To run this ansible script you must have at least 4 instances. 3 instances to setup Mesos master , marathon and zookeeper clusters. Along with Mesos master we install marathon and zookeeper. Another one or more instance/s to setup mesos agent/s. You can use ec2 ansible role provided with this playbook to spin up aws instances. To do that you need to set valid aws credentials. +To run this ansible script you must have at least 4 instances. 3 instances to setup Mesos master , marathon and zookeeper clusters. Along with Mesos master we install marathon and zookeeper. Another one or more instance/s to setup mesos agent/s. You can use __ec2__ ansible role provided with this playbook to spin up aws instances OR use the __openstack__ ansible role to spin up OpenStack instances. For either of these roles to work, you need to set valid credentials for AWS or OpenStack. -1. set valid aws credentials in `roles/ec2/vars/aws-credential.yml` if it doesn't work add folowing to ec2 task in `roles/ec2/tasks/main.yml` +### AWS Configuration & Provisioning + +1. Install the __boto__ python package using the following command: + `pip install boto` + +2. set valid aws credentials in `roles/ec2/vars/aws-credential.yml`. You need to set the following parameters: `aws_access_key: <your_valid_access_key>` - `aws_secret_key: <your_valid_secret_key?` + `aws_secret_key: <your_valid_secret_key>` + +3. Set ec2 instance names under ec2 task `with_items:` configurations + +After you set valid aws credentials and instance names run following ansible playbook command to spin up require aws ec2 instances. + + `ansible-playbook -i hosts site.yml -t "ec2"` + + +### OpenStack Configuration & Provisioning + +1. Install the __shade__ python package using the following command: + `pip install shade` + +2. set valid openstack credentials in `roles/openstack/vars/openstack-credential.yml`. You need to set the following parameters: -2. Set ec2 instance names under ec2 task `with_items:` configurations + `os_username: <your_valid_openstack_username>` - After you set valid aws credentials and instance names run following ansible playbook command to spin up require aws ec2 instances. + `os_password: <your_valid_openstack_password>` - `ansible-palybook -i hosts site.yml -t "ec2"` + `os_project_name: <your_valid_openstack_project_name>` -3. You need to know public ips of all the nodes and private ips of all mesos-master nodes. List all mesos master public ips under `[mesos-master]` section in `hosts` file. List all mesos-slave *(mesos-agent)* public ips under `[mesos-slave]` section. You need to set `my_id` hosts variable along with mesos-master ip addresses. This will use to set zookeeper id. + `os_auth_url: <your_valid_keystone_auth_url>` + + `os_region_name: <your_valid_openstack_region>` + + +3. Set OpenStack instance names under openstack task `with_items:` configurations + + +After you set valid OpenStack credentials and instance names, run the following ansible playbook command to spin up require OpenStack instances. + + `ansible-playbook -i hosts site.yml -t âopenstackâ` + + +## Installation + +1. You need to know public ips of all the nodes and private ips of all mesos-master nodes. List all mesos master public ips under `[mesos-master]` section in `hosts` file. List all mesos-slave *(mesos-agent)* public ips under `[mesos-slave]` section. You need to set `my_id` hosts variable along with mesos-master ip addresses. This will use to set zookeeper id. > [mesos-master] @@ -35,7 +69,7 @@ To run this ansible script you must have at least 4 instances. 3 instances to se > ... -4. Use mesos-master private ip addresses to set zookeeper servers properties in `roles/zookeeper/vars/main.yml` +2. Use mesos-master private ip addresses to set zookeeper servers properties in `roles/zookeeper/vars/main.yml` > zookeeper_servers: @@ -45,11 +79,11 @@ To run this ansible script you must have at least 4 instances. 3 instances to se > \- {id: "3", ip: "172.31.25.80"} -5. Set Mesos-master zookeeper quorum value in `group_vars/all.yml` file +3. Set Mesos-master zookeeper quorum value in `group_vars/all.yml` file > zk_quorum: 2 -6. Now we are set to deploy Apache Mesos cluster (Mesos-master, marthon, zookeeper, mesos-slave). Following ansible playbook command respectively setup mesos-master cluster , Mesos-slaves, and both mesos-master cluster and mesos-slave +4. Now we are set to deploy Apache Mesos cluster (Mesos-master, marthon, zookeeper, mesos-slave). Following ansible playbook command respectively setup mesos-master cluster , Mesos-slaves, and both mesos-master cluster and mesos-slave `ansible-playbook -i hosts site.yml -t "mesos-master"` @@ -57,9 +91,10 @@ To run this ansible script you must have at least 4 instances. 3 instances to se `ansible-playbook -i hosts site.yml -t "mesos"` - If everything works without any error, now you have running mesos cluster on aws instances. +If everything works without any error, now you have running mesos cluster on aws/openstack instances. + -## verifying installation +## Verifying installation 1. If your plays works without any error. Now you can access Mesos master console using `http://<master-ip>:5050` run following in one of Mesos master node to find which instance is the leader of cluster. @@ -80,3 +115,5 @@ To run this ansible script you must have at least 4 instances. 3 instances to se - zookeeper - install zookeeper - mesos-master - install mesos mastera and marathon - mesos-slave - install mesos slave +- ec2 - provision instances on aws +- openstack - provision instances on openstack http://git-wip-us.apache.org/repos/asf/airavata/blob/7f983b88/modules/cloud/ansible-playbooks/roles/openstack/tasks/main.yml ---------------------------------------------------------------------- diff --git a/modules/cloud/ansible-playbooks/roles/openstack/tasks/main.yml b/modules/cloud/ansible-playbooks/roles/openstack/tasks/main.yml new file mode 100644 index 0000000..27e7499 --- /dev/null +++ b/modules/cloud/ansible-playbooks/roles/openstack/tasks/main.yml @@ -0,0 +1,28 @@ +--- +- name: include credentials + include_vars: vars/openstack-credentials.yml + +- name: create openstack instance(s) + os_server: + state: present + auth: + auth_url: "{{os_auth_url}}" + username: "{{os_username}}" + password: "{{os_password}}" + project_name: "{{os_project_name}}" + + name: "{{item}}" + region_name: "{{os_region_name}}" + image: "{{centos7_gc_image}}" + key_name: "{{key_name}}" + timeout: 200 + flavor: "{{flavor_id}}" + + nics: + - net-name: "{{network_name}}" + security_groups: "{{security_group}}" + floating_ip_pools: "{{floating_ip_pool}}" + + with_items: + - test_ansible_1 + - test_ansible_2 http://git-wip-us.apache.org/repos/asf/airavata/blob/7f983b88/modules/cloud/ansible-playbooks/roles/openstack/vars/main.yml ---------------------------------------------------------------------- diff --git a/modules/cloud/ansible-playbooks/roles/openstack/vars/main.yml b/modules/cloud/ansible-playbooks/roles/openstack/vars/main.yml new file mode 100644 index 0000000..0a2ee2e --- /dev/null +++ b/modules/cloud/ansible-playbooks/roles/openstack/vars/main.yml @@ -0,0 +1,10 @@ +--- + +key_name: gourav-mac-key +centos7_gc_image: 1790e5c8-315a-4b9b-8b1f-46e47330d3cc +flavor_id: 1 + +security_group: Airavata + +network_name: airavata +floating_ip_pool: public \ No newline at end of file http://git-wip-us.apache.org/repos/asf/airavata/blob/7f983b88/modules/cloud/ansible-playbooks/roles/openstack/vars/openstack-credentials.yml ---------------------------------------------------------------------- diff --git a/modules/cloud/ansible-playbooks/roles/openstack/vars/openstack-credentials.yml b/modules/cloud/ansible-playbooks/roles/openstack/vars/openstack-credentials.yml new file mode 100644 index 0000000..c15e2d8 --- /dev/null +++ b/modules/cloud/ansible-playbooks/roles/openstack/vars/openstack-credentials.yml @@ -0,0 +1,10 @@ +--- + +os_username: xxxx +os_password: xxxx + +os_project_name: xxxx + +os_auth_url: xxxx + +os_region_name: xxxx http://git-wip-us.apache.org/repos/asf/airavata/blob/7f983b88/modules/cloud/ansible-playbooks/site.yml ---------------------------------------------------------------------- diff --git a/modules/cloud/ansible-playbooks/site.yml b/modules/cloud/ansible-playbooks/site.yml index 17258bd..b212f7f 100644 --- a/modules/cloud/ansible-playbooks/site.yml +++ b/modules/cloud/ansible-playbooks/site.yml @@ -4,6 +4,11 @@ roles: - ec2 +- hosts: localhost + tags: openstack + roles: + - openstack + - hosts: mesos-master tags: mesos, mesos-master roles:
