Repository: incubator-metron Updated Branches: refs/heads/master eb209f5c9 -> 34614cc90
METRON-196 Deployment Fails Without Ansible 2.0.0.2 closes apache/incubator-metron#499 Project: http://git-wip-us.apache.org/repos/asf/incubator-metron/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-metron/commit/34614cc9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-metron/tree/34614cc9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-metron/diff/34614cc9 Branch: refs/heads/master Commit: 34614cc9006a77826ed729f2c8a40f99bd84d4b4 Parents: eb209f5 Author: dlyle65535 <dlyle65...@gmail.com> Authored: Tue Apr 4 08:15:42 2017 -0400 Committer: David Lyle <dlyle65...@gmail.com> Committed: Tue Apr 4 08:15:42 2017 -0400 ---------------------------------------------------------------------- metron-deployment/README.md | 2 +- metron-deployment/amazon-ec2/README.md | 10 +++++----- metron-deployment/amazon-ec2/playbook.yml | 4 ++++ .../extra_modules/ambari_cluster_state.py | 2 +- .../extra_modules/ambari_service_state.py | 2 +- .../playbooks/metron_full_install.yml | 8 +++++++- .../roles/ambari_gather_facts/tasks/main.yml | 20 +++++++++----------- .../roles/ambari_master/tasks/main.yml | 2 +- .../vagrant/codelab-platform/README.md | 2 +- .../vagrant/full-dev-platform/README.md | 17 +++++------------ .../vagrant/quick-dev-platform/README.md | 2 +- 11 files changed, 36 insertions(+), 35 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/README.md b/metron-deployment/README.md index ee44a2f..66aa5f4 100644 --- a/metron-deployment/README.md +++ b/metron-deployment/README.md @@ -17,7 +17,7 @@ The following tools are required to run these scripts: - [Maven](https://maven.apache.org/) - [Git](https://git-scm.com/) -- [Ansible](http://www.ansible.com/) (version 2.0.0.2) +- [Ansible](http://www.ansible.com/) (2.0.0.2 or 2.2.2.0) - [Docker](https://www.docker.com/) (Docker for Mac on OSX) These scripts depend on two files for configuration: http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/amazon-ec2/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/amazon-ec2/README.md b/metron-deployment/amazon-ec2/README.md index 5f1cb38..6ffc9fa 100644 --- a/metron-deployment/amazon-ec2/README.md +++ b/metron-deployment/amazon-ec2/README.md @@ -12,7 +12,7 @@ Getting Started The host used to deploy Apache Metron will need the following software tools installed. The following versions are known to work as of the time of this writing, but by no means are these the only working versions. - - Ansible 2.0.0.2 + - Ansible 2.0.0.2 or 2.2.2.0 - Python 2.7.11 - Maven 3.3.9 @@ -26,14 +26,14 @@ Any platform that supports these tools is suitable, but the following instructio 2. With Homebrew installed, run the following command in a terminal to install all of the required tools. - ``` + ``` brew cask install java brew install maven git - brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/ee1273bf919a5e4e50838513a9e55ea423e1d7ce/Formula/ansible.rb - brew switch ansible 2.0.0.2 ``` -3. Ensure that a public SSH key is located at `~/.ssh/id_rsa.pub`. +3. Install Ansible by following the instructions [here](http://docs.ansible.com/ansible/intro_installation.html#latest-releases-via-pip). + +4. Ensure that a public SSH key is located at `~/.ssh/id_rsa.pub`. ``` $ cat ~/.ssh/id_rsa.pub http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/amazon-ec2/playbook.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/amazon-ec2/playbook.yml b/metron-deployment/amazon-ec2/playbook.yml index fb19d89..de51534 100644 --- a/metron-deployment/amazon-ec2/playbook.yml +++ b/metron-deployment/amazon-ec2/playbook.yml @@ -21,6 +21,10 @@ - hosts: localhost vars_files: - conf/defaults.yml + pre_tasks: + - name: Verify Ansible Version + fail: msg="Metron Requires Ansible 2.0.0.2 or 2.2.2.0, current version is {{ ansible_version }}" + when: "ansible_version.full | version_compare('2.2.2.0', '!=') and ansible_version.full | version_compare('2.0.0.2', '!=')" tasks: - include: tasks/create-keypair.yml - include: tasks/create-vpc.yml http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/extra_modules/ambari_cluster_state.py ---------------------------------------------------------------------- diff --git a/metron-deployment/extra_modules/ambari_cluster_state.py b/metron-deployment/extra_modules/ambari_cluster_state.py index 691d48e..0542526 100644 --- a/metron-deployment/extra_modules/ambari_cluster_state.py +++ b/metron-deployment/extra_modules/ambari_cluster_state.py @@ -142,7 +142,7 @@ def main(): blueprint_var=dict(type='dict', required=False), blueprint_name=dict(type='str', default=None, required=False), configurations=dict(type='list', default=None, required=False), - wait_for_complete=dict(default=False, required=False, choices=BOOLEANS), + wait_for_complete=dict(default=False, required=False, type='bool'), ) required_together = ['blueprint_var', 'blueprint_name'] http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/extra_modules/ambari_service_state.py ---------------------------------------------------------------------- diff --git a/metron-deployment/extra_modules/ambari_service_state.py b/metron-deployment/extra_modules/ambari_service_state.py index 6dbefa8..46d63f2 100644 --- a/metron-deployment/extra_modules/ambari_service_state.py +++ b/metron-deployment/extra_modules/ambari_service_state.py @@ -137,7 +137,7 @@ def main(): service_name=dict(type='str', required=False), component_name=dict(type='str', default=None, required=False), component_host=dict(type='str', default=None, required=False), - wait_for_complete=dict(default=False, required=False, choices=BOOLEANS), + wait_for_complete=dict(default=False, required=False, type='bool'), ) required_together = ['component_name', 'component_host'] http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/playbooks/metron_full_install.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/playbooks/metron_full_install.yml b/metron-deployment/playbooks/metron_full_install.yml index 3f1fa9a..670f5c8 100644 --- a/metron-deployment/playbooks/metron_full_install.yml +++ b/metron-deployment/playbooks/metron_full_install.yml @@ -15,6 +15,12 @@ # limitations under the License. # --- +- hosts: all + pre_tasks: + - name: Verify Ansible Version + fail: msg="Metron Requires Ansible 2.0.0.2 or 2.2.2.0, current version is {{ ansible_version }}" + when: "ansible_version.full | version_compare('2.2.2.0', '!=') and ansible_version.full | version_compare('2.0.0.2', '!=')" + - include: metron_build.yml tags: - build @@ -22,7 +28,7 @@ - include: ambari_install.yml tags: - ambari - + - include: metron_install.yml tags: - metron http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/roles/ambari_gather_facts/tasks/main.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/roles/ambari_gather_facts/tasks/main.yml b/metron-deployment/roles/ambari_gather_facts/tasks/main.yml index c976e27..2b37eec 100644 --- a/metron-deployment/roles/ambari_gather_facts/tasks/main.yml +++ b/metron-deployment/roles/ambari_gather_facts/tasks/main.yml @@ -118,20 +118,18 @@ when: kafka_broker_tag is undefined # -# kafka_broker_url, kafka_broker_port -# -- name: "Ask Ambari: kafka_broker_port, kafka_broker_url" - uri: - url: "http://{{ groups.ambari_master[0] }}:{{ ambari_port }}/api/v1/clusters/{{ cluster_name }}/configurations?type=kafka-broker&tag={{ kafka_broker_tag }}" - user: "{{ ambari_user }}" - password: "{{ ambari_password }}" - force_basic_auth: yes - return_content: yes - register: kafka_broker_properties_response +# kafka_broker_port +# +- name: "Ask Ambari: kafka_broker_port" + shell: > + curl -s -u {{ ambari_user }}:{{ ambari_password }} -X GET -H "X-Requested-By: ambari" "http://{{ groups.ambari_master[0] }}:{{ ambari_port }}/api/v1/clusters/{{ cluster_name }}/configurations?type=kafka-broker&tag={{ kafka_broker_tag }}" | python -c 'import sys, json; print json.load(sys.stdin)["items"][0]["properties"]["listeners"]' + args: + warn: false + register: kafka_broker_port_response when: (kafka_broker_url is undefined) or (kafka_broker_port is undefined) - set_fact: - kafka_broker_port: "{{ (kafka_broker_properties_response.content | from_json)['items'][0].properties['listeners'] | replace('PLAINTEXT://localhost:', '')}}" + kafka_broker_port: "{{ kafka_broker_port_response.stdout_lines[0] | replace('PLAINTEXT://localhost:', '')}}" when: kafka_broker_port is undefined - set_fact: http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/roles/ambari_master/tasks/main.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/roles/ambari_master/tasks/main.yml b/metron-deployment/roles/ambari_master/tasks/main.yml index 28b89e8..84696c1 100644 --- a/metron-deployment/roles/ambari_master/tasks/main.yml +++ b/metron-deployment/roles/ambari_master/tasks/main.yml @@ -36,7 +36,7 @@ - name: Setup ambari server shell: ambari-server setup -s && touch /etc/ambari-server/configured creates=/etc/ambari-server/configured register: ambari_server_setup - failed_when: "ambari_server_setup.stderr or 'FATAL' in ambari_server_setup.stdout" + failed_when: "(ambari_server_setup.stderr is defined and ambari_server_setup.stderr != '') or 'FATAL' in ambari_server_setup.stdout" - name: Copy MPack to Ambari Host copy: http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/vagrant/codelab-platform/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/vagrant/codelab-platform/README.md b/metron-deployment/vagrant/codelab-platform/README.md index b1da488..893f97d 100644 --- a/metron-deployment/vagrant/codelab-platform/README.md +++ b/metron-deployment/vagrant/codelab-platform/README.md @@ -10,7 +10,7 @@ Getting Started As with the Singlenode Full Image, the computer used to deploy Apache Metron will need to have the following components installed. - - [Ansible](https://github.com/ansible/ansible) 2.0.0.2 + - [Ansible](https://github.com/ansible/ansible) (2.0.0.2 or 2.2.2.0) - [Vagrant](https://www.vagrantup.com) 1.8.1 - [Virtualbox](https://virtualbox.org) 5.0.16 - Python 2.7.11 http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/vagrant/full-dev-platform/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/vagrant/full-dev-platform/README.md b/metron-deployment/vagrant/full-dev-platform/README.md index 03b2904..8bc78b3 100644 --- a/metron-deployment/vagrant/full-dev-platform/README.md +++ b/metron-deployment/vagrant/full-dev-platform/README.md @@ -12,7 +12,7 @@ Getting Started The computer used to deploy Apache Metron will need to have the following components installed. - - [Ansible](https://github.com/ansible/ansible) 2.0.0.2 + - [Ansible](https://github.com/ansible/ansible) (2.0.0.2 or 2.2.2.0) - [Vagrant](https://www.vagrantup.com) 1.8.1 - [Virtualbox](https://virtualbox.org) 5.0.16 - Python 2.7.11 @@ -29,26 +29,19 @@ Any platform that supports these tools is suitable, but the following instructio ``` brew cask install vagrant virtualbox java brew install maven git - brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/ee1273bf919a5e4e50838513a9e55ea423e1d7ce/Formula/ansible.rb - brew switch ansible 2.0.0.2 ``` -### Deploy Metron - -1. Build Metron +3. Install Ansible by following the instructions [here](http://docs.ansible.com/ansible/intro_installation.html#latest-releases-via-pip). - ``` - cd incubator-metron - mvn clean package -DskipTests - ``` +### Deploy Metron -2. Install Vagrant Hostmanager. +1. Install Vagrant Hostmanager. ``` vagrant plugin install vagrant-hostmanager ``` -3. Deploy Metron +2. Deploy Metron ``` cd metron-deployment/vagrant/full-dev-platform http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/34614cc9/metron-deployment/vagrant/quick-dev-platform/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/vagrant/quick-dev-platform/README.md b/metron-deployment/vagrant/quick-dev-platform/README.md index 1015563..71960ef 100644 --- a/metron-deployment/vagrant/quick-dev-platform/README.md +++ b/metron-deployment/vagrant/quick-dev-platform/README.md @@ -14,7 +14,7 @@ Getting Started As with the Full Development Platform (`metron-deployment/vagrant/full-dev-platform`), the computer used to deploy Apache Metron will need the following components installed. - - [Ansible](https://github.com/ansible/ansible) 2.0.0.2 + - [Ansible](https://github.com/ansible/ansible) (2.0.0.2 or 2.2.2.0) - [Vagrant](https://www.vagrantup.com) 1.8.1 - [Virtualbox](https://virtualbox.org) 5.0.16 - Python 2.7.11
