[4/4] incubator-myriad git commit: A full DC/OS cluster for Apache Myriad development and testing with resources such as Mesos-DNS, Minuteman load balancer (VIPs), Marathon meta-scheduler and so on.

[javiroman]

A full DC/OS cluster for Apache Myriad development and testing with resources
such as Mesos-DNS, Minuteman load balancer (VIPs), Marathon meta-scheduler and 
so on.

JIRA:
    [MYRIAD-262] https://issues.apache.org/jira/browse/MYRIAD-262

Pull Request:
    Closes #108

Author:  Javi Roman <jroman.espi...@gmail.com>
Date:    Sat Apr 7 10:59:50 2018 +0200


Project: http://git-wip-us.apache.org/repos/asf/incubator-myriad/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-myriad/commit/1c3857a8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-myriad/tree/1c3857a8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-myriad/diff/1c3857a8

Branch: refs/heads/master
Commit: 1c3857a8dde10cf162a7abef08170679b7ee1512
Parents: 2ce260e
Author: Javi Roman <jroman.espi...@gmail.com>
Authored: Sat Apr 7 10:59:50 2018 +0200
Committer: Javi Roman <javiro...@apache.org>
Committed: Tue Apr 10 08:14:15 2018 +0200

----------------------------------------------------------------------
 vagrant/libvirt/README.md                       |  55 +++++
 vagrant/libvirt/dcos/README.md                  |  81 ++++++
 vagrant/libvirt/dcos/Vagrantfile                |  94 +++++++
 vagrant/libvirt/dcos/ansible.cfg                |  76 ++++++
 vagrant/libvirt/dcos/check-env.sh               |  29 +++
 vagrant/libvirt/dcos/config/cluster-1m4a1p.yaml |  70 ++++++
 vagrant/libvirt/dcos/config/cluster-3m4a1p.yaml |  82 +++++++
 .../group_vars/all/dcos-versions.yaml           |  29 +++
 .../provisioning/group_vars/all/defaults.yml    |  44 ++++
 vagrant/libvirt/dcos/provisioning/inventory.py  | 232 ++++++++++++++++++
 .../provisioning/roles/common/handlers/main.yml |  21 ++
 .../provisioning/roles/common/tasks/main.yml    |  78 ++++++
 .../roles/common/templates/hostname-fix.j2      |  23 ++
 .../roles/common/templates/hosts.j2             |  27 ++
 .../roles/common/templates/ntp.conf.j2          |  25 ++
 .../roles/dcos-install/tasks/main.yml           |  43 ++++
 .../roles/dcos-setup/tasks/main.yml             | 130 ++++++++++
 .../provisioning/roles/docker/tasks/main.yml    |  63 +++++
 .../roles/passwordless/tasks/main.yml           |  38 +++
 .../provisioning/roles/registry/tasks/main.yml  |  32 +++
 vagrant/libvirt/dcos/provisioning/site.yml      |  63 +++++
 vagrant/libvirt/dcos/requirements.txt           |   1 +
 vagrant/libvirt/dcos/setup-env                  |  24 ++
 vagrant/libvirt/mesos/README.md                 | 245 +++++++++++++++++++
 vagrant/libvirt/mesos/Vagrantfile               |  92 +++++++
 vagrant/libvirt/mesos/ansible.cfg               |  79 ++++++
 vagrant/libvirt/mesos/check-env.sh              |  28 +++
 vagrant/libvirt/mesos/config/cluster-1m4a.yaml  |  64 +++++
 vagrant/libvirt/mesos/config/cluster-3m4a.yaml  |  76 ++++++
 vagrant/libvirt/mesos/docs/ansible.txt          |   1 +
 vagrant/libvirt/mesos/docs/build-mesos.md       | 135 ++++++++++
 vagrant/libvirt/mesos/docs/notes-myriad.txt     |  11 +
 vagrant/libvirt/mesos/docs/steps-hadoop.txt     | 145 +++++++++++
 .../provisioning/group_vars/all/defaults.yml    | 100 ++++++++
 vagrant/libvirt/mesos/provisioning/inventory.py | 219 +++++++++++++++++
 .../provisioning/roles/common/handlers/main.yml |  20 ++
 .../provisioning/roles/common/tasks/main.yml    | 104 ++++++++
 .../roles/common/templates/hostname-fix.j2      |  23 ++
 .../roles/common/templates/hosts.j2             |  26 ++
 .../roles/common/templates/ntp.conf.j2          |  24 ++
 .../provisioning/roles/common/vars/main.yml     |   2 +
 .../provisioning/roles/devel/handlers/main.yml  |  29 +++
 .../provisioning/roles/devel/tasks/main.yml     |  70 ++++++
 .../roles/devel/templates/exports.j2            |  34 +++
 .../provisioning/roles/dfs-start/tasks/main.yml |  39 +++
 .../provisioning/roles/docker/tasks/main.yml    |  63 +++++
 .../roles/hadoop/files/core-site.xml            |  23 ++
 .../roles/hadoop/files/hdfs-site.xml            |  34 +++
 .../provisioning/roles/hadoop/files/java.sh     |  20 ++
 .../roles/hadoop/files/mapred-site.xml          |  24 ++
 .../hadoop/files/myriad-config-default.yml      |  76 ++++++
 .../provisioning/roles/hadoop/files/slaves      |   3 +
 .../provisioning/roles/hadoop/files/workers     |   3 +
 .../roles/hadoop/files/yarn-site.xml            |  40 +++
 .../provisioning/roles/hadoop/tasks/build.yml   |  51 ++++
 .../roles/hadoop/tasks/datanode.yml             |  50 ++++
 .../provisioning/roles/hadoop/tasks/main.yml    |  37 +++
 .../roles/hadoop/tasks/namenode.yml             |  68 +++++
 .../roles/nfs-client/tasks/main.yml             |  31 +++
 .../roles/passwordless/tasks/build.yml          |  38 +++
 .../roles/passwordless/tasks/main.yml           |  24 ++
 .../roles/passwordless/tasks/master.yml         |  83 +++++++
 .../roles/yarn-start/tasks/main.yml             |  30 +++
 .../provisioning/roles/zookeeper/tasks/main.yml |  51 ++++
 vagrant/libvirt/mesos/provisioning/site.yml     |  63 +++++
 vagrant/libvirt/mesos/setup-env                 |  23 ++
 vagrant/mesos/libvirt/README.md                 |  55 -----
 vagrant/mesos/libvirt/mesos/README.md           | 245 -------------------
 vagrant/mesos/libvirt/mesos/Vagrantfile         |  92 -------
 vagrant/mesos/libvirt/mesos/ansible.cfg         |  79 ------
 vagrant/mesos/libvirt/mesos/check-env.sh        |  28 ---
 .../libvirt/mesos/config/cluster-1m4a.yaml      |  64 -----
 .../libvirt/mesos/config/cluster-3m4a.yaml      |  76 ------
 vagrant/mesos/libvirt/mesos/docs/ansible.txt    |   1 -
 vagrant/mesos/libvirt/mesos/docs/build-mesos.md | 135 ----------
 .../mesos/libvirt/mesos/docs/notes-myriad.txt   |  11 -
 .../mesos/libvirt/mesos/docs/steps-hadoop.txt   | 145 -----------
 .../provisioning/group_vars/all/defaults.yml    | 100 --------
 .../libvirt/mesos/provisioning/inventory.py     | 219 -----------------
 .../provisioning/roles/common/handlers/main.yml |  20 --
 .../provisioning/roles/common/tasks/main.yml    | 104 --------
 .../roles/common/templates/hostname-fix.j2      |  23 --
 .../roles/common/templates/hosts.j2             |  26 --
 .../roles/common/templates/ntp.conf.j2          |  24 --
 .../provisioning/roles/common/vars/main.yml     |   2 -
 .../provisioning/roles/devel/handlers/main.yml  |  29 ---
 .../provisioning/roles/devel/tasks/main.yml     |  70 ------
 .../roles/devel/templates/exports.j2            |  34 ---
 .../provisioning/roles/dfs-start/tasks/main.yml |  39 ---
 .../provisioning/roles/docker/tasks/main.yml    |  63 -----
 .../roles/hadoop/files/core-site.xml            |  23 --
 .../roles/hadoop/files/hdfs-site.xml            |  34 ---
 .../provisioning/roles/hadoop/files/java.sh     |  20 --
 .../roles/hadoop/files/mapred-site.xml          |  24 --
 .../hadoop/files/myriad-config-default.yml      |  76 ------
 .../provisioning/roles/hadoop/files/slaves      |   3 -
 .../provisioning/roles/hadoop/files/workers     |   3 -
 .../roles/hadoop/files/yarn-site.xml            |  40 ---
 .../provisioning/roles/hadoop/tasks/build.yml   |  51 ----
 .../roles/hadoop/tasks/datanode.yml             |  50 ----
 .../provisioning/roles/hadoop/tasks/main.yml    |  37 ---
 .../roles/hadoop/tasks/namenode.yml             |  68 -----
 .../roles/nfs-client/tasks/main.yml             |  31 ---
 .../roles/passwordless/tasks/build.yml          |  38 ---
 .../roles/passwordless/tasks/main.yml           |  24 --
 .../roles/passwordless/tasks/master.yml         |  83 -------
 .../roles/yarn-start/tasks/main.yml             |  30 ---
 .../provisioning/roles/zookeeper/tasks/main.yml |  51 ----
 .../mesos/libvirt/mesos/provisioning/site.yml   |  63 -----
 vagrant/mesos/libvirt/mesos/setup-env           |  23 --
 vagrant/mesos/virtualbox/Vagrantfile            |  79 ------
 .../virtualbox/provisioning/format_namenode.sh  |  23 --
 .../provisioning/install_default_jdk.sh         |  32 ---
 .../virtualbox/provisioning/install_docker.sh   |  25 --
 .../virtualbox/provisioning/install_hadoop.sh   | 115 ---------
 .../virtualbox/provisioning/install_mesos.sh    |  38 ---
 .../mesos/virtualbox/provisioning/shutdown.sh   |  30 ---
 .../virtualbox/provisioning/start_datanode.sh   |  24 --
 .../provisioning/start_historyserver.sh         |  23 --
 .../provisioning/start_mesos_master.sh          |  34 ---
 .../provisioning/start_mesos_slave.sh           |  38 ---
 .../virtualbox/provisioning/start_namenode.sh   |  27 --
 vagrant/virtualbox/mesos/Vagrantfile            |  79 ++++++
 .../mesos/provisioning/format_namenode.sh       |  23 ++
 .../mesos/provisioning/install_default_jdk.sh   |  32 +++
 .../mesos/provisioning/install_docker.sh        |  25 ++
 .../mesos/provisioning/install_hadoop.sh        | 115 +++++++++
 .../mesos/provisioning/install_mesos.sh         |  38 +++
 .../virtualbox/mesos/provisioning/shutdown.sh   |  30 +++
 .../mesos/provisioning/start_datanode.sh        |  24 ++
 .../mesos/provisioning/start_historyserver.sh   |  23 ++
 .../mesos/provisioning/start_mesos_master.sh    |  34 +++
 .../mesos/provisioning/start_mesos_slave.sh     |  38 +++
 .../mesos/provisioning/start_namenode.sh        |  27 ++
 134 files changed, 4249 insertions(+), 2944 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/README.md
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/README.md b/vagrant/libvirt/README.md
new file mode 100644
index 0000000..ced3779
--- /dev/null
+++ b/vagrant/libvirt/README.md
@@ -0,0 +1,55 @@
+# Myriad Vagrant based testbed setup
+
+This is a development environment for the Apache Myriad project based on
+Vagrant and libvirt. 
+
+You have to setup properly your environment in order to use Vagrant and
+libvirt. Follow the steps in the nex section for different environments.
+
+# Requeriments                   
+
+Download Vagrant from HashiCorp site:
+
+https://www.vagrantup.com/downloads.html
+
+## For Ubuntu users              
+
+```                              
+$ sudo apt-get install software-properties-common                 
+$ sudo apt-add-repository ppa:ansible/ansible                     
+$ sudo apt-get update            
+$ sudo apt-get install ansible   
+$ sudo apt-get install qemu-kvm libvirt-bin libvirt-dev           
+$ sudo apt-get install ruby-libvirt                               
+
+$ vagrant plugin install vagrant-libvirt
+Installing the 'vagrant-libvirt' plugin. This can take a few minutes...        
                                                     
+$ vagrant plugin list            
+vagrant-libvirt (0.0.40, system) 
+
+$ sudo usermod -G libvirtd -a $USER                               
+$ newgrp -                       
+``` 
+
+## For Fedora users              
+
+``` 
+dnf -y install qemu libvirt libvirt-devel ruby-devel gcc
+dnf -y install libxslt-devel libxml2-devel libvirt-devel 
+dnf -y install libguestfs-tools-c ruby-devel gcc
+``` 
+
+Testing Vagrant-libvirt:
+
+``` 
+$ mkdir test && cd test          
+$ vagrant init centos/7          
+$ vagrant up --provider libvirt  
+$ vagrant box list 
+centos/7                 (libvirt, 1702.01)                       
+$ vagrant status                 
+$ vagrant ssh                    
+$ vagrant destroy                
+$ cd .. && rm -fr test           
+``` 
+

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/README.md
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/README.md b/vagrant/libvirt/dcos/README.md
new file mode 100644
index 0000000..fd43ee4
--- /dev/null
+++ b/vagrant/libvirt/dcos/README.md
@@ -0,0 +1,81 @@
+# Mesosphere DC/OS Community Environment
+
+This is the Vagrant-libvirt based environment for testing Apache Myriad
+project in a DC/OS cluster.
+
+# Setup the environment
+
+In order to use this vagrant environment we have to enable a suitable
+environment. The key points of this environment are:
+
+- **DCOS_ARCH**: This variable has to match with the VMs architecture defined 
in
+  the config folder: *cluster-1m4a1p.yaml*, *cluster-3m4a1p.yaml*, and so on. 
For
+  example **DCOS_ARCH: 1m4a1p** matchs with the architecture 
*cluster-1m4a.yaml*, which
+  is a DC/OS master node (1m), four DC/OS private agents (4a), and one DC/OS
+  public agent (1p).
+
+- **DCOS_VERSION**: This variable set the DC/OS community to download. The next
+  section shows an example of configuration.
+
+```
+$ source setup-env 
+
+This Vagrant environment is ready for the following settings:                  
       
+
+- DCOS_ARCH: 1m4a1p                        
+- DCOS_VERSION: 1.11.0                     
+- HADOOP_VERSION: 2.7.0                    
+
+'vagrant up --provider=libvirt' and happy hacking! 
+``` 
+
+# Running the deployment
+
+```
+$ vagrant up  --provider=libvirt
+[...]
+PLAY RECAP
+*********************************************************************
+bt                         : ok=36   changed=28   unreachable=0    failed=0   
+m1                         : ok=28   changed=25   unreachable=0    failed=0   
+a1                         : ok=31   changed=28   unreachable=0    failed=0   
+a2                         : ok=31   changed=28   unreachable=0    failed=0   
+a3                         : ok=31   changed=28   unreachable=0    failed=0   
+a4                         : ok=31   changed=28   unreachable=0    failed=0   
+p1                         : ok=31   changed=28   unreachable=0    failed=0   
+```
+
+## Final VMs set
+
+```
+$ vagrant status   
+Current machine states:
+
+bt                        running (libvirt)
+m1                        running (libvirt)
+a1                        running (libvirt)
+a2                        running (libvirt)
+a3                        running (libvirt)
+a4                        running (libvirt)
+p1                        running (libvirt)
+```
+
+## Running DC/OS CLI
+
+```
+$ vagrant ssh bt
+[vagrant@bt ~]$ dcos auth login
+If your browser didn't open, please go to the following link:
+
+    http://m1/login?redirect_uri=urn:ietf:wg:oauth:2.0:oob
+
+[vagrant@bt ~]$ dcos node
+   HOSTNAME         IP                          ID                    TYPE     
         REGION  ZONE  
+ 100.1.10.102  100.1.10.102  637fb1cd-f4d4-427a-9418-092b80eb6000-S2  agent    
         None    None  
+ 100.1.10.103  100.1.10.103  637fb1cd-f4d4-427a-9418-092b80eb6000-S3  agent    
         None    None  
+ 100.1.10.104  100.1.10.104  637fb1cd-f4d4-427a-9418-092b80eb6000-S0  agent    
         None    None  
+ 100.1.10.105  100.1.10.105  637fb1cd-f4d4-427a-9418-092b80eb6000-S1  agent    
         None    None  
+ 100.1.10.106  100.1.10.106  637fb1cd-f4d4-427a-9418-092b80eb6000-S4  agent    
         None    None  
+master.mesos.  100.1.10.101    5ee304dc-18a0-4f7f-b1ef-591c249cbbeb   master 
(leader)   None    None 
+```
+

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/Vagrantfile
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/Vagrantfile b/vagrant/libvirt/dcos/Vagrantfile
new file mode 100644
index 0000000..886c227
--- /dev/null
+++ b/vagrant/libvirt/dcos/Vagrantfile
@@ -0,0 +1,94 @@
+# -*- mode: ruby -*-
+# vim: set ft=ruby ts=2 et :
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+unless defined? VAGRANTFILE_API_VERSION 
+    VAGRANTFILE_API_VERSION = "2"
+end
+
+# Tested with Vagrant version:
+Vagrant.require_version ">= 1.7.2"
+
+# Require YAML module
+require 'yaml'
+
+def fail_with_message(msg)
+    fail Vagrant::Errors::VagrantError.new, msg
+end
+
+def check_minimal_env()
+    if ENV['DCOS_ARCH'].nil? || ENV['DCOS_ARCH'].empty?
+        fail_with_message "You have to set DCOS_ARCH environment variable!"
+    end
+    if ENV['DCOS_VERSION'].nil? || ENV['DCOS_VERSION'].empty?
+        fail_with_message "You have to set DCOS_VERSION environment variable!"
+    end
+    if ENV['HADOOP_VERSION'].nil? || ENV['HADOOP_VERSION'].empty?
+        fail_with_message "You have to set HADOOP_VERSION environment 
variable!"
+    end
+end
+
+# Sanity checks
+check_minimal_env()
+
+# Read YAML file with cluster details: The cluster flavor.
+config_file = 'config/cluster-' + ENV['DCOS_ARCH'] + ".yaml"
+if File.exists?(config_file)
+    cluster = YAML.load_file(config_file)
+else
+    fail_with_message "Cluster definition file: #{config_file} was not found!"
+end
+
+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+  # Disabling the default /vagrant share
+  config.vm.synced_folder ".", "/vagrant", disabled: true
+  config.ssh.insert_key = false
+
+  i = 0
+  cluster.each do |servers|
+    # VM definition
+    config.vm.define servers["name"] do |node|
+      node.vm.box = servers["box"]
+      node.vm.hostname = servers["name"]
+      node.vm.network "private_network", ip: servers["ip"]
+      node.vm.provider :libvirt do |domain|
+        domain.uri = 'qemu+unix:///system'
+        domain.driver = 'kvm'
+        domain.memory = servers["mem"]
+        domain.cpus = servers['cpu']
+        if servers.key?('aditional_disk')
+            for disk in servers['aditional_disk']
+                domain.storage :file, :size => disk
+            end
+        end
+      end
+      i += 1
+      if i == cluster.length
+          node.vm.provision :ansible do |ansible|
+            ansible.verbose = "vv"
+            ansible.inventory_path = "provisioning/inventory.py"
+            ansible.playbook = "provisioning/site.yml"
+            ansible.limit = "all"
+          end # provison
+      end # if
+    end # config
+  end # cluster
+end # vagrant

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/ansible.cfg
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/ansible.cfg b/vagrant/libvirt/dcos/ansible.cfg
new file mode 100644
index 0000000..109fedb
--- /dev/null
+++ b/vagrant/libvirt/dcos/ansible.cfg
@@ -0,0 +1,76 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+[defaults]
+# Disable host key checking. If a host is not initially in ‘known_hosts’ 
this
+# will result in prompting for confirmation of the key, which results in an
+# interactive experience 
+host_key_checking = False
+
+# This is the default SSH timeout to use on connection attempts.
+timeout = 30
+
+# This options forces color mode even when running without a TTY.
+force_color = 1
+
+# Number of parallel processes to spawn when 
+# communicating with remote hosts. 
+forks = 10
+
+# Facts gathering: The value ‘smart’ means each new 
+# host that has no facts discovered will be scanned, 
+# but if the same host is addressed in multiple plays 
+# it will not be # contacted again in the playbook run. 
+# This ave fact gathering time.
+gathering = smart
+
+# Ansible will log information about executions at the designated location
+log_path=.vagrant/ansible.log
+
+# The retry files save path is where Ansible will save .retry files when a
+# playbook fails. The directory will be created if it does not already exist.
+retry_files_save_path = .vagrant/retry-files
+
+# Play execution strategies: Plays execution control for many hosts, controls
+# how plays run their multiple hosts. 
+#
+# linear (default): All hosts of a play have to finish one task before any of 
them can 
+#                   begin the next one. Using the number of forks (default 5)
+#                   to parallelize.
+# serial: Ensures one group of hosts in a play finishes its work before 
another group 
+#         can begin. This strategy will take a subset of hosts (the default is
+#         five) and execute all tasks (in the linear fashion) against those 
+#         hosts before moving to the next subset and starting from the 
beginning.
+# free: Allows each host to run until the end of the play as fast as it can.
+#       Allows all hosts to run independently of each other. “free” lets 
each
+#       host run independently, starting its next task as soon as it finishes
+#       the previous one, regardless of how far other hosts have gotten.
+#
+# By default Ansible will attempt to run on all hosts from a play in parallel.
+# With the serial strategy ansible will attempt to run on by defauult 5 of the
+# play.
+# It is designed so that each task will be run on all hosts before continuing 
on
+# to the next task. So if you have 3 tasks it will ensure task 1 runs on all
+# your hosts first, then task 2 is run, then task 3 is run. This is the 
"linear"
+# strategy.
+# Each play contains a list of tasks. Tasks are executed in order, one at a
+# time, against all machines matched by the host pattern of the play, before 
moving on to
+# the next task. It is important to understand that, within a play, all hosts
+# are going to get the same task directives. It is the purpose of a play to map
+# a selection of hosts to tasks.

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/check-env.sh
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/check-env.sh 
b/vagrant/libvirt/dcos/check-env.sh
new file mode 100644
index 0000000..d4c0d9a
--- /dev/null
+++ b/vagrant/libvirt/dcos/check-env.sh
@@ -0,0 +1,29 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+echo
+echo "This Vagrant environment is ready for the following settings:"
+echo
+
+[ -z $DCOS_ARCH ] || echo "- DCOS_ARCH: $DCOS_ARCH"
+[ -z $DCOS_VERSION ] || echo "- DCOS_VERSION: $DCOS_VERSION"
+[ -z $HADOOP_VERSION ] || echo "- HADOOP_VERSION: $HADOOP_VERSION"
+
+echo
+echo "'vagrant up --provider=libvirt' and happy hacking!"

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/config/cluster-1m4a1p.yaml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/config/cluster-1m4a1p.yaml 
b/vagrant/libvirt/dcos/config/cluster-1m4a1p.yaml
new file mode 100644
index 0000000..ecef066
--- /dev/null
+++ b/vagrant/libvirt/dcos/config/cluster-1m4a1p.yaml
@@ -0,0 +1,70 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: bt
+  type: bootstrap
+  box: centos/7
+  ip: 100.1.10.100
+  mem: 2048
+  cpu: 2
+- name: m1
+  type: master
+  box: centos/7
+  ip: 100.1.10.101
+  mem: 2048
+  cpu: 2
+- name: a1
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.102
+  mem: 2048
+  cpu: 2
+- name: a2
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.103
+  mem: 2048
+  cpu: 2
+  aditional_disk:
+    - 50G
+    - 50G
+- name: a3
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.104
+  mem: 2048
+  cpu: 2
+  aditional_disk:
+    - 50G
+    - 50G
+- name: a4
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.105
+  mem: 2048
+  cpu: 2
+  aditional_disk:
+    - 50G
+    - 50G
+- name: p1
+  type: agent-public
+  box: centos/7
+  ip: 100.1.10.106
+  mem: 2048
+  cpu: 2

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/config/cluster-3m4a1p.yaml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/config/cluster-3m4a1p.yaml 
b/vagrant/libvirt/dcos/config/cluster-3m4a1p.yaml
new file mode 100644
index 0000000..48d438f
--- /dev/null
+++ b/vagrant/libvirt/dcos/config/cluster-3m4a1p.yaml
@@ -0,0 +1,82 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: bt
+  type: bootstrap
+  box: centos/7
+  ip: 100.1.10.100
+  mem: 512
+  cpu: 1
+- name: m1
+  type: master
+  box: centos/7
+  ip: 100.1.10.101
+  mem: 1024
+  cpu: 2
+- name: m2
+  type: master
+  box: centos/7
+  ip: 100.1.10.102
+  mem: 1024
+  cpu: 2
+- name: m3
+  type: master
+  box: centos/7
+  ip: 100.1.10.103
+  mem: 1024
+  cpu: 2
+- name: a1
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.104
+  mem: 2048
+  cpu: 2
+- name: a2
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.105
+  mem: 2048
+  cpu: 2
+  aditional_disk:
+    - 50G
+    - 50G
+- name: a3
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.106
+  mem: 2048
+  cpu: 2
+  aditional_disk:
+    - 50G
+    - 50G
+- name: a4
+  type: agent-private
+  box: centos/7
+  ip: 100.1.10.107
+  mem: 2048
+  cpu: 2
+  aditional_disk:
+    - 50G
+    - 50G
+- name: p1
+  type: agent-public
+  box: centos/7
+  ip: 100.1.10.108
+  mem: 2048
+  cpu: 2

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/group_vars/all/dcos-versions.yaml
----------------------------------------------------------------------
diff --git 
a/vagrant/libvirt/dcos/provisioning/group_vars/all/dcos-versions.yaml 
b/vagrant/libvirt/dcos/provisioning/group_vars/all/dcos-versions.yaml
new file mode 100644
index 0000000..c4d5dcb
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/group_vars/all/dcos-versions.yaml
@@ -0,0 +1,29 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+versions:
+  '1.11.0':
+    channel: 'stable/1.11.0'
+    ref: 'b6d6ad4722600877fde2860122f870031d109da3'
+    sha256: 'f42800c773a79c8fa4b5ae837626805ac67420d388433bbc9891520b17668ba9'
+    cli: 'dcos-1.11'
+  '1.10.4':
+    channel: 'stable/1.10.4'
+    ref: '2d45a8f9e277a60007f277f70f01d076c913a7fe'
+    sha256: '108b0d66af1a96508219627f7dd897b3110235eb1d9729ae5a576025192ccb61'
+    cli: 'dcos-1.10'

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/group_vars/all/defaults.yml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/group_vars/all/defaults.yml 
b/vagrant/libvirt/dcos/provisioning/group_vars/all/defaults.yml
new file mode 100644
index 0000000..07e69e5
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/group_vars/all/defaults.yml
@@ -0,0 +1,44 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+# Minimal CentOS system packages
+system_packages:
+  - nc
+  - tar
+  - wget
+  - git
+  - docker-python
+  - epel-release
+  - vim
+  - tree
+  - ntp
+  - net-tools
+  - bind-utils
+  - xz 
+  - unzip 
+  - curl 
+  - ipset
+  - sysstat
+
+system_epel_packages:
+  - pdsh  
+  - sshpass
+
+with_dcos_version: "{{ lookup('env','DCOS_VERSION') }}"
+with_hadoop_version: "{{ lookup('env','HADOOP_VERSION') }}"

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/inventory.py
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/inventory.py 
b/vagrant/libvirt/dcos/provisioning/inventory.py
new file mode 100755
index 0000000..bee7313
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/inventory.py
@@ -0,0 +1,232 @@
+#!/usr/bin/env python2
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+import yaml
+import os
+import sys
+import paramiko
+import socket
+import time
+import logging
+
+
+def get_cluster_config_file():
+    '''Returns the file used as DCOS architecture'''
+    try:
+        ret = os.environ["DCOS_ARCH"]
+    except KeyError:
+        logging.debug(
+            "[Ansible] Please set environment variable DCOS_ARCH")
+        sys.exit(1)
+
+    return "config/cluster-" + ret + ".yaml"
+
+
+def get_cluster_config_yml():
+    '''Returns a list with hostname and IP'''
+    with open(get_cluster_config_file(), 'r') as f:
+        ret = yaml.load(f)
+
+    return ret
+
+
+def check_ssh(ip, user, key_file, initial_wait=0, interval=0, retries=1):
+    logging.debug("[Ansible] checking SSH availability for %s", ip)
+    ssh = paramiko.SSHClient()
+    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+
+    time.sleep(initial_wait)
+
+    for x in range(retries):
+        try:
+            ssh.connect(ip, username=user, key_filename=key_file)
+            return True
+        except (paramiko.BadHostKeyException,
+                paramiko.AuthenticationException,
+                paramiko.SSHException,
+                socket.error) as e:
+            logging.debug(e)
+            time.sleep(interval)
+
+    return False
+
+
+def check_ssh_available(cluster_yml):
+    ssh_key_path = os.environ["HOME"] + "/.vagrant.d/insecure_private_key"
+
+    # check if the hosts are ssh accesibles
+    for item in cluster_yml:
+        if check_ssh(item.get('ip'),
+                     "vagrant",
+                     ssh_key_path,
+                     2, 2, 3):
+            logging.debug(
+                "[Ansible] %s: SSH is OK for provisioning", item)
+        else:
+            logging.debug("[Ansible] %s: SSH not ready", item)
+            return False
+
+    return True
+
+
+logging.basicConfig(filename='.vagrant/inventory.log', level=logging.DEBUG)
+
+all_vm_accesibles = False
+logging.debug('[Ansible] getting host list from configuration')
+cluster_yml = get_cluster_config_yml()
+
+
+logging.debug("[Ansible] Sanity check loop for Ansible hosts")
+while not all_vm_accesibles:
+    logging.debug("[Ansible] Waiting for SSH to become available in all hosts")
+    if check_ssh_available(cluster_yml):
+        all_vm_accesibles = True
+
+
+class InventoryTemplate:
+    '''
+    {
+    "all": {
+        "hosts": ["bt","a1","a2","p1","m1","m2","m3"],
+        "vars": {
+                    "ansible_user": "vagrant",
+                    "ansible_become": "true"
+                }
+    },
+    "bootstrap": {
+        "hosts": ["bt"]
+    },
+    "dcos-masters": {
+        "hosts": ["m1","m3","m2"]
+    },
+    "dcos-agents-private": {
+        "hosts": ["a1","a2"]
+    },
+    "dcos-agents-public": {
+        "hosts": ["p1"]
+    },
+    "_meta": {
+        "hostvars": {
+               "bt": {"ansible_host": "192.168.121.56"},
+               "p1": {"ansible_host": "192.168.121.30"},
+               "a1": {"ansible_host": "192.168.121.248"},
+               "a2": {"ansible_host": "192.168.121.112"},
+               "m1": {"ansible_host": "192.168.121.80"},
+               "m2": {"ansible_host": "192.168.121.105"},
+               "m3": {"ansible_host": "192.168.121.150"}
+            }
+        }
+    }
+    '''
+
+    _template = """
+    {
+        "all": {
+            "hosts": [%(_get_all|_pattern_a)s],
+            "vars": {
+                "ansible_user": "vagrant",
+                "ansible_become": "true"
+            }
+        },
+        "bootstrap": {
+            "hosts": ["bt"]
+        },
+        "dcos-masters": {
+            "hosts": [%(_get_masters|_pattern_a)s]
+        },
+        "dcos-agents-private": {
+            "hosts": [%(_get_privates|_pattern_a)s]
+        },
+        "dcos-agents-public": {
+            "hosts": [%(_get_publics|_pattern_a)s]
+        },
+        "_meta": {
+            "hostvars": {
+                %(_get_all_hostip|_pattern_b)s
+            }
+        }
+    }
+    """
+
+    def __init__(self, dict={}):
+        self.dict = dict
+
+    def __str__(self):
+        return self._template % self
+
+    def __getitem__(self, key):
+        return self._process(key.split("|"))
+
+    def _process(self, l):
+        arg = l[0]
+        if len(l) == 1:
+            if arg in self.dict:
+                return self.dict[arg]
+            elif hasattr(self, arg) and callable(getattr(self, arg)):
+                return getattr(self, arg)()
+            else:
+                raise KeyError(arg)
+        else:
+            func = l[1]
+            return getattr(self, func)(self._process([arg]))
+
+    def _get_all(self):
+        cad = []
+        for i in cluster_yml:
+            cad.append(i.get('name'))
+        return cad
+
+    def _get_masters(self):
+        cad = []
+        for i in cluster_yml:
+            if i.get('type') == 'master':
+                cad.append(i.get('name'))
+        return cad
+
+    def _get_privates(iself):
+        cad = []
+        for i in cluster_yml:
+            if i.get('type') == 'agent-private':
+                cad.append(i.get('name'))
+        return cad
+
+    def _get_publics(iself):
+        cad = []
+        for i in cluster_yml:
+            if i.get('type') == 'agent-public':
+                cad.append(i.get('name'))
+        return cad
+
+    def _get_all_hostip(self):
+        cad = []
+        for i in cluster_yml:
+            cad.append("\"%s\": {\"ansible_host\": \"%s\"}" % (i.get('name'),
+                                                               i.get('ip')))
+        return cad
+
+    def _pattern_a(self, l):
+        return ",".join(["\"%s\"" % x for x in l])
+
+    def _pattern_b(self, l):
+        return ",".join(["%s" % x for x in l])
+
+
+print InventoryTemplate()

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/common/handlers/main.yml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/roles/common/handlers/main.yml 
b/vagrant/libvirt/dcos/provisioning/roles/common/handlers/main.yml
new file mode 100644
index 0000000..869f483
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/common/handlers/main.yml
@@ -0,0 +1,21 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: Restart ntp
+  service: name=ntpd state=restarted

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/common/tasks/main.yml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/roles/common/tasks/main.yml 
b/vagrant/libvirt/dcos/provisioning/roles/common/tasks/main.yml
new file mode 100644
index 0000000..f814085
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/common/tasks/main.yml
@@ -0,0 +1,78 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: Sanity checking Ansible version
+  run_once: true
+  delegate_to: localhost
+  assert:
+    that: 
+      - ansible_version.full|version_compare('2.2.0.0', '>')
+    msg: "Please install at least ansible 2.2.x.x in your host"
+
+- name: Install CentOS prerequisites
+  yum: name={{ item }} state=latest
+  with_items:
+    - "{{ system_packages }}"
+
+- name: Install CentOS EPEL prerequisites
+  yum: name={{ item }} state=latest
+  with_items:
+    - "{{ system_epel_packages }}"
+
+- name: Disabling SELinux (anti-pattern)
+  selinux:
+    policy: targeted
+    state: permissive
+
+- name: Effective disabling of SELinux (avoiting reboot)
+  command: /usr/sbin/setenforce 0
+
+- name: Adding group nogroup (similar to nobody)
+  group:
+    name: nogroup
+    state: present
+
+- name: Stopping Firewall rules (anti-pattner)
+  service: name=firewalld state=stopped enabled=no
+
+- name: Effective disabling of iptables rules
+  command: /usr/sbin/iptables -F
+
+- name: Disabling Chronyd NTP server
+  service: name=chronyd state=stopped enabled=no
+
+- name: Passwordless pre-step PasswordAuthentication enabling
+  lineinfile:
+    dest: /etc/ssh/sshd_config
+    backup: yes
+    regexp: '^PasswordAuthentication'
+    line: 'PasswordAuthentication yes'
+
+- name: Restarting SSH daemon
+  service: name=sshd state=restarted
+
+- name: Configure /etc/host files
+  template: src=hosts.j2 dest=/etc/hosts
+
+- name: Configure ntp servers
+  template: src=ntp.conf.j2 dest=/etc/ntp.conf
+  notify: Restart ntp
+
+- name: Enabled ntpd service
+  service: name=ntpd enabled=yes

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/common/templates/hostname-fix.j2
----------------------------------------------------------------------
diff --git 
a/vagrant/libvirt/dcos/provisioning/roles/common/templates/hostname-fix.j2 
b/vagrant/libvirt/dcos/provisioning/roles/common/templates/hostname-fix.j2
new file mode 100644
index 0000000..6a39a55
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/common/templates/hostname-fix.j2
@@ -0,0 +1,23 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+if ! grep -e "^127\.0\.1\.1.*`hostname`.*" /etc/hosts > /dev/null ; then
+  echo "127.0.1.1 `hostname`" >> /etc/hosts
+fi

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/common/templates/hosts.j2
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/roles/common/templates/hosts.j2 
b/vagrant/libvirt/dcos/provisioning/roles/common/templates/hosts.j2
new file mode 100644
index 0000000..b273ea1
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/common/templates/hosts.j2
@@ -0,0 +1,27 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+# {{ ansible_managed }}
+127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
+::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
+
+{% for item in play_hosts %}
+{% set short_name = item.split('.') %}
+{{ hostvars[item]['ansible_host'] }} {{ item }} 
+{% endfor %}

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/common/templates/ntp.conf.j2
----------------------------------------------------------------------
diff --git 
a/vagrant/libvirt/dcos/provisioning/roles/common/templates/ntp.conf.j2 
b/vagrant/libvirt/dcos/provisioning/roles/common/templates/ntp.conf.j2
new file mode 100644
index 0000000..0c54033
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/common/templates/ntp.conf.j2
@@ -0,0 +1,25 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+driftfile /var/lib/ntp/ntp.drift
+
+server 0.pool.ntp.org
+server 1.pool.ntp.org
+server 2.pool.ntp.org
+server 3.pool.ntp.org

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/dcos-install/tasks/main.yml
----------------------------------------------------------------------
diff --git 
a/vagrant/libvirt/dcos/provisioning/roles/dcos-install/tasks/main.yml 
b/vagrant/libvirt/dcos/provisioning/roles/dcos-install/tasks/main.yml
new file mode 100644
index 0000000..5ec6580
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/dcos-install/tasks/main.yml
@@ -0,0 +1,43 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: Waiting for bootstrap healthy state
+  wait_for:
+    host: bt 
+    port: 8080
+    delay: 5
+    timeout: 1200
+  tags: install
+
+- name: Download dcos_install.sh DC/OS Installer 
+  get_url:
+    url: http://bt:8080/dcos_install.sh
+    dest: /home/vagrant/dcos_install.sh
+    mode: 0655
+  tags: install
+
+# master, slave, slave_public
+- name: Executing DC/OS Node Installation
+  shell: >
+    bash dcos_install.sh "{{ node_type }}" && 
+    touch .skip
+  args:
+    chdir: /home/vagrant
+    creates: .skip
+  tags: install

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/dcos-setup/tasks/main.yml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/roles/dcos-setup/tasks/main.yml 
b/vagrant/libvirt/dcos/provisioning/roles/dcos-setup/tasks/main.yml
new file mode 100644
index 0000000..35e8af7
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/dcos-setup/tasks/main.yml
@@ -0,0 +1,130 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: Install CentOS prerequisites for Docker Ansible Module
+  yum: name={{ item }} state=latest
+  with_items:
+    - docker-python
+
+- name: Start and enable DNSMasq in Bootstrap
+  service: name=dnsmasq state=started enabled=yes
+
+- name: Pull NGINX Docker Image
+  docker_image:
+    name: nginx
+
+- name: Create working folder (genconf)
+  file: path=/home/vagrant/genconf state=directory
+
+- name: Download dcos_generate_config.sh installer (~1GB this process might 
takes a long time)
+  shell: >
+    curl -fLsS --retry 20 -Y 100000 -y 60 
+    https://downloads.dcos.io/dcos/{{ versions[with_dcos_version].channel 
}}/commit/{{ versions[with_dcos_version].ref }}/dcos_generate_config.sh 
+    -o dcos_generate_config.sh && touch .installerskip
+  args:
+    chdir: /home/vagrant
+    creates: .installerskip
+
+- name: Configure ip-detect helper for correct interface attaching
+  blockinfile:
+    dest: /home/vagrant/genconf/ip-detect
+    create: yes
+    marker: REMOVE
+    block: |
+      #!/usr/bin/env bash
+      set -o nounset -o errexit
+      export PATH=/usr/sbin:/usr/bin:$PATH
+      echo $(ip addr show eth1 | grep -Eo 
'[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | head -1)
+
+- name: Configure config.yaml for DC/OS deployment
+  blockinfile:
+    dest: /home/vagrant/genconf/config.yaml
+    create: yes
+    marker: REMOVE
+    block: |
+      ---
+      bootstrap_url: http://bt:8080
+      cluster_name: 'dcos-vagrant'
+      exhibitor_storage_backend: static
+      ip_detect_filename: /genconf/ip-detect
+      master_discovery: static
+      master_list:
+      - 100.1.10.101
+      resolvers:
+      - 8.8.8.8
+      ssh_port: 22
+      ssh_user: vagrant
+      superuser_username: admin
+      superuser_password_hash: 
$6$rounds=656000$J1uOqAb95/iblDzm$ejRz.4wZiBJyKK0Wlmv4zH4c1bRbO0/P1KbOLSm8NJfQChzWcDJ.twmbCq9LS83bWrt7S6ZaOzicwN/bBnC3S1
+      check_time: false
+
+- name: Remove markers from module blockinfile (workaround)
+  lineinfile: 
+    dest: "{{ item }}"
+    regexp: "^REMOVE"
+    state: absent
+  with_items:
+    - "/home/vagrant/genconf/ip-detect"
+    - "/home/vagrant/genconf/config.yaml"
+
+- name: Generating DC/OS config
+  shell: bash dcos_generate_config.sh && touch .skip
+  args:
+    chdir: /home/vagrant
+    creates: .skip
+
+# FIXME: hardcoded IP of bootstrap node
+- name: Running dockerized Nginx for installation
+  docker_container:
+    name: nginx-bootstrap
+    image: nginx
+    ports:
+      - "100.1.10.100:8080:80"
+    volumes:
+      - /home/vagrant/genconf/serve:/usr/share/nginx/html
+
+- name: Downloading DC/OS CLI
+  shell: >
+    curl -fLsS --retry 20 -Y 100000 -y 60 
+    https://downloads.dcos.io/binaries/cli/linux/x86-64/dcos-1.11/dcos
+    https://downloads.dcos.io/binaries/cli/linux/x86-64/{{ 
versions[with_dcos_version].cli }}/dcos
+    -o dcos && touch .cliskip
+  args:
+    chdir: /home/vagrant
+    creates: .cliskip
+  tags: cli
+
+- name: Copy DC/OS CLI to suitable PATH location
+  shell: cp -f dcos /usr/local/bin/dcos && chmod 755 /usr/local/bin/dcos && 
touch .skipcopy
+  args:
+    chdir: /home/vagrant
+    creates: .skipcopy
+  tags: cli
+
+- name: Delete DC/OS CLI downloaded
+  file: 
+    path: /home/vagrant/dcos
+    state: absent
+  tags: cli
+
+- name: Configure current configuration for master
+  shell: /usr/local/bin/dcos config set core.dcos_url http://m1
+  become: yes
+  become_user: vagrant
+  tags: cli

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/docker/tasks/main.yml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/roles/docker/tasks/main.yml 
b/vagrant/libvirt/dcos/provisioning/roles/docker/tasks/main.yml
new file mode 100644
index 0000000..e9e0b68
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/docker/tasks/main.yml
@@ -0,0 +1,63 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: Setup OverlayFS storage driver automatic loading
+  lineinfile: 
+    dest: /etc/modules-load.d/overlay.conf
+    create: yes
+    state: present
+    line: overlay
+
+- name: Load overlay kernel module (avoiting reboot)
+  modprobe: name=overlay state=present
+
+- name: Setup offial Docker repository
+  blockinfile:
+    dest: /etc/yum.repos.d/docker.repo
+    backup: yes
+    create: yes
+    block: |
+      [dockerrepo]
+      name=Docker Repository
+      baseurl=https://yum.dockerproject.org/repo/main/centos/7/
+      enabled=1
+      gpgcheck=1
+      gpgkey=https://yum.dockerproject.org/gpg
+
+- name: Configure systemd to run the Docker Daemon with OverlayFS (folder)
+  file: path=/etc/systemd/system/docker.service.d state=directory
+
+- name: Configure systemd to run the Docker Daemon with OverlayFS (control 
file)
+  blockinfile:
+    dest: /etc/systemd/system/docker.service.d/override.conf
+    backup: yes
+    create: yes
+    block: |
+      [Service]
+      ExecStart=
+      ExecStart=/usr/bin/docker daemon --insecure-registry bt:5000 
--storage-driver=overlay -H fd://
+
+- name: Installing Docker 1.11.12 version
+  yum: name=docker-engine-1.11.2 state=installed
+
+- name: Starting Docker daemon
+  service: name=docker state=started
+
+- name: Enabling Docker daemon
+  service: name=docker enabled=yes

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/passwordless/tasks/main.yml
----------------------------------------------------------------------
diff --git 
a/vagrant/libvirt/dcos/provisioning/roles/passwordless/tasks/main.yml 
b/vagrant/libvirt/dcos/provisioning/roles/passwordless/tasks/main.yml
new file mode 100644
index 0000000..6330a85
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/passwordless/tasks/main.yml
@@ -0,0 +1,38 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: Generate RSA key-pair for passwordless
+  shell: ssh-keygen -t rsa -f .ssh/id_rsa -q -N "" && touch .skiprsa
+  become: yes
+  become_user: vagrant
+  args:
+      chdir: /home/vagrant
+      creates: .skiprsa
+
+- name: Install boot host public key in a remote authorized_keys
+  shell: >
+    sshpass -p vagrant ssh-copy-id -i .ssh/id_rsa {{ item }} -o 
StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
+  become: yes
+  become_user: vagrant
+  args:
+    chdir: /home/vagrant
+  with_inventory_hostnames:
+    - all
+
+

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/roles/registry/tasks/main.yml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/roles/registry/tasks/main.yml 
b/vagrant/libvirt/dcos/provisioning/roles/registry/tasks/main.yml
new file mode 100644
index 0000000..d365b4e
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/roles/registry/tasks/main.yml
@@ -0,0 +1,32 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: Pull Docker Registry Image
+  docker_image:
+    name: registry
+  tags: registy
+
+- name: Running dockerized Nginx for installation
+  docker_container:
+    name: registry
+    image: registry
+    ports:                                 
+    - "5000:5000"                          
+    restart_policy: always
+  tags: registry

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/provisioning/site.yml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/provisioning/site.yml 
b/vagrant/libvirt/dcos/provisioning/site.yml
new file mode 100644
index 0000000..416aadf
--- /dev/null
+++ b/vagrant/libvirt/dcos/provisioning/site.yml
@@ -0,0 +1,63 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+# - Plays are map roles to hosts.  
+# - Roles are just abstractions around tasks
+# - Each "hosts:"-block (a play) will be executed 
+#   one by one. So you can use multiple
+#   blocks to force the desired execution order.
+# - The plays are executed from top to down order.
+# - Multiple roles declarations per play are executed
+#   non in special order, you cannot control the order
+#   of roles within a play.
+
+# Play 0
+- hosts: all
+  roles:
+    - common
+    - docker
+
+# Play 1
+- hosts: bootstrap 
+  roles:
+    - passwordless
+    - dcos-setup
+    - registry
+
+# Play 2
+- hosts: dcos-masters 
+  strategy: free
+  vars:
+    node_type: master
+  roles:
+    - dcos-install
+
+- hosts: dcos-agents-private 
+  strategy: free
+  vars:
+    node_type: slave
+  roles:
+    - dcos-install
+
+- hosts: dcos-agents-public
+  strategy: free
+  vars:
+    node_type: slave_public
+  roles:
+    - dcos-install

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/requirements.txt
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/requirements.txt 
b/vagrant/libvirt/dcos/requirements.txt
new file mode 100644
index 0000000..cd696f0
--- /dev/null
+++ b/vagrant/libvirt/dcos/requirements.txt
@@ -0,0 +1 @@
+sudo dnf install ansible -y

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/dcos/setup-env
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/dcos/setup-env b/vagrant/libvirt/dcos/setup-env
new file mode 100644
index 0000000..6296221
--- /dev/null
+++ b/vagrant/libvirt/dcos/setup-env
@@ -0,0 +1,24 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+export DCOS_ARCH=1m4a1p
+export DCOS_VERSION=1.11.0
+export HADOOP_VERSION=2.7.0
+
+sh check-env.sh

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/README.md
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/README.md b/vagrant/libvirt/mesos/README.md
new file mode 100644
index 0000000..02d6584
--- /dev/null
+++ b/vagrant/libvirt/mesos/README.md
@@ -0,0 +1,245 @@
+# Plain Apache Mesos Environment
+
+This is the Vagrant-libvirt based environment for testing Apache Myriad
+project in a Mesos cluster.
+
+# Setup the environment
+
+In order to use this vagrant environment we have to enable a suitable
+environment. The key points of this environment are:
+
+- **MESOS_ARCH**: This variable has to match with the VMs architecture defined 
in
+  the config folder: *cluster-1m4a.yaml*, *cluster-3m4a.yaml*, and so on. For
+  example **MESOS_ARCH: 1m4a** matchs with the architecture 
*cluster-1m4a.yaml*, which
+  is a Mesos master node (1m), and four Mesos agents (4a).
+
+- **HADOOP_VERSION**: This variable is used for downloading a binary 
distribution
+  of Apache HAdoop.
+
+- **ZOOKEEPER_VERSION**: For dowloading an Apache Zookeeper binary 
distribution.
+
+```
+$ source setup-env 
+
+This Vagrant environment is ready for the following settings:                  
       
+
+- MESOS_ARCH: 1m4a                         
+- HADOOP_VERSION: 2.7.0                    
+- ZOOKEEPER_VERSION: 3.4.11                
+
+'vagrant up --provider=libvirt' and happy hacking!  
+``` 
+
+# Running the deployment
+
+The VMs provisioning is based on Ansible, the provisioning is guided by the
+main playbook **provisioning/site.yml** and one special 
**provisioing/inventory.py**
+for creating a dynamic inventory. This dynamic inventory is created for
+managing the Ansible provisioning with a multi-machine Vagrant environment,
+because the Vagrant Ansible support is sometime problematic.
+
+```
+$ vagrant up  --provider=libvirt
+[...]
+PLAY RECAP
+*********************************************************************
+build                      : ok=36   changed=28   unreachable=0    failed=0   
+mesos-a1                   : ok=28   changed=25   unreachable=0    failed=0   
+mesos-a2                   : ok=31   changed=28   unreachable=0    failed=0   
+mesos-a3                   : ok=31   changed=28   unreachable=0    failed=0   
+mesos-a4                   : ok=31   changed=28   unreachable=0    failed=0   
+mesos-m1                   : ok=41   changed=38   unreachable=0    failed=0 
+```
+
+## Final VMs set
+
+```
+$ vagrant status   
+Current machine states:
+
+build                     running (libvirt)
+mesos-m1                  running (libvirt)
+mesos-a1                  running (libvirt)
+mesos-a2                  running (libvirt)
+mesos-a3                  running (libvirt)
+mesos-a4                  running (libvirt)
+```
+
+The VM dedicated for development is *build*, the development is maded in the
+*/opt* folder:
+
+```
+$ vagrant ssh build
+[vagrant@build ~]$ tree -d -L 1 /opt/
+/opt/
+├── hadoop
+├── mesos
+└── myriad
+```
+
+# Building the ecosystem
+
+The strategy is to build the target Apache Mesos version from source, and 
+build Myriad framework as follow:
+
+## Building Apache Mesos
+
+```
+[vagrant@build ~]$ cd /opt/mesos
+[vagrant@build ~]$ git checkout 0.28.1
+[vagrant@build ~]$ ./bootstrap
+[vagrant@build ~]$ mkdir build
+[vagrant@build ~]$ cd build
+[vagrant@build ~]$ ../configure
+[vagrant@build ~]$ make
+[...]
+running install_scripts
+creating build/bdist.linux-x86_64/wheel/mesos.native-0.28.1.dist-info/WHEEL
+make[2]: Leaving directory `/opt/mesos/build/src'
+make[1]: Leaving directory `/opt/mesos/build/src'
+```
+
+## Building Myriad Framework
+
+```
+[vagrant@build myriad]$ cd /opt/myriad
+[vagrant@build myriad]$ ./gradlew build
+[...]
+:myriad-scheduler:test                     
+:myriad-scheduler:check                    
+:myriad-scheduler:build                    
+
+BUILD SUCCESSFUL                           
+
+Total time: 3 mins 56.948 secs
+```
+
+# Running the ecosystem
+
+The ecosystem was built from **build** VM, however the execution of ecosystem
+is carried out from **mesos-m1** VM and mesos agents.
+
+## Running Mesos 3rdparty shipped Zookeeper
+
+We have to run a Zookeeper server, we can use the shipped Zookeeper expecified
+at ZOOKEEPER_VERSION, or we can use the already shipped Zookeeper at 3rdparty
+Mesos folder. Let see who to run the 3rdparty Zookeeper shipped with Mesos.
+
+```
+$ vagrant ssh mesos-m1
+[vagrant@mesos-m1 ~]$ cd /opt/mesos/build/3rdparty/zookeeper-3.4.5/
+[vagrant@mesos-m1 zookeeper-3.4.5]$ cp conf/zoo_sample.cfg conf/zoo.cfg
+[vagrant@mesos-m1 zookeeper-3.4.5]$ echo "server.1=mesos-m1:2888:3888" >> 
conf/zoo.cfg
+[vagrant@mesos-m1 zookeeper-3.4.5]$ bin/zkServer.sh start
+[vagrant@mesos-m1 zookeeper-3.4.5]$ echo ruok | nc 127.0.0.1 2181
+imok
+[vagrant@mesos-m1 ~]$ jps
+16290 Jps
+16253 QuorumPeerMain
+```
+
+## Running Apache Mesos
+
+- At mesos-m1
+
+```
+[vagrant@mesos-m1 ~]$ cd /opt/mesos/build
+[vagrant@mesos-m1 build]$ sudo ./bin/mesos-master.sh --ip=100.0.10.101 
--work_dir=/var/lib/mesos --zk=zk://mesos-m1:2181/mesos --quorum=1
+```
+
+- At agents mesos-a[1..4]:
+
+```
+[vagrant@mesos-a1 ~]$ cd /opt/mesos/build
+[vagrant@mesos-a1 build]$ sudo ./bin/mesos-slave.sh 
--master=zk://mesos-m1:2181/mesos --work_dir=/var/lib/mesos
+```
+
+*Note*: mesos-slave.sh was changed for newer versions of Mesos for 
mesos-agent.sh
+
+The Mesos Master web interface can be accessed via the URL:
+
+http://100.0.10.101:5050
+
+## Running Hadoop HDFS
+
+The Ansible provisioning must have Apache HDFS already running in the platform.
+For checking we can do the following:
+
+- At mesos-m1 master:
+
+```
+[vagrant@mesos-m1 ~]$ su - hdfs
+Password: (vagrant is the password)
+[hdfs@mesos-m1 ~]$ jps
+21249 NameNode
+16374 Jps
+21448 SecondaryNameNode
+[hdfs@mesos-m1 hadoop]$ cd /opt/hadoop
+[hdfs@mesos-m1 hadoop]$ bin/hdfs dfsadmin -printTopology
+Rack: /default-rack
+   100.0.10.103:50010 (mesos-a2)
+   100.0.10.104:50010 (mesos-a3)
+   100.0.10.105:50010 (mesos-a4)
+[hdfs@mesos-m1 hadoop]$ bin/hdfs dfsadmin -report
+[...]
+```
+
+The NN web interface can be accessed via the URL:
+
+http://100.0.10.101:50070/
+
+*Note*: Hadoop HDFS is running in mesos-a2, mesos-a3 and mesos-a4. 
+The Mesos agent mesos-a1 is dedicated for running Mesos tasks not related 
+with Hadoop.
+
+*Note*: You can stop or start HDFS manually from master node with:
+
+```
+[hdfs@mesos-m1 hadoop]$ cd /opt/hadoop
+[hdfs@mesos-m1 hadoop]$ sbin/stop-dfs.sh
+[hdfs@mesos-m1 hadoop]$ sbin/stop-dfs.sh
+```
+
+
+## Running Hadoop YARN Resource Manager (RM) with Myriad
+
+- At mesos-m1:
+
+```
+cd /opt
+
+# Copy the Myriad Scheduler
+cp -vf myriad/myriad-scheduler/build/libs/* hadoop/share/hadoop/yarn/lib/
+
+#  Copy the Myriad Executor
+cp -vf myriad/myriad-executor/build/libs/myriad-executor-0.2.0.jar 
hadoop/share/hadoop/yarn/lib/
+
+# Hadoop 2.7.x:
+$ su - yarn
+Password: (vagrant is the password)
+[yarn@mesos-m1 ~]$ cd /opt/hadoop/
+[yarn@mesos-m1 hadoop]$ sbin/yarn-daemon.sh --config /opt/hadoop/etc/hadoop/ 
start resourcemanager
+[yarn@mesos-m1 hadoop]$ jps                
+19283 ResourceManager                      
+19519 Jps 
+```
+
+The YARN web interface can be accessed via the URL:
+
+http://100.0.10.101:8088
+
+The Myriad web interface can be accessd via the URL:
+
+http://100.0.10.101:8192
+
+# Happy Hacking
+
+At this point you have a local development environment ready for Myriad
+Framework hacking!
+
+
+
+
+
+
+

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/Vagrantfile
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/Vagrantfile 
b/vagrant/libvirt/mesos/Vagrantfile
new file mode 100644
index 0000000..c5cc2af
--- /dev/null
+++ b/vagrant/libvirt/mesos/Vagrantfile
@@ -0,0 +1,92 @@
+# -*- mode: ruby -*-
+# vim: set ft=ruby ts=2 et :
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+VAGRANTFILE_API_VERSION = "2"
+
+# Tested with Vagrant version:
+Vagrant.require_version ">= 1.7.2"
+
+# Require YAML module
+require 'yaml'
+
+def fail_with_message(msg)
+    fail Vagrant::Errors::VagrantError.new, msg
+end
+
+def check_minimal_env()
+    if ENV['MESOS_ARCH'].nil? || ENV['MESOS_ARCH'].empty?
+        fail_with_message "You have to set MESOS_ARCH environment variable!"
+    end
+    if ENV['HADOOP_VERSION'].nil? || ENV['HADOOP_VERSION'].empty?
+        fail_with_message "You have to set HADOOP_VERSION environment 
variable!"
+    end
+    if ENV['ZOOKEEPER_VERSION'].nil? || ENV['ZOOKEEPER_VERSION'].empty?
+        fail_with_message "You have to set ZOOKEEPER environment variable!"
+    end
+end
+
+# Sanity checks
+check_minimal_env()
+
+# Read YAML file with cluster details: The cluster flavor.
+config_file = 'config/cluster-' + ENV['MESOS_ARCH'] + ".yaml"
+if File.exists?(config_file)
+    cluster = YAML.load_file(config_file)
+else
+    fail_with_message "Cluster definition file: #{config_file} was not found!"
+end
+
+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+  # Disabling the default /vagrant share
+  config.vm.synced_folder ".", "/vagrant", disabled: true
+  config.ssh.insert_key = false
+
+  i = 0
+  cluster.each do |servers|
+    # VM definition
+    config.vm.define servers["name"] do |node|
+      node.vm.box = servers["box"]
+      node.vm.hostname = servers["name"]
+      node.vm.network "private_network", ip: servers["ip"]
+      node.vm.provider :libvirt do |domain|
+        domain.uri = 'qemu+unix:///system'
+        domain.driver = 'kvm'
+        domain.memory = servers["mem"]
+        domain.cpus = servers['cpu']
+        if servers.key?('aditional_disk')
+            for disk in servers['aditional_disk']
+                domain.storage :file, :size => disk
+            end
+        end
+      end
+      i += 1
+      if i == cluster.length
+          node.vm.provision :ansible do |ansible|
+            ansible.verbose = "vv"
+            ansible.inventory_path = "provisioning/inventory.py"
+            ansible.playbook = "provisioning/site.yml"
+            ansible.limit = "all"
+          end # provison
+      end # if
+    end # config
+  end # cluster
+end # vagrant

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/ansible.cfg
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/ansible.cfg 
b/vagrant/libvirt/mesos/ansible.cfg
new file mode 100644
index 0000000..acbdc2e
--- /dev/null
+++ b/vagrant/libvirt/mesos/ansible.cfg
@@ -0,0 +1,79 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+[defaults]
+# Disable host key checking. If a host is not initially in ‘known_hosts’ 
this
+# will result in prompting for confirmation of the key, which results in an
+# interactive experience 
+host_key_checking = False
+
+# This is the default SSH timeout to use on connection attempts.
+timeout = 30
+
+# This options forces color mode even when running without a TTY.
+force_color = 1
+
+# Number of parallel processes to spawn when 
+# communicating with remote hosts. 
+forks = 10
+
+# Facts gathering: The value ‘smart’ means each new 
+# host that has no facts discovered will be scanned, 
+# but if the same host is addressed in multiple plays 
+# it will not be contacted again in the playbook run. 
+# This enhance fact gathering time.
+gathering = smart
+
+# Ansible will log information about executions at the designated location
+log_path=.vagrant/ansible.log
+
+# The retry files save path is where Ansible will save .retry files when a
+# playbook fails. The directory will be created if it does not already exist.
+retry_files_save_path = .vagrant/retry-files
+
+#
+# Play execution strategies explained:
+#
+# Plays execution control for many hosts, controls how plays run their multiple
+# hosts:
+#
+# linear (default): All hosts of a play have to finish one task before any of 
them can 
+#                   begin the next one. Using the number of forks (default 5)
+#                   to parallelize.
+# serial: Ensures one group of hosts in a play finishes its work before 
another group 
+#         can begin. This strategy will take a subset of hosts (the default is
+#         five) and execute all tasks (in the linear fashion) against those 
+#         hosts before moving to the next subset and starting from the 
beginning.
+# free: Allows each host to run until the end of the play as fast as it can.
+#       Allows all hosts to run independently of each other. “free” lets 
each
+#       host run independently, starting its next task as soon as it finishes
+#       the previous one, regardless of how far other hosts have gotten.
+#
+# By default Ansible will attempt to run on all hosts from a play in parallel.
+#
+# With the serial strategy ansible will attempt to run on by defauult 5 of the 
play.
+# It is designed so that each task will be run on all hosts before continuing 
on
+# to the next task. So if you have 3 tasks it will ensure task 1 runs on all
+# your hosts first, then task 2 is run, then task 3 is run. This is the 
"linear" strategy.
+#
+# Each play contains a list of tasks. Tasks are executed in order, one at a
+# time, against all machines matched by the host pattern of the play, before 
moving on to
+# the next task. It is important to understand that, within a play, all hosts
+# are going to get the same task directives. It is the purpose of a play to map
+# a selection of hosts to tasks.
+

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/check-env.sh
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/check-env.sh 
b/vagrant/libvirt/mesos/check-env.sh
new file mode 100644
index 0000000..c51ae75
--- /dev/null
+++ b/vagrant/libvirt/mesos/check-env.sh
@@ -0,0 +1,28 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+echo
+echo "This Vagrant environment is ready for the following settings:"
+echo
+
+[ -z $MESOS_ARCH ] || echo "- MESOS_ARCH: $MESOS_ARCH"
+[ -z $HADOOP_VERSION ] || echo "- HADOOP_VERSION: $HADOOP_VERSION"
+[ -z $ZOOKEEPER_VERSION ] || echo "- ZOOKEEPER_VERSION: $ZOOKEEPER_VERSION"
+
+echo
+echo "'vagrant up --provider=libvirt' and happy hacking!"

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/config/cluster-1m4a.yaml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/config/cluster-1m4a.yaml 
b/vagrant/libvirt/mesos/config/cluster-1m4a.yaml
new file mode 100644
index 0000000..3164e9f
--- /dev/null
+++ b/vagrant/libvirt/mesos/config/cluster-1m4a.yaml
@@ -0,0 +1,64 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: build
+  type: build
+  box: centos/7
+  ip: 100.0.10.100
+  mem: 4096
+  cpu: 2
+- name: mesos-m1
+  type: master
+  box: centos/7
+  ip: 100.0.10.101
+  mem: 2048
+  cpu: 2
+- name: mesos-a1
+  type: agent
+  box: centos/7
+  ip: 100.0.10.102
+  mem: 2048
+  cpu: 2
+- name: mesos-a2
+  type: agent
+  box: centos/7
+  ip: 100.0.10.103
+  mem: 2048
+  cpu: 2
+  aditional_disk:
+    - 50G
+    - 50G
+- name: mesos-a3
+  type: agent
+  box: centos/7
+  mem: 2048
+  ip: 100.0.10.104
+  aditional_disk:
+    - 50G
+    - 50G
+  cpu: 2
+- name: mesos-a4
+  type: agent
+  box: centos/7
+  mem: 2048
+  ip: 100.0.10.105
+  aditional_disk:
+    - 50G
+    - 50G
+  cpu: 2

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/config/cluster-3m4a.yaml
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/config/cluster-3m4a.yaml 
b/vagrant/libvirt/mesos/config/cluster-3m4a.yaml
new file mode 100644
index 0000000..cb51758
--- /dev/null
+++ b/vagrant/libvirt/mesos/config/cluster-3m4a.yaml
@@ -0,0 +1,76 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+---
+- name: build
+  type: build
+  box: centos/7
+  ip: 100.0.10.100
+  mem: 2048
+  cpu: 2
+- name: mesos-m1
+  type: master
+  box: centos/7
+  ip: 100.0.10.101
+  mem: 2048
+  cpu: 2
+- name: mesos-m2
+  type: master
+  box: centos/7
+  ip: 100.0.10.102
+  mem: 2048
+  cpu: 2
+- name: mesos-m3
+  type: master
+  box: centos/7
+  ip: 100.0.10.103
+  mem: 2048
+  cpu: 2
+- name: mesos-a1
+  type: agent
+  box: centos/7
+  ip: 100.0.10.104
+  mem: 2048
+  cpu: 2
+- name: mesos-a2
+  type: agent
+  box: centos/7
+  ip: 100.0.10.105
+  mem: 2048
+  cpu: 2
+  aditional_disk: 
+    - 50G
+    - 50G
+- name: mesos-a3
+  type: agent
+  box: centos/7
+  ip: 100.0.10.106
+  mem: 1024
+  cpu: 2
+  aditional_disk: 
+    - 50G
+    - 50G
+- name: mesos-a4
+  type: agent
+  box: centos/7
+  mem: 2048
+  ip: 100.0.10.107
+  aditional_disk:
+    - 50G
+    - 50G
+  cpu: 2

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/docs/ansible.txt
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/docs/ansible.txt 
b/vagrant/libvirt/mesos/docs/ansible.txt
new file mode 100644
index 0000000..d2c1d5b
--- /dev/null
+++ b/vagrant/libvirt/mesos/docs/ansible.txt
@@ -0,0 +1 @@
+ansible-playbook --private-key $HOME/.vagrant.d/insecure_private_key -i 
provisioning/inventory.py provisioning/site.yml --tags hadoop

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/docs/build-mesos.md
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/docs/build-mesos.md 
b/vagrant/libvirt/mesos/docs/build-mesos.md
new file mode 100644
index 0000000..99981f4
--- /dev/null
+++ b/vagrant/libvirt/mesos/docs/build-mesos.md
@@ -0,0 +1,135 @@
+# Building Apache Mesos
+
+Note: Last tested Mesos version for Myriad -> 0.28.1
+
+Notes about build Apache Mesos and some development hacking notes.
+
+# Building with autotools system
+
+```
+# Change working directory.
+$ cd mesos
+
+# Bootstrap (Only required if building from git repository).
+$ ./bootstrap
+
+# Configure and build.
+$ mkdir build
+$ cd build
+$ ../configure
+$ make
+
+# Run test suite.
+$ make check
+
+# Install (Optional).
+$ make install
+```
+
+# Building with new CMake system
+
+```
+curl -O https://cmake.org/files/v3.9/cmake-3.9.4-Linux-x86_64.tar.gz
+tar xvzf cmake-3.9.4-Linux-x86_64.tar.gz 
+export PATH=$PATH:$HOME/cmake-3.9.4-Linux-x86_64/bin
+
+cd mesos
+mkdir build && cd build
+cmake ..
+cmake --build .
+
+
+For cleaning generated build system files:
+
+rm build/CMakeCache.txt
+```
+
+# Running Apache Mesos
+
+## Master
+
+```
+cd /home/vagrant/mesos.git/build
+sudo ./bin/mesos-master.sh --ip=100.0.10.101 --work_dir=/var/lib/mesos
+```
+
+## Agents
+
+```
+cd /home/vagrant/mesos.git/build
+sudo ./bin/mesos-agent.sh --master=100.0.10.101:5050 --work_dir=/var/lib/mesos
+```
+
+## Running with 3rdparty shipped Zookeeper
+
+At master:
+
+```
+cd /home/vagrant/mesos.git/build/3rdparty/zookeeper-3.4.8
+cp conf/zoo_sample.cfg conf/zoo.cfg
+echo "server.1=mesos-m1:2888:3888" >> conf/zoo.cfg
+bin/zkServer.sh start
+echo ruok | nc 127.0.0.1 2181
+
+sudo ./bin/mesos-master.sh --ip=100.0.10.101 --work_dir=/var/lib/mesos 
--zk=zk://mesos-m1:2181/mesos --quorum=1
+```
+
+At agents:
+
+```
+sudo ./bin/mesos-agent.sh --master=zk://mesos-m1:2181/mesos 
--work_dir=/var/lib/mesos
+```
+
+# Debugging
+
+```
+gdbserver -> exec ${LIBTOOL} --mode=execute gdbserver 100.0.10.101:1234 \
+  /home/vagrant/mesos.git/build/src/mesos-master "${@}"
+
+
+(gdb) target remote mesos-m1:1234
+(gdb) continue
+(gdb) monitor exit
+```
+
+# Development Environment with Vim
+
+
+## Building clang-format
+
+```
+svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
+cd llvm/tools
+svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
+cd ../..
+mkdir build
+cd build
+cmake -G "Unix Makefiles" ../llvm
+```
+
+## Code navigation: cscope and tags
+
+```
+sudo yum install cscope ctags -y
+
+cd build
+make cscope
+make ctags
+```
+
+## Vimrc with clang-format for C++11 and Google Style
+
+```
+cd 
+curl -O 
https://raw.githubusercontent.com/javiroman/reproducible-research/master/vim-tmux/vimrc
+mv vimrc .vimrc
+mkdir -p .vim/bundle
+git clone https://github.com/VundleVim/Vundle.vim.git ~/.vim/bundle/Vundle.vim 
+vim
+:PluginInstall 
+or from command line: vim +PluginInstall +qall
+```
+
+
+
+

http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/1c3857a8/vagrant/libvirt/mesos/docs/notes-myriad.txt
----------------------------------------------------------------------
diff --git a/vagrant/libvirt/mesos/docs/notes-myriad.txt 
b/vagrant/libvirt/mesos/docs/notes-myriad.txt
new file mode 100644
index 0000000..1dd51d5
--- /dev/null
+++ b/vagrant/libvirt/mesos/docs/notes-myriad.txt
@@ -0,0 +1,11 @@
+# Copy the Myriad Scheduler
+cp -vf myriad/myriad-scheduler/build/libs/* hadoop/share/hadoop/yarn/lib/
+
+#  Copy the Myriad Executor
+cp -vf myriad/myriad-executor/build/libs/myriad-executor-0.2.0.jar 
hadoop/share/hadoop/yarn/lib/
+
+# Copy the myriad-config-default.yml
+cp myriad/myriad-scheduler/build/resources/main/myriad-config-default.yml 
hadoop/etc/hadoop/
+
+#Hadoop 2.7.x:
+#sbin/yarn-daemon.sh --config /opt/hadoop/etc/hadoop/ start resourcemanager