Hi Shameera,
I am trying to build a mesos cluster on EC2 using your playbooks. But I am
facing some issues. Please find the details below:
Details:
- I created 4 instances on EC2 (us-east-1 region) using the
cloud-provisioning module (CloudBridge python). Out of the 4, 3 were meant to
be mesos masters & 1 slave.
Note: The instance inbound & outbount traffic is wideopen.
- I skipped step-1 & step-2 in your README, since I manually
provisioned the instances. Next, I updated “hosts” file with public IPs for all
4 instances. And also updated the “roles/zookeeper/vars/main.yml” file with the
private IPs of 3 master instances.
- I executed the “ansible-playbook -i hosts site.yml -t
"mesos-master"” command, and I get the following error:
TASK [mesos-master : install firewalld] ****************************************
ok: [52.91.152.1]
ok: [52.87.235.79]
ok: [54.167.94.186]
TASK [mesos-master : start firewalld] ******************************************
ok: [52.91.152.1]
ok: [52.87.235.79]
ok: [54.167.94.186]
TASK [mesos-master : open ports] ***********************************************
ok: [52.91.152.1] => (item=5050/tcp)
ok: [52.87.235.79] => (item=5050/tcp)
ok: [54.167.94.186] => (item=5050/tcp)
ok: [52.87.235.79] => (item=8080/tcp)
ok: [54.167.94.186] => (item=8080/tcp)
ok: [52.91.152.1] => (item=8080/tcp)
TASK [mesos-master : install utility - TODO delete this] ***********************
ok: [52.91.152.1] => (item=[u'vim'])
ok: [52.87.235.79] => (item=[u'vim'])
ok: [54.167.94.186] => (item=[u'vim'])
TASK [mesos-master : add mesosphere rpm] ***************************************
ok: [52.91.152.1]
ok: [52.87.235.79]
ok: [54.167.94.186]
TASK [mesos-master : install mesos and marathon] *******************************
failed: [52.91.152.1] (item=[u'mesos', u'marathon']) => {"changed": true,
"failed": true, "item": ["mesos", "marathon"], "msg": "Error unpacking rpm
package marathon-1.3.0-1.0.506.el7.x86_64\n", "rc": 1, "results": ["All
packages providing mesos are up to date", "Loaded plugins:
fastestmirror\nLoading mirror speeds from cached hostfile\n * base:
mirrors.tripadvisor.com\n * extras: centos.hostingxtreme.com\n * updates:
mirrors.greenmountainaccess.net\nResolving Dependencies\n--> Running
transaction check\n---> Package marathon.x86_64 0:1.3.0-1.0.506.el7 will be
installed\n--> Finished Dependency Resolution\n\nDependencies
Resolved\n\n================================================================================\n
Package Arch Version Repository
Size\n================================================================================\nInstalling:\n
marathon x86_64 1.3.0-1.0.506.el7 mesosphere 17
M\n\nTransaction
Summary\n================================================================================\nInstall
1 Package\n\nTotal download size: 17 M\nInstalled size: 89 M\nDownloading
packages:\nRunning transaction check\nRunning transaction test\nTransaction
test succeeded\nRunning transaction\n Installing :
marathon-1.3.0-1.0.506.el7.x86_64 1/1 \nerror:
unpacking of archive failed on file /usr/bin/marathon;57daffff: cpio: read\n
Verifying : marathon-1.3.0-1.0.506.el7.x86_64 1/1
\n\nFailed:\n marathon.x86_64 0:1.3.0-1.0.506.el7
\n\nComplete!\n"]}
failed: [52.87.235.79] (item=[u'mesos', u'marathon']) => {"changed": true,
"failed": true, "item": ["mesos", "marathon"], "msg": "Error unpacking rpm
package marathon-1.3.0-1.0.506.el7.x86_64\n", "rc": 1, "results": ["All
packages providing mesos are up to date", "Loaded plugins:
fastestmirror\nLoading mirror speeds from cached hostfile\n * base:
mirrors.tripadvisor.com\n * extras: mirrors.evowise.com\n * updates:
mirrors.greenmountainaccess.net\nResolving Dependencies\n--> Running
transaction check\n---> Package marathon.x86_64 0:1.3.0-1.0.506.el7 will be
installed\n--> Finished Dependency Resolution\n\nDependencies
Resolved\n\n================================================================================\n
Package Arch Version Repository
Size\n================================================================================\nInstalling:\n
marathon x86_64 1.3.0-1.0.506.el7 mesosphere 17
M\n\nTransaction
Summary\n================================================================================\nInstall
1 Package\n\nTotal download size: 17 M\nInstalled size: 89 M\nDownloading
packages:\nRunning transaction check\nRunning transaction test\nTransaction
test succeeded\nRunning transaction\n Installing :
marathon-1.3.0-1.0.506.el7.x86_64 1/1 \nerror:
unpacking of archive failed on file /usr/bin/marathon;57daffff: cpio: read\n
Verifying : marathon-1.3.0-1.0.506.el7.x86_64 1/1
\n\nFailed:\n marathon.x86_64 0:1.3.0-1.0.506.el7
\n\nComplete!\n"]}
failed: [54.167.94.186] (item=[u'mesos', u'marathon']) => {"changed": true,
"failed": true, "item": ["mesos", "marathon"], "msg": "Error unpacking rpm
package marathon-1.3.0-1.0.506.el7.x86_64\n", "rc": 1, "results": ["All
packages providing mesos are up to date", "Loaded plugins:
fastestmirror\nLoading mirror speeds from cached hostfile\n * base:
mirrors.tripadvisor.com\n * extras: mirrors.evowise.com\n * updates:
mirrors.greenmountainaccess.net\nResolving Dependencies\n--> Running
transaction check\n---> Package marathon.x86_64 0:1.3.0-1.0.506.el7 will be
installed\n--> Finished Dependency Resolution\n\nDependencies
Resolved\n\n================================================================================\n
Package Arch Version Repository
Size\n================================================================================\nInstalling:\n
marathon x86_64 1.3.0-1.0.506.el7 mesosphere 17
M\n\nTransaction
Summary\n================================================================================\nInstall
1 Package\n\nTotal download size: 17 M\nInstalled size: 89 M\nDownloading
packages:\nRunning transaction check\nRunning transaction test\nTransaction
test succeeded\nRunning transaction\n Installing :
marathon-1.3.0-1.0.506.el7.x86_64 1/1 \nerror:
unpacking of archive failed on file /usr/bin/marathon;57daffff: cpio: read\n
Verifying : marathon-1.3.0-1.0.506.el7.x86_64 1/1
\n\nFailed:\n marathon.x86_64 0:1.3.0-1.0.506.el7
\n\nComplete!\n"]}
NO MORE HOSTS LEFT *************************************************************
RUNNING HANDLER [zookeeper : restart zookeeper] ********************************
[WARNING]: Could not create retry file 'site.retry'. [Errno 2] No such
file or directory: ''
PLAY RECAP *********************************************************************
52.87.235.79 : ok=17 changed=2 unreachable=0 failed=1
52.91.152.1 : ok=17 changed=2 unreachable=0 failed=1
54.167.94.186 : ok=17 changed=2 unreachable=0 failed=1
localhost : ok=1 changed=0 unreachable=0 failed=0
Is there some step that I am missing? It looks like the instances are not able
to communicate because of the firewall? This is just a wild guess. Any help
here is appreciated.
Thanks and Regards,
Gourav Shenoy
From: Shameera Rathnayaka <shameerai...@gmail.com>
Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org>
Date: Monday, September 12, 2016 at 11:19 AM
To: dev <dev@airavata.apache.org>
Subject: Spinup Mesos-Marathon Cluster for Hybrid Scheduling
Hi Dev,
The effort of getting use Cloud infrastructure to run MPI and BigData jobs
using Airavata, we use Apache Mesos as resource allocation framework to manage
different type of clusters (i.e HPC node cluster to run MPI jobs and spark,
Hadoop big data clusters to run bigdata applications). I came up with Ansible
script to spin up Mesos cluster on the target set of nodes. You can find the
script here https://github.com/shamrath/mesos-deployment I am thinking of move
this code to Airavata if all agreed. I would happy to answer any question
related to this.
Thanks,
Shameera.
--
Shameera Rathnayaka
