Hi Dev, The effort of getting use Cloud infrastructure to run MPI and BigData jobs using Airavata, we use Apache Mesos as resource allocation framework to manage different type of clusters (i.e HPC node cluster to run MPI jobs and spark, Hadoop big data clusters to run bigdata applications). I came up with Ansible script to spin up Mesos cluster on the target set of nodes. You can find the script here https://github.com/shamrath/mesos-deployment I am thinking of move this code to Airavata if all agreed. I would happy to answer any question related to this.
Thanks, Shameera. -- Shameera Rathnayaka
