Hello!

I would like to bring to your attention ElastiCluster [1] [2], a tool
for deploy verious kinds of compute clusters on IaaS clouds.  Thanks to
BigTop (and to the developers behind it!), ElastiCluster can also deploy
functional Hadoop+Spark clusters [3].

ElastiCluster does not use the BigTop provisioner, instead opts for its
own Ansible-based deployment playbooks: the provisioned software is
currently limited to Hadoop + Spark + Thriftserver (from BigTop 1.2.1),
but they can be integrated with other non-BigTop software (e.g.,
JupyterHub).

AFAIK, the main use for Hadoop+Spark on ElastiCluster so far has been
setting up small clusters for teaching purposes; I'd be glad for any
feedback, and especially if anyone is willing to try it for more
"serious" use cases, as well as discussing more general topics (here or
on the ElastiCluster mailing-list).

[1]: http://elasticluster.readthedocs.io/en/latest/
[2]: http://elasticluster.readthedocs.io/en/latest/install.html#quickstart
[3]: http://elasticluster.readthedocs.io/en/latest/playbooks.html#hadoop-spark

(I hope this kind of announcements is welcome on the list; I could find
no policy on allowed topics on the BigTop web site and the mailing list index.)

Kind regards,
Riccardo

--
Riccardo Murri

S3IT: Services and Support for Science IT
University of Zurich

Reply via email to