Wow! Really nice!
Thanks for sharing with us! Thanks, Bruno On 04/04/2018 11:59 AM, Riccardo Murri wrote:
Hello! I would like to bring to your attention ElastiCluster [1] [2], a tool for deploy verious kinds of compute clusters on IaaS clouds. Thanks to BigTop (and to the developers behind it!), ElastiCluster can also deploy functional Hadoop+Spark clusters [3]. ElastiCluster does not use the BigTop provisioner, instead opts for its own Ansible-based deployment playbooks: the provisioned software is currently limited to Hadoop + Spark + Thriftserver (from BigTop 1.2.1), but they can be integrated with other non-BigTop software (e.g., JupyterHub). AFAIK, the main use for Hadoop+Spark on ElastiCluster so far has been setting up small clusters for teaching purposes; I'd be glad for any feedback, and especially if anyone is willing to try it for more "serious" use cases, as well as discussing more general topics (here or on the ElastiCluster mailing-list). [1]: http://elasticluster.readthedocs.io/en/latest/ [2]: http://elasticluster.readthedocs.io/en/latest/install.html#quickstart [3]: http://elasticluster.readthedocs.io/en/latest/playbooks.html#hadoop-spark (I hope this kind of announcements is welcome on the list; I could find no policy on allowed topics on the BigTop web site and the mailing list index.) Kind regards, Riccardo -- Riccardo Murri S3IT: Services and Support for Science IT University of Zurich