Andre You can take a look at Apache Ambari. It allows you to spin up clusters very fast, add nodes and remove nodes. If you have an existing cluster and you want to run long running jobs, you can definitely use Apache Slider which is like Marathon in some ways.
There is doc https://cwiki.apache.org/confluence/display/AMBARI/Ambari Shivaji From: Joseph Jacks <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, January 14, 2015 at 4:11 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: (elastic) Hadoop YARN on mesos? Hey Andre, Have you seen: https://github.com/mesos/myriad JJ. On Wed, Jan 14, 2015 at 4:07 PM, Andre Kelpe <[email protected]<mailto:[email protected]>> wrote: Hi, we are currently looking for an easy way to spin up Hadoop YARN clusters on some hardware that we have lying around. I came across mesos and was wondering if somebody here is using it this way. What I am looking for is something where I say: Give me a cluster with n nodes. Run some tests and then destroy the Hadoop cluster. Similar to how EMR would work or how you work with vagrant locally. Is somebody here doing this and if so, do you have any starter docs for such a setup somewhere? Note that we def. need YARN and we want to use vanilla Apache Hadoop, so no commercial distribution. Thanks for your help! - Andre -- André Kelpe [email protected]<mailto:[email protected]> http://concurrentinc.com CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
