Hello! Thank you all for your answers! I guess I'll wait for adding in support for resource aware scheduling in a multi-tenant stand alone storm cluster. Regards, Florin
On Tue, Feb 2, 2016 at 1:00 AM, Erik Weathers <[email protected]> wrote: > hi Spico, > > As Bobby said, native Storm is going to have better support for this soon. > > FWIW, there is also the storm-on-mesos project, which we've been running > on for almost 2 years at Groupon. > > - https://github.com/mesos/storm > > Caveats: > > - storm's logviewer is unsupported > - scheduling can be suboptimal, causing topologies of different "size" > (resource requirements) to starve each other > - side effect of needing to dynamically calculate storm "slots" > from mesos resource offers, my team has a framework change we are > testing > that will improve that behavior. > - it's relatively complex to operate since there are so many different > moving parts / components > - mesos-master > - mesos-slave/agent (it's being renamed) > - storm nimbus (MesosNimbus -> the mesos scheduler) > - storm supervisor (MesosSupervisor -> the mesos executor) > - storm worker (the mesos task) > - ZooKeeper > - it probably won't work nicely with all the fancy security stuff that > has been added to Storm in 0.10.0+ > > - Erik > > On Mon, Feb 1, 2016 at 12:14 PM, Bobby Evans <[email protected]> wrote: > >> We are currently adding in support for resource aware scheduling in a >> multi-tenant stand alone storm cluster. It is still alpha quality but we >> plan on getting it into production at Yahoo this quarter. If you can wait >> that would be the preferred way I see to support your use case. >> >> - Bobby >> >> >> On Monday, February 1, 2016 12:16 PM, Spico Florin <[email protected]> >> wrote: >> >> >> Hello! >> I have an use case where we have to deploy many tpologies in a storm >> cluster. >> 1.we would like to know if running these topologies in combination with >> apache slider over yarn would bring us some benefits in terms of resource >> consumption? >> 2. in such cases (running many topolgies aprox 60) what are the best >> practices on how to run them over a cluster with a smart load balanced >> hardware resources consumption (cpu, ram)? >> I look forward for your answers. >> Regards, >> Florin >> >> >> >> >
