hi Spico, As Bobby said, native Storm is going to have better support for this soon.
FWIW, there is also the storm-on-mesos project, which we've been running on for almost 2 years at Groupon. - https://github.com/mesos/storm Caveats: - storm's logviewer is unsupported - scheduling can be suboptimal, causing topologies of different "size" (resource requirements) to starve each other - side effect of needing to dynamically calculate storm "slots" from mesos resource offers, my team has a framework change we are testing that will improve that behavior. - it's relatively complex to operate since there are so many different moving parts / components - mesos-master - mesos-slave/agent (it's being renamed) - storm nimbus (MesosNimbus -> the mesos scheduler) - storm supervisor (MesosSupervisor -> the mesos executor) - storm worker (the mesos task) - ZooKeeper - it probably won't work nicely with all the fancy security stuff that has been added to Storm in 0.10.0+ - Erik On Mon, Feb 1, 2016 at 12:14 PM, Bobby Evans <[email protected]> wrote: > We are currently adding in support for resource aware scheduling in a > multi-tenant stand alone storm cluster. It is still alpha quality but we > plan on getting it into production at Yahoo this quarter. If you can wait > that would be the preferred way I see to support your use case. > > - Bobby > > > On Monday, February 1, 2016 12:16 PM, Spico Florin <[email protected]> > wrote: > > > Hello! > I have an use case where we have to deploy many tpologies in a storm > cluster. > 1.we would like to know if running these topologies in combination with > apache slider over yarn would bring us some benefits in terms of resource > consumption? > 2. in such cases (running many topolgies aprox 60) what are the best > practices on how to run them over a cluster with a smart load balanced > hardware resources consumption (cpu, ram)? > I look forward for your answers. > Regards, > Florin > > > >
